mirror of
https://github.com/HeyPuter/puter.git
synced 2026-11-13 20:12:57 -10:00
Compare commits
52
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2b44aed144 | ||
|
|
647f28db37 | ||
|
|
30006e0108 | ||
|
|
71675d6eb7 | ||
|
|
6ec568d0b1 | ||
|
|
2394221745 | ||
|
|
ee2cdbfe1a | ||
|
|
9b6350beab | ||
|
|
b649d9ada7 | ||
|
|
50b0639d02 | ||
|
|
ab96702718 | ||
|
|
47f9b0bdf3 | ||
|
|
8c4b0eb1fc | ||
|
|
a88c1b9d65 | ||
|
|
9caaf7162c | ||
|
|
0191f2bfec | ||
|
|
7aadc48cca | ||
|
|
6b06793750 | ||
|
|
e441880f13 | ||
|
|
9a86803418 | ||
|
|
09c6ce6f46 | ||
|
|
0fdf40b8da | ||
|
|
5b850113ff | ||
|
|
51bf8697a8 | ||
|
|
8c14e7de30 | ||
|
|
84faf5c40c | ||
|
|
aba2666f26 | ||
|
|
abe6882de9 | ||
|
|
770af49528 | ||
|
|
47b4cc1158 | ||
|
|
e253a7f862 | ||
|
|
af35783df4 | ||
|
|
cb214f28cc | ||
|
|
8ed439e897 | ||
|
|
672a704193 | ||
|
|
35ae464d86 | ||
|
|
79f24ed3df | ||
|
|
f30d2134b1 | ||
|
|
c24c8589cb | ||
|
|
2f8f81e621 | ||
|
|
7d02c7f622 | ||
|
|
d2bbe2b5e2 | ||
|
|
3ff70ef97f | ||
|
|
f791024b2e | ||
|
|
cce9718b45 | ||
|
|
9758ada36e | ||
|
|
37dcee78f0 | ||
|
|
162398a827 | ||
|
|
6785b4d9cc | ||
|
|
d2acdafaed | ||
|
|
6a46363d6d | ||
|
|
2de9d509e8 |
No files matched your search
@@ -53,7 +53,7 @@ jobs:
|
||||
# step above fails; the tests skip themselves without the key.
|
||||
- name: Run live realtime conversation tests
|
||||
if: ${{ !cancelled() }}
|
||||
run: npm run setupExtensions && (cd src/puter-js && npm run build) && npx vitest run --config src/puter-js/tests/api/vitest.config.ts src/puter-js/tests/api/realtime/convoLive.test.ts
|
||||
run: npm run setupExtensions && (cd src/puter-js && npm run build) && npx vitest run --config src/puter-js/tests/api/vitest.config.ts src/puter-js/tests/api/realtime/realtimeLive.test.ts
|
||||
env:
|
||||
CI: 'true'
|
||||
PUTER_TEST_AI_OPENAI_API_KEY: ${{ secrets.PUTER_TEST_AI_OPENAI_API_KEY }}
|
||||
@@ -176,7 +176,10 @@ jobs:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v7
|
||||
|
||||
# Missing when the PR leg failed; `coverage (pr)` already reports that.
|
||||
- name: Download PR coverage
|
||||
id: pr-coverage
|
||||
continue-on-error: true
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
name: puterjs-coverage-pr
|
||||
@@ -186,6 +189,7 @@ jobs:
|
||||
# base run) — then report without the comparison column.
|
||||
- name: Download base coverage
|
||||
id: base-coverage
|
||||
if: steps.pr-coverage.outcome == 'success'
|
||||
continue-on-error: true
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
@@ -193,7 +197,7 @@ jobs:
|
||||
path: coverage-base
|
||||
|
||||
- name: Report coverage on PR
|
||||
if: steps.base-coverage.outcome == 'success'
|
||||
if: steps.pr-coverage.outcome == 'success' && steps.base-coverage.outcome == 'success'
|
||||
uses: davelosert/vitest-coverage-report-action@v2
|
||||
with:
|
||||
name: puter.js SDK
|
||||
@@ -202,7 +206,7 @@ jobs:
|
||||
json-summary-compare-path: coverage-base/coverage-summary.json
|
||||
|
||||
- name: Report coverage on PR (no base comparison)
|
||||
if: steps.base-coverage.outcome != 'success'
|
||||
if: steps.pr-coverage.outcome == 'success' && steps.base-coverage.outcome != 'success'
|
||||
uses: davelosert/vitest-coverage-report-action@v2
|
||||
with:
|
||||
name: puter.js SDK
|
||||
|
||||
@@ -159,7 +159,9 @@ The reference has two kinds of page:
|
||||
|
||||
An overview page answers "what is this API and is it what I need?", and a function page answers "how exactly does this call behave?". A change to one function usually only touches that function's page, so a new `kv.list()` option goes in `KV/list.md` without a matching update to `KV.md`. Don't copy a function's options, limits, or edge cases onto the overview page for completeness. Where a detail goes is a judgment call, not a hard rule: update the overview when a change affects how someone would describe the API as a whole.
|
||||
|
||||
When you add a page, add it to [src/docs/src/sidebar.js](src/docs/src/sidebar.js) under its area, or it won't show up in the navigation. When you add a runnable example, add its HTML to `src/docs/src/playground/examples/<slug>.html`, register the slug in [src/docs/src/examples.js](src/docs/src/examples.js), and link the code block to it by putting the slug after the language with a semicolon (` ```html;kv-set `).
|
||||
When you add a page, add it to [src/docs/src/sidebar.js](src/docs/src/sidebar.js) under its area, or it won't show up in the navigation.
|
||||
|
||||
**Sidebar order.** Order the API areas in the sidebar by how often developers use them, most common first. Overview stays first and Objects stays last. Inside an area, order the pages by use too, and keep pages that make up one feature next to each other, such as `share()`, `unshare()`, `listShared()`, and `listSharedByMe()`. A group goes where its most-used page would go. A helper that only supports another function, such as `listModels()` for `chat()`, goes by its own use. Put create, list, get, update, and delete in that order, and constants such as `MAX_KEY_SIZE` last. The function list on the area's overview page uses the sidebar's order. When you add a runnable example, add its HTML to `src/docs/src/playground/examples/<slug>.html`, register the slug in [src/docs/src/examples.js](src/docs/src/examples.js), and link the code block to it by putting the slug after the language with a semicolon (` ```html;kv-set `).
|
||||
|
||||
### Function page layout
|
||||
|
||||
@@ -200,6 +202,7 @@ These apply to reference pages and recipes alike.
|
||||
- **Describe behavior directly.** No quoted inner-voice phrases. Write "`isNew` is `true` when the person did not have access before", not "this tells 'I just gave this to someone' from 'they already had it'".
|
||||
- **Keep references easy to follow.** Name what a list or code block belongs to in the sentence directly above it. Name a thing once and keep that name. Introduce an actor as "a user" before giving them a role: "A user can create a team. The user who creates the team becomes its owner."
|
||||
- **Sentence mechanics.** Never start a sentence with inline code: write "To store data, use the [`set()`](/KV/set/) method." Link every inline mention of an API to its page, every time. No em dashes, and no colons or semicolons mid-sentence; a colon that introduces a code block is fine.
|
||||
- **No `##` or `###` headings inside tabs.** The table of contents lists every `##` and `###` heading on the page, including the ones in a hidden tab (an `example-content` block switched by the `example-group` pills), so its links lead to content the reader can't see. Put the heading above the tabs, or use `####` inside a tab, which the table of contents skips.
|
||||
- **Show the correct way.** No "don't" and no wrong-way-first examples, in recipes and playground examples alike. Mention a wrong approach only when developers already hit it in practice, and state how it fails as a fact: "`add()` reads a plain object as a path map, so the call rejects with …".
|
||||
|
||||
### Recipes
|
||||
|
||||
+216
-23
@@ -14,22 +14,41 @@ to its normal sign-in. Apps that need the desktop (built-in and godmode apps,
|
||||
private apps the user can't open, or a failed token request) open `/app/<name>`
|
||||
on the GUI origin, cut off from the embed. Website shortcuts open their external
|
||||
destination with no opener.
|
||||
The embed follows the OS's light or dark setting. A host with a fixed theme adds
|
||||
`theme=dark` or `theme=light` to the fragment (`#user=<uuid>&theme=dark`), since
|
||||
Chromium doesn't pass an iframe's `color-scheme` on to the page inside it.
|
||||
Apps opened from the embed carry `utm_source=puter.com&utm_medium=referral&utm_campaign=app_browser_embed`;
|
||||
the desktop and dashboard send `utm_campaign=desktop` and `utm_campaign=dashboard`.
|
||||
|
||||
## Session and framing
|
||||
|
||||
Only show the iframe after the user has signed in. The embed reads the existing
|
||||
GUI session from local storage on its own origin. If that storage holds no
|
||||
session, the embed gets the session from the GUI origin's session cookie through
|
||||
The embed shows the GUI session only to the user the host page is signed in as.
|
||||
The host names that user's `uuid` (from `puter.auth.getUser()`) in the iframe
|
||||
URL's fragment, as `/embed/apps#user=<uuid>`. The embed reads the session, asks
|
||||
`/whoami` which user it belongs to, and shows the signed-out view unless the two
|
||||
match. So a host that names no user, a malformed one, or a user other than the
|
||||
session's, gets the signed-out view, and so does a session the API refuses. A
|
||||
failed `/whoami` check shows the error state. The fragment never reaches the
|
||||
server and can only narrow what the embed shows: naming the session's own user
|
||||
gives the same view as a direct visit to the GUI. Changing the fragment of a
|
||||
loaded embed reloads it.
|
||||
|
||||
The embed reads the existing GUI session from local storage on its own origin.
|
||||
If that storage holds no session, the embed gets the session from the GUI origin's session cookie through
|
||||
`/get-gui-token`. This matters in Safari, which gives a framed origin separate
|
||||
local storage under each top-level origin, so even a sibling subdomain's iframe
|
||||
can't see the GUI's stored token. The session cookie is still sent, because the
|
||||
host is on the same site. The embed doesn't write that token back to storage. It
|
||||
does not create accounts, show a login dialog, or accept a token from its parent.
|
||||
A missing session or failed load displays an unavailable/error state. Being
|
||||
signed in on a subdomain alone does not create a session on the main GUI origin;
|
||||
the main GUI session must already exist.
|
||||
Being signed in on a subdomain alone does not create a session on the main GUI
|
||||
origin.
|
||||
|
||||
Without a session it can show, the embed shows the recommended apps (`RecommendedAppsService`)
|
||||
that the server put in the page, and nothing else: no installed apps, folders, or
|
||||
saved order, and no puter.js. Selecting one opens it at the same URL a signed-in
|
||||
launch uses, with no opener and no token on offer, so the app signs in on its own.
|
||||
Apps that need the desktop open `/app/<name>`. If the page carries no recommended
|
||||
apps, or a load fails, the embed displays an unavailable/error state.
|
||||
|
||||
The route's `Content-Security-Policy: frame-ancestors` allows the GUI origin and
|
||||
the embedding page's origin when it is exactly one label under `config.domain`
|
||||
@@ -53,14 +72,22 @@ and popups to escape the sandbox.
|
||||
|
||||
## Host integration
|
||||
|
||||
Create the iframe when “More apps” is opened, and remove it on close so reopening
|
||||
loads current apps and preferences. The host owns the modal and close control;
|
||||
there is no parent messaging API and no app list or credential is sent to it.
|
||||
The app browser fills its iframe and responds to its size.
|
||||
A first-party app that signs in with puter.js can place `<puter-user-menu>` from
|
||||
the SDK in its header instead (see `src/puter-js/src/ui/components.md`). It opens
|
||||
this embed with the signed-in user named and closes it when the app's sign-in
|
||||
changes. The rest of this section is for a host that builds the iframe itself.
|
||||
|
||||
This example assumes `puterOrigin` comes from the host app's deployment settings
|
||||
and the caller has already established the GUI session. Translate the button and
|
||||
frame labels using the host app's localization system.
|
||||
The iframe can be shown whether or not the user is signed in. Create it when
|
||||
“More apps” is opened, naming the signed-in user in its fragment, and remove it
|
||||
on close so reopening loads current apps and preferences. Remove it when the
|
||||
host's own sign-in changes too, such as on sign-out, so an open embed doesn't
|
||||
keep the previous user. The host owns the modal and close control;
|
||||
there is no parent messaging API and no app list or credential is sent to it.
|
||||
The app browser fills its iframe and responds to its size. It shows a spinner until its apps load.
|
||||
|
||||
This example assumes `puterOrigin` comes from the host app's deployment
|
||||
settings. Translate the button and frame labels using the host app's
|
||||
localization system.
|
||||
|
||||
```html
|
||||
<button id="more-apps" type="button">More apps</button>
|
||||
@@ -78,10 +105,14 @@ frame labels using the host app's localization system.
|
||||
const container = document.querySelector('#apps-frame');
|
||||
const trigger = document.querySelector('#more-apps');
|
||||
|
||||
trigger.addEventListener('click', () => {
|
||||
trigger.addEventListener('click', async () => {
|
||||
const url = new URL('/embed/apps', puterOrigin);
|
||||
// Signed out, the embed shows the recommended apps.
|
||||
const user = puter.auth.isSignedIn() ? await puter.auth.getUser().catch(() => null) : null;
|
||||
if (user) url.hash = `user=${encodeURIComponent(user.uuid)}`;
|
||||
const frame = document.createElement('iframe');
|
||||
frame.title = 'Your apps';
|
||||
frame.src = new URL('/embed/apps', puterOrigin).href;
|
||||
frame.src = url.href;
|
||||
container.replaceChildren(frame);
|
||||
dialog.showModal();
|
||||
});
|
||||
@@ -93,20 +124,182 @@ frame labels using the host app's localization system.
|
||||
</script>
|
||||
```
|
||||
|
||||
## Implementation and checks
|
||||
## Implementation
|
||||
|
||||
`createAppBrowser` in `src/gui/src/UI/Dashboard/AppBrowser.js` owns the shared UI
|
||||
and data behavior. `TabApps.js` injects desktop actions. `appsEmbed.js` supplies
|
||||
new-tab launching through `launchAppInTab` in `src/gui/src/helpers/appTabLaunch.js`,
|
||||
which builds the same app URL as `launch_app`, and loads only the embed entry, its
|
||||
SDK, and shared styles.
|
||||
The normal GUI webpack build emits `bundle.min.js` and `apps-embed.min.js`.
|
||||
No puter.js API or app-list endpoint changes are needed.
|
||||
and data behavior; its `guestApps` option is the signed-out list. `TabApps.js`
|
||||
injects desktop actions. `appsEmbed.js` supplies new-tab launching through
|
||||
`launchAppInTab` (signed in) and `guestAppLaunchUrl` (signed out) in
|
||||
`src/gui/src/helpers/appTabLaunch.js`, which build the same app URL as
|
||||
`launch_app`, and loads only the embed entry, its SDK, and shared styles. `embedHostUser` in
|
||||
`src/gui/src/helpers/embedHostUser.js` reads the host's user from the fragment, and
|
||||
`embedHostTheme.js` beside it applies the host's theme.
|
||||
The normal GUI webpack build emits `bundle.min.js`, `apps-embed.min.js`, and `feedback-embed.min.js`.
|
||||
The `/embed/apps` route puts the recommended list in the page's config as
|
||||
`guestApps`. No puter.js API or app-list endpoint changes are needed.
|
||||
|
||||
## Feedback dialog
|
||||
|
||||
`GET /embed/feedback` is the app feedback dialog (`puter.ui.showFeedbackDialog()`)
|
||||
for a first-party page, with the same framing rules, query redirect, and
|
||||
host-user binding as `/embed/apps`. It sends the feedback for the framing
|
||||
page's origin, which the server reads from the referrer and pins with
|
||||
`frame-ancestors`; opened top-level, it does nothing. Its page is transparent
|
||||
so the dialog's own scrim covers the host.
|
||||
|
||||
puter.js uses it on a page one label under the GUI origin, other than the
|
||||
subdomains the server refuses, when the page is signed in, framing it in a
|
||||
modal `<dialog>` that stays hidden until the embed is ready. A session check that takes longer than 200ms shows a loading scrim
|
||||
with a spinner in its place; Escape or a click on it cancels. The embed posts `{ msg: 'feedbackEmbed', state }` to its parent
|
||||
(`loading`, then `ready` or `unavailable`) and finally
|
||||
`{ msg: 'feedbackDialogClosed', sent }`. It reports `unavailable` without the
|
||||
named user's session, or while that user's email still needs confirming, and
|
||||
puter.js then uses the `/action/send-feedback` popup, which can sign in. So does
|
||||
a frame that never answers, such as one the host's CSP blocks. `feedbackEmbed.js`
|
||||
is the entry, sharing `embedSession` in `src/gui/src/helpers/embedSession.js`
|
||||
with the app browser; `showFeedbackEmbed` in `src/puter-js/src/lib/guiEmbed.js`
|
||||
is the host side.
|
||||
|
||||
## Sign-in and file pickers
|
||||
|
||||
`GET /embed/sign-in`, `/embed/show-open-file-picker`, `/embed/show-save-file-picker`
|
||||
and `/embed/show-directory-picker` serve the same GUI popups as `/action/<action>`
|
||||
for a first-party page, with the same framing rules, query redirect, and
|
||||
`no-store` as `/embed/apps`. Any other spelling of the path (another case, a
|
||||
trailing slash) redirects to it, since the shell boots the desktop for it. They render the normal GUI shell, which names the
|
||||
framing page (read from the referrer, pinned with `frame-ancestors`) as
|
||||
`gui_params.embed_host_origin`. The GUI then runs the popup boot unchanged, with
|
||||
the framing page as `window.opener`, and `window.close()` asks the page to
|
||||
remove the frame (`src/gui/src/helpers/popupFrame.js`). Without a framing page
|
||||
to answer (opened top-level, or framed by a refused page) it does nothing. The
|
||||
popup's parameters come from the fragment instead of the query: `msg_id`,
|
||||
`options` and `request_auth`, plus `user` for a picker.
|
||||
|
||||
The frame posts `{ msg: 'puterPopupFrame', state }` to its parent: `loading` as
|
||||
soon as the shell boots, `ready` once it shows a window or dialog, `unavailable`
|
||||
when the page has to use the popup, `provider` when the user picks a sign-in
|
||||
provider (below), and `closed` where a popup would close. Its
|
||||
answers (`puter.token`, `fileOpenPicked`, `directoryPicked`, `fileSaved`, the
|
||||
cancel messages, `sendMeFileData`) are the popup's.
|
||||
|
||||
puter.js uses them on a page one label under the GUI origin, other than the
|
||||
subdomains the server refuses, unless the page is cross-origin isolated or in a
|
||||
frame of another site (which `frame-ancestors` refuses too).
|
||||
`showPopupFrame` in `src/puter-js/src/lib/guiEmbed.js` shows the frame in a box
|
||||
the size of the popup window, over a scrim, hidden until `ready` and behind the
|
||||
loading spinner after 200ms. Escape closes it, on the page or in the frame. One shows per popup, as each
|
||||
popup has its own window name: a second picker request of the same kind
|
||||
replaces the one showing, and a `signIn()` call made while a framed sign-in is
|
||||
open settles with it. On
|
||||
screens up to 600px wide the box fills the screen. A frame that is refused,
|
||||
never answers, or reports `unavailable` falls back to the popup, opened through
|
||||
a consent dialog when the click that asked for it has expired. A reload or navigation the frame
|
||||
starts itself (logging out, a sign-out in another tab) names the frame as its
|
||||
referrer, so the server refuses it; the page then shows its spinner and opens
|
||||
the frame again, once per `ready`.
|
||||
|
||||
- **Sign-in** (`puter.auth.signIn()`) has no host-user binding: it is how the
|
||||
page gets a user. The box has a close button, which rejects with
|
||||
`auth_window_closed` as closing the popup does. Providers refuse to be
|
||||
framed, so choosing one posts `{ state: 'provider', url }` with the
|
||||
provider's start URL for this sign-in's `msg_id`, then `unavailable`. The
|
||||
page opens the popup straight at that URL, once it checks the URL is the
|
||||
GUI's `/auth/oidc/<provider>/start` for this `msg_id`, and the provider
|
||||
returns it as this sign-in's popup. `attempt_temp_user_creation` keeps the popup, since
|
||||
only the popup creates a temporary account.
|
||||
- **Pickers** frame only for a signed-in page. The GUI reports `unavailable`
|
||||
unless its session belongs to the user named in `user`, and the page then uses
|
||||
the popup, which can sign in and switch accounts. The token the GUI sends
|
||||
before a picker's answer isn't adopted, since it is for the page's own user.
|
||||
Dismissing the loading scrim settles the picker as cancelled.
|
||||
|
||||
## Upgrade
|
||||
|
||||
`GET /embed/upgrade` is the deployment's upgrade dialog for a first-party page,
|
||||
served like the popups above: the GUI shell with the same framing rules, query
|
||||
redirect, exact-path redirect and `no-store`, naming the framing page as
|
||||
`gui_params.embed_host_origin`. It runs only when framed by an allowed page.
|
||||
|
||||
puter.js frames it on a signed-in first-party page (the same pages as the
|
||||
pickers) for `puter.ui.requestUpgrade(details)`, and for the Upgrade button of
|
||||
the usage limit dialog a refused call shows there. Elsewhere both keep their
|
||||
behavior: inside an app `requestUpgrade()` asks the desktop, and on any other
|
||||
page the usage limit dialog opens the dashboard's upgrade page in a new tab.
|
||||
|
||||
The frame covers the page with no box or close button, and the GUI page is
|
||||
transparent (`html.popup-frame-transparent`) so the dialog's own scrim covers
|
||||
the host. The frame allows `payment` as well as `otp-credentials`. Escape on the
|
||||
page is left to the dialog, since one there shouldn't drop a payment in
|
||||
progress. Its fragment names `user` (the page's signed-in user) and, when the
|
||||
refusal gave them, `reason` (`funds`, `subscription` or `storage`) and `method`.
|
||||
|
||||
The GUI reports `unavailable` unless its session belongs to `user`, the account
|
||||
has no verification or password change waiting (which the frame couldn't be
|
||||
closed from), and the deployment defines `window.UIUpgradeAccount` with
|
||||
`UIUpgradeAccount.embeddable === true`. Otherwise it runs:
|
||||
|
||||
```js
|
||||
new UIUpgradeAccount({ attribution: { appUid, reason, method }, onClose }).open_as_window();
|
||||
```
|
||||
|
||||
`appUid` is the app the framing page's origin resolves to, never one the
|
||||
fragment names. `open_as_window()` resolves once the dialog shows, and the GUI
|
||||
then posts `ready`. While the page still hides the frame, which Chromium and
|
||||
WebKit lay out at 0x0, the GUI posts `ready` first and opens the dialog once
|
||||
the frame has a viewport, so a window it opens doesn't center on 0x0.
|
||||
`onClose({ upgraded })` must run exactly once, when the
|
||||
dialog closes or when it declines to open for the account; the GUI then posts
|
||||
`{ msg: 'puterPopupFrame', state: 'closed', upgraded }` and the page removes
|
||||
the frame. If `open_as_window()` rejects, the GUI posts `unavailable` instead. `requestUpgrade()` resolves `{ upgraded }`. A call made while the
|
||||
frame shows settles with it rather than replacing it, and a call refused while
|
||||
it shows still rejects but opens no usage limit dialog over it.
|
||||
|
||||
When the frame can't show the upgrade (refused, never ready, `unavailable`, or
|
||||
no user to name), puter.js opens the dashboard's upgrade page in a new tab with
|
||||
what was refused, through a consent dialog when the click has expired or the
|
||||
tab is blocked, and `requestUpgrade()` resolves `{ upgraded: false }`.
|
||||
|
||||
## Email confirmation
|
||||
|
||||
`GET /embed/request-email-confirmation` walks a first-party page's user through
|
||||
confirming their email, served like the upgrade: the GUI shell with the same
|
||||
framing rules, redirects and `no-store`, run only when framed by an allowed
|
||||
page. puter.js frames it on a signed-in first-party page for
|
||||
`puter.ui.requestEmailConfirmation()`. Inside an app the desktop shows the same
|
||||
windows over the app instead.
|
||||
|
||||
Like the upgrade, the frame covers the page with no box or close button, over a
|
||||
transparent GUI page, and its fragment names `user`. Escape closes it, on the
|
||||
page or on the save-account or code window. On a window, Escape closes that
|
||||
window as its close button does, so a code already sent still answers. The GUI
|
||||
reports `unavailable` unless its session belongs to `user` and the account has
|
||||
no phone, card or password check waiting.
|
||||
An email check is fine: the boot skips its own email gate for this action, so
|
||||
the user gets one code window they can close.
|
||||
|
||||
The GUI then runs `ensureConfirmedEmail` (`src/gui/src/helpers/ensureConfirmedEmail.js`),
|
||||
which the desktop's handler uses too: a temporary account gets the save-account
|
||||
window first, then an unconfirmed email gets the code window, each at most
|
||||
once. When the page's whoami already reads the email as confirmed,
|
||||
`requestEmailConfirmation()` resolves `{ response: true }` without framing
|
||||
anything, and a confirmation in the frame marks that whoami confirmed for the
|
||||
calls after it. An email the GUI reads as confirmed shows nothing. Before
|
||||
opening a window the GUI waits for the frame to have a viewport, as the upgrade
|
||||
does. It then posts
|
||||
`{ msg: 'puterPopupFrame', state: 'closed', confirmed }`, and
|
||||
`requestEmailConfirmation()` resolves `{ response: confirmed }`, the field an
|
||||
app reads from the desktop's answer. A call made while the frame shows settles
|
||||
with it. A frame that is refused, never ready, or `unavailable` resolves
|
||||
`{ response: false }`, since there's no popup to fall back to.
|
||||
|
||||
## Checks
|
||||
|
||||
Run the GUI component/browser tests (with the SDK bundle built and Playwright
|
||||
Chromium installed), and the backend route tests:
|
||||
|
||||
```sh
|
||||
npx vitest run --config src/gui/vitest.config.js src/gui/src/UI/Dashboard/AppBrowser
|
||||
npx vitest run --config src/gui/vitest.config.js src/gui/src/UI/Dashboard/AppBrowser src/feedbackEmbed
|
||||
npx vitest run src/puter-js/src/lib/guiEmbed.test.js src/puter-js/src/modules/UI.showFeedbackDialog.test.js src/puter-js/src/modules/Auth.signInFrame.test.js src/puter-js/src/modules/UI.pickerFrame.test.js src/puter-js/src/modules/UI.requestUpgrade.test.js src/puter-js/src/lib/upgradePrompt.test.js src/puter-js/src/modules/UsageLimitDialog.upgrade.test.js
|
||||
npx vitest run --config src/gui/vitest.config.js src/gui/src/helpers/popupFrame.test.js
|
||||
npx vitest run --config src/backend/vitest.config.ts src/backend/server.test.ts
|
||||
```
|
||||
@@ -420,7 +420,11 @@ export type EventMap = {
|
||||
old_username: string;
|
||||
new_username: string;
|
||||
};
|
||||
'user.email-changed': { user_id: number; new_email: string };
|
||||
'user.email-changed': {
|
||||
user_id: number;
|
||||
old_email: string | null;
|
||||
new_email: string;
|
||||
};
|
||||
/** An account was suspended, or its suspension lifted. Local events. */
|
||||
'user.suspended': { userId: number };
|
||||
'user.unsuspended': { userId: number };
|
||||
|
||||
@@ -21,6 +21,8 @@
|
||||
import { GetObjectCommand } from '@aws-sdk/client-s3';
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { existsSync } from 'node:fs';
|
||||
import { createServer, type Server } from 'node:http';
|
||||
import type { AddressInfo } from 'node:net';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
@@ -200,6 +202,77 @@ describe('S3Client — remote endpoint configuration', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('S3Client — bucket in another region', () => {
|
||||
let server: Server;
|
||||
let client: S3Client;
|
||||
let signedRegions: string[];
|
||||
|
||||
// Answers like a bucket that lives in us-west-2: a request signed for any
|
||||
// other region gets S3's 301 naming the bucket's region.
|
||||
beforeEach(async () => {
|
||||
signedRegions = [];
|
||||
server = createServer((req, res) => {
|
||||
const region =
|
||||
/Credential=[^/]+\/\d{8}\/([^/]+)\//.exec(
|
||||
req.headers.authorization ?? '',
|
||||
)?.[1] ?? '';
|
||||
signedRegions.push(region);
|
||||
if (region !== 'us-west-2') {
|
||||
res.writeHead(301, {
|
||||
'content-type': 'application/xml',
|
||||
'x-amz-bucket-region': 'us-west-2',
|
||||
});
|
||||
res.end(
|
||||
'<?xml version="1.0" encoding="UTF-8"?><Error><Code>PermanentRedirect</Code><Message>The bucket you are attempting to access must be addressed using the specified endpoint.</Message></Error>',
|
||||
);
|
||||
return;
|
||||
}
|
||||
res.writeHead(200, {
|
||||
'content-type': 'text/plain',
|
||||
'content-length': '5',
|
||||
});
|
||||
res.end('hello');
|
||||
});
|
||||
await new Promise<void>((resolve) =>
|
||||
server.listen(0, '127.0.0.1', resolve),
|
||||
);
|
||||
const { port } = server.address() as AddressInfo;
|
||||
client = new S3Client({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
s3: {
|
||||
s3Config: {
|
||||
endpoint: `http://127.0.0.1:${port}`,
|
||||
accessKeyId: 'fakeAccessKeyId',
|
||||
secretAccessKey: 'fakeSecretAccessKey',
|
||||
forcePathStyle: true,
|
||||
},
|
||||
},
|
||||
} as unknown as IConfig);
|
||||
await client.onServerStart();
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await client.onServerShutdown();
|
||||
server.closeAllConnections();
|
||||
await new Promise<void>((resolve) => server.close(() => resolve()));
|
||||
});
|
||||
|
||||
it("retries a read in the bucket's region when S3 redirects it", async () => {
|
||||
const result = await client
|
||||
.get('ap-northeast-2')
|
||||
.send(
|
||||
new GetObjectCommand({
|
||||
Bucket: 'other-bucket',
|
||||
Key: 'index.html',
|
||||
}),
|
||||
);
|
||||
|
||||
expect(await result.Body?.transformToString()).toBe('hello');
|
||||
expect(signedRegions).toEqual(['ap-northeast-2', 'us-west-2']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('S3Client — legacy storage migration', () => {
|
||||
let client: S3Client;
|
||||
let dir: string;
|
||||
|
||||
@@ -235,6 +235,9 @@ export class S3Client extends PuterClient {
|
||||
|
||||
const client = new AwsS3Client({
|
||||
region,
|
||||
// Rows without a stored region are read with the server's own, so
|
||||
// retry where S3's 301 says the bucket actually lives.
|
||||
followRegionRedirects: true,
|
||||
requestStreamBufferSize: 32 * 1024,
|
||||
requestHandler: new NodeHttpHandler({
|
||||
socketTimeout: 5000,
|
||||
|
||||
@@ -7468,6 +7468,11 @@ describe('AuthController user-protected mutations (validation paths)', () => {
|
||||
{ expiresIn: '1h' },
|
||||
);
|
||||
|
||||
const heard: Array<Record<string, unknown>> = [];
|
||||
eventClient.on('user.email-changed', (_k, data) => {
|
||||
heard.push(data as Record<string, unknown>);
|
||||
});
|
||||
|
||||
const req = makeReq({});
|
||||
(req as unknown as { query: Record<string, string> }).query = {
|
||||
token: linkJwt,
|
||||
@@ -7475,6 +7480,11 @@ describe('AuthController user-protected mutations (validation paths)', () => {
|
||||
const res = makeRes();
|
||||
await controller.handleChangeEmailConfirm(req, res);
|
||||
expect(res.sent).toMatch(/Email changed successfully/);
|
||||
expect(heard).toContainEqual({
|
||||
user_id: user.id,
|
||||
old_email: user.email ?? null,
|
||||
new_email: newEmail,
|
||||
});
|
||||
|
||||
const after = await server.stores.user.getById(user.id, {
|
||||
force: true,
|
||||
|
||||
@@ -2647,6 +2647,7 @@ export class AuthController extends PuterController {
|
||||
'user.email-changed' as never,
|
||||
{
|
||||
user_id: user.id,
|
||||
old_email: user.email ?? null,
|
||||
new_email: newEmail,
|
||||
} as never,
|
||||
{},
|
||||
|
||||
@@ -15,7 +15,6 @@ export interface OidcLoginContinuation {
|
||||
embeddedInPopup: boolean;
|
||||
openerOrigin: string | null;
|
||||
messageId: string | null;
|
||||
switchedToLogin: boolean;
|
||||
}
|
||||
|
||||
interface OidcReturnDeps {
|
||||
@@ -56,7 +55,6 @@ export function completeOidcLoginReturn(
|
||||
data.userUuid !== user.uuid ||
|
||||
typeof data.redirectUri !== 'string' ||
|
||||
typeof data.embeddedInPopup !== 'boolean' ||
|
||||
typeof data.switchedToLogin !== 'boolean' ||
|
||||
(data.openerOrigin !== null && typeof data.openerOrigin !== 'string') ||
|
||||
(data.messageId !== null && typeof data.messageId !== 'string')
|
||||
) {
|
||||
@@ -91,6 +89,5 @@ export function completeOidcLoginReturn(
|
||||
),
|
||||
);
|
||||
}
|
||||
if (data.switchedToLogin) append('oidc_switched', 'login');
|
||||
return target;
|
||||
}
|
||||
@@ -27,7 +27,7 @@ import type { Actor } from '../../core/actor.js';
|
||||
import { PuterRouter } from '../../core/http/PuterRouter.js';
|
||||
import { PuterServer } from '../../server.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
import { embedFrameAncestors } from './HomepageController.js';
|
||||
import { embedFrameAncestors, embedHostOrigin } from './HomepageController.js';
|
||||
|
||||
// ── Test harness ────────────────────────────────────────────────────
|
||||
//
|
||||
@@ -550,3 +550,26 @@ describe('embedFrameAncestors', () => {
|
||||
).toBe("'self'");
|
||||
});
|
||||
});
|
||||
|
||||
describe('embedHostOrigin', () => {
|
||||
const config = {
|
||||
origin: 'https://example.test',
|
||||
domain: 'example.test',
|
||||
api_base_url: 'https://backend.example.test',
|
||||
static_hosting_domain: 'site.example.test',
|
||||
};
|
||||
|
||||
it.each([
|
||||
['https://sheets.example.test/doc/1', 'https://sheets.example.test'],
|
||||
['https://backend.example.test/x', null],
|
||||
['https://api.example.test/', null],
|
||||
['https://someone.site.example.test/', null],
|
||||
['https://a.sheets.example.test/', null],
|
||||
['http://sheets.example.test/', null],
|
||||
['https://example.test/', null],
|
||||
['not a url', null],
|
||||
[undefined, null],
|
||||
])('%s frames as %s', (referer, expected) => {
|
||||
expect(embedHostOrigin(config, referer)).toBe(expected);
|
||||
});
|
||||
});
|
||||
@@ -31,8 +31,24 @@ import type {
|
||||
PuterHomepageService,
|
||||
PageMeta,
|
||||
LaunchOptions,
|
||||
PopupFrame,
|
||||
} from '../../services/homepage/PuterHomepageService';
|
||||
|
||||
/**
|
||||
* GUI popups a first-party page can show in a frame at `/embed/<action>`,
|
||||
* instead of opening `/action/<action>` in a window, plus the upgrade dialog,
|
||||
* which it would otherwise open in a dashboard tab, and the email confirmation,
|
||||
* which it otherwise can't show.
|
||||
*/
|
||||
export const FRAMED_POPUP_ACTIONS = [
|
||||
'sign-in',
|
||||
'show-open-file-picker',
|
||||
'show-save-file-picker',
|
||||
'show-directory-picker',
|
||||
'upgrade',
|
||||
'request-email-confirmation',
|
||||
] as const;
|
||||
|
||||
/** Longest description/og:description/twitter:description on `/app/<name>`. */
|
||||
export const APP_META_DESCRIPTION_MAX = 150;
|
||||
|
||||
@@ -61,32 +77,34 @@ export function appMetaDescription(text: string): string {
|
||||
return `${cut.replace(/[\s,;:.!?…—-]+$/, '')}…`;
|
||||
}
|
||||
|
||||
type EmbedFramingConfig = Pick<
|
||||
IConfig,
|
||||
| 'origin'
|
||||
| 'domain'
|
||||
| 'api_base_url'
|
||||
| 'static_hosting_domain'
|
||||
| 'static_hosting_domain_alt'
|
||||
| 'private_app_hosting_domain'
|
||||
| 'private_app_hosting_domain_alt'
|
||||
>;
|
||||
|
||||
/**
|
||||
* `frame-ancestors` for the app browser embed. A wildcard over `*.<domain>`
|
||||
* The origin of a first-party page that may frame an `/embed/*` page, named
|
||||
* exactly from the request's referrer, or null. A wildcard over `*.<domain>`
|
||||
* would also admit user-hosted sites whenever the hosting domains sit under it,
|
||||
* so the embedding page is named exactly, from the request's referrer. Only a
|
||||
* single label under `<domain>` qualifies, and not one this server answers
|
||||
* itself (the API, WebDAV, SDK and native-app hosts).
|
||||
* so only a single label under `<domain>` qualifies, and not one this server
|
||||
* answers itself (the API, WebDAV, SDK and native-app hosts).
|
||||
*/
|
||||
export function embedFrameAncestors(
|
||||
config: Pick<
|
||||
IConfig,
|
||||
| 'origin'
|
||||
| 'domain'
|
||||
| 'api_base_url'
|
||||
| 'static_hosting_domain'
|
||||
| 'static_hosting_domain_alt'
|
||||
| 'private_app_hosting_domain'
|
||||
| 'private_app_hosting_domain_alt'
|
||||
>,
|
||||
export function embedHostOrigin(
|
||||
config: EmbedFramingConfig,
|
||||
referer: string | undefined,
|
||||
): string {
|
||||
): string | null {
|
||||
const gui = new URL(config.origin ?? `https://${config.domain}`);
|
||||
let parent: URL;
|
||||
try {
|
||||
parent = new URL(referer ?? '');
|
||||
} catch {
|
||||
return "'self'";
|
||||
return null;
|
||||
}
|
||||
const host = parent.hostname.toLowerCase();
|
||||
const label = subdomainLabelOf(host, config.domain);
|
||||
@@ -105,7 +123,16 @@ export function embedFrameAncestors(
|
||||
!isReservedSubdomain(label) &&
|
||||
host !== new URL(config.api_base_url).hostname &&
|
||||
!hostingDomains.some((d) => host === d || host.endsWith(`.${d}`));
|
||||
return allowed ? `'self' ${parent.origin}` : "'self'";
|
||||
return allowed ? parent.origin : null;
|
||||
}
|
||||
|
||||
/** `frame-ancestors` for an `/embed/*` page: the GUI and {@link embedHostOrigin}. */
|
||||
export function embedFrameAncestors(
|
||||
config: EmbedFramingConfig,
|
||||
referer: string | undefined,
|
||||
): string {
|
||||
const host = embedHostOrigin(config, referer);
|
||||
return host ? `'self' ${host}` : "'self'";
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -142,6 +169,7 @@ export class HomepageController extends PuterController {
|
||||
res: express.Response,
|
||||
metaOverrides: Partial<PageMeta> = {},
|
||||
launch: LaunchOptions = {},
|
||||
popupFrame?: PopupFrame,
|
||||
) => {
|
||||
const meta = { ...defaultMeta(req), ...metaOverrides };
|
||||
const actor =
|
||||
@@ -150,19 +178,27 @@ export class HomepageController extends PuterController {
|
||||
actor?: Parameters<typeof homepage.send>[0]['actor'];
|
||||
}
|
||||
).actor ?? null;
|
||||
await homepage.send({ req, res, actor }, meta, launch);
|
||||
await homepage.send({ req, res, actor }, meta, launch, popupFrame);
|
||||
};
|
||||
|
||||
// -- Root + path-aliased shell routes ------------------------
|
||||
|
||||
router.get('/', {}, (req, res) => sendShell(req, res));
|
||||
|
||||
router.get('/embed/apps', {}, (req, res) => {
|
||||
// SDK bootstrap parameters must not change this page's account or API origin.
|
||||
// Checked on the raw URL; `req.query` drops pairs past the parser's limit.
|
||||
/**
|
||||
* Headers shared by the `/embed/*` pages; false when the request was
|
||||
* redirected instead. SDK bootstrap parameters must not change an
|
||||
* embed's account or API origin, so a query string is dropped. Checked
|
||||
* on the raw URL; `req.query` drops pairs past the parser's limit.
|
||||
*/
|
||||
const prepareEmbed = (
|
||||
req: express.Request,
|
||||
res: express.Response,
|
||||
path: string,
|
||||
): boolean => {
|
||||
if (req.originalUrl.includes('?')) {
|
||||
res.redirect('/embed/apps');
|
||||
return;
|
||||
res.redirect(path);
|
||||
return false;
|
||||
}
|
||||
res.removeHeader('X-Frame-Options');
|
||||
res.setHeader(
|
||||
@@ -170,9 +206,61 @@ export class HomepageController extends PuterController {
|
||||
`frame-ancestors ${embedFrameAncestors(this.config, req.get('referer'))}`,
|
||||
);
|
||||
res.setHeader('Cache-Control', 'no-store');
|
||||
res.send(homepage.renderAppBrowserEmbed());
|
||||
return true;
|
||||
};
|
||||
|
||||
router.get('/embed/apps', {}, async (req, res) => {
|
||||
if (!prepareEmbed(req, res, '/embed/apps')) return;
|
||||
// Shown to visitors without a session; a signed-in embed fetches
|
||||
// its own list, so a failure here only costs signed-out visitors.
|
||||
const guestApps = await this.services.recommendedApps
|
||||
.getRecommendedApps()
|
||||
.catch((): Array<Record<string, unknown>> => []);
|
||||
res.send(homepage.renderAppBrowserEmbed({ guestApps }));
|
||||
});
|
||||
|
||||
router.get('/embed/feedback', {}, (req, res) => {
|
||||
if (!prepareEmbed(req, res, '/embed/feedback')) return;
|
||||
// The feedback is recorded against this origin, which
|
||||
// frame-ancestors has just pinned to the actual framing page.
|
||||
res.send(
|
||||
homepage.renderFeedbackEmbed({
|
||||
hostOrigin: embedHostOrigin(
|
||||
this.config,
|
||||
req.get('referer'),
|
||||
),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
// The GUI runs the popup's own boot, answering the framing page
|
||||
// instead of an opener.
|
||||
for (const action of FRAMED_POPUP_ACTIONS) {
|
||||
const embedPath = `/embed/${action}`;
|
||||
router.get(embedPath, {}, async (req, res) => {
|
||||
// Routes match case-insensitively and with a trailing slash,
|
||||
// and the shell boots the desktop for any other spelling, so
|
||||
// only the exact one may be framed.
|
||||
if (req.originalUrl !== embedPath) {
|
||||
res.redirect(embedPath);
|
||||
return;
|
||||
}
|
||||
if (!prepareEmbed(req, res, embedPath)) return;
|
||||
await sendShell(
|
||||
req,
|
||||
res,
|
||||
{},
|
||||
{},
|
||||
{
|
||||
hostOrigin: embedHostOrigin(
|
||||
this.config,
|
||||
req.get('referer'),
|
||||
),
|
||||
},
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
router.get('/settings', {}, (req, res) => sendShell(req, res));
|
||||
router.get('/settings/*splat', {}, (req, res) => sendShell(req, res));
|
||||
|
||||
|
||||
@@ -1620,7 +1620,7 @@ describe('OIDCController signup callback', () => {
|
||||
expect(captured.redirectUrl).toContain('action=signup');
|
||||
});
|
||||
|
||||
it('creates a fresh user without oidc_switched on first signup', async () => {
|
||||
it('creates a fresh user on first signup', async () => {
|
||||
const state = oidc().signState({
|
||||
provider: 'custom',
|
||||
redirect_uri: TEST_ORIGIN + '/',
|
||||
@@ -1646,11 +1646,9 @@ describe('OIDCController signup callback', () => {
|
||||
);
|
||||
expect(captured.cookies).toHaveLength(1);
|
||||
expect(captured.redirectUrl).toBe(TEST_ORIGIN + '/');
|
||||
// Fresh creation — must NOT advertise "you've actually been signed in".
|
||||
expect(captured.redirectUrl).not.toContain('oidc_switched=login');
|
||||
});
|
||||
|
||||
it('adds oidc_switched=login when signup hits an already-linked account', async () => {
|
||||
it('signs in to an already-linked account on signup, returning as a login would', async () => {
|
||||
// Seed an account linked on (provider, sub) so the signup callback
|
||||
// resolves to `linked-sub` rather than creating a new user.
|
||||
const sub = `sub-${Math.random().toString(36).slice(2, 8)}`;
|
||||
@@ -1686,7 +1684,7 @@ describe('OIDCController signup callback', () => {
|
||||
res,
|
||||
);
|
||||
expect(captured.cookies).toHaveLength(1);
|
||||
expect(captured.redirectUrl).toContain('oidc_switched=login');
|
||||
expect(captured.redirectUrl).toBe(TEST_ORIGIN + '/');
|
||||
});
|
||||
|
||||
it('redirects to action=signup with auth_error when user resolution fails (unconfirmed email)', async () => {
|
||||
|
||||
@@ -688,14 +688,7 @@ export class OIDCController extends PuterController {
|
||||
);
|
||||
}
|
||||
|
||||
// If we landed on an existing account (either via the
|
||||
// provider_sub or via email match), signal the GUI so it can
|
||||
// render a "signed in" flow rather than "account created".
|
||||
const extra =
|
||||
resolved.origin === 'created'
|
||||
? undefined
|
||||
: { oidc_switched: 'login' };
|
||||
await this.#finishLogin(req, res, user, stateDecoded, extra);
|
||||
await this.#finishLogin(req, res, user, stateDecoded);
|
||||
};
|
||||
router.get('/auth/oidc/callback/signup', cbOpts, signupCb);
|
||||
router.post('/auth/oidc/callback/signup', cbOpts, signupCb);
|
||||
@@ -834,17 +827,14 @@ if (window.opener) {
|
||||
attempt = 0,
|
||||
): Promise<
|
||||
| { error: string; code?: string; requestCode?: string }
|
||||
| {
|
||||
user: import('../../stores/user/UserStore.js').UserRow;
|
||||
origin: 'linked-sub' | 'linked-email' | 'created';
|
||||
}
|
||||
| { user: import('../../stores/user/UserStore.js').UserRow }
|
||||
> {
|
||||
// 1. Existing provider/sub link.
|
||||
const linked = await this.services.oidc.findUserByProviderSub(
|
||||
provider,
|
||||
userinfo.sub,
|
||||
);
|
||||
if (linked) return { user: linked, origin: 'linked-sub' };
|
||||
if (linked) return { user: linked };
|
||||
|
||||
// 2/3. Email match branch.
|
||||
const claimedEmail =
|
||||
@@ -881,7 +871,7 @@ if (window.opener) {
|
||||
error: outcome.error ?? 'Failed to link provider.',
|
||||
};
|
||||
}
|
||||
return { user: byEmail, origin: 'linked-email' };
|
||||
return { user: byEmail };
|
||||
}
|
||||
}
|
||||
|
||||
@@ -911,7 +901,7 @@ if (window.opener) {
|
||||
requestCode: outcome.requestCode,
|
||||
};
|
||||
}
|
||||
return { user: outcome.user, origin: 'created' };
|
||||
return { user: outcome.user };
|
||||
}
|
||||
|
||||
async #processCallback(
|
||||
@@ -1000,7 +990,6 @@ if (window.opener) {
|
||||
res: Response,
|
||||
user: UserRow,
|
||||
stateDecoded: Record<string, unknown>,
|
||||
extraQueryParams?: Record<string, string>,
|
||||
): Promise<void> {
|
||||
const origin = (this.config.origin ?? '').replace(/\/$/, '');
|
||||
const continuation: OidcLoginContinuation = {
|
||||
@@ -1018,7 +1007,6 @@ if (window.opener) {
|
||||
stateDecoded.msg_id == null
|
||||
? null
|
||||
: String(stateDecoded.msg_id),
|
||||
switchedToLogin: extraQueryParams?.oidc_switched === 'login',
|
||||
};
|
||||
if (user.otp_enabled) {
|
||||
const challenge = this.services.token.sign(
|
||||
|
||||
@@ -139,7 +139,7 @@ describe('OIDC account second-factor login', () => {
|
||||
expect(await response.json()).toMatchObject({
|
||||
proceed: true,
|
||||
next_step: 'complete',
|
||||
redirectUrl: `${env.origin}/dashboard${flow === 'signup' ? '?oidc_switched=login' : ''}`,
|
||||
redirectUrl: `${env.origin}/dashboard`,
|
||||
});
|
||||
expect(
|
||||
response.headers
|
||||
|
||||
@@ -187,9 +187,9 @@ describe('createUserSubdomainNotFound', () => {
|
||||
});
|
||||
|
||||
it('passes through reserved subdomains (api, js, native apps, etc.)', () => {
|
||||
// `api`, `js`, `dav`, `docs`, `developer`, `editor`, `pdf`,
|
||||
// `api`, `js`, `dav`, `docs`, `developer`, `about`, `apps`,
|
||||
// `puter-app-icons`, etc. all pass through.
|
||||
for (const sub of ['api', 'js', 'docs', 'editor', 'puter-app-icons']) {
|
||||
for (const sub of ['api', 'js', 'docs', 'about', 'puter-app-icons']) {
|
||||
const { out, next } = run(
|
||||
createUserSubdomainNotFound(config),
|
||||
makeReq({
|
||||
@@ -865,13 +865,13 @@ describe('createNativeAppStatic', () => {
|
||||
|
||||
beforeAll(() => {
|
||||
root = mkdtempSync(nodePath.join(tmpdir(), 'native-apps-'));
|
||||
mkdirSync(nodePath.join(root, 'editor', 'assets'), { recursive: true });
|
||||
mkdirSync(nodePath.join(root, 'about', 'assets'), { recursive: true });
|
||||
writeFileSync(
|
||||
nodePath.join(root, 'editor', 'index.html'),
|
||||
'<h1>editor</h1>',
|
||||
nodePath.join(root, 'about', 'index.html'),
|
||||
'<h1>about</h1>',
|
||||
);
|
||||
writeFileSync(
|
||||
nodePath.join(root, 'editor', 'assets', 'app.js'),
|
||||
nodePath.join(root, 'about', 'assets', 'app.js'),
|
||||
'console.log(1);',
|
||||
);
|
||||
mkdirSync(nodePath.join(root, 'docs', 'dist'), { recursive: true });
|
||||
@@ -893,7 +893,7 @@ describe('createNativeAppStatic', () => {
|
||||
|
||||
it('leaves a native app name alone on a host outside our domains', async () => {
|
||||
const req = staticReq({
|
||||
subdomains: ['com', 'example', 'editor'],
|
||||
subdomains: ['com', 'example', 'about'],
|
||||
path: '/index.html',
|
||||
});
|
||||
(req as Request & { is_custom_domain?: boolean }).is_custom_domain =
|
||||
@@ -907,7 +907,7 @@ describe('createNativeAppStatic', () => {
|
||||
const { out, next } = await runStatic(
|
||||
createNativeAppStatic(config),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'editor'],
|
||||
subdomains: ['localhost', 'puter', 'about'],
|
||||
path: '/index.html',
|
||||
}),
|
||||
);
|
||||
@@ -915,17 +915,24 @@ describe('createNativeAppStatic', () => {
|
||||
expect(next).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('passes through subdomains that are not native apps', async () => {
|
||||
const { out, next } = await runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'api'],
|
||||
path: '/index.html',
|
||||
}),
|
||||
);
|
||||
expect(out.sent).toBeUndefined();
|
||||
expect(next).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
// `editor`, `markus` and `pdf` are web apps on standard host routing,
|
||||
// even with a folder of their own under the native apps root.
|
||||
it.each(['api', 'editor', 'markus', 'pdf'])(
|
||||
'passes %s through, as it is not a native app',
|
||||
async (sub) => {
|
||||
mkdirSync(nodePath.join(root, sub), { recursive: true });
|
||||
writeFileSync(nodePath.join(root, sub, 'index.html'), sub);
|
||||
const { out, next } = await runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', sub],
|
||||
path: '/index.html',
|
||||
}),
|
||||
);
|
||||
expect(out.sent).toBeUndefined();
|
||||
expect(next).toHaveBeenCalledTimes(1);
|
||||
},
|
||||
);
|
||||
|
||||
it('passes through when there is no subdomain at all', async () => {
|
||||
const { next } = await runStatic(
|
||||
@@ -939,13 +946,13 @@ describe('createNativeAppStatic', () => {
|
||||
const { out, next } = await runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'editor'],
|
||||
subdomains: ['localhost', 'puter', 'about'],
|
||||
path: '/index.html',
|
||||
}),
|
||||
);
|
||||
expect(out.sent).toEqual({
|
||||
path: '/index.html',
|
||||
root: nodePath.join(root, 'editor'),
|
||||
root: nodePath.join(root, 'about'),
|
||||
});
|
||||
expect(next).not.toHaveBeenCalled();
|
||||
});
|
||||
@@ -954,11 +961,11 @@ describe('createNativeAppStatic', () => {
|
||||
const { out } = await runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'EDITOR'],
|
||||
subdomains: ['localhost', 'puter', 'ABOUT'],
|
||||
path: '/index.html',
|
||||
}),
|
||||
);
|
||||
expect(out.sent?.root).toBe(nodePath.join(root, 'editor'));
|
||||
expect(out.sent?.root).toBe(nodePath.join(root, 'about'));
|
||||
});
|
||||
|
||||
it('serves docs out of its dist/ subdirectory, not the app root', async () => {
|
||||
@@ -979,7 +986,7 @@ describe('createNativeAppStatic', () => {
|
||||
const { out, next } = await runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'editor'],
|
||||
subdomains: ['localhost', 'puter', 'about'],
|
||||
path: '/assets',
|
||||
originalUrl: '/assets?v=2',
|
||||
}),
|
||||
@@ -995,7 +1002,7 @@ describe('createNativeAppStatic', () => {
|
||||
const { out } = await runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'editor'],
|
||||
subdomains: ['localhost', 'puter', 'about'],
|
||||
path: '/assets/',
|
||||
}),
|
||||
);
|
||||
@@ -1007,7 +1014,7 @@ describe('createNativeAppStatic', () => {
|
||||
const { out, next } = await runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'editor'],
|
||||
subdomains: ['localhost', 'puter', 'about'],
|
||||
path: '/nope.html',
|
||||
}),
|
||||
);
|
||||
@@ -1019,7 +1026,7 @@ describe('createNativeAppStatic', () => {
|
||||
const { next } = await runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'editor'],
|
||||
subdomains: ['localhost', 'puter', 'about'],
|
||||
path: '/index.html',
|
||||
}),
|
||||
new Error('send failed'),
|
||||
@@ -1032,7 +1039,7 @@ describe('createNativeAppStatic', () => {
|
||||
runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'editor'],
|
||||
subdomains: ['localhost', 'puter', 'about'],
|
||||
path: '/../../etc/passwd',
|
||||
}),
|
||||
),
|
||||
|
||||
@@ -27,15 +27,7 @@ import { HttpError, isHttpError } from '../HttpError';
|
||||
import { isLocalWorkerHost } from './localWorkerProxy';
|
||||
|
||||
/** Native-app subdomains served via `nativeAppStatic`. */
|
||||
const NATIVE_APP_SUBDOMAINS = [
|
||||
'about',
|
||||
'developer',
|
||||
'docs',
|
||||
'editor',
|
||||
'markus',
|
||||
'pdf',
|
||||
'apps',
|
||||
] as const;
|
||||
const NATIVE_APP_SUBDOMAINS = ['about', 'developer', 'docs', 'apps'] as const;
|
||||
|
||||
/** Subset served out of a `dist/` subdirectory rather than the app root. */
|
||||
const NATIVE_APPS_WITH_DIST = new Set(['docs', 'developer']);
|
||||
|
||||
@@ -2222,3 +2222,166 @@ describe('ChatCompletionDriver.countTokens', () => {
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
});
|
||||
|
||||
describe('ChatCompletionDriver service speed', () => {
|
||||
it.each([
|
||||
[{ serviceSpeed: 'ultrafast' }, 'fast', 0.028],
|
||||
[{ speed: 'fast' }, 'fast', 0.028],
|
||||
[{ serviceSpeed: 'standard', speed: 'fast' }, 'standard', 0.014],
|
||||
] as const)('uses the Claude catalog rate card for %j', async (options, speed, cents) => {
|
||||
const sent: Record<string, unknown>[] = [];
|
||||
vi.stubGlobal('fetch', async (input: Request | string, init?: RequestInit) => {
|
||||
const body = input instanceof Request
|
||||
? await input.clone().json()
|
||||
: JSON.parse(String(init?.body));
|
||||
sent.push(body);
|
||||
return new Response(JSON.stringify({
|
||||
id: 'claude-speed-test', type: 'message', role: 'assistant',
|
||||
model: body.model, stop_reason: 'end_turn',
|
||||
content: [{ type: 'text', text: 'OK' }],
|
||||
usage: { input_tokens: 10, output_tokens: 5, speed },
|
||||
}), { headers: { 'content-type': 'application/json' } });
|
||||
});
|
||||
const d = new ChatCompletionDriver(
|
||||
{ providers: { claude: { apiKey: 'test-key' }, ollama: { enabled: false } } } as never,
|
||||
server.clients, server.stores, server.services,
|
||||
);
|
||||
try {
|
||||
await d.onServerStart();
|
||||
const result = await withTestActor(() => d.complete({
|
||||
model: 'claude-opus-5-5', provider: 'claude',
|
||||
messages: [{ role: 'user', content: 'Hello' }],
|
||||
fallback: false, max_tokens: 32, normalize: true, ...options,
|
||||
}));
|
||||
expect(sent).toHaveLength(1);
|
||||
expect(sent[0]!.speed).toBe(speed === 'fast' ? 'fast' : undefined);
|
||||
expect(result).toMatchObject({
|
||||
message: { content: 'OK' }, usage: { usd_cents: cents },
|
||||
});
|
||||
} finally {
|
||||
d.onServerShutdown();
|
||||
vi.unstubAllGlobals();
|
||||
}
|
||||
});
|
||||
|
||||
it.each([
|
||||
null, false, true, 0, 1, -1, NaN, Infinity, '', 'invalid', 'FAST',
|
||||
'fast ', ' fast', 'ultraFast', [], ['fast'], {}, { speed: 'fast' },
|
||||
{ toString: 'fast' },
|
||||
])('rejects invalid speed %j before sending a request', async (serviceSpeed) => {
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
driver.complete({
|
||||
model: 'fake',
|
||||
messages: [{ role: 'user', content: 'Hello' }],
|
||||
serviceSpeed: serviceSpeed as never,
|
||||
}),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
legacyCode: 'invalid_service_speed',
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
['openai-responses', true],
|
||||
['openai-responses', false],
|
||||
['openai-completion', false],
|
||||
] as const)(
|
||||
'settles unavailable tiers at standard rates on %s with provider fallback disabled (admission rejection: %s)',
|
||||
async (provider, rejectFast) => {
|
||||
const sent: Record<string, unknown>[] = [];
|
||||
vi.stubGlobal(
|
||||
'fetch',
|
||||
async (_input: unknown, init?: RequestInit) => {
|
||||
const body =
|
||||
_input instanceof Request
|
||||
? await _input.clone().json()
|
||||
: JSON.parse(String(init?.body));
|
||||
sent.push(body);
|
||||
if (rejectFast && body.service_tier !== 'default') {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
error: {
|
||||
message: `${body.service_tier} service_tier capacity unavailable`,
|
||||
type: 'invalid_request_error',
|
||||
},
|
||||
}),
|
||||
{
|
||||
status: 400,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
},
|
||||
);
|
||||
}
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
id: 'response-test',
|
||||
object: 'response',
|
||||
model: body.model,
|
||||
status: 'completed',
|
||||
service_tier: 'default',
|
||||
output: [
|
||||
{
|
||||
type: 'message',
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{ type: 'output_text', text: 'OK' },
|
||||
],
|
||||
},
|
||||
],
|
||||
usage: {
|
||||
input_tokens: 10,
|
||||
output_tokens: 2,
|
||||
input_tokens_details: { cached_tokens: 0 },
|
||||
},
|
||||
}),
|
||||
{ headers: { 'content-type': 'application/json' } },
|
||||
);
|
||||
},
|
||||
);
|
||||
const d = new ChatCompletionDriver(
|
||||
{
|
||||
providers: {
|
||||
'openai-completion': { apiKey: 'test-key' },
|
||||
ollama: { enabled: false },
|
||||
},
|
||||
} as never,
|
||||
server.clients,
|
||||
server.stores,
|
||||
server.services,
|
||||
);
|
||||
try {
|
||||
await d.onServerStart();
|
||||
const result = (await withTestActor(() =>
|
||||
d.complete({
|
||||
model: 'gpt-6.1-sol',
|
||||
provider,
|
||||
messages: [{ role: 'user', content: 'Hello' }],
|
||||
serviceSpeed: 'ultrafast',
|
||||
fallback: false,
|
||||
max_tokens: 32,
|
||||
normalize: true,
|
||||
}),
|
||||
)) as {
|
||||
message: { content: string };
|
||||
usageDetails: { serviceSpeed: string };
|
||||
usage: { usd_cents: number };
|
||||
};
|
||||
expect(sent.map((body) => body.service_tier)).toEqual(
|
||||
rejectFast
|
||||
? ['ultrafast', 'priority', 'default']
|
||||
: ['ultrafast'],
|
||||
);
|
||||
expect(sent.every((body) => body.model === 'gpt-6.1-sol')).toBe(
|
||||
true,
|
||||
);
|
||||
expect(result.message.content).toBe('OK');
|
||||
expect(result.usageDetails.serviceSpeed).toBe('standard');
|
||||
expect(result.usage.usd_cents).toBeCloseTo(0.004, 8);
|
||||
} finally {
|
||||
d.onServerShutdown();
|
||||
vi.unstubAllGlobals();
|
||||
}
|
||||
},
|
||||
);
|
||||
});
|
||||
@@ -78,6 +78,11 @@ import type {
|
||||
UsageDetails,
|
||||
} from './types.js';
|
||||
import { normalize_tools_object } from './utils/FunctionCalling.js';
|
||||
import {
|
||||
selectServiceSpeed,
|
||||
lowerServiceSpeed,
|
||||
isServiceSpeedUnavailable,
|
||||
} from './utils/serviceSpeed.js';
|
||||
import {
|
||||
messagesHavePuterPaths,
|
||||
modelSupportsModality,
|
||||
@@ -674,6 +679,16 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
}
|
||||
|
||||
async complete(args: ICompleteArguments): Promise<IChatCompleteResult> {
|
||||
if (
|
||||
args.serviceSpeed !== undefined &&
|
||||
!['standard', 'fast', 'ultrafast'].includes(args.serviceSpeed)
|
||||
) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'serviceSpeed must be standard, fast, or ultrafast',
|
||||
{ legacyCode: 'invalid_service_speed' },
|
||||
);
|
||||
}
|
||||
const abortSignal = Context.get('abortSignal');
|
||||
abortSignal?.throwIfAborted();
|
||||
const actor = Context.get('actor');
|
||||
@@ -700,15 +715,16 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
args.model = this.#providers[intendedProvider].getDefaultModel();
|
||||
}
|
||||
|
||||
let model = await this.#resolveLoadedModel(
|
||||
const initialModel = await this.#resolveLoadedModel(
|
||||
args.model,
|
||||
intendedProvider,
|
||||
);
|
||||
if (!model) {
|
||||
if (!initialModel) {
|
||||
throw new HttpError(400, `Model not found: ${args.model}`, {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
let model: IChatModel = initialModel;
|
||||
// A pinned provider without a route for the model normally resolves
|
||||
// to another vendor; with `fallback: false` that is the one thing the
|
||||
// caller asked us not to do.
|
||||
@@ -840,18 +856,6 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
// The instant the current attempt is priced at, from its hold to its
|
||||
// charge, so a rate change mid-request can't split the two.
|
||||
let pricingTime = new Date();
|
||||
if (!useFakeProvider) {
|
||||
({
|
||||
hold,
|
||||
fundsCap,
|
||||
costFactor,
|
||||
promptTokenEstimate: attemptPromptTokenEstimate,
|
||||
} = await this.#applyCreditGate(actor, model, args, {
|
||||
promptTokenEstimate,
|
||||
requestedMaxTokens,
|
||||
pricingTime,
|
||||
}));
|
||||
}
|
||||
|
||||
// First attempt
|
||||
const provider = this.#providers[model.provider!];
|
||||
@@ -899,24 +903,134 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
return attempt;
|
||||
};
|
||||
|
||||
let variantSpeed: ICompleteArguments['serviceSpeed'];
|
||||
let creditGateError: unknown;
|
||||
const runAttempt = async (
|
||||
attemptProvider: IChatProvider,
|
||||
attemptModel: IChatModel,
|
||||
baseModel: IChatModel,
|
||||
) => {
|
||||
if (!useFakeProvider) {
|
||||
await this.#resolvePuterPaths(attemptProvider, args, actor);
|
||||
}
|
||||
// Last point where a caller who left costs nothing.
|
||||
abortSignal?.throwIfAborted();
|
||||
return attemptProvider.complete(
|
||||
{
|
||||
...args,
|
||||
const offers =
|
||||
args.serviceSpeed !== undefined &&
|
||||
args.serviceSpeed !== 'standard'
|
||||
? ((await attemptProvider.serviceSpeedModels?.(
|
||||
baseModel,
|
||||
)) ?? {})
|
||||
: {};
|
||||
let requestedSpeed =
|
||||
args.serviceSpeed ??
|
||||
(baseModel.provider === 'claude' && args.speed === 'fast'
|
||||
? 'fast'
|
||||
: undefined);
|
||||
for (;;) {
|
||||
const selection = selectServiceSpeed(
|
||||
baseModel,
|
||||
{ ...args, serviceSpeed: requestedSpeed },
|
||||
(id) => {
|
||||
const variant = this.#resolveModel(
|
||||
id,
|
||||
baseModel.provider!,
|
||||
);
|
||||
return variant &&
|
||||
variant.provider === baseModel.provider
|
||||
? variant
|
||||
: undefined;
|
||||
},
|
||||
offers,
|
||||
);
|
||||
const attemptModel = selection.model;
|
||||
const requestArgs = {
|
||||
...selection.args,
|
||||
model: attemptModel.id,
|
||||
provider: attemptModel.provider,
|
||||
},
|
||||
this.#catalogEntries.get(attemptModel),
|
||||
pricingTime,
|
||||
);
|
||||
};
|
||||
const catalogModel = this.#resolveModel(
|
||||
attemptModel.id,
|
||||
attemptModel.provider!,
|
||||
);
|
||||
const rawModel = this.#catalogEntries.get(
|
||||
catalogModel ?? baseModel,
|
||||
);
|
||||
const snapshot = rawModel
|
||||
? {
|
||||
...rawModel,
|
||||
costs: attemptModel.costs,
|
||||
pricingOverrides: attemptModel.pricingOverrides,
|
||||
serviceSpeedPricing: attemptModel.serviceSpeedPricing,
|
||||
}
|
||||
: attemptModel;
|
||||
if (!useFakeProvider) {
|
||||
try {
|
||||
({
|
||||
hold,
|
||||
fundsCap,
|
||||
costFactor,
|
||||
promptTokenEstimate: attemptPromptTokenEstimate,
|
||||
} = await this.#applyCreditGate(
|
||||
actor,
|
||||
attemptModel,
|
||||
requestArgs,
|
||||
{
|
||||
promptTokenEstimate,
|
||||
requestedMaxTokens,
|
||||
pricingTime,
|
||||
},
|
||||
));
|
||||
} catch (error) {
|
||||
creditGateError = error;
|
||||
throw error;
|
||||
}
|
||||
await this.#resolvePuterPaths(
|
||||
attemptProvider,
|
||||
requestArgs,
|
||||
actor,
|
||||
);
|
||||
}
|
||||
abortSignal?.throwIfAborted();
|
||||
try {
|
||||
const result = await attemptProvider.complete(
|
||||
requestArgs,
|
||||
snapshot,
|
||||
pricingTime,
|
||||
);
|
||||
model = attemptModel;
|
||||
variantSpeed =
|
||||
attemptModel.id !== baseModel.id
|
||||
? selection.speed
|
||||
: undefined;
|
||||
if (
|
||||
args.serviceSpeed !== undefined &&
|
||||
'message' in result
|
||||
) {
|
||||
const messageResult = result as IChatMessageResult;
|
||||
const details =
|
||||
messageResult.usageDetails ??
|
||||
usageDetailsFromUsage(messageResult.usage, model);
|
||||
messageResult.usageDetails = {
|
||||
...details,
|
||||
serviceSpeed:
|
||||
variantSpeed ??
|
||||
details?.serviceSpeed ??
|
||||
details?.speed ??
|
||||
'standard',
|
||||
};
|
||||
}
|
||||
return result;
|
||||
} catch (error) {
|
||||
await hold.release();
|
||||
hold = NO_CREDIT_HOLD;
|
||||
if (
|
||||
args.serviceSpeed === undefined ||
|
||||
selection.speed === 'standard' ||
|
||||
!isServiceSpeedUnavailable(
|
||||
error,
|
||||
attemptModel.id !== baseModel.id,
|
||||
)
|
||||
)
|
||||
throw error;
|
||||
requestedSpeed = lowerServiceSpeed(selection.speed);
|
||||
pricingTime = new Date();
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
try {
|
||||
@@ -984,29 +1098,17 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
// The previous attempt released its hold when it failed, so
|
||||
// this one starts from nothing held.
|
||||
pricingTime = new Date();
|
||||
if (!useFakeProvider) {
|
||||
({
|
||||
hold,
|
||||
fundsCap,
|
||||
costFactor,
|
||||
promptTokenEstimate: attemptPromptTokenEstimate,
|
||||
} = await this.#applyCreditGate(actor, fallback, args, {
|
||||
promptTokenEstimate,
|
||||
requestedMaxTokens,
|
||||
pricingTime,
|
||||
}));
|
||||
}
|
||||
|
||||
tried.add(routeId(fallback.provider!, fallback.id));
|
||||
|
||||
try {
|
||||
res = await runAttempt(fbProvider, fallback);
|
||||
model = fallback;
|
||||
lastError = null;
|
||||
console.warn(
|
||||
`[ai-chat] fallback served (${completionId}, ${first.provider}:${first.model} -> ${fallback.provider}:${fallback.id}, first ${first.status ?? 'none'}${first.upstreamType ? `/${first.upstreamType}` : ''})`,
|
||||
);
|
||||
} catch (fbErr) {
|
||||
if (fbErr === creditGateError) throw fbErr;
|
||||
await hold.release();
|
||||
hold = NO_CREDIT_HOLD;
|
||||
abortSignal?.throwIfAborted();
|
||||
@@ -1162,6 +1264,20 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
usageDetailsFromUsage(usage, model),
|
||||
);
|
||||
}
|
||||
if (
|
||||
args.serviceSpeed !== undefined &&
|
||||
chatStream.usageDetails
|
||||
) {
|
||||
const details = chatStream.usageDetails;
|
||||
chatStream.setUsageDetails({
|
||||
...details,
|
||||
serviceSpeed:
|
||||
variantSpeed ??
|
||||
details.serviceSpeed ??
|
||||
details.speed ??
|
||||
'standard',
|
||||
});
|
||||
}
|
||||
withClaudeThinkingTokens(
|
||||
enrichedUsage,
|
||||
model,
|
||||
@@ -1474,12 +1590,7 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
const metering = this.services.metering;
|
||||
const { requestedMaxTokens, pricingTime } = estimates;
|
||||
let { promptTokenEstimate } = estimates;
|
||||
// A provider can name its own cost keys for this request (Claude's
|
||||
// fast-mode rate card), cap its output, and size an extra reservation
|
||||
// the per-token math can't see (a web-search/advisor tool that could
|
||||
// run several times before the model answers) — the same clamps it
|
||||
// applies when building the request, so the hold and the request
|
||||
// agree.
|
||||
// Providers reserve tool calls using the same caps they send upstream.
|
||||
const pricing =
|
||||
this.#providers[model.provider!]?.requestPricing?.(args, model, {
|
||||
promptTokenEstimate,
|
||||
|
||||
@@ -41,6 +41,7 @@ import {
|
||||
meterChatUsage,
|
||||
type MeterChatUsageOptions,
|
||||
} from '../utils/meterChatUsage.js';
|
||||
import { servedServiceSpeed } from '../utils/serviceSpeed.js';
|
||||
import { catalogPrice, costKeys } from '../utils/pricing.js';
|
||||
import { normalizeModelKey } from '../utils/modelRouting.js';
|
||||
import * as OpenAIUtil from '../utils/OpenAIUtil.js';
|
||||
@@ -232,6 +233,8 @@ export class OpenAICompatProvider implements IChatProvider {
|
||||
)
|
||||
: {}),
|
||||
};
|
||||
if (args.serviceSpeed !== undefined && args.service_tier !== undefined)
|
||||
shared.service_tier = args.service_tier;
|
||||
const params = this.vendorParams(shared, args, model, actor);
|
||||
|
||||
let pricingTime = attemptPricingTime ?? new Date();
|
||||
@@ -271,15 +274,29 @@ export class OpenAICompatProvider implements IChatProvider {
|
||||
source,
|
||||
model,
|
||||
);
|
||||
if (usageDetails) source.setUsageDetails?.(usageDetails);
|
||||
const serviceSpeed = servedServiceSpeed(source);
|
||||
if (usageDetails)
|
||||
source.setUsageDetails?.({
|
||||
...usageDetails,
|
||||
...(args.serviceSpeed !== undefined
|
||||
? { serviceSpeed }
|
||||
: {}),
|
||||
});
|
||||
const metered = meterChatUsage(
|
||||
this.#metering,
|
||||
actor,
|
||||
this.meteringModelKey(model.id),
|
||||
model,
|
||||
usage,
|
||||
{ ...meterOptions, pricingTime },
|
||||
{ ...meterOptions, pricingTime, serviceSpeed },
|
||||
);
|
||||
if (args.serviceSpeed !== undefined && !usageDetails) {
|
||||
const details = OpenAIUtil.usageDetailsFromTrackedUsage(
|
||||
metered.usage,
|
||||
);
|
||||
if (details)
|
||||
source.setUsageDetails?.({ ...details, serviceSpeed });
|
||||
}
|
||||
source.setUsageCosts(metered.costs);
|
||||
return this.reportedUsage(metered.usage);
|
||||
},
|
||||
|
||||
@@ -174,6 +174,9 @@ export const mapAlibabaCatalogModel = (
|
||||
: undefined;
|
||||
return {
|
||||
...existing,
|
||||
serviceSpeedVariants: {
|
||||
fast: id === 'glm-5.2' ? 'glm-5.2-fast-preview' : `${id}-prime`,
|
||||
},
|
||||
id,
|
||||
puterId,
|
||||
name: raw.name ?? existing?.name ?? id,
|
||||
|
||||
@@ -37,6 +37,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-max-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 160,
|
||||
@@ -60,6 +61,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-plus-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -81,6 +83,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-plus-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -102,6 +105,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.6-plus-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 50,
|
||||
@@ -123,6 +127,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.6-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 19,
|
||||
@@ -143,6 +148,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.7-plus-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 50,
|
||||
@@ -164,6 +170,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.7-max-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
@@ -188,6 +195,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.8-max-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -210,6 +218,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.8-max-0902-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -234,6 +243,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 5,
|
||||
@@ -257,6 +267,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3-coder-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30,
|
||||
@@ -282,6 +293,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-397b-a17b-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 60,
|
||||
@@ -305,6 +317,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-122b-a10b-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -328,6 +341,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-35b-a3b-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 25,
|
||||
@@ -351,6 +365,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-27b-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30,
|
||||
@@ -376,6 +391,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.6-35b-a3b-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 24.8,
|
||||
@@ -399,6 +415,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.6-27b-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 60,
|
||||
@@ -422,6 +439,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3-vl-plus-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 20,
|
||||
@@ -447,6 +465,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3-omni-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 43,
|
||||
@@ -469,6 +488,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-mt-plus-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 246,
|
||||
@@ -491,6 +511,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-plus-2025-01-25-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -511,6 +532,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-plus-2025-04-28-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -531,6 +553,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-plus-2025-07-14-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -553,6 +576,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-plus-2025-12-01-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -574,6 +598,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.7-max-2026-05-20-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
@@ -594,6 +619,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.7-max-2026-06-08-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
@@ -614,6 +640,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.7-max-2026-05-17-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
@@ -635,6 +662,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.7-max-preview-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
@@ -657,6 +685,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-plus-2026-02-15-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -678,6 +707,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-plus-2026-04-20-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -700,6 +730,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 10,
|
||||
@@ -722,6 +753,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.7-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 3,
|
||||
@@ -743,6 +775,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.8-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 15,
|
||||
@@ -765,6 +798,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.8-27b-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 50,
|
||||
@@ -785,6 +819,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.8-2.4t-a95b-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -809,6 +844,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-omni-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -832,6 +868,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-omni-plus-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 140,
|
||||
@@ -855,6 +892,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.8-omni-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 15,
|
||||
@@ -877,6 +915,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-mt-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 16,
|
||||
@@ -897,6 +936,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-mt-lite-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 12,
|
||||
@@ -919,6 +959,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'glm-5.1-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 140,
|
||||
@@ -938,6 +979,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'glm-5.2-fast-preview' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 140,
|
||||
@@ -958,6 +1000,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'glm-5.2-fast-preview-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 280,
|
||||
@@ -981,6 +1024,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'deepseek-v4-flash-0731-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 44,
|
||||
@@ -1003,6 +1047,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'deepseek-v4.1-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30,
|
||||
@@ -1023,6 +1068,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'deepseek-v4-pro-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 240,
|
||||
@@ -1045,6 +1091,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'deepseek-v4-pro-0813-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 132,
|
||||
@@ -1065,6 +1112,7 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'kimi-k2.7-code-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 95,
|
||||
|
||||
@@ -20,6 +20,10 @@
|
||||
import { HttpError } from '@heyputer/backend/src/core/http/HttpError.js';
|
||||
import { OpenAI } from 'openai';
|
||||
import { ChatCompletionCreateParams } from 'openai/resources/index.js';
|
||||
import {
|
||||
serviceSpeedBillingModel,
|
||||
servedServiceSpeed,
|
||||
} from '../../utils/serviceSpeed.js';
|
||||
import { Context } from '../../../../core/context.js';
|
||||
import type { FSService } from '../../../../services/fs/FSService.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
@@ -153,6 +157,7 @@ export class AzureChatProvider implements IChatProvider {
|
||||
|
||||
async complete(
|
||||
params: ICompleteArguments,
|
||||
resolved?: IChatModel,
|
||||
): ReturnType<IChatProvider['complete']> {
|
||||
const {
|
||||
max_tokens,
|
||||
@@ -180,7 +185,9 @@ export class AzureChatProvider implements IChatProvider {
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
return await this.#responsesProvider.complete(params);
|
||||
return await (resolved
|
||||
? this.#responsesProvider.complete(params, resolved)
|
||||
: this.#responsesProvider.complete(params));
|
||||
}
|
||||
// Inline compaction is a Responses-API feature; chat.completions can't
|
||||
// express `context_management` or a `compaction` content block.
|
||||
@@ -194,7 +201,9 @@ export class AzureChatProvider implements IChatProvider {
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
return await this.#responsesProvider.complete(params);
|
||||
return await (resolved
|
||||
? this.#responsesProvider.complete(params, resolved)
|
||||
: this.#responsesProvider.complete(params));
|
||||
}
|
||||
// Validate messages
|
||||
if (!Array.isArray(messages)) {
|
||||
@@ -207,9 +216,11 @@ export class AzureChatProvider implements IChatProvider {
|
||||
model = model ?? this.#defaultModel;
|
||||
|
||||
const modelUsed =
|
||||
this.models().find((m) =>
|
||||
resolved ??
|
||||
(this.models().find((m) =>
|
||||
[m.id, ...(m.aliases || [])].includes(model),
|
||||
) || this.models().find((m) => m.id === this.getDefaultModel())!;
|
||||
) ||
|
||||
this.models().find((m) => m.id === this.getDefaultModel())!);
|
||||
|
||||
// messages.unshift({
|
||||
// role: 'system',
|
||||
@@ -279,6 +290,9 @@ export class AzureChatProvider implements IChatProvider {
|
||||
}),
|
||||
messages: messages,
|
||||
model: modelUsed.id,
|
||||
...(params.service_tier !== undefined
|
||||
? { service_tier: params.service_tier }
|
||||
: {}),
|
||||
...(mappedTools?.length ? { tools: mappedTools } : {}),
|
||||
...(max_tokens !== undefined
|
||||
? { max_completion_tokens: max_tokens }
|
||||
@@ -315,7 +329,12 @@ export class AzureChatProvider implements IChatProvider {
|
||||
);
|
||||
|
||||
return OpenAiUtil.handle_completion_output({
|
||||
usage_calculator: ({ usage }) => {
|
||||
usage_calculator: ({
|
||||
service_tier,
|
||||
usage,
|
||||
setUsageCosts,
|
||||
setUsageDetails,
|
||||
}) => {
|
||||
const cachedTokens =
|
||||
usage.prompt_tokens_details?.cached_tokens ?? 0;
|
||||
const trackedUsage = {
|
||||
@@ -335,9 +354,20 @@ export class AzureChatProvider implements IChatProvider {
|
||||
|
||||
const costsOverrideFromModel = buildCostsOverride(
|
||||
trackedUsage,
|
||||
modelUsed,
|
||||
serviceSpeedBillingModel(
|
||||
modelUsed,
|
||||
servedServiceSpeed({ service_tier }),
|
||||
),
|
||||
);
|
||||
|
||||
setUsageCosts(costsOverrideFromModel);
|
||||
if (params.serviceSpeed !== undefined)
|
||||
setUsageDetails({
|
||||
...OpenAiUtil.usageDetailsFromTrackedUsage(
|
||||
trackedUsage,
|
||||
)!,
|
||||
serviceSpeed: servedServiceSpeed({ service_tier }),
|
||||
});
|
||||
this.#meteringService.utilRecordUsageObject(
|
||||
trackedUsage,
|
||||
actor!,
|
||||
|
||||
@@ -19,6 +19,10 @@
|
||||
|
||||
import { OpenAI } from 'openai';
|
||||
import { ResponseCreateParams } from 'openai/resources/responses/responses.mjs';
|
||||
import {
|
||||
serviceSpeedBillingModel,
|
||||
servedServiceSpeed,
|
||||
} from '../../utils/serviceSpeed.js';
|
||||
import { Context } from '../../../../core/context.js';
|
||||
import type { FSService } from '../../../../services/fs/FSService.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
@@ -152,37 +156,41 @@ export class AzureResponsesProvider implements IChatProvider {
|
||||
: {};
|
||||
}
|
||||
|
||||
async complete({
|
||||
messages,
|
||||
model,
|
||||
max_tokens,
|
||||
moderation,
|
||||
tools,
|
||||
tool_choice,
|
||||
parallel_tool_calls,
|
||||
include,
|
||||
conversation,
|
||||
compaction,
|
||||
context_management,
|
||||
previous_response_id,
|
||||
instructions,
|
||||
metadata,
|
||||
prompt,
|
||||
prompt_cache_key,
|
||||
prompt_cache_retention,
|
||||
store,
|
||||
top_p,
|
||||
truncation,
|
||||
background,
|
||||
service_tier,
|
||||
verbosity,
|
||||
stream,
|
||||
reasoning,
|
||||
reasoning_effort,
|
||||
temperature,
|
||||
text,
|
||||
outputFormat,
|
||||
}: ICompleteArguments): ReturnType<IChatProvider['complete']> {
|
||||
async complete(
|
||||
{
|
||||
messages,
|
||||
model,
|
||||
max_tokens,
|
||||
moderation,
|
||||
tools,
|
||||
tool_choice,
|
||||
parallel_tool_calls,
|
||||
include,
|
||||
conversation,
|
||||
compaction,
|
||||
context_management,
|
||||
previous_response_id,
|
||||
instructions,
|
||||
metadata,
|
||||
prompt,
|
||||
prompt_cache_key,
|
||||
prompt_cache_retention,
|
||||
store,
|
||||
top_p,
|
||||
truncation,
|
||||
background,
|
||||
service_tier,
|
||||
serviceSpeed: requestedServiceSpeed,
|
||||
verbosity,
|
||||
stream,
|
||||
reasoning,
|
||||
reasoning_effort,
|
||||
temperature,
|
||||
text,
|
||||
outputFormat,
|
||||
}: ICompleteArguments,
|
||||
resolved?: IChatModel,
|
||||
): ReturnType<IChatProvider['complete']> {
|
||||
// Validate messages
|
||||
if (!Array.isArray(messages)) {
|
||||
throw new HttpError(400, '`messages` must be an array', {
|
||||
@@ -200,12 +208,13 @@ export class AzureResponsesProvider implements IChatProvider {
|
||||
model = model ?? this.#defaultModel;
|
||||
|
||||
const modelUsed =
|
||||
this.models({ no_restrictions: true }).find((m) =>
|
||||
resolved ??
|
||||
(this.models({ no_restrictions: true }).find((m) =>
|
||||
[m.id, ...(m.aliases || [])].includes(model),
|
||||
) ||
|
||||
this.models({ no_restrictions: true }).find(
|
||||
(m) => m.id === this.getDefaultModel(),
|
||||
)!;
|
||||
this.models({ no_restrictions: true }).find(
|
||||
(m) => m.id === this.getDefaultModel(),
|
||||
)!);
|
||||
|
||||
const userIdentifier = upstreamUserIdentifier(actor);
|
||||
// Cache key defaults to the actor identifier; see upstreamUserIdentifier.
|
||||
@@ -335,9 +344,11 @@ export class AzureResponsesProvider implements IChatProvider {
|
||||
return OpenAiUtil.handle_completion_output_responses_api({
|
||||
usage_calculator: ({
|
||||
usage,
|
||||
service_tier,
|
||||
webSearchCalls,
|
||||
tool_usage,
|
||||
setUsageCosts,
|
||||
setUsageDetails,
|
||||
}) => {
|
||||
const numWebSearches: number =
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
@@ -363,7 +374,10 @@ export class AzureResponsesProvider implements IChatProvider {
|
||||
|
||||
const costsOverrideFromModel = buildCostsOverride(
|
||||
trackedUsage,
|
||||
modelUsed,
|
||||
serviceSpeedBillingModel(
|
||||
modelUsed,
|
||||
servedServiceSpeed({ service_tier }),
|
||||
),
|
||||
);
|
||||
if (numWebSearches) {
|
||||
costsOverrideFromModel.web_search_calls =
|
||||
@@ -374,6 +388,14 @@ export class AzureResponsesProvider implements IChatProvider {
|
||||
});
|
||||
}
|
||||
|
||||
setUsageCosts?.(costsOverrideFromModel);
|
||||
if (requestedServiceSpeed !== undefined)
|
||||
setUsageDetails?.({
|
||||
...OpenAiUtil.usageDetailsFromTrackedUsage(
|
||||
trackedUsage,
|
||||
)!,
|
||||
serviceSpeed: servedServiceSpeed({ service_tier }),
|
||||
});
|
||||
this.#meteringService.utilRecordUsageObject(
|
||||
trackedUsage,
|
||||
actor,
|
||||
|
||||
@@ -40,7 +40,11 @@ const STANDALONE = [
|
||||
];
|
||||
|
||||
// Fields an Azure deployment genuinely differs from its source on, by puterId.
|
||||
const OVERRIDES: Record<string, string[]> = {};
|
||||
const OVERRIDES: Record<string, string[]> = {
|
||||
'azure:x-ai/grok-4.3': ['serviceSpeedRates'],
|
||||
'azure:x-ai/grok-4-20-non-reasoning': ['serviceSpeedRates'],
|
||||
'azure:x-ai/grok-4-20-reasoning': ['serviceSpeedRates'],
|
||||
};
|
||||
|
||||
const IDENTITY = ['id', 'puterId', 'aliases'];
|
||||
|
||||
@@ -65,6 +69,18 @@ const mirrored = AZURE_MODELS.filter(
|
||||
).map((m) => [m.puterId!, m] as const);
|
||||
|
||||
describe('AZURE_MODELS mirroring', () => {
|
||||
it('inherits OpenAI fast rates without enabling direct-only speed tiers', () => {
|
||||
for (const model of AZURE_MODELS) {
|
||||
expect(model.serviceSpeedRates?.ultrafast).toBeUndefined();
|
||||
if (model.puterId?.startsWith('azure:x-ai/')) {
|
||||
expect(model.serviceSpeedRates?.fast).toBeUndefined();
|
||||
}
|
||||
}
|
||||
const azure = AZURE_MODELS.find((model) => model.id === 'gpt-5.4')!;
|
||||
expect(azure.serviceSpeedRates).toEqual({
|
||||
fast: sourceOf(azure)!.serviceSpeedRates!.fast,
|
||||
});
|
||||
});
|
||||
it('has a direct-provider source for every entry not listed as standalone', () => {
|
||||
const orphans = mirrored
|
||||
.filter(([, m]) => !sourceOf(m))
|
||||
|
||||
@@ -22,10 +22,8 @@ import { OPEN_AI_MODELS } from '../openai/models.js';
|
||||
import { XAI_MODELS } from '../xai/models.js';
|
||||
|
||||
/**
|
||||
* An Azure deployment of a model we also serve direct. Every field but the
|
||||
* identity comes from the source entry, so prices, long-context tiers and
|
||||
* limits can't drift from it. `name` is the vendor-qualified name both catalogs
|
||||
* share (`openai/gpt-5.4`); extra fields override the source.
|
||||
* Azure shares its source model's list prices, context tiers, and limits.
|
||||
* Identity and available service speeds belong to the deployment.
|
||||
*/
|
||||
const mirror = (
|
||||
source: readonly IChatModel[],
|
||||
@@ -41,6 +39,11 @@ const mirror = (
|
||||
puterId: `azure:${name}`,
|
||||
id: azure.id,
|
||||
...shared,
|
||||
serviceSpeedRates:
|
||||
source === OPEN_AI_MODELS &&
|
||||
shared.serviceSpeedRates?.fast !== undefined
|
||||
? { fast: shared.serviceSpeedRates.fast }
|
||||
: undefined,
|
||||
aliases: [name],
|
||||
...azure,
|
||||
};
|
||||
@@ -127,6 +130,13 @@ export const AZURE_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 250,
|
||||
cached_tokens: 25,
|
||||
completion_tokens: 2000,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
@@ -171,6 +181,13 @@ export const AZURE_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 45,
|
||||
cached_tokens: 4.5,
|
||||
completion_tokens: 360,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 25,
|
||||
|
||||
@@ -147,6 +147,22 @@ describe.skipIf(skipUnlessEnv(ENV_VAR))('ClaudeProvider (integration)', () => {
|
||||
},
|
||||
);
|
||||
|
||||
it('returns a completion from claude-fable-5-2', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
|
||||
const provider = buildProvider();
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'claude-fable-5-2',
|
||||
messages: [{ role: 'user', content: 'Reply with OK.' }],
|
||||
reasoning_effort: 'low',
|
||||
max_tokens: 128,
|
||||
}),
|
||||
);
|
||||
const content = (result as {
|
||||
message: { content: Array<{ type: string; text?: string }> };
|
||||
}).message.content;
|
||||
expect(content.find((block) => block.type === 'text')?.text?.length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
// (a) Claude Code-shaped system blocks, attribution header first — the
|
||||
// pipeline must never merge or reorder system blocks: the header is
|
||||
// only recognized by Anthropic when it is the sole content of an
|
||||
|
||||
@@ -1173,6 +1173,7 @@ describe('ClaudeProvider.complete request shape', () => {
|
||||
});
|
||||
|
||||
it.each([
|
||||
'claude-fable-5-2',
|
||||
'claude-fable-5-1',
|
||||
'claude-haiku-5-5',
|
||||
'claude-opus-5-5',
|
||||
@@ -1198,6 +1199,7 @@ describe('ClaudeProvider.complete request shape', () => {
|
||||
);
|
||||
|
||||
it.each([
|
||||
'claude-fable-5-2',
|
||||
'claude-fable-5-1',
|
||||
'claude-haiku-5-5',
|
||||
'claude-opus-5-5',
|
||||
@@ -1303,6 +1305,73 @@ describe('ClaudeProvider model resolution', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it.each([
|
||||
'claude-fable-5-2',
|
||||
'claude-fable-5-2-latest',
|
||||
'claude-fable-5.2',
|
||||
'anthropic/claude-fable-5-2',
|
||||
])(
|
||||
'routes %s to Fable 5.2 with published rates',
|
||||
async (model) => {
|
||||
const { provider } = makeProvider();
|
||||
messagesCreateMock.mockResolvedValueOnce({
|
||||
content: [{ type: 'text', text: 'ok' }],
|
||||
usage: {
|
||||
input_tokens: 100,
|
||||
output_tokens: 50,
|
||||
cache_read_input_tokens: 1000,
|
||||
cache_creation_input_tokens: 200,
|
||||
cache_creation: {
|
||||
ephemeral_5m_input_tokens: 150,
|
||||
ephemeral_1h_input_tokens: 50,
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model,
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
|
||||
expect(messagesCreateMock.mock.calls[0]![0].model).toBe(
|
||||
'claude-fable-5-2',
|
||||
);
|
||||
const fable52 = CLAUDE_MODELS.find(
|
||||
(m) => m.id === 'claude-fable-5-2',
|
||||
)!;
|
||||
expect(fable52.costs).toMatchObject({
|
||||
input_tokens: 800,
|
||||
output_tokens: 4000,
|
||||
cache_read_input_tokens: 20,
|
||||
ephemeral_5m_input_tokens: 1000,
|
||||
ephemeral_1h_input_tokens: 1600,
|
||||
});
|
||||
expect(fable52.serviceSpeedRates?.fast).toMatchObject({
|
||||
input_tokens: 1600,
|
||||
output_tokens: 8000,
|
||||
cache_read_input_tokens: 40,
|
||||
ephemeral_5m_input_tokens: 2000,
|
||||
ephemeral_1h_input_tokens: 3200,
|
||||
});
|
||||
expect(fable52.context).toBe(1_000_000);
|
||||
expect(fable52.max_tokens).toBe(128_000);
|
||||
const [, , prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(prefix).toBe('claude:claude-fable-5-2');
|
||||
expect(overrides).toMatchObject({
|
||||
input_tokens: 100 * Number(fable52.costs.input_tokens),
|
||||
output_tokens: 50 * Number(fable52.costs.output_tokens),
|
||||
cache_read_input_tokens:
|
||||
1000 * Number(fable52.costs.cache_read_input_tokens),
|
||||
ephemeral_5m_input_tokens:
|
||||
150 * Number(fable52.costs.ephemeral_5m_input_tokens),
|
||||
ephemeral_1h_input_tokens:
|
||||
50 * Number(fable52.costs.ephemeral_1h_input_tokens),
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it('routes the bare claude-fable alias to fable 5.1 rather than fable 5', async () => {
|
||||
const { provider } = makeProvider();
|
||||
messagesCreateMock.mockResolvedValueOnce(baseResponse);
|
||||
@@ -2311,16 +2380,20 @@ describe('ClaudeProvider metering', () => {
|
||||
expect('thinking_tokens' in overrides).toBe(false);
|
||||
});
|
||||
|
||||
it('bills at the fast_* rates when the upstream reports usage.speed:"fast"', async () => {
|
||||
it.each(['claude-opus-5-5', 'claude-fable-5-2'])('bills at fast rates for %s when the upstream reports fast usage', async (model) => {
|
||||
const { provider } = makeProvider();
|
||||
messagesCreateMock.mockResolvedValueOnce({
|
||||
content: [{ type: 'text', text: 'hi' }],
|
||||
usage: { input_tokens: 10, output_tokens: 5, speed: 'fast' },
|
||||
usage: {
|
||||
input_tokens: 10, output_tokens: 5, speed: 'fast',
|
||||
cache_creation: { ephemeral_5m_input_tokens: 7, ephemeral_1h_input_tokens: 3 },
|
||||
cache_read_input_tokens: 20,
|
||||
},
|
||||
});
|
||||
|
||||
const result = (await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'claude-opus-5-5',
|
||||
model,
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
speed: 'fast',
|
||||
}),
|
||||
@@ -2331,11 +2404,15 @@ describe('ClaudeProvider metering', () => {
|
||||
expect('input_tokens' in result.usage).toBe(false);
|
||||
expect(result.usageDetails.speed).toBe('fast');
|
||||
|
||||
const opus = CLAUDE_MODELS.find((m) => m.id === 'claude-opus-5-5')!;
|
||||
const selectedModel = CLAUDE_MODELS.find((m) => m.id === model)!;
|
||||
const [, , prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(prefix).toBe('claude:claude-opus-5-5');
|
||||
expect(overrides.fast_input_tokens).toBe(10 * Number(opus.costs.fast_input_tokens));
|
||||
expect(overrides.fast_output_tokens).toBe(5 * Number(opus.costs.fast_output_tokens));
|
||||
expect(prefix).toBe(`claude:${model}`);
|
||||
const fastRates = selectedModel.serviceSpeedRates!.fast as Record<string, number>;
|
||||
expect(overrides.fast_input_tokens).toBe(10 * fastRates.input_tokens);
|
||||
expect(overrides.fast_output_tokens).toBe(5 * fastRates.output_tokens);
|
||||
expect(overrides.fast_ephemeral_5m_input_tokens).toBe(7 * fastRates.ephemeral_5m_input_tokens);
|
||||
expect(overrides.fast_ephemeral_1h_input_tokens).toBe(3 * fastRates.ephemeral_1h_input_tokens);
|
||||
expect(overrides.fast_cache_read_input_tokens).toBe(20 * fastRates.cache_read_input_tokens);
|
||||
});
|
||||
|
||||
it('bills at standard rates when fast mode was requested but not granted by the upstream', async () => {
|
||||
@@ -2357,13 +2434,15 @@ describe('ClaudeProvider metering', () => {
|
||||
expect('fast_input_tokens' in result.usage).toBe(false);
|
||||
});
|
||||
|
||||
it('retries at standard speed on Claude when fast mode is rate limited', async () => {
|
||||
it.each([
|
||||
[429, "This request would exceed your organization's rate limit of 0 fast mode input tokens per minute"],
|
||||
[529, 'Overloaded'],
|
||||
[529, 'Fast mode capacity unavailable'],
|
||||
])('retries at standard speed on Claude after fast capacity failure %s', async (status, message) => {
|
||||
const { provider } = makeProvider();
|
||||
const fastLimited = Object.assign(
|
||||
new Error(
|
||||
"This request would exceed your organization's rate limit of 0 fast mode input tokens per minute",
|
||||
),
|
||||
{ status: 429 },
|
||||
new Error(message),
|
||||
{ status },
|
||||
);
|
||||
messagesCreateMock.mockRejectedValueOnce(fastLimited).mockResolvedValueOnce({
|
||||
content: [{ type: 'text', text: 'hi' }],
|
||||
@@ -2587,13 +2666,13 @@ describe('ClaudeProvider.requestPricing', () => {
|
||||
const haiku = CLAUDE_MODELS.find((m) => m.id === 'claude-haiku-4-5-20251001')!;
|
||||
const opus55 = CLAUDE_MODELS.find((m) => m.id === 'claude-opus-5-5')!;
|
||||
|
||||
it('returns fast cost keys only when the model has fast rates and fast was requested', () => {
|
||||
it('leaves token reservation keys to the selected catalog rate card', () => {
|
||||
const { provider } = makeProvider();
|
||||
expect(
|
||||
provider.requestPricing({ speed: 'fast', tools: [] } as never, opus55, {
|
||||
promptTokenEstimate: 10,
|
||||
}),
|
||||
).toMatchObject({ inputKey: 'fast_input_tokens', outputKey: 'fast_output_tokens' });
|
||||
).toEqual({});
|
||||
expect(
|
||||
provider.requestPricing({ speed: 'fast', tools: [] } as never, haiku, {
|
||||
promptTokenEstimate: 10,
|
||||
|
||||
@@ -43,6 +43,7 @@ import {
|
||||
} from '../../utils/mediaParts.js';
|
||||
import { isToolChoice, toolChoiceFromWire } from '../../utils/openaiParams.js';
|
||||
import { buildCostsOverride } from '../../utils/pricing.js';
|
||||
import { serviceSpeedCosts } from '../../utils/serviceSpeed.js';
|
||||
import type {
|
||||
AIChatStream,
|
||||
AIChatTextStream,
|
||||
@@ -107,20 +108,21 @@ const toAnthropicMediaPart = (part: any): any => {
|
||||
return part;
|
||||
};
|
||||
|
||||
/** A 429 from fast mode's separate quota, not the model's normal limits. */
|
||||
const isFastModeRateLimit = (e: unknown): boolean => {
|
||||
/** Fast quota or overload failures can be retried at standard speed. */
|
||||
const isFastModeUnavailable = (e: unknown): boolean => {
|
||||
const err = e as { status?: unknown; message?: unknown };
|
||||
return (
|
||||
err?.status === 429 &&
|
||||
typeof err.message === 'string' &&
|
||||
/fast mode/i.test(err.message)
|
||||
typeof err?.message === 'string' &&
|
||||
((err?.status === 429 && /fast mode/i.test(err.message)) ||
|
||||
(err?.status === 529 && /overload|fast mode/i.test(err.message)))
|
||||
);
|
||||
};
|
||||
|
||||
// Models whose current turn rejects a non-default temperature/top_p/top_k
|
||||
// outright. Fable 5/5.1, Sonnet 5/5.5, Haiku 5.5, and Opus 4.7+.
|
||||
// outright. Fable 5/5.1/5.2, Sonnet 5/5.5, Haiku 5.5, and Opus 4.7+.
|
||||
const OMITS_SAMPLING_PARAMS = new Set([
|
||||
'claude-haiku-5-5',
|
||||
'claude-fable-5-2',
|
||||
'claude-fable-5-1',
|
||||
'claude-fable-5',
|
||||
'claude-sonnet-5-5',
|
||||
@@ -133,6 +135,7 @@ const OMITS_SAMPLING_PARAMS = new Set([
|
||||
|
||||
const SUPPORTS_EFFORT = new Set([
|
||||
'claude-haiku-5-5',
|
||||
'claude-fable-5-2',
|
||||
'claude-fable-5-1',
|
||||
'claude-fable-5',
|
||||
'claude-sonnet-5-5',
|
||||
@@ -298,20 +301,12 @@ export class ClaudeProvider implements IChatProvider {
|
||||
return `claude:${modelId}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sizes the credit hold for Claude-only extras the generic per-token math
|
||||
* can't see: fast mode's own rate card, and a web-search/advisor tool that
|
||||
* could run several times before the model answers.
|
||||
*/
|
||||
/** Reserves tools that can run several times before the model answers. */
|
||||
requestPricing(
|
||||
args: ICompleteArguments,
|
||||
model: IChatModel,
|
||||
est: { promptTokenEstimate: number },
|
||||
): { inputKey?: string; outputKey?: string; extraCost?: number } {
|
||||
const fast =
|
||||
args.speed === 'fast' &&
|
||||
model.costs?.fast_output_tokens !== undefined;
|
||||
|
||||
): { extraCost?: number } {
|
||||
let extraCost = 0;
|
||||
for (const raw of args.tools ?? []) {
|
||||
const tool = raw as Record<string, unknown>;
|
||||
@@ -350,12 +345,6 @@ export class ClaudeProvider implements IChatProvider {
|
||||
}
|
||||
|
||||
return {
|
||||
...(fast
|
||||
? {
|
||||
inputKey: 'fast_input_tokens',
|
||||
outputKey: 'fast_output_tokens',
|
||||
}
|
||||
: {}),
|
||||
...(extraCost > 0 ? { extraCost } : {}),
|
||||
};
|
||||
}
|
||||
@@ -364,11 +353,9 @@ export class ClaudeProvider implements IChatProvider {
|
||||
try {
|
||||
return await this.#completeOnce(args);
|
||||
} catch (e) {
|
||||
// Fast mode has its own quota. Anthropic's advice on that 429 is to
|
||||
// drop `speed` and run at standard speed; doing it here keeps the
|
||||
// request on Claude instead of marking the route unhealthy and
|
||||
// replaying it on a reseller.
|
||||
if (args.speed === 'fast' && isFastModeRateLimit(e)) {
|
||||
// Retry fast-capacity failures on Claude at standard speed before
|
||||
// considering a different provider.
|
||||
if (args.speed === 'fast' && isFastModeUnavailable(e)) {
|
||||
return await this.#completeOnce({ ...args, speed: undefined });
|
||||
}
|
||||
throw e;
|
||||
@@ -637,7 +624,7 @@ export class ClaudeProvider implements IChatProvider {
|
||||
: undefined;
|
||||
const fast =
|
||||
args.speed === 'fast' &&
|
||||
modelUsed.costs?.fast_output_tokens !== undefined;
|
||||
modelUsed.serviceSpeedRates?.fast !== undefined;
|
||||
const safeguardsPolicy = applySafeguardsPolicy(
|
||||
args.safeguards,
|
||||
args.anthropicBetas,
|
||||
@@ -1071,6 +1058,7 @@ export class ClaudeProvider implements IChatProvider {
|
||||
// keeps reasoning visible in the stream.
|
||||
if (
|
||||
modelId === 'claude-haiku-5-5' ||
|
||||
modelId === 'claude-fable-5-2' ||
|
||||
modelId === 'claude-fable-5-1' ||
|
||||
modelId === 'claude-fable-5' ||
|
||||
modelId === 'claude-sonnet-5-5' ||
|
||||
@@ -1505,7 +1493,8 @@ export class ClaudeProvider implements IChatProvider {
|
||||
|
||||
const fast =
|
||||
raw.speed === 'fast' &&
|
||||
modelUsed.costs?.fast_output_tokens !== undefined;
|
||||
modelUsed.serviceSpeedRates?.fast !== undefined;
|
||||
// Keep published usage keys while resolving rates from the common catalog shape.
|
||||
const executorUsage: Record<string, number> = fast
|
||||
? Object.fromEntries(
|
||||
Object.entries(exec).map(([k, v]) => [`fast_${k}`, v]),
|
||||
@@ -1517,7 +1506,24 @@ export class ClaudeProvider implements IChatProvider {
|
||||
const fetches = raw.server_tool_use?.web_fetch_requests ?? 0;
|
||||
if (fetches) executorUsage.web_fetch_requests = fetches;
|
||||
|
||||
const executorCosts = buildCostsOverride(executorUsage, modelUsed);
|
||||
const fastRates = fast
|
||||
? serviceSpeedCosts(modelUsed, 'fast')
|
||||
: undefined;
|
||||
const billingModel = fastRates
|
||||
? {
|
||||
...modelUsed,
|
||||
costs: {
|
||||
...modelUsed.costs,
|
||||
...Object.fromEntries(
|
||||
Object.keys(exec).map((key) => [
|
||||
`fast_${key}`,
|
||||
fastRates[key],
|
||||
]),
|
||||
),
|
||||
},
|
||||
}
|
||||
: modelUsed;
|
||||
const executorCosts = buildCostsOverride(executorUsage, billingModel);
|
||||
|
||||
let advisorUsage: Record<string, number> | undefined;
|
||||
let advisorCosts: Record<string, number> | undefined;
|
||||
|
||||
@@ -55,6 +55,46 @@ export const CLAUDE_MODELS: IChatModel[] = [
|
||||
context: 1_000_000,
|
||||
max_tokens: 128_000,
|
||||
},
|
||||
{
|
||||
puterId: 'anthropic:anthropic/claude-fable-5-2',
|
||||
id: 'claude-fable-5-2',
|
||||
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
release_date: '2026-11-09',
|
||||
aliases: [
|
||||
'claude-fable-5-2-latest',
|
||||
'claude-fable-5.2',
|
||||
'anthropic/claude-fable-5-2',
|
||||
],
|
||||
name: 'Claude Fable 5.2',
|
||||
midConversationSystem: true,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'input_tokens',
|
||||
output_cost_key: 'output_tokens',
|
||||
// Source: https://www.anthropic.com/claude-fable-and-mythos-5-2
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
input_tokens: 1600,
|
||||
output_tokens: 8000,
|
||||
ephemeral_5m_input_tokens: 1600 * 1.25,
|
||||
ephemeral_1h_input_tokens: 1600 * 2,
|
||||
cache_read_input_tokens: 1600 * 0.025,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
input_tokens: 800,
|
||||
ephemeral_5m_input_tokens: 800 * 1.25,
|
||||
ephemeral_1h_input_tokens: 800 * 2,
|
||||
// 0.025x input, not the usual 0.1x.
|
||||
cache_read_input_tokens: 800 * 0.025,
|
||||
output_tokens: 4000,
|
||||
web_search_requests: 1_000_000, // $10/1k requests
|
||||
},
|
||||
context: 1000000,
|
||||
max_tokens: 128000,
|
||||
},
|
||||
{
|
||||
puterId: 'anthropic:anthropic/claude-fable-5-1',
|
||||
id: 'claude-fable-5-1',
|
||||
@@ -190,6 +230,15 @@ export const CLAUDE_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'input_tokens',
|
||||
output_cost_key: 'output_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
input_tokens: 800,
|
||||
output_tokens: 4000,
|
||||
ephemeral_5m_input_tokens: 1000,
|
||||
ephemeral_1h_input_tokens: 1600,
|
||||
cache_read_input_tokens: 40,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
input_tokens: 400,
|
||||
@@ -199,12 +248,6 @@ export const CLAUDE_MODELS: IChatModel[] = [
|
||||
cache_read_input_tokens: 400 * 0.05,
|
||||
output_tokens: 2000,
|
||||
web_search_requests: 1_000_000,
|
||||
// Fast mode (`speed:'fast'`), live-verified.
|
||||
fast_input_tokens: 800,
|
||||
fast_output_tokens: 4000,
|
||||
fast_ephemeral_5m_input_tokens: 1000,
|
||||
fast_ephemeral_1h_input_tokens: 1600,
|
||||
fast_cache_read_input_tokens: 40, // 0.05x fast_input_tokens
|
||||
},
|
||||
context: 1000000,
|
||||
max_tokens: 128000,
|
||||
@@ -223,6 +266,15 @@ export const CLAUDE_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'input_tokens',
|
||||
output_cost_key: 'output_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
input_tokens: 1000,
|
||||
output_tokens: 5000,
|
||||
ephemeral_5m_input_tokens: 1250,
|
||||
ephemeral_1h_input_tokens: 2000,
|
||||
cache_read_input_tokens: 100,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
input_tokens: 500,
|
||||
@@ -231,11 +283,6 @@ export const CLAUDE_MODELS: IChatModel[] = [
|
||||
cache_read_input_tokens: 500 * 0.1,
|
||||
output_tokens: 2500,
|
||||
web_search_requests: 1_000_000,
|
||||
fast_input_tokens: 1000,
|
||||
fast_output_tokens: 5000,
|
||||
fast_ephemeral_5m_input_tokens: 1250,
|
||||
fast_ephemeral_1h_input_tokens: 2000,
|
||||
fast_cache_read_input_tokens: 100,
|
||||
},
|
||||
context: 1000000,
|
||||
max_tokens: 128000,
|
||||
@@ -258,6 +305,15 @@ export const CLAUDE_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'input_tokens',
|
||||
output_cost_key: 'output_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
input_tokens: 1000,
|
||||
output_tokens: 5000,
|
||||
ephemeral_5m_input_tokens: 1250,
|
||||
ephemeral_1h_input_tokens: 2000,
|
||||
cache_read_input_tokens: 100,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
input_tokens: 500,
|
||||
@@ -266,11 +322,6 @@ export const CLAUDE_MODELS: IChatModel[] = [
|
||||
cache_read_input_tokens: 500 * 0.1,
|
||||
output_tokens: 2500,
|
||||
web_search_requests: 1_000_000,
|
||||
fast_input_tokens: 1000,
|
||||
fast_output_tokens: 5000,
|
||||
fast_ephemeral_5m_input_tokens: 1250,
|
||||
fast_ephemeral_1h_input_tokens: 2000,
|
||||
fast_cache_read_input_tokens: 100,
|
||||
},
|
||||
context: 1000000,
|
||||
max_tokens: 128000,
|
||||
|
||||
@@ -84,8 +84,14 @@ export class GeminiChatProvider extends OpenAICompatProvider {
|
||||
chat: {
|
||||
completions: {
|
||||
create: (params, options) => {
|
||||
const { model } = params as { model: string };
|
||||
if (!GEMINI_NATIVE_MODELS.has(model)) {
|
||||
const { model, service_tier } = params as {
|
||||
model: string;
|
||||
service_tier?: string;
|
||||
};
|
||||
if (
|
||||
!GEMINI_NATIVE_MODELS.has(model) &&
|
||||
(service_tier === undefined || !native)
|
||||
) {
|
||||
return compat.chat.completions.create(
|
||||
params,
|
||||
options,
|
||||
@@ -103,7 +109,12 @@ export class GeminiChatProvider extends OpenAICompatProvider {
|
||||
},
|
||||
},
|
||||
};
|
||||
const models = native ? GEMINI_MODELS : COMPAT_ONLY_MODELS;
|
||||
const models: IChatModel[] = native
|
||||
? GEMINI_MODELS
|
||||
: COMPAT_ONLY_MODELS.map((model) => ({
|
||||
...model,
|
||||
serviceSpeeds: ['standard'],
|
||||
}));
|
||||
super(meteringService, {
|
||||
client,
|
||||
defaultModel: 'gemini-2.5-flash',
|
||||
@@ -183,7 +194,10 @@ export class GeminiChatProvider extends OpenAICompatProvider {
|
||||
}
|
||||
|
||||
protected override meteredUsage(source: UsageSource, model: IChatModel) {
|
||||
if (GEMINI_NATIVE_MODELS.has(model.id)) {
|
||||
if (
|
||||
GEMINI_NATIVE_MODELS.has(model.id) ||
|
||||
'promptTokenCount' in (source.usage ?? {})
|
||||
) {
|
||||
return nativeMeteredUsage(
|
||||
source.usage as unknown as
|
||||
GenerateContentResponseUsageMetadata | undefined,
|
||||
|
||||
@@ -80,6 +80,7 @@ interface ChatParams {
|
||||
temperature?: number;
|
||||
reasoning_effort?: 'minimal' | 'low' | 'medium' | 'high';
|
||||
stream?: boolean;
|
||||
service_tier?: string;
|
||||
}
|
||||
|
||||
/** Ids made up for calls Gemini returned without one; never sent back. */
|
||||
@@ -240,6 +241,12 @@ const toGenerateContentRequest = (
|
||||
? toolConfigOf(params.tool_choice)
|
||||
: undefined;
|
||||
const config: GenerateContentConfig = {
|
||||
...(params.service_tier !== undefined
|
||||
? {
|
||||
serviceTier:
|
||||
params.service_tier as GenerateContentConfig['serviceTier'],
|
||||
}
|
||||
: {}),
|
||||
...(signal ? { abortSignal: signal } : {}),
|
||||
...(systemInstruction ? { systemInstruction } : {}),
|
||||
...(params.max_completion_tokens !== undefined
|
||||
|
||||
@@ -142,6 +142,7 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: ARGON_LAUNCH_COSTS,
|
||||
pricingOverrides: listPricingFrom(
|
||||
'2026-12-01T00:00:00Z',
|
||||
@@ -174,6 +175,7 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
// Google serves this endpoint with gemini-3.6-flash and bills its rates.
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: FLASH_LAUNCH_COSTS,
|
||||
pricingOverrides: FLASH_LIST_PRICING,
|
||||
},
|
||||
@@ -201,6 +203,7 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30,
|
||||
@@ -228,6 +231,7 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: FLASH_LAUNCH_COSTS,
|
||||
pricingOverrides: FLASH_LIST_PRICING,
|
||||
},
|
||||
@@ -250,6 +254,7 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
// Google serves this endpoint with gemini-3.8-flash and bills its rates.
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: FLASH_LAUNCH_COSTS,
|
||||
pricingOverrides: FLASH_LIST_PRICING,
|
||||
},
|
||||
@@ -270,6 +275,7 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: FLASH_LAUNCH_COSTS,
|
||||
pricingOverrides: FLASH_LIST_PRICING,
|
||||
},
|
||||
@@ -291,6 +297,7 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 25,
|
||||
@@ -318,6 +325,7 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 50,
|
||||
@@ -344,6 +352,7 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30,
|
||||
@@ -374,6 +383,7 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 10,
|
||||
@@ -402,6 +412,7 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
@@ -448,6 +459,7 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
|
||||
@@ -266,6 +266,15 @@ export class InfronProvider extends OpenAICompatProvider {
|
||||
return coerced_models;
|
||||
}
|
||||
|
||||
async serviceSpeedModels(model: IChatModel) {
|
||||
const variant = (await this.models()).find(
|
||||
(entry) => entry.id === `${model.id}:priority`,
|
||||
);
|
||||
return variant
|
||||
? { fast: { ...variant, provider: model.provider } }
|
||||
: {};
|
||||
}
|
||||
|
||||
protected override vendorParams(
|
||||
params: Record<string, unknown>,
|
||||
_args: unknown,
|
||||
|
||||
@@ -43,6 +43,8 @@ export const MINIMAX_MODELS: MiniMaxChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.5 },
|
||||
serviceSpeedVariants: { fast: 'minimax-m3-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30, // $0.30 per 1M (input <= 512k tokens)
|
||||
@@ -70,6 +72,7 @@ export const MINIMAX_MODELS: MiniMaxChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'minimax-m2.7-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30, // $0.30 per 1M
|
||||
@@ -122,6 +125,7 @@ export const MINIMAX_MODELS: MiniMaxChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'minimax-m2.5-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30, // $0.30 per 1M
|
||||
@@ -174,6 +178,7 @@ export const MINIMAX_MODELS: MiniMaxChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'minimax-m2.1-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30, // $0.30 per 1M
|
||||
@@ -222,6 +227,7 @@ export const MINIMAX_MODELS: MiniMaxChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'minimax-m2-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30, // $0.30 per 1M
|
||||
|
||||
@@ -20,13 +20,12 @@
|
||||
/**
|
||||
* Offline unit tests for MistralAIProvider.
|
||||
*
|
||||
* Boots a real PuterServer (in-memory sqlite + dynamo + s3 + mock
|
||||
* redis) and constructs MistralAIProvider directly against the live
|
||||
* wired `MeteringService` so the recording side is exercised end-to-
|
||||
* end. The Mistral SDK is mocked at the module boundary (the real
|
||||
* network egress point) so the provider never reaches the network.
|
||||
* The companion integration test (MistralAiProvider.integration.test.ts)
|
||||
* exercises the real Mistral endpoint.
|
||||
* Boots a real PuterServer (in-memory sqlite + dynamo + s3 + mock redis) and
|
||||
* constructs MistralAIProvider directly against the live wired
|
||||
* `MeteringService` so the recording side is exercised end-to- end. The Mistral
|
||||
* SDK is mocked at the module boundary (the real network egress point) so the
|
||||
* provider never reaches the network. The companion integration test
|
||||
* (MistralAiProvider.integration.test.ts) exercises the real Mistral endpoint.
|
||||
*/
|
||||
|
||||
import { Context } from '../../../../core/context.js';
|
||||
@@ -147,7 +146,9 @@ describe('MistralAIProvider construction', () => {
|
||||
it('constructs the Mistral SDK with the configured API key', () => {
|
||||
makeProvider();
|
||||
expect(mistralCtor).toHaveBeenCalledTimes(1);
|
||||
expect(mistralCtor).toHaveBeenCalledWith({ apiKey: 'test-key' });
|
||||
expect(mistralCtor).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ apiKey: 'test-key' }),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -606,7 +607,11 @@ describe('MistralAIProvider.complete non-stream output', () => {
|
||||
if (stream) {
|
||||
streamMock.mockResolvedValueOnce(
|
||||
asAsyncIterable([
|
||||
{ data: { choices: [{ delta: { content: 'hello' } }] } },
|
||||
{
|
||||
data: {
|
||||
choices: [{ delta: { content: 'hello' } }],
|
||||
},
|
||||
},
|
||||
{ data: { choices: [], usage } },
|
||||
]),
|
||||
);
|
||||
@@ -643,7 +648,11 @@ describe('MistralAIProvider.complete non-stream output', () => {
|
||||
expect(result).toMatchObject({ message: { content: 'hello' } });
|
||||
}
|
||||
expect(recordSpy).toHaveBeenCalledWith(
|
||||
{ prompt_tokens: 5, cached_tokens: 1008, completion_tokens: 30 },
|
||||
{
|
||||
prompt_tokens: 5,
|
||||
cached_tokens: 1008,
|
||||
completion_tokens: 30,
|
||||
},
|
||||
SYSTEM_ACTOR,
|
||||
`mistral:${model}`,
|
||||
{
|
||||
@@ -688,8 +697,7 @@ describe('MistralAIProvider.complete non-stream output', () => {
|
||||
});
|
||||
|
||||
expect(recordSpy).toHaveBeenCalledTimes(1);
|
||||
const [usage, actor, prefix, overrides] =
|
||||
recordSpy.mock.calls[0]!;
|
||||
const [usage, actor, prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(usage).toEqual({
|
||||
prompt_tokens: 100,
|
||||
completion_tokens: 50,
|
||||
@@ -718,15 +726,11 @@ describe('MistralAIProvider.complete non-stream output', () => {
|
||||
content: [
|
||||
{
|
||||
type: 'thinking',
|
||||
thinking: [
|
||||
{ type: 'text', text: 'step one.' },
|
||||
],
|
||||
thinking: [{ type: 'text', text: 'step one.' }],
|
||||
},
|
||||
{
|
||||
type: 'thinking',
|
||||
thinking: [
|
||||
{ type: 'text', text: 'step two.' },
|
||||
],
|
||||
thinking: [{ type: 'text', text: 'step two.' }],
|
||||
},
|
||||
{ type: 'text', text: 'the answer' },
|
||||
],
|
||||
@@ -916,8 +920,7 @@ describe('MistralAIProvider.complete streaming', () => {
|
||||
|
||||
// mistral-small-2603: prompt=15, completion=60.
|
||||
expect(recordSpy).toHaveBeenCalledTimes(1);
|
||||
const [, , prefix, overrides] =
|
||||
recordSpy.mock.calls[0]!;
|
||||
const [, , prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(prefix).toBe('mistral:mistral-small-2603');
|
||||
expect(overrides).toMatchObject({
|
||||
prompt_tokens: 4 * 15,
|
||||
@@ -1115,7 +1118,7 @@ describe('MistralAIProvider image_url coercion', () => {
|
||||
usage: { promptTokens: 1, completionTokens: 1 },
|
||||
};
|
||||
|
||||
it('rewrites canonical image_url parts to the SDK\'s camelCase imageUrl', async () => {
|
||||
it("rewrites canonical image_url parts to the SDK's camelCase imageUrl", async () => {
|
||||
const { provider } = makeProvider();
|
||||
completeMock.mockResolvedValueOnce(baseCompletion);
|
||||
|
||||
@@ -1175,7 +1178,8 @@ describe('MistralAIProvider image_url coercion', () => {
|
||||
},
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url: 'https://example.com/already-flat.png',
|
||||
image_url:
|
||||
'https://example.com/already-flat.png',
|
||||
},
|
||||
],
|
||||
},
|
||||
@@ -1251,7 +1255,9 @@ describe('MistralAIProvider image_url coercion', () => {
|
||||
{ type: 'text', text: 'what is in this image?' },
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url: { url: 'https://example.com/photo.jpg' },
|
||||
image_url: {
|
||||
url: 'https://example.com/photo.jpg',
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
@@ -1260,7 +1266,11 @@ describe('MistralAIProvider image_url coercion', () => {
|
||||
);
|
||||
|
||||
const [args] = completeMock.mock.calls[0]!;
|
||||
const parts = args.messages[0].content as { type: string; text?: string; imageUrl?: unknown }[];
|
||||
const parts = args.messages[0].content as {
|
||||
type: string;
|
||||
text?: string;
|
||||
imageUrl?: unknown;
|
||||
}[];
|
||||
const textPart = parts.find((p) => p.type === 'text');
|
||||
const imgPart = parts.find((p) => p.type === 'image_url');
|
||||
expect(textPart?.text).toBe('what is in this image?');
|
||||
@@ -1300,3 +1310,41 @@ describe('MistralAIProvider.checkModeration', () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Mistral service speed wire mapping', () => {
|
||||
it('adds the tier after SDK serialization and removes its internal header', async () => {
|
||||
makeProvider();
|
||||
const options = mistralCtor.mock.calls[0][0] as {
|
||||
httpClient: import('@mistralai/mistralai/lib/http.js').HTTPClient;
|
||||
};
|
||||
let sent: Request | undefined;
|
||||
vi.stubGlobal('fetch', async (input: Request) => {
|
||||
sent = input;
|
||||
return new Response('{}', {
|
||||
headers: { 'content-type': 'application/json' },
|
||||
});
|
||||
});
|
||||
try {
|
||||
await options.httpClient.request(
|
||||
new Request('https://api.mistral.ai/v1/chat/completions', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
'x-puter-service-tier': 'auto',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: 'mistral-medium-latest',
|
||||
messages: [],
|
||||
}),
|
||||
}),
|
||||
);
|
||||
expect(sent?.headers.has('x-puter-service-tier')).toBe(false);
|
||||
expect(await sent?.json()).toMatchObject({
|
||||
model: 'mistral-medium-latest',
|
||||
service_tier: 'auto',
|
||||
});
|
||||
} finally {
|
||||
vi.unstubAllGlobals();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -17,6 +17,8 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { HTTPClient } from '@mistralai/mistralai/lib/http.js';
|
||||
import { servedServiceSpeed } from '../../utils/serviceSpeed.js';
|
||||
import { Mistral } from '@mistralai/mistralai';
|
||||
import { ChatCompletionResponse } from '@mistralai/mistralai/models/components/chatcompletionresponse.js';
|
||||
import { MistralError } from '@mistralai/mistralai/models/errors/mistralerror.js';
|
||||
@@ -26,6 +28,7 @@ import { Context } from '../../../../core/context.js';
|
||||
import { HttpError } from '../../../../core/http/HttpError.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
import type {
|
||||
IChatModel,
|
||||
IChatCompleteResult,
|
||||
IChatProvider,
|
||||
ICompleteArguments,
|
||||
@@ -106,6 +109,7 @@ const coerceMistralUsage = (usage: UsageInfo) => {
|
||||
prompt_tokens: (usage.promptTokens ?? 0) - cachedTokens,
|
||||
completion_tokens: usage.completionTokens ?? 0,
|
||||
prompt_tokens_details: { cached_tokens: cachedTokens },
|
||||
service_tier: usage.additionalProperties?.service_tier,
|
||||
};
|
||||
};
|
||||
|
||||
@@ -115,7 +119,24 @@ export class MistralAIProvider implements IChatProvider {
|
||||
#meteringService: MeteringService;
|
||||
|
||||
constructor(config: { apiKey: string }, meteringService: MeteringService) {
|
||||
const httpClient = new HTTPClient();
|
||||
// The SDK strips service_tier during body serialization.
|
||||
httpClient.addHook('beforeRequest', async (request) => {
|
||||
const tier = request.headers.get('x-puter-service-tier');
|
||||
if (!tier) return;
|
||||
const headers = new Headers(request.headers);
|
||||
headers.delete('x-puter-service-tier');
|
||||
const body = await request.json();
|
||||
return new Request(request, {
|
||||
headers,
|
||||
body: JSON.stringify({
|
||||
...body,
|
||||
service_tier: tier === 'auto' ? 'auto' : 'standard_only',
|
||||
}),
|
||||
});
|
||||
});
|
||||
this.#client = new Mistral({
|
||||
httpClient,
|
||||
apiKey: config.apiKey,
|
||||
});
|
||||
this.#meteringService = meteringService;
|
||||
@@ -181,17 +202,22 @@ export class MistralAIProvider implements IChatProvider {
|
||||
return `mistral:${modelId}`;
|
||||
}
|
||||
|
||||
async complete({
|
||||
messages,
|
||||
stream,
|
||||
model,
|
||||
tools,
|
||||
max_tokens,
|
||||
temperature,
|
||||
normalize,
|
||||
response,
|
||||
custom,
|
||||
}: ICompleteArguments): Promise<IChatCompleteResult> {
|
||||
async complete(
|
||||
{
|
||||
messages,
|
||||
stream,
|
||||
model,
|
||||
tools,
|
||||
max_tokens,
|
||||
temperature,
|
||||
normalize,
|
||||
response,
|
||||
custom,
|
||||
service_tier,
|
||||
serviceSpeed: requestedServiceSpeed,
|
||||
}: ICompleteArguments,
|
||||
resolved?: IChatModel,
|
||||
): Promise<IChatCompleteResult> {
|
||||
// Mistral's reasoning prompt mode: with `prompt_mode: 'reasoning'`,
|
||||
// reasoning-capable models return their thinking as structured ThinkChunk
|
||||
// content (which the splitter below separates into `reasoning`)
|
||||
@@ -217,10 +243,13 @@ export class MistralAIProvider implements IChatProvider {
|
||||
}
|
||||
|
||||
const selectedModel =
|
||||
(await this.models()).find((m) =>
|
||||
resolved ??
|
||||
((await this.models()).find((m) =>
|
||||
[m.id, ...(m.aliases || [])].includes(model),
|
||||
) ||
|
||||
(await this.models()).find((m) => m.id === this.getDefaultModel())!;
|
||||
(await this.models()).find(
|
||||
(m) => m.id === this.getDefaultModel(),
|
||||
)!);
|
||||
const actor = Context.get('actor');
|
||||
let completion: Awaited<
|
||||
ReturnType<Mistral['chat']['complete' | 'stream']>
|
||||
@@ -242,6 +271,9 @@ export class MistralAIProvider implements IChatProvider {
|
||||
// The SDK retries nothing by default and has no timeout once a
|
||||
// signal is passed; a stream is bounded by its caller instead.
|
||||
{
|
||||
...(service_tier !== undefined
|
||||
? { headers: { 'x-puter-service-tier': service_tier } }
|
||||
: {}),
|
||||
signal: stream
|
||||
? Context.get('upstreamSignal')
|
||||
: withSdkTimeout(Context.get('upstreamSignal')),
|
||||
@@ -398,7 +430,8 @@ export class MistralAIProvider implements IChatProvider {
|
||||
},
|
||||
completion: completion as ChatCompletionResponse,
|
||||
stream,
|
||||
usage_calculator: ({ usage, setUsageCosts }) => {
|
||||
usage_calculator: ({ usage, setUsageCosts, setUsageDetails }) => {
|
||||
const serviceSpeed = servedServiceSpeed({ usage });
|
||||
// `coerceMistralUsage` has already taken cached reads out of
|
||||
// `prompt_tokens`.
|
||||
const metered = meterChatUsage(
|
||||
@@ -407,8 +440,16 @@ export class MistralAIProvider implements IChatProvider {
|
||||
this.meteringModelKey(selectedModel.id),
|
||||
selectedModel,
|
||||
OpenAIUtil.extractMeteredUsage(usage),
|
||||
{ serviceSpeed },
|
||||
);
|
||||
setUsageCosts(metered.costs);
|
||||
if (requestedServiceSpeed !== undefined)
|
||||
setUsageDetails({
|
||||
...OpenAIUtil.usageDetailsFromTrackedUsage(
|
||||
metered.usage,
|
||||
)!,
|
||||
serviceSpeed,
|
||||
});
|
||||
return metered.usage;
|
||||
},
|
||||
});
|
||||
|
||||
@@ -42,6 +42,7 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 136,
|
||||
@@ -75,6 +76,7 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 150,
|
||||
@@ -105,6 +107,7 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 50,
|
||||
@@ -135,6 +138,7 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 15,
|
||||
@@ -159,6 +163,7 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 140,
|
||||
@@ -188,6 +193,7 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30,
|
||||
@@ -213,6 +219,7 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 20,
|
||||
@@ -238,6 +245,7 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 15,
|
||||
@@ -262,6 +270,7 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 10,
|
||||
|
||||
@@ -30,6 +30,7 @@ export const MOONSHOT_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'kimi-k3.1-pro-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200, // $2.00 per 1M
|
||||
@@ -77,6 +78,7 @@ export const MOONSHOT_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'kimi-k3-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 300, // $3.00 per 1M
|
||||
@@ -104,6 +106,7 @@ export const MOONSHOT_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'kimi-k2.6-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 95, // $0.95 per 1M
|
||||
@@ -126,6 +129,7 @@ export const MOONSHOT_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'kimi-k2.7-code-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 95,
|
||||
|
||||
@@ -20,6 +20,7 @@
|
||||
import { HttpError } from '@heyputer/backend/src/core/http/HttpError.js';
|
||||
import { OpenAI } from 'openai';
|
||||
import { ChatCompletionCreateParams } from 'openai/resources/index.js';
|
||||
import { servedServiceSpeed } from '../../utils/serviceSpeed.js';
|
||||
import { Context } from '../../../../core/context.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
import type {
|
||||
@@ -117,6 +118,7 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
|
||||
async complete(
|
||||
params: ICompleteArguments,
|
||||
resolved?: IChatModel,
|
||||
): ReturnType<IChatProvider['complete']> {
|
||||
const {
|
||||
max_tokens,
|
||||
@@ -134,6 +136,11 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
} = params;
|
||||
let { messages, model } = params;
|
||||
const cacheOptions = promptCacheOptionsToWire(prompt_cache_options);
|
||||
if (params.service_tier === 'ultrafast' && this.#responsesProvider) {
|
||||
return resolved
|
||||
? this.#responsesProvider.complete(params, resolved)
|
||||
: this.#responsesProvider.complete(params);
|
||||
}
|
||||
if (
|
||||
tools?.filter((e) => isWebSearchTool(e as Record<string, unknown>))
|
||||
.length
|
||||
@@ -147,7 +154,9 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
return await this.#responsesProvider.complete(params);
|
||||
return await (resolved
|
||||
? this.#responsesProvider.complete(params, resolved)
|
||||
: this.#responsesProvider.complete(params));
|
||||
}
|
||||
// Inline compaction is a Responses-API feature; chat.completions can't
|
||||
// express `context_management` or a `compaction` content block.
|
||||
@@ -161,7 +170,9 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
return await this.#responsesProvider.complete(params);
|
||||
return await (resolved
|
||||
? this.#responsesProvider.complete(params, resolved)
|
||||
: this.#responsesProvider.complete(params));
|
||||
}
|
||||
// Only the Responses API writes 12h cache entries or takes a lifetime
|
||||
// on a `prompt_cache_breakpoint`; Chat Completions rejects both.
|
||||
@@ -176,7 +187,9 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
return await this.#responsesProvider.complete(params);
|
||||
return await (resolved
|
||||
? this.#responsesProvider.complete(params, resolved)
|
||||
: this.#responsesProvider.complete(params));
|
||||
}
|
||||
// Validate messages
|
||||
if (!Array.isArray(messages)) {
|
||||
@@ -189,9 +202,11 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
model = model ?? this.#defaultModel;
|
||||
|
||||
const modelUsed =
|
||||
this.models().find((m) =>
|
||||
resolved ??
|
||||
(this.models().find((m) =>
|
||||
[m.id, ...(m.aliases || [])].includes(model),
|
||||
) || this.models().find((m) => m.id === this.getDefaultModel())!;
|
||||
) ||
|
||||
this.models().find((m) => m.id === this.getDefaultModel())!);
|
||||
|
||||
// messages.unshift({
|
||||
// role: 'system',
|
||||
@@ -248,6 +263,9 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
: {}),
|
||||
messages: messages,
|
||||
model: modelUsed.id,
|
||||
...(params.service_tier !== undefined
|
||||
? { service_tier: params.service_tier }
|
||||
: {}),
|
||||
...(mappedTools?.length ? { tools: mappedTools } : {}),
|
||||
...(max_tokens !== undefined
|
||||
? { max_completion_tokens: max_tokens }
|
||||
@@ -284,7 +302,13 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
);
|
||||
|
||||
return OpenAiUtil.handle_completion_output({
|
||||
usage_calculator: ({ usage, setUsageCosts, setUsageDetails }) => {
|
||||
usage_calculator: ({
|
||||
usage,
|
||||
service_tier,
|
||||
setUsageCosts,
|
||||
setUsageDetails,
|
||||
}) => {
|
||||
const serviceSpeed = servedServiceSpeed({ service_tier });
|
||||
// GPT-5.6 and later report cache writes inside
|
||||
// `prompt_tokens`, like cached reads; Chat Completions only
|
||||
// writes the default 30m entries, billed at 1.25x input.
|
||||
@@ -296,13 +320,17 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
this.meteringModelKey(modelUsed.id),
|
||||
modelUsed,
|
||||
trackedUsage,
|
||||
{ serviceSpeed },
|
||||
);
|
||||
setUsageCosts(metered.costs);
|
||||
setUsageDetails(
|
||||
OpenAiUtil.usageDetailsFromTrackedUsage(metered.usage, {
|
||||
setUsageDetails({
|
||||
...OpenAiUtil.usageDetailsFromTrackedUsage(metered.usage, {
|
||||
writeTtl: '30m',
|
||||
})!,
|
||||
);
|
||||
...(params.serviceSpeed !== undefined
|
||||
? { serviceSpeed }
|
||||
: {}),
|
||||
});
|
||||
return metered.usage;
|
||||
},
|
||||
stream,
|
||||
|
||||
@@ -19,6 +19,7 @@
|
||||
|
||||
import { OpenAI } from 'openai';
|
||||
import { ResponseCreateParams } from 'openai/resources/responses/responses.mjs';
|
||||
import { servedServiceSpeed } from '../../utils/serviceSpeed.js';
|
||||
import { Context } from '../../../../core/context.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
import type {
|
||||
@@ -134,38 +135,42 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
|
||||
return rate ? { extraCost: rate } : {};
|
||||
}
|
||||
|
||||
async complete({
|
||||
messages,
|
||||
model,
|
||||
max_tokens,
|
||||
moderation,
|
||||
tools,
|
||||
tool_choice,
|
||||
parallel_tool_calls,
|
||||
include,
|
||||
conversation,
|
||||
compaction,
|
||||
context_management,
|
||||
previous_response_id,
|
||||
instructions,
|
||||
metadata,
|
||||
prompt,
|
||||
prompt_cache_key,
|
||||
prompt_cache_retention,
|
||||
prompt_cache_options,
|
||||
store,
|
||||
top_p,
|
||||
truncation,
|
||||
background,
|
||||
service_tier,
|
||||
verbosity,
|
||||
stream,
|
||||
reasoning,
|
||||
reasoning_effort,
|
||||
temperature,
|
||||
text,
|
||||
outputFormat,
|
||||
}: ICompleteArguments): ReturnType<IChatProvider['complete']> {
|
||||
async complete(
|
||||
{
|
||||
messages,
|
||||
model,
|
||||
max_tokens,
|
||||
moderation,
|
||||
tools,
|
||||
tool_choice,
|
||||
parallel_tool_calls,
|
||||
include,
|
||||
conversation,
|
||||
compaction,
|
||||
context_management,
|
||||
previous_response_id,
|
||||
instructions,
|
||||
metadata,
|
||||
prompt,
|
||||
prompt_cache_key,
|
||||
prompt_cache_retention,
|
||||
prompt_cache_options,
|
||||
store,
|
||||
top_p,
|
||||
truncation,
|
||||
background,
|
||||
service_tier,
|
||||
serviceSpeed: requestedServiceSpeed,
|
||||
verbosity,
|
||||
stream,
|
||||
reasoning,
|
||||
reasoning_effort,
|
||||
temperature,
|
||||
text,
|
||||
outputFormat,
|
||||
}: ICompleteArguments,
|
||||
resolved?: IChatModel,
|
||||
): ReturnType<IChatProvider['complete']> {
|
||||
// Validate messages
|
||||
if (!Array.isArray(messages)) {
|
||||
throw new HttpError(400, '`messages` must be an array', {
|
||||
@@ -183,12 +188,13 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
|
||||
model = model ?? this.#defaultModel;
|
||||
|
||||
const modelUsed =
|
||||
this.models({ no_restrictions: true }).find((m) =>
|
||||
resolved ??
|
||||
(this.models({ no_restrictions: true }).find((m) =>
|
||||
[m.id, ...(m.aliases || [])].includes(model),
|
||||
) ||
|
||||
this.models({ no_restrictions: true }).find(
|
||||
(m) => m.id === this.getDefaultModel(),
|
||||
)!;
|
||||
this.models({ no_restrictions: true }).find(
|
||||
(m) => m.id === this.getDefaultModel(),
|
||||
)!);
|
||||
|
||||
// messages.unshift({
|
||||
// role: 'system',
|
||||
@@ -322,6 +328,7 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
|
||||
return OpenAiUtil.handle_completion_output_responses_api({
|
||||
usage_calculator: ({
|
||||
usage,
|
||||
service_tier,
|
||||
webSearchCalls,
|
||||
setUsageCosts,
|
||||
setUsageDetails,
|
||||
@@ -347,23 +354,31 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
|
||||
// model is a reasoning model, not a static per-model
|
||||
// cost-table entry — the model's rates would otherwise
|
||||
// price it (wrongly) at the input/output token rate.
|
||||
webSearchCalls
|
||||
? {
|
||||
costOverrides: {
|
||||
web_search_calls:
|
||||
webSearchCalls *
|
||||
(webSearchCallRate(tools, modelUsed.id) ??
|
||||
0),
|
||||
},
|
||||
}
|
||||
: {},
|
||||
{
|
||||
serviceSpeed: servedServiceSpeed({ service_tier }),
|
||||
...(webSearchCalls
|
||||
? {
|
||||
costOverrides: {
|
||||
web_search_calls:
|
||||
webSearchCalls *
|
||||
(webSearchCallRate(
|
||||
tools,
|
||||
modelUsed.id,
|
||||
) ?? 0),
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
},
|
||||
);
|
||||
setUsageCosts?.(metered.costs);
|
||||
setUsageDetails?.(
|
||||
OpenAiUtil.usageDetailsFromTrackedUsage(metered.usage, {
|
||||
setUsageDetails?.({
|
||||
...(requestedServiceSpeed !== undefined
|
||||
? { serviceSpeed: servedServiceSpeed({ service_tier }) }
|
||||
: {}),
|
||||
...OpenAiUtil.usageDetailsFromTrackedUsage(metered.usage, {
|
||||
writeTtl: '30m',
|
||||
})!,
|
||||
);
|
||||
});
|
||||
return metered.usage;
|
||||
},
|
||||
stream,
|
||||
|
||||
@@ -46,6 +46,7 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 100,
|
||||
@@ -72,6 +73,7 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2, ultrafast: 6 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -98,6 +100,7 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -124,6 +127,7 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 10,
|
||||
@@ -150,6 +154,7 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2, ultrafast: 6 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 1000,
|
||||
@@ -176,6 +181,7 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
// OpenAI's promotional pricing, guaranteed only through 2026-11-21.
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 400,
|
||||
@@ -201,6 +207,7 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -226,6 +233,7 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 20,
|
||||
@@ -251,6 +259,13 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 1250,
|
||||
cached_tokens: 125,
|
||||
completion_tokens: 7500,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 500,
|
||||
@@ -295,6 +310,13 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 500,
|
||||
cached_tokens: 50,
|
||||
completion_tokens: 3000,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
@@ -338,6 +360,13 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 150,
|
||||
cached_tokens: 15,
|
||||
completion_tokens: 900,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 75,
|
||||
@@ -425,6 +454,13 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 350,
|
||||
cached_tokens: 35,
|
||||
completion_tokens: 2800,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 175,
|
||||
@@ -446,6 +482,13 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 250,
|
||||
cached_tokens: 25,
|
||||
completion_tokens: 2000,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
@@ -467,6 +510,13 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 425,
|
||||
cached_tokens: 212.5,
|
||||
completion_tokens: 1700,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
@@ -490,6 +540,13 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 25,
|
||||
cached_tokens: 12.5,
|
||||
completion_tokens: 100,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 15,
|
||||
@@ -509,6 +566,13 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 350,
|
||||
cached_tokens: 87.5,
|
||||
completion_tokens: 1400,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -530,6 +594,13 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 70,
|
||||
cached_tokens: 17.5,
|
||||
completion_tokens: 280,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -571,6 +642,13 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 425,
|
||||
cached_tokens: 212.5,
|
||||
completion_tokens: 1700,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
|
||||
@@ -20,15 +20,14 @@
|
||||
/**
|
||||
* Offline unit tests for OpenRouterProvider.
|
||||
*
|
||||
* Boots a real PuterServer (in-memory sqlite + dynamo + s3 + mock
|
||||
* redis) and constructs OpenRouterProvider directly against the live
|
||||
* wired `MeteringService` so the recording side is exercised end-to-
|
||||
* end. OpenRouter is OpenAI-compatible, so the OpenAI SDK is mocked
|
||||
* at the module boundary; the model catalog is fetched via `axios`
|
||||
* which is mocked at its module boundary too. Both are the real
|
||||
* network egress points.
|
||||
* The companion integration test (OpenRouterProvider.integration.test.ts)
|
||||
* exercises the real OpenRouter endpoint.
|
||||
* Boots a real PuterServer (in-memory sqlite + dynamo + s3 + mock redis) and
|
||||
* constructs OpenRouterProvider directly against the live wired
|
||||
* `MeteringService` so the recording side is exercised end-to- end. OpenRouter
|
||||
* is OpenAI-compatible, so the OpenAI SDK is mocked at the module boundary; the
|
||||
* model catalog is fetched via `axios` which is mocked at its module boundary
|
||||
* too. Both are the real network egress points. The companion integration test
|
||||
* (OpenRouterProvider.integration.test.ts) exercises the real OpenRouter
|
||||
* endpoint.
|
||||
*/
|
||||
|
||||
import { Writable } from 'node:stream';
|
||||
@@ -300,16 +299,20 @@ describe('OpenRouterProvider model catalog', () => {
|
||||
it('coerces string prices and normalizes conditional tiers', async () => {
|
||||
const { provider } = makeProvider();
|
||||
const models = await provider.models();
|
||||
const sol = models.find((m) => m.id === 'openrouter:openai/gpt-6.1-sol')!;
|
||||
const sol = models.find(
|
||||
(m) => m.id === 'openrouter:openai/gpt-6.1-sol',
|
||||
)!;
|
||||
expect(sol.costs).toEqual({
|
||||
tokens: 1_000_000,
|
||||
prompt: 200,
|
||||
completion: 1000,
|
||||
});
|
||||
expect(sol.pricingOverrides).toEqual([{
|
||||
minPromptTokens: 272000,
|
||||
costs: { prompt: 400, completion: 1500 },
|
||||
}]);
|
||||
expect(sol.pricingOverrides).toEqual([
|
||||
{
|
||||
minPromptTokens: 272000,
|
||||
costs: { prompt: 400, completion: 1500 },
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('caches the coerced model list after the first axios round-trip', async () => {
|
||||
@@ -368,9 +371,12 @@ describe('OpenRouterProvider model catalog', () => {
|
||||
|
||||
it('does not reinterpret catalog-added timestamps as release dates', async () => {
|
||||
const { provider } = makeProvider();
|
||||
expect((await provider.models()).every(model => model.release_date === undefined)).toBe(true);
|
||||
expect(
|
||||
(await provider.models()).every(
|
||||
(model) => model.release_date === undefined,
|
||||
),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
});
|
||||
|
||||
// ── Disallowed model gate ───────────────────────────────────────────
|
||||
@@ -868,11 +874,14 @@ describe('OpenRouterProvider billed cost through the driver', () => {
|
||||
event.factor = factor;
|
||||
};
|
||||
server.clients.event.on('ai.cost.factor.*', listener as never);
|
||||
return () => server.clients.event.off('ai.cost.factor.*', listener as never);
|
||||
return () =>
|
||||
server.clients.event.off('ai.cost.factor.*', listener as never);
|
||||
};
|
||||
|
||||
/** µ¢ the ledger was handed for every usage recorded so far. */
|
||||
const ledgerTotal = (spy: MockInstance<MeteringService['batchIncrementUsages']>) =>
|
||||
/** Μ¢ the ledger was handed for every usage recorded so far. */
|
||||
const ledgerTotal = (
|
||||
spy: MockInstance<MeteringService['batchIncrementUsages']>,
|
||||
) =>
|
||||
spy.mock.calls
|
||||
.flatMap(([, usages]) => usages)
|
||||
.reduce((total, usage) => total + (usage.costOverride ?? 0), 0);
|
||||
@@ -928,7 +937,10 @@ describe('OpenRouterProvider billed cost through the driver', () => {
|
||||
it('reports what a billed completion was charged, after the cost factor', async () => {
|
||||
const driver = await makeDriver();
|
||||
const stopFactor = withCostFactor(1.6);
|
||||
const recorded = vi.spyOn(server.services.metering, 'batchIncrementUsages');
|
||||
const recorded = vi.spyOn(
|
||||
server.services.metering,
|
||||
'batchIncrementUsages',
|
||||
);
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
@@ -1023,7 +1035,10 @@ describe('OpenRouterProvider billed cost through the driver', () => {
|
||||
it('ends a stream with what it was charged at the catalog rates, after the cost factor', async () => {
|
||||
const driver = await makeDriver();
|
||||
const stopFactor = withCostFactor(1.6);
|
||||
const recorded = vi.spyOn(server.services.metering, 'batchIncrementUsages');
|
||||
const recorded = vi.spyOn(
|
||||
server.services.metering,
|
||||
'batchIncrementUsages',
|
||||
);
|
||||
createMock.mockReturnValueOnce(
|
||||
asAsyncIterable([
|
||||
{ choices: [{ delta: { content: 'hello' } }] },
|
||||
@@ -1126,31 +1141,180 @@ describe('OpenRouterProvider.checkModeration', () => {
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
describe('OpenRouter catalog refresh validation', () => {
|
||||
it('keeps missing model fields and prices unknown and blocks unpriced inference', async () => {
|
||||
axiosRequestMock.mockResolvedValue({ data: { data: [{ id: 'fixture/model', name: 'Fixture' }] } });
|
||||
axiosRequestMock.mockResolvedValue({
|
||||
data: { data: [{ id: 'fixture/model', name: 'Fixture' }] },
|
||||
});
|
||||
const { provider } = makeProvider();
|
||||
const [model] = await provider.models();
|
||||
expect(model.costs).toEqual({ tokens: 1_000_000 });
|
||||
expect(model.context).toBeUndefined();
|
||||
expect(model.max_tokens).toBe(0);
|
||||
await expect(withTestActor(() => provider.complete({ model: 'openrouter:fixture/model', messages: [] })))
|
||||
.rejects.toMatchObject({ legacyCode: 'model_pricing_unavailable' });
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'openrouter:fixture/model',
|
||||
messages: [],
|
||||
}),
|
||||
),
|
||||
).rejects.toMatchObject({ legacyCode: 'model_pricing_unavailable' });
|
||||
expect(createMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each(['malformed', 'failed', 'ambiguous conditional price'])('retains last-good data after a %s refresh', async failure => {
|
||||
vi.useFakeTimers({ toFake: ['Date'] });
|
||||
vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
it.each(['malformed', 'failed', 'ambiguous conditional price'])(
|
||||
'retains last-good data after a %s refresh',
|
||||
async (failure) => {
|
||||
vi.useFakeTimers({ toFake: ['Date'] });
|
||||
vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
const { provider } = makeProvider();
|
||||
const good = await provider.models();
|
||||
vi.advanceTimersByTime(15 * 60 * 1000 + 1);
|
||||
if (failure === 'failed')
|
||||
axiosRequestMock.mockRejectedValueOnce(
|
||||
new Error('unavailable'),
|
||||
);
|
||||
else if (failure === 'malformed')
|
||||
axiosRequestMock.mockResolvedValueOnce({ data: {} });
|
||||
else
|
||||
axiosRequestMock.mockResolvedValueOnce({
|
||||
data: {
|
||||
data: [
|
||||
{
|
||||
...SAMPLE_API_MODELS[0],
|
||||
pricing: {
|
||||
prompt: '0.000002',
|
||||
overrides: [
|
||||
{ min_prompt_tokens: 10, prompt: '' },
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
expect(await provider.models()).toBe(good);
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe('OpenRouter service speed offerings', () => {
|
||||
it('loads and caches tier prices while reserving for endpoint and context differences', async () => {
|
||||
const { provider } = makeProvider();
|
||||
const good = await provider.models();
|
||||
vi.advanceTimersByTime(15 * 60 * 1000 + 1);
|
||||
if (failure === 'failed') axiosRequestMock.mockRejectedValueOnce(new Error('unavailable'));
|
||||
else if (failure === 'malformed') axiosRequestMock.mockResolvedValueOnce({ data: {} });
|
||||
else axiosRequestMock.mockResolvedValueOnce({ data: { data: [{ ...SAMPLE_API_MODELS[0], pricing: {
|
||||
prompt: '0.000002', overrides: [{ min_prompt_tokens: 10, prompt: '' }],
|
||||
} }] } });
|
||||
expect(await provider.models()).toBe(good);
|
||||
const base = {
|
||||
...(await provider.models())[0],
|
||||
provider: 'openrouter',
|
||||
};
|
||||
axiosRequestMock.mockClear();
|
||||
axiosRequestMock.mockResolvedValueOnce({
|
||||
data: {
|
||||
data: {
|
||||
endpoints: [
|
||||
{
|
||||
service_tier: 'priority',
|
||||
pricing: {
|
||||
prompt: '0.000004',
|
||||
completion: '0.00002',
|
||||
input_cache_read: '0.0000002',
|
||||
},
|
||||
},
|
||||
{
|
||||
service_tier: 'priority',
|
||||
pricing: {
|
||||
prompt: '0.000005',
|
||||
completion: '0.00001',
|
||||
},
|
||||
},
|
||||
{
|
||||
service_tier: 'ultrafast',
|
||||
pricing: {
|
||||
prompt: '0.000012',
|
||||
completion: '0.00006',
|
||||
overrides: [
|
||||
{
|
||||
min_prompt_tokens: 272000,
|
||||
prompt: '0.000024',
|
||||
completion: '0.00009',
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
{
|
||||
service_tier: 'priority',
|
||||
pricing: { prompt: 'invalid' },
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
});
|
||||
const offers = await provider.serviceSpeedModels(base);
|
||||
expect(offers.fast?.costs).toMatchObject({
|
||||
prompt: 500,
|
||||
completion: 2000,
|
||||
input_cache_read: 20,
|
||||
});
|
||||
expect(offers.ultrafast?.costs).toMatchObject({
|
||||
prompt: 2400,
|
||||
completion: 9000,
|
||||
});
|
||||
expect(await provider.serviceSpeedModels(base)).toBe(offers);
|
||||
expect(axiosRequestMock).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('uses standard when the tier catalog cannot be fetched', async () => {
|
||||
const { provider } = makeProvider();
|
||||
const base = {
|
||||
...(await provider.models())[0],
|
||||
provider: 'openrouter',
|
||||
};
|
||||
axiosRequestMock.mockRejectedValueOnce(new Error('Unavailable'));
|
||||
expect(await provider.serviceSpeedModels(base)).toEqual({});
|
||||
});
|
||||
});
|
||||
|
||||
describe('OpenRouter speed billing', () => {
|
||||
it('requires an authoritative bill instead of charging the reservation ceiling', async () => {
|
||||
const { selectServiceSpeed } =
|
||||
await import('../../utils/serviceSpeed.js');
|
||||
const { provider } = makeProvider();
|
||||
const base = {
|
||||
...(await provider.models())[0],
|
||||
provider: 'openrouter',
|
||||
};
|
||||
const selected = selectServiceSpeed(
|
||||
base,
|
||||
{
|
||||
model: base.id,
|
||||
messages: [{ role: 'user', content: 'Hello' }],
|
||||
serviceSpeed: 'fast',
|
||||
},
|
||||
() => undefined,
|
||||
{
|
||||
fast: {
|
||||
...base,
|
||||
costs: { ...base.costs, prompt: 500, completion: 2000 },
|
||||
},
|
||||
},
|
||||
);
|
||||
const completion = {
|
||||
choices: [
|
||||
{
|
||||
index: 0,
|
||||
message: { role: 'assistant', content: 'OK' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: { prompt_tokens: 10, completion_tokens: 1 },
|
||||
};
|
||||
createMock.mockResolvedValueOnce({
|
||||
...completion,
|
||||
service_tier: 'priority',
|
||||
usage: { ...completion.usage, cost: undefined },
|
||||
});
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
provider.complete(selected.args, selected.model),
|
||||
),
|
||||
).rejects.toMatchObject({ legacyCode: 'model_pricing_unavailable' });
|
||||
expect(recordSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -22,8 +22,10 @@ import { OpenAI } from 'openai';
|
||||
import { HttpError, isHttpError } from '../../../../core/http/HttpError.js';
|
||||
import { UpstreamResponseError } from '../../../util/upstreamErrors.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
import type { IChatModel, ICompleteArguments } from '../../types.js';
|
||||
import type { IChatModel, ICompleteArguments, ModelCost } from '../../types.js';
|
||||
import { BoundedTtlMap } from '../../../../util/boundedTtlMap.js';
|
||||
import { cachedRemoteCatalog } from '../../utils/cachedRemoteCatalog.js';
|
||||
import { servedServiceSpeed } from '../../utils/serviceSpeed.js';
|
||||
import { catalogPrice } from '../../utils/pricing.js';
|
||||
import { mapOpenRouterPricing } from './pricing.js';
|
||||
import { sdkClientOptions } from '../../utils/sdkClient.js';
|
||||
@@ -64,6 +66,75 @@ const isPolicyBlock = (err: unknown): boolean => {
|
||||
export class OpenRouterProvider extends OpenAICompatProvider {
|
||||
#apiBaseUrl: string;
|
||||
|
||||
#speedCatalogs = new BoundedTtlMap<
|
||||
string,
|
||||
() => Promise<Partial<Record<'fast' | 'ultrafast', IChatModel>>>
|
||||
>({ maxEntries: 256 });
|
||||
|
||||
async serviceSpeedModels(model: IChatModel) {
|
||||
let load = this.#speedCatalogs.get(model.id);
|
||||
if (!load) {
|
||||
load = cachedRemoteCatalog({
|
||||
name: 'OpenRouter speed catalog',
|
||||
fallback: {} as Partial<
|
||||
Record<'fast' | 'ultrafast', IChatModel>
|
||||
>,
|
||||
fetch: async (signal) => {
|
||||
const id = model.id.replace(/^openrouter:/, '');
|
||||
const response = await axios.request({
|
||||
url: `${this.#apiBaseUrl}/models/${id}/endpoints`,
|
||||
method: 'GET',
|
||||
signal,
|
||||
maxContentLength: 2 * 1024 * 1024,
|
||||
});
|
||||
const offers: Partial<
|
||||
Record<'fast' | 'ultrafast', IChatModel>
|
||||
> = {};
|
||||
for (const endpoint of response.data?.data?.endpoints ??
|
||||
[]) {
|
||||
const speed =
|
||||
endpoint.service_tier === 'ultrafast'
|
||||
? 'ultrafast'
|
||||
: ['priority', 'fast'].includes(
|
||||
endpoint.service_tier,
|
||||
)
|
||||
? 'fast'
|
||||
: undefined;
|
||||
if (!speed || !endpoint.pricing) continue;
|
||||
const pricing = mapOpenRouterPricing(endpoint.pricing);
|
||||
if (
|
||||
pricing.costs.prompt === undefined ||
|
||||
pricing.costs.completion === undefined
|
||||
)
|
||||
continue;
|
||||
// Reserve for the most expensive eligible endpoint and context tier.
|
||||
const costs: ModelCost = {
|
||||
...offers[speed]?.costs,
|
||||
tokens: model.costs.tokens,
|
||||
};
|
||||
for (const rates of [
|
||||
pricing.costs,
|
||||
...pricing.overrides.map(
|
||||
(override) => override.costs,
|
||||
),
|
||||
]) {
|
||||
for (const [key, value] of Object.entries(rates))
|
||||
costs[key] = Math.max(costs[key] ?? 0, value);
|
||||
}
|
||||
offers[speed] = {
|
||||
...model,
|
||||
costs,
|
||||
pricingOverrides: [],
|
||||
};
|
||||
}
|
||||
return offers;
|
||||
},
|
||||
});
|
||||
this.#speedCatalogs.set(model.id, load);
|
||||
}
|
||||
return load();
|
||||
}
|
||||
|
||||
constructor(config: ChatProviderConfig, meteringService: MeteringService) {
|
||||
const apiBaseUrl = config.apiBaseUrl || 'https://openrouter.ai/api/v1';
|
||||
super(meteringService, {
|
||||
@@ -202,8 +273,17 @@ export class OpenRouterProvider extends OpenAICompatProvider {
|
||||
return { ...params, usage: { include: true } };
|
||||
}
|
||||
|
||||
protected override meteredUsage(source: UsageSource) {
|
||||
protected override meteredUsage(source: UsageSource, model: IChatModel) {
|
||||
const usage = source.usage as OpenrouterUsage;
|
||||
if (
|
||||
model.serviceSpeedPricing &&
|
||||
servedServiceSpeed(source) !== 'standard' &&
|
||||
catalogPrice(usage.cost) === undefined
|
||||
) {
|
||||
throw new HttpError(503, 'Service tier billing is unavailable', {
|
||||
legacyCode: 'model_pricing_unavailable',
|
||||
});
|
||||
}
|
||||
return {
|
||||
usage: {
|
||||
prompt:
|
||||
|
||||
@@ -20,13 +20,12 @@
|
||||
/**
|
||||
* Offline unit tests for XAIProvider.
|
||||
*
|
||||
* Boots a real PuterServer (in-memory sqlite + dynamo + s3 + mock
|
||||
* redis) and constructs XAIProvider directly against the live wired
|
||||
* `MeteringService` so the recording side is exercised end-to-end.
|
||||
* xAI is OpenAI-compatible so the OpenAI SDK is mocked at the module
|
||||
* boundary; that's the real network egress point. The companion
|
||||
* integration test (XAIProvider.integration.test.ts) exercises the
|
||||
* real xAI endpoint.
|
||||
* Boots a real PuterServer (in-memory sqlite + dynamo + s3 + mock redis) and
|
||||
* constructs XAIProvider directly against the live wired `MeteringService` so
|
||||
* the recording side is exercised end-to-end. xAI is OpenAI-compatible so the
|
||||
* OpenAI SDK is mocked at the module boundary; that's the real network egress
|
||||
* point. The companion integration test (XAIProvider.integration.test.ts)
|
||||
* exercises the real xAI endpoint.
|
||||
*/
|
||||
|
||||
import { Writable } from 'node:stream';
|
||||
@@ -933,8 +932,7 @@ describe('XAIProvider.complete non-stream output', () => {
|
||||
// namespace, actor, and a costsOverride priced from
|
||||
// models.ts (grok-4.6: prompt=200, completion=600, cached=50).
|
||||
expect(recordSpy).toHaveBeenCalledTimes(1);
|
||||
const [usage, actor, prefix, overrides] =
|
||||
recordSpy.mock.calls[0]!;
|
||||
const [usage, actor, prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(usage).toEqual({
|
||||
prompt_tokens: 90,
|
||||
completion_tokens: 50,
|
||||
@@ -1022,8 +1020,7 @@ describe('XAIProvider.complete non-stream output', () => {
|
||||
}),
|
||||
);
|
||||
|
||||
const [usage, , , overrides] =
|
||||
recordSpy.mock.calls[0]!;
|
||||
const [usage, , , overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(usage.cached_tokens).toBe(0);
|
||||
// 0 cached tokens × any rate = 0 metered cost.
|
||||
expect(overrides).toMatchObject({ cached_tokens: 0 });
|
||||
@@ -1081,8 +1078,7 @@ describe('XAIProvider.complete streaming', () => {
|
||||
|
||||
// grok-build-0.1 costs: prompt=100, completion=200, cached=20.
|
||||
expect(recordSpy).toHaveBeenCalledTimes(1);
|
||||
const [, , prefix, overrides] =
|
||||
recordSpy.mock.calls[0]!;
|
||||
const [, , prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(prefix).toBe('xai:grok-build-0.1');
|
||||
expect(overrides).toEqual({
|
||||
prompt_tokens: 3 * 100,
|
||||
@@ -1200,3 +1196,60 @@ describe('XAIProvider.checkModeration', () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('xAI service speed billing', () => {
|
||||
it('records the reported Priority charge with cached input and reasoning', async () => {
|
||||
const { selectServiceSpeed } =
|
||||
await import('../../utils/serviceSpeed.js');
|
||||
const { provider } = makeProvider();
|
||||
const base = {
|
||||
...(await provider.models()).find(
|
||||
(entry) => entry.id === 'grok-4.7',
|
||||
)!,
|
||||
provider: 'xai',
|
||||
};
|
||||
const selected = selectServiceSpeed(
|
||||
base,
|
||||
{
|
||||
model: base.id,
|
||||
messages: [{ role: 'user', content: 'Hello' }],
|
||||
serviceSpeed: 'ultrafast',
|
||||
},
|
||||
() => undefined,
|
||||
);
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
index: 0,
|
||||
message: { role: 'assistant', content: 'OK' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
service_tier: 'priority',
|
||||
usage: {
|
||||
prompt_tokens: 1247,
|
||||
completion_tokens: 1,
|
||||
total_tokens: 1282,
|
||||
prompt_tokens_details: { cached_tokens: 1152 },
|
||||
completion_tokens_details: { reasoning_tokens: 34 },
|
||||
cost_in_usd_ticks: 19520000,
|
||||
},
|
||||
});
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete(selected.args, selected.model),
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
usageDetails: {
|
||||
serviceSpeed: 'fast',
|
||||
outputTokens: 35,
|
||||
reasoningTokens: 34,
|
||||
},
|
||||
});
|
||||
expect(recordSpy).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ billedUsage: 1, completion_tokens: 35 }),
|
||||
expect.anything(),
|
||||
'xai:grok-4.7',
|
||||
expect.objectContaining({ billedUsage: 195200 }),
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -19,6 +19,8 @@
|
||||
|
||||
import { OpenAI } from 'openai';
|
||||
import { ChatCompletionCreateParams } from 'openai/resources/index.js';
|
||||
import { catalogPrice } from '../../utils/pricing.js';
|
||||
import { servedServiceSpeed } from '../../utils/serviceSpeed.js';
|
||||
import { Context } from '../../../../core/context.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
import * as OpenAIUtil from '../../utils/OpenAIUtil.js';
|
||||
@@ -74,7 +76,15 @@ export class XAIProvider implements IChatProvider {
|
||||
}
|
||||
|
||||
async complete(
|
||||
{ messages, stream, model, tools, max_tokens }: ICompleteArguments,
|
||||
{
|
||||
messages,
|
||||
stream,
|
||||
model,
|
||||
tools,
|
||||
max_tokens,
|
||||
service_tier,
|
||||
serviceSpeed: requestedServiceSpeed,
|
||||
}: ICompleteArguments,
|
||||
resolved?: IChatModel,
|
||||
): Promise<IChatCompleteResult> {
|
||||
const actor = Context.get('actor');
|
||||
@@ -88,6 +98,7 @@ export class XAIProvider implements IChatProvider {
|
||||
{
|
||||
messages,
|
||||
model: modelUsed.id,
|
||||
...(service_tier !== undefined ? { service_tier } : {}),
|
||||
...(tools ? { tools } : {}),
|
||||
...(max_tokens !== undefined ? { max_tokens } : {}),
|
||||
stream,
|
||||
@@ -107,18 +118,49 @@ export class XAIProvider implements IChatProvider {
|
||||
}
|
||||
|
||||
return OpenAIUtil.handle_completion_output({
|
||||
usage_calculator: ({ usage, setUsageCosts }) => {
|
||||
usage_calculator: ({
|
||||
usage,
|
||||
service_tier: servedTier,
|
||||
setUsageCosts,
|
||||
setUsageDetails,
|
||||
}) => {
|
||||
const serviceSpeed = servedServiceSpeed({
|
||||
service_tier: servedTier,
|
||||
});
|
||||
// xAI reports cached reads inside `prompt_tokens`; bill them
|
||||
// only at the cached rate.
|
||||
const metered = OpenAIUtil.extractMeteredUsage(usage);
|
||||
const reasoningTokens =
|
||||
usage.completion_tokens_details?.reasoning_tokens ?? 0;
|
||||
const trackedUsage = OpenAIUtil.splitCachedPrompt(usage);
|
||||
trackedUsage.completion_tokens += reasoningTokens;
|
||||
const reportedTicks = catalogPrice(
|
||||
(usage as typeof usage & { cost_in_usd_ticks?: number })
|
||||
.cost_in_usd_ticks,
|
||||
);
|
||||
const { costs } = meterChatUsage(
|
||||
this.#meteringService,
|
||||
actor,
|
||||
this.meteringModelKey(modelUsed.id),
|
||||
modelUsed,
|
||||
OpenAIUtil.splitCachedPrompt(usage),
|
||||
trackedUsage,
|
||||
{
|
||||
serviceSpeed,
|
||||
authoritativeUsd:
|
||||
reportedTicks === undefined
|
||||
? undefined
|
||||
: reportedTicks / 10_000_000_000,
|
||||
},
|
||||
);
|
||||
setUsageCosts(costs);
|
||||
if (requestedServiceSpeed !== undefined)
|
||||
setUsageDetails?.({
|
||||
...OpenAIUtil.usageDetailsFromTrackedUsage(
|
||||
trackedUsage,
|
||||
)!,
|
||||
serviceSpeed,
|
||||
...(reasoningTokens ? { reasoningTokens } : {}),
|
||||
});
|
||||
return metered;
|
||||
},
|
||||
stream,
|
||||
|
||||
@@ -42,6 +42,7 @@ export const XAI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -65,6 +66,7 @@ export const XAI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -87,6 +89,7 @@ export const XAI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -109,6 +112,7 @@ export const XAI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
@@ -151,6 +155,7 @@ export const XAI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
@@ -186,6 +191,7 @@ export const XAI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
@@ -217,6 +223,7 @@ export const XAI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
@@ -246,6 +253,7 @@ export const XAI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 100,
|
||||
|
||||
@@ -26,6 +26,8 @@
|
||||
|
||||
export type ModelCost = Record<string, number>;
|
||||
|
||||
export type ServiceSpeed = 'standard' | 'fast' | 'ultrafast';
|
||||
|
||||
export interface ModelModalities {
|
||||
input: string[];
|
||||
output: string[];
|
||||
@@ -67,6 +69,22 @@ export interface IChatModel<T extends ModelCost = ModelCost> extends Record<
|
||||
output_multiplier: number;
|
||||
};
|
||||
pricingOverrides?: ModelPricingOverride[];
|
||||
/**
|
||||
* A token-rate multiplier or explicit per-key rates merged with standard
|
||||
* costs.
|
||||
*/
|
||||
serviceSpeedRates?: Partial<
|
||||
Record<'fast' | 'ultrafast', number | ModelCost>
|
||||
>;
|
||||
serviceSpeedVariants?: { fast?: string };
|
||||
/** Request-local pricing snapshot; never part of the model catalog. */
|
||||
serviceSpeedPricing?: {
|
||||
speed: ServiceSpeed;
|
||||
standardCosts: ModelCost;
|
||||
standardOverrides?: ModelPricingOverride[];
|
||||
fastCosts?: ModelCost;
|
||||
ultrafastCosts?: ModelCost;
|
||||
};
|
||||
context?: number;
|
||||
max_tokens: number;
|
||||
subscriberOnly?: boolean;
|
||||
@@ -76,6 +94,8 @@ export interface IChatModel<T extends ModelCost = ModelCost> extends Record<
|
||||
tool_call?: boolean;
|
||||
responses_api?: boolean;
|
||||
responses_api_only?: boolean;
|
||||
/** Restricts service speeds for deployments with a limited API dialect. */
|
||||
serviceSpeeds?: ServiceSpeed[];
|
||||
/**
|
||||
* Omitted preserves sampling; reasoningDisabled requires explicit effort
|
||||
* none.
|
||||
@@ -163,6 +183,7 @@ export interface UsageDetails {
|
||||
webFetchRequests?: number;
|
||||
speed?: 'fast' | 'standard';
|
||||
serviceTier?: string;
|
||||
serviceSpeed?: ServiceSpeed;
|
||||
/**
|
||||
* Per-pass breakdown when the upstream ran several passes (compaction,
|
||||
* advisor).
|
||||
@@ -198,6 +219,7 @@ export interface ICompleteArguments {
|
||||
* request on another provider that serves the same model.
|
||||
*/
|
||||
fallback?: boolean;
|
||||
serviceSpeed?: ServiceSpeed;
|
||||
stream?: boolean;
|
||||
model: string;
|
||||
test_mode?: boolean;
|
||||
@@ -217,10 +239,7 @@ export interface ICompleteArguments {
|
||||
taskBudget?: { total: number; remaining?: number };
|
||||
/** Claude only: top-level automatic `cache_control`. */
|
||||
cacheControl?: CacheControl;
|
||||
/**
|
||||
* Claude only; forwarded only when the model's catalog entry has
|
||||
* `fast_output_tokens`.
|
||||
*/
|
||||
/** Claude only; fast is forwarded when the catalog declares fast rates. */
|
||||
speed?: 'fast' | 'standard';
|
||||
/** Claude only. */
|
||||
safeguards?: SafeguardRequest[];
|
||||
@@ -260,7 +279,15 @@ export interface ICompleteArguments {
|
||||
truncation?: 'auto' | 'disabled' | undefined;
|
||||
background?: boolean;
|
||||
service_tier?:
|
||||
'auto' | 'default' | 'flex' | 'scale' | 'priority' | undefined;
|
||||
| 'auto'
|
||||
| 'default'
|
||||
| 'flex'
|
||||
| 'scale'
|
||||
| 'priority'
|
||||
| 'fast'
|
||||
| 'ultrafast'
|
||||
| 'standard'
|
||||
| undefined;
|
||||
max_tokens?: number;
|
||||
temperature?: number;
|
||||
reasoning?: { effort: ReasoningEffort } | undefined;
|
||||
@@ -328,6 +355,10 @@ export interface IChatMessageResult {
|
||||
export type IChatCompleteResult = IChatStreamResult | IChatMessageResult;
|
||||
|
||||
export interface IChatProvider {
|
||||
/** Account/catalog-specific speed offerings, fetched before the credit hold. */
|
||||
serviceSpeedModels?(
|
||||
model: IChatModel,
|
||||
): Promise<Partial<Record<'fast' | 'ultrafast', IChatModel>>>;
|
||||
models(extra_params?: unknown): IChatModel[] | Promise<IChatModel[]>;
|
||||
getDefaultModel(): string;
|
||||
/**
|
||||
|
||||
@@ -650,12 +650,15 @@ export const create_chat_stream_handler =
|
||||
block.end();
|
||||
};
|
||||
|
||||
let serviceTier;
|
||||
let last_usage = null;
|
||||
let last_extra_content = null;
|
||||
let finish_reason = null;
|
||||
for await (let chunk of completion) {
|
||||
await chatStream.waitForDrain();
|
||||
chunk = deviations.chunk_but_like_actually(chunk);
|
||||
if (chunk.service_tier !== undefined)
|
||||
serviceTier = chunk.service_tier;
|
||||
const chunk_usage = deviations.index_usage_from_stream_chunk(chunk);
|
||||
if (chunk_usage) last_usage = chunk_usage;
|
||||
if (chunk.choices.length < 1) continue;
|
||||
@@ -746,6 +749,7 @@ export const create_chat_stream_handler =
|
||||
const usage = last_usage
|
||||
? usage_calculator({
|
||||
usage: last_usage,
|
||||
service_tier: serviceTier,
|
||||
extra_content: last_extra_content,
|
||||
setUsageCosts: (c) => chatStream.setUsageCosts(c),
|
||||
setUsageDetails: (d) => {
|
||||
@@ -779,6 +783,7 @@ export const create_chat_stream_handler_responses_api =
|
||||
const message = chatStream.message();
|
||||
const textblock = message.contentBlock({ type: 'text' });
|
||||
|
||||
let serviceTier;
|
||||
let last_usage = null;
|
||||
let completed = false;
|
||||
let sawFunctionCall = false;
|
||||
@@ -787,6 +792,8 @@ export const create_chat_stream_handler_responses_api =
|
||||
let toolUsage = null;
|
||||
for await (const chunk of completion) {
|
||||
await chatStream.waitForDrain();
|
||||
if (chunk.response?.service_tier !== undefined)
|
||||
serviceTier = chunk.response.service_tier;
|
||||
if (chunk.type === 'response.failed' || chunk.type === 'error') {
|
||||
throw new Error(
|
||||
chunk.response?.error?.message ??
|
||||
@@ -918,6 +925,7 @@ export const create_chat_stream_handler_responses_api =
|
||||
const usage = last_usage
|
||||
? usage_calculator({
|
||||
usage: last_usage,
|
||||
service_tier: serviceTier,
|
||||
webSearchCalls,
|
||||
tool_usage: toolUsage,
|
||||
setUsageCosts: (c) => chatStream.setUsageCosts(c),
|
||||
@@ -972,6 +980,7 @@ export const handle_completion_output = async (
|
||||
/**
|
||||
* @type {Record<string, unknown> & {
|
||||
* usage_calculator: (args: {
|
||||
* service_tier?: string;
|
||||
* usage: import('openai/resources/completions.mjs').CompletionUsage;
|
||||
* setUsageCosts: (costs: Record<string, number>) => void;
|
||||
* setUsageDetails: (
|
||||
@@ -1066,6 +1075,7 @@ export const handle_completion_output = async (
|
||||
* @param {((text: string) => Promise<{ flagged: boolean }>) | undefined} [params.moderate]
|
||||
* @param {(args: {
|
||||
* usage: import('openai/resources/responses/responses.mjs').ResponseUsage;
|
||||
* service_tier?: string;
|
||||
* webSearchCalls?: number;
|
||||
* tool_usage?: Record<string, unknown>;
|
||||
* setUsageCosts?: (costs: Record<string, number>) => void;
|
||||
|
||||
@@ -19,13 +19,15 @@
|
||||
|
||||
import type { Actor } from '../../../core/actor.js';
|
||||
import type { MeteringService } from '../../../services/metering/MeteringService.js';
|
||||
import type { IChatModel } from '../types.js';
|
||||
import type { IChatModel, ServiceSpeed } from '../types.js';
|
||||
import { serviceSpeedBillingModel } from './serviceSpeed.js';
|
||||
import { buildCostsOverride } from './pricing.js';
|
||||
|
||||
/** Microcents per US dollar. */
|
||||
const MICROCENTS_PER_USD = 100_000_000;
|
||||
|
||||
export interface MeterChatUsageOptions {
|
||||
serviceSpeed?: ServiceSpeed;
|
||||
/**
|
||||
* What the upstream says it billed, in USD. Recorded as one `billedUsage`
|
||||
* line instead of pricing the tokens, which are still recorded at zero.
|
||||
@@ -77,10 +79,15 @@ export const meterChatUsage = (
|
||||
authoritativeUsd,
|
||||
costOverrides,
|
||||
pricingTime,
|
||||
serviceSpeed = 'standard',
|
||||
}: MeterChatUsageOptions = {},
|
||||
): { usage: Record<string, number>; costs: Record<string, number> } => {
|
||||
const priced = {
|
||||
...buildCostsOverride(usage, model, pricingTime),
|
||||
...buildCostsOverride(
|
||||
usage,
|
||||
serviceSpeedBillingModel(model, serviceSpeed),
|
||||
pricingTime,
|
||||
),
|
||||
...costOverrides,
|
||||
};
|
||||
if (authoritativeUsd !== undefined) {
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* Puter is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with Puter. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import type { IChatModel, ServiceSpeed } from '../types.js';
|
||||
import { ALIBABA_MODELS } from '../providers/alibaba/models.js';
|
||||
import { AZURE_MODELS } from '../providers/azure/models.js';
|
||||
import { BYTEPLUS_MODELS } from '../providers/byteplus/models.js';
|
||||
import { CLAUDE_MODELS } from '../providers/claude/models.js';
|
||||
import { DEEPSEEK_MODELS } from '../providers/deepseek/models.js';
|
||||
import { GEMINI_MODELS } from '../providers/gemini/models.js';
|
||||
import { GROQ_MODELS } from '../providers/groq/models.js';
|
||||
import { HOONIFY_MODELS } from '../providers/hoonify/models.js';
|
||||
import { META_MODELS } from '../providers/meta/models.js';
|
||||
import { MINIMAX_MODELS } from '../providers/minimax/models.js';
|
||||
import { MISTRAL_MODELS } from '../providers/mistral/models.js';
|
||||
import { MOONSHOT_MODELS } from '../providers/moonshot/models.js';
|
||||
import { OPEN_AI_MODELS } from '../providers/openai/models.js';
|
||||
import { XAI_MODELS } from '../providers/xai/models.js';
|
||||
import { ZAI_MODELS } from '../providers/zai/models.js';
|
||||
import { shouldPresentAsOpenAI } from './normalizeToOpenAI.js';
|
||||
import {
|
||||
isServiceSpeedUnavailable,
|
||||
lowerServiceSpeed,
|
||||
selectServiceSpeed,
|
||||
servedServiceSpeed,
|
||||
serviceSpeedBillingModel,
|
||||
} from './serviceSpeed.js';
|
||||
|
||||
const catalogs: Array<[string, IChatModel[]]> = [
|
||||
['alibaba', ALIBABA_MODELS],
|
||||
['azure-openai', AZURE_MODELS],
|
||||
['byteplus', BYTEPLUS_MODELS],
|
||||
['claude', CLAUDE_MODELS],
|
||||
['deepseek', DEEPSEEK_MODELS],
|
||||
['gemini', GEMINI_MODELS],
|
||||
['groq', GROQ_MODELS],
|
||||
['hoonify', HOONIFY_MODELS],
|
||||
['meta', META_MODELS],
|
||||
['minimax', MINIMAX_MODELS],
|
||||
['mistral', MISTRAL_MODELS],
|
||||
['moonshotai', MOONSHOT_MODELS],
|
||||
['openai-completion', OPEN_AI_MODELS],
|
||||
['openai-responses', OPEN_AI_MODELS],
|
||||
['xai', XAI_MODELS],
|
||||
['zai', ZAI_MODELS],
|
||||
];
|
||||
const speeds: ServiceSpeed[] = ['standard', 'fast', 'ultrafast'];
|
||||
|
||||
describe('service speed seeded fuzzing', () => {
|
||||
it('preserves catalog, request, normalization, and billing invariants across model changes', () => {
|
||||
let seed = 0x51eed;
|
||||
const pick = <T>(values: T[]): T => {
|
||||
seed = (Math.imul(seed, 1664525) + 1013904223) >>> 0;
|
||||
return values[seed % values.length]!;
|
||||
};
|
||||
const seenProviders = new Set<string>();
|
||||
for (let index = 0; index < 10_000; index++) {
|
||||
const [provider, catalog] = pick(catalogs);
|
||||
seenProviders.add(provider);
|
||||
const base = { ...structuredClone(pick(catalog)), provider };
|
||||
const before = structuredClone(base);
|
||||
Object.freeze(base.costs);
|
||||
Object.freeze(base);
|
||||
const args = Object.freeze({
|
||||
model: base.id,
|
||||
messages: [{ role: 'user', content: `Case ${index}` }],
|
||||
serviceSpeed: pick([...speeds, undefined]),
|
||||
normalize: pick([true, false, undefined]),
|
||||
temperature: pick([0, 0.5, 1]),
|
||||
service_tier: pick(['flex' as const, 'ultrafast' as const, undefined]),
|
||||
speed: pick(['standard' as const, 'fast' as const, undefined]),
|
||||
});
|
||||
const selected = selectServiceSpeed(base, args, (id) => {
|
||||
const variant = catalog.find((entry) => entry.id === id);
|
||||
return variant && { ...variant, provider };
|
||||
});
|
||||
expect(base).toEqual(before);
|
||||
expect(selected.args.messages).toBe(args.messages);
|
||||
expect(selected.args.temperature).toBe(args.temperature);
|
||||
expect(selected.model.provider).toBe(provider);
|
||||
expect(selected.model.release_date).toBe(base.release_date);
|
||||
expect(
|
||||
shouldPresentAsOpenAI(selected.args, selected.model.release_date),
|
||||
).toBe(shouldPresentAsOpenAI(args, base.release_date));
|
||||
if (args.serviceSpeed === undefined) {
|
||||
expect(selected.args).toBe(args);
|
||||
expect(selected.model).toBe(base);
|
||||
continue;
|
||||
}
|
||||
expect(speeds.indexOf(selected.speed)).toBeLessThanOrEqual(
|
||||
speeds.indexOf(args.serviceSpeed),
|
||||
);
|
||||
expect(selected.args.serviceSpeed).toBe(selected.speed);
|
||||
if (args.serviceSpeed === 'standard') {
|
||||
expect(selected.args.speed).not.toBe('fast');
|
||||
expect(selected.args.service_tier).not.toBe('ultrafast');
|
||||
}
|
||||
for (const cost of Object.values(selected.model.costs)) {
|
||||
expect(Number.isFinite(cost)).toBe(true);
|
||||
expect(cost).toBeGreaterThanOrEqual(0);
|
||||
}
|
||||
if (selected.model.serviceSpeedPricing) {
|
||||
const settled = serviceSpeedBillingModel(selected.model, 'standard');
|
||||
expect(settled.costs).toEqual(base.costs);
|
||||
expect(settled.pricingOverrides).toEqual(base.pricingOverrides);
|
||||
}
|
||||
let speed = selected.speed;
|
||||
for (let attempts = 0; speed !== 'standard'; attempts++) {
|
||||
expect(attempts).toBeLessThan(2);
|
||||
speed = lowerServiceSpeed(speed);
|
||||
}
|
||||
}
|
||||
expect([...seenProviders].sort()).toEqual(catalogs.map(([provider]) => provider).sort());
|
||||
});
|
||||
|
||||
it('handles malformed upstream tier and admission-error metadata without coercion or retries', () => {
|
||||
const values = [null, false, 0, 429, '', 'unrelated', [], {},
|
||||
['service_tier capacity unavailable'],
|
||||
{ toString: null }, { toString: 'priority capacity unavailable' }];
|
||||
for (const value of values) {
|
||||
expect(isServiceSpeedUnavailable(value)).toBe(false);
|
||||
for (const status of [undefined, 400, 401, 403, 404, 429, 500, 503]) {
|
||||
for (const variant of [false, true]) {
|
||||
expect(isServiceSpeedUnavailable({ status, message: value }, variant)).toBe(false);
|
||||
expect(isServiceSpeedUnavailable({ status, error: { message: value } }, variant)).toBe(false);
|
||||
}
|
||||
}
|
||||
expect(servedServiceSpeed({ usage: value })).toBe('standard');
|
||||
expect(servedServiceSpeed({ service_tier: value })).toBe('standard');
|
||||
expect(servedServiceSpeed({ usage: { speed: value, service_tier: value } })).toBe('standard');
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,148 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
makeMeteringStub,
|
||||
optionalEnv,
|
||||
withTestActor,
|
||||
} from '../../integrationTestUtil.js';
|
||||
import type {
|
||||
IChatMessageResult,
|
||||
IChatProvider,
|
||||
ICompleteArguments,
|
||||
} from '../types.js';
|
||||
import { OpenAiResponsesChatProvider } from '../providers/openai/OpenAiChatResponsesProvider.js';
|
||||
import { GeminiChatProvider } from '../providers/gemini/GeminiChatProvider.js';
|
||||
import { MiniMaxProvider } from '../providers/minimax/MiniMaxProvider.js';
|
||||
import { MoonshotProvider } from '../providers/moonshot/MoonshotProvider.js';
|
||||
import { MistralAIProvider } from '../providers/mistral/MistralAiProvider.js';
|
||||
import { selectServiceSpeed } from './serviceSpeed.js';
|
||||
|
||||
type Case = {
|
||||
name: string;
|
||||
env: string;
|
||||
provider: string;
|
||||
model: string;
|
||||
build: (key: string) => IChatProvider;
|
||||
};
|
||||
const cases: Case[] = [
|
||||
{
|
||||
name: 'OpenAI Ultrafast',
|
||||
env: 'PUTER_TEST_AI_OPENAI_API_KEY',
|
||||
provider: 'openai-responses',
|
||||
model: 'gpt-6.1-sol',
|
||||
build: (key) =>
|
||||
new OpenAiResponsesChatProvider(
|
||||
{ apiKey: key },
|
||||
makeMeteringStub(),
|
||||
),
|
||||
},
|
||||
{
|
||||
name: 'Gemini Priority',
|
||||
env: 'PUTER_TEST_AI_GEMINI_API_KEY',
|
||||
provider: 'gemini',
|
||||
model: 'gemini-3.8-flash',
|
||||
build: (key) =>
|
||||
new GeminiChatProvider({ apiKey: key }, makeMeteringStub()),
|
||||
},
|
||||
{
|
||||
name: 'MiniMax Priority',
|
||||
env: 'PUTER_TEST_AI_MINIMAX_API_KEY',
|
||||
provider: 'minimax',
|
||||
model: 'minimax-m3',
|
||||
build: (key) =>
|
||||
new MiniMaxProvider({ apiKey: key }, makeMeteringStub()),
|
||||
},
|
||||
{
|
||||
name: 'Moonshot Highspeed',
|
||||
env: 'PUTER_TEST_AI_MOONSHOT_API_KEY',
|
||||
provider: 'moonshotai',
|
||||
model: 'kimi-k3.1-pro',
|
||||
build: (key) =>
|
||||
new MoonshotProvider({ apiKey: key }, makeMeteringStub()),
|
||||
},
|
||||
{
|
||||
name: 'Mistral served tier',
|
||||
env: 'PUTER_TEST_AI_MISTRAL_API_KEY',
|
||||
provider: 'mistral',
|
||||
model: 'mistral-medium-latest',
|
||||
build: (key) =>
|
||||
new MistralAIProvider({ apiKey: key }, makeMeteringStub()),
|
||||
},
|
||||
];
|
||||
|
||||
for (const testCase of cases) {
|
||||
describe.skipIf(!optionalEnv(testCase.env))(
|
||||
`serviceSpeed ${testCase.name} (integration)`,
|
||||
() => {
|
||||
it(
|
||||
'completes with a selected speed and normalized text',
|
||||
{ timeout: 90_000 },
|
||||
async () => {
|
||||
const provider = testCase.build(optionalEnv(testCase.env)!);
|
||||
const catalog = (await provider.models()).map((model) => ({
|
||||
...model,
|
||||
provider: testCase.provider,
|
||||
}));
|
||||
const base = catalog.find(
|
||||
(model) =>
|
||||
model.id === testCase.model ||
|
||||
model.aliases?.includes(testCase.model),
|
||||
)!;
|
||||
expect(base).toBeDefined();
|
||||
const args: ICompleteArguments = {
|
||||
model: base.id,
|
||||
messages: [
|
||||
{ role: 'user', content: 'Reply with exactly OK.' },
|
||||
],
|
||||
serviceSpeed: 'ultrafast',
|
||||
max_tokens: 256,
|
||||
reasoning_effort: 'low',
|
||||
normalize: true,
|
||||
};
|
||||
const selection = selectServiceSpeed(base, args, (id) =>
|
||||
catalog.find((model) => model.id === id),
|
||||
);
|
||||
expect(selection.speed).toBe(
|
||||
testCase.provider === 'openai-responses'
|
||||
? 'ultrafast'
|
||||
: 'fast',
|
||||
);
|
||||
const result = (await withTestActor(() =>
|
||||
provider.complete(
|
||||
{ ...selection.args, model: selection.model.id },
|
||||
selection.model,
|
||||
),
|
||||
)) as IChatMessageResult;
|
||||
expect(typeof result.message.content).toBe('string');
|
||||
expect(result.message.content.length).toBeGreaterThan(0);
|
||||
if (
|
||||
['openai-responses', 'gemini', 'minimax'].includes(
|
||||
testCase.provider,
|
||||
)
|
||||
)
|
||||
expect(result.usageDetails?.serviceSpeed).toBe(
|
||||
selection.speed,
|
||||
);
|
||||
},
|
||||
);
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,368 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { Writable } from 'node:stream';
|
||||
import { AIChatStream } from './Streaming.js';
|
||||
import {
|
||||
handle_completion_output,
|
||||
handle_completion_output_responses_api,
|
||||
} from './OpenAIUtil.js';
|
||||
import type { IChatModel, ServiceSpeed } from '../types.js';
|
||||
import {
|
||||
selectServiceSpeed,
|
||||
servedServiceSpeed,
|
||||
serviceSpeedBillingModel,
|
||||
isServiceSpeedUnavailable,
|
||||
} from './serviceSpeed.js';
|
||||
import { meterChatUsage } from './meterChatUsage.js';
|
||||
import { OPEN_AI_MODELS } from '../providers/openai/models.js';
|
||||
import { XAI_MODELS } from '../providers/xai/models.js';
|
||||
import { GEMINI_MODELS } from '../providers/gemini/models.js';
|
||||
import { CLAUDE_MODELS } from '../providers/claude/models.js';
|
||||
|
||||
const model = (provider: string, id: string): IChatModel => ({
|
||||
provider,
|
||||
id,
|
||||
serviceSpeedRates: [...OPEN_AI_MODELS, ...XAI_MODELS, ...GEMINI_MODELS, ...CLAUDE_MODELS]
|
||||
.find((entry) => entry.id === id)?.serviceSpeedRates,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
max_tokens: 1000,
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
completion_tokens: 1000,
|
||||
cached_tokens: 10,
|
||||
web_search_requests: 1_000_000,
|
||||
},
|
||||
});
|
||||
const request = (serviceSpeed?: ServiceSpeed) => ({
|
||||
model: 'test',
|
||||
messages: [{ role: 'user', content: 'Hello' }],
|
||||
serviceSpeed,
|
||||
});
|
||||
const noVariant = () => undefined;
|
||||
|
||||
describe('service speed selection', () => {
|
||||
it.each([2, { prompt_tokens: 400, completion_tokens: 2000 }])(
|
||||
'uses catalog rates independently of provider and model names: %j',
|
||||
(rates) => {
|
||||
const entry = {
|
||||
...model('catalog-provider', 'renamed-model'),
|
||||
serviceSpeedRates: { fast: rates },
|
||||
};
|
||||
const selected = selectServiceSpeed(entry, request('ultrafast'), noVariant);
|
||||
expect(selected.speed).toBe('fast');
|
||||
expect(selected.model.costs.prompt_tokens).toBe(400);
|
||||
expect(selected.model.costs.completion_tokens).toBe(2000);
|
||||
expect(entry.costs.prompt_tokens).toBe(200);
|
||||
},
|
||||
);
|
||||
it('preserves an omitted option and native parameters', () => {
|
||||
const entry = model('openai-responses', 'gpt-6.1-sol');
|
||||
const args = { ...request(), service_tier: 'flex' as const };
|
||||
const selected = selectServiceSpeed(entry, args, noVariant);
|
||||
expect(selected.model).toBe(entry);
|
||||
expect(selected.args).toBe(args);
|
||||
});
|
||||
it.each([
|
||||
'deepseek',
|
||||
'zai',
|
||||
'groq',
|
||||
'byteplus',
|
||||
'together-ai',
|
||||
'meta',
|
||||
'ollama',
|
||||
'hoonify',
|
||||
'neuralwatt',
|
||||
'fake-chat',
|
||||
])('uses standard when %s has no verified faster offering', (provider) => {
|
||||
const selected = selectServiceSpeed(
|
||||
model(provider, 'some-model'),
|
||||
request('ultrafast'),
|
||||
noVariant,
|
||||
);
|
||||
expect(selected.speed).toBe('standard');
|
||||
expect(selected.args.service_tier).toBeUndefined();
|
||||
});
|
||||
it.each(['standard', 'fast', 'ultrafast'] as const)(
|
||||
'selects %s without changing the model or catalog prices',
|
||||
(speed) => {
|
||||
const entry = model('openai-responses', 'gpt-6.1-sol');
|
||||
const selected = selectServiceSpeed(
|
||||
entry,
|
||||
request(speed),
|
||||
noVariant,
|
||||
);
|
||||
expect(selected.speed).toBe(speed);
|
||||
expect(selected.model.id).toBe(entry.id);
|
||||
expect(selected.model.costs.prompt_tokens).toBe(
|
||||
200 * { standard: 1, fast: 2, ultrafast: 6 }[speed],
|
||||
);
|
||||
expect(selected.model.costs.web_search_requests).toBe(1_000_000);
|
||||
expect(entry.costs.prompt_tokens).toBe(200);
|
||||
},
|
||||
);
|
||||
it('downgrades ultrafast to fast on a model without ultrafast', () => {
|
||||
const selected = selectServiceSpeed(
|
||||
model('xai', 'grok-4.7'),
|
||||
request('ultrafast'),
|
||||
noVariant,
|
||||
);
|
||||
expect(selected.speed).toBe('fast');
|
||||
expect(selected.args.service_tier).toBe('priority');
|
||||
});
|
||||
it('keeps a compatibility-only Gemini deployment at standard speed', () => {
|
||||
const entry = {
|
||||
...model('gemini', 'gemini-3.8-flash'),
|
||||
serviceSpeeds: ['standard'] as ServiceSpeed[],
|
||||
};
|
||||
expect(
|
||||
selectServiceSpeed(entry, request('ultrafast'), noVariant).speed,
|
||||
).toBe('standard');
|
||||
});
|
||||
it('uses Claude speed independently of its service tier', () => {
|
||||
const entry = model('claude', 'claude-opus-5-5');
|
||||
expect(
|
||||
selectServiceSpeed(entry, request('ultrafast'), noVariant).args
|
||||
.speed,
|
||||
).toBe('fast');
|
||||
expect(
|
||||
selectServiceSpeed(entry, request('ultrafast'), noVariant).args
|
||||
.service_tier,
|
||||
).toBeUndefined();
|
||||
expect(
|
||||
servedServiceSpeed({
|
||||
usage: { speed: 'fast', service_tier: 'standard' },
|
||||
}),
|
||||
).toBe('fast');
|
||||
});
|
||||
it.each([
|
||||
['minimax', 'minimax-m2.7', 'minimax-m2.7-highspeed'],
|
||||
['moonshotai', 'kimi-k3.1-pro', 'kimi-k3.1-pro-highspeed'],
|
||||
['alibaba', 'glm-5.3', 'glm-5.3-prime'],
|
||||
['alibaba', 'glm-5.2', 'glm-5.2-fast-preview'],
|
||||
['infron', 'infron:vendor/model', 'infron:vendor/model:priority'],
|
||||
])(
|
||||
'selects an equivalent %s variant and preserves normalization policy',
|
||||
(provider, id, variantId) => {
|
||||
const base = {
|
||||
...model(provider, id),
|
||||
serviceSpeedVariants: { fast: variantId },
|
||||
release_date: '2026-08-01',
|
||||
};
|
||||
const variant = {
|
||||
...model(provider, variantId),
|
||||
release_date: '2026-10-01',
|
||||
};
|
||||
const selected = selectServiceSpeed(
|
||||
base,
|
||||
request('ultrafast'),
|
||||
(candidate) => (candidate === variantId ? variant : undefined),
|
||||
);
|
||||
expect(selected.model.id).toBe(variantId);
|
||||
expect(selected.model.release_date).toBe(base.release_date);
|
||||
expect(selected.speed).toBe('fast');
|
||||
expect(
|
||||
selectServiceSpeed(base, request('fast'), noVariant).speed,
|
||||
).toBe('standard');
|
||||
},
|
||||
);
|
||||
it('keeps caller media on Gemini compatibility when native speed cannot carry it', () => {
|
||||
const args = {
|
||||
...request('fast'),
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{
|
||||
type: 'video_url',
|
||||
video_url: { url: 'data:video/mp4;base64,AA==' },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
expect(
|
||||
selectServiceSpeed(
|
||||
model('gemini', 'gemini-3.8-flash'),
|
||||
args,
|
||||
noVariant,
|
||||
).speed,
|
||||
).toBe('standard');
|
||||
});
|
||||
});
|
||||
|
||||
describe('served speed billing', () => {
|
||||
it.each(['standard', 'fast', 'ultrafast'] as const)(
|
||||
'settles an ultrafast hold at the served %s rate',
|
||||
(speed) => {
|
||||
const selected = selectServiceSpeed(
|
||||
model('openai-responses', 'gpt-6.1-sol'),
|
||||
request('ultrafast'),
|
||||
noVariant,
|
||||
).model;
|
||||
const recorded: Array<Record<string, number> | undefined> = [];
|
||||
const meter = {
|
||||
utilRecordUsageObject: (
|
||||
_usage: Record<string, number>,
|
||||
_actor: unknown,
|
||||
_key: string,
|
||||
costs?: Record<string, number>,
|
||||
) => {
|
||||
recorded.push(costs);
|
||||
},
|
||||
};
|
||||
const result = meterChatUsage(
|
||||
meter as never,
|
||||
undefined,
|
||||
'openai:gpt-6.1-sol',
|
||||
selected,
|
||||
{ prompt_tokens: 10, completion_tokens: 5, cached_tokens: 20 },
|
||||
{ serviceSpeed: speed },
|
||||
);
|
||||
expect(result.costs.prompt_tokens).toBe(
|
||||
2000 * { standard: 1, fast: 2, ultrafast: 6 }[speed],
|
||||
);
|
||||
expect(result.costs.cached_tokens).toBe(
|
||||
200 * { standard: 1, fast: 2, ultrafast: 6 }[speed],
|
||||
);
|
||||
expect(recorded).toEqual([result.costs]);
|
||||
},
|
||||
);
|
||||
it('restores standard scheduled and long-context rates after a downgrade', () => {
|
||||
const entry = model('openai-responses', 'gpt-6.1-sol');
|
||||
entry.pricingOverrides = [
|
||||
{
|
||||
minPromptTokens: 100,
|
||||
costs: { prompt_tokens: 400, completion_tokens: 1500 },
|
||||
},
|
||||
];
|
||||
const selected = selectServiceSpeed(
|
||||
entry,
|
||||
request('ultrafast'),
|
||||
noVariant,
|
||||
).model;
|
||||
expect(
|
||||
serviceSpeedBillingModel(selected, 'standard').pricingOverrides,
|
||||
).toEqual(entry.pricingOverrides);
|
||||
expect(
|
||||
serviceSpeedBillingModel(selected, 'fast').pricingOverrides?.[0]
|
||||
.costs.prompt_tokens,
|
||||
).toBe(800);
|
||||
});
|
||||
it.each([
|
||||
[{ service_tier: 'priority' }, 'fast'],
|
||||
[{ usage: { serviceTier: 'priority' } }, 'fast'],
|
||||
[{ usage: { service_tier: 'standard' } }, 'standard'],
|
||||
[{ service_tier: 'ultrafast' }, 'ultrafast'],
|
||||
[{}, 'standard'],
|
||||
])('reads actual speed from %j', (source, speed) => {
|
||||
expect(servedServiceSpeed(source)).toBe(speed);
|
||||
});
|
||||
});
|
||||
|
||||
describe('speed admission failures', () => {
|
||||
it.each([
|
||||
[{ status: 429, message: 'Fast mode rate limit exceeded' }, true],
|
||||
[{ status: 403, message: 'Ultrafast access unavailable' }, true],
|
||||
[{ status: 400, message: 'service_tier is not supported' }, true],
|
||||
[{ status: 401, message: 'Invalid API key' }, false],
|
||||
[{ status: 429, message: 'Rate limit exceeded' }, false],
|
||||
[{ status: 400, message: 'Invalid messages' }, false],
|
||||
])('classifies %j without swallowing unrelated errors', (error, retry) => {
|
||||
expect(isServiceSpeedUnavailable(error)).toBe(retry);
|
||||
});
|
||||
it('downgrades a missing speed variant without retrying an invalid base model', () => {
|
||||
const error = { status: 404, message: 'Model not found' };
|
||||
expect(isServiceSpeedUnavailable(error, true)).toBe(true);
|
||||
expect(isServiceSpeedUnavailable(error)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('streamed service tier metadata', () => {
|
||||
it('retains a tier from an earlier Chat Completions chunk until usage arrives', async () => {
|
||||
let servedTier: string | undefined;
|
||||
const result = await handle_completion_output({
|
||||
stream: true,
|
||||
completion: (async function* () {
|
||||
yield {
|
||||
service_tier: 'priority',
|
||||
choices: [{ index: 0, delta: { content: 'OK' } }],
|
||||
};
|
||||
yield {
|
||||
choices: [{ index: 0, delta: {}, finish_reason: 'stop' }],
|
||||
};
|
||||
yield {
|
||||
choices: [],
|
||||
usage: { prompt_tokens: 10, completion_tokens: 1 },
|
||||
};
|
||||
})(),
|
||||
usage_calculator: (source) => {
|
||||
servedTier = source.service_tier;
|
||||
return source.usage;
|
||||
},
|
||||
});
|
||||
const chatStream = new AIChatStream({
|
||||
stream: new Writable({
|
||||
write(_chunk, _encoding, callback) {
|
||||
callback();
|
||||
},
|
||||
}),
|
||||
});
|
||||
await result.init_chat_stream!({ chatStream });
|
||||
expect(servedTier).toBe('priority');
|
||||
});
|
||||
|
||||
it('uses the final Responses tier when the provider downgrades a stream', async () => {
|
||||
let servedTier: string | undefined;
|
||||
const result = await handle_completion_output_responses_api({
|
||||
stream: true,
|
||||
completion: (async function* () {
|
||||
yield {
|
||||
type: 'response.created',
|
||||
response: { service_tier: 'ultrafast' },
|
||||
};
|
||||
yield { type: 'response.output_text.delta', delta: 'OK' };
|
||||
yield {
|
||||
type: 'response.completed',
|
||||
response: {
|
||||
service_tier: 'default',
|
||||
usage: { input_tokens: 10, output_tokens: 1 },
|
||||
},
|
||||
};
|
||||
})(),
|
||||
usage_calculator: (source) => {
|
||||
servedTier = source.service_tier;
|
||||
return source.usage;
|
||||
},
|
||||
});
|
||||
const chatStream = new AIChatStream({
|
||||
stream: new Writable({
|
||||
write(_chunk, _encoding, callback) {
|
||||
callback();
|
||||
},
|
||||
}),
|
||||
});
|
||||
await result.init_chat_stream!({ chatStream });
|
||||
expect(servedTier).toBe('default');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,261 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { requiredInputModalities } from './mediaParts.js';
|
||||
import type {
|
||||
IChatModel,
|
||||
ICompleteArguments,
|
||||
ModelCost,
|
||||
ServiceSpeed,
|
||||
} from '../types.js';
|
||||
|
||||
export const lowerServiceSpeed = (speed: ServiceSpeed): ServiceSpeed =>
|
||||
speed === 'ultrafast' ? 'fast' : 'standard';
|
||||
|
||||
const tokenRates = (costs: ModelCost, multiplier: number): ModelCost =>
|
||||
Object.fromEntries(
|
||||
Object.entries(costs).map(([key, value]) => [
|
||||
key,
|
||||
key !== 'tokens' &&
|
||||
/token|^prompt$|^completion$|^input_cache_/.test(key)
|
||||
? value * multiplier
|
||||
: value,
|
||||
]),
|
||||
);
|
||||
|
||||
export const serviceSpeedCosts = (
|
||||
model: IChatModel,
|
||||
speed: 'fast' | 'ultrafast',
|
||||
): ModelCost | undefined => {
|
||||
if (model.serviceSpeeds && !model.serviceSpeeds.includes(speed))
|
||||
return undefined;
|
||||
const rates = model.serviceSpeedRates?.[speed];
|
||||
return typeof rates === 'number'
|
||||
? tokenRates(model.costs, rates)
|
||||
: rates
|
||||
? { ...model.costs, ...rates }
|
||||
: undefined;
|
||||
};
|
||||
|
||||
export interface ServiceSpeedSelection {
|
||||
model: IChatModel;
|
||||
args: ICompleteArguments;
|
||||
speed: ServiceSpeed;
|
||||
}
|
||||
|
||||
/** Select only an equivalent model from the same provider's loaded catalog. */
|
||||
export const selectServiceSpeed = (
|
||||
model: IChatModel,
|
||||
args: ICompleteArguments,
|
||||
findVariant: (id: string) => IChatModel | undefined,
|
||||
offers: Partial<Record<'fast' | 'ultrafast', IChatModel>> = {},
|
||||
): ServiceSpeedSelection => {
|
||||
if (args.serviceSpeed === undefined)
|
||||
return { model, args, speed: 'standard' };
|
||||
const provider = model.provider;
|
||||
const fast =
|
||||
offers.fast?.costs ??
|
||||
(provider === 'gemini' &&
|
||||
(requiredInputModalities(args.messages).some(
|
||||
(modality) => !['text', 'image'].includes(modality),
|
||||
) ||
|
||||
args.messages.some(
|
||||
(message) =>
|
||||
Array.isArray(message.content) &&
|
||||
message.content.some(
|
||||
(part: Record<string, unknown>) =>
|
||||
part &&
|
||||
[
|
||||
'input_audio',
|
||||
'audio_url',
|
||||
'document',
|
||||
'file',
|
||||
'input_file',
|
||||
].includes(String(part.type)),
|
||||
),
|
||||
))
|
||||
? undefined
|
||||
: serviceSpeedCosts(model, 'fast'));
|
||||
const ultra =
|
||||
offers.ultrafast?.costs ?? serviceSpeedCosts(model, 'ultrafast');
|
||||
let speed = args.serviceSpeed;
|
||||
const variantId = model.serviceSpeedVariants?.fast;
|
||||
const variant =
|
||||
offers.fast && offers.fast.id !== model.id
|
||||
? offers.fast
|
||||
: variantId
|
||||
? findVariant(variantId)
|
||||
: undefined;
|
||||
while (speed !== 'standard') {
|
||||
if (speed === 'ultrafast' && ultra) break;
|
||||
if (speed === 'fast' && (fast || variant)) break;
|
||||
speed = lowerServiceSpeed(speed);
|
||||
}
|
||||
const request: ICompleteArguments = {
|
||||
...args,
|
||||
serviceSpeed: speed,
|
||||
speed: undefined,
|
||||
service_tier: undefined,
|
||||
};
|
||||
if (speed === 'fast' && variant) {
|
||||
return {
|
||||
model: { ...variant, release_date: model.release_date },
|
||||
args: request,
|
||||
speed,
|
||||
};
|
||||
}
|
||||
if (provider === 'gemini' && fast)
|
||||
request.service_tier = speed === 'fast' ? 'priority' : 'standard';
|
||||
else if (provider === 'claude')
|
||||
request.speed = speed === 'fast' ? 'fast' : 'standard';
|
||||
else if (provider === 'mistral')
|
||||
request.service_tier = speed === 'fast' ? 'auto' : 'default';
|
||||
else if (
|
||||
provider === 'minimax' &&
|
||||
model.serviceSpeedRates?.fast !== undefined
|
||||
)
|
||||
request.service_tier = speed === 'fast' ? 'priority' : 'standard';
|
||||
else if (fast || ultra)
|
||||
request.service_tier =
|
||||
speed === 'ultrafast'
|
||||
? 'ultrafast'
|
||||
: speed === 'fast'
|
||||
? 'priority'
|
||||
: 'default';
|
||||
if (
|
||||
speed === 'standard' &&
|
||||
[
|
||||
'openai-completion',
|
||||
'openai-responses',
|
||||
'azure-openai',
|
||||
'azure-openai-responses',
|
||||
'openrouter',
|
||||
].includes(provider ?? '')
|
||||
)
|
||||
request.service_tier = 'default';
|
||||
const selectedCosts =
|
||||
speed === 'ultrafast' ? ultra : speed === 'fast' ? fast : undefined;
|
||||
return {
|
||||
args: request,
|
||||
speed,
|
||||
model: selectedCosts
|
||||
? {
|
||||
...model,
|
||||
costs: selectedCosts,
|
||||
pricingOverrides: model.pricingOverrides?.map((override) => ({
|
||||
...override,
|
||||
costs: Object.fromEntries(
|
||||
Object.entries(override.costs).map(([key, value]) => [
|
||||
key,
|
||||
model.costs[key] > 0
|
||||
? (value *
|
||||
(selectedCosts[key] ??
|
||||
model.costs[key])) /
|
||||
model.costs[key]
|
||||
: value,
|
||||
]),
|
||||
),
|
||||
})),
|
||||
serviceSpeedPricing: {
|
||||
speed,
|
||||
standardCosts: model.costs,
|
||||
standardOverrides: model.pricingOverrides,
|
||||
fastCosts: fast,
|
||||
ultrafastCosts: ultra,
|
||||
},
|
||||
}
|
||||
: model,
|
||||
};
|
||||
};
|
||||
|
||||
/** The response is authoritative; an absent tier never earns a premium. */
|
||||
export const servedServiceSpeed = (
|
||||
source: Record<string, unknown>,
|
||||
): ServiceSpeed => {
|
||||
const usage = source.usage as Record<string, unknown> | undefined;
|
||||
const tier =
|
||||
usage?.speed ??
|
||||
usage?.serviceTier ??
|
||||
usage?.service_tier ??
|
||||
source.service_tier;
|
||||
return tier === 'ultrafast'
|
||||
? 'ultrafast'
|
||||
: tier === 'fast' || tier === 'priority'
|
||||
? 'fast'
|
||||
: 'standard';
|
||||
};
|
||||
|
||||
export const serviceSpeedBillingModel = (
|
||||
model: IChatModel,
|
||||
speed: ServiceSpeed,
|
||||
): IChatModel => {
|
||||
const pricing = model.serviceSpeedPricing;
|
||||
if (!pricing || pricing.speed === speed) return model;
|
||||
const costs =
|
||||
speed === 'fast'
|
||||
? (pricing.fastCosts ?? pricing.standardCosts)
|
||||
: pricing.standardCosts;
|
||||
return {
|
||||
...model,
|
||||
costs,
|
||||
pricingOverrides: pricing.standardOverrides?.map((override) => ({
|
||||
...override,
|
||||
costs: Object.fromEntries(
|
||||
Object.entries(override.costs).map(([key, value]) => [
|
||||
key,
|
||||
pricing.standardCosts[key] > 0
|
||||
? (value * (costs[key] ?? pricing.standardCosts[key])) /
|
||||
pricing.standardCosts[key]
|
||||
: value,
|
||||
]),
|
||||
),
|
||||
})),
|
||||
};
|
||||
};
|
||||
|
||||
/** Retry only failures that explicitly concern admission to the faster tier. */
|
||||
export const isServiceSpeedUnavailable = (
|
||||
error: unknown,
|
||||
variant = false,
|
||||
): boolean => {
|
||||
const e = error as {
|
||||
status?: number;
|
||||
statusCode?: number;
|
||||
message?: unknown;
|
||||
error?: { message?: unknown };
|
||||
};
|
||||
const status = e?.status ?? e?.statusCode;
|
||||
const message = e?.error?.message ?? e?.message ?? '';
|
||||
if (typeof message !== 'string') return false;
|
||||
if (
|
||||
variant &&
|
||||
[400, 404].includes(status ?? 0) &&
|
||||
/model.*(?:not found|not exist|unavailable)/i.test(message)
|
||||
)
|
||||
return true;
|
||||
return (
|
||||
[400, 403, 429, 503].includes(status ?? 0) &&
|
||||
/fast mode|ultrafast|service[_ ]?tier|priority (?:tier|processing|capacity)|speed parameter/i.test(
|
||||
message,
|
||||
) &&
|
||||
/unavailable|unsupported|not support|rate.limit|quota|capacity|not enabled|not allowed|access|exceed|invalid value/i.test(
|
||||
message,
|
||||
)
|
||||
);
|
||||
};
|
||||
+108
-1
@@ -352,6 +352,10 @@ describe('PuterServer host header validation', () => {
|
||||
expect(res.body).toContain('/dist/apps-embed.min.js');
|
||||
expect(res.body).not.toContain('/dist/bundle.min.js');
|
||||
expect(res.body).toContain(`http://api.puter.localhost:${port}`);
|
||||
const config = JSON.parse(
|
||||
res.body.match(/id="app-browser-config">(.*?)<\/script>/)[1],
|
||||
);
|
||||
expect(Array.isArray(config.guestApps)).toBe(true);
|
||||
}
|
||||
const gui = await request('/embed/apps', {
|
||||
host: `puter.localhost:${port}`,
|
||||
@@ -364,12 +368,115 @@ describe('PuterServer host header validation', () => {
|
||||
expect(api.status).toBe(404);
|
||||
});
|
||||
|
||||
it('lets a deployment subdomain frame the feedback embed and names it', async () => {
|
||||
const parent = `http://spreadsheet.puter.localhost:${port}`;
|
||||
const res = await request('/embed/feedback', {
|
||||
host: `puter.localhost:${port}`,
|
||||
referer: `${parent}/sheet/1`,
|
||||
});
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.headers['x-frame-options']).toBeUndefined();
|
||||
expect(res.headers['content-security-policy']).toBe(
|
||||
`frame-ancestors 'self' ${parent}`,
|
||||
);
|
||||
expect(res.headers['cache-control']).toBe('no-store');
|
||||
expect(res.body).toContain('/dist/feedback-embed.min.js');
|
||||
expect(res.body).not.toContain('/dist/bundle.min.js');
|
||||
const config = JSON.parse(
|
||||
res.body.match(/id="feedback-embed-config">(.*?)<\/script>/)[1],
|
||||
);
|
||||
expect(config).toEqual({
|
||||
apiOrigin: `http://api.puter.localhost:${port}`,
|
||||
hostOrigin: parent,
|
||||
});
|
||||
|
||||
const hosted = await request('/embed/feedback', {
|
||||
host: `puter.localhost:${port}`,
|
||||
referer: `http://someone.site.puter.localhost:${port}/`,
|
||||
});
|
||||
expect(hosted.headers['content-security-policy']).toBe("frame-ancestors 'self'");
|
||||
expect(
|
||||
JSON.parse(
|
||||
hosted.body.match(/id="feedback-embed-config">(.*?)<\/script>/)[1],
|
||||
).hostOrigin,
|
||||
).toBeNull();
|
||||
|
||||
const redirected = await request('/embed/feedback?puter.api_origin=https://other.test', {
|
||||
host: `puter.localhost:${port}`,
|
||||
});
|
||||
expect(redirected.status).toBe(302);
|
||||
expect(redirected.headers.location).toBe('/embed/feedback');
|
||||
});
|
||||
|
||||
it.each([
|
||||
'sign-in',
|
||||
'show-open-file-picker',
|
||||
'show-save-file-picker',
|
||||
'show-directory-picker',
|
||||
'upgrade',
|
||||
'request-email-confirmation',
|
||||
])('lets a deployment subdomain frame the %s popup and names it to the GUI', async (action) => {
|
||||
const parent = `http://spreadsheet.puter.localhost:${port}`;
|
||||
const guiParams = (body: string) =>
|
||||
JSON.parse(body.match(/gui\((\{.*\})\);/)[1]);
|
||||
const res = await request(`/embed/${action}`, {
|
||||
host: `puter.localhost:${port}`,
|
||||
referer: `${parent}/sheet/1`,
|
||||
});
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.headers['x-frame-options']).toBeUndefined();
|
||||
expect(res.headers['content-security-policy']).toBe(
|
||||
`frame-ancestors 'self' ${parent}`,
|
||||
);
|
||||
expect(res.headers['cache-control']).toBe('no-store');
|
||||
expect(guiParams(res.body).embed_host_origin).toBe(parent);
|
||||
|
||||
const hosted = await request(`/embed/${action}`, {
|
||||
host: `puter.localhost:${port}`,
|
||||
referer: `http://someone.site.puter.localhost:${port}/`,
|
||||
});
|
||||
expect(hosted.headers['content-security-policy']).toBe("frame-ancestors 'self'");
|
||||
expect(guiParams(hosted.body).embed_host_origin).toBeNull();
|
||||
|
||||
const redirected = await request(`/embed/${action}?auth_token=x&embedded_in_popup=true`, {
|
||||
host: `puter.localhost:${port}`,
|
||||
referer: `${parent}/sheet/1`,
|
||||
});
|
||||
expect(redirected.status).toBe(302);
|
||||
expect(redirected.headers.location).toBe(`/embed/${action}`);
|
||||
expect(redirected.headers['x-frame-options']).toBe('SAMEORIGIN');
|
||||
});
|
||||
|
||||
it.each(['/embed/SIGN-IN', '/embed/Show-Open-File-Picker', '/embed/sign-in/', '/embed/Upgrade', '/embed/upgrade/', '/embed/Request-Email-Confirmation'])(
|
||||
'lets a first-party page frame only the exact spelling of %s',
|
||||
async (spelling) => {
|
||||
const res = await request(spelling, {
|
||||
host: `puter.localhost:${port}`,
|
||||
referer: `http://spreadsheet.puter.localhost:${port}/`,
|
||||
});
|
||||
expect(res.status).toBe(302);
|
||||
expect(res.headers.location).toBe(spelling.toLowerCase().replace(/\/$/, ''));
|
||||
expect(res.headers['x-frame-options']).toBe('SAMEORIGIN');
|
||||
expect(res.headers['content-security-policy']).toBeUndefined();
|
||||
},
|
||||
);
|
||||
|
||||
it('names no framing page to the popups it serves in a window', async () => {
|
||||
const res = await request('/action/sign-in?embedded_in_popup=true', {
|
||||
host: `puter.localhost:${port}`,
|
||||
referer: `http://spreadsheet.puter.localhost:${port}/`,
|
||||
});
|
||||
expect(res.headers['x-frame-options']).toBe('SAMEORIGIN');
|
||||
expect(res.headers['content-security-policy']).toBeUndefined();
|
||||
expect(res.body).not.toContain('embed_host_origin');
|
||||
});
|
||||
|
||||
it.each([
|
||||
['a user-hosted site', 'http://someone.site.puter.localhost:PORT/'],
|
||||
['a private app host', 'http://someone.app.puter.localhost:PORT/'],
|
||||
['the API host', 'http://api.puter.localhost:PORT/'],
|
||||
['the WebDAV host', 'http://dav.puter.localhost:PORT/'],
|
||||
['a native app host', 'http://editor.puter.localhost:PORT/'],
|
||||
['a native app host', 'http://about.puter.localhost:PORT/'],
|
||||
['a nested subdomain', 'http://a.spreadsheet.puter.localhost:PORT/'],
|
||||
['another site', 'http://evil.test:PORT/'],
|
||||
['another port', 'http://spreadsheet.puter.localhost:1/'],
|
||||
|
||||
@@ -27,17 +27,16 @@ import { PuterService } from '../types.js';
|
||||
*/
|
||||
export const RECOMMENDED_APP_NAMES = [
|
||||
'builder',
|
||||
'word-processor',
|
||||
'spreadsheet',
|
||||
'presentation',
|
||||
'whiteboard',
|
||||
'pdf-editor',
|
||||
'contacts',
|
||||
'calendar',
|
||||
'meetings',
|
||||
'teamchat',
|
||||
'email',
|
||||
'whiteboard',
|
||||
'spreadsheet',
|
||||
'word-processor',
|
||||
'presentation',
|
||||
'pdf-editor',
|
||||
'terminal',
|
||||
'dev-center',
|
||||
'cap-table',
|
||||
'invoices',
|
||||
|
||||
@@ -144,11 +144,22 @@ const createInline = async (
|
||||
return String(response.body.subId);
|
||||
};
|
||||
|
||||
const delivered = (count = 1): Promise<void> =>
|
||||
vi.waitFor(() => expect(sinkPosts.length).toBeGreaterThanOrEqual(count), {
|
||||
timeout: 30_000,
|
||||
interval: 50,
|
||||
});
|
||||
/**
|
||||
* The post each label made, in order, once all have arrived. Matched by label
|
||||
* since earlier tests' subscriptions stay live and post to the same sink.
|
||||
*/
|
||||
const delivered = async (...labels: string[]): Promise<SinkPost[]> => {
|
||||
await vi.waitFor(
|
||||
() =>
|
||||
expect(sinkPosts.map((post) => post.label)).toEqual(
|
||||
expect.arrayContaining(labels),
|
||||
),
|
||||
{ timeout: 30_000, interval: 50 },
|
||||
);
|
||||
return labels.map((label) =>
|
||||
sinkPosts.find((post) => post.label === label)!,
|
||||
);
|
||||
};
|
||||
|
||||
beforeAll(async () => {
|
||||
env = await setupPuterTestEnv({
|
||||
@@ -255,14 +266,11 @@ describe('an inline handler in a user scope, end to end', () => {
|
||||
await env.server.services.fs.touch(holder.id, {
|
||||
path: `${holder.anchor}/note.txt`,
|
||||
});
|
||||
await delivered();
|
||||
const [post] = await delivered('inline');
|
||||
|
||||
expect(sinkPosts[0]).toMatchObject({
|
||||
label: 'inline',
|
||||
path: `${holder.anchor}/note.txt`,
|
||||
});
|
||||
expect(post.path).toBe(`${holder.anchor}/note.txt`);
|
||||
const { actor } = await env.server.services.auth.authenticate(
|
||||
sinkPosts[0].token!,
|
||||
post.token!,
|
||||
);
|
||||
expect(actor!.user?.id).toBe(holder.id);
|
||||
expect(actor!.effectiveApp?.uid).toBe(appUid);
|
||||
@@ -297,14 +305,14 @@ describe('an inline handler in a user scope, end to end', () => {
|
||||
await env.server.services.fs.touch(holder.id, {
|
||||
path: `${holder.anchor}/shared-${i}.txt`,
|
||||
});
|
||||
await delivered(2);
|
||||
const runAs = new Set<number>();
|
||||
for (const post of sinkPosts)
|
||||
runAs.add(
|
||||
(await env.server.services.auth.authenticate(post.token!))
|
||||
.actor!.user!.id,
|
||||
// The first test's 'inline' handler also posts for holders[0].
|
||||
const posts = await delivered('first', 'second');
|
||||
for (const [i, post] of posts.entries()) {
|
||||
const { actor } = await env.server.services.auth.authenticate(
|
||||
post.token!,
|
||||
);
|
||||
expect(runAs).toEqual(new Set(holders.map((h) => h.id)));
|
||||
expect(actor!.user!.id).toBe(holders[i].id);
|
||||
}
|
||||
create.mockRestore();
|
||||
},
|
||||
);
|
||||
|
||||
@@ -63,7 +63,7 @@ describe('app browser embed shell', () => {
|
||||
const html = service.renderAppBrowserEmbed();
|
||||
const configJson = html.match(/id="app-browser-config">(.*?)<\/script>/)![1];
|
||||
expect(JSON.parse(configJson)).toEqual({
|
||||
apiOrigin: 'https://api.deployment.test', sdkUrl,
|
||||
apiOrigin: 'https://api.deployment.test', sdkUrl, guestApps: [],
|
||||
});
|
||||
expect(html).toContain('/dist/apps-embed.min.js');
|
||||
expect(html).not.toContain('/dist/bundle.min.js');
|
||||
@@ -95,6 +95,41 @@ describe('app browser embed shell', () => {
|
||||
const configJson = html.match(/id="app-browser-config">(.*?)<\/script>/)![1];
|
||||
expect(JSON.parse(configJson).sdkUrl).toBe(sdkUrl);
|
||||
});
|
||||
|
||||
it('carries the apps shown without a session, escaped', () => {
|
||||
const guestApps = [
|
||||
{ name: 'notes', title: '</script><script>unexpected()</script>' },
|
||||
];
|
||||
const html = makeService({}).renderAppBrowserEmbed({ guestApps });
|
||||
expect(html).not.toContain('<script>unexpected()');
|
||||
const configJson = html.match(/id="app-browser-config">(.*?)<\/script>/)![1];
|
||||
expect(JSON.parse(configJson).guestApps).toEqual(guestApps);
|
||||
});
|
||||
});
|
||||
|
||||
describe('feedback embed shell', () => {
|
||||
it('carries the framing page and API origin, and loads only its own script', () => {
|
||||
const html = makeService({
|
||||
api_base_url: 'https://api.deployment.test',
|
||||
gui_bundle: '/static/bundle.min.js?v=42',
|
||||
}).renderFeedbackEmbed({ hostOrigin: 'https://notes.deployment.test' });
|
||||
const configJson = html.match(/id="feedback-embed-config">(.*?)<\/script>/)![1];
|
||||
expect(JSON.parse(configJson)).toEqual({
|
||||
apiOrigin: 'https://api.deployment.test',
|
||||
hostOrigin: 'https://notes.deployment.test',
|
||||
});
|
||||
expect(html).toContain('<script defer src="/static/feedback-embed.min.js?v=42"></script>');
|
||||
expect(html).toContain('<link rel="stylesheet" href="/css/feedback-embed.css">');
|
||||
expect(html).toContain('<main id="feedback-embed"></main>');
|
||||
expect(html).not.toContain('bundle.min.js');
|
||||
expect(html).not.toContain('apps-embed');
|
||||
});
|
||||
|
||||
it('carries no framing page when none is allowed', () => {
|
||||
const html = makeService({}).renderFeedbackEmbed({ hostOrigin: null });
|
||||
const configJson = html.match(/id="feedback-embed-config">(.*?)<\/script>/)![1];
|
||||
expect(JSON.parse(configJson).hostOrigin).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
/** Capture what the service sends, and return the rendered HTML. */
|
||||
|
||||
@@ -48,6 +48,15 @@ export interface LaunchOptions {
|
||||
on_initialized?: Array<Record<string, unknown>>;
|
||||
}
|
||||
|
||||
/**
|
||||
* A GUI popup a first-party page frames at `/embed/<action>`. `hostOrigin` is
|
||||
* that page, read from the referrer and pinned with frame-ancestors; null when
|
||||
* no allowed page frames it.
|
||||
*/
|
||||
export interface PopupFrame {
|
||||
hostOrigin: string | null;
|
||||
}
|
||||
|
||||
interface PuterGuiAddonsEvent {
|
||||
req: Request;
|
||||
path: string;
|
||||
@@ -167,15 +176,54 @@ export class PuterHomepageService extends PuterService {
|
||||
this.#guiParams[key] = val;
|
||||
}
|
||||
|
||||
renderAppBrowserEmbed(): string {
|
||||
const configJson = JSON.stringify({
|
||||
apiOrigin: this.config.api_base_url,
|
||||
sdkUrl:
|
||||
this.config.env === 'dev'
|
||||
? '/sdk/puter.dev.js'
|
||||
: (this.config.gui_puterjs_bundle ??
|
||||
'https://js.puter.com/v2/'),
|
||||
}).replace(/</g, '\\u003c');
|
||||
/** `guestApps` is what the embed shows a visitor without a session. */
|
||||
renderAppBrowserEmbed({
|
||||
guestApps = [],
|
||||
}: { guestApps?: Array<Record<string, unknown>> } = {}): string {
|
||||
return this.#renderEmbed({
|
||||
rootId: 'app-browser',
|
||||
rootClass: 'dashboard',
|
||||
stylesheet: 'apps-embed.css',
|
||||
script: 'apps-embed.min.js',
|
||||
config: {
|
||||
apiOrigin: this.config.api_base_url,
|
||||
sdkUrl:
|
||||
this.config.env === 'dev'
|
||||
? '/sdk/puter.dev.js'
|
||||
: (this.config.gui_puterjs_bundle ??
|
||||
'https://js.puter.com/v2/'),
|
||||
guestApps,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* `hostOrigin` is the first-party page framing the dialog, which the
|
||||
* feedback is sent for; null when no allowed page frames it.
|
||||
*/
|
||||
renderFeedbackEmbed({ hostOrigin }: { hostOrigin: string | null }): string {
|
||||
return this.#renderEmbed({
|
||||
rootId: 'feedback-embed',
|
||||
stylesheet: 'feedback-embed.css',
|
||||
script: 'feedback-embed.min.js',
|
||||
config: { apiOrigin: this.config.api_base_url, hostOrigin },
|
||||
});
|
||||
}
|
||||
|
||||
#renderEmbed({
|
||||
rootId,
|
||||
rootClass,
|
||||
stylesheet,
|
||||
script,
|
||||
config,
|
||||
}: {
|
||||
rootId: string;
|
||||
rootClass?: string;
|
||||
stylesheet: string;
|
||||
script: string;
|
||||
config: Record<string, unknown>;
|
||||
}): string {
|
||||
const configJson = JSON.stringify(config).replace(/</g, '\\u003c');
|
||||
|
||||
return `<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
@@ -186,27 +234,27 @@ export class PuterHomepageService extends PuterService {
|
||||
<title>${encode(String(this.config.gui_params?.title ?? 'Puter'))}</title>
|
||||
<link rel="stylesheet" href="/css/normalize.css">
|
||||
<link rel="stylesheet" href="/css/dashboard.css">
|
||||
<link rel="stylesheet" href="/css/apps-embed.css">
|
||||
<link rel="stylesheet" href="/css/${stylesheet}">
|
||||
</head>
|
||||
<body>
|
||||
<main id="app-browser" class="dashboard"></main>
|
||||
<script type="application/json" id="app-browser-config">${configJson}</script>
|
||||
<script defer src="${encode(this.#appBrowserEmbedScript())}"></script>
|
||||
<main id="${rootId}"${rootClass ? ` class="${rootClass}"` : ''}></main>
|
||||
<script type="application/json" id="${rootId}-config">${configJson}</script>
|
||||
<script defer src="${encode(this.#embedScript(script))}"></script>
|
||||
</body>
|
||||
</html>`;
|
||||
}
|
||||
|
||||
/**
|
||||
* The embed's script is built beside the GUI bundle, so it is served from
|
||||
* Embed scripts are built beside the GUI bundle, so they are served from
|
||||
* wherever `gui_bundle` is (a CDN in production), with the same query.
|
||||
*/
|
||||
#appBrowserEmbedScript(): string {
|
||||
#embedScript(name: string): string {
|
||||
const base = 'http://relative.invalid';
|
||||
const bundle = new URL(
|
||||
this.config.gui_bundle ?? '/dist/bundle.min.js',
|
||||
base,
|
||||
);
|
||||
const script = new URL('apps-embed.min.js', bundle);
|
||||
const script = new URL(name, bundle);
|
||||
script.search = bundle.search;
|
||||
return script.origin === base
|
||||
? `${script.pathname}${script.search}`
|
||||
@@ -264,6 +312,7 @@ export class PuterHomepageService extends PuterService {
|
||||
ctx: { req: Request; res: Response; actor?: Actor | null },
|
||||
meta: PageMeta,
|
||||
launchOptions: LaunchOptions = {},
|
||||
popupFrame?: PopupFrame,
|
||||
): Promise<void> {
|
||||
const { req, res, actor } = ctx;
|
||||
|
||||
@@ -288,6 +337,7 @@ export class PuterHomepageService extends PuterService {
|
||||
{ req, actor: actor ?? null },
|
||||
meta,
|
||||
launchOptions,
|
||||
popupFrame,
|
||||
);
|
||||
res.send(html);
|
||||
}
|
||||
@@ -298,6 +348,7 @@ export class PuterHomepageService extends PuterService {
|
||||
ctx: { req: Request; actor: Actor | null },
|
||||
meta: PageMeta,
|
||||
launchOptions: LaunchOptions,
|
||||
popupFrame?: PopupFrame,
|
||||
): Promise<string> {
|
||||
const { req, actor } = ctx;
|
||||
const env = this.config.env ?? 'prod';
|
||||
@@ -343,6 +394,8 @@ export class PuterHomepageService extends PuterService {
|
||||
eventsNotifications: notificationsFoldInEnabled(this.config),
|
||||
...meta,
|
||||
launch_options: launchOptions,
|
||||
// After `meta`, so nothing page-specific can name the opener.
|
||||
...(popupFrame ? { embed_host_origin: popupFrame.hostOrigin } : {}),
|
||||
};
|
||||
|
||||
const event: PuterGuiAddonsEvent = {
|
||||
|
||||
@@ -78,7 +78,10 @@ describe('LocalWorkerService shutdown disposal', () => {
|
||||
disposals[0].mockImplementationOnce(async () => {
|
||||
await held;
|
||||
});
|
||||
vi.useFakeTimers();
|
||||
vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout'] });
|
||||
// The test server keeps arming timers from real callbacks while fakes
|
||||
// are installed, so only what shutdown arms synchronously is checked.
|
||||
const arm = vi.spyOn(globalThis, 'setTimeout');
|
||||
try {
|
||||
let finished = false;
|
||||
const draining = Promise.resolve(service.onServerShutdown()).then(
|
||||
@@ -86,12 +89,13 @@ describe('LocalWorkerService shutdown disposal', () => {
|
||||
finished = true;
|
||||
},
|
||||
);
|
||||
expect(arm.mock.calls.map(([, ms]) => ms)).toEqual([5_000]);
|
||||
await vi.advanceTimersByTimeAsync(4_999);
|
||||
expect(finished).toBe(false);
|
||||
await vi.advanceTimersByTimeAsync(1);
|
||||
await draining;
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
} finally {
|
||||
arm.mockRestore();
|
||||
release();
|
||||
vi.useRealTimers();
|
||||
}
|
||||
|
||||
@@ -2860,8 +2860,12 @@ describe('OpenAIRealtimeRelayService relay', () => {
|
||||
),
|
||||
).toHaveLength(64),
|
||||
);
|
||||
// Holds are reserved and usage recorded one at a time, which a loaded
|
||||
// runner takes past vi.waitFor's default second.
|
||||
const settled = (check: () => Promise<void> | void) =>
|
||||
vi.waitFor(check, { timeout: 5_000 });
|
||||
// The pool grew one item per commit, with nothing spare at the cap.
|
||||
await vi.waitFor(async () =>
|
||||
await settled(async () =>
|
||||
expect(
|
||||
await server.services.metering.getOutstandingHolds(actor),
|
||||
).toBe(64 * 10_896_000),
|
||||
@@ -2890,7 +2894,7 @@ describe('OpenAIRealtimeRelayService relay', () => {
|
||||
for (let batch = 0; batch < 8; batch++) {
|
||||
for (let i = Math.max(1, batch * 8); i < (batch + 1) * 8; i++)
|
||||
complete(i);
|
||||
await vi.waitFor(() =>
|
||||
await settled(() =>
|
||||
expect(
|
||||
recorded.mock.calls
|
||||
.flatMap(([, rows]) => rows)
|
||||
@@ -2906,13 +2910,13 @@ describe('OpenAIRealtimeRelayService relay', () => {
|
||||
.filter((row) => row.usageType.includes('transcription_'))
|
||||
.reduce((sum, row) => sum + row.costOverride!, 0),
|
||||
).toBe(640);
|
||||
await vi.waitFor(async () =>
|
||||
await settled(async () =>
|
||||
expect(
|
||||
await server.services.metering.getOutstandingHolds(actor),
|
||||
).toBe(2 * 10_896_000),
|
||||
);
|
||||
c.ws.close();
|
||||
await vi.waitFor(async () =>
|
||||
await settled(async () =>
|
||||
expect(
|
||||
await server.services.metering.getOutstandingHolds(actor),
|
||||
).toBe(0),
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
# OpenAI Realtime relay
|
||||
|
||||
The relay serves `/puterai/openai/v1/realtime` on the API host. The SDK exposes
|
||||
it through [`puter.ai.convo()`](../../../docs/src/AI/convo.md).
|
||||
it through [`puter.ai.realtime()`](../../../docs/src/AI/realtime.md).
|
||||
|
||||
**Beta:** This API is incomplete. Its methods, options, events, and behavior may change abruptly, including breaking changes without notice.
|
||||
|
||||
Direct clients are documented in the [Realtime WebSocket API reference](../../../docs/src/AI/realtime.md).
|
||||
Direct clients are documented in the [Realtime WebSocket API reference](../../../docs/src/AI/realtime-websocket.md).
|
||||
|
||||
## Transport
|
||||
|
||||
@@ -27,7 +27,7 @@ The socket opens before authentication completes. Authentication and model
|
||||
failures arrive as OpenAI-shaped error events followed by a close frame.
|
||||
|
||||
```js
|
||||
const conversation = await puter.ai.convo({ output: ['text'] });
|
||||
const conversation = await puter.ai.realtime({ output: ['text'] });
|
||||
conversation.on('error', (event) => console.error(event));
|
||||
conversation.on('text', ({ delta }) => console.log(delta));
|
||||
conversation.on('turn.done', () => conversation.close());
|
||||
|
||||
@@ -1542,18 +1542,22 @@ describe('AppStore detailed stats', () => {
|
||||
let db;
|
||||
|
||||
const APP_UID = `app-stats-${Date.now()}`;
|
||||
const nowSeconds = Math.floor(Date.now() / 1000);
|
||||
// The windows come from the clock, so it is pinned: mid-week and
|
||||
// mid-month, and minutes clear of midnight in any time zone.
|
||||
const NOW = new Date('2026-06-17T12:07:30Z');
|
||||
const nowSeconds = Math.floor(NOW.getTime() / 1000);
|
||||
|
||||
beforeAll(async () => {
|
||||
server = await setupTestServer();
|
||||
appStore = server.stores.app;
|
||||
db = server.clients.db;
|
||||
await clearAppCache(server.clients.redis);
|
||||
vi.useFakeTimers({ toFake: ['Date'], now: NOW });
|
||||
|
||||
// Two users; the repeat shares a timestamp so it is always same-hour.
|
||||
// Two users in one hour. Windows end at now, exclusive.
|
||||
for (const [userId, offset] of [
|
||||
[1, 0],
|
||||
[1, 0],
|
||||
[1, 60],
|
||||
[1, 60],
|
||||
[2, 120],
|
||||
]) {
|
||||
await db.write(
|
||||
@@ -1564,6 +1568,7 @@ describe('AppStore detailed stats', () => {
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
vi.useRealTimers();
|
||||
await server?.shutdown();
|
||||
});
|
||||
|
||||
|
||||
@@ -154,6 +154,9 @@ export const setupTestServer = async (
|
||||
redis: { useMock: true },
|
||||
// Suites run system jobs themselves, not on the poll timer.
|
||||
jobs: { enabled: false },
|
||||
// The health loop's timed `SELECT 1` would land in tests that
|
||||
// count queries.
|
||||
server_health: { disabled_checks: ['database-liveness'] },
|
||||
s3: { localConfig: { inMemory: true } },
|
||||
no_default_user: true,
|
||||
no_devwatch: true,
|
||||
|
||||
@@ -1102,8 +1102,9 @@ interface IConfigOptional {
|
||||
/** Force the bundled GUI even in dev. Default: false. */
|
||||
use_bundled_gui: boolean;
|
||||
/**
|
||||
* Override the GUI bundle JS path. Default: `/dist/bundle.min.js`. The app
|
||||
* browser embed loads `apps-embed.min.js` from the same directory.
|
||||
* Override the GUI bundle JS path. Default: `/dist/bundle.min.js`. The
|
||||
* `/embed/*` pages load `apps-embed.min.js` and `feedback-embed.min.js`
|
||||
* from the same directory.
|
||||
*/
|
||||
gui_bundle: string;
|
||||
/** Override the GUI CSS path when bundled. Default: `/dist/bundle.min.css`. */
|
||||
|
||||
@@ -37,6 +37,9 @@ const repoRoot = path.resolve(backendDir, '../..');
|
||||
const isPgmockMode =
|
||||
(process.env.PUTER_TEST_DB_ENGINE ?? '').toLowerCase() === 'postgres';
|
||||
const pgmockTimeoutMs = 600_000;
|
||||
// Loaded CI runners stall timers well past the 5 s / 10 s defaults; a real
|
||||
// hang still fails, just later.
|
||||
const ciTimeouts = { testTimeout: 20_000, hookTimeout: 60_000 };
|
||||
|
||||
// Tests that always boot pgmock regardless of PUTER_TEST_DB_ENGINE. Too slow
|
||||
// for CI; run locally or via `npm run test:backend:postgres`.
|
||||
@@ -103,9 +106,11 @@ export const lowerDecoratorsPlugin = {
|
||||
loader: 'ts',
|
||||
target: 'es2024',
|
||||
sourcefile: id,
|
||||
sourcemap: 'inline',
|
||||
sourcemap: 'external',
|
||||
});
|
||||
return { code: result.code, map: null };
|
||||
// Without the map, coverage and stack traces report esbuild's
|
||||
// output lines instead of the source's.
|
||||
return { code: result.code, map: result.map };
|
||||
},
|
||||
};
|
||||
|
||||
@@ -145,7 +150,9 @@ export default defineConfig(({ mode }) => ({
|
||||
hookTimeout: pgmockTimeoutMs,
|
||||
fileParallelism: false,
|
||||
}
|
||||
: {}),
|
||||
: isCi
|
||||
? ciTimeouts
|
||||
: {}),
|
||||
coverage: {
|
||||
provider: 'v8',
|
||||
reporter: isCi
|
||||
|
||||
+13
-5
@@ -247,10 +247,10 @@ function generateTableOfContentsHTML (htmlContent, title) {
|
||||
html += '<nav class="toc-nav">';
|
||||
|
||||
const cleanTitle = removeTags(title);
|
||||
html += `<a href="#" class="toc-link toc-level-1" data-level="1">${cleanTitle}</a>`;
|
||||
html += `<a href="#" class="toc-link toc-level-1" data-level="1">${breakAfterDots(cleanTitle)}</a>`;
|
||||
|
||||
for ( const heading of headings ) {
|
||||
html += `<a href="#${heading.slug}" class="toc-link toc-level-${heading.level}" data-level="${heading.level}">${heading.text}</a>`;
|
||||
html += `<a href="#${heading.slug}" class="toc-link toc-level-${heading.level}" data-level="${heading.level}">${breakAfterDots(heading.text)}</a>`;
|
||||
}
|
||||
|
||||
html += '</nav>';
|
||||
@@ -457,10 +457,12 @@ function generateDocsHTML (filePath, rootDir, page, isIndex = false) {
|
||||
html += '</div>';
|
||||
// content
|
||||
html += `<div id="docs-content-${page.slug ?? ''}" class="docs-content col-xl-7 col-lg-7 col-md-12 col-sm-12 col-xs-12">`;
|
||||
// context menu
|
||||
// title, with the context menu beside it
|
||||
const pageTitle = breakAfterDots(page.page_title ?? page.title);
|
||||
html += '<div class="page-title-row">';
|
||||
html += `<h1>${page.icon ? `<img src="/${baseURL}${page.icon}" style="opacity:0.5; width: 24px; height: 24px; margin-right: 10px;">` : '' }${pageTitle}</h1>`;
|
||||
html += generateMenuHTML();
|
||||
|
||||
html += `<h1>${page.icon ? `<img src="/${baseURL}${page.icon}" style="opacity:0.5; width: 24px; height: 24px; margin-right: 10px;">` : '' }${page.page_title ?? page.title}</h1>`;
|
||||
html += '</div>';
|
||||
// Platform compatibility badges
|
||||
html += generatePlatformCompatibilityHTML(frontMatter);
|
||||
html += '<hr class="hr-inset">';
|
||||
@@ -852,6 +854,12 @@ function removeTags (html) {
|
||||
return html.replace(/<[^>]*>?/gm, '');
|
||||
}
|
||||
|
||||
// Let a long title wrap after a dot, so `puter.auth.getDetailedAppUsage()`
|
||||
// never splits inside the method name.
|
||||
function breakAfterDots (html) {
|
||||
return html.replace(/\.(?=\w)/g, '.<wbr>');
|
||||
}
|
||||
|
||||
// Replace '/path/to/root-directory' with the path to your root directory
|
||||
generateDocumentation('./src');
|
||||
generateRedirects();
|
||||
|
||||
+8
-8
@@ -194,18 +194,18 @@ Choose a model and compare provider rates in the [AI model directory](https://de
|
||||
These AI features are supported out of the box when using Puter.js:
|
||||
|
||||
- **[`puter.ai.chat()`](/AI/chat/)** - Chat with AI models like Claude, GPT, and others
|
||||
- **[`puter.ai.txt2img()`](/AI/txt2img/)** - Generate images from text descriptions
|
||||
- **[`puter.ai.txt2speech()`](/AI/txt2speech/)** - Convert text to speech
|
||||
- **[`puter.ai.speech2txt()`](/AI/speech2txt/)** - Transcribe audio recordings into text
|
||||
- **[`puter.ai.img2txt()`](/AI/img2txt/)** - Extract text from images (OCR)
|
||||
- **[`puter.ai.txt2vid()`](/AI/txt2vid/)** - Generate short video clips from text or a reference image with Wan, Seedance, Veo, Kling, MiniMax H3 and other models, or edit a video you supply with Gemini Omni, FLUX 3 or Kling Omni
|
||||
- **[`puter.ai.speech2speech()`](/AI/speech2speech/)** - Convert speech in one voice to another voice
|
||||
- **[`puter.ai.realtime()`](/AI/realtime/)** - Prepare a beta realtime conversation with optional text, microphone, playback, and interruption helpers
|
||||
- **[Realtime WebSocket API](/AI/realtime-websocket/)** - Connect directly without puter.js. This API is in beta and may change abruptly.
|
||||
- **[`puter.ai.listModels()`](/AI/listModels/)** - List available AI chat models (and providers) that Puter currently exposes.
|
||||
- **[`puter.ai.listModelProviders()`](/AI/listModelProviders/)** - List the AI providers that Puter currently exposes.
|
||||
- **[`puter.ai.txt2img()`](/AI/txt2img/)** - Generate images from text descriptions
|
||||
- **[`puter.ai.img2txt()`](/AI/img2txt/)** - Extract text from images (OCR)
|
||||
- **[`puter.ai.txt2speech()`](/AI/txt2speech/)** - Convert text to speech
|
||||
- **[`puter.ai.txt2speech.listEngines()`](/AI/txt2speech.listEngines/)** - List available TTS engines/models
|
||||
- **[`puter.ai.txt2speech.listVoices()`](/AI/txt2speech.listVoices/)** - List available TTS voices
|
||||
- **[`puter.ai.speech2speech()`](/AI/speech2speech/)** - Convert speech in one voice to another voice
|
||||
- **[`puter.ai.txt2vid()`](/AI/txt2vid/)** - Generate short video clips from text or a reference image with Wan, Seedance, Veo, Kling, MiniMax H3 and other models, or edit a video you supply with Gemini Omni, FLUX 3 or Kling Omni
|
||||
- **[`puter.ai.speech2txt()`](/AI/speech2txt/)** - Transcribe audio recordings into text
|
||||
- **[Realtime WebSocket API](/AI/realtime/)** - Connect directly without puter.js. This API is in beta and may change abruptly.
|
||||
- **[`puter.ai.convo()`](/AI/convo/)** - Prepare a beta realtime conversation with optional text, microphone, playback, and interruption helpers
|
||||
|
||||
## Examples
|
||||
|
||||
|
||||
+81
-18
@@ -29,24 +29,25 @@ An object containing the following properties:
|
||||
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `model` | String | The model you want to use for the completion. If not specified, defaults to `gpt-5.4-nano`. More than 500 models are available from providers including OpenAI, Anthropic, Google, Alibaba Cloud, xAI, Mistral, OpenRouter, Infron, and others. For a full list, see the [AI models list](https://developer.puter.com/ai/models/) page. |
|
||||
| `stream` | Boolean | A boolean indicating whether you want to stream the completion. Defaults to `false`. |
|
||||
| `max_tokens` | Number | The maximum number of tokens to generate in the completion. By default, the specific model's maximum is used. |
|
||||
| `temperature` | Number | A number between 0 and 2 indicating the randomness of the completion. Lower values make the output more focused and deterministic, while higher values make it more random. By default, the model's own temperature is used. See [Model-specific features](#model-specific-features). |
|
||||
| `reasoning_effort` / `reasoning.effort` | String | Controls how much effort reasoning models spend thinking. Supported values: `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Lower values give faster responses with less reasoning. OpenAI models, Moonshot's Kimi K3.1 Pro models, Meta's Muse Spark models, `gemini-4-argon`, `gemini-3.1-flash-lite` and `gemini-3-flash-preview` only. See [Model-specific features](#model-specific-features). |
|
||||
| `thinking` | Object \| null | Extended-thinking control: `{ type: "adaptive" \| "enabled" \| "disabled" \| "between_tools", budgetTokens?, display? }`. Pass `null` to send no thinking configuration at all (the model's own default); leaving it unset falls back to the legacy derivation from `reasoning_effort`. `budgetTokens` only applies to `type: "enabled"`. |
|
||||
| `verbosity` / `text.verbosity` | String | Controls how long or short responses are. Supported values: `low`, `medium`, and `high`. Lower values give shorter responses. OpenAI models only. |
|
||||
| `tools` | Array | Function definitions the AI can call. See [Function Calling](#function-calling) for details. |
|
||||
| `toolChoice` | String \| Object | Which tool, if any, the model must call. Pass `"auto"` (the model decides), `"none"` (never call a tool), `"required"` (call at least one), or an object naming a specific tool: `{ type: "tool", name: "..." }`. Requires `tools`. |
|
||||
| `parallelToolCalls` | Boolean | Whether the model may call more than one tool in a single turn. Left unset, tool calls run one at a time. |
|
||||
| `streamToolInput` | Boolean | Streaming only. When `true`, a tool call's arguments arrive incrementally as `tool_use_start` / `tool_input_delta` chunks instead of a single `tool_use` chunk once the call completes. |
|
||||
| `normalize` | Boolean | Controls the format of the non-streaming response. See [Response normalization](#response-normalization). |
|
||||
| `compaction` | Boolean \| Object | Opt into inline context compaction for long conversations. See [Compaction](#compaction). |
|
||||
| `signal` | AbortSignal | Abort the request, including while waiting for the next streamed chunk. Cancellation rejects with the signal reason (normally an `AbortError`). |
|
||||
| `provider` | String | Pin the request to a specific provider, for example `openrouter` or `infron`. Without it, Puter selects a provider for the requested model. Call [`puter.ai.listModelProviders()`](/AI/listModelProviders) for the available values, and [`puter.ai.listModels(provider)`](/AI/listModels) for the models a given provider serves. See [Providers](#providers). |
|
||||
| `fallback` | Boolean | Set to `false` to fail the request when the first provider tried fails, instead of retrying it on another provider that serves the same model. Defaults to `true`. See [Providers](#providers). |
|
||||
| `outputFormat` | Object | Constrains the response to a JSON schema: `{ type: "json_schema", schema, name?, strict? }`. |
|
||||
| `stopSequences` | Array of String | Up to the model's limit of strings; generation stops the moment one of them appears in the output. |
|
||||
| `model` | `String` | The model you want to use for the completion. If not specified, defaults to `gpt-5.4-nano`. More than 500 models are available from providers including OpenAI, Anthropic, Google, Alibaba Cloud, xAI, Mistral, OpenRouter, Infron, and others. For a full list, see the [AI models list](https://developer.puter.com/ai/models/) page. |
|
||||
| `serviceSpeed` | `String` | Preferred service speed, `"standard"`, `"fast"`, or `"ultrafast"`. Unavailable speeds fall back from ultrafast to fast, then standard. Omitting it preserves the model's default service. Faster service may cost more. See [Service speed](#service-speed) and [Model-specific features](#model-specific-features). |
|
||||
| `stream` | `Boolean` | A boolean indicating whether you want to stream the completion. Defaults to `false`. |
|
||||
| `max_tokens` | `Number` | The maximum number of tokens to generate in the completion. By default, the specific model's maximum is used. |
|
||||
| `temperature` | `Number` | A number between 0 and 2 indicating the randomness of the completion. Lower values make the output more focused and deterministic, while higher values make it more random. By default, the model's own temperature is used. See [Model-specific features](#model-specific-features). |
|
||||
| `reasoning_effort` / `reasoning.effort` | `String` | Controls how much effort reasoning models spend thinking. Supported values: `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Lower values give faster responses with less reasoning. OpenAI models, Moonshot's Kimi K3.1 Pro models, Meta's Muse Spark models, `gemini-4-argon`, `gemini-3.1-flash-lite` and `gemini-3-flash-preview` only. See [Model-specific features](#model-specific-features). |
|
||||
| `thinking` | `Object \| null` | Extended-thinking control: `{ type: "adaptive" \| "enabled" \| "disabled" \| "between_tools", budgetTokens?, display? }`. Pass `null` to send no thinking configuration at all (the model's own default); leaving it unset falls back to the legacy derivation from `reasoning_effort`. `budgetTokens` only applies to `type: "enabled"`. |
|
||||
| `verbosity` / `text.verbosity` | `String` | Controls how long or short responses are. Supported values: `low`, `medium`, and `high`. Lower values give shorter responses. OpenAI models only. |
|
||||
| `tools` | `Array` | Function definitions the AI can call. See [Function Calling](#function-calling) for details. |
|
||||
| `toolChoice` | `String \| Object` | Which tool, if any, the model must call. Pass `"auto"` (the model decides), `"none"` (never call a tool), `"required"` (call at least one), or an object naming a specific tool: `{ type: "tool", name: "..." }`. Requires `tools`. |
|
||||
| `parallelToolCalls` | `Boolean` | Whether the model may call more than one tool in a single turn. Left unset, tool calls run one at a time. |
|
||||
| `streamToolInput` | `Boolean` | Streaming only. When `true`, a tool call's arguments arrive incrementally as `tool_use_start` / `tool_input_delta` chunks instead of a single `tool_use` chunk once the call completes. |
|
||||
| `normalize` | `Boolean` | Controls the format of the non-streaming response. See [Response normalization](#response-normalization). |
|
||||
| `compaction` | `Boolean \| Object` | Opt into inline context compaction for long conversations. See [Compaction](#compaction). |
|
||||
| `signal` | `AbortSignal` | Abort the request, including while waiting for the next streamed chunk. Cancellation rejects with the signal reason (normally an `AbortError`). |
|
||||
| `provider` | `String` | Pin the request to a specific provider, for example `openrouter` or `infron`. Without it, Puter selects a provider for the requested model. Call [`puter.ai.listModelProviders()`](/AI/listModelProviders) for the available values, and [`puter.ai.listModels(provider)`](/AI/listModels) for the models a given provider serves. See [Providers](#providers). |
|
||||
| `fallback` | `Boolean` | Set to `false` to fail the request when the first provider tried fails, instead of retrying it on another provider that serves the same model. Defaults to `true`. See [Providers](#providers). |
|
||||
| `outputFormat` | `Object` | Constrains the response to a JSON schema: `{ type: "json_schema", schema, name?, strict? }`. |
|
||||
| `stopSequences` | `Array<String>` | Up to the model's limit of strings; generation stops the moment one of them appears in the output. |
|
||||
|
||||
Some models accept additional options or handle these options differently. See [Model-specific features](#model-specific-features).
|
||||
|
||||
@@ -138,6 +139,8 @@ by a newline. Use the async iterator to receive every chunk as an object.
|
||||
|
||||
In case of an error, the `Promise` will reject with an error message.
|
||||
|
||||
An unrecognized `serviceSpeed` value rejects with `invalid_service_speed`.
|
||||
|
||||
For streaming responses, an error from the AI provider is returned as a chunk
|
||||
with `type: "error"` and a `message` describing the failure. This also happens
|
||||
if the provider stops sending the response before it is complete. Check for
|
||||
@@ -370,10 +373,20 @@ const response = await puter.ai.chat('Hello', {
|
||||
|
||||
With `provider` set as well, the request is answered by that provider or fails. If that provider does not serve the model, the request fails with a `400` instead of going to another provider. The error is the same one you would get if every provider had failed, for example `400 upstream_provider_unavailable` or `429 upstream_rate_limited`.
|
||||
|
||||
## Service speed
|
||||
|
||||
The `serviceSpeed` option selects service capacity independently of reasoning effort. If the requested speed is unavailable for the selected model, the request uses the next supported speed in the order `ultrafast`, `fast`, `standard`.
|
||||
|
||||
A faster tier that rejects a request because its capacity or access is unavailable can retry at a lower speed on the same provider. The `fallback` option controls retries on other providers separately. Once streaming output starts, a failed stream ends with an error instead of restarting at another speed.
|
||||
|
||||
The `usageDetails.serviceSpeed` field reports the served speed. Billing uses the tier or equivalent model that served the request. A provider downgrade is billed at the lower tier's rates. Response normalization follows the same rule as a request without this option, including when a faster equivalent model is selected.
|
||||
|
||||
## Model-specific features
|
||||
|
||||
### OpenAI models
|
||||
|
||||
Fast service is available on models with a published Fast rate card. Ultrafast is available for GPT-6.1 Sol and GPT-6 Astra. Other models fall back to fast or standard. Cache and long-context rates apply to the served tier.
|
||||
|
||||
#### `temperature`
|
||||
|
||||
Some OpenAI reasoning models ignore `temperature` and `top_p`. GPT-5.6, GPT-6 Sol/Luna, and GPT-6.1 Terra accept them only with `reasoning_effort: 'none'`, and GPT-6 Astra, GPT-6.1 Sol, and GPT-5.3 Codex always ignore them.
|
||||
@@ -418,6 +431,8 @@ The `usageDetails` object on every result carries the same counts under provider
|
||||
|
||||
### Anthropic models
|
||||
|
||||
Fast service is available on Claude models with a Fast rate card. Ultrafast falls back to fast. When the account has no Fast quota, the request retries at standard speed. The upstream service tier and inference speed are separate fields.
|
||||
|
||||
#### Prompt Caching
|
||||
|
||||
Specific to Anthropic models, you can use the cache control feature, allowing you to optimize costs for repeated prompts.
|
||||
@@ -442,6 +457,8 @@ You can find the implementation in our [prompt caching example](/playground/ai-c
|
||||
|
||||
### Gemini models
|
||||
|
||||
Fast service uses Priority inference on supported text and image requests. Other media use standard service to preserve input handling.
|
||||
|
||||
#### `temperature`
|
||||
|
||||
`gemini-4-argon` ignores `temperature`. `gemini-3.1-flash-lite` and `gemini-3-flash-preview` use it.
|
||||
@@ -543,6 +560,8 @@ for await (const part of resp) {
|
||||
|
||||
### Moonshot Kimi models
|
||||
|
||||
Models with an equivalent Highspeed variant select that variant for fast service. Other models use standard service.
|
||||
|
||||
#### `reasoning_effort`
|
||||
|
||||
Kimi K3.1 Pro accepts `low`, `high` and `max`, uses `max` when you leave it unset, and fails with `400 bad_request` on any other value.
|
||||
@@ -557,6 +576,30 @@ Moonshot's Kimi K3 and Kimi K3.1 Pro models cache prompt prefixes on their own a
|
||||
|
||||
Muse Spark always reasons, so `none` is ignored for it.
|
||||
|
||||
### xAI models
|
||||
|
||||
Fast service requests Priority processing. Token charges, including cached input and reasoning, follow the served tier.
|
||||
|
||||
### MiniMax models
|
||||
|
||||
MiniMax M3 uses Priority admission for fast service. Earlier models with an equivalent Highspeed variant use that variant. Other models use standard service.
|
||||
|
||||
### Mistral models
|
||||
|
||||
Fast service requests Priority when the account and model are eligible. The provider can serve the request at standard speed.
|
||||
|
||||
### Alibaba Cloud models
|
||||
|
||||
Models with an equivalent Prime or Fast variant select that variant for fast service. Other models use standard service.
|
||||
|
||||
### OpenRouter models
|
||||
|
||||
Faster tiers come from the selected model's endpoint catalog. A tier can fall back upstream. Billing uses the endpoint that serves the request.
|
||||
|
||||
### Infron models
|
||||
|
||||
Models with a Priority offering select that offering for fast service. Other models use standard service.
|
||||
|
||||
## Examples
|
||||
|
||||
### Ask GPT-6 Luna a question
|
||||
@@ -1092,3 +1135,23 @@ Policy 8 - Account Management: Each Enterprise and Ultimate customer is assigned
|
||||
</body>
|
||||
</html>
|
||||
```
|
||||
|
||||
### Select service speed
|
||||
|
||||
For example, request the fastest supported service:
|
||||
|
||||
```html;ai-chat-service-speed
|
||||
<html>
|
||||
<body>
|
||||
<script src="https://js.puter.com/v2/"></script>
|
||||
<script type="module">
|
||||
const response = await puter.ai.chat('Explain closures in one paragraph.', {
|
||||
model: 'gpt-6.1-sol',
|
||||
serviceSpeed: 'ultrafast',
|
||||
});
|
||||
puter.print(response.message.content);
|
||||
puter.print(response.usageDetails.serviceSpeed);
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
```
|
||||
@@ -1,143 +0,0 @@
|
||||
---
|
||||
title: puter.ai.convo()
|
||||
description: Prepare an OpenAI realtime conversation with optional audio and text helpers.
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
The conversation API prepares a realtime session, verifies Puter sign-in, and lets you connect when your app is ready. Realtime conversations require a paid Puter plan; usage is billed to the signed-in user. Native OpenAI events and convenience methods use the same session. Microphone capture and speaker playback start when your app requests them.
|
||||
|
||||
In Node.js, the SDK requires a global `WebSocket`, available in Node.js 22 or later.
|
||||
|
||||
**Beta:** This API is incomplete. Its methods, options, events, and behavior may change abruptly, including breaking changes without notice.
|
||||
|
||||
To connect without puter.js, see the [Realtime WebSocket API](/AI/realtime/).
|
||||
|
||||
## Syntax
|
||||
|
||||
```js
|
||||
const conversation = await puter.ai.convo(options = {});
|
||||
await conversation.connect();
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
||||
The options object accepts these fields.
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
| --- | --- | --- | --- |
|
||||
| `model` | String | `gpt-realtime-2.1-mini` | Also accepts `gpt-realtime-2.1`, `gpt-realtime-2`, `gpt-realtime-1.5`, `gpt-realtime`, and `gpt-realtime-mini`. |
|
||||
| `instructions` | String | Provider default | Session instructions, up to 8192 characters. |
|
||||
| `output` | Array of Strings | `['audio']` | Accepts `['text']`, `['audio']`, or both. Audio includes a transcript. Requesting both uses audio output and emits transcript deltas through the `text` event. |
|
||||
| `voice` | String | Provider default | OpenAI voice identifier, up to 128 characters. |
|
||||
| `turnDetection` | Boolean | `true` | Whether the provider detects the end of spoken turns automatically. Set to `false` for manual audio commits. |
|
||||
|
||||
## Return value
|
||||
|
||||
A Promise resolves to a prepared conversation after options and sign-in are checked. In a browser, preparation opens the Puter sign-in prompt if needed. Preparation opens no realtime connection and requests no microphone access.
|
||||
|
||||
The conversation exposes these properties and methods.
|
||||
|
||||
| Member | Behavior |
|
||||
| --- | --- |
|
||||
| `state` | `prepared`, `connecting`, `connected`, or `closed`. |
|
||||
| `model` | Resolved model name. |
|
||||
| `connect()` | Opens the connection and applies settings. Resolves after the provider acknowledges session configuration. Concurrent calls share one connection Promise, including bounded retries for transient setup failures. Calling it while connected opens no additional connection. During a reconnection, it resolves when the reconnection succeeds and rejects if the conversation closes instead. |
|
||||
| `configure(options)` | Updates the given session settings and resolves after the provider acknowledges this update. Settings it isn't given, including ones set through native events, are left alone. Accepts the factory options except `model`. One update can be pending at a time. Provider rules apply to changes such as switching a voice after audio has been generated. |
|
||||
| `sendText(text)` | Adds a user message and requests a response. Text must be nonempty and at most 8192 characters. |
|
||||
| `appendAudio(pcm)` | Appends an `Int16Array` containing PCM16, 24 kHz mono audio. A chunk contains at most 12000 samples. |
|
||||
| `commitAudio()` | Commits buffered audio and requests a response. Requires turn detection to be off, through `turnDetection: false` or a native session update. The provider requires at least 100 ms of buffered audio. |
|
||||
| `interrupt(position?)` | Stops built-in playback immediately and waits for cancellation and playback-history synchronization. A reply requested but not yet started is cancelled as soon as it starts. Custom players provide `{ itemId, contentIndex, playedMs }`. An idle call resolves immediately. Readiness failures reject the returned Promise with `session_not_ready` or `session_closed`. |
|
||||
| `send(event)` | Sends a native OpenAI JSON event. |
|
||||
| `on(type, handler)` | Registers a listener and returns an unsubscribe function. Use `'*'` to receive every native and convenience event. |
|
||||
| `off(type, handler)` | Removes a listener. |
|
||||
| `close()` | Closes the session, rejects pending operations, and releases SDK-owned capture and playback resources. Repeated calls have no additional effect. |
|
||||
|
||||
Attach listeners before connecting to receive initial session events. Closing during connection rejects the pending attempt.
|
||||
|
||||
If the connection drops because of network loss, a server restart or a provider error, the conversation reconnects on its own. Normal closes with code 1000 end the conversation. While it reconnects, `state` is `connecting`, a `reconnecting` event fires, and pending operations reject with `connection_lost`. The new session gets every setting the provider acknowledged, from the factory options, `configure()` and native `session.update` events. Then `ready` fires again with `reconnected: true`. Listeners, the microphone and the speaker carry over. Microphone audio captured while reconnecting is dropped.
|
||||
|
||||
The conversation also restores recent context. Before `ready` fires, it adds back up to 40 of the most recent conversation items, 48 KiB in total. Restored items are user text, user speech that has a transcript, assistant replies as text, and function calls with their outputs. User speech has a transcript only when input transcription is configured through native session settings. An assistant reply that was interrupted, or still playing when the connection dropped, is cut to what the user saw or heard. The cut is estimated from how much of the audio played, so it can be off by a few words. Images, speech without a transcript, deleted items, and replies still generating when the connection dropped are left out. Restored items get new item IDs, don't fire item events, and count as input tokens on the next response. Item IDs from before the reconnect don't work in later events. Restoration uses a contiguous suffix of restorable items. An oversized newest item leaves that suffix empty. Older items left out to fit the window fire a `context_trimmed` event with the count in `omittedItems`; this is routine and not an error. A provider refusal or a failure to send the restored items fires one `error` event with code `context_restore_failed` per restoration. In both cases the conversation stays connected. Items are preserved without truncating text, function arguments or function results.
|
||||
|
||||
When Puter knows ahead of time that the server a session runs on will restart, it sends a `puter.reconnect` event, which also reaches your listeners. Without that notice, a restart drops the connection and the conversation reconnects as above. Within a few seconds, the conversation opens a replacement session in the background while the current one keeps working. It switches once no reply is generating or playing, no user turn is being spoken or transcribed, no function call is waiting for its output, and nothing you sent is waiting on the provider. The switch goes through the same steps as a reconnect, and the `reconnecting` event has code 1012. If the replacement can't open yet, the conversation keeps the current session and tries again later. A refusal that retrying can't fix, such as `insufficient_funds`, stops the attempts. If the server closes the session first, the conversation switches to the replacement, or reconnects if none is open.
|
||||
|
||||
The conversation makes up to ten attempts over about a minute. In a browser that reports being offline, it first waits up to two minutes for the connection to come back. If every attempt fails, or the server refuses with an error such as `insufficient_funds`, the conversation closes. A closed conversation cannot reconnect. To start another session, prepare a new one.
|
||||
|
||||
Native event definitions are listed in the [OpenAI realtime reference](https://developers.openai.com/api/reference/resources/realtime/server-events). Native events retain their provider fields. Sending native events preserves direct control of the session. Puter refuses `conversation.item.retrieve` events and MCP tools (`"type": "mcp"`) in `session.update` and `response.create` events. Sending one fires an `error` event with code `invalid_event`, and the conversation closes with code 1008. Audio helpers expect the session to use PCM16 at 24 kHz.
|
||||
|
||||
## Convenience events
|
||||
|
||||
The conversation emits these events alongside native events.
|
||||
|
||||
| Event | Fields |
|
||||
| --- | --- |
|
||||
| `ready` | `model`, `provider`, and `reconnected: true` after a reconnection |
|
||||
| `reconnecting` | `code`, `reason` of the dropped connection. After a `puter.reconnect` event, `code` is 1012. |
|
||||
| `context_trimmed` | `omittedItems`, the number of restorable items left out of the restored context after a reconnection. Fires before `ready`. |
|
||||
| `text` | `delta`, `responseId` |
|
||||
| `transcript` | `role`, `text`, `final`. Assistant deltas use `final: false`. Completed assistant and user transcripts use `final: true`. User transcripts require transcription to be configured through native session settings. |
|
||||
| `audio` | `pcm`, `sampleRate`, `responseId`, `itemId`, `contentIndex`. The PCM chunk is an `Int16Array`. |
|
||||
| `interrupted` | `source` (`manual` or `speech`), `responseId` |
|
||||
| `turn.done` | `responseId`, native provider `usage`, `status` |
|
||||
| `close` | `code`, `reason` |
|
||||
|
||||
Each event includes its `type`. When the provider event leaves out an ID, `responseId` and `itemId` are empty strings and `contentIndex` is `0`. The `interrupted` event signals local interruption immediately. Its corresponding operation resolves after the provider acknowledges cancellation and any requested truncation. Late audio from a canceled response remains visible as native events and is excluded from convenience audio and built-in playback.
|
||||
|
||||
## Browser media
|
||||
|
||||
The optional microphone and speaker helpers require a browser with Web Audio support. Microphone capture also requires media permissions and a secure context. Start audio from a user action so the browser can allow playback.
|
||||
|
||||
| Member | Behavior |
|
||||
| --- | --- |
|
||||
| `microphone.start()` | Requests microphone access and streams PCM16, resampled to 24 kHz, in 20 ms chunks through an AudioWorklet. If the page or network falls behind, chunks are dropped rather than stopping capture. |
|
||||
| `microphone.mute()` | Disables capture tracks and stops forwarding captured chunks. |
|
||||
| `microphone.unmute()` | Enables capture tracks and forwarding. |
|
||||
| `microphone.stop()` | Stops capture tracks and releases capture resources. A `start()` still waiting on the permission prompt rejects with `session_closed`. |
|
||||
| `speaker.start({ bargeIn: true })` | Starts playback. Barge-in defaults to `true`. With barge-in enabled, provider speech-start events interrupt playback and synchronize played audio with conversation history. Starting the speaker requests no microphone access. |
|
||||
| `speaker.stop()` | Clears playback and releases its audio context. |
|
||||
|
||||
Both start methods require a connected session. Capture and playback are independent, so custom audio input can use the built-in speaker, and built-in capture can use custom playback.
|
||||
|
||||
## Errors and limits
|
||||
|
||||
Failures use objects containing `code` and `message`. Preparation can reject with `invalid_argument`, `model_not_found`, `auth_required`, `auth_canceled`, or `auth_failed`. Connection can reject with `subscription_required` or `insufficient_funds`, which also show the Puter upgrade prompt. Connection and asynchronous operations can reject with provider errors, `session_closed`, `connection_lost`, `operation_timeout`, or `operation_in_progress`. Sending before readiness, or while reconnecting, throws `session_not_ready`.
|
||||
|
||||
Invalid input throws `invalid_argument`, `invalid_event`, or `input_too_large`. A full SDK send queue throws `queue_full`. Media helpers report `unsupported_environment`, `permission_denied`, `media_error`, `media_timeout`, or `queue_full`. Runtime media errors and `context_restore_failed` arrive through the `error` event. Provider errors also arrive as native `error` events with a nested `error` object.
|
||||
|
||||
Each native event is capped at 64 KiB. The SDK bounds JSON traversal, outgoing queues, acknowledgment waits, microphone work, and playback. All numeric limits are listed on [rate limits and quotas](/rate-limits-and-quotas/).
|
||||
|
||||
## Examples
|
||||
|
||||
A text conversation uses convenience methods for input and streamed output.
|
||||
|
||||
```js
|
||||
const conversation = await puter.ai.convo({
|
||||
output: ['text'],
|
||||
instructions: 'Keep replies short.',
|
||||
});
|
||||
|
||||
conversation.on('text', ({ delta }) => console.log(delta));
|
||||
conversation.on('turn.done', () => conversation.close());
|
||||
conversation.on('error', (event) => console.error(event));
|
||||
|
||||
await conversation.connect();
|
||||
conversation.sendText('Say hello in three words.');
|
||||
```
|
||||
|
||||
A voice conversation enables the microphone and speaker after connection.
|
||||
|
||||
```js
|
||||
const conversation = await puter.ai.convo();
|
||||
conversation.on('transcript', ({ role, text }) => console.log(role, text));
|
||||
conversation.on('error', (event) => console.error(event));
|
||||
|
||||
await conversation.connect();
|
||||
await conversation.speaker.start();
|
||||
await conversation.microphone.start();
|
||||
|
||||
// To interrupt a response and submit a replacement text turn:
|
||||
await conversation.interrupt();
|
||||
conversation.sendText('Actually, make it Wednesday.');
|
||||
|
||||
// When the user ends the conversation:
|
||||
conversation.close();
|
||||
```
|
||||
@@ -22,6 +22,8 @@ A `Promise` that will resolve to an array of string containing each AI providers
|
||||
|
||||
## Examples
|
||||
|
||||
### List all model providers
|
||||
|
||||
```html;ai-list-model-providers
|
||||
<html>
|
||||
<body>
|
||||
|
||||
@@ -43,13 +43,15 @@ Example model entry:
|
||||
]
|
||||
```
|
||||
|
||||
<div class="info">Successful results are cached for 60 seconds per provider.</div>
|
||||
|
||||
## Errors
|
||||
|
||||
Network and server failures reject the promise; they do not resolve to an empty
|
||||
catalogue. Network failures use `{ message, code: "network_error" }`, invalid
|
||||
catalogue responses use `invalid_response`, idle timeouts use `request_timeout`,
|
||||
and server errors pass through unchanged.
|
||||
|
||||
<div class="info">Successful results are cached for 60 seconds per provider.</div>
|
||||
|
||||
## Model variants
|
||||
|
||||
Some models have variants. A variant is the same model with a different price, rate limit, or speed. The variant is a suffix on the model ID, separated by a colon.
|
||||
@@ -64,6 +66,8 @@ The model ID without a suffix is the standard variant. Not every model has varia
|
||||
|
||||
## Examples
|
||||
|
||||
### List all models
|
||||
|
||||
```html;ai-list-models
|
||||
<html>
|
||||
<body>
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
---
|
||||
title: Realtime WebSocket API (Beta)
|
||||
description: Connect to OpenAI realtime conversations through Puter without puter.js.
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
The Realtime WebSocket API lets your client send and receive native OpenAI realtime JSON events through Puter.
|
||||
|
||||
Puter authenticates the user and charges their account for usage.
|
||||
|
||||
**Beta:** This API is incomplete. Its methods, options, events, and behavior may change abruptly, including breaking changes without notice.
|
||||
|
||||
For optional text, microphone, playback, and interruption helpers, use [`puter.ai.realtime()`](/AI/realtime/).
|
||||
|
||||
## Connection and authentication
|
||||
|
||||
Connect to this URL on the Puter API host:
|
||||
|
||||
```text
|
||||
wss://api.puter.com/puterai/openai/v1/realtime?model=gpt-realtime-2.1-mini
|
||||
```
|
||||
|
||||
The `model` query parameter defaults to `gpt-realtime-2.1-mini`. The model is fixed for the connection. A session update that changes it is rejected. It also accepts `gpt-realtime-2.1`, `gpt-realtime-2`, `gpt-realtime-1.5`, `gpt-realtime`, and `gpt-realtime-mini`.
|
||||
|
||||
Authenticate with a Puter session, app, or full-access API token. Puter supplies the provider credential. Server clients pass this header:
|
||||
|
||||
```text
|
||||
Authorization: Bearer <PUTER_TOKEN>
|
||||
```
|
||||
|
||||
Browser WebSocket clients pass `realtime` and `openai-insecure-api-key.<PUTER_TOKEN>` as subprotocols. The server echoes only `realtime`. The endpoint also accepts an `auth_token` query parameter. Prefer the authorization header where supported because URLs and subprotocols can appear in logs. Authentication takes precedence in this order, header, subprotocol, then query parameter.
|
||||
|
||||
The socket can open before authentication and provider setup finish. Listen for errors and closure immediately. Wait for `session.created`, send your session settings, then wait for `session.updated` before submitting a turn.
|
||||
|
||||
To open a replacement while another connection remains healthy, set the optional `purpose=handover` query parameter. These attempts share a smaller per-credential budget, checked before the general connection-attempt budget. Initial connections and dropped-connection recovery use the URL without this parameter. The limits are listed in [rate limits and quotas](/rate-limits-and-quotas/).
|
||||
|
||||
## Text example
|
||||
|
||||
Install the `ws` package and set `PUTER_TOKEN` to your Puter token. Run this example as a Node.js ES module:
|
||||
|
||||
```js
|
||||
import WebSocket from 'ws';
|
||||
|
||||
const socket = new WebSocket(
|
||||
'wss://api.puter.com/puterai/openai/v1/realtime?model=gpt-realtime-2.1-mini',
|
||||
{ headers: { Authorization: `Bearer ${process.env.PUTER_TOKEN}` } },
|
||||
);
|
||||
const send = (event) => socket.send(JSON.stringify(event));
|
||||
|
||||
socket.on('message', (data) => {
|
||||
const event = JSON.parse(data.toString());
|
||||
|
||||
if (event.type === 'session.created') {
|
||||
send({
|
||||
type: 'session.update',
|
||||
session: {
|
||||
type: 'realtime',
|
||||
output_modalities: ['text'],
|
||||
instructions: 'Keep replies short.',
|
||||
},
|
||||
});
|
||||
}
|
||||
if (event.type === 'session.updated') {
|
||||
send({
|
||||
type: 'conversation.item.create',
|
||||
item: {
|
||||
type: 'message',
|
||||
role: 'user',
|
||||
content: [{ type: 'input_text', text: 'Hello!' }],
|
||||
},
|
||||
});
|
||||
send({ type: 'response.create' });
|
||||
}
|
||||
if (event.type === 'response.output_text.delta')
|
||||
process.stdout.write(event.delta);
|
||||
if (event.type === 'error')
|
||||
console.error(event.error);
|
||||
if (event.type === 'response.done')
|
||||
socket.close();
|
||||
});
|
||||
socket.on('error', console.error);
|
||||
socket.on('close', (code, reason) => console.log(code, reason.toString()));
|
||||
```
|
||||
|
||||
## Events, audio, and interruption
|
||||
|
||||
Each message contains one JSON object with a nonempty string `type` of at most 128 characters. Binary messages are rejected. Native events retain their provider fields. The endpoint doesn't deliver `rate_limits.updated`, which describes Puter's provider account rather than yours, and it removes provider organization IDs from error messages. The endpoint exposes the WebSocket event protocol. Your client manages capture, playback, acknowledgments, and timeouts.
|
||||
|
||||
The endpoint refuses `conversation.item.retrieve` events and MCP tools (`"type": "mcp"`) in `session.update` and `response.create` events. A refused event isn't sent to the provider. The session gets an `invalid_event` error and closes with 1008.
|
||||
|
||||
PCM16 audio configured at 24 kHz mono is sent as base64 chunks in `input_audio_buffer.append` events. Output arrives in `response.output_audio.delta` events. With manual turn detection, commit buffered input with `input_audio_buffer.commit`, then request a reply with `response.create`.
|
||||
|
||||
To interrupt, stop local playback immediately and send `response.cancel` with the active `response_id`. If some generated audio was not played, send `conversation.item.truncate` with its `item_id`, `content_index`, and `audio_end_ms` set to the amount actually played. Wait for `response.done` and any requested `conversation.item.truncated` acknowledgment before submitting a replacement turn.
|
||||
|
||||
Connection loss ends the session. Reconnecting creates a new conversation. Tool execution and automatic reconnection belong to your client.
|
||||
|
||||
## Server restarts
|
||||
|
||||
When Puter knows ahead of time that the server a session runs on will restart, it sends each open session this event, at most once a minute:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "puter.reconnect",
|
||||
"event_id": "event_puter_1767225600000",
|
||||
"reason": "server_restarting",
|
||||
"close_in_ms": 90000
|
||||
}
|
||||
```
|
||||
|
||||
The `reason` field is `server_restarting`. The `close_in_ms` field, when present, is about how long until the server closes the session itself. The session keeps working until then. Open a new connection after a short, randomized delay, and once it gets `session.created` and your client is idle, with no response generating and no user turn in progress, move to it and close the old one. A connection that reaches a server that is also restarting gets a `server_restarting` error and closes with 1012. In that case keep the old session and try again after a short, randomized delay. If the server closes the old session first, the close code is 1012, and your client reconnects the same way. A restart can also drop the connection without any notice, so treat an unexpected close the same way.
|
||||
|
||||
A new connection starts an empty conversation. To keep context, add earlier items back with `conversation.item.create` before you request a response. Event types that start with `puter.` come from Puter, not the provider.
|
||||
|
||||
## Errors, billing, and limits
|
||||
|
||||
API errors arrive as `{ "type": "error", "error": { "code": "...", "message": "..." } }`. Handle socket errors and close frames too. Invalid, missing, or revoked credentials use `invalid_api_key`. If Puter can't check a credential or the account's plan, the error is `internal_error` and the session closes with 1011, so try again after a short, randomized delay. Other relay codes include `permission_denied`, `model_not_found`, `insufficient_funds`, `subscription_required`, `rate_limit_exceeded`, `invalid_event`, `connection_timeout`, `provider_unavailable`, `upstream_error`, and `server_restarting`.
|
||||
|
||||
Policy failures generally close with 1008, oversized client messages with 1009, upstream or setup failures with 1011, server restarts with 1012, and temporary server overload with 1013. Reconnect after 1011, 1012, or 1013. Authentication and model failures can arrive after the WebSocket handshake succeeds.
|
||||
|
||||
The endpoint requires a paid Puter plan. Usage is charged after completed responses and input transcriptions. While a response generates, part of your balance is held for it, so a session opens only if your balance covers one response. If the provider connection closes before a response reports its usage, the response is charged the full amount held for it. Each account can open 20 sessions at once by default. Each JSON message you send is limited to 64 KiB, including reassembled fragments. Messages from the provider can be up to 4 MiB. Stream audio in small chunks. All message rates, queue caps, timeouts, and other bounds are published in [rate limits and quotas](/rate-limits-and-quotas/).
|
||||
|
||||
Input transcription holds a small amount of your balance per transcribed item, priced at the transcription model you select. Two items are reserved ahead of your audio and replenished as the provider commits them, so a session holds only its spare items plus the items still awaiting a transcript. Changing the transcription model reprices the spare items. If your balance can't cover another item, audio waits while earlier items settle; once none are left to settle, the session closes with `insufficient_funds`. Failed transcriptions, items without final usage, and audio whose commit the provider never acknowledges before the connection closes are each charged that model's fixed per-item amount. Unacknowledged audio is charged one item per started 160 seconds. With server turn detection, only detected speech counts, so silence is never charged. The amounts, timeout and audio limits are listed in [rate limits and quotas](/rate-limits-and-quotas/).
|
||||
|
||||
Provider-reported realtime models must have a known price. A missing model uses the last known model. An explicitly unpriced model closes the connection with `upstream_error`.
|
||||
|
||||
After recording usage, the connection can continue while settled credit is positive, including a fractional balance. Each new billable operation still needs its full reservation after accounting for other outstanding holds.
|
||||
+129
-102
@@ -1,124 +1,151 @@
|
||||
---
|
||||
title: Realtime WebSocket API (Beta)
|
||||
description: Connect to OpenAI realtime conversations through Puter without puter.js.
|
||||
title: puter.ai.realtime()
|
||||
description: Prepare an OpenAI realtime conversation with optional audio and text helpers.
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
The Realtime WebSocket API lets your client send and receive native OpenAI realtime JSON events through Puter. Puter authenticates the user and charges their account for usage.
|
||||
Prepares a realtime OpenAI conversation that your app can connect to when it's ready.
|
||||
|
||||
Preparing the conversation verifies Puter sign-in. Realtime conversations require a paid Puter plan, and usage is billed to the signed-in user. Native OpenAI events and convenience methods use the same session. Microphone capture and speaker playback start when your app requests them.
|
||||
|
||||
In Node.js, the SDK requires a global `WebSocket`, available in Node.js 22 or later.
|
||||
|
||||
**Beta:** This API is incomplete. Its methods, options, events, and behavior may change abruptly, including breaking changes without notice.
|
||||
|
||||
For optional text, microphone, playback, and interruption helpers, use [`puter.ai.convo()`](/AI/convo/).
|
||||
To connect without puter.js, see the [Realtime WebSocket API](/AI/realtime-websocket/).
|
||||
|
||||
## Connection and authentication
|
||||
|
||||
Connect to this URL on the Puter API host:
|
||||
|
||||
```text
|
||||
wss://api.puter.com/puterai/openai/v1/realtime?model=gpt-realtime-2.1-mini
|
||||
```
|
||||
|
||||
The `model` query parameter defaults to `gpt-realtime-2.1-mini`. The model is fixed for the connection. A session update that changes it is rejected. It also accepts `gpt-realtime-2.1`, `gpt-realtime-2`, `gpt-realtime-1.5`, `gpt-realtime`, and `gpt-realtime-mini`.
|
||||
|
||||
Authenticate with a Puter session, app, or full-access API token. Puter supplies the provider credential. Server clients pass this header:
|
||||
|
||||
```text
|
||||
Authorization: Bearer <PUTER_TOKEN>
|
||||
```
|
||||
|
||||
Browser WebSocket clients pass `realtime` and `openai-insecure-api-key.<PUTER_TOKEN>` as subprotocols. The server echoes only `realtime`. The endpoint also accepts an `auth_token` query parameter. Prefer the authorization header where supported because URLs and subprotocols can appear in logs. Authentication takes precedence in this order, header, subprotocol, then query parameter.
|
||||
|
||||
The socket can open before authentication and provider setup finish. Listen for errors and closure immediately. Wait for `session.created`, send your session settings, then wait for `session.updated` before submitting a turn.
|
||||
|
||||
To open a replacement while another connection remains healthy, set the optional `purpose=handover` query parameter. These attempts share a smaller per-credential budget, checked before the general connection-attempt budget. Initial connections and dropped-connection recovery use the URL without this parameter. The limits are listed in [rate limits and quotas](/rate-limits-and-quotas/).
|
||||
|
||||
## Text example
|
||||
|
||||
Install the `ws` package and set `PUTER_TOKEN` to your Puter token. Run this example as a Node.js ES module:
|
||||
## Syntax
|
||||
|
||||
```js
|
||||
import WebSocket from 'ws';
|
||||
const conversation = await puter.ai.realtime(options = {});
|
||||
await conversation.connect();
|
||||
```
|
||||
|
||||
const socket = new WebSocket(
|
||||
'wss://api.puter.com/puterai/openai/v1/realtime?model=gpt-realtime-2.1-mini',
|
||||
{ headers: { Authorization: `Bearer ${process.env.PUTER_TOKEN}` } },
|
||||
);
|
||||
const send = (event) => socket.send(JSON.stringify(event));
|
||||
## Parameters
|
||||
|
||||
socket.on('message', (data) => {
|
||||
const event = JSON.parse(data.toString());
|
||||
#### `options` (Object) (Optional)
|
||||
|
||||
if (event.type === 'session.created') {
|
||||
send({
|
||||
type: 'session.update',
|
||||
session: {
|
||||
type: 'realtime',
|
||||
output_modalities: ['text'],
|
||||
instructions: 'Keep replies short.',
|
||||
},
|
||||
});
|
||||
}
|
||||
if (event.type === 'session.updated') {
|
||||
send({
|
||||
type: 'conversation.item.create',
|
||||
item: {
|
||||
type: 'message',
|
||||
role: 'user',
|
||||
content: [{ type: 'input_text', text: 'Hello!' }],
|
||||
},
|
||||
});
|
||||
send({ type: 'response.create' });
|
||||
}
|
||||
if (event.type === 'response.output_text.delta')
|
||||
process.stdout.write(event.delta);
|
||||
if (event.type === 'error')
|
||||
console.error(event.error);
|
||||
if (event.type === 'response.done')
|
||||
socket.close();
|
||||
The options object accepts these fields.
|
||||
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `model` | `String` | The realtime model. Defaults to `gpt-realtime-2.1-mini`. Also accepts `gpt-realtime-2.1`, `gpt-realtime-2`, `gpt-realtime-1.5`, `gpt-realtime`, and `gpt-realtime-mini`. |
|
||||
| `instructions` | `String` | Session instructions, up to 8192 characters. Defaults to the provider's default. |
|
||||
| `output` | `Array<String>` | Accepts `['text']`, `['audio']`, or both. Defaults to `['audio']`. Audio includes a transcript. Requesting both uses audio output and emits transcript deltas through the `text` event. |
|
||||
| `voice` | `String` | OpenAI voice identifier, up to 128 characters. Defaults to the provider's default. |
|
||||
| `turnDetection` | `Boolean` | Whether the provider detects the end of spoken turns automatically. Defaults to `true`. Set to `false` for manual audio commits. |
|
||||
|
||||
## Return value
|
||||
|
||||
A Promise resolves to a prepared conversation after options and sign-in are checked. In a browser, preparation opens the Puter sign-in prompt if needed. Preparation opens no realtime connection and requests no microphone access.
|
||||
|
||||
The conversation exposes these properties and methods.
|
||||
|
||||
| Member | Behavior |
|
||||
| --- | --- |
|
||||
| `state` | `prepared`, `connecting`, `connected`, or `closed`. |
|
||||
| `model` | Resolved model name. |
|
||||
| `connect()` | Opens the connection and applies settings. Resolves after the provider acknowledges session configuration. Concurrent calls share one connection Promise, including bounded retries for transient setup failures. Calling it while connected opens no additional connection. During a reconnection, it resolves when the reconnection succeeds and rejects if the conversation closes instead. |
|
||||
| `configure(options)` | Updates the given session settings and resolves after the provider acknowledges this update. Settings it isn't given, including ones set through native events, are left alone. Accepts the factory options except `model`. One update can be pending at a time. Provider rules apply to changes such as switching a voice after audio has been generated. |
|
||||
| `sendText(text)` | Adds a user message and requests a response. Text must be nonempty and at most 8192 characters. |
|
||||
| `appendAudio(pcm)` | Appends an `Int16Array` containing PCM16, 24 kHz mono audio. A chunk contains at most 12000 samples. |
|
||||
| `commitAudio()` | Commits buffered audio and requests a response. Requires turn detection to be off, through `turnDetection: false` or a native session update. The provider requires at least 100 ms of buffered audio. |
|
||||
| `interrupt(position?)` | Stops built-in playback immediately and waits for cancellation and playback-history synchronization. A reply requested but not yet started is cancelled as soon as it starts. Custom players provide `{ itemId, contentIndex, playedMs }`. An idle call resolves immediately. Readiness failures reject the returned Promise with `session_not_ready` or `session_closed`. |
|
||||
| `send(event)` | Sends a native OpenAI JSON event. |
|
||||
| `on(type, handler)` | Registers a listener and returns an unsubscribe function. Use `'*'` to receive every native and convenience event. |
|
||||
| `off(type, handler)` | Removes a listener. |
|
||||
| `close()` | Closes the session, rejects pending operations, and releases SDK-owned capture and playback resources. Repeated calls have no additional effect. |
|
||||
|
||||
Attach listeners before connecting to receive initial session events. Closing during connection rejects the pending attempt.
|
||||
|
||||
If the connection drops because of network loss, a server restart or a provider error, the conversation reconnects on its own. Normal closes with code 1000 end the conversation. While it reconnects, `state` is `connecting`, a `reconnecting` event fires, and pending operations reject with `connection_lost`. The new session gets every setting the provider acknowledged, from the factory options, `configure()` and native `session.update` events. Then `ready` fires again with `reconnected: true`. Listeners, the microphone and the speaker carry over. Microphone audio captured while reconnecting is dropped.
|
||||
|
||||
The conversation also restores recent context. Before `ready` fires, it adds back up to 40 of the most recent conversation items, 48 KiB in total. Restored items are user text, user speech that has a transcript, assistant replies as text, and function calls with their outputs. User speech has a transcript only when input transcription is configured through native session settings. An assistant reply that was interrupted, or still playing when the connection dropped, is cut to what the user saw or heard. The cut is estimated from how much of the audio played, so it can be off by a few words. Images, speech without a transcript, deleted items, and replies still generating when the connection dropped are left out. Restored items get new item IDs, don't fire item events, and count as input tokens on the next response. Item IDs from before the reconnect don't work in later events. Restoration uses a contiguous suffix of restorable items. An oversized newest item leaves that suffix empty. Older items left out to fit the window fire a `context_trimmed` event with the count in `omittedItems`; this is routine and not an error. A provider refusal or a failure to send the restored items fires one `error` event with code `context_restore_failed` per restoration. In both cases the conversation stays connected. Items are preserved without truncating text, function arguments or function results.
|
||||
|
||||
When Puter knows ahead of time that the server a session runs on will restart, it sends a `puter.reconnect` event, which also reaches your listeners. Without that notice, a restart drops the connection and the conversation reconnects as above. Within a few seconds, the conversation opens a replacement session in the background while the current one keeps working. It switches once no reply is generating or playing, no user turn is being spoken or transcribed, no function call is waiting for its output, and nothing you sent is waiting on the provider. The switch goes through the same steps as a reconnect, and the `reconnecting` event has code 1012. If the replacement can't open yet, the conversation keeps the current session and tries again later. A refusal that retrying can't fix, such as `insufficient_funds`, stops the attempts. If the server closes the session first, the conversation switches to the replacement, or reconnects if none is open.
|
||||
|
||||
The conversation makes up to ten attempts over about a minute. In a browser that reports being offline, it first waits up to two minutes for the connection to come back. If every attempt fails, or the server refuses with an error such as `insufficient_funds`, the conversation closes. A closed conversation cannot reconnect. To start another session, prepare a new one.
|
||||
|
||||
Native event definitions are listed in the [OpenAI realtime reference](https://developers.openai.com/api/reference/resources/realtime/server-events). Native events retain their provider fields. Sending native events preserves direct control of the session. Puter refuses `conversation.item.retrieve` events and MCP tools (`"type": "mcp"`) in `session.update` and `response.create` events. Sending one fires an `error` event with code `invalid_event`, and the conversation closes with code 1008. Audio helpers expect the session to use PCM16 at 24 kHz.
|
||||
|
||||
## Errors and limits
|
||||
|
||||
Failures use objects containing `code` and `message`. Preparation can reject with `invalid_argument`, `model_not_found`, `auth_required`, `auth_canceled`, or `auth_failed`. Connection can reject with `subscription_required` or `insufficient_funds`, which also show the Puter upgrade prompt. Connection and asynchronous operations can reject with provider errors, `session_closed`, `connection_lost`, `operation_timeout`, or `operation_in_progress`. Sending before readiness, or while reconnecting, throws `session_not_ready`.
|
||||
|
||||
Invalid input throws `invalid_argument`, `invalid_event`, or `input_too_large`. A full SDK send queue throws `queue_full`. Media helpers report `unsupported_environment`, `permission_denied`, `media_error`, `media_timeout`, or `queue_full`. Runtime media errors and `context_restore_failed` arrive through the `error` event. Provider errors also arrive as native `error` events with a nested `error` object.
|
||||
|
||||
Each native event is capped at 64 KiB. The SDK bounds JSON traversal, outgoing queues, acknowledgment waits, microphone work, and playback. All numeric limits are listed on [rate limits and quotas](/rate-limits-and-quotas/).
|
||||
|
||||
## Convenience events
|
||||
|
||||
The conversation emits these events alongside native events.
|
||||
|
||||
| Event | Fields |
|
||||
| --- | --- |
|
||||
| `ready` | `model`, `provider`, and `reconnected: true` after a reconnection |
|
||||
| `reconnecting` | `code`, `reason` of the dropped connection. After a `puter.reconnect` event, `code` is 1012. |
|
||||
| `context_trimmed` | `omittedItems`, the number of restorable items left out of the restored context after a reconnection. Fires before `ready`. |
|
||||
| `text` | `delta`, `responseId` |
|
||||
| `transcript` | `role`, `text`, `final`. Assistant deltas use `final: false`. Completed assistant and user transcripts use `final: true`. User transcripts require transcription to be configured through native session settings. |
|
||||
| `audio` | `pcm`, `sampleRate`, `responseId`, `itemId`, `contentIndex`. The PCM chunk is an `Int16Array`. |
|
||||
| `interrupted` | `source` (`manual` or `speech`), `responseId` |
|
||||
| `turn.done` | `responseId`, native provider `usage`, `status` |
|
||||
| `close` | `code`, `reason` |
|
||||
|
||||
Each event includes its `type`. When the provider event leaves out an ID, `responseId` and `itemId` are empty strings and `contentIndex` is `0`. The `interrupted` event signals local interruption immediately. Its corresponding operation resolves after the provider acknowledges cancellation and any requested truncation. Late audio from a canceled response remains visible as native events and is excluded from convenience audio and built-in playback.
|
||||
|
||||
## Browser media
|
||||
|
||||
The optional microphone and speaker helpers require a browser with Web Audio support. Microphone capture also requires media permissions and a secure context. Start audio from a user action so the browser can allow playback.
|
||||
|
||||
| Member | Behavior |
|
||||
| --- | --- |
|
||||
| `microphone.start()` | Requests microphone access and streams PCM16, resampled to 24 kHz, in 20 ms chunks through an AudioWorklet. If the page or network falls behind, chunks are dropped rather than stopping capture. |
|
||||
| `microphone.mute()` | Disables capture tracks and stops forwarding captured chunks. |
|
||||
| `microphone.unmute()` | Enables capture tracks and forwarding. |
|
||||
| `microphone.stop()` | Stops capture tracks and releases capture resources. A `start()` still waiting on the permission prompt rejects with `session_closed`. |
|
||||
| `speaker.start({ bargeIn: true })` | Starts playback. Barge-in defaults to `true`. With barge-in enabled, provider speech-start events interrupt playback and synchronize played audio with conversation history. Starting the speaker requests no microphone access. |
|
||||
| `speaker.stop()` | Clears playback and releases its audio context. |
|
||||
|
||||
Both start methods require a connected session. Capture and playback are independent, so custom audio input can use the built-in speaker, and built-in capture can use custom playback.
|
||||
|
||||
## Examples
|
||||
|
||||
### Have a text conversation
|
||||
|
||||
A text conversation uses convenience methods for input and streamed output.
|
||||
|
||||
```js
|
||||
const conversation = await puter.ai.realtime({
|
||||
output: ['text'],
|
||||
instructions: 'Keep replies short.',
|
||||
});
|
||||
socket.on('error', console.error);
|
||||
socket.on('close', (code, reason) => console.log(code, reason.toString()));
|
||||
|
||||
conversation.on('text', ({ delta }) => console.log(delta));
|
||||
conversation.on('turn.done', () => conversation.close());
|
||||
conversation.on('error', (event) => console.error(event));
|
||||
|
||||
await conversation.connect();
|
||||
conversation.sendText('Say hello in three words.');
|
||||
```
|
||||
|
||||
## Events, audio, and interruption
|
||||
### Have a voice conversation
|
||||
|
||||
Each message contains one JSON object with a nonempty string `type` of at most 128 characters. Binary messages are rejected. Native events retain their provider fields. The endpoint doesn't deliver `rate_limits.updated`, which describes Puter's provider account rather than yours, and it removes provider organization IDs from error messages. The endpoint exposes the WebSocket event protocol. Your client manages capture, playback, acknowledgments, and timeouts.
|
||||
A voice conversation enables the microphone and speaker after connection.
|
||||
|
||||
The endpoint refuses `conversation.item.retrieve` events and MCP tools (`"type": "mcp"`) in `session.update` and `response.create` events. A refused event isn't sent to the provider. The session gets an `invalid_event` error and closes with 1008.
|
||||
```js
|
||||
const conversation = await puter.ai.realtime();
|
||||
conversation.on('transcript', ({ role, text }) => console.log(role, text));
|
||||
conversation.on('error', (event) => console.error(event));
|
||||
|
||||
PCM16 audio configured at 24 kHz mono is sent as base64 chunks in `input_audio_buffer.append` events. Output arrives in `response.output_audio.delta` events. With manual turn detection, commit buffered input with `input_audio_buffer.commit`, then request a reply with `response.create`.
|
||||
await conversation.connect();
|
||||
await conversation.speaker.start();
|
||||
await conversation.microphone.start();
|
||||
|
||||
To interrupt, stop local playback immediately and send `response.cancel` with the active `response_id`. If some generated audio was not played, send `conversation.item.truncate` with its `item_id`, `content_index`, and `audio_end_ms` set to the amount actually played. Wait for `response.done` and any requested `conversation.item.truncated` acknowledgment before submitting a replacement turn.
|
||||
// To interrupt a response and submit a replacement text turn:
|
||||
await conversation.interrupt();
|
||||
conversation.sendText('Actually, make it Wednesday.');
|
||||
|
||||
Connection loss ends the session. Reconnecting creates a new conversation. Tool execution and automatic reconnection belong to your client.
|
||||
|
||||
## Server restarts
|
||||
|
||||
When Puter knows ahead of time that the server a session runs on will restart, it sends each open session this event, at most once a minute:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "puter.reconnect",
|
||||
"event_id": "event_puter_1767225600000",
|
||||
"reason": "server_restarting",
|
||||
"close_in_ms": 90000
|
||||
}
|
||||
// When the user ends the conversation:
|
||||
conversation.close();
|
||||
```
|
||||
|
||||
The `reason` field is `server_restarting`. The `close_in_ms` field, when present, is about how long until the server closes the session itself. The session keeps working until then. Open a new connection after a short, randomized delay, and once it gets `session.created` and your client is idle, with no response generating and no user turn in progress, move to it and close the old one. A connection that reaches a server that is also restarting gets a `server_restarting` error and closes with 1012. In that case keep the old session and try again after a short, randomized delay. If the server closes the old session first, the close code is 1012, and your client reconnects the same way. A restart can also drop the connection without any notice, so treat an unexpected close the same way.
|
||||
|
||||
A new connection starts an empty conversation. To keep context, add earlier items back with `conversation.item.create` before you request a response. Event types that start with `puter.` come from Puter, not the provider.
|
||||
|
||||
## Errors, billing, and limits
|
||||
|
||||
API errors arrive as `{ "type": "error", "error": { "code": "...", "message": "..." } }`. Handle socket errors and close frames too. Invalid, missing, or revoked credentials use `invalid_api_key`. If Puter can't check a credential or the account's plan, the error is `internal_error` and the session closes with 1011, so try again after a short, randomized delay. Other relay codes include `permission_denied`, `model_not_found`, `insufficient_funds`, `subscription_required`, `rate_limit_exceeded`, `invalid_event`, `connection_timeout`, `provider_unavailable`, `upstream_error`, and `server_restarting`.
|
||||
|
||||
Policy failures generally close with 1008, oversized client messages with 1009, upstream or setup failures with 1011, server restarts with 1012, and temporary server overload with 1013. Reconnect after 1011, 1012, or 1013. Authentication and model failures can arrive after the WebSocket handshake succeeds.
|
||||
|
||||
The endpoint requires a paid Puter plan. Usage is charged after completed responses and input transcriptions. While a response generates, part of your balance is held for it, so a session opens only if your balance covers one response. If the provider connection closes before a response reports its usage, the response is charged the full amount held for it. Each account can open 20 sessions at once by default. Each JSON message you send is limited to 64 KiB, including reassembled fragments. Messages from the provider can be up to 4 MiB. Stream audio in small chunks. All message rates, queue caps, timeouts, and other bounds are published in [rate limits and quotas](/rate-limits-and-quotas/).
|
||||
|
||||
Input transcription holds a small amount of your balance per transcribed item, priced at the transcription model you select. Two items are reserved ahead of your audio and replenished as the provider commits them, so a session holds only its spare items plus the items still awaiting a transcript. Changing the transcription model reprices the spare items. If your balance can't cover another item, audio waits while earlier items settle; once none are left to settle, the session closes with `insufficient_funds`. Failed transcriptions, items without final usage, and audio whose commit the provider never acknowledges before the connection closes are each charged that model's fixed per-item amount. Unacknowledged audio is charged one item per started 160 seconds. With server turn detection, only detected speech counts, so silence is never charged. The amounts, timeout and audio limits are listed in [rate limits and quotas](/rate-limits-and-quotas/).
|
||||
|
||||
Provider-reported realtime models must have a known price. A missing model uses the last known model. An explicitly unpriced model closes the connection with `upstream_error`.
|
||||
|
||||
After recording usage, the connection can continue while settled credit is positive, including a fractional balance. Each new billable operation still needs its full reservation after accounting for other outstanding holds.
|
||||
@@ -54,7 +54,7 @@ Example response:
|
||||
|
||||
## Examples
|
||||
|
||||
<strong class="example-title">List engines for a specific provider</strong>
|
||||
### List engines for a specific provider
|
||||
|
||||
```html;ai-txt2speech-list-engines
|
||||
<html>
|
||||
@@ -73,7 +73,7 @@ Example response:
|
||||
</html>
|
||||
```
|
||||
|
||||
<strong class="example-title">List engines using options object</strong>
|
||||
### List engines using options object
|
||||
|
||||
```js
|
||||
const engines = await puter.ai.txt2speech.listEngines({ provider: 'elevenlabs' });
|
||||
|
||||
@@ -4,7 +4,9 @@ description: List available TTS voices, optionally filtered by provider.
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
Returns the voices available from a TTS provider. Each voice entry includes metadata such as language, category, and supported models.
|
||||
Returns the voices available from a TTS provider.
|
||||
|
||||
Each voice entry includes metadata such as language, category, and supported models.
|
||||
|
||||
## Syntax
|
||||
|
||||
@@ -52,7 +54,7 @@ Example response (with `provider: 'all'`):
|
||||
|
||||
## Examples
|
||||
|
||||
<strong class="example-title">List voices for a provider</strong>
|
||||
### List voices for a provider
|
||||
|
||||
```html;ai-txt2speech-list-voices
|
||||
<html>
|
||||
@@ -71,7 +73,7 @@ Example response (with `provider: 'all'`):
|
||||
</html>
|
||||
```
|
||||
|
||||
<strong class="example-title">List all default (AWS Polly) voices</strong>
|
||||
### List all default (AWS Polly) voices
|
||||
|
||||
```js
|
||||
const voices = await puter.ai.txt2speech.listVoices();
|
||||
@@ -81,7 +83,7 @@ for (const voice of voices) {
|
||||
}
|
||||
```
|
||||
|
||||
<strong class="example-title">List Gemini voices</strong>
|
||||
### List Gemini voices
|
||||
|
||||
```js
|
||||
const voices = await puter.ai.txt2speech.listVoices({ provider: 'gemini' });
|
||||
|
||||
@@ -4,7 +4,9 @@ description: Convert text to speech with AI using multiple languages, voices, an
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
Converts text into speech using AI. Supports multiple languages and voices.
|
||||
Converts text into speech using AI.
|
||||
|
||||
It supports multiple languages and voices.
|
||||
|
||||
## Syntax
|
||||
|
||||
|
||||
@@ -183,9 +183,9 @@ These Apps API are supported out of the box when using Puter.js:
|
||||
|
||||
- **[`puter.apps.create()`](/Apps/create/)** - Create a new application
|
||||
- **[`puter.apps.list()`](/Apps/list/)** - List all applications
|
||||
- **[`puter.apps.delete()`](/Apps/delete/)** - Delete an application
|
||||
- **[`puter.apps.update()`](/Apps/update/)** - Update application settings
|
||||
- **[`puter.apps.get()`](/Apps/get/)** - Get information about a specific application
|
||||
- **[`puter.apps.update()`](/Apps/update/)** - Update application settings
|
||||
- **[`puter.apps.delete()`](/Apps/delete/)** - Delete an application
|
||||
- **[`puter.apps.checkName()`](/Apps/checkName/)** - Check whether an app name is available
|
||||
|
||||
## Examples
|
||||
|
||||
@@ -4,7 +4,9 @@ description: Check whether an app name is still available.
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
Checks whether an app name is free, without creating anything. App names are unique across all Puter accounts, so a name is free only when no app on any account uses it. Useful before calling [`puter.apps.create()`](/Apps/create/), which rejects when the name is already taken.
|
||||
Checks whether an app name is free, without creating anything.
|
||||
|
||||
App names are unique across all Puter accounts, so a name is free only when no app on any account uses it. Useful before calling [`puter.apps.create()`](/Apps/create/), which rejects when the name is already taken.
|
||||
|
||||
## Syntax
|
||||
|
||||
@@ -24,7 +26,7 @@ A `Promise` that will resolve to an object describing the name's availability.
|
||||
|
||||
## Examples
|
||||
|
||||
<strong class="example-title">Check a name before creating the app</strong>
|
||||
### Check a name before creating the app
|
||||
|
||||
```html
|
||||
<html>
|
||||
|
||||
+17
-13
@@ -4,7 +4,9 @@ description: Create apps in the Puter desktop environment.
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
Creates a Puter app with the given name. The app will be created in the user's apps, and will be accessible to this app. The app will be created with no permissions, and will not be able to access any data until permissions are granted to it.
|
||||
Creates a Puter app with the given name.
|
||||
|
||||
The app will be created in the user's apps, and will be accessible to this app. The app will be created with no permissions, and will not be able to access any data until permissions are granted to it.
|
||||
|
||||
## Syntax
|
||||
|
||||
@@ -39,17 +41,19 @@ The title of the app. If this parameter is not provided, the app will be created
|
||||
|
||||
An object containing the options for the app to create. The object can contain the following properties:
|
||||
|
||||
- `name` (String) (required): The name of the app to create. App names are unique across all Puter accounts. If any app already has this name, the promise will be rejected.
|
||||
- `indexURL` (String) (required): The URL of the app's index page. This URL must be accessible to the user. If this parameter is not provided, the promise will be rejected.
|
||||
- `title` (String) (optional): The human-readable title of the app. If this parameter is not provided, the app will be created with `name` as its title.
|
||||
- `description` (String) (optional): The description of the app aimed at the end user.
|
||||
- `icon` (String) (optional): The new icon of the app.
|
||||
- `maximizeOnStart` (Boolean) (optional): Whether the app should be maximized when it is started. Defaults to `false`.
|
||||
- `filetypeAssociations` (Array<String>) (optional): An array of strings representing the filetypes that the app can open. Defaults to `[]`. File extentions and MIME types are supported; For example, `[".txt", ".md", "application/pdf"]` would allow the app to open `.txt`, `.md`, and PDF files. At most 200 entries, each at most 60 characters.
|
||||
- `dedupeName` (Boolean) (optional) - Whether to deduplicate the app name if it already exists. Defaults to `false`.
|
||||
- `background` (Boolean) (optional) - Whether the app should run in the background. Defaults to `false`.
|
||||
- `feedbackEnabled` (Boolean) (optional) - Whether users can send feedback to you through [`puter.ui.showFeedbackDialog()`](/UI/showFeedbackDialog/). Defaults to `false`.
|
||||
- `metadata` (Object) (optional) - An object containing custom metadata for the app. This can be used to store arbitrary key-value pairs associated with the app. At most 16 KiB when serialized as JSON.
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `name` | `String` | Required. The name of the app to create. App names are unique across all Puter accounts. If any app already has this name, the promise will be rejected. |
|
||||
| `indexURL` | `String` | Required. The URL of the app's index page. This URL must be accessible to the user. If it's missing, the promise will be rejected. |
|
||||
| `title` | `String` | The human-readable title of the app. If it's missing, the app will be created with `name` as its title. |
|
||||
| `description` | `String` | The description of the app aimed at the end user. |
|
||||
| `icon` | `String` | The new icon of the app. |
|
||||
| `filetypeAssociations` | `Array<String>` | An array of strings representing the filetypes that the app can open. Defaults to `[]`. File extensions and MIME types are supported. For example, `[".txt", ".md", "application/pdf"]` would allow the app to open `.txt`, `.md`, and PDF files. At most 200 entries, each at most 60 characters. |
|
||||
| `maximizeOnStart` | `Boolean` | Whether the app should be maximized when it is started. Defaults to `false`. |
|
||||
| `background` | `Boolean` | Whether the app should run in the background. Defaults to `false`. |
|
||||
| `feedbackEnabled` | `Boolean` | Whether users can send feedback to you through [`puter.ui.showFeedbackDialog()`](/UI/showFeedbackDialog/). Defaults to `false`. |
|
||||
| `metadata` | `Object` | An object containing custom metadata for the app. This can be used to store arbitrary key-value pairs associated with the app. At most 16 KiB when serialized as JSON. |
|
||||
| `dedupeName` | `Boolean` | Whether to deduplicate the app name if it already exists. Defaults to `false`. |
|
||||
|
||||
## Return value
|
||||
|
||||
@@ -57,7 +61,7 @@ A `Promise` that will resolve to the [`CreateAppResult`](/Objects/createappresul
|
||||
|
||||
## Examples
|
||||
|
||||
<strong class="example-title">Create an app pointing to example.com</strong>
|
||||
### Create an app pointing to example.com
|
||||
|
||||
```html;app-create
|
||||
<html>
|
||||
|
||||
@@ -24,7 +24,7 @@ A `Promise` that will resolve to an object `{ success: true, uid: <app uid> }` i
|
||||
|
||||
## Examples
|
||||
|
||||
<strong class="example-title">Create a random app then delete it</strong>
|
||||
### Create a random app then delete it
|
||||
|
||||
```html;app-delete
|
||||
<html>
|
||||
|
||||
@@ -4,7 +4,9 @@ description: Retrieve details of your Puter app.
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
Returns an app with the given name. If the app does not exist, the promise will be rejected.
|
||||
Returns an app with the given name.
|
||||
|
||||
If the app does not exist, the promise will be rejected.
|
||||
|
||||
## Syntax
|
||||
|
||||
@@ -19,13 +21,14 @@ puter.apps.get(name, options)
|
||||
|
||||
The name of the app to get.
|
||||
|
||||
### options (optional)
|
||||
#### `options` (Object) (optional)
|
||||
|
||||
An object containing the following properties:
|
||||
|
||||
- `stats_period` (optional): A string representing the period for which to get the user and open count. Possible values are `today`, `yesterday`, `7d`, `30d`, `this_month`, `last_month`, `this_year`, `last_year`, `month_to_date`, `year_to_date`, `last_12_months`. Default is `all` (all time).
|
||||
|
||||
- `icon_size` (optional): An integer representing the size of the icons to return. Possible values are `null`, `16`, `32`, `64`, `128`, `256`, and `512`. Default is `null` (the original size).
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `icon_size` | `Number \| null` | An integer representing the size of the icons to return. Possible values are `null`, `16`, `32`, `64`, `128`, `256`, and `512`. Default is `null` (the original size). |
|
||||
| `stats_period` | `String` | The period for which to get the user and open count. Possible values are `today`, `yesterday`, `7d`, `30d`, `this_month`, `last_month`, `this_year`, `last_year`, `month_to_date`, `year_to_date`, `last_12_months`. Default is `all` (all time). |
|
||||
|
||||
## Return value
|
||||
|
||||
@@ -33,7 +36,7 @@ A `Promise` that will resolve to the [`App`](/Objects/app/) object with the give
|
||||
|
||||
## Examples
|
||||
|
||||
<strong class="example-title">Create a random app then get it</strong>
|
||||
### Create a random app then get it
|
||||
|
||||
```html;app-get
|
||||
<html>
|
||||
|
||||
+13
-15
@@ -4,7 +4,9 @@ description: List all apps in your Puter account.
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
Returns an array of all apps belonging to the user and that this app has access to. If the user has no apps, the array will be empty.
|
||||
Returns an array of all apps belonging to the user and that this app has access to.
|
||||
|
||||
If the user has no apps, the array will be empty.
|
||||
|
||||
## Syntax
|
||||
|
||||
@@ -19,19 +21,15 @@ puter.apps.list(options)
|
||||
|
||||
An object containing the following properties:
|
||||
|
||||
- `stats_period` (optional): A string representing the period for which to get the user and open count. Possible values are `today`, `yesterday`, `7d`, `30d`, `this_month`, `last_month`, `this_year`, `last_year`, `month_to_date`, `year_to_date`, `last_12_months`. Default is `all` (all time).
|
||||
|
||||
- `icon_size` (optional): An integer representing the size of the icons to return. Possible values are `null`, `16`, `32`, `64`, `128`, `256`, and `512`. Default is `null` (the original size).
|
||||
|
||||
- `limit` (optional): Maximum number of apps to return in a single call.
|
||||
|
||||
- `offset` (optional): Skips the given number of apps. Prefer `cursor` for paging through large lists.
|
||||
|
||||
- `cursor` (optional): Opts into paginated results. Pass `null` for the first page, then the `cursor` from each page to fetch the next one.
|
||||
|
||||
- `includeTotal` (optional): If `true`, the paginated result includes a `total` count of the user's apps.
|
||||
|
||||
- `stream` (optional): If `true`, the method returns an async iterator of page objects instead of a promise, for use with `for await ... of`. Combine with `limit` to control the page size, or `cursor` to resume from a previous page. Cannot be combined with `offset`. With `includeTotal`, only the first page carries `total`.
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `limit` | `Number` | Maximum number of apps to return in a single call. |
|
||||
| `cursor` | `String \| null` | Opts into paginated results. Pass `null` for the first page, then the `cursor` from each page to fetch the next one. |
|
||||
| `stream` | `Boolean` | If `true`, the method returns an async iterator of page objects instead of a promise, for use with `for await ... of`. Combine with `limit` to control the page size, or `cursor` to resume from a previous page. Cannot be combined with `offset`. With `includeTotal`, only the first page carries `total`. |
|
||||
| `includeTotal` | `Boolean` | If `true`, the paginated result includes a `total` count of the user's apps. |
|
||||
| `offset` | `Number` | Skips the given number of apps. Prefer `cursor` for paging through large lists. |
|
||||
| `icon_size` | `Number \| null` | An integer representing the size of the icons to return. Possible values are `null`, `16`, `32`, `64`, `128`, `256`, and `512`. Default is `null` (the original size). |
|
||||
| `stats_period` | `String` | The period for which to get the user and open count. Possible values are `today`, `yesterday`, `7d`, `30d`, `this_month`, `last_month`, `this_year`, `last_year`, `month_to_date`, `year_to_date`, `last_12_months`. Default is `all` (all time). |
|
||||
|
||||
## Return value
|
||||
|
||||
@@ -57,7 +55,7 @@ for await (const page of puter.apps.list({ stream: true })) {
|
||||
|
||||
## Examples
|
||||
|
||||
<strong class="example-title">Create 3 random apps and then list them</strong>
|
||||
### Create 3 random apps and then list them
|
||||
|
||||
```html;app-list
|
||||
<html>
|
||||
|
||||
+14
-11
@@ -17,16 +17,19 @@ The name of the app to update.
|
||||
|
||||
#### `attributes` (required)
|
||||
An object containing the attributes to update. The object can contain the following properties:
|
||||
- `name` (optional): The new name of the app. App names are unique across all Puter accounts. If any app already has this name, the promise will be rejected.
|
||||
- `indexURL` (optional): The new URL of the app's index page. This URL must be accessible to the user.
|
||||
- `title` (optional): The new title of the app.
|
||||
- `description` (optional): The new description of the app aimed at the end user.
|
||||
- `icon` (optional): The new icon of the app.
|
||||
- `maximizeOnStart` (optional): Whether the app should be maximized when it is started. Defaults to `false`.
|
||||
- `background` (optional): Whether the app should run in the background. Defaults to `false`.
|
||||
- `feedbackEnabled` (optional): Whether users can send feedback to you through [`puter.ui.showFeedbackDialog()`](/UI/showFeedbackDialog/). Omitted leaves the app's current value unchanged.
|
||||
- `filetypeAssociations` (optional): An array of strings representing the filetypes that the app can open. Defaults to `[]`. File extentions and MIME types are supported; For example, `[".txt", ".md", "application/pdf"]` would allow the app to open `.txt`, `.md`, and PDF files. At most 200 entries, each at most 60 characters.
|
||||
- `metadata` (optional): An object containing custom metadata for the app. This can be used to store arbitrary key-value pairs associated with the app. At most 16 KiB when serialized as JSON.
|
||||
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `title` | `String` | The new title of the app. |
|
||||
| `indexURL` | `String` | The new URL of the app's index page. This URL must be accessible to the user. |
|
||||
| `description` | `String` | The new description of the app aimed at the end user. |
|
||||
| `icon` | `String` | The new icon of the app. |
|
||||
| `name` | `String` | The new name of the app. App names are unique across all Puter accounts. If any app already has this name, the promise will be rejected. |
|
||||
| `filetypeAssociations` | `Array<String>` | An array of strings representing the filetypes that the app can open. Defaults to `[]`. File extensions and MIME types are supported. For example, `[".txt", ".md", "application/pdf"]` would allow the app to open `.txt`, `.md`, and PDF files. At most 200 entries, each at most 60 characters. |
|
||||
| `maximizeOnStart` | `Boolean` | Whether the app should be maximized when it is started. Defaults to `false`. |
|
||||
| `background` | `Boolean` | Whether the app should run in the background. Defaults to `false`. |
|
||||
| `feedbackEnabled` | `Boolean` | Whether users can send feedback to you through [`puter.ui.showFeedbackDialog()`](/UI/showFeedbackDialog/). Omitted leaves the app's current value unchanged. |
|
||||
| `metadata` | `Object` | An object containing custom metadata for the app. This can be used to store arbitrary key-value pairs associated with the app. At most 16 KiB when serialized as JSON. |
|
||||
|
||||
Changing `indexURL` can merge the app into an existing app for that hosted origin. The returned app can have a different UID. The deleted source app's KV data and KV shares are removed. Data already belonging to the surviving app is retained.
|
||||
|
||||
@@ -35,7 +38,7 @@ A `Promise` that will resolve to the [`App`](/Objects/app/) object that was upda
|
||||
|
||||
## Examples
|
||||
|
||||
<strong class="example-title">Create a random app then change its title</strong>
|
||||
### Create a random app then change its title
|
||||
|
||||
```html;app-update
|
||||
<html>
|
||||
|
||||
@@ -98,8 +98,8 @@ This is essential for users to access the various Puter.js APIs integrated into
|
||||
These authentication features are supported out of the box when using Puter.js:
|
||||
|
||||
- **[`puter.auth.signIn()`](/Auth/signIn/)** - Sign in a user
|
||||
- **[`puter.auth.signOut()`](/Auth/signOut/)** - Sign out the current user
|
||||
- **[`puter.auth.isSignedIn()`](/Auth/isSignedIn/)** - Check if a user is signed in
|
||||
- **[`puter.auth.signOut()`](/Auth/signOut/)** - Sign out the current user
|
||||
- **[`puter.auth.getUser()`](/Auth/getUser/)** - Get information about the current user
|
||||
- **[`puter.auth.getProfile()`](/Auth/getProfile/)** - Get a user's profile, when available
|
||||
- **[`puter.auth.getProfilePicture()`](/Auth/getProfilePicture/)** - Get a user's profile picture, when available
|
||||
|
||||
@@ -29,7 +29,9 @@ The id of the application.
|
||||
|
||||
A `Promise` that resolves to a [`DetailedAppUsage`](/Objects/detailedappusage) object containing resource usage statistics for the given application.
|
||||
|
||||
## Example
|
||||
## Examples
|
||||
|
||||
### Get an app's usage
|
||||
|
||||
```html
|
||||
<html>
|
||||
|
||||
@@ -26,7 +26,9 @@ None
|
||||
|
||||
A `Promise` that resolves to a [`MonthlyUsage`](/Objects/monthlyusage) object containing the user's monthly usage information.
|
||||
|
||||
## Example
|
||||
## Examples
|
||||
|
||||
### Get the user's monthly usage
|
||||
|
||||
```html;auth-get-monthly-usage
|
||||
<html>
|
||||
|
||||
@@ -4,7 +4,9 @@ description: Read a user's profile, returning null when it is not available.
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
Returns a user's profile: their picture, display name, and bio. Users set these on the [Account page](https://puter.com/dashboard#account) of their Puter dashboard. A user can always read their own profile. Another user's profile is available only while that user is on a paid plan; otherwise the method resolves to `null`, the same as for a user who does not exist.
|
||||
Returns a user's profile, which holds their picture, display name, and bio.
|
||||
|
||||
Users set these on the [Account page](https://puter.com/dashboard#account) of their Puter dashboard. A user can always read their own profile. Another user's profile is available only while that user is on a paid plan; otherwise the method resolves to `null`, the same as for a user who does not exist.
|
||||
|
||||
## Syntax
|
||||
|
||||
@@ -15,7 +17,7 @@ puter.auth.getProfile(username);
|
||||
|
||||
## Parameters
|
||||
|
||||
### `username` (optional)
|
||||
#### `username` (optional)
|
||||
|
||||
The username to look up. Defaults to the signed-in user.
|
||||
|
||||
@@ -25,7 +27,9 @@ A promise that resolves to a [`UserProfile`](/Objects/userprofile) object, or `n
|
||||
|
||||
When signed out with no `username`, it resolves to `null` without opening a sign-in prompt.
|
||||
|
||||
## Example
|
||||
## Examples
|
||||
|
||||
### Show the user's profile
|
||||
|
||||
```html
|
||||
<html>
|
||||
|
||||
@@ -4,7 +4,9 @@ description: Read a user's profile picture, returning null when unavailable.
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
Returns the `picture` field of a user's profile (see [`puter.auth.getProfile()`](/Auth/getProfile/)). A user can always read their own picture. Another user's picture is available only while that user is on a paid plan.
|
||||
Returns the `picture` field of a user's profile (see [`puter.auth.getProfile()`](/Auth/getProfile/)).
|
||||
|
||||
A user can always read their own picture. Another user's picture is available only while that user is on a paid plan.
|
||||
|
||||
## Syntax
|
||||
|
||||
@@ -15,7 +17,7 @@ puter.auth.getProfilePicture(username);
|
||||
|
||||
## Parameters
|
||||
|
||||
### `username` (optional)
|
||||
#### `username` (optional)
|
||||
|
||||
The username to look up. Defaults to the signed-in user.
|
||||
|
||||
@@ -27,7 +29,9 @@ Missing users, profiles that are not public to the caller, profiles with no pict
|
||||
|
||||
When signed out with no `username`, it resolves to `null` without opening a sign-in prompt.
|
||||
|
||||
## Example
|
||||
## Examples
|
||||
|
||||
### Show the user's profile picture
|
||||
|
||||
```html
|
||||
<html>
|
||||
|
||||
@@ -20,7 +20,9 @@ None
|
||||
|
||||
A promise that resolves to a [`User`](/Objects/user) object containing the user's basic information.
|
||||
|
||||
## Example
|
||||
## Examples
|
||||
|
||||
### Get the signed-in user
|
||||
|
||||
```html;auth-get-user
|
||||
<html>
|
||||
|
||||
@@ -20,7 +20,9 @@ None
|
||||
|
||||
Returns `true` if the user is signed in, `false` otherwise.
|
||||
|
||||
## Example
|
||||
## Examples
|
||||
|
||||
### Check whether the user is signed in
|
||||
|
||||
```html;auth-is-signed-in
|
||||
<html>
|
||||
|
||||
@@ -4,7 +4,9 @@ description: Initiate sign in process in your application with user's Puter acco
|
||||
platforms: [websites]
|
||||
---
|
||||
|
||||
Initiates the sign in process for the user. This will open a popup window with the appropriate authentication method. Puter automatically handles the authentication process and will resolve the promise when the user has signed in.
|
||||
Initiates the sign in process for the user.
|
||||
|
||||
This will open a popup window with the appropriate authentication method. Puter automatically handles the authentication process and will resolve the promise when the user has signed in. Puter's own apps on puter.com subdomains show the sign-in in a frame over the page instead, and open the popup when the user picks a sign-in provider such as Google.
|
||||
|
||||
It is important to note that all essential methods in Puter handle authentication automatically. This method is only necessary if you want to handle authentication manually, for example if you want to build your own custom authentication flow.
|
||||
|
||||
@@ -27,11 +29,12 @@ puter.auth.signIn(options)
|
||||
|
||||
#### `options` (optional)
|
||||
|
||||
`options` is an object with the following properties:
|
||||
An object with the following properties:
|
||||
|
||||
- `attempt_temp_user_creation`: A boolean value that indicates whether to Puter should automatically create a temporary user. This is useful if you want to quickly onboard a user without requiring them to sign up. They can always sign up later if they want to.
|
||||
|
||||
- `request_auth`: A boolean value that asks the popup to let the user re-pick their account, even when your site already holds a token for them. Puter otherwise skips that prompt for a site it has seen before. Useful for an explicit "switch account" button.
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `attempt_temp_user_creation` | `Boolean` | Whether Puter should automatically create a temporary user. This is useful if you want to quickly onboard a user without requiring them to sign up. They can always sign up later if they want to. |
|
||||
| `request_auth` | `Boolean` | Asks the popup to let the user re-pick their account, even when your site already holds a token for them. Puter otherwise skips that prompt for a site it has seen before. Useful for an explicit "switch account" button. |
|
||||
|
||||
Each call completes from its own sign-in window, including when multiple calls run at once.
|
||||
|
||||
@@ -39,7 +42,7 @@ Each call completes from its own sign-in window, including when multiple calls r
|
||||
|
||||
A `Promise` that will resolve to a [`SignInResult`](/Objects/signinresult/) object when the user has signed in.
|
||||
|
||||
## Rejection
|
||||
## Errors
|
||||
|
||||
The promise will reject with an object containing an `error` code and a human-readable `msg` in the following cases:
|
||||
|
||||
@@ -53,7 +56,9 @@ The promise will reject with an object containing an `error` code and a human-re
|
||||
|
||||
The promise may also reject with the failure response returned by the authentication window itself.
|
||||
|
||||
## Example
|
||||
## Examples
|
||||
|
||||
### Sign in from a button click
|
||||
|
||||
```html;auth-sign-in
|
||||
<html>
|
||||
|
||||
@@ -22,7 +22,9 @@ None
|
||||
|
||||
A `Promise` that resolves with nothing once the user is signed out and, on a website, this browser's push subscription for your app is ended. It doesn't reject.
|
||||
|
||||
## Example
|
||||
## Examples
|
||||
|
||||
### Sign the user out
|
||||
|
||||
```html;auth-sign-out
|
||||
<html>
|
||||
|
||||
@@ -5,7 +5,9 @@ platforms: [websites, apps, nodejs, workers]
|
||||
draft: true
|
||||
---
|
||||
|
||||
Changes fields on the signed-in user's profile and returns the updated profile. Fields left out of the patch are left as they are; `null` clears a field.
|
||||
Changes fields on the signed-in user's profile and returns the updated profile.
|
||||
|
||||
Fields left out of the patch are left as they are; `null` clears a field.
|
||||
|
||||
## Syntax
|
||||
|
||||
@@ -15,19 +17,27 @@ puter.auth.updateProfile(patch);
|
||||
|
||||
## Parameters
|
||||
|
||||
### `patch` (required)
|
||||
#### `patch` (required)
|
||||
|
||||
An object with any of these fields:
|
||||
|
||||
- `picture` (String or `null`): a base64 image data URL such as `data:image/png;base64,...`. At most 512 KiB.
|
||||
- `displayName` (String or `null`): at most 64 characters. Surrounding whitespace is removed.
|
||||
- `bio` (String or `null`): at most 280 characters. Surrounding whitespace is removed.
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `displayName` | `String \| null` | The user's display name, at most 64 characters. Surrounding whitespace is removed. |
|
||||
| `picture` | `String \| null` | A base64 image data URL such as `data:image/png;base64,...`. At most 512 KiB. |
|
||||
| `bio` | `String \| null` | The user's bio, at most 280 characters. Surrounding whitespace is removed. |
|
||||
|
||||
Any other field is rejected.
|
||||
|
||||
## Return value
|
||||
|
||||
A promise that resolves to the stored [`UserProfile`](/Objects/userprofile). It rejects with an object carrying a `code` and `message` when the patch is not accepted:
|
||||
A promise that resolves to the stored [`UserProfile`](/Objects/userprofile).
|
||||
|
||||
The profile belongs to the account, so the call must be made with a user session; an app acting on a user's behalf cannot change it.
|
||||
|
||||
## Errors
|
||||
|
||||
The promise rejects with an object carrying a `code` and `message` when the patch is not accepted:
|
||||
|
||||
| `code` | Meaning |
|
||||
| --------------------------- | ----------------------------------------------------- |
|
||||
@@ -37,9 +47,9 @@ A promise that resolves to the stored [`UserProfile`](/Objects/userprofile). It
|
||||
| `profile_picture_too_large` | `picture` is over the size limit |
|
||||
| `profile_field_too_long` | `displayName` or `bio` is over its length limit |
|
||||
|
||||
The profile belongs to the account, so the call must be made with a user session; an app acting on a user's behalf cannot change it.
|
||||
## Examples
|
||||
|
||||
## Example
|
||||
### Change the user's display name
|
||||
|
||||
```html;auth-update-profile
|
||||
<html>
|
||||
|
||||
@@ -88,6 +88,6 @@ router.post('/notify', async ({ request, user }) => {
|
||||
|
||||
## Functions
|
||||
|
||||
- **[`puter.email.sendTransactional()`](/Email/sendTransactional/)** - Send a transactional email from your app
|
||||
- **[`puter.email.list()`](/Email/list/)** - List the messages in the user's mailbox, newest first
|
||||
- **[`puter.email.get()`](/Email/get/)** - Fetch and parse one message, attachments included
|
||||
- **[`puter.email.sendTransactional()`](/Email/sendTransactional/)** - Send a transactional email from your app
|
||||
@@ -4,7 +4,9 @@ description: Fetch and parse one message from the user's mailbox by its id.
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
Reads one message by the `id` a listing returned. The whole raw message is downloaded and parsed, so the result carries the full subject, sender and recipients, the text and HTML bodies, every header, and each attachment with its bytes. Pass `raw: true` to get the unparsed `message/rfc822` bytes as a `Blob` instead.
|
||||
Reads one message by the `id` a listing returned.
|
||||
|
||||
The whole raw message is downloaded and parsed, so the result carries the full subject, sender and recipients, the text and HTML bodies, every header, and each attachment with its bytes. Pass `raw: true` to get the unparsed `message/rfc822` bytes as a `Blob` instead.
|
||||
|
||||
Reading the mailbox requires read access to the `~/.mail` folder, the same grant [`puter.email.list()`](/Email/list/) needs.
|
||||
|
||||
@@ -23,9 +25,11 @@ The message id from `puter.email.list()`. Looks up the inbox.
|
||||
|
||||
#### `options` (Object)
|
||||
|
||||
- `id` (String) (required): The message id.
|
||||
- `folder` (String): `'inbox'` or `'sent'`. Default `'inbox'`. A message id is unique within a folder, so a sent message must be read with `folder: 'sent'`.
|
||||
- `raw` (Boolean): When `true`, resolve with the raw message `Blob` and skip parsing. Default `false`.
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `id` | `String` | Required. The message id. |
|
||||
| `folder` | `String` | Either `'inbox'` or `'sent'`. Default `'inbox'`. A message id is unique within a folder, so a sent message must be read with `folder: 'sent'`. |
|
||||
| `raw` | `Boolean` | When `true`, resolve with the raw message `Blob` and skip parsing. Default `false`. |
|
||||
|
||||
## Return value
|
||||
|
||||
@@ -45,7 +49,7 @@ Rejects with an object carrying a `code`:
|
||||
|
||||
## Examples
|
||||
|
||||
<strong class="example-title">Read the newest message in the inbox</strong>
|
||||
### Read the newest message in the inbox
|
||||
|
||||
```html
|
||||
<html>
|
||||
@@ -66,7 +70,7 @@ Rejects with an object carrying a `code`:
|
||||
</html>
|
||||
```
|
||||
|
||||
<strong class="example-title">Save every attachment of a message to the user's Documents</strong>
|
||||
### Save every attachment of a message to the user's Documents
|
||||
|
||||
```html
|
||||
<html>
|
||||
@@ -89,7 +93,7 @@ Rejects with an object carrying a `code`:
|
||||
</html>
|
||||
```
|
||||
|
||||
<strong class="example-title">Download the raw message</strong>
|
||||
### Download the raw message
|
||||
|
||||
```html
|
||||
<html>
|
||||
|
||||
@@ -4,7 +4,9 @@ description: List the messages in the user's mailbox, newest first, one page at
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
Lists the messages in one folder of the user's mailbox, newest first. Every call returns one page of `items` plus, while more pages exist, a `cursor` to pass back for the next one. Listing reads the mailbox's folder structure only, so it never downloads a message: each item carries the subject, date, and size, and `puter.email.get()` fetches the rest.
|
||||
Lists the messages in one folder of the user's mailbox, newest first.
|
||||
|
||||
Every call returns one page of `items` plus, while more pages exist, a `cursor` to pass back for the next one. Listing reads the mailbox's folder structure only, so it never downloads a message: each item carries the subject, date, and size, and `puter.email.get()` fetches the rest.
|
||||
|
||||
The mailbox belongs to the user, so it is protected like the rest of their files. An app reads it only after the user grants read access to the `~/.mail` folder. To ask for it, call [`puter.perms.request()`](/Perms/request/):
|
||||
|
||||
@@ -23,10 +25,12 @@ puter.email.list(options)
|
||||
|
||||
#### `options` (Object) (optional)
|
||||
|
||||
- `folder` (String): `'inbox'` for mail received at the user's address, `'sent'` for their outgoing copies. Default `'inbox'`.
|
||||
- `limit` (Number): Maximum messages per page. Default `50`, capped at `1000`.
|
||||
- `cursor` (String | null): The `cursor` from the previous page. `null` or absent fetches the first page.
|
||||
- `stream` (Boolean): When `true`, returns an async iterator of pages for `for await ... of` instead of a single page.
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `limit` | `Number` | Maximum messages per page. Default `50`, capped at `1000`. |
|
||||
| `folder` | `String` | Use `'inbox'` for mail received at the user's address, or `'sent'` for their outgoing copies. Default `'inbox'`. |
|
||||
| `cursor` | `String \| null` | The `cursor` from the previous page. Pass `null` or leave it out to fetch the first page. |
|
||||
| `stream` | `Boolean` | When `true`, returns an async iterator of pages for `for await ... of` instead of a single page. |
|
||||
|
||||
`offset` and `includeTotal` are not supported for mailboxes; passing either rejects with `invalid_request`.
|
||||
|
||||
@@ -45,7 +49,7 @@ Rejects with an object carrying a `code`:
|
||||
|
||||
## Examples
|
||||
|
||||
<strong class="example-title">List the ten newest messages</strong>
|
||||
### List the ten newest messages
|
||||
|
||||
```html
|
||||
<html>
|
||||
@@ -64,7 +68,7 @@ Rejects with an object carrying a `code`:
|
||||
</html>
|
||||
```
|
||||
|
||||
<strong class="example-title">Walk the whole sent folder with a cursor</strong>
|
||||
### Walk the whole sent folder with a cursor
|
||||
|
||||
```html
|
||||
<html>
|
||||
@@ -86,7 +90,7 @@ Rejects with an object carrying a `code`:
|
||||
</html>
|
||||
```
|
||||
|
||||
<strong class="example-title">Stream pages</strong>
|
||||
### Stream pages
|
||||
|
||||
```html
|
||||
<html>
|
||||
|
||||
@@ -4,7 +4,9 @@ description: Send a transactional email from your app — authorized by a worker
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
Sends one transactional email from your app. Every send must be authorized by a worker: call it from inside a worker, or pass the worker's token as `emailAccessToken`. The account that makes the call is the one billed and rate-limited.
|
||||
Sends one transactional email from your app.
|
||||
|
||||
Every send must be authorized by a worker: call it from inside a worker, or pass the worker's token as `emailAccessToken`. The account that makes the call is the one billed and rate-limited.
|
||||
|
||||
<div class="info">Transactional email is available to accounts on a paid plan for now. A call from a free account fails with <code>402</code> <code>subscription_required</code>. Under the <a href="/user-pays-model/">User-Pays Model</a> that is the plan of the <em>calling</em> account — your users when your app sends with their session, your own when the worker sends directly.</div>
|
||||
|
||||
@@ -31,21 +33,25 @@ A plain-text body. Shorthand for `text`; use the options form for anything else.
|
||||
|
||||
#### `options` (Object)
|
||||
|
||||
- `to` (String | Array) (required) - Recipient address(es).
|
||||
- `subject` (String) (required) - As above.
|
||||
- `text` (String) - Plain-text body. At least one of `text` and `html` is required.
|
||||
- `html` (String) - HTML body. An inline image is shown with `<img src="cid:...">` — see `cid` under `attachments`.
|
||||
- `cc` (String | Array) - Carbon-copy recipient(s).
|
||||
- `bcc` (String | Array) - Blind-copy recipient(s). At most **10** recipients across `to`, `cc` and `bcc`.
|
||||
- `replyTo` (String) - Where replies go. Defaults to the confirmed account email of the worker's owner when the worker has no app binding, or its owner owns the bound app and that app wasn't created by another app. Otherwise, or while the address is unconfirmed, the message goes out without a Reply-To.
|
||||
- `emailAccessToken` (String) - A worker's auth token — inside a worker, `me.puter.authToken`. Required when the caller is not itself a worker. The caller stays the billed and rate-limited identity.
|
||||
- `attachments` (Array) - Up to **10** attachments, **25 MB** in total. Each is an object:
|
||||
- `filename` (String) - Required with `content`; defaults to the file's name for a `path`/`uid` attachment.
|
||||
- `content` (String) - The file body as base64. Mutually exclusive with `path`/`uid`.
|
||||
- `path` (String) - A Puter path (`~/` allowed, meaning the caller's home). The file is read on the server with the caller's file permissions only, and streamed rather than uploaded — prefer this over `content` for anything larger than a few hundred kilobytes.
|
||||
- `uid` (String) - A Puter file uid, as an alternative to `path`. Read with the caller's file permissions, falling back to the worker owner's — use a uid to attach a file from the worker owner's storage.
|
||||
- `contentType` (String) - MIME type. Detected from the file for `path`/`uid` attachments.
|
||||
- `cid` (String) - Content-ID that makes the attachment an inline part of the `html` body: with `cid: 'logo'`, `<img src="cid:logo">` shows it. Printable ASCII with no whitespace or angle brackets. Inline parts are billed like any other attachment.
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `to` | `String \| Array` | Required. One recipient address, or an array of them. |
|
||||
| `subject` | `String` | Required. As above. |
|
||||
| `text` | `String` | Plain-text body. At least one of `text` and `html` is required. |
|
||||
| `html` | `String` | HTML body. An inline image is shown with `<img src="cid:...">`. See `attachments.cid`. |
|
||||
| `emailAccessToken` | `String` | A worker's auth token. Inside a worker, that's `me.puter.authToken`. Required when the caller is not itself a worker. The caller stays the billed and rate-limited identity. |
|
||||
| `attachments` | `Array` | Up to **10** attachments, **25 MB** in total. Each attachment is an object with the `attachments.*` fields in the rows that follow. |
|
||||
| `attachments.path` | `String` | A Puter path (`~/` allowed, meaning the caller's home). The file is read on the server with the caller's file permissions only, and streamed rather than uploaded. Prefer this over `content` for anything larger than a few hundred kilobytes. |
|
||||
| `attachments.uid` | `String` | A Puter file uid, as an alternative to `path`. Read with the caller's file permissions, falling back to the worker owner's. Use a uid to attach a file from the worker owner's storage. |
|
||||
| `attachments.content` | `String` | The file body as base64. Mutually exclusive with `path`/`uid`. |
|
||||
| `attachments.filename` | `String` | Required with `content`. Defaults to the file's name for a `path`/`uid` attachment. |
|
||||
| `attachments.contentType` | `String` | MIME type. Detected from the file for `path`/`uid` attachments. |
|
||||
| `attachments.cid` | `String` | Content-ID that makes the attachment an inline part of the `html` body. With `cid: 'logo'`, `<img src="cid:logo">` shows it. Printable ASCII with no whitespace or angle brackets. Inline parts are billed like any other attachment. |
|
||||
| `replyTo` | `String` | Where replies go. Defaults to the confirmed account email of the worker's owner when the worker has no app binding, or its owner owns the bound app and that app wasn't created by another app. Otherwise, or while the address is unconfirmed, the message goes out without a Reply-To. |
|
||||
| `cc` | `String \| Array` | Carbon-copy recipient(s). |
|
||||
| `bcc` | `String \| Array` | Blind-copy recipient(s). |
|
||||
|
||||
A message can have at most **10** recipients across `to`, `cc` and `bcc`.
|
||||
|
||||
## Return value
|
||||
|
||||
@@ -56,7 +62,11 @@ A `Promise` that resolves to an object:
|
||||
- `suppressed` (Array) - Recipients (lowercased) dropped because they unsubscribed from your app's mail. The message went to the others.
|
||||
- `failed` (Array) - Recipients (lowercased) whose delivery failed. Everyone else got their copy — retry with just these addresses.
|
||||
|
||||
Each recipient gets a private copy of the message, so the unsubscribe link in one copy can only act for the mailbox that received it. Per-recipient failures come back in `failed` rather than failing the call; the promise rejects only when no recipient could be delivered, or when every `to` recipient has unsubscribed (`400`).
|
||||
Each recipient gets a private copy of the message, so the unsubscribe link in one copy can only act for the mailbox that received it. Per-recipient failures come back in `failed` rather than failing the call.
|
||||
|
||||
## Errors
|
||||
|
||||
The promise rejects only when no recipient could be delivered, or when every `to` recipient has unsubscribed (`400`).
|
||||
|
||||
In case of an error, the `Promise` rejects with `{ message, code }`. Codes you may see:
|
||||
|
||||
@@ -90,7 +100,7 @@ Sends are rate-limited per calling account; the numbers are on [Rate Limits and
|
||||
|
||||
## Examples
|
||||
|
||||
<strong class="example-title">Send a plain-text notification from a worker</strong>
|
||||
### Send a plain-text notification from a worker
|
||||
|
||||
```js
|
||||
// The worker is the caller, so its owner is billed and rate-limited.
|
||||
@@ -104,7 +114,7 @@ router.post('/api/alert', async ({ request }) => {
|
||||
});
|
||||
```
|
||||
|
||||
<strong class="example-title">Let the user pay for their own sends</strong>
|
||||
### Let the user pay for their own sends
|
||||
|
||||
```js
|
||||
// The user's session makes the call; the worker's token authorizes it.
|
||||
@@ -122,7 +132,7 @@ router.post('/api/receipt', async ({ request, user }) => {
|
||||
});
|
||||
```
|
||||
|
||||
<strong class="example-title">Attach a file and show an inline image</strong>
|
||||
### Attach a file and show an inline image
|
||||
|
||||
```js
|
||||
// The uid of a file in the worker owner's storage, e.g. from
|
||||
|
||||
@@ -4,9 +4,11 @@ description: Read what a subject recorded while nothing was listening.
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
Reads the events a subject has stored, a page at a time.
|
||||
|
||||
<div class="info">The Events API is in beta. Event shapes, limits, and behavior may change between releases.</div>
|
||||
|
||||
Reads the events a subject has stored, a page at a time. A subscription only delivers while something is listening, and `fetch()` catches up on what happened while the client was closed or offline. Only `notif:` subjects (the notification mailbox) store events. Subjects for `fs:` and `kv:` are refused with `fetch_unsupported_subject`.
|
||||
A subscription only delivers while something is listening, and `fetch()` catches up on what happened while the client was closed or offline. Only `notif:` subjects (the notification mailbox) store events. Subjects for `fs:` and `kv:` are refused with `fetch_unsupported_subject`.
|
||||
|
||||
It's a plain query. Nothing is registered and no position is saved, so calling it twice returns the same result. You keep the `cursor` and pass it back.
|
||||
|
||||
@@ -25,11 +27,13 @@ puter.events.fetch(options)
|
||||
|
||||
#### `options` (Object) (required)
|
||||
|
||||
- `subject` (String) (required): What to read. Use `notif:app-user` for your app's notifications for the user, and `notif:developer` for notifications about your app, for its owner. The full form is `notif:<appId>:<audience>`, where the audience is `account`, `developer`, or `app-user`. Apps and websites never see `account` notifications, so `notif:account` returns an empty page for them.
|
||||
- `cursor` (String): The `cursor` from the previous page. Leave it off to start from the oldest notification still kept. A read from a cursor returns the notifications created after it, plus any notification [replaced](/Notifications/notify/#groups) since that page was read, even one from an earlier page.
|
||||
- `after` (String): The older name for `cursor`, still accepted. If both are given, `cursor` is used.
|
||||
- `from` (String): Where a read without a cursor starts. `'start'`, the default, starts from the oldest notification still kept. `'end'` skips what's already there: it returns no items and a `cursor` at the newest notification, and reads from that cursor return only notifications created or replaced after it. Ignored when a cursor is given.
|
||||
- `limit` (Number): Events per page. Defaults to 50, max 200.
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `subject` | `String` | Required. What to read. Use `notif:app-user` for your app's notifications for the user, and `notif:developer` for notifications about your app, for its owner. The full form is `notif:<appId>:<audience>`, where the audience is `account`, `developer`, or `app-user`. Apps and websites never see `account` notifications, so `notif:account` returns an empty page for them. |
|
||||
| `cursor` | `String` | The `cursor` from the previous page. Leave it off to start from the oldest notification still kept. A read from a cursor returns the notifications created after it, plus any notification [replaced](/Notifications/notify/#groups) since that page was read, even one from an earlier page. |
|
||||
| `limit` | `Number` | Events per page. Defaults to 50, max 200. |
|
||||
| `from` | `String` | Where a read without a cursor starts. The default, `'start'`, starts from the oldest notification still kept. With `'end'`, the read skips what's already there. It returns no items and a `cursor` at the newest notification, and reads from that cursor return only notifications created or replaced after it. Ignored when a cursor is given. |
|
||||
| `after` | `String` | The older name for `cursor`, still accepted. If both are given, `cursor` is used. |
|
||||
|
||||
## Return value
|
||||
|
||||
@@ -42,18 +46,18 @@ Each item is a notification event:
|
||||
|
||||
| Field | Type | Description |
|
||||
| --- | --- | --- |
|
||||
| `id` | String | The notification's uid. Live deliveries carry the same id, so you can drop duplicates. A replacement keeps the id, so tell it apart by `replaced` and `ts`. |
|
||||
| `subject` | String | `notif:<appId or userId>:<audience>`, the part of the mailbox it belongs to. |
|
||||
| `op` | String | Always `post`. |
|
||||
| `uid` | String | The notification's uid. |
|
||||
| `type` | String | The kind of notification, such as `share.received`, `app.worker.deployed`, or `app.message`. The types are listed below. |
|
||||
| `audience` | String | `account`, `developer`, or `app-user`. |
|
||||
| `appUid` | String \| null | The app it's about, or `null` for one that names no app. |
|
||||
| `notification` | Object | The stored payload. It always has `type`, and every type except `kv.share.received` has a `title`. Other fields depend on `type`. |
|
||||
| `replaced` | Boolean | Present and `true` when the notification replaced an earlier one with the same `uid`, as an app's notification with a [`group`](/Notifications/notify/#groups) does. |
|
||||
| `self` | Boolean | Always `true`, since a mailbox is your own. |
|
||||
| `ts` | Number | When it was created, in milliseconds since the epoch. On a replacement, when it was replaced, and the same on its live and fetched copies. |
|
||||
| `seq` | Number | Position within the page. |
|
||||
| `id` | `String` | The notification's uid. Live deliveries carry the same id, so you can drop duplicates. A replacement keeps the id, so tell it apart by `replaced` and `ts`. |
|
||||
| `subject` | `String` | `notif:<appId or userId>:<audience>`, the part of the mailbox it belongs to. |
|
||||
| `op` | `String` | Always `post`. |
|
||||
| `uid` | `String` | The notification's uid. |
|
||||
| `type` | `String` | The kind of notification, such as `share.received`, `app.worker.deployed`, or `app.message`. The types are listed below. |
|
||||
| `audience` | `String` | `account`, `developer`, or `app-user`. |
|
||||
| `appUid` | `String \| null` | The app it's about, or `null` for one that names no app. |
|
||||
| `notification` | `Object` | The stored payload. It always has `type`, and every type except `kv.share.received` has a `title`. Other fields depend on `type`. |
|
||||
| `replaced` | `Boolean` | Present and `true` when the notification replaced an earlier one with the same `uid`, as an app's notification with a [`group`](/Notifications/notify/#groups) does. |
|
||||
| `self` | `Boolean` | Always `true`, since a mailbox is your own. |
|
||||
| `ts` | `Number` | When it was created, in milliseconds since the epoch. On a replacement, when it was replaced, and the same on its live and fetched copies. |
|
||||
| `seq` | `Number` | Position within the page. |
|
||||
|
||||
Besides `type` and `title`, `notification` carries these fields for each type:
|
||||
|
||||
@@ -74,6 +78,8 @@ Apps see `developer` notifications only for apps the user owns. Asking for a par
|
||||
|
||||
Notifications are kept for the server's retention window. A client away for longer starts from what's left.
|
||||
|
||||
## Errors
|
||||
|
||||
The promise rejects with `{ message, code }`:
|
||||
|
||||
| `code` | Meaning |
|
||||
@@ -89,7 +95,7 @@ The promise rejects with `{ message, code }`:
|
||||
|
||||
## Examples
|
||||
|
||||
<strong class="example-title">Catch up on everything missed</strong>
|
||||
### Catch up on everything missed
|
||||
|
||||
```html;events-fetch
|
||||
<html>
|
||||
@@ -118,7 +124,7 @@ The promise rejects with `{ message, code }`:
|
||||
</html>
|
||||
```
|
||||
|
||||
<strong class="example-title">Read the missed ones, then keep listening</strong>
|
||||
### Read the missed ones, then keep listening
|
||||
|
||||
```html;events-fetch-then-listen
|
||||
<html>
|
||||
|
||||
Loaded 100 of 408 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user