mirror of
https://github.com/HeyPuter/puter.git
synced 2026-11-13 20:12:57 -10:00
Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f3542a6aa3 | ||
|
|
5a59db7b6b |
No files matched your search
@@ -53,7 +53,7 @@ jobs:
|
||||
# step above fails; the tests skip themselves without the key.
|
||||
- name: Run live realtime conversation tests
|
||||
if: ${{ !cancelled() }}
|
||||
run: npm run setupExtensions && (cd src/puter-js && npm run build) && npx vitest run --config src/puter-js/tests/api/vitest.config.ts src/puter-js/tests/api/realtime/realtimeLive.test.ts
|
||||
run: npm run setupExtensions && (cd src/puter-js && npm run build) && npx vitest run --config src/puter-js/tests/api/vitest.config.ts src/puter-js/tests/api/realtime/convoLive.test.ts
|
||||
env:
|
||||
CI: 'true'
|
||||
PUTER_TEST_AI_OPENAI_API_KEY: ${{ secrets.PUTER_TEST_AI_OPENAI_API_KEY }}
|
||||
@@ -176,10 +176,7 @@ jobs:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v7
|
||||
|
||||
# Missing when the PR leg failed; `coverage (pr)` already reports that.
|
||||
- name: Download PR coverage
|
||||
id: pr-coverage
|
||||
continue-on-error: true
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
name: puterjs-coverage-pr
|
||||
@@ -189,7 +186,6 @@ jobs:
|
||||
# base run) — then report without the comparison column.
|
||||
- name: Download base coverage
|
||||
id: base-coverage
|
||||
if: steps.pr-coverage.outcome == 'success'
|
||||
continue-on-error: true
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
@@ -197,7 +193,7 @@ jobs:
|
||||
path: coverage-base
|
||||
|
||||
- name: Report coverage on PR
|
||||
if: steps.pr-coverage.outcome == 'success' && steps.base-coverage.outcome == 'success'
|
||||
if: steps.base-coverage.outcome == 'success'
|
||||
uses: davelosert/vitest-coverage-report-action@v2
|
||||
with:
|
||||
name: puter.js SDK
|
||||
@@ -206,7 +202,7 @@ jobs:
|
||||
json-summary-compare-path: coverage-base/coverage-summary.json
|
||||
|
||||
- name: Report coverage on PR (no base comparison)
|
||||
if: steps.pr-coverage.outcome == 'success' && steps.base-coverage.outcome != 'success'
|
||||
if: steps.base-coverage.outcome != 'success'
|
||||
uses: davelosert/vitest-coverage-report-action@v2
|
||||
with:
|
||||
name: puter.js SDK
|
||||
|
||||
@@ -159,42 +159,13 @@ The reference has two kinds of page:
|
||||
|
||||
An overview page answers "what is this API and is it what I need?", and a function page answers "how exactly does this call behave?". A change to one function usually only touches that function's page, so a new `kv.list()` option goes in `KV/list.md` without a matching update to `KV.md`. Don't copy a function's options, limits, or edge cases onto the overview page for completeness. Where a detail goes is a judgment call, not a hard rule: update the overview when a change affects how someone would describe the API as a whole.
|
||||
|
||||
When you add a page, add it to [src/docs/src/sidebar.js](src/docs/src/sidebar.js) under its area, or it won't show up in the navigation.
|
||||
|
||||
**Sidebar order.** Order the API areas in the sidebar by how often developers use them, most common first. Overview stays first and Objects stays last. Inside an area, order the pages by use too, and keep pages that make up one feature next to each other, such as `share()`, `unshare()`, `listShared()`, and `listSharedByMe()`. A group goes where its most-used page would go. A helper that only supports another function, such as `listModels()` for `chat()`, goes by its own use. Put create, list, get, update, and delete in that order, and constants such as `MAX_KEY_SIZE` last. The function list on the area's overview page uses the sidebar's order. When you add a runnable example, add its HTML to `src/docs/src/playground/examples/<slug>.html`, register the slug in [src/docs/src/examples.js](src/docs/src/examples.js), and link the code block to it by putting the slug after the language with a semicolon (` ```html;kv-set `).
|
||||
|
||||
### Function page layout
|
||||
|
||||
A function page opens with one sentence that says what the function does, in its own paragraph, such as "Given a prompt, generate an image using AI." Put any more detail in the paragraphs after it.
|
||||
|
||||
After the intro, a function page has these `##` sections, in this order:
|
||||
|
||||
1. Syntax
|
||||
2. Parameters
|
||||
3. Return value
|
||||
4. Errors, if the function has any
|
||||
5. Sections for specific topics, if the page needs them, with `## Model-specific features` last
|
||||
6. Examples
|
||||
|
||||
[chat.md](src/docs/src/AI/chat.md) follows this layout.
|
||||
|
||||
- **Options as a table.** Document an options object as a table with Option, Type, and Description columns. Give each option its own row, even when two options go together, such as `width` and `height`. Order the rows by how often developers use them, most common first, without group rows. In chat.md that's the core options, then reasoning, tools, response handling, request control, and output.
|
||||
- **Common options in the table, model-specific behavior in its own section.** A page whose function works with many models or providers, such as [chat.md](src/docs/src/AI/chat.md) and [txt2img.md](src/docs/src/AI/txt2img.md), splits its options this way:
|
||||
- **The options table** under Parameters lists the options that work with every model, and describes each one the way it works for every model.
|
||||
- **`## Model-specific features`**, the last section before Examples, covers everything else. It has a `###` for each model family or provider, named after it, such as "OpenAI models" or "Cloudflare models". Under it go the options only some models accept, and the common options a model handles differently, such as a narrower range, other allowed values, or an option it ignores.
|
||||
- **Write there only what's different.** A `###` lists an option only when that model or provider adds it or handles it differently. An option that works the same everywhere stays out, even if the provider supports it, and a model or provider with nothing different gets no `###`.
|
||||
- **Model-specific features is the only place for model-specific details.** Every limit, format, allowed value, default, price, or quirk that applies to some models goes there, in that model's or provider's `###`, and nowhere else. The options table and topic sections (such as Image inputs or Video input) describe what's true for every model and link to Model-specific features for the rest. Don't add a table or sentence there that compares providers or repeats a model's limits. Each copy is another source of truth to keep in sync, and copies drift when a model changes.
|
||||
- **Link the two.** Directly under the options table, put `Some models accept additional options or handle these options differently. See [Model-specific features](#model-specific-features).` A row whose option behaves differently on some models can also end with `See [Model-specific features](#model-specific-features).`
|
||||
- **A long topic gets its own `##` section.** When explaining an option or feature takes more than a sentence or two, such as function calling or output dimensions, give it a section between Errors and Examples and link to it from the options table. This keeps the reference sections short.
|
||||
- **Usage patterns are examples.** A way of using the API, such as stopping a stream, goes in a runnable example that the relevant section links to ("See the [stop a stream example](/playground/ai-chat-stop-stream/)."), not in a section of its own.
|
||||
- **Each example has a `###` heading** under `## Examples`, so every example shows in the table of contents.
|
||||
When you add a page, add it to [src/docs/src/sidebar.js](src/docs/src/sidebar.js) under its area, or it won't show up in the navigation. When you add a runnable example, add its HTML to `src/docs/src/playground/examples/<slug>.html`, register the slug in [src/docs/src/examples.js](src/docs/src/examples.js), and link the code block to it by putting the slug after the language with a semicolon (` ```html;kv-set `).
|
||||
|
||||
### Writing docs
|
||||
|
||||
These apply to reference pages and recipes alike.
|
||||
|
||||
- **Document the developer-facing side only.** The reader is building on top of puter.js, so cover what affects their code: signatures, defaults, return shapes, errors, limits, and behavior they can observe. Leave out internal workings (which service handles the call, storage layout, caching, retries) unless they change what the developer has to do. [rate-limits-and-quotas.md](src/docs/src/rate-limits-and-quotas.md) is the exception: it publishes every limit, including timeouts and caps that come from how the backend works.
|
||||
- **Document Puter as it runs on puter.com.** The docs are for developers building on production Puter. Leave out behavior that only happens on a self-hosted or differently configured server, such as a provider that isn't set up there.
|
||||
- **Describe the current state.** Docs aren't a changelog. Write how the API behaves now, without "now", "no longer", "existing code is unaffected", or other wording that compares it to an older version.
|
||||
- **Verify before you document.** Run the calls you describe against a running Puter instead of inferring behavior from the source.
|
||||
- **Write plainly.** Use the literal words a developer would say out loud, with contractions and everyday words. No idioms or figurative verbs ("walk away from something shared with you" → "remove your own access", "nothing to page through" → "without pagination"). No formal register ("present only while more pages remain" → "includes a `cursor` if there are more pages to load"). Prefer dev terms such as atomic, concurrent, and insertion order.
|
||||
@@ -202,7 +173,6 @@ These apply to reference pages and recipes alike.
|
||||
- **Describe behavior directly.** No quoted inner-voice phrases. Write "`isNew` is `true` when the person did not have access before", not "this tells 'I just gave this to someone' from 'they already had it'".
|
||||
- **Keep references easy to follow.** Name what a list or code block belongs to in the sentence directly above it. Name a thing once and keep that name. Introduce an actor as "a user" before giving them a role: "A user can create a team. The user who creates the team becomes its owner."
|
||||
- **Sentence mechanics.** Never start a sentence with inline code: write "To store data, use the [`set()`](/KV/set/) method." Link every inline mention of an API to its page, every time. No em dashes, and no colons or semicolons mid-sentence; a colon that introduces a code block is fine.
|
||||
- **No `##` or `###` headings inside tabs.** The table of contents lists every `##` and `###` heading on the page, including the ones in a hidden tab (an `example-content` block switched by the `example-group` pills), so its links lead to content the reader can't see. Put the heading above the tabs, or use `####` inside a tab, which the table of contents skips.
|
||||
- **Show the correct way.** No "don't" and no wrong-way-first examples, in recipes and playground examples alike. Mention a wrong approach only when developers already hit it in practice, and state how it fails as a fact: "`add()` reads a plain object as a path map, so the call rejects with …".
|
||||
|
||||
### Recipes
|
||||
|
||||
@@ -1,305 +0,0 @@
|
||||
# Embedded app browser
|
||||
|
||||
`GET /embed/apps` on the deployment's main GUI origin renders the same app browser
|
||||
as the dashboard's Apps tab, without the desktop or dashboard navigation.
|
||||
|
||||
It supports search, paging, saved folders, and saved ordering. It uses the same
|
||||
installed-app and recommendation data as the dashboard, including the user's
|
||||
removed recommendations. Editing, adding, reordering, renaming, and uninstalling
|
||||
are disabled. Selecting an app opens it in a new browser tab at its own URL, the
|
||||
same way the dashboard does, and the host page stays open. The app's token isn't
|
||||
put in the URL: the app's SDK asks the embed for it once the tab loads, so the
|
||||
embed must still be open then. If the iframe is removed first, the app falls back
|
||||
to its normal sign-in. Apps that need the desktop (built-in and godmode apps,
|
||||
private apps the user can't open, or a failed token request) open `/app/<name>`
|
||||
on the GUI origin, cut off from the embed. Website shortcuts open their external
|
||||
destination with no opener.
|
||||
The embed follows the OS's light or dark setting. A host with a fixed theme adds
|
||||
`theme=dark` or `theme=light` to the fragment (`#user=<uuid>&theme=dark`), since
|
||||
Chromium doesn't pass an iframe's `color-scheme` on to the page inside it.
|
||||
Apps opened from the embed carry `utm_source=puter.com&utm_medium=referral&utm_campaign=app_browser_embed`;
|
||||
the desktop and dashboard send `utm_campaign=desktop` and `utm_campaign=dashboard`.
|
||||
|
||||
## Session and framing
|
||||
|
||||
The embed shows the GUI session only to the user the host page is signed in as.
|
||||
The host names that user's `uuid` (from `puter.auth.getUser()`) in the iframe
|
||||
URL's fragment, as `/embed/apps#user=<uuid>`. The embed reads the session, asks
|
||||
`/whoami` which user it belongs to, and shows the signed-out view unless the two
|
||||
match. So a host that names no user, a malformed one, or a user other than the
|
||||
session's, gets the signed-out view, and so does a session the API refuses. A
|
||||
failed `/whoami` check shows the error state. The fragment never reaches the
|
||||
server and can only narrow what the embed shows: naming the session's own user
|
||||
gives the same view as a direct visit to the GUI. Changing the fragment of a
|
||||
loaded embed reloads it.
|
||||
|
||||
The embed reads the existing GUI session from local storage on its own origin.
|
||||
If that storage holds no session, the embed gets the session from the GUI origin's session cookie through
|
||||
`/get-gui-token`. This matters in Safari, which gives a framed origin separate
|
||||
local storage under each top-level origin, so even a sibling subdomain's iframe
|
||||
can't see the GUI's stored token. The session cookie is still sent, because the
|
||||
host is on the same site. The embed doesn't write that token back to storage. It
|
||||
does not create accounts, show a login dialog, or accept a token from its parent.
|
||||
Being signed in on a subdomain alone does not create a session on the main GUI
|
||||
origin.
|
||||
|
||||
Without a session it can show, the embed shows the recommended apps (`RecommendedAppsService`)
|
||||
that the server put in the page, and nothing else: no installed apps, folders, or
|
||||
saved order, and no puter.js. Selecting one opens it at the same URL a signed-in
|
||||
launch uses, with no opener and no token on offer, so the app signs in on its own.
|
||||
Apps that need the desktop open `/app/<name>`. If the page carries no recommended
|
||||
apps, or a load fails, the embed displays an unavailable/error state.
|
||||
|
||||
The route's `Content-Security-Policy: frame-ancestors` allows the GUI origin and
|
||||
the embedding page's origin when it is exactly one label under `config.domain`
|
||||
with the scheme and port from `config.origin`. Subdomains the server answers
|
||||
itself (`api`, `dav`, `js`, `www`, the native-app hosts, and the host of
|
||||
`api_base_url`) and the user-content hosting domains (`static_hosting_domain`,
|
||||
`private_app_hosting_domain`, and their `_alt` variants) are refused. For
|
||||
example, `https://puter.com` permits `https://spreadsheet.puter.com` but not
|
||||
`https://api.puter.com`, and `http://puter.localhost:4100` permits
|
||||
`http://spreadsheet.puter.localhost:4100` but not
|
||||
`http://someone.site.puter.localhost:4100`. The embedding page is read from the
|
||||
iframe request's referrer, so the host must not suppress it (the browser default
|
||||
`strict-origin-when-cross-origin` is enough). All ancestors in a nested iframe
|
||||
chain must be allowed. Other GUI routes retain `X-Frame-Options: SAMEORIGIN`.
|
||||
|
||||
The embed accepts no query parameters. Requests with a query string redirect to
|
||||
the same embed path so SDK bootstrap parameters cannot override the session or API
|
||||
origin. The host needs to allow the GUI origin in its own `frame-src` CSP if it
|
||||
sets one. If sandboxing the iframe, allow scripts, same-origin storage, popups,
|
||||
and popups to escape the sandbox.
|
||||
|
||||
## Host integration
|
||||
|
||||
A first-party app that signs in with puter.js can place `<puter-user-menu>` from
|
||||
the SDK in its header instead (see `src/puter-js/src/ui/components.md`). It opens
|
||||
this embed with the signed-in user named and closes it when the app's sign-in
|
||||
changes. The rest of this section is for a host that builds the iframe itself.
|
||||
|
||||
The iframe can be shown whether or not the user is signed in. Create it when
|
||||
“More apps” is opened, naming the signed-in user in its fragment, and remove it
|
||||
on close so reopening loads current apps and preferences. Remove it when the
|
||||
host's own sign-in changes too, such as on sign-out, so an open embed doesn't
|
||||
keep the previous user. The host owns the modal and close control;
|
||||
there is no parent messaging API and no app list or credential is sent to it.
|
||||
The app browser fills its iframe and responds to its size. It shows a spinner until its apps load.
|
||||
|
||||
This example assumes `puterOrigin` comes from the host app's deployment
|
||||
settings. Translate the button and frame labels using the host app's
|
||||
localization system.
|
||||
|
||||
```html
|
||||
<button id="more-apps" type="button">More apps</button>
|
||||
<dialog id="apps-dialog" aria-label="More apps">
|
||||
<button id="close-apps" type="button">Close</button>
|
||||
<div id="apps-frame"></div>
|
||||
</dialog>
|
||||
<style>
|
||||
#apps-dialog { padding: 12px; width: min(900px, 90vw); }
|
||||
#apps-frame iframe { width: 100%; height: min(650px, 75vh); border: 0; }
|
||||
</style>
|
||||
<script type="module">
|
||||
const puterOrigin = 'https://puter.com'; // Use this deployment's configured origin.
|
||||
const dialog = document.querySelector('#apps-dialog');
|
||||
const container = document.querySelector('#apps-frame');
|
||||
const trigger = document.querySelector('#more-apps');
|
||||
|
||||
trigger.addEventListener('click', async () => {
|
||||
const url = new URL('/embed/apps', puterOrigin);
|
||||
// Signed out, the embed shows the recommended apps.
|
||||
const user = puter.auth.isSignedIn() ? await puter.auth.getUser().catch(() => null) : null;
|
||||
if (user) url.hash = `user=${encodeURIComponent(user.uuid)}`;
|
||||
const frame = document.createElement('iframe');
|
||||
frame.title = 'Your apps';
|
||||
frame.src = url.href;
|
||||
container.replaceChildren(frame);
|
||||
dialog.showModal();
|
||||
});
|
||||
document.querySelector('#close-apps').addEventListener('click', () => dialog.close());
|
||||
dialog.addEventListener('close', () => {
|
||||
container.replaceChildren();
|
||||
trigger.focus();
|
||||
});
|
||||
</script>
|
||||
```
|
||||
|
||||
## Implementation
|
||||
|
||||
`createAppBrowser` in `src/gui/src/UI/Dashboard/AppBrowser.js` owns the shared UI
|
||||
and data behavior; its `guestApps` option is the signed-out list. `TabApps.js`
|
||||
injects desktop actions. `appsEmbed.js` supplies new-tab launching through
|
||||
`launchAppInTab` (signed in) and `guestAppLaunchUrl` (signed out) in
|
||||
`src/gui/src/helpers/appTabLaunch.js`, which build the same app URL as
|
||||
`launch_app`, and loads only the embed entry, its SDK, and shared styles. `embedHostUser` in
|
||||
`src/gui/src/helpers/embedHostUser.js` reads the host's user from the fragment, and
|
||||
`embedHostTheme.js` beside it applies the host's theme.
|
||||
The normal GUI webpack build emits `bundle.min.js`, `apps-embed.min.js`, and `feedback-embed.min.js`.
|
||||
The `/embed/apps` route puts the recommended list in the page's config as
|
||||
`guestApps`. No puter.js API or app-list endpoint changes are needed.
|
||||
|
||||
## Feedback dialog
|
||||
|
||||
`GET /embed/feedback` is the app feedback dialog (`puter.ui.showFeedbackDialog()`)
|
||||
for a first-party page, with the same framing rules, query redirect, and
|
||||
host-user binding as `/embed/apps`. It sends the feedback for the framing
|
||||
page's origin, which the server reads from the referrer and pins with
|
||||
`frame-ancestors`; opened top-level, it does nothing. Its page is transparent
|
||||
so the dialog's own scrim covers the host.
|
||||
|
||||
puter.js uses it on a page one label under the GUI origin, other than the
|
||||
subdomains the server refuses, when the page is signed in, framing it in a
|
||||
modal `<dialog>` that stays hidden until the embed is ready. A session check that takes longer than 200ms shows a loading scrim
|
||||
with a spinner in its place; Escape or a click on it cancels. The embed posts `{ msg: 'feedbackEmbed', state }` to its parent
|
||||
(`loading`, then `ready` or `unavailable`) and finally
|
||||
`{ msg: 'feedbackDialogClosed', sent }`. It reports `unavailable` without the
|
||||
named user's session, or while that user's email still needs confirming, and
|
||||
puter.js then uses the `/action/send-feedback` popup, which can sign in. So does
|
||||
a frame that never answers, such as one the host's CSP blocks. `feedbackEmbed.js`
|
||||
is the entry, sharing `embedSession` in `src/gui/src/helpers/embedSession.js`
|
||||
with the app browser; `showFeedbackEmbed` in `src/puter-js/src/lib/guiEmbed.js`
|
||||
is the host side.
|
||||
|
||||
## Sign-in and file pickers
|
||||
|
||||
`GET /embed/sign-in`, `/embed/show-open-file-picker`, `/embed/show-save-file-picker`
|
||||
and `/embed/show-directory-picker` serve the same GUI popups as `/action/<action>`
|
||||
for a first-party page, with the same framing rules, query redirect, and
|
||||
`no-store` as `/embed/apps`. Any other spelling of the path (another case, a
|
||||
trailing slash) redirects to it, since the shell boots the desktop for it. They render the normal GUI shell, which names the
|
||||
framing page (read from the referrer, pinned with `frame-ancestors`) as
|
||||
`gui_params.embed_host_origin`. The GUI then runs the popup boot unchanged, with
|
||||
the framing page as `window.opener`, and `window.close()` asks the page to
|
||||
remove the frame (`src/gui/src/helpers/popupFrame.js`). Without a framing page
|
||||
to answer (opened top-level, or framed by a refused page) it does nothing. The
|
||||
popup's parameters come from the fragment instead of the query: `msg_id`,
|
||||
`options` and `request_auth`, plus `user` for a picker.
|
||||
|
||||
The frame posts `{ msg: 'puterPopupFrame', state }` to its parent: `loading` as
|
||||
soon as the shell boots, `ready` once it shows a window or dialog, `unavailable`
|
||||
when the page has to use the popup, `provider` when the user picks a sign-in
|
||||
provider (below), and `closed` where a popup would close. Its
|
||||
answers (`puter.token`, `fileOpenPicked`, `directoryPicked`, `fileSaved`, the
|
||||
cancel messages, `sendMeFileData`) are the popup's.
|
||||
|
||||
puter.js uses them on a page one label under the GUI origin, other than the
|
||||
subdomains the server refuses, unless the page is cross-origin isolated or in a
|
||||
frame of another site (which `frame-ancestors` refuses too).
|
||||
`showPopupFrame` in `src/puter-js/src/lib/guiEmbed.js` shows the frame in a box
|
||||
the size of the popup window, over a scrim, hidden until `ready` and behind the
|
||||
loading spinner after 200ms. Escape closes it, on the page or in the frame. One shows per popup, as each
|
||||
popup has its own window name: a second picker request of the same kind
|
||||
replaces the one showing, and a `signIn()` call made while a framed sign-in is
|
||||
open settles with it. On
|
||||
screens up to 600px wide the box fills the screen. A frame that is refused,
|
||||
never answers, or reports `unavailable` falls back to the popup, opened through
|
||||
a consent dialog when the click that asked for it has expired. A reload or navigation the frame
|
||||
starts itself (logging out, a sign-out in another tab) names the frame as its
|
||||
referrer, so the server refuses it; the page then shows its spinner and opens
|
||||
the frame again, once per `ready`.
|
||||
|
||||
- **Sign-in** (`puter.auth.signIn()`) has no host-user binding: it is how the
|
||||
page gets a user. The box has a close button, which rejects with
|
||||
`auth_window_closed` as closing the popup does. Providers refuse to be
|
||||
framed, so choosing one posts `{ state: 'provider', url }` with the
|
||||
provider's start URL for this sign-in's `msg_id`, then `unavailable`. The
|
||||
page opens the popup straight at that URL, once it checks the URL is the
|
||||
GUI's `/auth/oidc/<provider>/start` for this `msg_id`, and the provider
|
||||
returns it as this sign-in's popup. `attempt_temp_user_creation` keeps the popup, since
|
||||
only the popup creates a temporary account.
|
||||
- **Pickers** frame only for a signed-in page. The GUI reports `unavailable`
|
||||
unless its session belongs to the user named in `user`, and the page then uses
|
||||
the popup, which can sign in and switch accounts. The token the GUI sends
|
||||
before a picker's answer isn't adopted, since it is for the page's own user.
|
||||
Dismissing the loading scrim settles the picker as cancelled.
|
||||
|
||||
## Upgrade
|
||||
|
||||
`GET /embed/upgrade` is the deployment's upgrade dialog for a first-party page,
|
||||
served like the popups above: the GUI shell with the same framing rules, query
|
||||
redirect, exact-path redirect and `no-store`, naming the framing page as
|
||||
`gui_params.embed_host_origin`. It runs only when framed by an allowed page.
|
||||
|
||||
puter.js frames it on a signed-in first-party page (the same pages as the
|
||||
pickers) for `puter.ui.requestUpgrade(details)`, and for the Upgrade button of
|
||||
the usage limit dialog a refused call shows there. Elsewhere both keep their
|
||||
behavior: inside an app `requestUpgrade()` asks the desktop, and on any other
|
||||
page the usage limit dialog opens the dashboard's upgrade page in a new tab.
|
||||
|
||||
The frame covers the page with no box or close button, and the GUI page is
|
||||
transparent (`html.popup-frame-transparent`) so the dialog's own scrim covers
|
||||
the host. The frame allows `payment` as well as `otp-credentials`. Escape on the
|
||||
page is left to the dialog, since one there shouldn't drop a payment in
|
||||
progress. Its fragment names `user` (the page's signed-in user) and, when the
|
||||
refusal gave them, `reason` (`funds`, `subscription` or `storage`) and `method`.
|
||||
|
||||
The GUI reports `unavailable` unless its session belongs to `user`, the account
|
||||
has no verification or password change waiting (which the frame couldn't be
|
||||
closed from), and the deployment defines `window.UIUpgradeAccount` with
|
||||
`UIUpgradeAccount.embeddable === true`. Otherwise it runs:
|
||||
|
||||
```js
|
||||
new UIUpgradeAccount({ attribution: { appUid, reason, method }, onClose }).open_as_window();
|
||||
```
|
||||
|
||||
`appUid` is the app the framing page's origin resolves to, never one the
|
||||
fragment names. `open_as_window()` resolves once the dialog shows, and the GUI
|
||||
then posts `ready`. While the page still hides the frame, which Chromium and
|
||||
WebKit lay out at 0x0, the GUI posts `ready` first and opens the dialog once
|
||||
the frame has a viewport, so a window it opens doesn't center on 0x0.
|
||||
`onClose({ upgraded })` must run exactly once, when the
|
||||
dialog closes or when it declines to open for the account; the GUI then posts
|
||||
`{ msg: 'puterPopupFrame', state: 'closed', upgraded }` and the page removes
|
||||
the frame. If `open_as_window()` rejects, the GUI posts `unavailable` instead. `requestUpgrade()` resolves `{ upgraded }`. A call made while the
|
||||
frame shows settles with it rather than replacing it, and a call refused while
|
||||
it shows still rejects but opens no usage limit dialog over it.
|
||||
|
||||
When the frame can't show the upgrade (refused, never ready, `unavailable`, or
|
||||
no user to name), puter.js opens the dashboard's upgrade page in a new tab with
|
||||
what was refused, through a consent dialog when the click has expired or the
|
||||
tab is blocked, and `requestUpgrade()` resolves `{ upgraded: false }`.
|
||||
|
||||
## Email confirmation
|
||||
|
||||
`GET /embed/request-email-confirmation` walks a first-party page's user through
|
||||
confirming their email, served like the upgrade: the GUI shell with the same
|
||||
framing rules, redirects and `no-store`, run only when framed by an allowed
|
||||
page. puter.js frames it on a signed-in first-party page for
|
||||
`puter.ui.requestEmailConfirmation()`. Inside an app the desktop shows the same
|
||||
windows over the app instead.
|
||||
|
||||
Like the upgrade, the frame covers the page with no box or close button, over a
|
||||
transparent GUI page, and its fragment names `user`. Escape closes it, on the
|
||||
page or on the save-account or code window. On a window, Escape closes that
|
||||
window as its close button does, so a code already sent still answers. The GUI
|
||||
reports `unavailable` unless its session belongs to `user` and the account has
|
||||
no phone, card or password check waiting.
|
||||
An email check is fine: the boot skips its own email gate for this action, so
|
||||
the user gets one code window they can close.
|
||||
|
||||
The GUI then runs `ensureConfirmedEmail` (`src/gui/src/helpers/ensureConfirmedEmail.js`),
|
||||
which the desktop's handler uses too: a temporary account gets the save-account
|
||||
window first, then an unconfirmed email gets the code window, each at most
|
||||
once. When the page's whoami already reads the email as confirmed,
|
||||
`requestEmailConfirmation()` resolves `{ response: true }` without framing
|
||||
anything, and a confirmation in the frame marks that whoami confirmed for the
|
||||
calls after it. An email the GUI reads as confirmed shows nothing. Before
|
||||
opening a window the GUI waits for the frame to have a viewport, as the upgrade
|
||||
does. It then posts
|
||||
`{ msg: 'puterPopupFrame', state: 'closed', confirmed }`, and
|
||||
`requestEmailConfirmation()` resolves `{ response: confirmed }`, the field an
|
||||
app reads from the desktop's answer. A call made while the frame shows settles
|
||||
with it. A frame that is refused, never ready, or `unavailable` resolves
|
||||
`{ response: false }`, since there's no popup to fall back to.
|
||||
|
||||
## Checks
|
||||
|
||||
Run the GUI component/browser tests (with the SDK bundle built and Playwright
|
||||
Chromium installed), and the backend route tests:
|
||||
|
||||
```sh
|
||||
npx vitest run --config src/gui/vitest.config.js src/gui/src/UI/Dashboard/AppBrowser src/feedbackEmbed
|
||||
npx vitest run src/puter-js/src/lib/guiEmbed.test.js src/puter-js/src/modules/UI.showFeedbackDialog.test.js src/puter-js/src/modules/Auth.signInFrame.test.js src/puter-js/src/modules/UI.pickerFrame.test.js src/puter-js/src/modules/UI.requestUpgrade.test.js src/puter-js/src/lib/upgradePrompt.test.js src/puter-js/src/modules/UsageLimitDialog.upgrade.test.js
|
||||
npx vitest run --config src/gui/vitest.config.js src/gui/src/helpers/popupFrame.test.js
|
||||
npx vitest run --config src/backend/vitest.config.ts src/backend/server.test.ts
|
||||
```
|
||||
+6
-7
@@ -1,6 +1,6 @@
|
||||
# Browser push client
|
||||
|
||||
Puter's desktop shows pushes through `/sw.js`. Core shows Settings → Notifications only when the `pushEnabled` gui param is `true`; core never sets it. The worker is registered only when push is turned on in the browser, and a load removes one left without a subscription. A backend extension that delivers pushes sets it with `services.homepage.setGuiParam('pushEnabled', true)` and can load its own code into the worker with `services.homepage.registerServiceWorkerScript(path)`.
|
||||
Puter's desktop shows pushes through `/sw.js`. Core registers it, and shows Settings → Notifications, only when the `pushEnabled` gui param is `true`; core never sets it. A backend extension that delivers pushes sets it with `services.homepage.setGuiParam('pushEnabled', true)` and can load its own code into the worker with `services.homepage.registerServiceWorkerScript(path)`.
|
||||
|
||||
Core subscribes the browser to a push service only when the server also publishes `pushWebPushKey` (see [Built-in Web Push client](#built-in-web-push-client)), and it has no `/push/*` routes of its own. The device, app, preference and subscription endpoints below come from that extension. On a deployment without one, such as most self-hosted servers, they answer core's plain 404 (`code: 'not_found'`), which `puter.notifications` reports as `push_unavailable` and the Notifications tab shows as unavailable. A device the extension can't find is `device_not_found`.
|
||||
|
||||
@@ -22,13 +22,12 @@ When the browser replaces the subscription (`pushsubscriptionchange`), `sw.js` p
|
||||
|
||||
Before a notification is pushed, the server asks the open clients whether they received it live. It sends `notif.receipt` with `{ uid }` on the user's sockets for the app that sent it, or on the desktop's sockets for Puter's own notifications. The SDK remembers the uids its page received through `onNotification()`, `onPush()` or the desktop's feed, and answers once its page has that notification, or with `{ received: false }` when it never gets it. The server waits up to 5 seconds (`notificationAckTimeoutMs`).
|
||||
|
||||
An answer carries `visibilityState` and `focused` as they were when the page received the notification. Pages never show a browser notification themselves; pushes do.
|
||||
An answer carries `visibilityState`, `focused` and `showsOwn`, the last being whether the page shows the notification itself. A website page shows it itself when it is hidden or unfocused, has an `onPush()` handler and notification permission, and no other tab of the same site is focused. Inside the desktop, the desktop shows it, and an app's `onPush()` result is never shown as a browser notification.
|
||||
|
||||
- An answer from a page that was visible and focused stops the push (`handled` on `notif.delivery.settled`). An older client that doesn't send `focused` counts while visible.
|
||||
- Otherwise the notification is pushed, to every browser the user subscribed. A website's pushes go only to the browsers that subscribed through its own worker, never to Puter's `/sw.js`. An app inside the desktop pushes through `/sw.js`, which shows the app's name and icon.
|
||||
- A client that can't answer, such as an older puter.js, doesn't hold the push back.
|
||||
|
||||
The ask reaches only the sockets in the asking region. So that other regions hear about a page too, a page that receives a notification live while visible and focused also sends `notif.handled` with `{ uid }` on the socket it came on. Core relays it as the node-local `notif.handled` event with the socket's user and app, at most 60 a minute per user; an extension that delivers pushes decides what to do with it.
|
||||
- An answer from a page that is visible and focused, or that shows the notification itself, stops the push.
|
||||
- Otherwise the notification is pushed. A website's pushes go only to the browsers that subscribed through its own worker, never to Puter's `/sw.js`. An app inside the desktop pushes through `/sw.js`, which shows the app's name and icon.
|
||||
- A hidden desktop tab shows no banner, so answers only from hidden desktop tabs don't stop a push of Puter's own notifications.
|
||||
- A client that can't answer, such as an older puter.js, holds the push back while it is connected.
|
||||
|
||||
## Registering
|
||||
|
||||
|
||||
@@ -318,13 +318,3 @@ describe('AbstractDatabaseClient — SQL generation', () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
it('builds an insert for batchWrite and shares it with insert', async () => {
|
||||
const client = new FixtureDatabaseClient([]);
|
||||
const entry = client.insertEntry('widget', { name: "reader's", enabled: true });
|
||||
expect(entry.values).toEqual(["reader's", true]);
|
||||
expect(entry.statement).toContain('VALUES (?, ?)');
|
||||
await client.insert('widget', { name: "reader's", enabled: true });
|
||||
expect(client.writes).toEqual([{ query: entry.statement, params: entry.values }]);
|
||||
});
|
||||
@@ -110,23 +110,14 @@ export class AbstractDatabaseClient extends PuterClient {
|
||||
tableName: string,
|
||||
data: Record<string, unknown>,
|
||||
): Promise<WriteResult> {
|
||||
const entry = this.insertEntry(tableName, data);
|
||||
return this.write(
|
||||
entry.statement + this.returningIdClause(),
|
||||
entry.values,
|
||||
);
|
||||
}
|
||||
|
||||
/** An insert that can join other writes in a transaction. */
|
||||
insertEntry(tableName: string, data: Record<string, unknown>): BatchEntry {
|
||||
const cols = Object.keys(data);
|
||||
return {
|
||||
statement:
|
||||
`INSERT INTO ${this.quoteIdentifier(tableName)} ` +
|
||||
`(${cols.map((c) => this.quoteIdentifier(c)).join(', ')}) ` +
|
||||
`VALUES (${cols.map(() => '?').join(', ')})`,
|
||||
values: Object.values(data),
|
||||
};
|
||||
const values = Object.values(data);
|
||||
const sql =
|
||||
`INSERT INTO ${this.quoteIdentifier(tableName)} ` +
|
||||
`(${cols.map((c) => this.quoteIdentifier(c)).join(', ')}) ` +
|
||||
`VALUES (${cols.map(() => '?').join(', ')})` +
|
||||
this.returningIdClause();
|
||||
return this.write(sql, values);
|
||||
}
|
||||
|
||||
/** Whether `read()` goes to a separate replica rather than the primary. */
|
||||
|
||||
@@ -117,7 +117,7 @@ export type EventMap = {
|
||||
model_used: string;
|
||||
service_used: string;
|
||||
intended_service: string;
|
||||
model_details?: {
|
||||
model_details: {
|
||||
id: string;
|
||||
provider: string;
|
||||
input_cost_key: string;
|
||||
@@ -420,11 +420,7 @@ export type EventMap = {
|
||||
old_username: string;
|
||||
new_username: string;
|
||||
};
|
||||
'user.email-changed': {
|
||||
user_id: number;
|
||||
old_email: string | null;
|
||||
new_email: string;
|
||||
};
|
||||
'user.email-changed': { user_id: number; new_email: string };
|
||||
/** An account was suspended, or its suspension lifted. Local events. */
|
||||
'user.suspended': { userId: number };
|
||||
'user.unsuspended': { userId: number };
|
||||
@@ -560,14 +556,6 @@ export type EventMap = {
|
||||
'kv.share.retired': { handles: string[]; userId: number };
|
||||
|
||||
// ---- Notifications ----
|
||||
/** An app message was acknowledged after its state committed. */
|
||||
'notif.acknowledged': {
|
||||
userId: number;
|
||||
uid: string;
|
||||
type: string;
|
||||
audience: string;
|
||||
appUid: string;
|
||||
};
|
||||
/**
|
||||
* A notification row exists. Emitted after the insert commits — nothing
|
||||
* pushes a uid before there is a row to name — and carries the scope tuple
|
||||
@@ -601,8 +589,6 @@ export type EventMap = {
|
||||
value: Record<string, unknown>;
|
||||
/** An earlier notification rewritten in place. */
|
||||
updated?: true;
|
||||
/** Replaced an earlier notification in its group. */
|
||||
replaced?: true;
|
||||
/** Names this one send, for a receipt asked about it. */
|
||||
sendId: string;
|
||||
/** The same `sendId` may be emitted again, as a delayed send is. */
|
||||
@@ -640,18 +626,6 @@ export type EventMap = {
|
||||
/** A peer region holds a socket for the user. */
|
||||
peerRegions: boolean;
|
||||
responders: NotifDeliveryResponder[];
|
||||
/** A page here got it live while visible and focused. */
|
||||
handled: boolean;
|
||||
};
|
||||
/**
|
||||
* A page reported that it got a notification live while visible and
|
||||
* focused. Node-local; `appUid` is the reporting socket's app, absent for
|
||||
* the desktop.
|
||||
*/
|
||||
'notif.handled': {
|
||||
userId: number;
|
||||
appUid?: string;
|
||||
uid: string;
|
||||
};
|
||||
|
||||
// ---- Metering ----
|
||||
@@ -877,12 +851,6 @@ export type EventMap = {
|
||||
/** One access token was revoked; only its own connections should drop. */
|
||||
'auth.access-token.revoked': { token_uid: string };
|
||||
|
||||
/** Same-cluster eviction for raw realtime WebSockets. */
|
||||
'pubsub.realtime.auth-revoked': {
|
||||
sessionUids?: string[];
|
||||
tokenUid?: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* A grant was withdrawn, so whatever was standing on it has to be settled
|
||||
* rather than left to fail its next check.
|
||||
@@ -1108,6 +1076,11 @@ export interface NotifDeliveryResponder {
|
||||
visibilityState: 'visible' | 'hidden' | 'unknown';
|
||||
/** The page's window has focus. Absent from older clients. */
|
||||
focused?: boolean;
|
||||
/**
|
||||
* The page shows its own browser notification while hidden or unfocused.
|
||||
* Absent from older clients.
|
||||
*/
|
||||
showsOwn?: boolean;
|
||||
}
|
||||
|
||||
export type EventMetadata = {
|
||||
|
||||
@@ -347,46 +347,6 @@ describe('DispatcherInvokeTransport with every connection busy', () => {
|
||||
held[INVOKE_CONNECTIONS].answer();
|
||||
await expect(after).resolves.toMatchObject({ status: 200 });
|
||||
});
|
||||
|
||||
it('rechecks the guard once a connection frees, and drops a superseded call', async () => {
|
||||
const { transport, held, answerAll } = await saturated();
|
||||
|
||||
let current = true;
|
||||
const guard = vi.fn(async () => current);
|
||||
const waiting = transport.send({ ...CALL, beforeSend: guard });
|
||||
current = false;
|
||||
held[0].answer();
|
||||
|
||||
await expect(waiting).resolves.toEqual({
|
||||
status: null,
|
||||
error: 'invocation superseded',
|
||||
});
|
||||
expect(guard).toHaveBeenCalledTimes(1);
|
||||
expect(held).toHaveLength(INVOKE_CONNECTIONS);
|
||||
await answerAll();
|
||||
});
|
||||
|
||||
it('fails a call whose guard throws after the wait, and frees its slot', async () => {
|
||||
const { transport, held, answerAll } = await saturated();
|
||||
|
||||
const waiting = transport.send({
|
||||
...CALL,
|
||||
beforeSend: async () => {
|
||||
throw new Error('lease store down');
|
||||
},
|
||||
});
|
||||
held[0].answer();
|
||||
await expect(waiting).rejects.toThrow('lease store down');
|
||||
expect(held).toHaveLength(INVOKE_CONNECTIONS);
|
||||
|
||||
await answerAll();
|
||||
const after = transport.send(CALL);
|
||||
await vi.waitFor(() =>
|
||||
expect(held).toHaveLength(INVOKE_CONNECTIONS + 1),
|
||||
);
|
||||
held[INVOKE_CONNECTIONS].answer();
|
||||
await expect(after).resolves.toMatchObject({ status: 200 });
|
||||
});
|
||||
});
|
||||
|
||||
describe('DispatcherInvokeTransport schedule lane', () => {
|
||||
@@ -811,138 +771,6 @@ describe('DispatcherInvokeTransport batches', () => {
|
||||
]);
|
||||
});
|
||||
|
||||
it('leaves a call superseded while gathered out of the batch', async () => {
|
||||
const { fetchImpl, batches, singles } = streamingFetch();
|
||||
const transport = transportFor(fetchImpl, { batchWindowMs: 20 });
|
||||
let current = true;
|
||||
const sent = [
|
||||
transport.send(CALL),
|
||||
transport.send({ ...CALL, beforeSend: async () => current }),
|
||||
transport.send({ ...CALL, script: 'evw-other' }),
|
||||
];
|
||||
current = false;
|
||||
await vi.waitFor(() => expect(batches).toHaveLength(1));
|
||||
expect(batches[0].body.calls.map((call) => call.script)).toEqual([
|
||||
'evw-test',
|
||||
'evw-other',
|
||||
]);
|
||||
await expect(sent[1]).resolves.toEqual({
|
||||
status: null,
|
||||
error: 'invocation superseded',
|
||||
});
|
||||
batches[0].answer({ i: 0, status: 200, handled: true });
|
||||
batches[0].answer({ i: 1, status: 200, handled: true });
|
||||
await expect(Promise.all([sent[0], sent[2]])).resolves.toEqual([
|
||||
{ status: 200, handled: true },
|
||||
{ status: 200, handled: true },
|
||||
]);
|
||||
expect(singles).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('sends nothing when every gathered call was superseded', async () => {
|
||||
const fetchImpl = vi.fn(
|
||||
stubFetch(200, { [EVENTS_HANDLED_HEADER]: '1' }),
|
||||
);
|
||||
const transport = transportFor(fetchImpl as unknown as FetchImpl);
|
||||
const superseded = { ...CALL, beforeSend: async () => false };
|
||||
await expect(
|
||||
Promise.all([
|
||||
transport.send(superseded),
|
||||
transport.send(superseded),
|
||||
]),
|
||||
).resolves.toEqual([
|
||||
{ status: null, error: 'invocation superseded' },
|
||||
{ status: null, error: 'invocation superseded' },
|
||||
]);
|
||||
expect(fetchImpl).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('fails only the call whose guard throws at flush', async () => {
|
||||
const { fetchImpl, batches, singles } = streamingFetch();
|
||||
const transport = transportFor(fetchImpl);
|
||||
const failing = transport.send({
|
||||
...CALL,
|
||||
beforeSend: async () => {
|
||||
throw new Error('lease store down');
|
||||
},
|
||||
});
|
||||
const kept = transport.send(CALL);
|
||||
await expect(failing).rejects.toThrow('lease store down');
|
||||
// One call left, so it goes alone.
|
||||
await expect(kept).resolves.toEqual({ status: 200, handled: true });
|
||||
expect(batches).toHaveLength(0);
|
||||
expect(singles).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('rechecks the guard before resending a batch call on its own', async () => {
|
||||
const urls: string[] = [];
|
||||
let current = true;
|
||||
const fetchImpl = (async (url: string) => {
|
||||
urls.push(url);
|
||||
if (url.endsWith(EVENTS_DISPATCH_BATCH_PATH)) {
|
||||
// Superseded while the batch was out.
|
||||
current = false;
|
||||
return new Response('no-such-route', {
|
||||
status: 404,
|
||||
headers: {
|
||||
[EVENTS_DISPATCH_ERROR_HEADER]: 'no-such-route',
|
||||
},
|
||||
});
|
||||
}
|
||||
return new Response(null, {
|
||||
status: 200,
|
||||
headers: { [EVENTS_HANDLED_HEADER]: '1' },
|
||||
});
|
||||
}) as unknown as FetchImpl;
|
||||
const transport = transportFor(fetchImpl);
|
||||
|
||||
await expect(
|
||||
Promise.all([
|
||||
transport.send({ ...CALL, beforeSend: async () => current }),
|
||||
transport.send(CALL),
|
||||
]),
|
||||
).resolves.toEqual([
|
||||
{ status: null, error: 'invocation superseded' },
|
||||
{ status: 200, handled: true },
|
||||
]);
|
||||
expect(urls.map((url) => new URL(url).pathname)).toEqual([
|
||||
EVENTS_DISPATCH_BATCH_PATH,
|
||||
EVENTS_DISPATCH_PATH,
|
||||
]);
|
||||
});
|
||||
|
||||
it('answers a schedule run superseded in the batch as the early check does', async () => {
|
||||
const { fetchImpl, batches } = streamingFetch();
|
||||
const client = new EventsWorkerInvokerClient({
|
||||
events: { invokeTimeoutMs: 5_000 },
|
||||
} as unknown as IConfig);
|
||||
client.setTransport(transportFor(fetchImpl, { batchWindowMs: 20 }));
|
||||
let current = true;
|
||||
const guard = vi.fn(async () => current);
|
||||
|
||||
const run = client.invoke(REQUEST, {
|
||||
lane: 'schedule',
|
||||
beforeSend: guard,
|
||||
});
|
||||
const other = client.invoke(REQUEST);
|
||||
const another = client.invoke(REQUEST);
|
||||
await vi.waitFor(() => expect(guard).toHaveBeenCalledTimes(1));
|
||||
current = false;
|
||||
|
||||
await vi.waitFor(() => expect(batches).toHaveLength(1));
|
||||
expect(batches[0].body.calls).toHaveLength(2);
|
||||
await expect(run).resolves.toEqual(
|
||||
await client.invoke(REQUEST, { beforeSend: async () => false }),
|
||||
);
|
||||
await expect(run).resolves.toMatchObject({
|
||||
outcome: 'unavailable',
|
||||
error: 'invocation superseded',
|
||||
});
|
||||
batches[0].answer({ i: 0, status: 200, handled: true });
|
||||
batches[0].answer({ i: 1, status: 200, handled: true });
|
||||
await Promise.all([other, another]);
|
||||
});
|
||||
|
||||
it('builds each body when the batch goes out', async () => {
|
||||
const { fetchImpl, batches } = streamingFetch();
|
||||
const transport = transportFor(fetchImpl);
|
||||
@@ -999,26 +827,6 @@ describe('DispatcherInvokeTransport batches', () => {
|
||||
});
|
||||
|
||||
describe('EventsWorkerInvokerClient invoke options', () => {
|
||||
it('does not send a superseded invocation', async () => {
|
||||
const client = makeClient();
|
||||
const send = vi.fn(async () => ({ status: 200, handled: true }));
|
||||
client.setTransport({ send });
|
||||
const result = await client.invoke(REQUEST, { beforeSend: async () => false });
|
||||
expect(result.outcome).toBe('unavailable');
|
||||
expect(send).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('rechecks before resending after a deployment', async () => {
|
||||
const client = makeClient();
|
||||
let current = true;
|
||||
const send = vi.fn(async () => ({ status: null, dispatchReason: 'missing' }));
|
||||
client.setTransport({ send });
|
||||
client.setMissHandler(async () => { current = false; return 'deployed'; });
|
||||
const result = await client.invoke(REQUEST, { beforeSend: async () => current });
|
||||
expect(result.outcome).toBe('unavailable');
|
||||
expect(send).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('stamps the event as each attempt goes out', async () => {
|
||||
const bodies: string[] = [];
|
||||
const client = makeClient();
|
||||
|
||||
@@ -282,11 +282,6 @@ export interface EventsInvokeCall {
|
||||
* connection first, so a send time stamped into it is the real one.
|
||||
*/
|
||||
render?: () => string;
|
||||
/**
|
||||
* Rechecked by a transport that waits before sending; false drops the call
|
||||
* with {@link supersededAnswer}.
|
||||
*/
|
||||
beforeSend?: () => Promise<boolean>;
|
||||
}
|
||||
|
||||
/** What one attempt at delivering an invocation came back with. */
|
||||
@@ -339,8 +334,6 @@ export interface WorkerInvokeResult {
|
||||
/** How one invocation is sent. */
|
||||
export interface WorkerInvokeOptions {
|
||||
lane?: InvokeLane;
|
||||
/** Rechecked before each send, including a resend after deployment. */
|
||||
beforeSend?: () => Promise<boolean>;
|
||||
/** Applied to the event as each attempt is sent. */
|
||||
stampEvent?: (event: unknown) => unknown;
|
||||
}
|
||||
@@ -348,12 +341,6 @@ export interface WorkerInvokeOptions {
|
||||
const NO_TRANSPORT =
|
||||
'no events dispatcher configured (url and internal secret)';
|
||||
|
||||
/** What a call whose `beforeSend` said no answers instead of being sent. */
|
||||
const supersededAnswer = (): EventsInvokeAnswer => ({
|
||||
status: null,
|
||||
error: 'invocation superseded',
|
||||
});
|
||||
|
||||
export class EventsWorkerInvokerClient extends PuterClient {
|
||||
/** Set only where invocations do not leave the process — local development. */
|
||||
#transport: EventsInvokeTransport | null = null;
|
||||
@@ -408,13 +395,11 @@ export class EventsWorkerInvokerClient extends PuterClient {
|
||||
const margin = Math.min(EVENTS_DEADLINE_MARGIN_MS, timeoutMs / 2);
|
||||
const deadline =
|
||||
Date.now() + timeoutMs * EVENTS_WORKER_LEASE_FACTOR - margin;
|
||||
const send = async (
|
||||
const send = (
|
||||
deployed: boolean,
|
||||
sendTimeoutMs = timeoutMs,
|
||||
): Promise<EventsInvokeAnswer> => {
|
||||
if (options.beforeSend && !(await options.beforeSend()))
|
||||
return supersededAnswer();
|
||||
return transport.send({
|
||||
): Promise<EventsInvokeAnswer> =>
|
||||
transport.send({
|
||||
script: request.script,
|
||||
appUid: request.appUid,
|
||||
...(request.holder ? { holder: request.holder } : {}),
|
||||
@@ -424,11 +409,7 @@ export class EventsWorkerInvokerClient extends PuterClient {
|
||||
...(deployed ? { deployed: true } : {}),
|
||||
...(options.lane ? { lane: options.lane } : {}),
|
||||
...(stampEvent ? { render } : {}),
|
||||
...(options.beforeSend
|
||||
? { beforeSend: options.beforeSend }
|
||||
: {}),
|
||||
});
|
||||
};
|
||||
|
||||
// A send may wait for a connection for half its timeout before the
|
||||
// run's own timeout starts, so it is given what is left divided by that.
|
||||
@@ -553,8 +534,6 @@ export class EventsWorkerInvokerClient extends PuterClient {
|
||||
interface BatchedCall {
|
||||
call: EventsInvokeCall;
|
||||
resolve: (answer: EventsInvokeAnswer) => void;
|
||||
/** For a `beforeSend` that threw. */
|
||||
reject: (err: unknown) => void;
|
||||
/** When its wait for a connection began, for what is left of it. */
|
||||
waitingSince: number;
|
||||
}
|
||||
@@ -625,14 +604,9 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
|
||||
error: 'no free connection to the events dispatcher',
|
||||
};
|
||||
try {
|
||||
if (this.#batchSize === 1) {
|
||||
// The wait for a connection may have outlived the call.
|
||||
if (call.beforeSend && !(await call.beforeSend()))
|
||||
return supersededAnswer();
|
||||
return await this.#sendOne(call);
|
||||
}
|
||||
return await new Promise<EventsInvokeAnswer>((resolve, reject) => {
|
||||
this.#batch.push({ call, resolve, reject, waitingSince });
|
||||
if (this.#batchSize === 1) return await this.#sendOne(call);
|
||||
return await new Promise<EventsInvokeAnswer>((resolve) => {
|
||||
this.#batch.push({ call, resolve, waitingSince });
|
||||
if (this.#batch.length >= this.#batchSize) this.#flush();
|
||||
else
|
||||
this.#batchTimer ??= setTimeout(
|
||||
@@ -652,12 +626,6 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
|
||||
const batch = this.#batch;
|
||||
this.#batch = [];
|
||||
if (batch.length === 0) return;
|
||||
if (batch.some(({ call }) => call.beforeSend))
|
||||
void this.#sendStillCurrent(batch);
|
||||
else this.#sendGathered(batch);
|
||||
}
|
||||
|
||||
#sendGathered(batch: BatchedCall[]): void {
|
||||
if (batch.length === 1 || Date.now() < this.#batchUnsupportedUntil) {
|
||||
for (const { call, resolve } of batch)
|
||||
void this.#sendOne(call).then(resolve);
|
||||
@@ -666,26 +634,6 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
|
||||
void this.#sendBatch(batch);
|
||||
}
|
||||
|
||||
/** Drops calls superseded while they were gathered, then sends the rest. */
|
||||
async #sendStillCurrent(batch: BatchedCall[]): Promise<void> {
|
||||
const checked = await Promise.all(
|
||||
batch.map(async (entry) => {
|
||||
if (!entry.call.beforeSend) return entry;
|
||||
try {
|
||||
if (await entry.call.beforeSend()) return entry;
|
||||
entry.resolve(supersededAnswer());
|
||||
} catch (err) {
|
||||
entry.reject(err);
|
||||
}
|
||||
return null;
|
||||
}),
|
||||
);
|
||||
const current = checked.filter(
|
||||
(entry): entry is BatchedCall => entry !== null,
|
||||
);
|
||||
if (current.length > 0) this.#sendGathered(current);
|
||||
}
|
||||
|
||||
/** One call on the single route. */
|
||||
async #sendOne(call: EventsInvokeCall): Promise<EventsInvokeAnswer> {
|
||||
try {
|
||||
@@ -740,15 +688,13 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
|
||||
let open = batch.length;
|
||||
const answered = batch.map(() => false);
|
||||
const timers: ReturnType<typeof setTimeout>[] = [];
|
||||
const finish = (i: number, done: (entry: BatchedCall) => void) => {
|
||||
const settle = (i: number, answer: EventsInvokeAnswer): void => {
|
||||
if (answered[i]) return;
|
||||
answered[i] = true;
|
||||
clearTimeout(timers[i]);
|
||||
done(batch[i]);
|
||||
batch[i].resolve(answer);
|
||||
if (--open === 0) hangUp.abort();
|
||||
};
|
||||
const settle = (i: number, answer: EventsInvokeAnswer): void =>
|
||||
finish(i, (entry) => entry.resolve(answer));
|
||||
const settleRest = (answer: EventsInvokeAnswer): void => {
|
||||
for (let i = 0; i < batch.length; i++) settle(i, answer);
|
||||
};
|
||||
@@ -831,22 +777,8 @@ export class DispatcherInvokeTransport implements EventsInvokeTransport {
|
||||
this.#batchUnsupportedUntil =
|
||||
Date.now() + BATCH_UNSUPPORTED_RETRY_MS;
|
||||
batch.forEach(({ call }, i) => {
|
||||
if (answered[i]) return;
|
||||
const { beforeSend } = call;
|
||||
if (!beforeSend) {
|
||||
if (!answered[i])
|
||||
void this.#sendOne(call).then((a) => settle(i, a));
|
||||
return;
|
||||
}
|
||||
// The batch round trip may have outlived the call.
|
||||
void (async () => {
|
||||
try {
|
||||
if (!(await beforeSend()))
|
||||
return settle(i, supersededAnswer());
|
||||
} catch (err) {
|
||||
return finish(i, (entry) => entry.reject(err));
|
||||
}
|
||||
settle(i, await this.#sendOne(call));
|
||||
})();
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -21,8 +21,6 @@
|
||||
import { GetObjectCommand } from '@aws-sdk/client-s3';
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { existsSync } from 'node:fs';
|
||||
import { createServer, type Server } from 'node:http';
|
||||
import type { AddressInfo } from 'node:net';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
@@ -202,77 +200,6 @@ describe('S3Client — remote endpoint configuration', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('S3Client — bucket in another region', () => {
|
||||
let server: Server;
|
||||
let client: S3Client;
|
||||
let signedRegions: string[];
|
||||
|
||||
// Answers like a bucket that lives in us-west-2: a request signed for any
|
||||
// other region gets S3's 301 naming the bucket's region.
|
||||
beforeEach(async () => {
|
||||
signedRegions = [];
|
||||
server = createServer((req, res) => {
|
||||
const region =
|
||||
/Credential=[^/]+\/\d{8}\/([^/]+)\//.exec(
|
||||
req.headers.authorization ?? '',
|
||||
)?.[1] ?? '';
|
||||
signedRegions.push(region);
|
||||
if (region !== 'us-west-2') {
|
||||
res.writeHead(301, {
|
||||
'content-type': 'application/xml',
|
||||
'x-amz-bucket-region': 'us-west-2',
|
||||
});
|
||||
res.end(
|
||||
'<?xml version="1.0" encoding="UTF-8"?><Error><Code>PermanentRedirect</Code><Message>The bucket you are attempting to access must be addressed using the specified endpoint.</Message></Error>',
|
||||
);
|
||||
return;
|
||||
}
|
||||
res.writeHead(200, {
|
||||
'content-type': 'text/plain',
|
||||
'content-length': '5',
|
||||
});
|
||||
res.end('hello');
|
||||
});
|
||||
await new Promise<void>((resolve) =>
|
||||
server.listen(0, '127.0.0.1', resolve),
|
||||
);
|
||||
const { port } = server.address() as AddressInfo;
|
||||
client = new S3Client({
|
||||
port: 0,
|
||||
extensions: [],
|
||||
s3: {
|
||||
s3Config: {
|
||||
endpoint: `http://127.0.0.1:${port}`,
|
||||
accessKeyId: 'fakeAccessKeyId',
|
||||
secretAccessKey: 'fakeSecretAccessKey',
|
||||
forcePathStyle: true,
|
||||
},
|
||||
},
|
||||
} as unknown as IConfig);
|
||||
await client.onServerStart();
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await client.onServerShutdown();
|
||||
server.closeAllConnections();
|
||||
await new Promise<void>((resolve) => server.close(() => resolve()));
|
||||
});
|
||||
|
||||
it("retries a read in the bucket's region when S3 redirects it", async () => {
|
||||
const result = await client
|
||||
.get('ap-northeast-2')
|
||||
.send(
|
||||
new GetObjectCommand({
|
||||
Bucket: 'other-bucket',
|
||||
Key: 'index.html',
|
||||
}),
|
||||
);
|
||||
|
||||
expect(await result.Body?.transformToString()).toBe('hello');
|
||||
expect(signedRegions).toEqual(['ap-northeast-2', 'us-west-2']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('S3Client — legacy storage migration', () => {
|
||||
let client: S3Client;
|
||||
let dir: string;
|
||||
|
||||
@@ -235,9 +235,6 @@ export class S3Client extends PuterClient {
|
||||
|
||||
const client = new AwsS3Client({
|
||||
region,
|
||||
// Rows without a stored region are read with the server's own, so
|
||||
// retry where S3's 301 says the bucket actually lives.
|
||||
followRegionRedirects: true,
|
||||
requestStreamBufferSize: 32 * 1024,
|
||||
requestHandler: new NodeHttpHandler({
|
||||
socketTimeout: 5000,
|
||||
|
||||
@@ -7468,11 +7468,6 @@ describe('AuthController user-protected mutations (validation paths)', () => {
|
||||
{ expiresIn: '1h' },
|
||||
);
|
||||
|
||||
const heard: Array<Record<string, unknown>> = [];
|
||||
eventClient.on('user.email-changed', (_k, data) => {
|
||||
heard.push(data as Record<string, unknown>);
|
||||
});
|
||||
|
||||
const req = makeReq({});
|
||||
(req as unknown as { query: Record<string, string> }).query = {
|
||||
token: linkJwt,
|
||||
@@ -7480,11 +7475,6 @@ describe('AuthController user-protected mutations (validation paths)', () => {
|
||||
const res = makeRes();
|
||||
await controller.handleChangeEmailConfirm(req, res);
|
||||
expect(res.sent).toMatch(/Email changed successfully/);
|
||||
expect(heard).toContainEqual({
|
||||
user_id: user.id,
|
||||
old_email: user.email ?? null,
|
||||
new_email: newEmail,
|
||||
});
|
||||
|
||||
const after = await server.stores.user.getById(user.id, {
|
||||
force: true,
|
||||
|
||||
@@ -2647,7 +2647,6 @@ export class AuthController extends PuterController {
|
||||
'user.email-changed' as never,
|
||||
{
|
||||
user_id: user.id,
|
||||
old_email: user.email ?? null,
|
||||
new_email: newEmail,
|
||||
} as never,
|
||||
{},
|
||||
|
||||
@@ -15,6 +15,7 @@ export interface OidcLoginContinuation {
|
||||
embeddedInPopup: boolean;
|
||||
openerOrigin: string | null;
|
||||
messageId: string | null;
|
||||
switchedToLogin: boolean;
|
||||
}
|
||||
|
||||
interface OidcReturnDeps {
|
||||
@@ -55,6 +56,7 @@ export function completeOidcLoginReturn(
|
||||
data.userUuid !== user.uuid ||
|
||||
typeof data.redirectUri !== 'string' ||
|
||||
typeof data.embeddedInPopup !== 'boolean' ||
|
||||
typeof data.switchedToLogin !== 'boolean' ||
|
||||
(data.openerOrigin !== null && typeof data.openerOrigin !== 'string') ||
|
||||
(data.messageId !== null && typeof data.messageId !== 'string')
|
||||
) {
|
||||
@@ -89,5 +91,6 @@ export function completeOidcLoginReturn(
|
||||
),
|
||||
);
|
||||
}
|
||||
if (data.switchedToLogin) append('oidc_switched', 'login');
|
||||
return target;
|
||||
}
|
||||
@@ -236,17 +236,6 @@ export class EventsController extends PuterController {
|
||||
});
|
||||
}
|
||||
|
||||
@Post('/schedules/update', {
|
||||
subdomain: 'api',
|
||||
requireAuth: true,
|
||||
allowAccessToken: true,
|
||||
})
|
||||
async updateSchedule(req: Request, res: Response): Promise<void> {
|
||||
const actor = this.#requireActor(req);
|
||||
const { scheduleId, changes } = bodyRecord(req);
|
||||
res.json(await this.#schedules().update(actor, scheduleId, changes));
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /events/schedules/remove — end one schedule. An id the caller does
|
||||
* not hold reads as absent.
|
||||
|
||||
@@ -75,7 +75,6 @@ const fakeSchedules = () =>
|
||||
total: 3,
|
||||
})),
|
||||
get: vi.fn(async () => view),
|
||||
update: vi.fn(async () => view),
|
||||
remove: vi.fn(async () => {}),
|
||||
receivePeerChanges: vi.fn(async () => ({ applied: 2 })),
|
||||
prepareSendAt: vi.fn(),
|
||||
@@ -130,10 +129,6 @@ const routes: Array<[string, (response: Response) => Promise<void>]> = [
|
||||
(r) =>
|
||||
events.getSchedule(req({ params: { scheduleId: SCHEDULE_ID } }), r),
|
||||
],
|
||||
[
|
||||
'POST /events/schedules/update',
|
||||
(r) => events.updateSchedule(req({ body: { scheduleId: SCHEDULE_ID, changes: { every: '2h' } } }), r),
|
||||
],
|
||||
[
|
||||
'POST /events/schedules/remove',
|
||||
(r) =>
|
||||
@@ -255,16 +250,6 @@ describe('with services.schedules', () => {
|
||||
expect(sent.body).toEqual(view);
|
||||
});
|
||||
|
||||
it('updates through update and answers its view', async () => {
|
||||
const schedules = fakeSchedules();
|
||||
setSchedules(schedules);
|
||||
const { response, sent } = res();
|
||||
const changes = { every: '2h' };
|
||||
await events.updateSchedule(req({ body: { scheduleId: SCHEDULE_ID, changes } }), response);
|
||||
expect(schedules.update).toHaveBeenCalledWith(actor, SCHEDULE_ID, changes);
|
||||
expect(sent.body).toEqual(view);
|
||||
});
|
||||
|
||||
it('removes through remove and answers {}', async () => {
|
||||
const schedules = fakeSchedules();
|
||||
setSchedules(schedules);
|
||||
|
||||
@@ -1,77 +0,0 @@
|
||||
import type { Request, Response } from 'express';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { makeActor } from '../../core/actor.js';
|
||||
import type {
|
||||
ScheduleServiceContract,
|
||||
ScheduleView,
|
||||
} from '../../core/schedules.js';
|
||||
import { EventsController } from './EventsController.js';
|
||||
|
||||
const actor = makeActor({
|
||||
user: { id: 1, uuid: 'user-1', username: 'user1' } as never,
|
||||
});
|
||||
const request = (body: unknown) => ({ actor, body }) as Request;
|
||||
const controller = (schedules?: Pick<ScheduleServiceContract, 'update'>) =>
|
||||
new EventsController(
|
||||
{} as never,
|
||||
{} as never,
|
||||
{} as never,
|
||||
{ schedules } as never,
|
||||
{} as never,
|
||||
);
|
||||
|
||||
describe('update schedule route contract', () => {
|
||||
it('answers events_disabled when schedules are unavailable', async () => {
|
||||
await expect(
|
||||
controller().updateSchedule(
|
||||
request({ scheduleId: 'id', changes: {} }),
|
||||
{} as Response,
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
legacyCode: 'events_disabled',
|
||||
statusCode: 503,
|
||||
noAlarm: true,
|
||||
});
|
||||
});
|
||||
|
||||
it('passes the actor, id and changes through and returns the updated view', async () => {
|
||||
const view = {
|
||||
scheduleId: 'id',
|
||||
kind: 'recurring',
|
||||
state: 'suspended',
|
||||
} as ScheduleView;
|
||||
const changes = { cron: '0 9 * * *', tz: 'UTC' };
|
||||
const calls: unknown[][] = [];
|
||||
const events = controller({
|
||||
update: async (...args) => {
|
||||
calls.push(args);
|
||||
return view;
|
||||
},
|
||||
});
|
||||
let sent: unknown;
|
||||
await events.updateSchedule(request({ scheduleId: 'id', changes }), {
|
||||
json: (value: unknown) => {
|
||||
sent = value;
|
||||
},
|
||||
} as Response);
|
||||
expect(calls).toEqual([[actor, 'id', changes]]);
|
||||
expect(sent).toBe(view);
|
||||
});
|
||||
|
||||
it('preserves validation and sending errors from the implementation', async () => {
|
||||
const error = {
|
||||
code: 'events_schedule_sending',
|
||||
message: 'Already sending',
|
||||
};
|
||||
await expect(
|
||||
controller({
|
||||
update: async () => {
|
||||
throw error;
|
||||
},
|
||||
}).updateSchedule(
|
||||
request({ scheduleId: 'id', changes: {} }),
|
||||
{} as Response,
|
||||
),
|
||||
).rejects.toBe(error);
|
||||
});
|
||||
});
|
||||
@@ -27,7 +27,6 @@ import type { Actor } from '../../core/actor.js';
|
||||
import { PuterRouter } from '../../core/http/PuterRouter.js';
|
||||
import { PuterServer } from '../../server.js';
|
||||
import { setupTestServer } from '../../testUtil.js';
|
||||
import { embedFrameAncestors, embedHostOrigin } from './HomepageController.js';
|
||||
|
||||
// ── Test harness ────────────────────────────────────────────────────
|
||||
//
|
||||
@@ -530,46 +529,3 @@ describe('HomepageController GET /sw.js', () => {
|
||||
expect(route?.options.subdomain).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('embedFrameAncestors', () => {
|
||||
const config = {
|
||||
origin: 'https://example.test',
|
||||
domain: 'example.test',
|
||||
api_base_url: 'https://backend.example.test',
|
||||
};
|
||||
|
||||
it('names a sibling subdomain that embeds the browser', () => {
|
||||
expect(
|
||||
embedFrameAncestors(config, 'https://sheets.example.test/doc/1'),
|
||||
).toBe("'self' https://sheets.example.test");
|
||||
});
|
||||
|
||||
it('refuses the configured API host even off the api label', () => {
|
||||
expect(
|
||||
embedFrameAncestors(config, 'https://backend.example.test/x'),
|
||||
).toBe("'self'");
|
||||
});
|
||||
});
|
||||
|
||||
describe('embedHostOrigin', () => {
|
||||
const config = {
|
||||
origin: 'https://example.test',
|
||||
domain: 'example.test',
|
||||
api_base_url: 'https://backend.example.test',
|
||||
static_hosting_domain: 'site.example.test',
|
||||
};
|
||||
|
||||
it.each([
|
||||
['https://sheets.example.test/doc/1', 'https://sheets.example.test'],
|
||||
['https://backend.example.test/x', null],
|
||||
['https://api.example.test/', null],
|
||||
['https://someone.site.example.test/', null],
|
||||
['https://a.sheets.example.test/', null],
|
||||
['http://sheets.example.test/', null],
|
||||
['https://example.test/', null],
|
||||
['not a url', null],
|
||||
[undefined, null],
|
||||
])('%s frames as %s', (referer, expected) => {
|
||||
expect(embedHostOrigin(config, referer)).toBe(expected);
|
||||
});
|
||||
});
|
||||
@@ -22,33 +22,12 @@ import path from 'node:path';
|
||||
import { PuterController } from '../types.js';
|
||||
import { toAppShellView } from '../../util/appShellView.js';
|
||||
import type { PuterRouter } from '../../core/http/PuterRouter';
|
||||
import {
|
||||
isReservedSubdomain,
|
||||
subdomainLabelOf,
|
||||
} from '../../core/http/middleware/hostRedirects.js';
|
||||
import type { IConfig } from '../../types';
|
||||
import type {
|
||||
PuterHomepageService,
|
||||
PageMeta,
|
||||
LaunchOptions,
|
||||
PopupFrame,
|
||||
} from '../../services/homepage/PuterHomepageService';
|
||||
|
||||
/**
|
||||
* GUI popups a first-party page can show in a frame at `/embed/<action>`,
|
||||
* instead of opening `/action/<action>` in a window, plus the upgrade dialog,
|
||||
* which it would otherwise open in a dashboard tab, and the email confirmation,
|
||||
* which it otherwise can't show.
|
||||
*/
|
||||
export const FRAMED_POPUP_ACTIONS = [
|
||||
'sign-in',
|
||||
'show-open-file-picker',
|
||||
'show-save-file-picker',
|
||||
'show-directory-picker',
|
||||
'upgrade',
|
||||
'request-email-confirmation',
|
||||
] as const;
|
||||
|
||||
/** Longest description/og:description/twitter:description on `/app/<name>`. */
|
||||
export const APP_META_DESCRIPTION_MAX = 150;
|
||||
|
||||
@@ -64,9 +43,9 @@ const APP_LANDING_LIMIT = {
|
||||
};
|
||||
|
||||
/**
|
||||
* App descriptions can run long; search snippets and social cards want a short
|
||||
* blurb. Collapses whitespace, then cuts at a word boundary with an ellipsis so
|
||||
* the result never exceeds `APP_META_DESCRIPTION_MAX`.
|
||||
* App descriptions can run long; search snippets and social cards want a
|
||||
* short blurb. Collapses whitespace, then cuts at a word boundary with an
|
||||
* ellipsis so the result never exceeds `APP_META_DESCRIPTION_MAX`.
|
||||
*/
|
||||
export function appMetaDescription(text: string): string {
|
||||
const clean = text.replace(/\s+/g, ' ').trim();
|
||||
@@ -77,64 +56,6 @@ export function appMetaDescription(text: string): string {
|
||||
return `${cut.replace(/[\s,;:.!?…—-]+$/, '')}…`;
|
||||
}
|
||||
|
||||
type EmbedFramingConfig = Pick<
|
||||
IConfig,
|
||||
| 'origin'
|
||||
| 'domain'
|
||||
| 'api_base_url'
|
||||
| 'static_hosting_domain'
|
||||
| 'static_hosting_domain_alt'
|
||||
| 'private_app_hosting_domain'
|
||||
| 'private_app_hosting_domain_alt'
|
||||
>;
|
||||
|
||||
/**
|
||||
* The origin of a first-party page that may frame an `/embed/*` page, named
|
||||
* exactly from the request's referrer, or null. A wildcard over `*.<domain>`
|
||||
* would also admit user-hosted sites whenever the hosting domains sit under it,
|
||||
* so only a single label under `<domain>` qualifies, and not one this server
|
||||
* answers itself (the API, WebDAV, SDK and native-app hosts).
|
||||
*/
|
||||
export function embedHostOrigin(
|
||||
config: EmbedFramingConfig,
|
||||
referer: string | undefined,
|
||||
): string | null {
|
||||
const gui = new URL(config.origin ?? `https://${config.domain}`);
|
||||
let parent: URL;
|
||||
try {
|
||||
parent = new URL(referer ?? '');
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
const host = parent.hostname.toLowerCase();
|
||||
const label = subdomainLabelOf(host, config.domain);
|
||||
const hostingDomains = [
|
||||
config.static_hosting_domain,
|
||||
config.static_hosting_domain_alt,
|
||||
config.private_app_hosting_domain,
|
||||
config.private_app_hosting_domain_alt,
|
||||
]
|
||||
.map((d) => (d ?? '').toLowerCase().split(':')[0])
|
||||
.filter(Boolean);
|
||||
const allowed =
|
||||
parent.protocol === gui.protocol &&
|
||||
parent.port === gui.port &&
|
||||
label !== undefined &&
|
||||
!isReservedSubdomain(label) &&
|
||||
host !== new URL(config.api_base_url).hostname &&
|
||||
!hostingDomains.some((d) => host === d || host.endsWith(`.${d}`));
|
||||
return allowed ? parent.origin : null;
|
||||
}
|
||||
|
||||
/** `frame-ancestors` for an `/embed/*` page: the GUI and {@link embedHostOrigin}. */
|
||||
export function embedFrameAncestors(
|
||||
config: EmbedFramingConfig,
|
||||
referer: string | undefined,
|
||||
): string {
|
||||
const host = embedHostOrigin(config, referer);
|
||||
return host ? `'self' ${host}` : "'self'";
|
||||
}
|
||||
|
||||
/**
|
||||
* Routes that render the Puter GUI shell, plus a catch-all static fallback
|
||||
* under `<gui_assets_root>/src` for non-dist/src paths (images, fonts, lib
|
||||
@@ -169,7 +90,6 @@ export class HomepageController extends PuterController {
|
||||
res: express.Response,
|
||||
metaOverrides: Partial<PageMeta> = {},
|
||||
launch: LaunchOptions = {},
|
||||
popupFrame?: PopupFrame,
|
||||
) => {
|
||||
const meta = { ...defaultMeta(req), ...metaOverrides };
|
||||
const actor =
|
||||
@@ -178,89 +98,13 @@ export class HomepageController extends PuterController {
|
||||
actor?: Parameters<typeof homepage.send>[0]['actor'];
|
||||
}
|
||||
).actor ?? null;
|
||||
await homepage.send({ req, res, actor }, meta, launch, popupFrame);
|
||||
await homepage.send({ req, res, actor }, meta, launch);
|
||||
};
|
||||
|
||||
// -- Root + path-aliased shell routes ------------------------
|
||||
|
||||
router.get('/', {}, (req, res) => sendShell(req, res));
|
||||
|
||||
/**
|
||||
* Headers shared by the `/embed/*` pages; false when the request was
|
||||
* redirected instead. SDK bootstrap parameters must not change an
|
||||
* embed's account or API origin, so a query string is dropped. Checked
|
||||
* on the raw URL; `req.query` drops pairs past the parser's limit.
|
||||
*/
|
||||
const prepareEmbed = (
|
||||
req: express.Request,
|
||||
res: express.Response,
|
||||
path: string,
|
||||
): boolean => {
|
||||
if (req.originalUrl.includes('?')) {
|
||||
res.redirect(path);
|
||||
return false;
|
||||
}
|
||||
res.removeHeader('X-Frame-Options');
|
||||
res.setHeader(
|
||||
'Content-Security-Policy',
|
||||
`frame-ancestors ${embedFrameAncestors(this.config, req.get('referer'))}`,
|
||||
);
|
||||
res.setHeader('Cache-Control', 'no-store');
|
||||
return true;
|
||||
};
|
||||
|
||||
router.get('/embed/apps', {}, async (req, res) => {
|
||||
if (!prepareEmbed(req, res, '/embed/apps')) return;
|
||||
// Shown to visitors without a session; a signed-in embed fetches
|
||||
// its own list, so a failure here only costs signed-out visitors.
|
||||
const guestApps = await this.services.recommendedApps
|
||||
.getRecommendedApps()
|
||||
.catch((): Array<Record<string, unknown>> => []);
|
||||
res.send(homepage.renderAppBrowserEmbed({ guestApps }));
|
||||
});
|
||||
|
||||
router.get('/embed/feedback', {}, (req, res) => {
|
||||
if (!prepareEmbed(req, res, '/embed/feedback')) return;
|
||||
// The feedback is recorded against this origin, which
|
||||
// frame-ancestors has just pinned to the actual framing page.
|
||||
res.send(
|
||||
homepage.renderFeedbackEmbed({
|
||||
hostOrigin: embedHostOrigin(
|
||||
this.config,
|
||||
req.get('referer'),
|
||||
),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
// The GUI runs the popup's own boot, answering the framing page
|
||||
// instead of an opener.
|
||||
for (const action of FRAMED_POPUP_ACTIONS) {
|
||||
const embedPath = `/embed/${action}`;
|
||||
router.get(embedPath, {}, async (req, res) => {
|
||||
// Routes match case-insensitively and with a trailing slash,
|
||||
// and the shell boots the desktop for any other spelling, so
|
||||
// only the exact one may be framed.
|
||||
if (req.originalUrl !== embedPath) {
|
||||
res.redirect(embedPath);
|
||||
return;
|
||||
}
|
||||
if (!prepareEmbed(req, res, embedPath)) return;
|
||||
await sendShell(
|
||||
req,
|
||||
res,
|
||||
{},
|
||||
{},
|
||||
{
|
||||
hostOrigin: embedHostOrigin(
|
||||
this.config,
|
||||
req.get('referer'),
|
||||
),
|
||||
},
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
router.get('/settings', {}, (req, res) => sendShell(req, res));
|
||||
router.get('/settings/*splat', {}, (req, res) => sendShell(req, res));
|
||||
|
||||
|
||||
@@ -1620,7 +1620,7 @@ describe('OIDCController signup callback', () => {
|
||||
expect(captured.redirectUrl).toContain('action=signup');
|
||||
});
|
||||
|
||||
it('creates a fresh user on first signup', async () => {
|
||||
it('creates a fresh user without oidc_switched on first signup', async () => {
|
||||
const state = oidc().signState({
|
||||
provider: 'custom',
|
||||
redirect_uri: TEST_ORIGIN + '/',
|
||||
@@ -1646,9 +1646,11 @@ describe('OIDCController signup callback', () => {
|
||||
);
|
||||
expect(captured.cookies).toHaveLength(1);
|
||||
expect(captured.redirectUrl).toBe(TEST_ORIGIN + '/');
|
||||
// Fresh creation — must NOT advertise "you've actually been signed in".
|
||||
expect(captured.redirectUrl).not.toContain('oidc_switched=login');
|
||||
});
|
||||
|
||||
it('signs in to an already-linked account on signup, returning as a login would', async () => {
|
||||
it('adds oidc_switched=login when signup hits an already-linked account', async () => {
|
||||
// Seed an account linked on (provider, sub) so the signup callback
|
||||
// resolves to `linked-sub` rather than creating a new user.
|
||||
const sub = `sub-${Math.random().toString(36).slice(2, 8)}`;
|
||||
@@ -1684,7 +1686,7 @@ describe('OIDCController signup callback', () => {
|
||||
res,
|
||||
);
|
||||
expect(captured.cookies).toHaveLength(1);
|
||||
expect(captured.redirectUrl).toBe(TEST_ORIGIN + '/');
|
||||
expect(captured.redirectUrl).toContain('oidc_switched=login');
|
||||
});
|
||||
|
||||
it('redirects to action=signup with auth_error when user resolution fails (unconfirmed email)', async () => {
|
||||
|
||||
@@ -688,7 +688,14 @@ export class OIDCController extends PuterController {
|
||||
);
|
||||
}
|
||||
|
||||
await this.#finishLogin(req, res, user, stateDecoded);
|
||||
// If we landed on an existing account (either via the
|
||||
// provider_sub or via email match), signal the GUI so it can
|
||||
// render a "signed in" flow rather than "account created".
|
||||
const extra =
|
||||
resolved.origin === 'created'
|
||||
? undefined
|
||||
: { oidc_switched: 'login' };
|
||||
await this.#finishLogin(req, res, user, stateDecoded, extra);
|
||||
};
|
||||
router.get('/auth/oidc/callback/signup', cbOpts, signupCb);
|
||||
router.post('/auth/oidc/callback/signup', cbOpts, signupCb);
|
||||
@@ -827,14 +834,17 @@ if (window.opener) {
|
||||
attempt = 0,
|
||||
): Promise<
|
||||
| { error: string; code?: string; requestCode?: string }
|
||||
| { user: import('../../stores/user/UserStore.js').UserRow }
|
||||
| {
|
||||
user: import('../../stores/user/UserStore.js').UserRow;
|
||||
origin: 'linked-sub' | 'linked-email' | 'created';
|
||||
}
|
||||
> {
|
||||
// 1. Existing provider/sub link.
|
||||
const linked = await this.services.oidc.findUserByProviderSub(
|
||||
provider,
|
||||
userinfo.sub,
|
||||
);
|
||||
if (linked) return { user: linked };
|
||||
if (linked) return { user: linked, origin: 'linked-sub' };
|
||||
|
||||
// 2/3. Email match branch.
|
||||
const claimedEmail =
|
||||
@@ -871,7 +881,7 @@ if (window.opener) {
|
||||
error: outcome.error ?? 'Failed to link provider.',
|
||||
};
|
||||
}
|
||||
return { user: byEmail };
|
||||
return { user: byEmail, origin: 'linked-email' };
|
||||
}
|
||||
}
|
||||
|
||||
@@ -901,7 +911,7 @@ if (window.opener) {
|
||||
requestCode: outcome.requestCode,
|
||||
};
|
||||
}
|
||||
return { user: outcome.user };
|
||||
return { user: outcome.user, origin: 'created' };
|
||||
}
|
||||
|
||||
async #processCallback(
|
||||
@@ -990,6 +1000,7 @@ if (window.opener) {
|
||||
res: Response,
|
||||
user: UserRow,
|
||||
stateDecoded: Record<string, unknown>,
|
||||
extraQueryParams?: Record<string, string>,
|
||||
): Promise<void> {
|
||||
const origin = (this.config.origin ?? '').replace(/\/$/, '');
|
||||
const continuation: OidcLoginContinuation = {
|
||||
@@ -1007,6 +1018,7 @@ if (window.opener) {
|
||||
stateDecoded.msg_id == null
|
||||
? null
|
||||
: String(stateDecoded.msg_id),
|
||||
switchedToLogin: extraQueryParams?.oidc_switched === 'login',
|
||||
};
|
||||
if (user.otp_enabled) {
|
||||
const challenge = this.services.token.sign(
|
||||
|
||||
@@ -139,7 +139,7 @@ describe('OIDC account second-factor login', () => {
|
||||
expect(await response.json()).toMatchObject({
|
||||
proceed: true,
|
||||
next_step: 'complete',
|
||||
redirectUrl: `${env.origin}/dashboard`,
|
||||
redirectUrl: `${env.origin}/dashboard${flow === 'signup' ? '?oidc_switched=login' : ''}`,
|
||||
});
|
||||
expect(
|
||||
response.headers
|
||||
|
||||
@@ -201,7 +201,6 @@ describe('PuterAIController.openaiResponses parameter forwarding', () => {
|
||||
context_management: [{ type: 'compaction' }],
|
||||
prompt_cache_key: 'ck',
|
||||
prompt_cache_retention: '24h',
|
||||
prompt_cache_options: { ttl: '12h', prewarm: true },
|
||||
store: true,
|
||||
truncation: 'auto',
|
||||
service_tier: 'default',
|
||||
@@ -227,9 +226,6 @@ describe('PuterAIController.openaiResponses parameter forwarding', () => {
|
||||
context_management: [{ type: 'compaction' }],
|
||||
prompt_cache_key: 'ck',
|
||||
prompt_cache_retention: '24h',
|
||||
// Forwarded as sent; the provider keeps only the fields it
|
||||
// accepts.
|
||||
prompt_cache_options: { ttl: '12h', prewarm: true },
|
||||
// forced false regardless of the client's request.
|
||||
store: false,
|
||||
truncation: 'auto',
|
||||
@@ -363,18 +359,13 @@ describe('PuterAIController.openaiResponses input normalization', () => {
|
||||
role: 'user',
|
||||
content: [
|
||||
'a bare string part',
|
||||
{
|
||||
type: 'input_text',
|
||||
text: 'typed text',
|
||||
prompt_cache_breakpoint: { mode: 'explicit', ttl: '12h' },
|
||||
},
|
||||
{ type: 'input_text', text: 'typed text' },
|
||||
{ type: 'output_text', text: 'echoed text' },
|
||||
{
|
||||
type: 'input_image',
|
||||
detail: 'high',
|
||||
image_url: 'https://img.test/a.png',
|
||||
file_id: 'file_img',
|
||||
prompt_cache_breakpoint: { mode: 'explicit' },
|
||||
},
|
||||
{ type: 'input_audio', input_audio: { data: 'AAA' } },
|
||||
{
|
||||
@@ -395,18 +386,13 @@ describe('PuterAIController.openaiResponses input normalization', () => {
|
||||
role: 'user',
|
||||
content: [
|
||||
{ type: 'text', text: 'a bare string part' },
|
||||
{
|
||||
type: 'text',
|
||||
text: 'typed text',
|
||||
prompt_cache_breakpoint: { mode: 'explicit', ttl: '12h' },
|
||||
},
|
||||
{ type: 'text', text: 'typed text' },
|
||||
{ type: 'text', text: 'echoed text' },
|
||||
{
|
||||
type: 'image_url',
|
||||
detail: 'high',
|
||||
image_url: { url: 'https://img.test/a.png' },
|
||||
file_id: 'file_img',
|
||||
prompt_cache_breakpoint: { mode: 'explicit' },
|
||||
},
|
||||
{ type: 'input_audio', input_audio: { data: 'AAA' } },
|
||||
{
|
||||
@@ -653,33 +639,6 @@ describe('PuterAIController.openaiResponses output items', () => {
|
||||
// -- /openai/v1/chat/completions edges -------------------------------
|
||||
|
||||
describe('PuterAIController.openaiChatCompletions translation edges', () => {
|
||||
it('forwards the prompt-cache fields to the driver', async () => {
|
||||
const completeSpy = stubChatComplete({
|
||||
message: { role: 'assistant', content: 'ok' },
|
||||
});
|
||||
|
||||
const { res } = makeRes();
|
||||
await controller.openaiChatCompletions(
|
||||
makeReq({
|
||||
body: {
|
||||
model: 'gpt-test',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
prompt_cache_key: 'ck',
|
||||
prompt_cache_retention: 'in_memory',
|
||||
prompt_cache_options: { ttl: '30m' },
|
||||
},
|
||||
}),
|
||||
res,
|
||||
);
|
||||
|
||||
const args = completeSpy.mock.calls[0]![0] as Record<string, unknown>;
|
||||
expect(args).toMatchObject({
|
||||
prompt_cache_key: 'ck',
|
||||
prompt_cache_retention: 'in_memory',
|
||||
prompt_cache_options: { ttl: '30m' },
|
||||
});
|
||||
});
|
||||
|
||||
it('derives tool_calls from tool_use content blocks when the message has none', async () => {
|
||||
stubChatComplete({
|
||||
message: {
|
||||
|
||||
@@ -447,7 +447,6 @@ export class PuterAIController extends PuterController {
|
||||
body.reasoning_effort as ICompleteArguments['reasoning_effort'],
|
||||
}
|
||||
: {}),
|
||||
...promptCacheArgsFromBody(body),
|
||||
...openaiCompatProvider(body),
|
||||
...(body.fallback === false ? { fallback: false } : {}),
|
||||
};
|
||||
@@ -843,7 +842,15 @@ export class PuterAIController extends PuterController {
|
||||
body.compaction as ICompleteArguments['compaction'],
|
||||
}
|
||||
: {}),
|
||||
...promptCacheArgsFromBody(body),
|
||||
...(body.prompt_cache_key
|
||||
? { prompt_cache_key: String(body.prompt_cache_key) }
|
||||
: {}),
|
||||
...(body.prompt_cache_retention
|
||||
? {
|
||||
prompt_cache_retention:
|
||||
body.prompt_cache_retention as ICompleteArguments['prompt_cache_retention'],
|
||||
}
|
||||
: {}),
|
||||
// forced off regardless of what the caller sent — OpenAI
|
||||
// defaults `store` to true, which would persist state server-side
|
||||
// on OpenAI's end that this route gives no way to retrieve.
|
||||
@@ -1336,33 +1343,6 @@ const openaiCompatProvider = (
|
||||
return { provider: DEFAULTS.openaiChat };
|
||||
};
|
||||
|
||||
/**
|
||||
* The prompt-cache fields both OpenAI-shaped routes forward as sent; the
|
||||
* provider checks the values.
|
||||
*/
|
||||
const promptCacheArgsFromBody = (
|
||||
body: Record<string, unknown>,
|
||||
): Pick<
|
||||
ICompleteArguments,
|
||||
'prompt_cache_key' | 'prompt_cache_retention' | 'prompt_cache_options'
|
||||
> => ({
|
||||
...(body.prompt_cache_key
|
||||
? { prompt_cache_key: String(body.prompt_cache_key) }
|
||||
: {}),
|
||||
...(body.prompt_cache_retention
|
||||
? {
|
||||
prompt_cache_retention:
|
||||
body.prompt_cache_retention as ICompleteArguments['prompt_cache_retention'],
|
||||
}
|
||||
: {}),
|
||||
...(body.prompt_cache_options
|
||||
? {
|
||||
prompt_cache_options:
|
||||
body.prompt_cache_options as ICompleteArguments['prompt_cache_options'],
|
||||
}
|
||||
: {}),
|
||||
});
|
||||
|
||||
const randomId = (): string => crypto.randomUUID().replace(/-/g, '');
|
||||
const generateId = (prefix: string): string => `${prefix}_${randomId()}`;
|
||||
|
||||
@@ -1610,24 +1590,12 @@ const parseJsonMaybe = (value: unknown): unknown => {
|
||||
}
|
||||
};
|
||||
|
||||
/** An OpenAI `prompt_cache_breakpoint` marker, kept on the translated part. */
|
||||
const cacheBreakpointOf = (
|
||||
p: Record<string, unknown>,
|
||||
): { prompt_cache_breakpoint?: unknown } =>
|
||||
p.prompt_cache_breakpoint && typeof p.prompt_cache_breakpoint === 'object'
|
||||
? { prompt_cache_breakpoint: p.prompt_cache_breakpoint }
|
||||
: {};
|
||||
|
||||
const normalizeContentPart = (part: unknown): Record<string, unknown> => {
|
||||
if (typeof part === 'string') return { type: 'text', text: part };
|
||||
if (!part || typeof part !== 'object') return { type: 'text', text: '' };
|
||||
const p = part as Record<string, unknown>;
|
||||
if (p.type === 'input_text' || p.type === 'output_text') {
|
||||
return {
|
||||
type: 'text',
|
||||
text: String(p.text ?? ''),
|
||||
...cacheBreakpointOf(p),
|
||||
};
|
||||
return { type: 'text', text: String(p.text ?? '') };
|
||||
}
|
||||
if (p.type === 'input_image') {
|
||||
return {
|
||||
@@ -1635,7 +1603,6 @@ const normalizeContentPart = (part: unknown): Record<string, unknown> => {
|
||||
...(p.detail ? { detail: p.detail } : {}),
|
||||
...(p.image_url ? { image_url: { url: p.image_url } } : {}),
|
||||
...(p.file_id ? { file_id: p.file_id } : {}),
|
||||
...cacheBreakpointOf(p),
|
||||
};
|
||||
}
|
||||
if (p.type === 'input_audio')
|
||||
@@ -1647,7 +1614,6 @@ const normalizeContentPart = (part: unknown): Record<string, unknown> => {
|
||||
...(p.file_id ? { file_id: p.file_id } : {}),
|
||||
...(p.file_url ? { file_url: p.file_url } : {}),
|
||||
...(p.filename ? { filename: p.filename } : {}),
|
||||
...cacheBreakpointOf(p),
|
||||
};
|
||||
}
|
||||
return p;
|
||||
|
||||
@@ -187,9 +187,9 @@ describe('createUserSubdomainNotFound', () => {
|
||||
});
|
||||
|
||||
it('passes through reserved subdomains (api, js, native apps, etc.)', () => {
|
||||
// `api`, `js`, `dav`, `docs`, `developer`, `about`, `apps`,
|
||||
// `api`, `js`, `dav`, `docs`, `developer`, `editor`, `pdf`,
|
||||
// `puter-app-icons`, etc. all pass through.
|
||||
for (const sub of ['api', 'js', 'docs', 'about', 'puter-app-icons']) {
|
||||
for (const sub of ['api', 'js', 'docs', 'editor', 'puter-app-icons']) {
|
||||
const { out, next } = run(
|
||||
createUserSubdomainNotFound(config),
|
||||
makeReq({
|
||||
@@ -865,13 +865,13 @@ describe('createNativeAppStatic', () => {
|
||||
|
||||
beforeAll(() => {
|
||||
root = mkdtempSync(nodePath.join(tmpdir(), 'native-apps-'));
|
||||
mkdirSync(nodePath.join(root, 'about', 'assets'), { recursive: true });
|
||||
mkdirSync(nodePath.join(root, 'editor', 'assets'), { recursive: true });
|
||||
writeFileSync(
|
||||
nodePath.join(root, 'about', 'index.html'),
|
||||
'<h1>about</h1>',
|
||||
nodePath.join(root, 'editor', 'index.html'),
|
||||
'<h1>editor</h1>',
|
||||
);
|
||||
writeFileSync(
|
||||
nodePath.join(root, 'about', 'assets', 'app.js'),
|
||||
nodePath.join(root, 'editor', 'assets', 'app.js'),
|
||||
'console.log(1);',
|
||||
);
|
||||
mkdirSync(nodePath.join(root, 'docs', 'dist'), { recursive: true });
|
||||
@@ -893,7 +893,7 @@ describe('createNativeAppStatic', () => {
|
||||
|
||||
it('leaves a native app name alone on a host outside our domains', async () => {
|
||||
const req = staticReq({
|
||||
subdomains: ['com', 'example', 'about'],
|
||||
subdomains: ['com', 'example', 'editor'],
|
||||
path: '/index.html',
|
||||
});
|
||||
(req as Request & { is_custom_domain?: boolean }).is_custom_domain =
|
||||
@@ -907,7 +907,7 @@ describe('createNativeAppStatic', () => {
|
||||
const { out, next } = await runStatic(
|
||||
createNativeAppStatic(config),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'about'],
|
||||
subdomains: ['localhost', 'puter', 'editor'],
|
||||
path: '/index.html',
|
||||
}),
|
||||
);
|
||||
@@ -915,24 +915,17 @@ describe('createNativeAppStatic', () => {
|
||||
expect(next).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
// `editor`, `markus` and `pdf` are web apps on standard host routing,
|
||||
// even with a folder of their own under the native apps root.
|
||||
it.each(['api', 'editor', 'markus', 'pdf'])(
|
||||
'passes %s through, as it is not a native app',
|
||||
async (sub) => {
|
||||
mkdirSync(nodePath.join(root, sub), { recursive: true });
|
||||
writeFileSync(nodePath.join(root, sub, 'index.html'), sub);
|
||||
const { out, next } = await runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', sub],
|
||||
path: '/index.html',
|
||||
}),
|
||||
);
|
||||
expect(out.sent).toBeUndefined();
|
||||
expect(next).toHaveBeenCalledTimes(1);
|
||||
},
|
||||
);
|
||||
it('passes through subdomains that are not native apps', async () => {
|
||||
const { out, next } = await runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'api'],
|
||||
path: '/index.html',
|
||||
}),
|
||||
);
|
||||
expect(out.sent).toBeUndefined();
|
||||
expect(next).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('passes through when there is no subdomain at all', async () => {
|
||||
const { next } = await runStatic(
|
||||
@@ -946,13 +939,13 @@ describe('createNativeAppStatic', () => {
|
||||
const { out, next } = await runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'about'],
|
||||
subdomains: ['localhost', 'puter', 'editor'],
|
||||
path: '/index.html',
|
||||
}),
|
||||
);
|
||||
expect(out.sent).toEqual({
|
||||
path: '/index.html',
|
||||
root: nodePath.join(root, 'about'),
|
||||
root: nodePath.join(root, 'editor'),
|
||||
});
|
||||
expect(next).not.toHaveBeenCalled();
|
||||
});
|
||||
@@ -961,11 +954,11 @@ describe('createNativeAppStatic', () => {
|
||||
const { out } = await runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'ABOUT'],
|
||||
subdomains: ['localhost', 'puter', 'EDITOR'],
|
||||
path: '/index.html',
|
||||
}),
|
||||
);
|
||||
expect(out.sent?.root).toBe(nodePath.join(root, 'about'));
|
||||
expect(out.sent?.root).toBe(nodePath.join(root, 'editor'));
|
||||
});
|
||||
|
||||
it('serves docs out of its dist/ subdirectory, not the app root', async () => {
|
||||
@@ -986,7 +979,7 @@ describe('createNativeAppStatic', () => {
|
||||
const { out, next } = await runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'about'],
|
||||
subdomains: ['localhost', 'puter', 'editor'],
|
||||
path: '/assets',
|
||||
originalUrl: '/assets?v=2',
|
||||
}),
|
||||
@@ -1002,7 +995,7 @@ describe('createNativeAppStatic', () => {
|
||||
const { out } = await runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'about'],
|
||||
subdomains: ['localhost', 'puter', 'editor'],
|
||||
path: '/assets/',
|
||||
}),
|
||||
);
|
||||
@@ -1014,7 +1007,7 @@ describe('createNativeAppStatic', () => {
|
||||
const { out, next } = await runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'about'],
|
||||
subdomains: ['localhost', 'puter', 'editor'],
|
||||
path: '/nope.html',
|
||||
}),
|
||||
);
|
||||
@@ -1026,7 +1019,7 @@ describe('createNativeAppStatic', () => {
|
||||
const { next } = await runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'about'],
|
||||
subdomains: ['localhost', 'puter', 'editor'],
|
||||
path: '/index.html',
|
||||
}),
|
||||
new Error('send failed'),
|
||||
@@ -1039,7 +1032,7 @@ describe('createNativeAppStatic', () => {
|
||||
runStatic(
|
||||
withRoot(),
|
||||
staticReq({
|
||||
subdomains: ['localhost', 'puter', 'about'],
|
||||
subdomains: ['localhost', 'puter', 'editor'],
|
||||
path: '/../../etc/passwd',
|
||||
}),
|
||||
),
|
||||
|
||||
@@ -27,7 +27,15 @@ import { HttpError, isHttpError } from '../HttpError';
|
||||
import { isLocalWorkerHost } from './localWorkerProxy';
|
||||
|
||||
/** Native-app subdomains served via `nativeAppStatic`. */
|
||||
const NATIVE_APP_SUBDOMAINS = ['about', 'developer', 'docs', 'apps'] as const;
|
||||
const NATIVE_APP_SUBDOMAINS = [
|
||||
'about',
|
||||
'developer',
|
||||
'docs',
|
||||
'editor',
|
||||
'markus',
|
||||
'pdf',
|
||||
'apps',
|
||||
] as const;
|
||||
|
||||
/** Subset served out of a `dist/` subdirectory rather than the app root. */
|
||||
const NATIVE_APPS_WITH_DIST = new Set(['docs', 'developer']);
|
||||
@@ -103,10 +111,6 @@ export const subdomainLabelOf = (
|
||||
return label && !label.includes('.') ? label : undefined;
|
||||
};
|
||||
|
||||
/** Whether core itself answers `<name>.<domain>` (`www` redirects to the root). */
|
||||
export const isReservedSubdomain = (name: string): boolean =>
|
||||
name === 'www' || RESERVED_SUBDOMAINS.has(name);
|
||||
|
||||
/**
|
||||
* Record a claim on `<name>.<domain>`. Throws on a malformed or uppercase
|
||||
* label, a subdomain core already handles, or a name that is already claimed.
|
||||
@@ -119,7 +123,7 @@ export const addClaimedSubdomain = (
|
||||
if (typeof name !== 'string' || !SUBDOMAIN_LABEL.test(name)) {
|
||||
throw new Error(`Invalid subdomain label: ${JSON.stringify(name)}`);
|
||||
}
|
||||
if (isReservedSubdomain(name)) {
|
||||
if (name === 'www' || RESERVED_SUBDOMAINS.has(name)) {
|
||||
throw new Error(`Subdomain '${name}' is reserved`);
|
||||
}
|
||||
if (claims.has(name)) {
|
||||
|
||||
@@ -170,16 +170,8 @@ export interface ScheduleServiceContract {
|
||||
request: ScheduleListRequest,
|
||||
): Promise<PageResult<ScheduleView>>;
|
||||
get(actor: Actor, scheduleId: unknown): Promise<ScheduleView>;
|
||||
update(
|
||||
actor: Actor,
|
||||
scheduleId: unknown,
|
||||
changes: unknown,
|
||||
): Promise<ScheduleView>;
|
||||
remove(actor: Actor, scheduleId: unknown): Promise<void>;
|
||||
/** `results`: each change's index outcome, in order (`null` if dropped). */
|
||||
receivePeerChanges(
|
||||
body: unknown,
|
||||
): Promise<{ applied: number; results?: Array<number | null> }>;
|
||||
receivePeerChanges(body: unknown): Promise<{ applied: number }>;
|
||||
/** Checks a caller supplying `sendAt`; `events_disabled` when off. */
|
||||
prepareSendAt(actor: Actor, sendAt: unknown): Promise<DeferredSendPlan>;
|
||||
createDeferredAppMessage(
|
||||
|
||||
@@ -2222,166 +2222,3 @@ describe('ChatCompletionDriver.countTokens', () => {
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
});
|
||||
});
|
||||
|
||||
describe('ChatCompletionDriver service speed', () => {
|
||||
it.each([
|
||||
[{ serviceSpeed: 'ultrafast' }, 'fast', 0.028],
|
||||
[{ speed: 'fast' }, 'fast', 0.028],
|
||||
[{ serviceSpeed: 'standard', speed: 'fast' }, 'standard', 0.014],
|
||||
] as const)('uses the Claude catalog rate card for %j', async (options, speed, cents) => {
|
||||
const sent: Record<string, unknown>[] = [];
|
||||
vi.stubGlobal('fetch', async (input: Request | string, init?: RequestInit) => {
|
||||
const body = input instanceof Request
|
||||
? await input.clone().json()
|
||||
: JSON.parse(String(init?.body));
|
||||
sent.push(body);
|
||||
return new Response(JSON.stringify({
|
||||
id: 'claude-speed-test', type: 'message', role: 'assistant',
|
||||
model: body.model, stop_reason: 'end_turn',
|
||||
content: [{ type: 'text', text: 'OK' }],
|
||||
usage: { input_tokens: 10, output_tokens: 5, speed },
|
||||
}), { headers: { 'content-type': 'application/json' } });
|
||||
});
|
||||
const d = new ChatCompletionDriver(
|
||||
{ providers: { claude: { apiKey: 'test-key' }, ollama: { enabled: false } } } as never,
|
||||
server.clients, server.stores, server.services,
|
||||
);
|
||||
try {
|
||||
await d.onServerStart();
|
||||
const result = await withTestActor(() => d.complete({
|
||||
model: 'claude-opus-5-5', provider: 'claude',
|
||||
messages: [{ role: 'user', content: 'Hello' }],
|
||||
fallback: false, max_tokens: 32, normalize: true, ...options,
|
||||
}));
|
||||
expect(sent).toHaveLength(1);
|
||||
expect(sent[0]!.speed).toBe(speed === 'fast' ? 'fast' : undefined);
|
||||
expect(result).toMatchObject({
|
||||
message: { content: 'OK' }, usage: { usd_cents: cents },
|
||||
});
|
||||
} finally {
|
||||
d.onServerShutdown();
|
||||
vi.unstubAllGlobals();
|
||||
}
|
||||
});
|
||||
|
||||
it.each([
|
||||
null, false, true, 0, 1, -1, NaN, Infinity, '', 'invalid', 'FAST',
|
||||
'fast ', ' fast', 'ultraFast', [], ['fast'], {}, { speed: 'fast' },
|
||||
{ toString: 'fast' },
|
||||
])('rejects invalid speed %j before sending a request', async (serviceSpeed) => {
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
driver.complete({
|
||||
model: 'fake',
|
||||
messages: [{ role: 'user', content: 'Hello' }],
|
||||
serviceSpeed: serviceSpeed as never,
|
||||
}),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
statusCode: 400,
|
||||
legacyCode: 'invalid_service_speed',
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
['openai-responses', true],
|
||||
['openai-responses', false],
|
||||
['openai-completion', false],
|
||||
] as const)(
|
||||
'settles unavailable tiers at standard rates on %s with provider fallback disabled (admission rejection: %s)',
|
||||
async (provider, rejectFast) => {
|
||||
const sent: Record<string, unknown>[] = [];
|
||||
vi.stubGlobal(
|
||||
'fetch',
|
||||
async (_input: unknown, init?: RequestInit) => {
|
||||
const body =
|
||||
_input instanceof Request
|
||||
? await _input.clone().json()
|
||||
: JSON.parse(String(init?.body));
|
||||
sent.push(body);
|
||||
if (rejectFast && body.service_tier !== 'default') {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
error: {
|
||||
message: `${body.service_tier} service_tier capacity unavailable`,
|
||||
type: 'invalid_request_error',
|
||||
},
|
||||
}),
|
||||
{
|
||||
status: 400,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
},
|
||||
);
|
||||
}
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
id: 'response-test',
|
||||
object: 'response',
|
||||
model: body.model,
|
||||
status: 'completed',
|
||||
service_tier: 'default',
|
||||
output: [
|
||||
{
|
||||
type: 'message',
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{ type: 'output_text', text: 'OK' },
|
||||
],
|
||||
},
|
||||
],
|
||||
usage: {
|
||||
input_tokens: 10,
|
||||
output_tokens: 2,
|
||||
input_tokens_details: { cached_tokens: 0 },
|
||||
},
|
||||
}),
|
||||
{ headers: { 'content-type': 'application/json' } },
|
||||
);
|
||||
},
|
||||
);
|
||||
const d = new ChatCompletionDriver(
|
||||
{
|
||||
providers: {
|
||||
'openai-completion': { apiKey: 'test-key' },
|
||||
ollama: { enabled: false },
|
||||
},
|
||||
} as never,
|
||||
server.clients,
|
||||
server.stores,
|
||||
server.services,
|
||||
);
|
||||
try {
|
||||
await d.onServerStart();
|
||||
const result = (await withTestActor(() =>
|
||||
d.complete({
|
||||
model: 'gpt-6.1-sol',
|
||||
provider,
|
||||
messages: [{ role: 'user', content: 'Hello' }],
|
||||
serviceSpeed: 'ultrafast',
|
||||
fallback: false,
|
||||
max_tokens: 32,
|
||||
normalize: true,
|
||||
}),
|
||||
)) as {
|
||||
message: { content: string };
|
||||
usageDetails: { serviceSpeed: string };
|
||||
usage: { usd_cents: number };
|
||||
};
|
||||
expect(sent.map((body) => body.service_tier)).toEqual(
|
||||
rejectFast
|
||||
? ['ultrafast', 'priority', 'default']
|
||||
: ['ultrafast'],
|
||||
);
|
||||
expect(sent.every((body) => body.model === 'gpt-6.1-sol')).toBe(
|
||||
true,
|
||||
);
|
||||
expect(result.message.content).toBe('OK');
|
||||
expect(result.usageDetails.serviceSpeed).toBe('standard');
|
||||
expect(result.usage.usd_cents).toBeCloseTo(0.004, 8);
|
||||
} finally {
|
||||
d.onServerShutdown();
|
||||
vi.unstubAllGlobals();
|
||||
}
|
||||
},
|
||||
);
|
||||
});
|
||||
@@ -78,11 +78,6 @@ import type {
|
||||
UsageDetails,
|
||||
} from './types.js';
|
||||
import { normalize_tools_object } from './utils/FunctionCalling.js';
|
||||
import {
|
||||
selectServiceSpeed,
|
||||
lowerServiceSpeed,
|
||||
isServiceSpeedUnavailable,
|
||||
} from './utils/serviceSpeed.js';
|
||||
import {
|
||||
messagesHavePuterPaths,
|
||||
modelSupportsModality,
|
||||
@@ -679,16 +674,6 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
}
|
||||
|
||||
async complete(args: ICompleteArguments): Promise<IChatCompleteResult> {
|
||||
if (
|
||||
args.serviceSpeed !== undefined &&
|
||||
!['standard', 'fast', 'ultrafast'].includes(args.serviceSpeed)
|
||||
) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'serviceSpeed must be standard, fast, or ultrafast',
|
||||
{ legacyCode: 'invalid_service_speed' },
|
||||
);
|
||||
}
|
||||
const abortSignal = Context.get('abortSignal');
|
||||
abortSignal?.throwIfAborted();
|
||||
const actor = Context.get('actor');
|
||||
@@ -715,16 +700,15 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
args.model = this.#providers[intendedProvider].getDefaultModel();
|
||||
}
|
||||
|
||||
const initialModel = await this.#resolveLoadedModel(
|
||||
let model = await this.#resolveLoadedModel(
|
||||
args.model,
|
||||
intendedProvider,
|
||||
);
|
||||
if (!initialModel) {
|
||||
if (!model) {
|
||||
throw new HttpError(400, `Model not found: ${args.model}`, {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
let model: IChatModel = initialModel;
|
||||
// A pinned provider without a route for the model normally resolves
|
||||
// to another vendor; with `fallback: false` that is the one thing the
|
||||
// caller asked us not to do.
|
||||
@@ -856,6 +840,18 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
// The instant the current attempt is priced at, from its hold to its
|
||||
// charge, so a rate change mid-request can't split the two.
|
||||
let pricingTime = new Date();
|
||||
if (!useFakeProvider) {
|
||||
({
|
||||
hold,
|
||||
fundsCap,
|
||||
costFactor,
|
||||
promptTokenEstimate: attemptPromptTokenEstimate,
|
||||
} = await this.#applyCreditGate(actor, model, args, {
|
||||
promptTokenEstimate,
|
||||
requestedMaxTokens,
|
||||
pricingTime,
|
||||
}));
|
||||
}
|
||||
|
||||
// First attempt
|
||||
const provider = this.#providers[model.provider!];
|
||||
@@ -903,134 +899,24 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
return attempt;
|
||||
};
|
||||
|
||||
let variantSpeed: ICompleteArguments['serviceSpeed'];
|
||||
let creditGateError: unknown;
|
||||
const runAttempt = async (
|
||||
attemptProvider: IChatProvider,
|
||||
baseModel: IChatModel,
|
||||
attemptModel: IChatModel,
|
||||
) => {
|
||||
const offers =
|
||||
args.serviceSpeed !== undefined &&
|
||||
args.serviceSpeed !== 'standard'
|
||||
? ((await attemptProvider.serviceSpeedModels?.(
|
||||
baseModel,
|
||||
)) ?? {})
|
||||
: {};
|
||||
let requestedSpeed =
|
||||
args.serviceSpeed ??
|
||||
(baseModel.provider === 'claude' && args.speed === 'fast'
|
||||
? 'fast'
|
||||
: undefined);
|
||||
for (;;) {
|
||||
const selection = selectServiceSpeed(
|
||||
baseModel,
|
||||
{ ...args, serviceSpeed: requestedSpeed },
|
||||
(id) => {
|
||||
const variant = this.#resolveModel(
|
||||
id,
|
||||
baseModel.provider!,
|
||||
);
|
||||
return variant &&
|
||||
variant.provider === baseModel.provider
|
||||
? variant
|
||||
: undefined;
|
||||
},
|
||||
offers,
|
||||
);
|
||||
const attemptModel = selection.model;
|
||||
const requestArgs = {
|
||||
...selection.args,
|
||||
if (!useFakeProvider) {
|
||||
await this.#resolvePuterPaths(attemptProvider, args, actor);
|
||||
}
|
||||
// Last point where a caller who left costs nothing.
|
||||
abortSignal?.throwIfAborted();
|
||||
return attemptProvider.complete(
|
||||
{
|
||||
...args,
|
||||
model: attemptModel.id,
|
||||
provider: attemptModel.provider,
|
||||
};
|
||||
const catalogModel = this.#resolveModel(
|
||||
attemptModel.id,
|
||||
attemptModel.provider!,
|
||||
);
|
||||
const rawModel = this.#catalogEntries.get(
|
||||
catalogModel ?? baseModel,
|
||||
);
|
||||
const snapshot = rawModel
|
||||
? {
|
||||
...rawModel,
|
||||
costs: attemptModel.costs,
|
||||
pricingOverrides: attemptModel.pricingOverrides,
|
||||
serviceSpeedPricing: attemptModel.serviceSpeedPricing,
|
||||
}
|
||||
: attemptModel;
|
||||
if (!useFakeProvider) {
|
||||
try {
|
||||
({
|
||||
hold,
|
||||
fundsCap,
|
||||
costFactor,
|
||||
promptTokenEstimate: attemptPromptTokenEstimate,
|
||||
} = await this.#applyCreditGate(
|
||||
actor,
|
||||
attemptModel,
|
||||
requestArgs,
|
||||
{
|
||||
promptTokenEstimate,
|
||||
requestedMaxTokens,
|
||||
pricingTime,
|
||||
},
|
||||
));
|
||||
} catch (error) {
|
||||
creditGateError = error;
|
||||
throw error;
|
||||
}
|
||||
await this.#resolvePuterPaths(
|
||||
attemptProvider,
|
||||
requestArgs,
|
||||
actor,
|
||||
);
|
||||
}
|
||||
abortSignal?.throwIfAborted();
|
||||
try {
|
||||
const result = await attemptProvider.complete(
|
||||
requestArgs,
|
||||
snapshot,
|
||||
pricingTime,
|
||||
);
|
||||
model = attemptModel;
|
||||
variantSpeed =
|
||||
attemptModel.id !== baseModel.id
|
||||
? selection.speed
|
||||
: undefined;
|
||||
if (
|
||||
args.serviceSpeed !== undefined &&
|
||||
'message' in result
|
||||
) {
|
||||
const messageResult = result as IChatMessageResult;
|
||||
const details =
|
||||
messageResult.usageDetails ??
|
||||
usageDetailsFromUsage(messageResult.usage, model);
|
||||
messageResult.usageDetails = {
|
||||
...details,
|
||||
serviceSpeed:
|
||||
variantSpeed ??
|
||||
details?.serviceSpeed ??
|
||||
details?.speed ??
|
||||
'standard',
|
||||
};
|
||||
}
|
||||
return result;
|
||||
} catch (error) {
|
||||
await hold.release();
|
||||
hold = NO_CREDIT_HOLD;
|
||||
if (
|
||||
args.serviceSpeed === undefined ||
|
||||
selection.speed === 'standard' ||
|
||||
!isServiceSpeedUnavailable(
|
||||
error,
|
||||
attemptModel.id !== baseModel.id,
|
||||
)
|
||||
)
|
||||
throw error;
|
||||
requestedSpeed = lowerServiceSpeed(selection.speed);
|
||||
pricingTime = new Date();
|
||||
}
|
||||
}
|
||||
},
|
||||
this.#catalogEntries.get(attemptModel),
|
||||
pricingTime,
|
||||
);
|
||||
};
|
||||
|
||||
try {
|
||||
@@ -1098,17 +984,29 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
// The previous attempt released its hold when it failed, so
|
||||
// this one starts from nothing held.
|
||||
pricingTime = new Date();
|
||||
if (!useFakeProvider) {
|
||||
({
|
||||
hold,
|
||||
fundsCap,
|
||||
costFactor,
|
||||
promptTokenEstimate: attemptPromptTokenEstimate,
|
||||
} = await this.#applyCreditGate(actor, fallback, args, {
|
||||
promptTokenEstimate,
|
||||
requestedMaxTokens,
|
||||
pricingTime,
|
||||
}));
|
||||
}
|
||||
|
||||
tried.add(routeId(fallback.provider!, fallback.id));
|
||||
|
||||
try {
|
||||
res = await runAttempt(fbProvider, fallback);
|
||||
model = fallback;
|
||||
lastError = null;
|
||||
console.warn(
|
||||
`[ai-chat] fallback served (${completionId}, ${first.provider}:${first.model} -> ${fallback.provider}:${fallback.id}, first ${first.status ?? 'none'}${first.upstreamType ? `/${first.upstreamType}` : ''})`,
|
||||
);
|
||||
} catch (fbErr) {
|
||||
if (fbErr === creditGateError) throw fbErr;
|
||||
await hold.release();
|
||||
hold = NO_CREDIT_HOLD;
|
||||
abortSignal?.throwIfAborted();
|
||||
@@ -1264,20 +1162,6 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
usageDetailsFromUsage(usage, model),
|
||||
);
|
||||
}
|
||||
if (
|
||||
args.serviceSpeed !== undefined &&
|
||||
chatStream.usageDetails
|
||||
) {
|
||||
const details = chatStream.usageDetails;
|
||||
chatStream.setUsageDetails({
|
||||
...details,
|
||||
serviceSpeed:
|
||||
variantSpeed ??
|
||||
details.serviceSpeed ??
|
||||
details.speed ??
|
||||
'standard',
|
||||
});
|
||||
}
|
||||
withClaudeThinkingTokens(
|
||||
enrichedUsage,
|
||||
model,
|
||||
@@ -1590,7 +1474,12 @@ export class ChatCompletionDriver extends PuterDriver {
|
||||
const metering = this.services.metering;
|
||||
const { requestedMaxTokens, pricingTime } = estimates;
|
||||
let { promptTokenEstimate } = estimates;
|
||||
// Providers reserve tool calls using the same caps they send upstream.
|
||||
// A provider can name its own cost keys for this request (Claude's
|
||||
// fast-mode rate card), cap its output, and size an extra reservation
|
||||
// the per-token math can't see (a web-search/advisor tool that could
|
||||
// run several times before the model answers) — the same clamps it
|
||||
// applies when building the request, so the hold and the request
|
||||
// agree.
|
||||
const pricing =
|
||||
this.#providers[model.provider!]?.requestPricing?.(args, model, {
|
||||
promptTokenEstimate,
|
||||
|
||||
@@ -41,7 +41,6 @@ import {
|
||||
meterChatUsage,
|
||||
type MeterChatUsageOptions,
|
||||
} from '../utils/meterChatUsage.js';
|
||||
import { servedServiceSpeed } from '../utils/serviceSpeed.js';
|
||||
import { catalogPrice, costKeys } from '../utils/pricing.js';
|
||||
import { normalizeModelKey } from '../utils/modelRouting.js';
|
||||
import * as OpenAIUtil from '../utils/OpenAIUtil.js';
|
||||
@@ -233,8 +232,6 @@ export class OpenAICompatProvider implements IChatProvider {
|
||||
)
|
||||
: {}),
|
||||
};
|
||||
if (args.serviceSpeed !== undefined && args.service_tier !== undefined)
|
||||
shared.service_tier = args.service_tier;
|
||||
const params = this.vendorParams(shared, args, model, actor);
|
||||
|
||||
let pricingTime = attemptPricingTime ?? new Date();
|
||||
@@ -274,29 +271,15 @@ export class OpenAICompatProvider implements IChatProvider {
|
||||
source,
|
||||
model,
|
||||
);
|
||||
const serviceSpeed = servedServiceSpeed(source);
|
||||
if (usageDetails)
|
||||
source.setUsageDetails?.({
|
||||
...usageDetails,
|
||||
...(args.serviceSpeed !== undefined
|
||||
? { serviceSpeed }
|
||||
: {}),
|
||||
});
|
||||
if (usageDetails) source.setUsageDetails?.(usageDetails);
|
||||
const metered = meterChatUsage(
|
||||
this.#metering,
|
||||
actor,
|
||||
this.meteringModelKey(model.id),
|
||||
model,
|
||||
usage,
|
||||
{ ...meterOptions, pricingTime, serviceSpeed },
|
||||
{ ...meterOptions, pricingTime },
|
||||
);
|
||||
if (args.serviceSpeed !== undefined && !usageDetails) {
|
||||
const details = OpenAIUtil.usageDetailsFromTrackedUsage(
|
||||
metered.usage,
|
||||
);
|
||||
if (details)
|
||||
source.setUsageDetails?.({ ...details, serviceSpeed });
|
||||
}
|
||||
source.setUsageCosts(metered.costs);
|
||||
return this.reportedUsage(metered.usage);
|
||||
},
|
||||
|
||||
@@ -174,9 +174,6 @@ export const mapAlibabaCatalogModel = (
|
||||
: undefined;
|
||||
return {
|
||||
...existing,
|
||||
serviceSpeedVariants: {
|
||||
fast: id === 'glm-5.2' ? 'glm-5.2-fast-preview' : `${id}-prime`,
|
||||
},
|
||||
id,
|
||||
puterId,
|
||||
name: raw.name ?? existing?.name ?? id,
|
||||
|
||||
@@ -37,7 +37,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-max-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 160,
|
||||
@@ -61,7 +60,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-plus-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -83,7 +81,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-plus-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -105,7 +102,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.6-plus-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 50,
|
||||
@@ -127,7 +123,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.6-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 19,
|
||||
@@ -148,7 +143,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.7-plus-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 50,
|
||||
@@ -170,7 +164,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.7-max-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
@@ -195,7 +188,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.8-max-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -218,7 +210,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.8-max-0902-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -243,7 +234,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 5,
|
||||
@@ -267,7 +257,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3-coder-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30,
|
||||
@@ -293,7 +282,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-397b-a17b-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 60,
|
||||
@@ -317,7 +305,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-122b-a10b-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -341,7 +328,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-35b-a3b-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 25,
|
||||
@@ -365,7 +351,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-27b-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30,
|
||||
@@ -391,7 +376,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.6-35b-a3b-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 24.8,
|
||||
@@ -415,7 +399,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.6-27b-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 60,
|
||||
@@ -439,7 +422,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3-vl-plus-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 20,
|
||||
@@ -465,7 +447,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3-omni-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 43,
|
||||
@@ -488,7 +469,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-mt-plus-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 246,
|
||||
@@ -511,7 +491,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-plus-2025-01-25-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -532,7 +511,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-plus-2025-04-28-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -553,7 +531,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-plus-2025-07-14-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -576,7 +553,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-plus-2025-12-01-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -598,7 +574,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.7-max-2026-05-20-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
@@ -619,7 +594,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.7-max-2026-06-08-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
@@ -640,7 +614,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.7-max-2026-05-17-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
@@ -662,7 +635,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.7-max-preview-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
@@ -685,7 +657,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-plus-2026-02-15-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -707,7 +678,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-plus-2026-04-20-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -730,7 +700,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 10,
|
||||
@@ -753,7 +722,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.7-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 3,
|
||||
@@ -775,7 +743,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.8-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 15,
|
||||
@@ -798,7 +765,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.8-27b-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 50,
|
||||
@@ -819,7 +785,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.8-2.4t-a95b-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -844,7 +809,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-omni-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -868,7 +832,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.5-omni-plus-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 140,
|
||||
@@ -892,7 +855,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen3.8-omni-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 15,
|
||||
@@ -915,7 +877,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-mt-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 16,
|
||||
@@ -936,7 +897,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'qwen-mt-lite-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 12,
|
||||
@@ -959,7 +919,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'glm-5.1-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 140,
|
||||
@@ -979,7 +938,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'glm-5.2-fast-preview' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 140,
|
||||
@@ -1000,7 +958,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'glm-5.2-fast-preview-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 280,
|
||||
@@ -1024,7 +981,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'deepseek-v4-flash-0731-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 44,
|
||||
@@ -1047,7 +1003,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'deepseek-v4.1-flash-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30,
|
||||
@@ -1068,7 +1023,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'deepseek-v4-pro-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 240,
|
||||
@@ -1091,7 +1045,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'deepseek-v4-pro-0813-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 132,
|
||||
@@ -1112,7 +1065,6 @@ export const ALIBABA_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'kimi-k2.7-code-prime' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 95,
|
||||
|
||||
@@ -20,10 +20,6 @@
|
||||
import { HttpError } from '@heyputer/backend/src/core/http/HttpError.js';
|
||||
import { OpenAI } from 'openai';
|
||||
import { ChatCompletionCreateParams } from 'openai/resources/index.js';
|
||||
import {
|
||||
serviceSpeedBillingModel,
|
||||
servedServiceSpeed,
|
||||
} from '../../utils/serviceSpeed.js';
|
||||
import { Context } from '../../../../core/context.js';
|
||||
import type { FSService } from '../../../../services/fs/FSService.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
@@ -157,7 +153,6 @@ export class AzureChatProvider implements IChatProvider {
|
||||
|
||||
async complete(
|
||||
params: ICompleteArguments,
|
||||
resolved?: IChatModel,
|
||||
): ReturnType<IChatProvider['complete']> {
|
||||
const {
|
||||
max_tokens,
|
||||
@@ -185,9 +180,7 @@ export class AzureChatProvider implements IChatProvider {
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
return await (resolved
|
||||
? this.#responsesProvider.complete(params, resolved)
|
||||
: this.#responsesProvider.complete(params));
|
||||
return await this.#responsesProvider.complete(params);
|
||||
}
|
||||
// Inline compaction is a Responses-API feature; chat.completions can't
|
||||
// express `context_management` or a `compaction` content block.
|
||||
@@ -201,9 +194,7 @@ export class AzureChatProvider implements IChatProvider {
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
return await (resolved
|
||||
? this.#responsesProvider.complete(params, resolved)
|
||||
: this.#responsesProvider.complete(params));
|
||||
return await this.#responsesProvider.complete(params);
|
||||
}
|
||||
// Validate messages
|
||||
if (!Array.isArray(messages)) {
|
||||
@@ -216,11 +207,9 @@ export class AzureChatProvider implements IChatProvider {
|
||||
model = model ?? this.#defaultModel;
|
||||
|
||||
const modelUsed =
|
||||
resolved ??
|
||||
(this.models().find((m) =>
|
||||
this.models().find((m) =>
|
||||
[m.id, ...(m.aliases || [])].includes(model),
|
||||
) ||
|
||||
this.models().find((m) => m.id === this.getDefaultModel())!);
|
||||
) || this.models().find((m) => m.id === this.getDefaultModel())!;
|
||||
|
||||
// messages.unshift({
|
||||
// role: 'system',
|
||||
@@ -290,9 +279,6 @@ export class AzureChatProvider implements IChatProvider {
|
||||
}),
|
||||
messages: messages,
|
||||
model: modelUsed.id,
|
||||
...(params.service_tier !== undefined
|
||||
? { service_tier: params.service_tier }
|
||||
: {}),
|
||||
...(mappedTools?.length ? { tools: mappedTools } : {}),
|
||||
...(max_tokens !== undefined
|
||||
? { max_completion_tokens: max_tokens }
|
||||
@@ -329,12 +315,7 @@ export class AzureChatProvider implements IChatProvider {
|
||||
);
|
||||
|
||||
return OpenAiUtil.handle_completion_output({
|
||||
usage_calculator: ({
|
||||
service_tier,
|
||||
usage,
|
||||
setUsageCosts,
|
||||
setUsageDetails,
|
||||
}) => {
|
||||
usage_calculator: ({ usage }) => {
|
||||
const cachedTokens =
|
||||
usage.prompt_tokens_details?.cached_tokens ?? 0;
|
||||
const trackedUsage = {
|
||||
@@ -354,20 +335,9 @@ export class AzureChatProvider implements IChatProvider {
|
||||
|
||||
const costsOverrideFromModel = buildCostsOverride(
|
||||
trackedUsage,
|
||||
serviceSpeedBillingModel(
|
||||
modelUsed,
|
||||
servedServiceSpeed({ service_tier }),
|
||||
),
|
||||
modelUsed,
|
||||
);
|
||||
|
||||
setUsageCosts(costsOverrideFromModel);
|
||||
if (params.serviceSpeed !== undefined)
|
||||
setUsageDetails({
|
||||
...OpenAiUtil.usageDetailsFromTrackedUsage(
|
||||
trackedUsage,
|
||||
)!,
|
||||
serviceSpeed: servedServiceSpeed({ service_tier }),
|
||||
});
|
||||
this.#meteringService.utilRecordUsageObject(
|
||||
trackedUsage,
|
||||
actor!,
|
||||
|
||||
@@ -449,22 +449,6 @@ describe('AzureResponsesProvider.complete request shape', () => {
|
||||
expect(args.service_tier).toBe('default');
|
||||
});
|
||||
|
||||
it('sends the wire spelling of the in-memory cache retention', async () => {
|
||||
const provider = makeProvider();
|
||||
responsesCreateMock.mockResolvedValueOnce(okResponse);
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-5.3-codex',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
prompt_cache_retention: 'in-memory',
|
||||
} as never),
|
||||
);
|
||||
|
||||
const [args] = responsesCreateMock.mock.calls[0]!;
|
||||
expect(args.prompt_cache_retention).toBe('in_memory');
|
||||
});
|
||||
|
||||
it.each([
|
||||
['previous_response_id', 'resp_1'],
|
||||
['conversation', 'conv_1'],
|
||||
|
||||
@@ -19,10 +19,6 @@
|
||||
|
||||
import { OpenAI } from 'openai';
|
||||
import { ResponseCreateParams } from 'openai/resources/responses/responses.mjs';
|
||||
import {
|
||||
serviceSpeedBillingModel,
|
||||
servedServiceSpeed,
|
||||
} from '../../utils/serviceSpeed.js';
|
||||
import { Context } from '../../../../core/context.js';
|
||||
import type { FSService } from '../../../../services/fs/FSService.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
@@ -39,7 +35,6 @@ import * as OpenAiUtil from '../../utils/OpenAIUtil.js';
|
||||
import {
|
||||
clampReasoningEffort,
|
||||
openAICompatParams,
|
||||
promptCacheRetentionToWire,
|
||||
rejectStatefulResponsesFields,
|
||||
} from '../../utils/openaiParams.js';
|
||||
import { buildCostsOverride } from '../../utils/pricing.js';
|
||||
@@ -156,41 +151,37 @@ export class AzureResponsesProvider implements IChatProvider {
|
||||
: {};
|
||||
}
|
||||
|
||||
async complete(
|
||||
{
|
||||
messages,
|
||||
model,
|
||||
max_tokens,
|
||||
moderation,
|
||||
tools,
|
||||
tool_choice,
|
||||
parallel_tool_calls,
|
||||
include,
|
||||
conversation,
|
||||
compaction,
|
||||
context_management,
|
||||
previous_response_id,
|
||||
instructions,
|
||||
metadata,
|
||||
prompt,
|
||||
prompt_cache_key,
|
||||
prompt_cache_retention,
|
||||
store,
|
||||
top_p,
|
||||
truncation,
|
||||
background,
|
||||
service_tier,
|
||||
serviceSpeed: requestedServiceSpeed,
|
||||
verbosity,
|
||||
stream,
|
||||
reasoning,
|
||||
reasoning_effort,
|
||||
temperature,
|
||||
text,
|
||||
outputFormat,
|
||||
}: ICompleteArguments,
|
||||
resolved?: IChatModel,
|
||||
): ReturnType<IChatProvider['complete']> {
|
||||
async complete({
|
||||
messages,
|
||||
model,
|
||||
max_tokens,
|
||||
moderation,
|
||||
tools,
|
||||
tool_choice,
|
||||
parallel_tool_calls,
|
||||
include,
|
||||
conversation,
|
||||
compaction,
|
||||
context_management,
|
||||
previous_response_id,
|
||||
instructions,
|
||||
metadata,
|
||||
prompt,
|
||||
prompt_cache_key,
|
||||
prompt_cache_retention,
|
||||
store,
|
||||
top_p,
|
||||
truncation,
|
||||
background,
|
||||
service_tier,
|
||||
verbosity,
|
||||
stream,
|
||||
reasoning,
|
||||
reasoning_effort,
|
||||
temperature,
|
||||
text,
|
||||
outputFormat,
|
||||
}: ICompleteArguments): ReturnType<IChatProvider['complete']> {
|
||||
// Validate messages
|
||||
if (!Array.isArray(messages)) {
|
||||
throw new HttpError(400, '`messages` must be an array', {
|
||||
@@ -208,20 +199,16 @@ export class AzureResponsesProvider implements IChatProvider {
|
||||
model = model ?? this.#defaultModel;
|
||||
|
||||
const modelUsed =
|
||||
resolved ??
|
||||
(this.models({ no_restrictions: true }).find((m) =>
|
||||
this.models({ no_restrictions: true }).find((m) =>
|
||||
[m.id, ...(m.aliases || [])].includes(model),
|
||||
) ||
|
||||
this.models({ no_restrictions: true }).find(
|
||||
(m) => m.id === this.getDefaultModel(),
|
||||
)!);
|
||||
this.models({ no_restrictions: true }).find(
|
||||
(m) => m.id === this.getDefaultModel(),
|
||||
)!;
|
||||
|
||||
const userIdentifier = upstreamUserIdentifier(actor);
|
||||
// Cache key defaults to the actor identifier; see upstreamUserIdentifier.
|
||||
const cacheKey = prompt_cache_key ?? userIdentifier;
|
||||
const cacheRetention = promptCacheRetentionToWire(
|
||||
prompt_cache_retention,
|
||||
);
|
||||
|
||||
// Resolve any `puter_path` content parts into inline base64 data URLs
|
||||
// before the Responses API sees them.
|
||||
@@ -320,8 +307,8 @@ export class AzureResponsesProvider implements IChatProvider {
|
||||
...(instructions !== undefined ? { instructions } : {}),
|
||||
...(metadata !== undefined ? { metadata } : {}),
|
||||
...(cacheKey !== undefined ? { prompt_cache_key: cacheKey } : {}),
|
||||
...(cacheRetention !== undefined
|
||||
? { prompt_cache_retention: cacheRetention }
|
||||
...(prompt_cache_retention !== undefined
|
||||
? { prompt_cache_retention }
|
||||
: {}),
|
||||
...(store !== undefined ? { store } : {}),
|
||||
...(max_tokens !== undefined
|
||||
@@ -344,11 +331,9 @@ export class AzureResponsesProvider implements IChatProvider {
|
||||
return OpenAiUtil.handle_completion_output_responses_api({
|
||||
usage_calculator: ({
|
||||
usage,
|
||||
service_tier,
|
||||
webSearchCalls,
|
||||
tool_usage,
|
||||
setUsageCosts,
|
||||
setUsageDetails,
|
||||
}) => {
|
||||
const numWebSearches: number =
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
@@ -374,10 +359,7 @@ export class AzureResponsesProvider implements IChatProvider {
|
||||
|
||||
const costsOverrideFromModel = buildCostsOverride(
|
||||
trackedUsage,
|
||||
serviceSpeedBillingModel(
|
||||
modelUsed,
|
||||
servedServiceSpeed({ service_tier }),
|
||||
),
|
||||
modelUsed,
|
||||
);
|
||||
if (numWebSearches) {
|
||||
costsOverrideFromModel.web_search_calls =
|
||||
@@ -388,14 +370,6 @@ export class AzureResponsesProvider implements IChatProvider {
|
||||
});
|
||||
}
|
||||
|
||||
setUsageCosts?.(costsOverrideFromModel);
|
||||
if (requestedServiceSpeed !== undefined)
|
||||
setUsageDetails?.({
|
||||
...OpenAiUtil.usageDetailsFromTrackedUsage(
|
||||
trackedUsage,
|
||||
)!,
|
||||
serviceSpeed: servedServiceSpeed({ service_tier }),
|
||||
});
|
||||
this.#meteringService.utilRecordUsageObject(
|
||||
trackedUsage,
|
||||
actor,
|
||||
|
||||
@@ -40,11 +40,7 @@ const STANDALONE = [
|
||||
];
|
||||
|
||||
// Fields an Azure deployment genuinely differs from its source on, by puterId.
|
||||
const OVERRIDES: Record<string, string[]> = {
|
||||
'azure:x-ai/grok-4.3': ['serviceSpeedRates'],
|
||||
'azure:x-ai/grok-4-20-non-reasoning': ['serviceSpeedRates'],
|
||||
'azure:x-ai/grok-4-20-reasoning': ['serviceSpeedRates'],
|
||||
};
|
||||
const OVERRIDES: Record<string, string[]> = {};
|
||||
|
||||
const IDENTITY = ['id', 'puterId', 'aliases'];
|
||||
|
||||
@@ -69,18 +65,6 @@ const mirrored = AZURE_MODELS.filter(
|
||||
).map((m) => [m.puterId!, m] as const);
|
||||
|
||||
describe('AZURE_MODELS mirroring', () => {
|
||||
it('inherits OpenAI fast rates without enabling direct-only speed tiers', () => {
|
||||
for (const model of AZURE_MODELS) {
|
||||
expect(model.serviceSpeedRates?.ultrafast).toBeUndefined();
|
||||
if (model.puterId?.startsWith('azure:x-ai/')) {
|
||||
expect(model.serviceSpeedRates?.fast).toBeUndefined();
|
||||
}
|
||||
}
|
||||
const azure = AZURE_MODELS.find((model) => model.id === 'gpt-5.4')!;
|
||||
expect(azure.serviceSpeedRates).toEqual({
|
||||
fast: sourceOf(azure)!.serviceSpeedRates!.fast,
|
||||
});
|
||||
});
|
||||
it('has a direct-provider source for every entry not listed as standalone', () => {
|
||||
const orphans = mirrored
|
||||
.filter(([, m]) => !sourceOf(m))
|
||||
|
||||
@@ -22,8 +22,10 @@ import { OPEN_AI_MODELS } from '../openai/models.js';
|
||||
import { XAI_MODELS } from '../xai/models.js';
|
||||
|
||||
/**
|
||||
* Azure shares its source model's list prices, context tiers, and limits.
|
||||
* Identity and available service speeds belong to the deployment.
|
||||
* An Azure deployment of a model we also serve direct. Every field but the
|
||||
* identity comes from the source entry, so prices, long-context tiers and
|
||||
* limits can't drift from it. `name` is the vendor-qualified name both catalogs
|
||||
* share (`openai/gpt-5.4`); extra fields override the source.
|
||||
*/
|
||||
const mirror = (
|
||||
source: readonly IChatModel[],
|
||||
@@ -39,11 +41,6 @@ const mirror = (
|
||||
puterId: `azure:${name}`,
|
||||
id: azure.id,
|
||||
...shared,
|
||||
serviceSpeedRates:
|
||||
source === OPEN_AI_MODELS &&
|
||||
shared.serviceSpeedRates?.fast !== undefined
|
||||
? { fast: shared.serviceSpeedRates.fast }
|
||||
: undefined,
|
||||
aliases: [name],
|
||||
...azure,
|
||||
};
|
||||
@@ -130,13 +127,6 @@ export const AZURE_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 250,
|
||||
cached_tokens: 25,
|
||||
completion_tokens: 2000,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
@@ -181,13 +171,6 @@ export const AZURE_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 45,
|
||||
cached_tokens: 4.5,
|
||||
completion_tokens: 360,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 25,
|
||||
|
||||
@@ -147,22 +147,6 @@ describe.skipIf(skipUnlessEnv(ENV_VAR))('ClaudeProvider (integration)', () => {
|
||||
},
|
||||
);
|
||||
|
||||
it('returns a completion from claude-fable-5-2', { timeout: INTEGRATION_TEST_TIMEOUT_MS }, async () => {
|
||||
const provider = buildProvider();
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'claude-fable-5-2',
|
||||
messages: [{ role: 'user', content: 'Reply with OK.' }],
|
||||
reasoning_effort: 'low',
|
||||
max_tokens: 128,
|
||||
}),
|
||||
);
|
||||
const content = (result as {
|
||||
message: { content: Array<{ type: string; text?: string }> };
|
||||
}).message.content;
|
||||
expect(content.find((block) => block.type === 'text')?.text?.length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
// (a) Claude Code-shaped system blocks, attribution header first — the
|
||||
// pipeline must never merge or reorder system blocks: the header is
|
||||
// only recognized by Anthropic when it is the sole content of an
|
||||
|
||||
@@ -1173,7 +1173,6 @@ describe('ClaudeProvider.complete request shape', () => {
|
||||
});
|
||||
|
||||
it.each([
|
||||
'claude-fable-5-2',
|
||||
'claude-fable-5-1',
|
||||
'claude-haiku-5-5',
|
||||
'claude-opus-5-5',
|
||||
@@ -1199,7 +1198,6 @@ describe('ClaudeProvider.complete request shape', () => {
|
||||
);
|
||||
|
||||
it.each([
|
||||
'claude-fable-5-2',
|
||||
'claude-fable-5-1',
|
||||
'claude-haiku-5-5',
|
||||
'claude-opus-5-5',
|
||||
@@ -1305,73 +1303,6 @@ describe('ClaudeProvider model resolution', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it.each([
|
||||
'claude-fable-5-2',
|
||||
'claude-fable-5-2-latest',
|
||||
'claude-fable-5.2',
|
||||
'anthropic/claude-fable-5-2',
|
||||
])(
|
||||
'routes %s to Fable 5.2 with published rates',
|
||||
async (model) => {
|
||||
const { provider } = makeProvider();
|
||||
messagesCreateMock.mockResolvedValueOnce({
|
||||
content: [{ type: 'text', text: 'ok' }],
|
||||
usage: {
|
||||
input_tokens: 100,
|
||||
output_tokens: 50,
|
||||
cache_read_input_tokens: 1000,
|
||||
cache_creation_input_tokens: 200,
|
||||
cache_creation: {
|
||||
ephemeral_5m_input_tokens: 150,
|
||||
ephemeral_1h_input_tokens: 50,
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model,
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
}),
|
||||
);
|
||||
|
||||
expect(messagesCreateMock.mock.calls[0]![0].model).toBe(
|
||||
'claude-fable-5-2',
|
||||
);
|
||||
const fable52 = CLAUDE_MODELS.find(
|
||||
(m) => m.id === 'claude-fable-5-2',
|
||||
)!;
|
||||
expect(fable52.costs).toMatchObject({
|
||||
input_tokens: 800,
|
||||
output_tokens: 4000,
|
||||
cache_read_input_tokens: 20,
|
||||
ephemeral_5m_input_tokens: 1000,
|
||||
ephemeral_1h_input_tokens: 1600,
|
||||
});
|
||||
expect(fable52.serviceSpeedRates?.fast).toMatchObject({
|
||||
input_tokens: 1600,
|
||||
output_tokens: 8000,
|
||||
cache_read_input_tokens: 40,
|
||||
ephemeral_5m_input_tokens: 2000,
|
||||
ephemeral_1h_input_tokens: 3200,
|
||||
});
|
||||
expect(fable52.context).toBe(1_000_000);
|
||||
expect(fable52.max_tokens).toBe(128_000);
|
||||
const [, , prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(prefix).toBe('claude:claude-fable-5-2');
|
||||
expect(overrides).toMatchObject({
|
||||
input_tokens: 100 * Number(fable52.costs.input_tokens),
|
||||
output_tokens: 50 * Number(fable52.costs.output_tokens),
|
||||
cache_read_input_tokens:
|
||||
1000 * Number(fable52.costs.cache_read_input_tokens),
|
||||
ephemeral_5m_input_tokens:
|
||||
150 * Number(fable52.costs.ephemeral_5m_input_tokens),
|
||||
ephemeral_1h_input_tokens:
|
||||
50 * Number(fable52.costs.ephemeral_1h_input_tokens),
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it('routes the bare claude-fable alias to fable 5.1 rather than fable 5', async () => {
|
||||
const { provider } = makeProvider();
|
||||
messagesCreateMock.mockResolvedValueOnce(baseResponse);
|
||||
@@ -2380,20 +2311,16 @@ describe('ClaudeProvider metering', () => {
|
||||
expect('thinking_tokens' in overrides).toBe(false);
|
||||
});
|
||||
|
||||
it.each(['claude-opus-5-5', 'claude-fable-5-2'])('bills at fast rates for %s when the upstream reports fast usage', async (model) => {
|
||||
it('bills at the fast_* rates when the upstream reports usage.speed:"fast"', async () => {
|
||||
const { provider } = makeProvider();
|
||||
messagesCreateMock.mockResolvedValueOnce({
|
||||
content: [{ type: 'text', text: 'hi' }],
|
||||
usage: {
|
||||
input_tokens: 10, output_tokens: 5, speed: 'fast',
|
||||
cache_creation: { ephemeral_5m_input_tokens: 7, ephemeral_1h_input_tokens: 3 },
|
||||
cache_read_input_tokens: 20,
|
||||
},
|
||||
usage: { input_tokens: 10, output_tokens: 5, speed: 'fast' },
|
||||
});
|
||||
|
||||
const result = (await withTestActor(() =>
|
||||
provider.complete({
|
||||
model,
|
||||
model: 'claude-opus-5-5',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
speed: 'fast',
|
||||
}),
|
||||
@@ -2404,15 +2331,11 @@ describe('ClaudeProvider metering', () => {
|
||||
expect('input_tokens' in result.usage).toBe(false);
|
||||
expect(result.usageDetails.speed).toBe('fast');
|
||||
|
||||
const selectedModel = CLAUDE_MODELS.find((m) => m.id === model)!;
|
||||
const opus = CLAUDE_MODELS.find((m) => m.id === 'claude-opus-5-5')!;
|
||||
const [, , prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(prefix).toBe(`claude:${model}`);
|
||||
const fastRates = selectedModel.serviceSpeedRates!.fast as Record<string, number>;
|
||||
expect(overrides.fast_input_tokens).toBe(10 * fastRates.input_tokens);
|
||||
expect(overrides.fast_output_tokens).toBe(5 * fastRates.output_tokens);
|
||||
expect(overrides.fast_ephemeral_5m_input_tokens).toBe(7 * fastRates.ephemeral_5m_input_tokens);
|
||||
expect(overrides.fast_ephemeral_1h_input_tokens).toBe(3 * fastRates.ephemeral_1h_input_tokens);
|
||||
expect(overrides.fast_cache_read_input_tokens).toBe(20 * fastRates.cache_read_input_tokens);
|
||||
expect(prefix).toBe('claude:claude-opus-5-5');
|
||||
expect(overrides.fast_input_tokens).toBe(10 * Number(opus.costs.fast_input_tokens));
|
||||
expect(overrides.fast_output_tokens).toBe(5 * Number(opus.costs.fast_output_tokens));
|
||||
});
|
||||
|
||||
it('bills at standard rates when fast mode was requested but not granted by the upstream', async () => {
|
||||
@@ -2434,15 +2357,13 @@ describe('ClaudeProvider metering', () => {
|
||||
expect('fast_input_tokens' in result.usage).toBe(false);
|
||||
});
|
||||
|
||||
it.each([
|
||||
[429, "This request would exceed your organization's rate limit of 0 fast mode input tokens per minute"],
|
||||
[529, 'Overloaded'],
|
||||
[529, 'Fast mode capacity unavailable'],
|
||||
])('retries at standard speed on Claude after fast capacity failure %s', async (status, message) => {
|
||||
it('retries at standard speed on Claude when fast mode is rate limited', async () => {
|
||||
const { provider } = makeProvider();
|
||||
const fastLimited = Object.assign(
|
||||
new Error(message),
|
||||
{ status },
|
||||
new Error(
|
||||
"This request would exceed your organization's rate limit of 0 fast mode input tokens per minute",
|
||||
),
|
||||
{ status: 429 },
|
||||
);
|
||||
messagesCreateMock.mockRejectedValueOnce(fastLimited).mockResolvedValueOnce({
|
||||
content: [{ type: 'text', text: 'hi' }],
|
||||
@@ -2666,13 +2587,13 @@ describe('ClaudeProvider.requestPricing', () => {
|
||||
const haiku = CLAUDE_MODELS.find((m) => m.id === 'claude-haiku-4-5-20251001')!;
|
||||
const opus55 = CLAUDE_MODELS.find((m) => m.id === 'claude-opus-5-5')!;
|
||||
|
||||
it('leaves token reservation keys to the selected catalog rate card', () => {
|
||||
it('returns fast cost keys only when the model has fast rates and fast was requested', () => {
|
||||
const { provider } = makeProvider();
|
||||
expect(
|
||||
provider.requestPricing({ speed: 'fast', tools: [] } as never, opus55, {
|
||||
promptTokenEstimate: 10,
|
||||
}),
|
||||
).toEqual({});
|
||||
).toMatchObject({ inputKey: 'fast_input_tokens', outputKey: 'fast_output_tokens' });
|
||||
expect(
|
||||
provider.requestPricing({ speed: 'fast', tools: [] } as never, haiku, {
|
||||
promptTokenEstimate: 10,
|
||||
|
||||
@@ -43,7 +43,6 @@ import {
|
||||
} from '../../utils/mediaParts.js';
|
||||
import { isToolChoice, toolChoiceFromWire } from '../../utils/openaiParams.js';
|
||||
import { buildCostsOverride } from '../../utils/pricing.js';
|
||||
import { serviceSpeedCosts } from '../../utils/serviceSpeed.js';
|
||||
import type {
|
||||
AIChatStream,
|
||||
AIChatTextStream,
|
||||
@@ -108,21 +107,20 @@ const toAnthropicMediaPart = (part: any): any => {
|
||||
return part;
|
||||
};
|
||||
|
||||
/** Fast quota or overload failures can be retried at standard speed. */
|
||||
const isFastModeUnavailable = (e: unknown): boolean => {
|
||||
/** A 429 from fast mode's separate quota, not the model's normal limits. */
|
||||
const isFastModeRateLimit = (e: unknown): boolean => {
|
||||
const err = e as { status?: unknown; message?: unknown };
|
||||
return (
|
||||
typeof err?.message === 'string' &&
|
||||
((err?.status === 429 && /fast mode/i.test(err.message)) ||
|
||||
(err?.status === 529 && /overload|fast mode/i.test(err.message)))
|
||||
err?.status === 429 &&
|
||||
typeof err.message === 'string' &&
|
||||
/fast mode/i.test(err.message)
|
||||
);
|
||||
};
|
||||
|
||||
// Models whose current turn rejects a non-default temperature/top_p/top_k
|
||||
// outright. Fable 5/5.1/5.2, Sonnet 5/5.5, Haiku 5.5, and Opus 4.7+.
|
||||
// outright. Fable 5/5.1, Sonnet 5/5.5, Haiku 5.5, and Opus 4.7+.
|
||||
const OMITS_SAMPLING_PARAMS = new Set([
|
||||
'claude-haiku-5-5',
|
||||
'claude-fable-5-2',
|
||||
'claude-fable-5-1',
|
||||
'claude-fable-5',
|
||||
'claude-sonnet-5-5',
|
||||
@@ -135,7 +133,6 @@ const OMITS_SAMPLING_PARAMS = new Set([
|
||||
|
||||
const SUPPORTS_EFFORT = new Set([
|
||||
'claude-haiku-5-5',
|
||||
'claude-fable-5-2',
|
||||
'claude-fable-5-1',
|
||||
'claude-fable-5',
|
||||
'claude-sonnet-5-5',
|
||||
@@ -301,12 +298,20 @@ export class ClaudeProvider implements IChatProvider {
|
||||
return `claude:${modelId}`;
|
||||
}
|
||||
|
||||
/** Reserves tools that can run several times before the model answers. */
|
||||
/**
|
||||
* Sizes the credit hold for Claude-only extras the generic per-token math
|
||||
* can't see: fast mode's own rate card, and a web-search/advisor tool that
|
||||
* could run several times before the model answers.
|
||||
*/
|
||||
requestPricing(
|
||||
args: ICompleteArguments,
|
||||
model: IChatModel,
|
||||
est: { promptTokenEstimate: number },
|
||||
): { extraCost?: number } {
|
||||
): { inputKey?: string; outputKey?: string; extraCost?: number } {
|
||||
const fast =
|
||||
args.speed === 'fast' &&
|
||||
model.costs?.fast_output_tokens !== undefined;
|
||||
|
||||
let extraCost = 0;
|
||||
for (const raw of args.tools ?? []) {
|
||||
const tool = raw as Record<string, unknown>;
|
||||
@@ -345,6 +350,12 @@ export class ClaudeProvider implements IChatProvider {
|
||||
}
|
||||
|
||||
return {
|
||||
...(fast
|
||||
? {
|
||||
inputKey: 'fast_input_tokens',
|
||||
outputKey: 'fast_output_tokens',
|
||||
}
|
||||
: {}),
|
||||
...(extraCost > 0 ? { extraCost } : {}),
|
||||
};
|
||||
}
|
||||
@@ -353,9 +364,11 @@ export class ClaudeProvider implements IChatProvider {
|
||||
try {
|
||||
return await this.#completeOnce(args);
|
||||
} catch (e) {
|
||||
// Retry fast-capacity failures on Claude at standard speed before
|
||||
// considering a different provider.
|
||||
if (args.speed === 'fast' && isFastModeUnavailable(e)) {
|
||||
// Fast mode has its own quota. Anthropic's advice on that 429 is to
|
||||
// drop `speed` and run at standard speed; doing it here keeps the
|
||||
// request on Claude instead of marking the route unhealthy and
|
||||
// replaying it on a reseller.
|
||||
if (args.speed === 'fast' && isFastModeRateLimit(e)) {
|
||||
return await this.#completeOnce({ ...args, speed: undefined });
|
||||
}
|
||||
throw e;
|
||||
@@ -624,7 +637,7 @@ export class ClaudeProvider implements IChatProvider {
|
||||
: undefined;
|
||||
const fast =
|
||||
args.speed === 'fast' &&
|
||||
modelUsed.serviceSpeedRates?.fast !== undefined;
|
||||
modelUsed.costs?.fast_output_tokens !== undefined;
|
||||
const safeguardsPolicy = applySafeguardsPolicy(
|
||||
args.safeguards,
|
||||
args.anthropicBetas,
|
||||
@@ -1058,7 +1071,6 @@ export class ClaudeProvider implements IChatProvider {
|
||||
// keeps reasoning visible in the stream.
|
||||
if (
|
||||
modelId === 'claude-haiku-5-5' ||
|
||||
modelId === 'claude-fable-5-2' ||
|
||||
modelId === 'claude-fable-5-1' ||
|
||||
modelId === 'claude-fable-5' ||
|
||||
modelId === 'claude-sonnet-5-5' ||
|
||||
@@ -1493,8 +1505,7 @@ export class ClaudeProvider implements IChatProvider {
|
||||
|
||||
const fast =
|
||||
raw.speed === 'fast' &&
|
||||
modelUsed.serviceSpeedRates?.fast !== undefined;
|
||||
// Keep published usage keys while resolving rates from the common catalog shape.
|
||||
modelUsed.costs?.fast_output_tokens !== undefined;
|
||||
const executorUsage: Record<string, number> = fast
|
||||
? Object.fromEntries(
|
||||
Object.entries(exec).map(([k, v]) => [`fast_${k}`, v]),
|
||||
@@ -1506,24 +1517,7 @@ export class ClaudeProvider implements IChatProvider {
|
||||
const fetches = raw.server_tool_use?.web_fetch_requests ?? 0;
|
||||
if (fetches) executorUsage.web_fetch_requests = fetches;
|
||||
|
||||
const fastRates = fast
|
||||
? serviceSpeedCosts(modelUsed, 'fast')
|
||||
: undefined;
|
||||
const billingModel = fastRates
|
||||
? {
|
||||
...modelUsed,
|
||||
costs: {
|
||||
...modelUsed.costs,
|
||||
...Object.fromEntries(
|
||||
Object.keys(exec).map((key) => [
|
||||
`fast_${key}`,
|
||||
fastRates[key],
|
||||
]),
|
||||
),
|
||||
},
|
||||
}
|
||||
: modelUsed;
|
||||
const executorCosts = buildCostsOverride(executorUsage, billingModel);
|
||||
const executorCosts = buildCostsOverride(executorUsage, modelUsed);
|
||||
|
||||
let advisorUsage: Record<string, number> | undefined;
|
||||
let advisorCosts: Record<string, number> | undefined;
|
||||
|
||||
@@ -55,46 +55,6 @@ export const CLAUDE_MODELS: IChatModel[] = [
|
||||
context: 1_000_000,
|
||||
max_tokens: 128_000,
|
||||
},
|
||||
{
|
||||
puterId: 'anthropic:anthropic/claude-fable-5-2',
|
||||
id: 'claude-fable-5-2',
|
||||
modalities: { input: ['text', 'image', 'pdf'], output: ['text'] },
|
||||
open_weights: false,
|
||||
tool_call: true,
|
||||
release_date: '2026-11-09',
|
||||
aliases: [
|
||||
'claude-fable-5-2-latest',
|
||||
'claude-fable-5.2',
|
||||
'anthropic/claude-fable-5-2',
|
||||
],
|
||||
name: 'Claude Fable 5.2',
|
||||
midConversationSystem: true,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'input_tokens',
|
||||
output_cost_key: 'output_tokens',
|
||||
// Source: https://www.anthropic.com/claude-fable-and-mythos-5-2
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
input_tokens: 1600,
|
||||
output_tokens: 8000,
|
||||
ephemeral_5m_input_tokens: 1600 * 1.25,
|
||||
ephemeral_1h_input_tokens: 1600 * 2,
|
||||
cache_read_input_tokens: 1600 * 0.025,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
input_tokens: 800,
|
||||
ephemeral_5m_input_tokens: 800 * 1.25,
|
||||
ephemeral_1h_input_tokens: 800 * 2,
|
||||
// 0.025x input, not the usual 0.1x.
|
||||
cache_read_input_tokens: 800 * 0.025,
|
||||
output_tokens: 4000,
|
||||
web_search_requests: 1_000_000, // $10/1k requests
|
||||
},
|
||||
context: 1000000,
|
||||
max_tokens: 128000,
|
||||
},
|
||||
{
|
||||
puterId: 'anthropic:anthropic/claude-fable-5-1',
|
||||
id: 'claude-fable-5-1',
|
||||
@@ -230,15 +190,6 @@ export const CLAUDE_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'input_tokens',
|
||||
output_cost_key: 'output_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
input_tokens: 800,
|
||||
output_tokens: 4000,
|
||||
ephemeral_5m_input_tokens: 1000,
|
||||
ephemeral_1h_input_tokens: 1600,
|
||||
cache_read_input_tokens: 40,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
input_tokens: 400,
|
||||
@@ -248,6 +199,12 @@ export const CLAUDE_MODELS: IChatModel[] = [
|
||||
cache_read_input_tokens: 400 * 0.05,
|
||||
output_tokens: 2000,
|
||||
web_search_requests: 1_000_000,
|
||||
// Fast mode (`speed:'fast'`), live-verified.
|
||||
fast_input_tokens: 800,
|
||||
fast_output_tokens: 4000,
|
||||
fast_ephemeral_5m_input_tokens: 1000,
|
||||
fast_ephemeral_1h_input_tokens: 1600,
|
||||
fast_cache_read_input_tokens: 40, // 0.05x fast_input_tokens
|
||||
},
|
||||
context: 1000000,
|
||||
max_tokens: 128000,
|
||||
@@ -266,15 +223,6 @@ export const CLAUDE_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'input_tokens',
|
||||
output_cost_key: 'output_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
input_tokens: 1000,
|
||||
output_tokens: 5000,
|
||||
ephemeral_5m_input_tokens: 1250,
|
||||
ephemeral_1h_input_tokens: 2000,
|
||||
cache_read_input_tokens: 100,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
input_tokens: 500,
|
||||
@@ -283,6 +231,11 @@ export const CLAUDE_MODELS: IChatModel[] = [
|
||||
cache_read_input_tokens: 500 * 0.1,
|
||||
output_tokens: 2500,
|
||||
web_search_requests: 1_000_000,
|
||||
fast_input_tokens: 1000,
|
||||
fast_output_tokens: 5000,
|
||||
fast_ephemeral_5m_input_tokens: 1250,
|
||||
fast_ephemeral_1h_input_tokens: 2000,
|
||||
fast_cache_read_input_tokens: 100,
|
||||
},
|
||||
context: 1000000,
|
||||
max_tokens: 128000,
|
||||
@@ -305,15 +258,6 @@ export const CLAUDE_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'input_tokens',
|
||||
output_cost_key: 'output_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
input_tokens: 1000,
|
||||
output_tokens: 5000,
|
||||
ephemeral_5m_input_tokens: 1250,
|
||||
ephemeral_1h_input_tokens: 2000,
|
||||
cache_read_input_tokens: 100,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
input_tokens: 500,
|
||||
@@ -322,6 +266,11 @@ export const CLAUDE_MODELS: IChatModel[] = [
|
||||
cache_read_input_tokens: 500 * 0.1,
|
||||
output_tokens: 2500,
|
||||
web_search_requests: 1_000_000,
|
||||
fast_input_tokens: 1000,
|
||||
fast_output_tokens: 5000,
|
||||
fast_ephemeral_5m_input_tokens: 1250,
|
||||
fast_ephemeral_1h_input_tokens: 2000,
|
||||
fast_cache_read_input_tokens: 100,
|
||||
},
|
||||
context: 1000000,
|
||||
max_tokens: 128000,
|
||||
|
||||
@@ -84,14 +84,8 @@ export class GeminiChatProvider extends OpenAICompatProvider {
|
||||
chat: {
|
||||
completions: {
|
||||
create: (params, options) => {
|
||||
const { model, service_tier } = params as {
|
||||
model: string;
|
||||
service_tier?: string;
|
||||
};
|
||||
if (
|
||||
!GEMINI_NATIVE_MODELS.has(model) &&
|
||||
(service_tier === undefined || !native)
|
||||
) {
|
||||
const { model } = params as { model: string };
|
||||
if (!GEMINI_NATIVE_MODELS.has(model)) {
|
||||
return compat.chat.completions.create(
|
||||
params,
|
||||
options,
|
||||
@@ -109,12 +103,7 @@ export class GeminiChatProvider extends OpenAICompatProvider {
|
||||
},
|
||||
},
|
||||
};
|
||||
const models: IChatModel[] = native
|
||||
? GEMINI_MODELS
|
||||
: COMPAT_ONLY_MODELS.map((model) => ({
|
||||
...model,
|
||||
serviceSpeeds: ['standard'],
|
||||
}));
|
||||
const models = native ? GEMINI_MODELS : COMPAT_ONLY_MODELS;
|
||||
super(meteringService, {
|
||||
client,
|
||||
defaultModel: 'gemini-2.5-flash',
|
||||
@@ -194,10 +183,7 @@ export class GeminiChatProvider extends OpenAICompatProvider {
|
||||
}
|
||||
|
||||
protected override meteredUsage(source: UsageSource, model: IChatModel) {
|
||||
if (
|
||||
GEMINI_NATIVE_MODELS.has(model.id) ||
|
||||
'promptTokenCount' in (source.usage ?? {})
|
||||
) {
|
||||
if (GEMINI_NATIVE_MODELS.has(model.id)) {
|
||||
return nativeMeteredUsage(
|
||||
source.usage as unknown as
|
||||
GenerateContentResponseUsageMetadata | undefined,
|
||||
|
||||
@@ -80,7 +80,6 @@ interface ChatParams {
|
||||
temperature?: number;
|
||||
reasoning_effort?: 'minimal' | 'low' | 'medium' | 'high';
|
||||
stream?: boolean;
|
||||
service_tier?: string;
|
||||
}
|
||||
|
||||
/** Ids made up for calls Gemini returned without one; never sent back. */
|
||||
@@ -241,12 +240,6 @@ const toGenerateContentRequest = (
|
||||
? toolConfigOf(params.tool_choice)
|
||||
: undefined;
|
||||
const config: GenerateContentConfig = {
|
||||
...(params.service_tier !== undefined
|
||||
? {
|
||||
serviceTier:
|
||||
params.service_tier as GenerateContentConfig['serviceTier'],
|
||||
}
|
||||
: {}),
|
||||
...(signal ? { abortSignal: signal } : {}),
|
||||
...(systemInstruction ? { systemInstruction } : {}),
|
||||
...(params.max_completion_tokens !== undefined
|
||||
|
||||
@@ -142,7 +142,6 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: ARGON_LAUNCH_COSTS,
|
||||
pricingOverrides: listPricingFrom(
|
||||
'2026-12-01T00:00:00Z',
|
||||
@@ -175,7 +174,6 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
// Google serves this endpoint with gemini-3.6-flash and bills its rates.
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: FLASH_LAUNCH_COSTS,
|
||||
pricingOverrides: FLASH_LIST_PRICING,
|
||||
},
|
||||
@@ -203,7 +201,6 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30,
|
||||
@@ -231,7 +228,6 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: FLASH_LAUNCH_COSTS,
|
||||
pricingOverrides: FLASH_LIST_PRICING,
|
||||
},
|
||||
@@ -254,7 +250,6 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
// Google serves this endpoint with gemini-3.8-flash and bills its rates.
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: FLASH_LAUNCH_COSTS,
|
||||
pricingOverrides: FLASH_LIST_PRICING,
|
||||
},
|
||||
@@ -275,7 +270,6 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: FLASH_LAUNCH_COSTS,
|
||||
pricingOverrides: FLASH_LIST_PRICING,
|
||||
},
|
||||
@@ -297,7 +291,6 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 25,
|
||||
@@ -325,7 +318,6 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 50,
|
||||
@@ -352,7 +344,6 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30,
|
||||
@@ -383,7 +374,6 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 10,
|
||||
@@ -412,7 +402,6 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
@@ -459,7 +448,6 @@ export const GEMINI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.8 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
|
||||
@@ -266,15 +266,6 @@ export class InfronProvider extends OpenAICompatProvider {
|
||||
return coerced_models;
|
||||
}
|
||||
|
||||
async serviceSpeedModels(model: IChatModel) {
|
||||
const variant = (await this.models()).find(
|
||||
(entry) => entry.id === `${model.id}:priority`,
|
||||
);
|
||||
return variant
|
||||
? { fast: { ...variant, provider: model.provider } }
|
||||
: {};
|
||||
}
|
||||
|
||||
protected override vendorParams(
|
||||
params: Record<string, unknown>,
|
||||
_args: unknown,
|
||||
|
||||
@@ -23,7 +23,6 @@ import { HttpError } from '../../../../core/http/HttpError.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
import { upstreamUserIdentifier } from '../../../util/upstreamIdentifier.js';
|
||||
import type { IChatModel, ICompleteArguments } from '../../types.js';
|
||||
import { promptCacheRetentionToWire } from '../../utils/openaiParams.js';
|
||||
import { sdkClientOptions } from '../../utils/sdkClient.js';
|
||||
import {
|
||||
type ChatProviderConfig,
|
||||
@@ -108,9 +107,12 @@ export class MetaProvider extends OpenAICompatProvider {
|
||||
? requestedEffort
|
||||
: undefined;
|
||||
|
||||
const cacheRetention = promptCacheRetentionToWire(
|
||||
args.prompt_cache_retention,
|
||||
);
|
||||
// Puter spells the in-memory retention with a hyphen; Meta's enum
|
||||
// uses an underscore.
|
||||
const cacheRetention =
|
||||
args.prompt_cache_retention === 'in-memory'
|
||||
? 'in_memory'
|
||||
: args.prompt_cache_retention;
|
||||
|
||||
// The identifier is Puter's abuse attribution, so `custom` can't
|
||||
// override it. Cache key defaults to it; see upstreamUserIdentifier.
|
||||
|
||||
@@ -43,8 +43,6 @@ export const MINIMAX_MODELS: MiniMaxChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.5 },
|
||||
serviceSpeedVariants: { fast: 'minimax-m3-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30, // $0.30 per 1M (input <= 512k tokens)
|
||||
@@ -72,7 +70,6 @@ export const MINIMAX_MODELS: MiniMaxChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'minimax-m2.7-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30, // $0.30 per 1M
|
||||
@@ -125,7 +122,6 @@ export const MINIMAX_MODELS: MiniMaxChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'minimax-m2.5-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30, // $0.30 per 1M
|
||||
@@ -178,7 +174,6 @@ export const MINIMAX_MODELS: MiniMaxChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'minimax-m2.1-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30, // $0.30 per 1M
|
||||
@@ -227,7 +222,6 @@ export const MINIMAX_MODELS: MiniMaxChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'minimax-m2-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30, // $0.30 per 1M
|
||||
|
||||
@@ -20,12 +20,13 @@
|
||||
/**
|
||||
* Offline unit tests for MistralAIProvider.
|
||||
*
|
||||
* Boots a real PuterServer (in-memory sqlite + dynamo + s3 + mock redis) and
|
||||
* constructs MistralAIProvider directly against the live wired
|
||||
* `MeteringService` so the recording side is exercised end-to- end. The Mistral
|
||||
* SDK is mocked at the module boundary (the real network egress point) so the
|
||||
* provider never reaches the network. The companion integration test
|
||||
* (MistralAiProvider.integration.test.ts) exercises the real Mistral endpoint.
|
||||
* Boots a real PuterServer (in-memory sqlite + dynamo + s3 + mock
|
||||
* redis) and constructs MistralAIProvider directly against the live
|
||||
* wired `MeteringService` so the recording side is exercised end-to-
|
||||
* end. The Mistral SDK is mocked at the module boundary (the real
|
||||
* network egress point) so the provider never reaches the network.
|
||||
* The companion integration test (MistralAiProvider.integration.test.ts)
|
||||
* exercises the real Mistral endpoint.
|
||||
*/
|
||||
|
||||
import { Context } from '../../../../core/context.js';
|
||||
@@ -146,9 +147,7 @@ describe('MistralAIProvider construction', () => {
|
||||
it('constructs the Mistral SDK with the configured API key', () => {
|
||||
makeProvider();
|
||||
expect(mistralCtor).toHaveBeenCalledTimes(1);
|
||||
expect(mistralCtor).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ apiKey: 'test-key' }),
|
||||
);
|
||||
expect(mistralCtor).toHaveBeenCalledWith({ apiKey: 'test-key' });
|
||||
});
|
||||
});
|
||||
|
||||
@@ -607,11 +606,7 @@ describe('MistralAIProvider.complete non-stream output', () => {
|
||||
if (stream) {
|
||||
streamMock.mockResolvedValueOnce(
|
||||
asAsyncIterable([
|
||||
{
|
||||
data: {
|
||||
choices: [{ delta: { content: 'hello' } }],
|
||||
},
|
||||
},
|
||||
{ data: { choices: [{ delta: { content: 'hello' } }] } },
|
||||
{ data: { choices: [], usage } },
|
||||
]),
|
||||
);
|
||||
@@ -648,11 +643,7 @@ describe('MistralAIProvider.complete non-stream output', () => {
|
||||
expect(result).toMatchObject({ message: { content: 'hello' } });
|
||||
}
|
||||
expect(recordSpy).toHaveBeenCalledWith(
|
||||
{
|
||||
prompt_tokens: 5,
|
||||
cached_tokens: 1008,
|
||||
completion_tokens: 30,
|
||||
},
|
||||
{ prompt_tokens: 5, cached_tokens: 1008, completion_tokens: 30 },
|
||||
SYSTEM_ACTOR,
|
||||
`mistral:${model}`,
|
||||
{
|
||||
@@ -697,7 +688,8 @@ describe('MistralAIProvider.complete non-stream output', () => {
|
||||
});
|
||||
|
||||
expect(recordSpy).toHaveBeenCalledTimes(1);
|
||||
const [usage, actor, prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
const [usage, actor, prefix, overrides] =
|
||||
recordSpy.mock.calls[0]!;
|
||||
expect(usage).toEqual({
|
||||
prompt_tokens: 100,
|
||||
completion_tokens: 50,
|
||||
@@ -726,11 +718,15 @@ describe('MistralAIProvider.complete non-stream output', () => {
|
||||
content: [
|
||||
{
|
||||
type: 'thinking',
|
||||
thinking: [{ type: 'text', text: 'step one.' }],
|
||||
thinking: [
|
||||
{ type: 'text', text: 'step one.' },
|
||||
],
|
||||
},
|
||||
{
|
||||
type: 'thinking',
|
||||
thinking: [{ type: 'text', text: 'step two.' }],
|
||||
thinking: [
|
||||
{ type: 'text', text: 'step two.' },
|
||||
],
|
||||
},
|
||||
{ type: 'text', text: 'the answer' },
|
||||
],
|
||||
@@ -920,7 +916,8 @@ describe('MistralAIProvider.complete streaming', () => {
|
||||
|
||||
// mistral-small-2603: prompt=15, completion=60.
|
||||
expect(recordSpy).toHaveBeenCalledTimes(1);
|
||||
const [, , prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
const [, , prefix, overrides] =
|
||||
recordSpy.mock.calls[0]!;
|
||||
expect(prefix).toBe('mistral:mistral-small-2603');
|
||||
expect(overrides).toMatchObject({
|
||||
prompt_tokens: 4 * 15,
|
||||
@@ -1118,7 +1115,7 @@ describe('MistralAIProvider image_url coercion', () => {
|
||||
usage: { promptTokens: 1, completionTokens: 1 },
|
||||
};
|
||||
|
||||
it("rewrites canonical image_url parts to the SDK's camelCase imageUrl", async () => {
|
||||
it('rewrites canonical image_url parts to the SDK\'s camelCase imageUrl', async () => {
|
||||
const { provider } = makeProvider();
|
||||
completeMock.mockResolvedValueOnce(baseCompletion);
|
||||
|
||||
@@ -1178,8 +1175,7 @@ describe('MistralAIProvider image_url coercion', () => {
|
||||
},
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url:
|
||||
'https://example.com/already-flat.png',
|
||||
image_url: 'https://example.com/already-flat.png',
|
||||
},
|
||||
],
|
||||
},
|
||||
@@ -1255,9 +1251,7 @@ describe('MistralAIProvider image_url coercion', () => {
|
||||
{ type: 'text', text: 'what is in this image?' },
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url: {
|
||||
url: 'https://example.com/photo.jpg',
|
||||
},
|
||||
image_url: { url: 'https://example.com/photo.jpg' },
|
||||
},
|
||||
],
|
||||
},
|
||||
@@ -1266,11 +1260,7 @@ describe('MistralAIProvider image_url coercion', () => {
|
||||
);
|
||||
|
||||
const [args] = completeMock.mock.calls[0]!;
|
||||
const parts = args.messages[0].content as {
|
||||
type: string;
|
||||
text?: string;
|
||||
imageUrl?: unknown;
|
||||
}[];
|
||||
const parts = args.messages[0].content as { type: string; text?: string; imageUrl?: unknown }[];
|
||||
const textPart = parts.find((p) => p.type === 'text');
|
||||
const imgPart = parts.find((p) => p.type === 'image_url');
|
||||
expect(textPart?.text).toBe('what is in this image?');
|
||||
@@ -1310,41 +1300,3 @@ describe('MistralAIProvider.checkModeration', () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Mistral service speed wire mapping', () => {
|
||||
it('adds the tier after SDK serialization and removes its internal header', async () => {
|
||||
makeProvider();
|
||||
const options = mistralCtor.mock.calls[0][0] as {
|
||||
httpClient: import('@mistralai/mistralai/lib/http.js').HTTPClient;
|
||||
};
|
||||
let sent: Request | undefined;
|
||||
vi.stubGlobal('fetch', async (input: Request) => {
|
||||
sent = input;
|
||||
return new Response('{}', {
|
||||
headers: { 'content-type': 'application/json' },
|
||||
});
|
||||
});
|
||||
try {
|
||||
await options.httpClient.request(
|
||||
new Request('https://api.mistral.ai/v1/chat/completions', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
'x-puter-service-tier': 'auto',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: 'mistral-medium-latest',
|
||||
messages: [],
|
||||
}),
|
||||
}),
|
||||
);
|
||||
expect(sent?.headers.has('x-puter-service-tier')).toBe(false);
|
||||
expect(await sent?.json()).toMatchObject({
|
||||
model: 'mistral-medium-latest',
|
||||
service_tier: 'auto',
|
||||
});
|
||||
} finally {
|
||||
vi.unstubAllGlobals();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -17,8 +17,6 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { HTTPClient } from '@mistralai/mistralai/lib/http.js';
|
||||
import { servedServiceSpeed } from '../../utils/serviceSpeed.js';
|
||||
import { Mistral } from '@mistralai/mistralai';
|
||||
import { ChatCompletionResponse } from '@mistralai/mistralai/models/components/chatcompletionresponse.js';
|
||||
import { MistralError } from '@mistralai/mistralai/models/errors/mistralerror.js';
|
||||
@@ -28,7 +26,6 @@ import { Context } from '../../../../core/context.js';
|
||||
import { HttpError } from '../../../../core/http/HttpError.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
import type {
|
||||
IChatModel,
|
||||
IChatCompleteResult,
|
||||
IChatProvider,
|
||||
ICompleteArguments,
|
||||
@@ -109,7 +106,6 @@ const coerceMistralUsage = (usage: UsageInfo) => {
|
||||
prompt_tokens: (usage.promptTokens ?? 0) - cachedTokens,
|
||||
completion_tokens: usage.completionTokens ?? 0,
|
||||
prompt_tokens_details: { cached_tokens: cachedTokens },
|
||||
service_tier: usage.additionalProperties?.service_tier,
|
||||
};
|
||||
};
|
||||
|
||||
@@ -119,24 +115,7 @@ export class MistralAIProvider implements IChatProvider {
|
||||
#meteringService: MeteringService;
|
||||
|
||||
constructor(config: { apiKey: string }, meteringService: MeteringService) {
|
||||
const httpClient = new HTTPClient();
|
||||
// The SDK strips service_tier during body serialization.
|
||||
httpClient.addHook('beforeRequest', async (request) => {
|
||||
const tier = request.headers.get('x-puter-service-tier');
|
||||
if (!tier) return;
|
||||
const headers = new Headers(request.headers);
|
||||
headers.delete('x-puter-service-tier');
|
||||
const body = await request.json();
|
||||
return new Request(request, {
|
||||
headers,
|
||||
body: JSON.stringify({
|
||||
...body,
|
||||
service_tier: tier === 'auto' ? 'auto' : 'standard_only',
|
||||
}),
|
||||
});
|
||||
});
|
||||
this.#client = new Mistral({
|
||||
httpClient,
|
||||
apiKey: config.apiKey,
|
||||
});
|
||||
this.#meteringService = meteringService;
|
||||
@@ -202,22 +181,17 @@ export class MistralAIProvider implements IChatProvider {
|
||||
return `mistral:${modelId}`;
|
||||
}
|
||||
|
||||
async complete(
|
||||
{
|
||||
messages,
|
||||
stream,
|
||||
model,
|
||||
tools,
|
||||
max_tokens,
|
||||
temperature,
|
||||
normalize,
|
||||
response,
|
||||
custom,
|
||||
service_tier,
|
||||
serviceSpeed: requestedServiceSpeed,
|
||||
}: ICompleteArguments,
|
||||
resolved?: IChatModel,
|
||||
): Promise<IChatCompleteResult> {
|
||||
async complete({
|
||||
messages,
|
||||
stream,
|
||||
model,
|
||||
tools,
|
||||
max_tokens,
|
||||
temperature,
|
||||
normalize,
|
||||
response,
|
||||
custom,
|
||||
}: ICompleteArguments): Promise<IChatCompleteResult> {
|
||||
// Mistral's reasoning prompt mode: with `prompt_mode: 'reasoning'`,
|
||||
// reasoning-capable models return their thinking as structured ThinkChunk
|
||||
// content (which the splitter below separates into `reasoning`)
|
||||
@@ -243,13 +217,10 @@ export class MistralAIProvider implements IChatProvider {
|
||||
}
|
||||
|
||||
const selectedModel =
|
||||
resolved ??
|
||||
((await this.models()).find((m) =>
|
||||
(await this.models()).find((m) =>
|
||||
[m.id, ...(m.aliases || [])].includes(model),
|
||||
) ||
|
||||
(await this.models()).find(
|
||||
(m) => m.id === this.getDefaultModel(),
|
||||
)!);
|
||||
(await this.models()).find((m) => m.id === this.getDefaultModel())!;
|
||||
const actor = Context.get('actor');
|
||||
let completion: Awaited<
|
||||
ReturnType<Mistral['chat']['complete' | 'stream']>
|
||||
@@ -271,9 +242,6 @@ export class MistralAIProvider implements IChatProvider {
|
||||
// The SDK retries nothing by default and has no timeout once a
|
||||
// signal is passed; a stream is bounded by its caller instead.
|
||||
{
|
||||
...(service_tier !== undefined
|
||||
? { headers: { 'x-puter-service-tier': service_tier } }
|
||||
: {}),
|
||||
signal: stream
|
||||
? Context.get('upstreamSignal')
|
||||
: withSdkTimeout(Context.get('upstreamSignal')),
|
||||
@@ -430,8 +398,7 @@ export class MistralAIProvider implements IChatProvider {
|
||||
},
|
||||
completion: completion as ChatCompletionResponse,
|
||||
stream,
|
||||
usage_calculator: ({ usage, setUsageCosts, setUsageDetails }) => {
|
||||
const serviceSpeed = servedServiceSpeed({ usage });
|
||||
usage_calculator: ({ usage, setUsageCosts }) => {
|
||||
// `coerceMistralUsage` has already taken cached reads out of
|
||||
// `prompt_tokens`.
|
||||
const metered = meterChatUsage(
|
||||
@@ -440,16 +407,8 @@ export class MistralAIProvider implements IChatProvider {
|
||||
this.meteringModelKey(selectedModel.id),
|
||||
selectedModel,
|
||||
OpenAIUtil.extractMeteredUsage(usage),
|
||||
{ serviceSpeed },
|
||||
);
|
||||
setUsageCosts(metered.costs);
|
||||
if (requestedServiceSpeed !== undefined)
|
||||
setUsageDetails({
|
||||
...OpenAIUtil.usageDetailsFromTrackedUsage(
|
||||
metered.usage,
|
||||
)!,
|
||||
serviceSpeed,
|
||||
});
|
||||
return metered.usage;
|
||||
},
|
||||
});
|
||||
|
||||
@@ -42,7 +42,6 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 136,
|
||||
@@ -76,7 +75,6 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 150,
|
||||
@@ -107,7 +105,6 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 50,
|
||||
@@ -138,7 +135,6 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 15,
|
||||
@@ -163,7 +159,6 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 140,
|
||||
@@ -193,7 +188,6 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 30,
|
||||
@@ -219,7 +213,6 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 20,
|
||||
@@ -245,7 +238,6 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 15,
|
||||
@@ -270,7 +262,6 @@ export const MISTRAL_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 1.75 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 10,
|
||||
|
||||
@@ -30,7 +30,6 @@ export const MOONSHOT_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'kimi-k3.1-pro-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200, // $2.00 per 1M
|
||||
@@ -78,7 +77,6 @@ export const MOONSHOT_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'kimi-k3-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 300, // $3.00 per 1M
|
||||
@@ -106,7 +104,6 @@ export const MOONSHOT_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'kimi-k2.6-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 95, // $0.95 per 1M
|
||||
@@ -129,7 +126,6 @@ export const MOONSHOT_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedVariants: { fast: 'kimi-k2.7-code-highspeed' },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 95,
|
||||
|
||||
+1
-157
@@ -249,130 +249,6 @@ describe('OpenAiChatProvider.complete argument validation', () => {
|
||||
expect(result).toEqual({ delegated: true });
|
||||
});
|
||||
|
||||
it('delegates a 12h cache lifetime to the Responses provider, which is the only API that writes it', async () => {
|
||||
const { provider } = makeProvider();
|
||||
const sibling = {
|
||||
complete: vi.fn().mockResolvedValue({ delegated: true }),
|
||||
};
|
||||
provider.setResponsesProvider(
|
||||
sibling as unknown as Parameters<
|
||||
typeof provider.setResponsesProvider
|
||||
>[0],
|
||||
);
|
||||
|
||||
const params = {
|
||||
model: 'gpt-6-luna',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
prompt_cache_options: { ttl: '12h' as const },
|
||||
};
|
||||
const result = await withTestActor(() => provider.complete(params));
|
||||
|
||||
expect(createMock).not.toHaveBeenCalled();
|
||||
expect(sibling.complete).toHaveBeenCalledWith(params);
|
||||
expect(result).toEqual({ delegated: true });
|
||||
});
|
||||
|
||||
it.each([
|
||||
['30m', { mode: 'explicit', ttl: '30m' }],
|
||||
['12h', { mode: 'explicit', ttl: '12h' }],
|
||||
])(
|
||||
'delegates a %s breakpoint lifetime to the Responses provider, since Chat Completions rejects the field',
|
||||
async (_ttl, breakpoint) => {
|
||||
const { provider } = makeProvider();
|
||||
const sibling = {
|
||||
complete: vi.fn().mockResolvedValue({ delegated: true }),
|
||||
};
|
||||
provider.setResponsesProvider(
|
||||
sibling as unknown as Parameters<
|
||||
typeof provider.setResponsesProvider
|
||||
>[0],
|
||||
);
|
||||
|
||||
const params = {
|
||||
model: 'gpt-6-luna',
|
||||
messages: [
|
||||
{
|
||||
role: 'system',
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: 'long prefix',
|
||||
prompt_cache_breakpoint: breakpoint,
|
||||
},
|
||||
],
|
||||
},
|
||||
{ role: 'user', content: 'hi' },
|
||||
],
|
||||
};
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete(params as never),
|
||||
);
|
||||
|
||||
expect(createMock).not.toHaveBeenCalled();
|
||||
expect(sibling.complete).toHaveBeenCalledWith(params);
|
||||
expect(result).toEqual({ delegated: true });
|
||||
},
|
||||
);
|
||||
|
||||
it('keeps a breakpoint without a lifetime on the Chat Completions path, intact', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
message: { content: 'ok', role: 'assistant' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: { prompt_tokens: 1, completion_tokens: 1 },
|
||||
});
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-6-luna',
|
||||
messages: [
|
||||
{
|
||||
role: 'system',
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: 'long prefix',
|
||||
prompt_cache_breakpoint: { mode: 'explicit' },
|
||||
},
|
||||
],
|
||||
},
|
||||
{ role: 'user', content: 'hi' },
|
||||
],
|
||||
} as never),
|
||||
);
|
||||
|
||||
const [args] = createMock.mock.calls[0]!;
|
||||
expect(args.messages[0]).toMatchObject({
|
||||
role: 'system',
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: 'long prefix',
|
||||
prompt_cache_breakpoint: { mode: 'explicit' },
|
||||
},
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
it('throws 400 for a 12h cache lifetime without a Responses sibling provider', async () => {
|
||||
const { provider } = makeProvider();
|
||||
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-6-luna',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
prompt_cache_options: { ttl: '12h' },
|
||||
}),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
expect(createMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('requestPricing delegates to the Responses sibling (web_search\'s own hold), and no-ops without one', async () => {
|
||||
const { provider } = makeProvider();
|
||||
const args = {
|
||||
@@ -473,31 +349,6 @@ describe('OpenAiChatProvider.complete request shape', () => {
|
||||
expect(args.safety_identifier).toBe('puter-u42');
|
||||
});
|
||||
|
||||
it('forwards 30m prompt_cache_options and the wire spelling of prompt_cache_retention', async () => {
|
||||
const { provider } = makeProvider();
|
||||
createMock.mockResolvedValueOnce(baseCompletion);
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-6-luna',
|
||||
messages: [{ role: 'user', content: 'hello' }],
|
||||
prompt_cache_retention: 'in-memory',
|
||||
prompt_cache_options: {
|
||||
mode: 'explicit',
|
||||
ttl: '30m',
|
||||
prewarm: true,
|
||||
} as never,
|
||||
}),
|
||||
);
|
||||
|
||||
const [args] = createMock.mock.calls[0]!;
|
||||
expect(args.prompt_cache_retention).toBe('in_memory');
|
||||
expect(args.prompt_cache_options).toEqual({
|
||||
mode: 'explicit',
|
||||
ttl: '30m',
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
'gpt-6-astra',
|
||||
'gpt-6.1-sol',
|
||||
@@ -814,7 +665,7 @@ describe('OpenAiChatProvider.complete non-stream output', () => {
|
||||
},
|
||||
});
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-6-luna',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
@@ -834,13 +685,6 @@ describe('OpenAiChatProvider.complete non-stream output', () => {
|
||||
cached_tokens: 1000 * Number(luna.costs.cached_tokens),
|
||||
cache_write_tokens: 3000 * Number(luna.costs.cache_write_tokens),
|
||||
});
|
||||
// Chat Completions only writes OpenAI's default 30m entries.
|
||||
expect((result as { usageDetails: unknown }).usageDetails).toEqual({
|
||||
inputTokens: 1000,
|
||||
outputTokens: 12,
|
||||
cacheReadTokens: 1000,
|
||||
cacheWrite30mTokens: 3000,
|
||||
});
|
||||
});
|
||||
|
||||
it('zeroes cached_tokens when prompt_tokens_details is missing', async () => {
|
||||
|
||||
@@ -20,7 +20,6 @@
|
||||
import { HttpError } from '@heyputer/backend/src/core/http/HttpError.js';
|
||||
import { OpenAI } from 'openai';
|
||||
import { ChatCompletionCreateParams } from 'openai/resources/index.js';
|
||||
import { servedServiceSpeed } from '../../utils/serviceSpeed.js';
|
||||
import { Context } from '../../../../core/context.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
import type {
|
||||
@@ -36,10 +35,7 @@ import { make_openai_tools } from '../../utils/FunctionCalling.js';
|
||||
import * as OpenAiUtil from '../../utils/OpenAIUtil.js';
|
||||
import {
|
||||
clampReasoningEffort,
|
||||
messagesHaveCacheBreakpointTtl,
|
||||
openAICompatParams,
|
||||
promptCacheOptionsToWire,
|
||||
promptCacheRetentionToWire,
|
||||
} from '../../utils/openaiParams.js';
|
||||
import { OPEN_AI_MODELS } from './models.js';
|
||||
import type { OpenAiResponsesChatProvider } from './OpenAiChatResponsesProvider.js';
|
||||
@@ -118,7 +114,6 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
|
||||
async complete(
|
||||
params: ICompleteArguments,
|
||||
resolved?: IChatModel,
|
||||
): ReturnType<IChatProvider['complete']> {
|
||||
const {
|
||||
max_tokens,
|
||||
@@ -131,16 +126,8 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
temperature,
|
||||
text,
|
||||
prompt_cache_key,
|
||||
prompt_cache_retention,
|
||||
prompt_cache_options,
|
||||
} = params;
|
||||
let { messages, model } = params;
|
||||
const cacheOptions = promptCacheOptionsToWire(prompt_cache_options);
|
||||
if (params.service_tier === 'ultrafast' && this.#responsesProvider) {
|
||||
return resolved
|
||||
? this.#responsesProvider.complete(params, resolved)
|
||||
: this.#responsesProvider.complete(params);
|
||||
}
|
||||
if (
|
||||
tools?.filter((e) => isWebSearchTool(e as Record<string, unknown>))
|
||||
.length
|
||||
@@ -154,9 +141,7 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
return await (resolved
|
||||
? this.#responsesProvider.complete(params, resolved)
|
||||
: this.#responsesProvider.complete(params));
|
||||
return await this.#responsesProvider.complete(params);
|
||||
}
|
||||
// Inline compaction is a Responses-API feature; chat.completions can't
|
||||
// express `context_management` or a `compaction` content block.
|
||||
@@ -170,26 +155,7 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
return await (resolved
|
||||
? this.#responsesProvider.complete(params, resolved)
|
||||
: this.#responsesProvider.complete(params));
|
||||
}
|
||||
// Only the Responses API writes 12h cache entries or takes a lifetime
|
||||
// on a `prompt_cache_breakpoint`; Chat Completions rejects both.
|
||||
if (
|
||||
cacheOptions?.ttl === '12h' ||
|
||||
messagesHaveCacheBreakpointTtl(messages)
|
||||
) {
|
||||
if (!this.#responsesProvider) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'a prompt cache lifetime requires the OpenAI Responses provider, which is not configured',
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
return await (resolved
|
||||
? this.#responsesProvider.complete(params, resolved)
|
||||
: this.#responsesProvider.complete(params));
|
||||
return await this.#responsesProvider.complete(params);
|
||||
}
|
||||
// Validate messages
|
||||
if (!Array.isArray(messages)) {
|
||||
@@ -202,11 +168,9 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
model = model ?? this.#defaultModel;
|
||||
|
||||
const modelUsed =
|
||||
resolved ??
|
||||
(this.models().find((m) =>
|
||||
this.models().find((m) =>
|
||||
[m.id, ...(m.aliases || [])].includes(model),
|
||||
) ||
|
||||
this.models().find((m) => m.id === this.getDefaultModel())!);
|
||||
) || this.models().find((m) => m.id === this.getDefaultModel())!;
|
||||
|
||||
// messages.unshift({
|
||||
// role: 'system',
|
||||
@@ -216,9 +180,6 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
const userIdentifier = upstreamUserIdentifier(actor);
|
||||
// Cache key defaults to the actor identifier; see upstreamUserIdentifier.
|
||||
const cacheKey = prompt_cache_key ?? userIdentifier;
|
||||
const cacheRetention = promptCacheRetentionToWire(
|
||||
prompt_cache_retention,
|
||||
);
|
||||
|
||||
// Strip Anthropic-only shape a fallback-replayed message can carry
|
||||
// (thinking/server-tool blocks, cache_control, citations) before the
|
||||
@@ -255,17 +216,8 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
user: userIdentifier,
|
||||
safety_identifier: userIdentifier,
|
||||
...(cacheKey !== undefined ? { prompt_cache_key: cacheKey } : {}),
|
||||
...(cacheRetention !== undefined
|
||||
? { prompt_cache_retention: cacheRetention }
|
||||
: {}),
|
||||
...(cacheOptions !== undefined
|
||||
? { prompt_cache_options: cacheOptions }
|
||||
: {}),
|
||||
messages: messages,
|
||||
model: modelUsed.id,
|
||||
...(params.service_tier !== undefined
|
||||
? { service_tier: params.service_tier }
|
||||
: {}),
|
||||
...(mappedTools?.length ? { tools: mappedTools } : {}),
|
||||
...(max_tokens !== undefined
|
||||
? { max_completion_tokens: max_tokens }
|
||||
@@ -302,17 +254,28 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
);
|
||||
|
||||
return OpenAiUtil.handle_completion_output({
|
||||
usage_calculator: ({
|
||||
usage,
|
||||
service_tier,
|
||||
setUsageCosts,
|
||||
setUsageDetails,
|
||||
}) => {
|
||||
const serviceSpeed = servedServiceSpeed({ service_tier });
|
||||
// GPT-5.6 and later report cache writes inside
|
||||
// `prompt_tokens`, like cached reads; Chat Completions only
|
||||
// writes the default 30m entries, billed at 1.25x input.
|
||||
const trackedUsage = OpenAiUtil.splitCachedPrompt(usage);
|
||||
usage_calculator: ({ usage, setUsageCosts }) => {
|
||||
const cachedTokens =
|
||||
usage.prompt_tokens_details?.cached_tokens ?? 0;
|
||||
// GPT-5.6 and later bill cache writes at 1.25x input. They're
|
||||
// reported inside `prompt_tokens`, like cached reads.
|
||||
// The SDK doesn't type `cache_write_tokens` yet.
|
||||
const cacheWriteTokens =
|
||||
(
|
||||
usage.prompt_tokens_details as
|
||||
{ cache_write_tokens?: number } | undefined
|
||||
)?.cache_write_tokens ?? 0;
|
||||
const trackedUsage = {
|
||||
prompt_tokens:
|
||||
(usage.prompt_tokens ?? 0) -
|
||||
cachedTokens -
|
||||
cacheWriteTokens,
|
||||
completion_tokens: usage.completion_tokens ?? 0,
|
||||
cached_tokens: cachedTokens,
|
||||
...(cacheWriteTokens
|
||||
? { cache_write_tokens: cacheWriteTokens }
|
||||
: {}),
|
||||
};
|
||||
|
||||
const metered = meterChatUsage(
|
||||
this.#meteringService,
|
||||
@@ -320,17 +283,8 @@ export class OpenAiChatProvider implements IChatProvider {
|
||||
this.meteringModelKey(modelUsed.id),
|
||||
modelUsed,
|
||||
trackedUsage,
|
||||
{ serviceSpeed },
|
||||
);
|
||||
setUsageCosts(metered.costs);
|
||||
setUsageDetails({
|
||||
...OpenAiUtil.usageDetailsFromTrackedUsage(metered.usage, {
|
||||
writeTtl: '30m',
|
||||
})!,
|
||||
...(params.serviceSpeed !== undefined
|
||||
? { serviceSpeed }
|
||||
: {}),
|
||||
});
|
||||
return metered.usage;
|
||||
},
|
||||
stream,
|
||||
|
||||
@@ -246,7 +246,6 @@ describe('OpenAiResponsesChatProvider model catalog', () => {
|
||||
prompt_tokens: input,
|
||||
cached_tokens: cached,
|
||||
cache_write_tokens: input * 1.25,
|
||||
cache_write_12h_tokens: input * 2.5,
|
||||
completion_tokens: output,
|
||||
},
|
||||
context: 1_050_000,
|
||||
@@ -845,7 +844,7 @@ describe('OpenAiResponsesChatProvider.complete non-stream output', () => {
|
||||
},
|
||||
});
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-6-luna',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
@@ -868,115 +867,6 @@ describe('OpenAiResponsesChatProvider.complete non-stream output', () => {
|
||||
cached_tokens: 1000 * Number(luna.costs.cached_tokens),
|
||||
cache_write_tokens: 3000 * Number(luna.costs.cache_write_tokens),
|
||||
});
|
||||
// Writes without a per-lifetime breakdown are OpenAI's default 30m
|
||||
// entries, and the neutral breakdown says so.
|
||||
expect((result as { usageDetails: unknown }).usageDetails).toEqual({
|
||||
inputTokens: 1000,
|
||||
outputTokens: 20,
|
||||
cacheReadTokens: 1000,
|
||||
cacheWrite30mTokens: 3000,
|
||||
});
|
||||
});
|
||||
|
||||
it('bills 12h cache writes at 2.5x input and the other writes at 1.25x', async () => {
|
||||
const luna = OPEN_AI_MODELS.find((m) => m.id === 'gpt-6-luna')!;
|
||||
const { provider } = makeProvider();
|
||||
// One request can write both lifetimes: an explicit 12h breakpoint
|
||||
// plus the implicit 30m one OpenAI adds at the end of the prompt.
|
||||
responsesCreateMock.mockResolvedValueOnce({
|
||||
output: [{ role: 'assistant' }],
|
||||
output_text: 'ok',
|
||||
usage: {
|
||||
input_tokens: 6026,
|
||||
output_tokens: 5,
|
||||
input_tokens_details: {
|
||||
cached_tokens: 0,
|
||||
cache_write_tokens: 6023,
|
||||
cache_write_tokens_details: {
|
||||
ttl_12h_tokens: 6012,
|
||||
ttl_30m_tokens: 11,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-6-luna',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
prompt_cache_options: { ttl: '12h' },
|
||||
}),
|
||||
);
|
||||
|
||||
const [usage, , , overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(usage).toEqual({
|
||||
prompt_tokens: 3,
|
||||
completion_tokens: 5,
|
||||
cached_tokens: 0,
|
||||
cache_write_tokens: 11,
|
||||
cache_write_12h_tokens: 6012,
|
||||
});
|
||||
expect(luna.costs.cache_write_12h_tokens).toBe(
|
||||
Number(luna.costs.prompt_tokens) * 2.5,
|
||||
);
|
||||
expect(overrides).toEqual({
|
||||
prompt_tokens: 3 * Number(luna.costs.prompt_tokens),
|
||||
completion_tokens: 5 * Number(luna.costs.completion_tokens),
|
||||
cached_tokens: 0,
|
||||
cache_write_tokens: 11 * Number(luna.costs.cache_write_tokens),
|
||||
cache_write_12h_tokens:
|
||||
6012 * Number(luna.costs.cache_write_12h_tokens),
|
||||
});
|
||||
expect((result as { usageDetails: unknown }).usageDetails).toEqual({
|
||||
inputTokens: 3,
|
||||
outputTokens: 5,
|
||||
cacheWrite30mTokens: 11,
|
||||
cacheWrite12hTokens: 6012,
|
||||
});
|
||||
});
|
||||
|
||||
it('forwards prompt_cache_options and the wire spelling of prompt_cache_retention', async () => {
|
||||
const { provider } = makeProvider();
|
||||
responsesCreateMock.mockResolvedValueOnce({
|
||||
output: [{ role: 'assistant' }],
|
||||
output_text: 'ok',
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
});
|
||||
|
||||
await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-6-luna',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
prompt_cache_retention: 'in-memory',
|
||||
prompt_cache_options: {
|
||||
mode: 'explicit',
|
||||
ttl: '12h',
|
||||
comparison_response_id: 'resp_1',
|
||||
} as never,
|
||||
}),
|
||||
);
|
||||
|
||||
const [args] = responsesCreateMock.mock.calls[0]!;
|
||||
expect(args.prompt_cache_retention).toBe('in_memory');
|
||||
expect(args.prompt_cache_options).toEqual({
|
||||
mode: 'explicit',
|
||||
ttl: '12h',
|
||||
});
|
||||
});
|
||||
|
||||
it('rejects a cache lifetime OpenAI does not offer before calling upstream', async () => {
|
||||
const { provider } = makeProvider();
|
||||
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-6-luna',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
prompt_cache_options: { ttl: '2h' } as never,
|
||||
}),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
expect(responsesCreateMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each([
|
||||
@@ -1178,69 +1068,6 @@ describe('OpenAiResponsesChatProvider.complete streaming', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('labels streamed cache writes by lifetime on the final usage chunk', async () => {
|
||||
const luna = OPEN_AI_MODELS.find((m) => m.id === 'gpt-6-luna')!;
|
||||
const { provider } = makeProvider();
|
||||
responsesCreateMock.mockReturnValueOnce(
|
||||
asAsyncIterable([
|
||||
{ type: 'response.output_text.delta', delta: 'ok' },
|
||||
{
|
||||
type: 'response.completed',
|
||||
response: {
|
||||
usage: {
|
||||
input_tokens: 6026,
|
||||
output_tokens: 5,
|
||||
input_tokens_details: {
|
||||
cached_tokens: 0,
|
||||
cache_write_tokens: 6023,
|
||||
cache_write_tokens_details: {
|
||||
ttl_12h_tokens: 6012,
|
||||
ttl_30m_tokens: 11,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
]),
|
||||
);
|
||||
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'gpt-6-luna',
|
||||
messages: [{ role: 'user', content: 'say ok' }],
|
||||
stream: true,
|
||||
prompt_cache_options: { ttl: '12h' },
|
||||
}),
|
||||
);
|
||||
const harness = makeCapturingChatStream();
|
||||
await (
|
||||
result as {
|
||||
init_chat_stream: (p: { chatStream: unknown }) => Promise<void>;
|
||||
}
|
||||
).init_chat_stream({ chatStream: harness.chatStream });
|
||||
|
||||
const usageEvent = harness.events().find((e) => e.type === 'usage');
|
||||
expect(usageEvent?.usage).toEqual({
|
||||
prompt_tokens: 3,
|
||||
completion_tokens: 5,
|
||||
cached_tokens: 0,
|
||||
cache_write_tokens: 11,
|
||||
cache_write_12h_tokens: 6012,
|
||||
});
|
||||
expect(usageEvent?.usageDetails).toEqual({
|
||||
inputTokens: 3,
|
||||
outputTokens: 5,
|
||||
cacheWrite30mTokens: 11,
|
||||
cacheWrite12hTokens: 6012,
|
||||
});
|
||||
const [, , , overrides] = recordSpy.mock.calls[0]!;
|
||||
expect(overrides).toMatchObject({
|
||||
cache_write_tokens: 11 * Number(luna.costs.cache_write_tokens),
|
||||
cache_write_12h_tokens:
|
||||
6012 * Number(luna.costs.cache_write_12h_tokens),
|
||||
});
|
||||
});
|
||||
|
||||
it('emits a tool_use block when response.output_item.done arrives with a function_call', async () => {
|
||||
const { provider } = makeProvider();
|
||||
responsesCreateMock.mockReturnValueOnce(
|
||||
|
||||
@@ -19,7 +19,6 @@
|
||||
|
||||
import { OpenAI } from 'openai';
|
||||
import { ResponseCreateParams } from 'openai/resources/responses/responses.mjs';
|
||||
import { servedServiceSpeed } from '../../utils/serviceSpeed.js';
|
||||
import { Context } from '../../../../core/context.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
import type {
|
||||
@@ -33,8 +32,6 @@ import * as OpenAiUtil from '../../utils/OpenAIUtil.js';
|
||||
import {
|
||||
clampReasoningEffort,
|
||||
openAICompatParams,
|
||||
promptCacheOptionsToWire,
|
||||
promptCacheRetentionToWire,
|
||||
rejectStatefulResponsesFields,
|
||||
} from '../../utils/openaiParams.js';
|
||||
import { responseSamplingParams } from '../../utils/responseSampling.js';
|
||||
@@ -135,42 +132,37 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
|
||||
return rate ? { extraCost: rate } : {};
|
||||
}
|
||||
|
||||
async complete(
|
||||
{
|
||||
messages,
|
||||
model,
|
||||
max_tokens,
|
||||
moderation,
|
||||
tools,
|
||||
tool_choice,
|
||||
parallel_tool_calls,
|
||||
include,
|
||||
conversation,
|
||||
compaction,
|
||||
context_management,
|
||||
previous_response_id,
|
||||
instructions,
|
||||
metadata,
|
||||
prompt,
|
||||
prompt_cache_key,
|
||||
prompt_cache_retention,
|
||||
prompt_cache_options,
|
||||
store,
|
||||
top_p,
|
||||
truncation,
|
||||
background,
|
||||
service_tier,
|
||||
serviceSpeed: requestedServiceSpeed,
|
||||
verbosity,
|
||||
stream,
|
||||
reasoning,
|
||||
reasoning_effort,
|
||||
temperature,
|
||||
text,
|
||||
outputFormat,
|
||||
}: ICompleteArguments,
|
||||
resolved?: IChatModel,
|
||||
): ReturnType<IChatProvider['complete']> {
|
||||
async complete({
|
||||
messages,
|
||||
model,
|
||||
max_tokens,
|
||||
moderation,
|
||||
tools,
|
||||
tool_choice,
|
||||
parallel_tool_calls,
|
||||
include,
|
||||
conversation,
|
||||
compaction,
|
||||
context_management,
|
||||
previous_response_id,
|
||||
instructions,
|
||||
metadata,
|
||||
prompt,
|
||||
prompt_cache_key,
|
||||
prompt_cache_retention,
|
||||
store,
|
||||
top_p,
|
||||
truncation,
|
||||
background,
|
||||
service_tier,
|
||||
verbosity,
|
||||
stream,
|
||||
reasoning,
|
||||
reasoning_effort,
|
||||
temperature,
|
||||
text,
|
||||
outputFormat,
|
||||
}: ICompleteArguments): ReturnType<IChatProvider['complete']> {
|
||||
// Validate messages
|
||||
if (!Array.isArray(messages)) {
|
||||
throw new HttpError(400, '`messages` must be an array', {
|
||||
@@ -188,13 +180,12 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
|
||||
model = model ?? this.#defaultModel;
|
||||
|
||||
const modelUsed =
|
||||
resolved ??
|
||||
(this.models({ no_restrictions: true }).find((m) =>
|
||||
this.models({ no_restrictions: true }).find((m) =>
|
||||
[m.id, ...(m.aliases || [])].includes(model),
|
||||
) ||
|
||||
this.models({ no_restrictions: true }).find(
|
||||
(m) => m.id === this.getDefaultModel(),
|
||||
)!);
|
||||
this.models({ no_restrictions: true }).find(
|
||||
(m) => m.id === this.getDefaultModel(),
|
||||
)!;
|
||||
|
||||
// messages.unshift({
|
||||
// role: 'system',
|
||||
@@ -204,10 +195,6 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
|
||||
const userIdentifier = upstreamUserIdentifier(actor);
|
||||
// Cache key defaults to the actor identifier; see upstreamUserIdentifier.
|
||||
const cacheKey = prompt_cache_key ?? userIdentifier;
|
||||
const cacheRetention = promptCacheRetentionToWire(
|
||||
prompt_cache_retention,
|
||||
);
|
||||
const cacheOptions = promptCacheOptionsToWire(prompt_cache_options);
|
||||
|
||||
if (tools) {
|
||||
// Unravel tools to OpenAI Responses API format
|
||||
@@ -301,11 +288,8 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
|
||||
...(instructions !== undefined ? { instructions } : {}),
|
||||
...(metadata !== undefined ? { metadata } : {}),
|
||||
...(cacheKey !== undefined ? { prompt_cache_key: cacheKey } : {}),
|
||||
...(cacheRetention !== undefined
|
||||
? { prompt_cache_retention: cacheRetention }
|
||||
: {}),
|
||||
...(cacheOptions !== undefined
|
||||
? { prompt_cache_options: cacheOptions }
|
||||
...(prompt_cache_retention !== undefined
|
||||
? { prompt_cache_retention }
|
||||
: {}),
|
||||
...(store !== undefined ? { store } : {}),
|
||||
...(max_tokens !== undefined
|
||||
@@ -326,18 +310,24 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
|
||||
{ signal: Context.get('upstreamSignal') },
|
||||
);
|
||||
return OpenAiUtil.handle_completion_output_responses_api({
|
||||
usage_calculator: ({
|
||||
usage,
|
||||
service_tier,
|
||||
webSearchCalls,
|
||||
setUsageCosts,
|
||||
setUsageDetails,
|
||||
}) => {
|
||||
// GPT-5.6 and later report cache writes inside `input_tokens`,
|
||||
// like cached reads, split by lifetime (30m and 12h entries
|
||||
// cost 1.25x and 2.5x input to write).
|
||||
usage_calculator: ({ usage, webSearchCalls, setUsageCosts }) => {
|
||||
const cachedTokens =
|
||||
(usage as any).input_tokens_details?.cached_tokens ?? 0;
|
||||
// GPT-5.6 and later bill cache writes at 1.25x input. They're
|
||||
// reported inside `input_tokens`, like cached reads.
|
||||
const cacheWriteTokens =
|
||||
(usage as any).input_tokens_details?.cache_write_tokens ??
|
||||
0;
|
||||
const trackedUsage: Record<string, number> = {
|
||||
...OpenAiUtil.splitCachedPrompt(usage),
|
||||
prompt_tokens:
|
||||
((usage as any).input_tokens ?? 0) -
|
||||
cachedTokens -
|
||||
cacheWriteTokens,
|
||||
completion_tokens: (usage as any).output_tokens ?? 0,
|
||||
cached_tokens: cachedTokens,
|
||||
...(cacheWriteTokens
|
||||
? { cache_write_tokens: cacheWriteTokens }
|
||||
: {}),
|
||||
...(webSearchCalls
|
||||
? { web_search_calls: webSearchCalls }
|
||||
: {}),
|
||||
@@ -354,31 +344,18 @@ export class OpenAiResponsesChatProvider implements IChatProvider {
|
||||
// model is a reasoning model, not a static per-model
|
||||
// cost-table entry — the model's rates would otherwise
|
||||
// price it (wrongly) at the input/output token rate.
|
||||
{
|
||||
serviceSpeed: servedServiceSpeed({ service_tier }),
|
||||
...(webSearchCalls
|
||||
? {
|
||||
costOverrides: {
|
||||
web_search_calls:
|
||||
webSearchCalls *
|
||||
(webSearchCallRate(
|
||||
tools,
|
||||
modelUsed.id,
|
||||
) ?? 0),
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
},
|
||||
webSearchCalls
|
||||
? {
|
||||
costOverrides: {
|
||||
web_search_calls:
|
||||
webSearchCalls *
|
||||
(webSearchCallRate(tools, modelUsed.id) ??
|
||||
0),
|
||||
},
|
||||
}
|
||||
: {},
|
||||
);
|
||||
setUsageCosts?.(metered.costs);
|
||||
setUsageDetails?.({
|
||||
...(requestedServiceSpeed !== undefined
|
||||
? { serviceSpeed: servedServiceSpeed({ service_tier }) }
|
||||
: {}),
|
||||
...OpenAiUtil.usageDetailsFromTrackedUsage(metered.usage, {
|
||||
writeTtl: '30m',
|
||||
})!,
|
||||
});
|
||||
return metered.usage;
|
||||
},
|
||||
stream,
|
||||
|
||||
@@ -23,8 +23,6 @@ import type { IChatModel } from '../../types.js';
|
||||
|
||||
// Prompts over 272K input tokens are billed at 2x input (cached reads and
|
||||
// cache writes included) and 1.5x output for the full request.
|
||||
// GPT-5.6 and later bill cache writes by the lifetime requested: 1.25x input
|
||||
// for the default 30m entries, 2.5x for 12h entries.
|
||||
const GPT_LONG_CONTEXT_PRICING = {
|
||||
threshold: 272_000,
|
||||
input_multiplier: 2,
|
||||
@@ -46,13 +44,11 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 100,
|
||||
cached_tokens: 5,
|
||||
cache_write_tokens: 100 * 1.25,
|
||||
cache_write_12h_tokens: 100 * 2.5,
|
||||
completion_tokens: 500,
|
||||
},
|
||||
long_context_pricing: GPT_LONG_CONTEXT_PRICING,
|
||||
@@ -73,13 +69,11 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2, ultrafast: 6 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
cached_tokens: 10,
|
||||
cache_write_tokens: 200 * 1.25,
|
||||
cache_write_12h_tokens: 200 * 2.5,
|
||||
completion_tokens: 1000,
|
||||
},
|
||||
long_context_pricing: GPT_LONG_CONTEXT_PRICING,
|
||||
@@ -100,13 +94,11 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
cached_tokens: 20,
|
||||
cache_write_tokens: 200 * 1.25,
|
||||
cache_write_12h_tokens: 200 * 2.5,
|
||||
completion_tokens: 1000,
|
||||
},
|
||||
long_context_pricing: GPT_LONG_CONTEXT_PRICING,
|
||||
@@ -127,13 +119,11 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 10,
|
||||
cached_tokens: 1,
|
||||
cache_write_tokens: 10 * 1.25,
|
||||
cache_write_12h_tokens: 10 * 2.5,
|
||||
completion_tokens: 50,
|
||||
},
|
||||
long_context_pricing: GPT_LONG_CONTEXT_PRICING,
|
||||
@@ -154,13 +144,11 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2, ultrafast: 6 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 1000,
|
||||
cached_tokens: 100,
|
||||
cache_write_tokens: 1000 * 1.25,
|
||||
cache_write_12h_tokens: 1000 * 2.5,
|
||||
completion_tokens: 5000,
|
||||
},
|
||||
long_context_pricing: GPT_LONG_CONTEXT_PRICING,
|
||||
@@ -181,13 +169,11 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
// OpenAI's promotional pricing, guaranteed only through 2026-11-21.
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 400,
|
||||
cached_tokens: 40,
|
||||
cache_write_tokens: 400 * 1.25,
|
||||
cache_write_12h_tokens: 400 * 2.5,
|
||||
completion_tokens: 2000,
|
||||
},
|
||||
long_context_pricing: GPT_LONG_CONTEXT_PRICING,
|
||||
@@ -207,13 +193,11 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
cached_tokens: 20,
|
||||
cache_write_tokens: 200 * 1.25,
|
||||
cache_write_12h_tokens: 200 * 2.5,
|
||||
completion_tokens: 1200,
|
||||
},
|
||||
long_context_pricing: GPT_LONG_CONTEXT_PRICING,
|
||||
@@ -233,13 +217,11 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 20,
|
||||
cached_tokens: 2,
|
||||
cache_write_tokens: 20 * 1.25,
|
||||
cache_write_12h_tokens: 20 * 2.5,
|
||||
completion_tokens: 120,
|
||||
},
|
||||
long_context_pricing: GPT_LONG_CONTEXT_PRICING,
|
||||
@@ -259,13 +241,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 1250,
|
||||
cached_tokens: 125,
|
||||
completion_tokens: 7500,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 500,
|
||||
@@ -310,13 +285,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 500,
|
||||
cached_tokens: 50,
|
||||
completion_tokens: 3000,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
@@ -360,13 +328,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 150,
|
||||
cached_tokens: 15,
|
||||
completion_tokens: 900,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 75,
|
||||
@@ -454,13 +415,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 350,
|
||||
cached_tokens: 35,
|
||||
completion_tokens: 2800,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 175,
|
||||
@@ -482,13 +436,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 250,
|
||||
cached_tokens: 25,
|
||||
completion_tokens: 2000,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
@@ -510,13 +457,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 425,
|
||||
cached_tokens: 212.5,
|
||||
completion_tokens: 1700,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
@@ -540,13 +480,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 25,
|
||||
cached_tokens: 12.5,
|
||||
completion_tokens: 100,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 15,
|
||||
@@ -566,13 +499,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 350,
|
||||
cached_tokens: 87.5,
|
||||
completion_tokens: 1400,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -594,13 +520,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 70,
|
||||
cached_tokens: 17.5,
|
||||
completion_tokens: 280,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 40,
|
||||
@@ -642,13 +561,6 @@ export const OPEN_AI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: {
|
||||
fast: {
|
||||
prompt_tokens: 425,
|
||||
cached_tokens: 212.5,
|
||||
completion_tokens: 1700,
|
||||
},
|
||||
},
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 250,
|
||||
|
||||
@@ -20,14 +20,15 @@
|
||||
/**
|
||||
* Offline unit tests for OpenRouterProvider.
|
||||
*
|
||||
* Boots a real PuterServer (in-memory sqlite + dynamo + s3 + mock redis) and
|
||||
* constructs OpenRouterProvider directly against the live wired
|
||||
* `MeteringService` so the recording side is exercised end-to- end. OpenRouter
|
||||
* is OpenAI-compatible, so the OpenAI SDK is mocked at the module boundary; the
|
||||
* model catalog is fetched via `axios` which is mocked at its module boundary
|
||||
* too. Both are the real network egress points. The companion integration test
|
||||
* (OpenRouterProvider.integration.test.ts) exercises the real OpenRouter
|
||||
* endpoint.
|
||||
* Boots a real PuterServer (in-memory sqlite + dynamo + s3 + mock
|
||||
* redis) and constructs OpenRouterProvider directly against the live
|
||||
* wired `MeteringService` so the recording side is exercised end-to-
|
||||
* end. OpenRouter is OpenAI-compatible, so the OpenAI SDK is mocked
|
||||
* at the module boundary; the model catalog is fetched via `axios`
|
||||
* which is mocked at its module boundary too. Both are the real
|
||||
* network egress points.
|
||||
* The companion integration test (OpenRouterProvider.integration.test.ts)
|
||||
* exercises the real OpenRouter endpoint.
|
||||
*/
|
||||
|
||||
import { Writable } from 'node:stream';
|
||||
@@ -299,20 +300,16 @@ describe('OpenRouterProvider model catalog', () => {
|
||||
it('coerces string prices and normalizes conditional tiers', async () => {
|
||||
const { provider } = makeProvider();
|
||||
const models = await provider.models();
|
||||
const sol = models.find(
|
||||
(m) => m.id === 'openrouter:openai/gpt-6.1-sol',
|
||||
)!;
|
||||
const sol = models.find((m) => m.id === 'openrouter:openai/gpt-6.1-sol')!;
|
||||
expect(sol.costs).toEqual({
|
||||
tokens: 1_000_000,
|
||||
prompt: 200,
|
||||
completion: 1000,
|
||||
});
|
||||
expect(sol.pricingOverrides).toEqual([
|
||||
{
|
||||
minPromptTokens: 272000,
|
||||
costs: { prompt: 400, completion: 1500 },
|
||||
},
|
||||
]);
|
||||
expect(sol.pricingOverrides).toEqual([{
|
||||
minPromptTokens: 272000,
|
||||
costs: { prompt: 400, completion: 1500 },
|
||||
}]);
|
||||
});
|
||||
|
||||
it('caches the coerced model list after the first axios round-trip', async () => {
|
||||
@@ -371,12 +368,9 @@ describe('OpenRouterProvider model catalog', () => {
|
||||
|
||||
it('does not reinterpret catalog-added timestamps as release dates', async () => {
|
||||
const { provider } = makeProvider();
|
||||
expect(
|
||||
(await provider.models()).every(
|
||||
(model) => model.release_date === undefined,
|
||||
),
|
||||
).toBe(true);
|
||||
expect((await provider.models()).every(model => model.release_date === undefined)).toBe(true);
|
||||
});
|
||||
|
||||
});
|
||||
|
||||
// ── Disallowed model gate ───────────────────────────────────────────
|
||||
@@ -874,14 +868,11 @@ describe('OpenRouterProvider billed cost through the driver', () => {
|
||||
event.factor = factor;
|
||||
};
|
||||
server.clients.event.on('ai.cost.factor.*', listener as never);
|
||||
return () =>
|
||||
server.clients.event.off('ai.cost.factor.*', listener as never);
|
||||
return () => server.clients.event.off('ai.cost.factor.*', listener as never);
|
||||
};
|
||||
|
||||
/** Μ¢ the ledger was handed for every usage recorded so far. */
|
||||
const ledgerTotal = (
|
||||
spy: MockInstance<MeteringService['batchIncrementUsages']>,
|
||||
) =>
|
||||
/** µ¢ the ledger was handed for every usage recorded so far. */
|
||||
const ledgerTotal = (spy: MockInstance<MeteringService['batchIncrementUsages']>) =>
|
||||
spy.mock.calls
|
||||
.flatMap(([, usages]) => usages)
|
||||
.reduce((total, usage) => total + (usage.costOverride ?? 0), 0);
|
||||
@@ -937,10 +928,7 @@ describe('OpenRouterProvider billed cost through the driver', () => {
|
||||
it('reports what a billed completion was charged, after the cost factor', async () => {
|
||||
const driver = await makeDriver();
|
||||
const stopFactor = withCostFactor(1.6);
|
||||
const recorded = vi.spyOn(
|
||||
server.services.metering,
|
||||
'batchIncrementUsages',
|
||||
);
|
||||
const recorded = vi.spyOn(server.services.metering, 'batchIncrementUsages');
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
@@ -1035,10 +1023,7 @@ describe('OpenRouterProvider billed cost through the driver', () => {
|
||||
it('ends a stream with what it was charged at the catalog rates, after the cost factor', async () => {
|
||||
const driver = await makeDriver();
|
||||
const stopFactor = withCostFactor(1.6);
|
||||
const recorded = vi.spyOn(
|
||||
server.services.metering,
|
||||
'batchIncrementUsages',
|
||||
);
|
||||
const recorded = vi.spyOn(server.services.metering, 'batchIncrementUsages');
|
||||
createMock.mockReturnValueOnce(
|
||||
asAsyncIterable([
|
||||
{ choices: [{ delta: { content: 'hello' } }] },
|
||||
@@ -1141,180 +1126,31 @@ describe('OpenRouterProvider.checkModeration', () => {
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
describe('OpenRouter catalog refresh validation', () => {
|
||||
it('keeps missing model fields and prices unknown and blocks unpriced inference', async () => {
|
||||
axiosRequestMock.mockResolvedValue({
|
||||
data: { data: [{ id: 'fixture/model', name: 'Fixture' }] },
|
||||
});
|
||||
axiosRequestMock.mockResolvedValue({ data: { data: [{ id: 'fixture/model', name: 'Fixture' }] } });
|
||||
const { provider } = makeProvider();
|
||||
const [model] = await provider.models();
|
||||
expect(model.costs).toEqual({ tokens: 1_000_000 });
|
||||
expect(model.context).toBeUndefined();
|
||||
expect(model.max_tokens).toBe(0);
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
provider.complete({
|
||||
model: 'openrouter:fixture/model',
|
||||
messages: [],
|
||||
}),
|
||||
),
|
||||
).rejects.toMatchObject({ legacyCode: 'model_pricing_unavailable' });
|
||||
await expect(withTestActor(() => provider.complete({ model: 'openrouter:fixture/model', messages: [] })))
|
||||
.rejects.toMatchObject({ legacyCode: 'model_pricing_unavailable' });
|
||||
expect(createMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each(['malformed', 'failed', 'ambiguous conditional price'])(
|
||||
'retains last-good data after a %s refresh',
|
||||
async (failure) => {
|
||||
vi.useFakeTimers({ toFake: ['Date'] });
|
||||
vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
const { provider } = makeProvider();
|
||||
const good = await provider.models();
|
||||
vi.advanceTimersByTime(15 * 60 * 1000 + 1);
|
||||
if (failure === 'failed')
|
||||
axiosRequestMock.mockRejectedValueOnce(
|
||||
new Error('unavailable'),
|
||||
);
|
||||
else if (failure === 'malformed')
|
||||
axiosRequestMock.mockResolvedValueOnce({ data: {} });
|
||||
else
|
||||
axiosRequestMock.mockResolvedValueOnce({
|
||||
data: {
|
||||
data: [
|
||||
{
|
||||
...SAMPLE_API_MODELS[0],
|
||||
pricing: {
|
||||
prompt: '0.000002',
|
||||
overrides: [
|
||||
{ min_prompt_tokens: 10, prompt: '' },
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
expect(await provider.models()).toBe(good);
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe('OpenRouter service speed offerings', () => {
|
||||
it('loads and caches tier prices while reserving for endpoint and context differences', async () => {
|
||||
it.each(['malformed', 'failed', 'ambiguous conditional price'])('retains last-good data after a %s refresh', async failure => {
|
||||
vi.useFakeTimers({ toFake: ['Date'] });
|
||||
vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
const { provider } = makeProvider();
|
||||
const base = {
|
||||
...(await provider.models())[0],
|
||||
provider: 'openrouter',
|
||||
};
|
||||
axiosRequestMock.mockClear();
|
||||
axiosRequestMock.mockResolvedValueOnce({
|
||||
data: {
|
||||
data: {
|
||||
endpoints: [
|
||||
{
|
||||
service_tier: 'priority',
|
||||
pricing: {
|
||||
prompt: '0.000004',
|
||||
completion: '0.00002',
|
||||
input_cache_read: '0.0000002',
|
||||
},
|
||||
},
|
||||
{
|
||||
service_tier: 'priority',
|
||||
pricing: {
|
||||
prompt: '0.000005',
|
||||
completion: '0.00001',
|
||||
},
|
||||
},
|
||||
{
|
||||
service_tier: 'ultrafast',
|
||||
pricing: {
|
||||
prompt: '0.000012',
|
||||
completion: '0.00006',
|
||||
overrides: [
|
||||
{
|
||||
min_prompt_tokens: 272000,
|
||||
prompt: '0.000024',
|
||||
completion: '0.00009',
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
{
|
||||
service_tier: 'priority',
|
||||
pricing: { prompt: 'invalid' },
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
});
|
||||
const offers = await provider.serviceSpeedModels(base);
|
||||
expect(offers.fast?.costs).toMatchObject({
|
||||
prompt: 500,
|
||||
completion: 2000,
|
||||
input_cache_read: 20,
|
||||
});
|
||||
expect(offers.ultrafast?.costs).toMatchObject({
|
||||
prompt: 2400,
|
||||
completion: 9000,
|
||||
});
|
||||
expect(await provider.serviceSpeedModels(base)).toBe(offers);
|
||||
expect(axiosRequestMock).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('uses standard when the tier catalog cannot be fetched', async () => {
|
||||
const { provider } = makeProvider();
|
||||
const base = {
|
||||
...(await provider.models())[0],
|
||||
provider: 'openrouter',
|
||||
};
|
||||
axiosRequestMock.mockRejectedValueOnce(new Error('Unavailable'));
|
||||
expect(await provider.serviceSpeedModels(base)).toEqual({});
|
||||
});
|
||||
});
|
||||
|
||||
describe('OpenRouter speed billing', () => {
|
||||
it('requires an authoritative bill instead of charging the reservation ceiling', async () => {
|
||||
const { selectServiceSpeed } =
|
||||
await import('../../utils/serviceSpeed.js');
|
||||
const { provider } = makeProvider();
|
||||
const base = {
|
||||
...(await provider.models())[0],
|
||||
provider: 'openrouter',
|
||||
};
|
||||
const selected = selectServiceSpeed(
|
||||
base,
|
||||
{
|
||||
model: base.id,
|
||||
messages: [{ role: 'user', content: 'Hello' }],
|
||||
serviceSpeed: 'fast',
|
||||
},
|
||||
() => undefined,
|
||||
{
|
||||
fast: {
|
||||
...base,
|
||||
costs: { ...base.costs, prompt: 500, completion: 2000 },
|
||||
},
|
||||
},
|
||||
);
|
||||
const completion = {
|
||||
choices: [
|
||||
{
|
||||
index: 0,
|
||||
message: { role: 'assistant', content: 'OK' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
usage: { prompt_tokens: 10, completion_tokens: 1 },
|
||||
};
|
||||
createMock.mockResolvedValueOnce({
|
||||
...completion,
|
||||
service_tier: 'priority',
|
||||
usage: { ...completion.usage, cost: undefined },
|
||||
});
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
provider.complete(selected.args, selected.model),
|
||||
),
|
||||
).rejects.toMatchObject({ legacyCode: 'model_pricing_unavailable' });
|
||||
expect(recordSpy).not.toHaveBeenCalled();
|
||||
const good = await provider.models();
|
||||
vi.advanceTimersByTime(15 * 60 * 1000 + 1);
|
||||
if (failure === 'failed') axiosRequestMock.mockRejectedValueOnce(new Error('unavailable'));
|
||||
else if (failure === 'malformed') axiosRequestMock.mockResolvedValueOnce({ data: {} });
|
||||
else axiosRequestMock.mockResolvedValueOnce({ data: { data: [{ ...SAMPLE_API_MODELS[0], pricing: {
|
||||
prompt: '0.000002', overrides: [{ min_prompt_tokens: 10, prompt: '' }],
|
||||
} }] } });
|
||||
expect(await provider.models()).toBe(good);
|
||||
});
|
||||
});
|
||||
@@ -22,10 +22,8 @@ import { OpenAI } from 'openai';
|
||||
import { HttpError, isHttpError } from '../../../../core/http/HttpError.js';
|
||||
import { UpstreamResponseError } from '../../../util/upstreamErrors.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
import type { IChatModel, ICompleteArguments, ModelCost } from '../../types.js';
|
||||
import { BoundedTtlMap } from '../../../../util/boundedTtlMap.js';
|
||||
import type { IChatModel, ICompleteArguments } from '../../types.js';
|
||||
import { cachedRemoteCatalog } from '../../utils/cachedRemoteCatalog.js';
|
||||
import { servedServiceSpeed } from '../../utils/serviceSpeed.js';
|
||||
import { catalogPrice } from '../../utils/pricing.js';
|
||||
import { mapOpenRouterPricing } from './pricing.js';
|
||||
import { sdkClientOptions } from '../../utils/sdkClient.js';
|
||||
@@ -66,75 +64,6 @@ const isPolicyBlock = (err: unknown): boolean => {
|
||||
export class OpenRouterProvider extends OpenAICompatProvider {
|
||||
#apiBaseUrl: string;
|
||||
|
||||
#speedCatalogs = new BoundedTtlMap<
|
||||
string,
|
||||
() => Promise<Partial<Record<'fast' | 'ultrafast', IChatModel>>>
|
||||
>({ maxEntries: 256 });
|
||||
|
||||
async serviceSpeedModels(model: IChatModel) {
|
||||
let load = this.#speedCatalogs.get(model.id);
|
||||
if (!load) {
|
||||
load = cachedRemoteCatalog({
|
||||
name: 'OpenRouter speed catalog',
|
||||
fallback: {} as Partial<
|
||||
Record<'fast' | 'ultrafast', IChatModel>
|
||||
>,
|
||||
fetch: async (signal) => {
|
||||
const id = model.id.replace(/^openrouter:/, '');
|
||||
const response = await axios.request({
|
||||
url: `${this.#apiBaseUrl}/models/${id}/endpoints`,
|
||||
method: 'GET',
|
||||
signal,
|
||||
maxContentLength: 2 * 1024 * 1024,
|
||||
});
|
||||
const offers: Partial<
|
||||
Record<'fast' | 'ultrafast', IChatModel>
|
||||
> = {};
|
||||
for (const endpoint of response.data?.data?.endpoints ??
|
||||
[]) {
|
||||
const speed =
|
||||
endpoint.service_tier === 'ultrafast'
|
||||
? 'ultrafast'
|
||||
: ['priority', 'fast'].includes(
|
||||
endpoint.service_tier,
|
||||
)
|
||||
? 'fast'
|
||||
: undefined;
|
||||
if (!speed || !endpoint.pricing) continue;
|
||||
const pricing = mapOpenRouterPricing(endpoint.pricing);
|
||||
if (
|
||||
pricing.costs.prompt === undefined ||
|
||||
pricing.costs.completion === undefined
|
||||
)
|
||||
continue;
|
||||
// Reserve for the most expensive eligible endpoint and context tier.
|
||||
const costs: ModelCost = {
|
||||
...offers[speed]?.costs,
|
||||
tokens: model.costs.tokens,
|
||||
};
|
||||
for (const rates of [
|
||||
pricing.costs,
|
||||
...pricing.overrides.map(
|
||||
(override) => override.costs,
|
||||
),
|
||||
]) {
|
||||
for (const [key, value] of Object.entries(rates))
|
||||
costs[key] = Math.max(costs[key] ?? 0, value);
|
||||
}
|
||||
offers[speed] = {
|
||||
...model,
|
||||
costs,
|
||||
pricingOverrides: [],
|
||||
};
|
||||
}
|
||||
return offers;
|
||||
},
|
||||
});
|
||||
this.#speedCatalogs.set(model.id, load);
|
||||
}
|
||||
return load();
|
||||
}
|
||||
|
||||
constructor(config: ChatProviderConfig, meteringService: MeteringService) {
|
||||
const apiBaseUrl = config.apiBaseUrl || 'https://openrouter.ai/api/v1';
|
||||
super(meteringService, {
|
||||
@@ -273,17 +202,8 @@ export class OpenRouterProvider extends OpenAICompatProvider {
|
||||
return { ...params, usage: { include: true } };
|
||||
}
|
||||
|
||||
protected override meteredUsage(source: UsageSource, model: IChatModel) {
|
||||
protected override meteredUsage(source: UsageSource) {
|
||||
const usage = source.usage as OpenrouterUsage;
|
||||
if (
|
||||
model.serviceSpeedPricing &&
|
||||
servedServiceSpeed(source) !== 'standard' &&
|
||||
catalogPrice(usage.cost) === undefined
|
||||
) {
|
||||
throw new HttpError(503, 'Service tier billing is unavailable', {
|
||||
legacyCode: 'model_pricing_unavailable',
|
||||
});
|
||||
}
|
||||
return {
|
||||
usage: {
|
||||
prompt:
|
||||
|
||||
@@ -20,12 +20,13 @@
|
||||
/**
|
||||
* Offline unit tests for XAIProvider.
|
||||
*
|
||||
* Boots a real PuterServer (in-memory sqlite + dynamo + s3 + mock redis) and
|
||||
* constructs XAIProvider directly against the live wired `MeteringService` so
|
||||
* the recording side is exercised end-to-end. xAI is OpenAI-compatible so the
|
||||
* OpenAI SDK is mocked at the module boundary; that's the real network egress
|
||||
* point. The companion integration test (XAIProvider.integration.test.ts)
|
||||
* exercises the real xAI endpoint.
|
||||
* Boots a real PuterServer (in-memory sqlite + dynamo + s3 + mock
|
||||
* redis) and constructs XAIProvider directly against the live wired
|
||||
* `MeteringService` so the recording side is exercised end-to-end.
|
||||
* xAI is OpenAI-compatible so the OpenAI SDK is mocked at the module
|
||||
* boundary; that's the real network egress point. The companion
|
||||
* integration test (XAIProvider.integration.test.ts) exercises the
|
||||
* real xAI endpoint.
|
||||
*/
|
||||
|
||||
import { Writable } from 'node:stream';
|
||||
@@ -932,7 +933,8 @@ describe('XAIProvider.complete non-stream output', () => {
|
||||
// namespace, actor, and a costsOverride priced from
|
||||
// models.ts (grok-4.6: prompt=200, completion=600, cached=50).
|
||||
expect(recordSpy).toHaveBeenCalledTimes(1);
|
||||
const [usage, actor, prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
const [usage, actor, prefix, overrides] =
|
||||
recordSpy.mock.calls[0]!;
|
||||
expect(usage).toEqual({
|
||||
prompt_tokens: 90,
|
||||
completion_tokens: 50,
|
||||
@@ -1020,7 +1022,8 @@ describe('XAIProvider.complete non-stream output', () => {
|
||||
}),
|
||||
);
|
||||
|
||||
const [usage, , , overrides] = recordSpy.mock.calls[0]!;
|
||||
const [usage, , , overrides] =
|
||||
recordSpy.mock.calls[0]!;
|
||||
expect(usage.cached_tokens).toBe(0);
|
||||
// 0 cached tokens × any rate = 0 metered cost.
|
||||
expect(overrides).toMatchObject({ cached_tokens: 0 });
|
||||
@@ -1078,7 +1081,8 @@ describe('XAIProvider.complete streaming', () => {
|
||||
|
||||
// grok-build-0.1 costs: prompt=100, completion=200, cached=20.
|
||||
expect(recordSpy).toHaveBeenCalledTimes(1);
|
||||
const [, , prefix, overrides] = recordSpy.mock.calls[0]!;
|
||||
const [, , prefix, overrides] =
|
||||
recordSpy.mock.calls[0]!;
|
||||
expect(prefix).toBe('xai:grok-build-0.1');
|
||||
expect(overrides).toEqual({
|
||||
prompt_tokens: 3 * 100,
|
||||
@@ -1196,60 +1200,3 @@ describe('XAIProvider.checkModeration', () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('xAI service speed billing', () => {
|
||||
it('records the reported Priority charge with cached input and reasoning', async () => {
|
||||
const { selectServiceSpeed } =
|
||||
await import('../../utils/serviceSpeed.js');
|
||||
const { provider } = makeProvider();
|
||||
const base = {
|
||||
...(await provider.models()).find(
|
||||
(entry) => entry.id === 'grok-4.7',
|
||||
)!,
|
||||
provider: 'xai',
|
||||
};
|
||||
const selected = selectServiceSpeed(
|
||||
base,
|
||||
{
|
||||
model: base.id,
|
||||
messages: [{ role: 'user', content: 'Hello' }],
|
||||
serviceSpeed: 'ultrafast',
|
||||
},
|
||||
() => undefined,
|
||||
);
|
||||
createMock.mockResolvedValueOnce({
|
||||
choices: [
|
||||
{
|
||||
index: 0,
|
||||
message: { role: 'assistant', content: 'OK' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
service_tier: 'priority',
|
||||
usage: {
|
||||
prompt_tokens: 1247,
|
||||
completion_tokens: 1,
|
||||
total_tokens: 1282,
|
||||
prompt_tokens_details: { cached_tokens: 1152 },
|
||||
completion_tokens_details: { reasoning_tokens: 34 },
|
||||
cost_in_usd_ticks: 19520000,
|
||||
},
|
||||
});
|
||||
const result = await withTestActor(() =>
|
||||
provider.complete(selected.args, selected.model),
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
usageDetails: {
|
||||
serviceSpeed: 'fast',
|
||||
outputTokens: 35,
|
||||
reasoningTokens: 34,
|
||||
},
|
||||
});
|
||||
expect(recordSpy).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ billedUsage: 1, completion_tokens: 35 }),
|
||||
expect.anything(),
|
||||
'xai:grok-4.7',
|
||||
expect.objectContaining({ billedUsage: 195200 }),
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -19,8 +19,6 @@
|
||||
|
||||
import { OpenAI } from 'openai';
|
||||
import { ChatCompletionCreateParams } from 'openai/resources/index.js';
|
||||
import { catalogPrice } from '../../utils/pricing.js';
|
||||
import { servedServiceSpeed } from '../../utils/serviceSpeed.js';
|
||||
import { Context } from '../../../../core/context.js';
|
||||
import type { MeteringService } from '../../../../services/metering/MeteringService.js';
|
||||
import * as OpenAIUtil from '../../utils/OpenAIUtil.js';
|
||||
@@ -76,15 +74,7 @@ export class XAIProvider implements IChatProvider {
|
||||
}
|
||||
|
||||
async complete(
|
||||
{
|
||||
messages,
|
||||
stream,
|
||||
model,
|
||||
tools,
|
||||
max_tokens,
|
||||
service_tier,
|
||||
serviceSpeed: requestedServiceSpeed,
|
||||
}: ICompleteArguments,
|
||||
{ messages, stream, model, tools, max_tokens }: ICompleteArguments,
|
||||
resolved?: IChatModel,
|
||||
): Promise<IChatCompleteResult> {
|
||||
const actor = Context.get('actor');
|
||||
@@ -98,7 +88,6 @@ export class XAIProvider implements IChatProvider {
|
||||
{
|
||||
messages,
|
||||
model: modelUsed.id,
|
||||
...(service_tier !== undefined ? { service_tier } : {}),
|
||||
...(tools ? { tools } : {}),
|
||||
...(max_tokens !== undefined ? { max_tokens } : {}),
|
||||
stream,
|
||||
@@ -118,49 +107,18 @@ export class XAIProvider implements IChatProvider {
|
||||
}
|
||||
|
||||
return OpenAIUtil.handle_completion_output({
|
||||
usage_calculator: ({
|
||||
usage,
|
||||
service_tier: servedTier,
|
||||
setUsageCosts,
|
||||
setUsageDetails,
|
||||
}) => {
|
||||
const serviceSpeed = servedServiceSpeed({
|
||||
service_tier: servedTier,
|
||||
});
|
||||
usage_calculator: ({ usage, setUsageCosts }) => {
|
||||
// xAI reports cached reads inside `prompt_tokens`; bill them
|
||||
// only at the cached rate.
|
||||
const metered = OpenAIUtil.extractMeteredUsage(usage);
|
||||
const reasoningTokens =
|
||||
usage.completion_tokens_details?.reasoning_tokens ?? 0;
|
||||
const trackedUsage = OpenAIUtil.splitCachedPrompt(usage);
|
||||
trackedUsage.completion_tokens += reasoningTokens;
|
||||
const reportedTicks = catalogPrice(
|
||||
(usage as typeof usage & { cost_in_usd_ticks?: number })
|
||||
.cost_in_usd_ticks,
|
||||
);
|
||||
const { costs } = meterChatUsage(
|
||||
this.#meteringService,
|
||||
actor,
|
||||
this.meteringModelKey(modelUsed.id),
|
||||
modelUsed,
|
||||
trackedUsage,
|
||||
{
|
||||
serviceSpeed,
|
||||
authoritativeUsd:
|
||||
reportedTicks === undefined
|
||||
? undefined
|
||||
: reportedTicks / 10_000_000_000,
|
||||
},
|
||||
OpenAIUtil.splitCachedPrompt(usage),
|
||||
);
|
||||
setUsageCosts(costs);
|
||||
if (requestedServiceSpeed !== undefined)
|
||||
setUsageDetails?.({
|
||||
...OpenAIUtil.usageDetailsFromTrackedUsage(
|
||||
trackedUsage,
|
||||
)!,
|
||||
serviceSpeed,
|
||||
...(reasoningTokens ? { reasoningTokens } : {}),
|
||||
});
|
||||
return metered;
|
||||
},
|
||||
stream,
|
||||
|
||||
@@ -42,7 +42,6 @@ export const XAI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -66,7 +65,6 @@ export const XAI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -89,7 +87,6 @@ export const XAI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
@@ -112,7 +109,6 @@ export const XAI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
@@ -155,7 +151,6 @@ export const XAI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
@@ -191,7 +186,6 @@ export const XAI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
@@ -223,7 +217,6 @@ export const XAI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 125,
|
||||
@@ -253,7 +246,6 @@ export const XAI_MODELS: IChatModel[] = [
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
serviceSpeedRates: { fast: 2 },
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 100,
|
||||
|
||||
@@ -26,8 +26,6 @@
|
||||
|
||||
export type ModelCost = Record<string, number>;
|
||||
|
||||
export type ServiceSpeed = 'standard' | 'fast' | 'ultrafast';
|
||||
|
||||
export interface ModelModalities {
|
||||
input: string[];
|
||||
output: string[];
|
||||
@@ -69,22 +67,6 @@ export interface IChatModel<T extends ModelCost = ModelCost> extends Record<
|
||||
output_multiplier: number;
|
||||
};
|
||||
pricingOverrides?: ModelPricingOverride[];
|
||||
/**
|
||||
* A token-rate multiplier or explicit per-key rates merged with standard
|
||||
* costs.
|
||||
*/
|
||||
serviceSpeedRates?: Partial<
|
||||
Record<'fast' | 'ultrafast', number | ModelCost>
|
||||
>;
|
||||
serviceSpeedVariants?: { fast?: string };
|
||||
/** Request-local pricing snapshot; never part of the model catalog. */
|
||||
serviceSpeedPricing?: {
|
||||
speed: ServiceSpeed;
|
||||
standardCosts: ModelCost;
|
||||
standardOverrides?: ModelPricingOverride[];
|
||||
fastCosts?: ModelCost;
|
||||
ultrafastCosts?: ModelCost;
|
||||
};
|
||||
context?: number;
|
||||
max_tokens: number;
|
||||
subscriberOnly?: boolean;
|
||||
@@ -94,8 +76,6 @@ export interface IChatModel<T extends ModelCost = ModelCost> extends Record<
|
||||
tool_call?: boolean;
|
||||
responses_api?: boolean;
|
||||
responses_api_only?: boolean;
|
||||
/** Restricts service speeds for deployments with a limited API dialect. */
|
||||
serviceSpeeds?: ServiceSpeed[];
|
||||
/**
|
||||
* Omitted preserves sampling; reasoningDisabled requires explicit effort
|
||||
* none.
|
||||
@@ -165,25 +145,18 @@ export type StopReason =
|
||||
| 'model_context_window_exceeded'
|
||||
| (string & {});
|
||||
|
||||
/**
|
||||
* Provider-neutral usage breakdown. `inputTokens` excludes cache reads/writes.
|
||||
* Cache writes are keyed by the lifetime the entry was written with: Claude and
|
||||
* Moonshot write 5m and 1h entries, OpenAI writes 30m and 12h entries.
|
||||
*/
|
||||
/** Provider-neutral usage breakdown. `inputTokens` excludes cache reads/writes. */
|
||||
export interface UsageDetails {
|
||||
inputTokens: number;
|
||||
outputTokens: number; // includes reasoning
|
||||
cacheReadTokens?: number;
|
||||
cacheWrite5mTokens?: number;
|
||||
cacheWrite30mTokens?: number;
|
||||
cacheWrite1hTokens?: number;
|
||||
cacheWrite12hTokens?: number;
|
||||
reasoningTokens?: number;
|
||||
webSearchRequests?: number;
|
||||
webFetchRequests?: number;
|
||||
speed?: 'fast' | 'standard';
|
||||
serviceTier?: string;
|
||||
serviceSpeed?: ServiceSpeed;
|
||||
/**
|
||||
* Per-pass breakdown when the upstream ran several passes (compaction,
|
||||
* advisor).
|
||||
@@ -195,22 +168,10 @@ export interface UsageDetails {
|
||||
outputTokens: number;
|
||||
cacheReadTokens?: number;
|
||||
cacheWrite5mTokens?: number;
|
||||
cacheWrite30mTokens?: number;
|
||||
cacheWrite1hTokens?: number;
|
||||
cacheWrite12hTokens?: number;
|
||||
}>;
|
||||
}
|
||||
|
||||
/**
|
||||
* OpenAI `prompt_cache_options`, GPT-5.6 and later. `ttl` is the minimum
|
||||
* lifetime of every cache entry the request writes; `12h` entries cost twice
|
||||
* what `30m` entries do to write and only the Responses API writes them.
|
||||
*/
|
||||
export interface PromptCacheOptions {
|
||||
mode?: 'implicit' | 'explicit';
|
||||
ttl?: '30m' | '12h';
|
||||
}
|
||||
|
||||
export interface ICompleteArguments {
|
||||
messages: PuterMessage[];
|
||||
provider?: string;
|
||||
@@ -219,7 +180,6 @@ export interface ICompleteArguments {
|
||||
* request on another provider that serves the same model.
|
||||
*/
|
||||
fallback?: boolean;
|
||||
serviceSpeed?: ServiceSpeed;
|
||||
stream?: boolean;
|
||||
model: string;
|
||||
test_mode?: boolean;
|
||||
@@ -239,7 +199,10 @@ export interface ICompleteArguments {
|
||||
taskBudget?: { total: number; remaining?: number };
|
||||
/** Claude only: top-level automatic `cache_control`. */
|
||||
cacheControl?: CacheControl;
|
||||
/** Claude only; fast is forwarded when the catalog declares fast rates. */
|
||||
/**
|
||||
* Claude only; forwarded only when the model's catalog entry has
|
||||
* `fast_output_tokens`.
|
||||
*/
|
||||
speed?: 'fast' | 'standard';
|
||||
/** Claude only. */
|
||||
safeguards?: SafeguardRequest[];
|
||||
@@ -271,23 +234,13 @@ export interface ICompleteArguments {
|
||||
metadata?: Record<string, string>;
|
||||
prompt?: unknown;
|
||||
prompt_cache_key?: string;
|
||||
/** `in-memory` is accepted as a spelling of OpenAI's `in_memory`. */
|
||||
prompt_cache_retention?: 'in-memory' | 'in_memory' | '24h' | undefined;
|
||||
prompt_cache_options?: PromptCacheOptions;
|
||||
prompt_cache_retention?: 'in-memory' | '24h' | undefined;
|
||||
store?: boolean;
|
||||
top_p?: number;
|
||||
truncation?: 'auto' | 'disabled' | undefined;
|
||||
background?: boolean;
|
||||
service_tier?:
|
||||
| 'auto'
|
||||
| 'default'
|
||||
| 'flex'
|
||||
| 'scale'
|
||||
| 'priority'
|
||||
| 'fast'
|
||||
| 'ultrafast'
|
||||
| 'standard'
|
||||
| undefined;
|
||||
'auto' | 'default' | 'flex' | 'scale' | 'priority' | undefined;
|
||||
max_tokens?: number;
|
||||
temperature?: number;
|
||||
reasoning?: { effort: ReasoningEffort } | undefined;
|
||||
@@ -355,10 +308,6 @@ export interface IChatMessageResult {
|
||||
export type IChatCompleteResult = IChatStreamResult | IChatMessageResult;
|
||||
|
||||
export interface IChatProvider {
|
||||
/** Account/catalog-specific speed offerings, fetched before the credit hold. */
|
||||
serviceSpeedModels?(
|
||||
model: IChatModel,
|
||||
): Promise<Partial<Record<'fast' | 'ultrafast', IChatModel>>>;
|
||||
models(extra_params?: unknown): IChatModel[] | Promise<IChatModel[]>;
|
||||
getDefaultModel(): string;
|
||||
/**
|
||||
|
||||
@@ -481,86 +481,43 @@ export const extractMeteredUsage = (usage) => {
|
||||
};
|
||||
|
||||
/**
|
||||
* @typedef {{
|
||||
* cached_tokens?: number | null;
|
||||
* cache_write_tokens?: number | null;
|
||||
* cache_write_tokens_details?: {
|
||||
* ttl_30m_tokens?: number | null;
|
||||
* ttl_12h_tokens?: number | null;
|
||||
* } | null;
|
||||
* }} OpenAiCacheDetails
|
||||
*/
|
||||
|
||||
/**
|
||||
* OpenAI-shaped usage as it's metered. The prompt count reports cached reads
|
||||
* and cache writes inside it, so they come out of it and are priced once each
|
||||
* under their own keys: `cached_tokens` for reads, `cache_write_tokens` for
|
||||
* writes at the model's default lifetime and `cache_write_12h_tokens` for
|
||||
* writes at OpenAI's 12h lifetime. Without the per-lifetime breakdown every
|
||||
* write is a default-lifetime write: Chat Completions writes no 12h entries,
|
||||
* and Responses reports the breakdown whenever a request asked for them. Reads
|
||||
* the Chat Completions (`prompt_tokens`) and Responses (`input_tokens`)
|
||||
* spellings alike.
|
||||
* Chat Completions usage as it's metered: `prompt_tokens` reports cached reads
|
||||
* inside its count, so they come out of it and are priced only once, under
|
||||
* `cached_tokens`.
|
||||
*
|
||||
* @param {{
|
||||
* prompt_tokens?: number;
|
||||
* completion_tokens?: number;
|
||||
* prompt_tokens_details?: OpenAiCacheDetails | null;
|
||||
* input_tokens?: number;
|
||||
* output_tokens?: number;
|
||||
* input_tokens_details?: OpenAiCacheDetails | null;
|
||||
* }
|
||||
* | undefined} usage
|
||||
* @param {import('openai/resources/completions.mjs').CompletionUsage | undefined} usage
|
||||
*/
|
||||
export const splitCachedPrompt = (usage) => {
|
||||
const details = usage?.prompt_tokens_details ?? usage?.input_tokens_details;
|
||||
const details = usage?.prompt_tokens_details;
|
||||
const cachedTokens = details?.cached_tokens ?? 0;
|
||||
// Moonshot and GPT-5.6+ count the prompt tokens they wrote to the prefix
|
||||
// cache here; they're billed at the model's write rate, not the prompt rate.
|
||||
// Moonshot counts the prompt tokens it wrote to its prefix cache here;
|
||||
// they're billed at the model's write rate, not the plain prompt rate.
|
||||
const writeTokens =
|
||||
typeof details?.cache_write_tokens === 'number'
|
||||
? details.cache_write_tokens
|
||||
: undefined;
|
||||
const reported12h = details?.cache_write_tokens_details?.ttl_12h_tokens;
|
||||
const write12hTokens =
|
||||
typeof reported12h === 'number'
|
||||
? Math.min(reported12h, writeTokens ?? 0)
|
||||
: undefined;
|
||||
return {
|
||||
prompt_tokens:
|
||||
(usage?.prompt_tokens ?? usage?.input_tokens ?? 0) -
|
||||
cachedTokens -
|
||||
(writeTokens ?? 0),
|
||||
completion_tokens:
|
||||
usage?.completion_tokens ?? usage?.output_tokens ?? 0,
|
||||
(usage?.prompt_tokens ?? 0) - cachedTokens - (writeTokens ?? 0),
|
||||
completion_tokens: usage?.completion_tokens ?? 0,
|
||||
cached_tokens: cachedTokens,
|
||||
...(writeTokens !== undefined
|
||||
? { cache_write_tokens: writeTokens - (write12hTokens ?? 0) }
|
||||
: {}),
|
||||
...(write12hTokens !== undefined
|
||||
? { cache_write_12h_tokens: write12hTokens }
|
||||
? { cache_write_tokens: writeTokens }
|
||||
: {}),
|
||||
};
|
||||
};
|
||||
|
||||
/**
|
||||
* The `{prompt_tokens, completion_tokens, cached_tokens, cache_write_tokens,
|
||||
* cache_write_12h_tokens}` shape every OpenAI-family `usage_calculator` already
|
||||
* returns for billing, mapped to the provider-neutral `UsageDetails` the
|
||||
* end-of-stream/non-stream result carries. `inputTokens`/`outputTokens` are the
|
||||
* same numbers already billed — this never introduces a new figure, only
|
||||
* exposes the breakdown.
|
||||
* The `{prompt_tokens, completion_tokens, cached_tokens, cache_write_tokens}`
|
||||
* shape every OpenAI-family `usage_calculator` already returns for billing,
|
||||
* mapped to the provider-neutral `UsageDetails` the end-of-stream/non-stream
|
||||
* result carries. `inputTokens`/`outputTokens` are the same numbers already
|
||||
* billed — this never introduces a new figure, only exposes the breakdown.
|
||||
*
|
||||
* @param {Record<string, number> | undefined | null} trackedUsage
|
||||
* @param {{ writeTtl?: '5m' | '30m' }} [opts] The lifetime `cache_write_tokens`
|
||||
* were written with: 5m unless the provider says otherwise (OpenAI's default
|
||||
* entries live 30m).
|
||||
* @returns {import('../types.js').UsageDetails | undefined}
|
||||
*/
|
||||
export const usageDetailsFromTrackedUsage = (
|
||||
trackedUsage,
|
||||
{ writeTtl = '5m' } = {},
|
||||
) => {
|
||||
export const usageDetailsFromTrackedUsage = (trackedUsage) => {
|
||||
if (!trackedUsage || typeof trackedUsage !== 'object') return undefined;
|
||||
// OpenRouter, Infron and Ollama track `{prompt, completion,
|
||||
// input_cache_read}` instead; any other shape is left to the driver.
|
||||
@@ -569,19 +526,12 @@ export const usageDetailsFromTrackedUsage = (
|
||||
if (input === undefined && output === undefined) return undefined;
|
||||
const cacheRead =
|
||||
trackedUsage.cached_tokens ?? trackedUsage.input_cache_read;
|
||||
const cacheWrite = trackedUsage.cache_write_tokens;
|
||||
return {
|
||||
inputTokens: input ?? 0,
|
||||
outputTokens: output ?? 0,
|
||||
...(cacheRead ? { cacheReadTokens: cacheRead } : {}),
|
||||
...(cacheWrite && writeTtl === '30m'
|
||||
? { cacheWrite30mTokens: cacheWrite }
|
||||
: {}),
|
||||
...(cacheWrite && writeTtl !== '30m'
|
||||
? { cacheWrite5mTokens: cacheWrite }
|
||||
: {}),
|
||||
...(trackedUsage.cache_write_12h_tokens
|
||||
? { cacheWrite12hTokens: trackedUsage.cache_write_12h_tokens }
|
||||
...(trackedUsage.cache_write_tokens
|
||||
? { cacheWrite5mTokens: trackedUsage.cache_write_tokens }
|
||||
: {}),
|
||||
};
|
||||
};
|
||||
@@ -650,15 +600,12 @@ export const create_chat_stream_handler =
|
||||
block.end();
|
||||
};
|
||||
|
||||
let serviceTier;
|
||||
let last_usage = null;
|
||||
let last_extra_content = null;
|
||||
let finish_reason = null;
|
||||
for await (let chunk of completion) {
|
||||
await chatStream.waitForDrain();
|
||||
chunk = deviations.chunk_but_like_actually(chunk);
|
||||
if (chunk.service_tier !== undefined)
|
||||
serviceTier = chunk.service_tier;
|
||||
const chunk_usage = deviations.index_usage_from_stream_chunk(chunk);
|
||||
if (chunk_usage) last_usage = chunk_usage;
|
||||
if (chunk.choices.length < 1) continue;
|
||||
@@ -749,7 +696,6 @@ export const create_chat_stream_handler =
|
||||
const usage = last_usage
|
||||
? usage_calculator({
|
||||
usage: last_usage,
|
||||
service_tier: serviceTier,
|
||||
extra_content: last_extra_content,
|
||||
setUsageCosts: (c) => chatStream.setUsageCosts(c),
|
||||
setUsageDetails: (d) => {
|
||||
@@ -783,7 +729,6 @@ export const create_chat_stream_handler_responses_api =
|
||||
const message = chatStream.message();
|
||||
const textblock = message.contentBlock({ type: 'text' });
|
||||
|
||||
let serviceTier;
|
||||
let last_usage = null;
|
||||
let completed = false;
|
||||
let sawFunctionCall = false;
|
||||
@@ -792,8 +737,6 @@ export const create_chat_stream_handler_responses_api =
|
||||
let toolUsage = null;
|
||||
for await (const chunk of completion) {
|
||||
await chatStream.waitForDrain();
|
||||
if (chunk.response?.service_tier !== undefined)
|
||||
serviceTier = chunk.response.service_tier;
|
||||
if (chunk.type === 'response.failed' || chunk.type === 'error') {
|
||||
throw new Error(
|
||||
chunk.response?.error?.message ??
|
||||
@@ -921,17 +864,12 @@ export const create_chat_stream_handler_responses_api =
|
||||
// Missing usage is left undefined rather than fed to the calculator —
|
||||
// see the sibling handler above, including why usage is reported
|
||||
// before the block flushes.
|
||||
let usageDetails;
|
||||
const usage = last_usage
|
||||
? usage_calculator({
|
||||
usage: last_usage,
|
||||
service_tier: serviceTier,
|
||||
webSearchCalls,
|
||||
tool_usage: toolUsage,
|
||||
setUsageCosts: (c) => chatStream.setUsageCosts(c),
|
||||
setUsageDetails: (d) => {
|
||||
usageDetails = d;
|
||||
},
|
||||
})
|
||||
: undefined;
|
||||
chatStream.reportUsage(usage);
|
||||
@@ -939,7 +877,7 @@ export const create_chat_stream_handler_responses_api =
|
||||
reason:
|
||||
incompleteReason ?? (sawFunctionCall ? 'tool_use' : 'end_turn'),
|
||||
});
|
||||
usageDetails ??= usageDetailsFromTrackedUsage(usage);
|
||||
const usageDetails = usageDetailsFromTrackedUsage(usage);
|
||||
if (usageDetails) chatStream.setUsageDetails(usageDetails);
|
||||
|
||||
textblock.end();
|
||||
@@ -980,7 +918,6 @@ export const handle_completion_output = async (
|
||||
/**
|
||||
* @type {Record<string, unknown> & {
|
||||
* usage_calculator: (args: {
|
||||
* service_tier?: string;
|
||||
* usage: import('openai/resources/completions.mjs').CompletionUsage;
|
||||
* setUsageCosts: (costs: Record<string, number>) => void;
|
||||
* setUsageDetails: (
|
||||
@@ -1075,11 +1012,9 @@ export const handle_completion_output = async (
|
||||
* @param {((text: string) => Promise<{ flagged: boolean }>) | undefined} [params.moderate]
|
||||
* @param {(args: {
|
||||
* usage: import('openai/resources/responses/responses.mjs').ResponseUsage;
|
||||
* service_tier?: string;
|
||||
* webSearchCalls?: number;
|
||||
* tool_usage?: Record<string, unknown>;
|
||||
* setUsageCosts?: (costs: Record<string, number>) => void;
|
||||
* setUsageDetails?: (details: import('../types').UsageDetails) => void;
|
||||
* }) => unknown} params.usage_calculator
|
||||
* @param {() => Promise<void>} [params.finally_fn]
|
||||
* @returns {ReturnType<import('../types').IChatProvider['complete']>}
|
||||
@@ -1221,15 +1156,12 @@ export const handle_completion_output_responses_api = async ({
|
||||
setUsageCosts: (c) => {
|
||||
ret.usageCosts = c;
|
||||
},
|
||||
setUsageDetails: (d) => {
|
||||
ret.usageDetails = d;
|
||||
},
|
||||
})
|
||||
: {
|
||||
input_tokens: completion.usage.input_tokens,
|
||||
output_tokens: completion.usage.output_tokens,
|
||||
};
|
||||
if (usage_calculator && ret.usageDetails === undefined) {
|
||||
if (usage_calculator) {
|
||||
const usageDetails = usageDetailsFromTrackedUsage(ret.usage);
|
||||
if (usageDetails) ret.usageDetails = usageDetails;
|
||||
}
|
||||
|
||||
@@ -247,40 +247,6 @@ describe('usageDetailsFromTrackedUsage', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('labels cache writes by the lifetime the provider says it wrote them with', () => {
|
||||
expect(
|
||||
usageDetailsFromTrackedUsage(
|
||||
{
|
||||
prompt_tokens: 10,
|
||||
completion_tokens: 5,
|
||||
cached_tokens: 3,
|
||||
cache_write_tokens: 11,
|
||||
cache_write_12h_tokens: 6012,
|
||||
},
|
||||
{ writeTtl: '30m' },
|
||||
),
|
||||
).toEqual({
|
||||
inputTokens: 10,
|
||||
outputTokens: 5,
|
||||
cacheReadTokens: 3,
|
||||
cacheWrite30mTokens: 11,
|
||||
cacheWrite12hTokens: 6012,
|
||||
});
|
||||
// Zero counts, as OpenAI reports on a cache hit, add no field.
|
||||
expect(
|
||||
usageDetailsFromTrackedUsage(
|
||||
{
|
||||
prompt_tokens: 10,
|
||||
completion_tokens: 5,
|
||||
cached_tokens: 3,
|
||||
cache_write_tokens: 0,
|
||||
cache_write_12h_tokens: 0,
|
||||
},
|
||||
{ writeTtl: '30m' },
|
||||
),
|
||||
).toEqual({ inputTokens: 10, outputTokens: 5, cacheReadTokens: 3 });
|
||||
});
|
||||
|
||||
it('returns undefined for a missing/non-object usage', () => {
|
||||
expect(usageDetailsFromTrackedUsage(undefined)).toBeUndefined();
|
||||
expect(usageDetailsFromTrackedUsage(null)).toBeUndefined();
|
||||
@@ -996,66 +962,6 @@ describe('splitCachedPrompt', () => {
|
||||
cached_tokens: 3,
|
||||
});
|
||||
});
|
||||
|
||||
it('reads the Responses usage spelling', () => {
|
||||
expect(
|
||||
splitCachedPrompt({
|
||||
input_tokens: 6026,
|
||||
output_tokens: 5,
|
||||
input_tokens_details: { cached_tokens: 0, cache_write_tokens: 6023 },
|
||||
}),
|
||||
).toEqual({
|
||||
prompt_tokens: 3,
|
||||
completion_tokens: 5,
|
||||
cached_tokens: 0,
|
||||
cache_write_tokens: 6023,
|
||||
});
|
||||
});
|
||||
|
||||
it('splits 12h writes out of the write count when the breakdown is reported', () => {
|
||||
// One request can write both lifetimes: an explicit 12h breakpoint
|
||||
// plus the implicit 30m one OpenAI adds at the end of the prompt.
|
||||
expect(
|
||||
splitCachedPrompt({
|
||||
input_tokens: 6026,
|
||||
output_tokens: 5,
|
||||
input_tokens_details: {
|
||||
cached_tokens: 0,
|
||||
cache_write_tokens: 6023,
|
||||
cache_write_tokens_details: {
|
||||
ttl_12h_tokens: 6012,
|
||||
ttl_30m_tokens: 11,
|
||||
},
|
||||
},
|
||||
}),
|
||||
).toEqual({
|
||||
prompt_tokens: 3,
|
||||
completion_tokens: 5,
|
||||
cached_tokens: 0,
|
||||
cache_write_tokens: 11,
|
||||
cache_write_12h_tokens: 6012,
|
||||
});
|
||||
});
|
||||
|
||||
it('never reports more 12h writes than writes in total', () => {
|
||||
expect(
|
||||
splitCachedPrompt({
|
||||
input_tokens: 100,
|
||||
output_tokens: 1,
|
||||
input_tokens_details: {
|
||||
cached_tokens: 0,
|
||||
cache_write_tokens: 40,
|
||||
cache_write_tokens_details: { ttl_12h_tokens: 50 },
|
||||
},
|
||||
}),
|
||||
).toEqual({
|
||||
prompt_tokens: 60,
|
||||
completion_tokens: 1,
|
||||
cached_tokens: 0,
|
||||
cache_write_tokens: 0,
|
||||
cache_write_12h_tokens: 40,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// ── recorded usage costs ────────────────────────────────────────────
|
||||
|
||||
@@ -19,15 +19,13 @@
|
||||
|
||||
import type { Actor } from '../../../core/actor.js';
|
||||
import type { MeteringService } from '../../../services/metering/MeteringService.js';
|
||||
import type { IChatModel, ServiceSpeed } from '../types.js';
|
||||
import { serviceSpeedBillingModel } from './serviceSpeed.js';
|
||||
import type { IChatModel } from '../types.js';
|
||||
import { buildCostsOverride } from './pricing.js';
|
||||
|
||||
/** Microcents per US dollar. */
|
||||
const MICROCENTS_PER_USD = 100_000_000;
|
||||
|
||||
export interface MeterChatUsageOptions {
|
||||
serviceSpeed?: ServiceSpeed;
|
||||
/**
|
||||
* What the upstream says it billed, in USD. Recorded as one `billedUsage`
|
||||
* line instead of pricing the tokens, which are still recorded at zero.
|
||||
@@ -79,15 +77,10 @@ export const meterChatUsage = (
|
||||
authoritativeUsd,
|
||||
costOverrides,
|
||||
pricingTime,
|
||||
serviceSpeed = 'standard',
|
||||
}: MeterChatUsageOptions = {},
|
||||
): { usage: Record<string, number>; costs: Record<string, number> } => {
|
||||
const priced = {
|
||||
...buildCostsOverride(
|
||||
usage,
|
||||
serviceSpeedBillingModel(model, serviceSpeed),
|
||||
pricingTime,
|
||||
),
|
||||
...buildCostsOverride(usage, model, pricingTime),
|
||||
...costOverrides,
|
||||
};
|
||||
if (authoritativeUsd !== undefined) {
|
||||
|
||||
@@ -22,14 +22,11 @@ import { describe, expect, it } from 'vitest';
|
||||
import type { ICompleteArguments } from '../types.js';
|
||||
import {
|
||||
clampReasoningEffort,
|
||||
messagesHaveCacheBreakpointTtl,
|
||||
openAICompatParams,
|
||||
outputFormatFromResponseFormat,
|
||||
outputFormatFromResponsesText,
|
||||
outputFormatToResponseFormat,
|
||||
outputFormatToResponsesText,
|
||||
promptCacheOptionsToWire,
|
||||
promptCacheRetentionToWire,
|
||||
rejectStatefulResponsesFields,
|
||||
toolChoiceFromWire,
|
||||
toolChoiceToWire,
|
||||
@@ -337,109 +334,3 @@ describe('rejectStatefulResponsesFields', () => {
|
||||
).toThrow(/previous_response_id/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('promptCacheRetentionToWire', () => {
|
||||
it('spells the in-memory retention the way the upstreams expect', () => {
|
||||
expect(promptCacheRetentionToWire('in-memory')).toBe('in_memory');
|
||||
expect(promptCacheRetentionToWire('in_memory')).toBe('in_memory');
|
||||
expect(promptCacheRetentionToWire('24h')).toBe('24h');
|
||||
expect(promptCacheRetentionToWire(undefined)).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('promptCacheOptionsToWire', () => {
|
||||
it('forwards mode and ttl and nothing else', () => {
|
||||
expect(
|
||||
promptCacheOptionsToWire({
|
||||
mode: 'explicit',
|
||||
ttl: '12h',
|
||||
prewarm: true,
|
||||
comparison_response_id: 'resp_1',
|
||||
}),
|
||||
).toEqual({ mode: 'explicit', ttl: '12h' });
|
||||
expect(promptCacheOptionsToWire({ ttl: '30m' })).toEqual({
|
||||
ttl: '30m',
|
||||
});
|
||||
});
|
||||
|
||||
it('sends nothing for an absent or empty object', () => {
|
||||
expect(promptCacheOptionsToWire(undefined)).toBeUndefined();
|
||||
expect(promptCacheOptionsToWire(null)).toBeUndefined();
|
||||
expect(promptCacheOptionsToWire({})).toBeUndefined();
|
||||
expect(promptCacheOptionsToWire({ prewarm: true })).toBeUndefined();
|
||||
});
|
||||
|
||||
it('rejects a lifetime or mode OpenAI does not offer', () => {
|
||||
expect(() => promptCacheOptionsToWire({ ttl: '2h' })).toThrow(
|
||||
expect.objectContaining({ statusCode: 400 }),
|
||||
);
|
||||
expect(() => promptCacheOptionsToWire({ ttl: '24h' })).toThrow(
|
||||
/30m.*12h/,
|
||||
);
|
||||
expect(() => promptCacheOptionsToWire({ mode: 'auto' })).toThrow(
|
||||
/implicit.*explicit/,
|
||||
);
|
||||
expect(() => promptCacheOptionsToWire('12h')).toThrow(
|
||||
/must be an object/,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('messagesHaveCacheBreakpointTtl', () => {
|
||||
it('finds a breakpoint lifetime on any content part', () => {
|
||||
expect(
|
||||
messagesHaveCacheBreakpointTtl([
|
||||
{ role: 'user', content: 'plain' },
|
||||
{
|
||||
role: 'system',
|
||||
content: [
|
||||
{ type: 'text', text: 'a' },
|
||||
{
|
||||
type: 'text',
|
||||
text: 'b',
|
||||
prompt_cache_breakpoint: {
|
||||
mode: 'explicit',
|
||||
ttl: '30m',
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
]),
|
||||
).toBe(true);
|
||||
expect(
|
||||
messagesHaveCacheBreakpointTtl([
|
||||
{
|
||||
role: 'user',
|
||||
content: {
|
||||
type: 'text',
|
||||
text: 'a',
|
||||
prompt_cache_breakpoint: { ttl: '12h' },
|
||||
},
|
||||
},
|
||||
]),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it('ignores breakpoints without a lifetime and odd message shapes', () => {
|
||||
expect(
|
||||
messagesHaveCacheBreakpointTtl([
|
||||
{
|
||||
role: 'system',
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: 'a',
|
||||
prompt_cache_breakpoint: { mode: 'explicit' },
|
||||
},
|
||||
'bare string part',
|
||||
null,
|
||||
],
|
||||
},
|
||||
{ role: 'user', content: null },
|
||||
null,
|
||||
]),
|
||||
).toBe(false);
|
||||
expect(messagesHaveCacheBreakpointTtl(undefined)).toBe(false);
|
||||
expect(messagesHaveCacheBreakpointTtl('nope')).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -29,7 +29,6 @@ import { HttpError } from '../../../core/http/HttpError.js';
|
||||
import type {
|
||||
ICompleteArguments,
|
||||
OutputFormat,
|
||||
PromptCacheOptions,
|
||||
ReasoningEffort,
|
||||
ToolChoice,
|
||||
} from '../types.js';
|
||||
@@ -306,80 +305,6 @@ export const openAICompatParams = (
|
||||
return out;
|
||||
};
|
||||
|
||||
// -- prompt caching ----------------------------------------------------------
|
||||
|
||||
/**
|
||||
* The wire spelling of `prompt_cache_retention`. Puter also accepts the
|
||||
* hyphenated `in-memory`, which every OpenAI-shaped upstream rejects.
|
||||
*/
|
||||
export const promptCacheRetentionToWire = (
|
||||
retention: ICompleteArguments['prompt_cache_retention'],
|
||||
): 'in_memory' | '24h' | undefined =>
|
||||
retention === 'in-memory' ? 'in_memory' : retention;
|
||||
|
||||
const PROMPT_CACHE_MODES = new Set(['implicit', 'explicit']);
|
||||
const PROMPT_CACHE_TTLS = new Set(['30m', '12h']);
|
||||
|
||||
/**
|
||||
* `prompt_cache_options` as forwarded upstream: `mode` and `ttl` only, each
|
||||
* checked against the values OpenAI accepts, so a bad value fails here with a
|
||||
* clear message instead of as an upstream 400 the fallback loop may hide.
|
||||
* `comparison_response_id` names a stored response, which Puter never keeps,
|
||||
* and `prewarm` is dropped with it.
|
||||
*/
|
||||
export const promptCacheOptionsToWire = (
|
||||
options: unknown,
|
||||
): PromptCacheOptions | undefined => {
|
||||
if (options === undefined || options === null) return undefined;
|
||||
if (typeof options !== 'object' || Array.isArray(options)) {
|
||||
throw new HttpError(400, '`prompt_cache_options` must be an object', {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
}
|
||||
const { mode, ttl } = options as Record<string, unknown>;
|
||||
if (mode !== undefined && !PROMPT_CACHE_MODES.has(mode as string)) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'`prompt_cache_options.mode` must be `implicit` or `explicit`',
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
if (ttl !== undefined && !PROMPT_CACHE_TTLS.has(ttl as string)) {
|
||||
throw new HttpError(
|
||||
400,
|
||||
'`prompt_cache_options.ttl` must be `30m` or `12h`',
|
||||
{ legacyCode: 'bad_request' },
|
||||
);
|
||||
}
|
||||
const wire: PromptCacheOptions = {
|
||||
...(mode !== undefined
|
||||
? { mode: mode as PromptCacheOptions['mode'] }
|
||||
: {}),
|
||||
...(ttl !== undefined ? { ttl: ttl as PromptCacheOptions['ttl'] } : {}),
|
||||
};
|
||||
return Object.keys(wire).length ? wire : undefined;
|
||||
};
|
||||
|
||||
/**
|
||||
* Whether any content part carries a `prompt_cache_breakpoint` with its own
|
||||
* `ttl`. Chat Completions rejects the field whatever its value; only the
|
||||
* Responses API honours a per-breakpoint lifetime.
|
||||
*/
|
||||
export const messagesHaveCacheBreakpointTtl = (messages: unknown): boolean =>
|
||||
Array.isArray(messages) &&
|
||||
messages.some((message) => {
|
||||
const content = (message as { content?: unknown } | null)?.content;
|
||||
const parts = Array.isArray(content) ? content : [content];
|
||||
return parts.some(
|
||||
(part) =>
|
||||
(
|
||||
part as {
|
||||
prompt_cache_breakpoint?: { ttl?: unknown } | null;
|
||||
} | null
|
||||
)?.prompt_cache_breakpoint?.ttl !== undefined,
|
||||
);
|
||||
});
|
||||
|
||||
// -- Responses server-side state ---------------------------------------------
|
||||
|
||||
const STATEFUL_RESPONSES_FIELDS = [
|
||||
|
||||
@@ -1,150 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* Puter is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with Puter. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import type { IChatModel, ServiceSpeed } from '../types.js';
|
||||
import { ALIBABA_MODELS } from '../providers/alibaba/models.js';
|
||||
import { AZURE_MODELS } from '../providers/azure/models.js';
|
||||
import { BYTEPLUS_MODELS } from '../providers/byteplus/models.js';
|
||||
import { CLAUDE_MODELS } from '../providers/claude/models.js';
|
||||
import { DEEPSEEK_MODELS } from '../providers/deepseek/models.js';
|
||||
import { GEMINI_MODELS } from '../providers/gemini/models.js';
|
||||
import { GROQ_MODELS } from '../providers/groq/models.js';
|
||||
import { HOONIFY_MODELS } from '../providers/hoonify/models.js';
|
||||
import { META_MODELS } from '../providers/meta/models.js';
|
||||
import { MINIMAX_MODELS } from '../providers/minimax/models.js';
|
||||
import { MISTRAL_MODELS } from '../providers/mistral/models.js';
|
||||
import { MOONSHOT_MODELS } from '../providers/moonshot/models.js';
|
||||
import { OPEN_AI_MODELS } from '../providers/openai/models.js';
|
||||
import { XAI_MODELS } from '../providers/xai/models.js';
|
||||
import { ZAI_MODELS } from '../providers/zai/models.js';
|
||||
import { shouldPresentAsOpenAI } from './normalizeToOpenAI.js';
|
||||
import {
|
||||
isServiceSpeedUnavailable,
|
||||
lowerServiceSpeed,
|
||||
selectServiceSpeed,
|
||||
servedServiceSpeed,
|
||||
serviceSpeedBillingModel,
|
||||
} from './serviceSpeed.js';
|
||||
|
||||
const catalogs: Array<[string, IChatModel[]]> = [
|
||||
['alibaba', ALIBABA_MODELS],
|
||||
['azure-openai', AZURE_MODELS],
|
||||
['byteplus', BYTEPLUS_MODELS],
|
||||
['claude', CLAUDE_MODELS],
|
||||
['deepseek', DEEPSEEK_MODELS],
|
||||
['gemini', GEMINI_MODELS],
|
||||
['groq', GROQ_MODELS],
|
||||
['hoonify', HOONIFY_MODELS],
|
||||
['meta', META_MODELS],
|
||||
['minimax', MINIMAX_MODELS],
|
||||
['mistral', MISTRAL_MODELS],
|
||||
['moonshotai', MOONSHOT_MODELS],
|
||||
['openai-completion', OPEN_AI_MODELS],
|
||||
['openai-responses', OPEN_AI_MODELS],
|
||||
['xai', XAI_MODELS],
|
||||
['zai', ZAI_MODELS],
|
||||
];
|
||||
const speeds: ServiceSpeed[] = ['standard', 'fast', 'ultrafast'];
|
||||
|
||||
describe('service speed seeded fuzzing', () => {
|
||||
it('preserves catalog, request, normalization, and billing invariants across model changes', () => {
|
||||
let seed = 0x51eed;
|
||||
const pick = <T>(values: T[]): T => {
|
||||
seed = (Math.imul(seed, 1664525) + 1013904223) >>> 0;
|
||||
return values[seed % values.length]!;
|
||||
};
|
||||
const seenProviders = new Set<string>();
|
||||
for (let index = 0; index < 10_000; index++) {
|
||||
const [provider, catalog] = pick(catalogs);
|
||||
seenProviders.add(provider);
|
||||
const base = { ...structuredClone(pick(catalog)), provider };
|
||||
const before = structuredClone(base);
|
||||
Object.freeze(base.costs);
|
||||
Object.freeze(base);
|
||||
const args = Object.freeze({
|
||||
model: base.id,
|
||||
messages: [{ role: 'user', content: `Case ${index}` }],
|
||||
serviceSpeed: pick([...speeds, undefined]),
|
||||
normalize: pick([true, false, undefined]),
|
||||
temperature: pick([0, 0.5, 1]),
|
||||
service_tier: pick(['flex' as const, 'ultrafast' as const, undefined]),
|
||||
speed: pick(['standard' as const, 'fast' as const, undefined]),
|
||||
});
|
||||
const selected = selectServiceSpeed(base, args, (id) => {
|
||||
const variant = catalog.find((entry) => entry.id === id);
|
||||
return variant && { ...variant, provider };
|
||||
});
|
||||
expect(base).toEqual(before);
|
||||
expect(selected.args.messages).toBe(args.messages);
|
||||
expect(selected.args.temperature).toBe(args.temperature);
|
||||
expect(selected.model.provider).toBe(provider);
|
||||
expect(selected.model.release_date).toBe(base.release_date);
|
||||
expect(
|
||||
shouldPresentAsOpenAI(selected.args, selected.model.release_date),
|
||||
).toBe(shouldPresentAsOpenAI(args, base.release_date));
|
||||
if (args.serviceSpeed === undefined) {
|
||||
expect(selected.args).toBe(args);
|
||||
expect(selected.model).toBe(base);
|
||||
continue;
|
||||
}
|
||||
expect(speeds.indexOf(selected.speed)).toBeLessThanOrEqual(
|
||||
speeds.indexOf(args.serviceSpeed),
|
||||
);
|
||||
expect(selected.args.serviceSpeed).toBe(selected.speed);
|
||||
if (args.serviceSpeed === 'standard') {
|
||||
expect(selected.args.speed).not.toBe('fast');
|
||||
expect(selected.args.service_tier).not.toBe('ultrafast');
|
||||
}
|
||||
for (const cost of Object.values(selected.model.costs)) {
|
||||
expect(Number.isFinite(cost)).toBe(true);
|
||||
expect(cost).toBeGreaterThanOrEqual(0);
|
||||
}
|
||||
if (selected.model.serviceSpeedPricing) {
|
||||
const settled = serviceSpeedBillingModel(selected.model, 'standard');
|
||||
expect(settled.costs).toEqual(base.costs);
|
||||
expect(settled.pricingOverrides).toEqual(base.pricingOverrides);
|
||||
}
|
||||
let speed = selected.speed;
|
||||
for (let attempts = 0; speed !== 'standard'; attempts++) {
|
||||
expect(attempts).toBeLessThan(2);
|
||||
speed = lowerServiceSpeed(speed);
|
||||
}
|
||||
}
|
||||
expect([...seenProviders].sort()).toEqual(catalogs.map(([provider]) => provider).sort());
|
||||
});
|
||||
|
||||
it('handles malformed upstream tier and admission-error metadata without coercion or retries', () => {
|
||||
const values = [null, false, 0, 429, '', 'unrelated', [], {},
|
||||
['service_tier capacity unavailable'],
|
||||
{ toString: null }, { toString: 'priority capacity unavailable' }];
|
||||
for (const value of values) {
|
||||
expect(isServiceSpeedUnavailable(value)).toBe(false);
|
||||
for (const status of [undefined, 400, 401, 403, 404, 429, 500, 503]) {
|
||||
for (const variant of [false, true]) {
|
||||
expect(isServiceSpeedUnavailable({ status, message: value }, variant)).toBe(false);
|
||||
expect(isServiceSpeedUnavailable({ status, error: { message: value } }, variant)).toBe(false);
|
||||
}
|
||||
}
|
||||
expect(servedServiceSpeed({ usage: value })).toBe('standard');
|
||||
expect(servedServiceSpeed({ service_tier: value })).toBe('standard');
|
||||
expect(servedServiceSpeed({ usage: { speed: value, service_tier: value } })).toBe('standard');
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1,148 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
makeMeteringStub,
|
||||
optionalEnv,
|
||||
withTestActor,
|
||||
} from '../../integrationTestUtil.js';
|
||||
import type {
|
||||
IChatMessageResult,
|
||||
IChatProvider,
|
||||
ICompleteArguments,
|
||||
} from '../types.js';
|
||||
import { OpenAiResponsesChatProvider } from '../providers/openai/OpenAiChatResponsesProvider.js';
|
||||
import { GeminiChatProvider } from '../providers/gemini/GeminiChatProvider.js';
|
||||
import { MiniMaxProvider } from '../providers/minimax/MiniMaxProvider.js';
|
||||
import { MoonshotProvider } from '../providers/moonshot/MoonshotProvider.js';
|
||||
import { MistralAIProvider } from '../providers/mistral/MistralAiProvider.js';
|
||||
import { selectServiceSpeed } from './serviceSpeed.js';
|
||||
|
||||
type Case = {
|
||||
name: string;
|
||||
env: string;
|
||||
provider: string;
|
||||
model: string;
|
||||
build: (key: string) => IChatProvider;
|
||||
};
|
||||
const cases: Case[] = [
|
||||
{
|
||||
name: 'OpenAI Ultrafast',
|
||||
env: 'PUTER_TEST_AI_OPENAI_API_KEY',
|
||||
provider: 'openai-responses',
|
||||
model: 'gpt-6.1-sol',
|
||||
build: (key) =>
|
||||
new OpenAiResponsesChatProvider(
|
||||
{ apiKey: key },
|
||||
makeMeteringStub(),
|
||||
),
|
||||
},
|
||||
{
|
||||
name: 'Gemini Priority',
|
||||
env: 'PUTER_TEST_AI_GEMINI_API_KEY',
|
||||
provider: 'gemini',
|
||||
model: 'gemini-3.8-flash',
|
||||
build: (key) =>
|
||||
new GeminiChatProvider({ apiKey: key }, makeMeteringStub()),
|
||||
},
|
||||
{
|
||||
name: 'MiniMax Priority',
|
||||
env: 'PUTER_TEST_AI_MINIMAX_API_KEY',
|
||||
provider: 'minimax',
|
||||
model: 'minimax-m3',
|
||||
build: (key) =>
|
||||
new MiniMaxProvider({ apiKey: key }, makeMeteringStub()),
|
||||
},
|
||||
{
|
||||
name: 'Moonshot Highspeed',
|
||||
env: 'PUTER_TEST_AI_MOONSHOT_API_KEY',
|
||||
provider: 'moonshotai',
|
||||
model: 'kimi-k3.1-pro',
|
||||
build: (key) =>
|
||||
new MoonshotProvider({ apiKey: key }, makeMeteringStub()),
|
||||
},
|
||||
{
|
||||
name: 'Mistral served tier',
|
||||
env: 'PUTER_TEST_AI_MISTRAL_API_KEY',
|
||||
provider: 'mistral',
|
||||
model: 'mistral-medium-latest',
|
||||
build: (key) =>
|
||||
new MistralAIProvider({ apiKey: key }, makeMeteringStub()),
|
||||
},
|
||||
];
|
||||
|
||||
for (const testCase of cases) {
|
||||
describe.skipIf(!optionalEnv(testCase.env))(
|
||||
`serviceSpeed ${testCase.name} (integration)`,
|
||||
() => {
|
||||
it(
|
||||
'completes with a selected speed and normalized text',
|
||||
{ timeout: 90_000 },
|
||||
async () => {
|
||||
const provider = testCase.build(optionalEnv(testCase.env)!);
|
||||
const catalog = (await provider.models()).map((model) => ({
|
||||
...model,
|
||||
provider: testCase.provider,
|
||||
}));
|
||||
const base = catalog.find(
|
||||
(model) =>
|
||||
model.id === testCase.model ||
|
||||
model.aliases?.includes(testCase.model),
|
||||
)!;
|
||||
expect(base).toBeDefined();
|
||||
const args: ICompleteArguments = {
|
||||
model: base.id,
|
||||
messages: [
|
||||
{ role: 'user', content: 'Reply with exactly OK.' },
|
||||
],
|
||||
serviceSpeed: 'ultrafast',
|
||||
max_tokens: 256,
|
||||
reasoning_effort: 'low',
|
||||
normalize: true,
|
||||
};
|
||||
const selection = selectServiceSpeed(base, args, (id) =>
|
||||
catalog.find((model) => model.id === id),
|
||||
);
|
||||
expect(selection.speed).toBe(
|
||||
testCase.provider === 'openai-responses'
|
||||
? 'ultrafast'
|
||||
: 'fast',
|
||||
);
|
||||
const result = (await withTestActor(() =>
|
||||
provider.complete(
|
||||
{ ...selection.args, model: selection.model.id },
|
||||
selection.model,
|
||||
),
|
||||
)) as IChatMessageResult;
|
||||
expect(typeof result.message.content).toBe('string');
|
||||
expect(result.message.content.length).toBeGreaterThan(0);
|
||||
if (
|
||||
['openai-responses', 'gemini', 'minimax'].includes(
|
||||
testCase.provider,
|
||||
)
|
||||
)
|
||||
expect(result.usageDetails?.serviceSpeed).toBe(
|
||||
selection.speed,
|
||||
);
|
||||
},
|
||||
);
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -1,368 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { Writable } from 'node:stream';
|
||||
import { AIChatStream } from './Streaming.js';
|
||||
import {
|
||||
handle_completion_output,
|
||||
handle_completion_output_responses_api,
|
||||
} from './OpenAIUtil.js';
|
||||
import type { IChatModel, ServiceSpeed } from '../types.js';
|
||||
import {
|
||||
selectServiceSpeed,
|
||||
servedServiceSpeed,
|
||||
serviceSpeedBillingModel,
|
||||
isServiceSpeedUnavailable,
|
||||
} from './serviceSpeed.js';
|
||||
import { meterChatUsage } from './meterChatUsage.js';
|
||||
import { OPEN_AI_MODELS } from '../providers/openai/models.js';
|
||||
import { XAI_MODELS } from '../providers/xai/models.js';
|
||||
import { GEMINI_MODELS } from '../providers/gemini/models.js';
|
||||
import { CLAUDE_MODELS } from '../providers/claude/models.js';
|
||||
|
||||
const model = (provider: string, id: string): IChatModel => ({
|
||||
provider,
|
||||
id,
|
||||
serviceSpeedRates: [...OPEN_AI_MODELS, ...XAI_MODELS, ...GEMINI_MODELS, ...CLAUDE_MODELS]
|
||||
.find((entry) => entry.id === id)?.serviceSpeedRates,
|
||||
costs_currency: 'usd-cents',
|
||||
input_cost_key: 'prompt_tokens',
|
||||
output_cost_key: 'completion_tokens',
|
||||
max_tokens: 1000,
|
||||
costs: {
|
||||
tokens: 1_000_000,
|
||||
prompt_tokens: 200,
|
||||
completion_tokens: 1000,
|
||||
cached_tokens: 10,
|
||||
web_search_requests: 1_000_000,
|
||||
},
|
||||
});
|
||||
const request = (serviceSpeed?: ServiceSpeed) => ({
|
||||
model: 'test',
|
||||
messages: [{ role: 'user', content: 'Hello' }],
|
||||
serviceSpeed,
|
||||
});
|
||||
const noVariant = () => undefined;
|
||||
|
||||
describe('service speed selection', () => {
|
||||
it.each([2, { prompt_tokens: 400, completion_tokens: 2000 }])(
|
||||
'uses catalog rates independently of provider and model names: %j',
|
||||
(rates) => {
|
||||
const entry = {
|
||||
...model('catalog-provider', 'renamed-model'),
|
||||
serviceSpeedRates: { fast: rates },
|
||||
};
|
||||
const selected = selectServiceSpeed(entry, request('ultrafast'), noVariant);
|
||||
expect(selected.speed).toBe('fast');
|
||||
expect(selected.model.costs.prompt_tokens).toBe(400);
|
||||
expect(selected.model.costs.completion_tokens).toBe(2000);
|
||||
expect(entry.costs.prompt_tokens).toBe(200);
|
||||
},
|
||||
);
|
||||
it('preserves an omitted option and native parameters', () => {
|
||||
const entry = model('openai-responses', 'gpt-6.1-sol');
|
||||
const args = { ...request(), service_tier: 'flex' as const };
|
||||
const selected = selectServiceSpeed(entry, args, noVariant);
|
||||
expect(selected.model).toBe(entry);
|
||||
expect(selected.args).toBe(args);
|
||||
});
|
||||
it.each([
|
||||
'deepseek',
|
||||
'zai',
|
||||
'groq',
|
||||
'byteplus',
|
||||
'together-ai',
|
||||
'meta',
|
||||
'ollama',
|
||||
'hoonify',
|
||||
'neuralwatt',
|
||||
'fake-chat',
|
||||
])('uses standard when %s has no verified faster offering', (provider) => {
|
||||
const selected = selectServiceSpeed(
|
||||
model(provider, 'some-model'),
|
||||
request('ultrafast'),
|
||||
noVariant,
|
||||
);
|
||||
expect(selected.speed).toBe('standard');
|
||||
expect(selected.args.service_tier).toBeUndefined();
|
||||
});
|
||||
it.each(['standard', 'fast', 'ultrafast'] as const)(
|
||||
'selects %s without changing the model or catalog prices',
|
||||
(speed) => {
|
||||
const entry = model('openai-responses', 'gpt-6.1-sol');
|
||||
const selected = selectServiceSpeed(
|
||||
entry,
|
||||
request(speed),
|
||||
noVariant,
|
||||
);
|
||||
expect(selected.speed).toBe(speed);
|
||||
expect(selected.model.id).toBe(entry.id);
|
||||
expect(selected.model.costs.prompt_tokens).toBe(
|
||||
200 * { standard: 1, fast: 2, ultrafast: 6 }[speed],
|
||||
);
|
||||
expect(selected.model.costs.web_search_requests).toBe(1_000_000);
|
||||
expect(entry.costs.prompt_tokens).toBe(200);
|
||||
},
|
||||
);
|
||||
it('downgrades ultrafast to fast on a model without ultrafast', () => {
|
||||
const selected = selectServiceSpeed(
|
||||
model('xai', 'grok-4.7'),
|
||||
request('ultrafast'),
|
||||
noVariant,
|
||||
);
|
||||
expect(selected.speed).toBe('fast');
|
||||
expect(selected.args.service_tier).toBe('priority');
|
||||
});
|
||||
it('keeps a compatibility-only Gemini deployment at standard speed', () => {
|
||||
const entry = {
|
||||
...model('gemini', 'gemini-3.8-flash'),
|
||||
serviceSpeeds: ['standard'] as ServiceSpeed[],
|
||||
};
|
||||
expect(
|
||||
selectServiceSpeed(entry, request('ultrafast'), noVariant).speed,
|
||||
).toBe('standard');
|
||||
});
|
||||
it('uses Claude speed independently of its service tier', () => {
|
||||
const entry = model('claude', 'claude-opus-5-5');
|
||||
expect(
|
||||
selectServiceSpeed(entry, request('ultrafast'), noVariant).args
|
||||
.speed,
|
||||
).toBe('fast');
|
||||
expect(
|
||||
selectServiceSpeed(entry, request('ultrafast'), noVariant).args
|
||||
.service_tier,
|
||||
).toBeUndefined();
|
||||
expect(
|
||||
servedServiceSpeed({
|
||||
usage: { speed: 'fast', service_tier: 'standard' },
|
||||
}),
|
||||
).toBe('fast');
|
||||
});
|
||||
it.each([
|
||||
['minimax', 'minimax-m2.7', 'minimax-m2.7-highspeed'],
|
||||
['moonshotai', 'kimi-k3.1-pro', 'kimi-k3.1-pro-highspeed'],
|
||||
['alibaba', 'glm-5.3', 'glm-5.3-prime'],
|
||||
['alibaba', 'glm-5.2', 'glm-5.2-fast-preview'],
|
||||
['infron', 'infron:vendor/model', 'infron:vendor/model:priority'],
|
||||
])(
|
||||
'selects an equivalent %s variant and preserves normalization policy',
|
||||
(provider, id, variantId) => {
|
||||
const base = {
|
||||
...model(provider, id),
|
||||
serviceSpeedVariants: { fast: variantId },
|
||||
release_date: '2026-08-01',
|
||||
};
|
||||
const variant = {
|
||||
...model(provider, variantId),
|
||||
release_date: '2026-10-01',
|
||||
};
|
||||
const selected = selectServiceSpeed(
|
||||
base,
|
||||
request('ultrafast'),
|
||||
(candidate) => (candidate === variantId ? variant : undefined),
|
||||
);
|
||||
expect(selected.model.id).toBe(variantId);
|
||||
expect(selected.model.release_date).toBe(base.release_date);
|
||||
expect(selected.speed).toBe('fast');
|
||||
expect(
|
||||
selectServiceSpeed(base, request('fast'), noVariant).speed,
|
||||
).toBe('standard');
|
||||
},
|
||||
);
|
||||
it('keeps caller media on Gemini compatibility when native speed cannot carry it', () => {
|
||||
const args = {
|
||||
...request('fast'),
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{
|
||||
type: 'video_url',
|
||||
video_url: { url: 'data:video/mp4;base64,AA==' },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
expect(
|
||||
selectServiceSpeed(
|
||||
model('gemini', 'gemini-3.8-flash'),
|
||||
args,
|
||||
noVariant,
|
||||
).speed,
|
||||
).toBe('standard');
|
||||
});
|
||||
});
|
||||
|
||||
describe('served speed billing', () => {
|
||||
it.each(['standard', 'fast', 'ultrafast'] as const)(
|
||||
'settles an ultrafast hold at the served %s rate',
|
||||
(speed) => {
|
||||
const selected = selectServiceSpeed(
|
||||
model('openai-responses', 'gpt-6.1-sol'),
|
||||
request('ultrafast'),
|
||||
noVariant,
|
||||
).model;
|
||||
const recorded: Array<Record<string, number> | undefined> = [];
|
||||
const meter = {
|
||||
utilRecordUsageObject: (
|
||||
_usage: Record<string, number>,
|
||||
_actor: unknown,
|
||||
_key: string,
|
||||
costs?: Record<string, number>,
|
||||
) => {
|
||||
recorded.push(costs);
|
||||
},
|
||||
};
|
||||
const result = meterChatUsage(
|
||||
meter as never,
|
||||
undefined,
|
||||
'openai:gpt-6.1-sol',
|
||||
selected,
|
||||
{ prompt_tokens: 10, completion_tokens: 5, cached_tokens: 20 },
|
||||
{ serviceSpeed: speed },
|
||||
);
|
||||
expect(result.costs.prompt_tokens).toBe(
|
||||
2000 * { standard: 1, fast: 2, ultrafast: 6 }[speed],
|
||||
);
|
||||
expect(result.costs.cached_tokens).toBe(
|
||||
200 * { standard: 1, fast: 2, ultrafast: 6 }[speed],
|
||||
);
|
||||
expect(recorded).toEqual([result.costs]);
|
||||
},
|
||||
);
|
||||
it('restores standard scheduled and long-context rates after a downgrade', () => {
|
||||
const entry = model('openai-responses', 'gpt-6.1-sol');
|
||||
entry.pricingOverrides = [
|
||||
{
|
||||
minPromptTokens: 100,
|
||||
costs: { prompt_tokens: 400, completion_tokens: 1500 },
|
||||
},
|
||||
];
|
||||
const selected = selectServiceSpeed(
|
||||
entry,
|
||||
request('ultrafast'),
|
||||
noVariant,
|
||||
).model;
|
||||
expect(
|
||||
serviceSpeedBillingModel(selected, 'standard').pricingOverrides,
|
||||
).toEqual(entry.pricingOverrides);
|
||||
expect(
|
||||
serviceSpeedBillingModel(selected, 'fast').pricingOverrides?.[0]
|
||||
.costs.prompt_tokens,
|
||||
).toBe(800);
|
||||
});
|
||||
it.each([
|
||||
[{ service_tier: 'priority' }, 'fast'],
|
||||
[{ usage: { serviceTier: 'priority' } }, 'fast'],
|
||||
[{ usage: { service_tier: 'standard' } }, 'standard'],
|
||||
[{ service_tier: 'ultrafast' }, 'ultrafast'],
|
||||
[{}, 'standard'],
|
||||
])('reads actual speed from %j', (source, speed) => {
|
||||
expect(servedServiceSpeed(source)).toBe(speed);
|
||||
});
|
||||
});
|
||||
|
||||
describe('speed admission failures', () => {
|
||||
it.each([
|
||||
[{ status: 429, message: 'Fast mode rate limit exceeded' }, true],
|
||||
[{ status: 403, message: 'Ultrafast access unavailable' }, true],
|
||||
[{ status: 400, message: 'service_tier is not supported' }, true],
|
||||
[{ status: 401, message: 'Invalid API key' }, false],
|
||||
[{ status: 429, message: 'Rate limit exceeded' }, false],
|
||||
[{ status: 400, message: 'Invalid messages' }, false],
|
||||
])('classifies %j without swallowing unrelated errors', (error, retry) => {
|
||||
expect(isServiceSpeedUnavailable(error)).toBe(retry);
|
||||
});
|
||||
it('downgrades a missing speed variant without retrying an invalid base model', () => {
|
||||
const error = { status: 404, message: 'Model not found' };
|
||||
expect(isServiceSpeedUnavailable(error, true)).toBe(true);
|
||||
expect(isServiceSpeedUnavailable(error)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('streamed service tier metadata', () => {
|
||||
it('retains a tier from an earlier Chat Completions chunk until usage arrives', async () => {
|
||||
let servedTier: string | undefined;
|
||||
const result = await handle_completion_output({
|
||||
stream: true,
|
||||
completion: (async function* () {
|
||||
yield {
|
||||
service_tier: 'priority',
|
||||
choices: [{ index: 0, delta: { content: 'OK' } }],
|
||||
};
|
||||
yield {
|
||||
choices: [{ index: 0, delta: {}, finish_reason: 'stop' }],
|
||||
};
|
||||
yield {
|
||||
choices: [],
|
||||
usage: { prompt_tokens: 10, completion_tokens: 1 },
|
||||
};
|
||||
})(),
|
||||
usage_calculator: (source) => {
|
||||
servedTier = source.service_tier;
|
||||
return source.usage;
|
||||
},
|
||||
});
|
||||
const chatStream = new AIChatStream({
|
||||
stream: new Writable({
|
||||
write(_chunk, _encoding, callback) {
|
||||
callback();
|
||||
},
|
||||
}),
|
||||
});
|
||||
await result.init_chat_stream!({ chatStream });
|
||||
expect(servedTier).toBe('priority');
|
||||
});
|
||||
|
||||
it('uses the final Responses tier when the provider downgrades a stream', async () => {
|
||||
let servedTier: string | undefined;
|
||||
const result = await handle_completion_output_responses_api({
|
||||
stream: true,
|
||||
completion: (async function* () {
|
||||
yield {
|
||||
type: 'response.created',
|
||||
response: { service_tier: 'ultrafast' },
|
||||
};
|
||||
yield { type: 'response.output_text.delta', delta: 'OK' };
|
||||
yield {
|
||||
type: 'response.completed',
|
||||
response: {
|
||||
service_tier: 'default',
|
||||
usage: { input_tokens: 10, output_tokens: 1 },
|
||||
},
|
||||
};
|
||||
})(),
|
||||
usage_calculator: (source) => {
|
||||
servedTier = source.service_tier;
|
||||
return source.usage;
|
||||
},
|
||||
});
|
||||
const chatStream = new AIChatStream({
|
||||
stream: new Writable({
|
||||
write(_chunk, _encoding, callback) {
|
||||
callback();
|
||||
},
|
||||
}),
|
||||
});
|
||||
await result.init_chat_stream!({ chatStream });
|
||||
expect(servedTier).toBe('default');
|
||||
});
|
||||
});
|
||||
@@ -1,261 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) 2024-present Puter Technologies Inc.
|
||||
*
|
||||
* This file is part of Puter.
|
||||
*
|
||||
* Puter is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
import { requiredInputModalities } from './mediaParts.js';
|
||||
import type {
|
||||
IChatModel,
|
||||
ICompleteArguments,
|
||||
ModelCost,
|
||||
ServiceSpeed,
|
||||
} from '../types.js';
|
||||
|
||||
export const lowerServiceSpeed = (speed: ServiceSpeed): ServiceSpeed =>
|
||||
speed === 'ultrafast' ? 'fast' : 'standard';
|
||||
|
||||
const tokenRates = (costs: ModelCost, multiplier: number): ModelCost =>
|
||||
Object.fromEntries(
|
||||
Object.entries(costs).map(([key, value]) => [
|
||||
key,
|
||||
key !== 'tokens' &&
|
||||
/token|^prompt$|^completion$|^input_cache_/.test(key)
|
||||
? value * multiplier
|
||||
: value,
|
||||
]),
|
||||
);
|
||||
|
||||
export const serviceSpeedCosts = (
|
||||
model: IChatModel,
|
||||
speed: 'fast' | 'ultrafast',
|
||||
): ModelCost | undefined => {
|
||||
if (model.serviceSpeeds && !model.serviceSpeeds.includes(speed))
|
||||
return undefined;
|
||||
const rates = model.serviceSpeedRates?.[speed];
|
||||
return typeof rates === 'number'
|
||||
? tokenRates(model.costs, rates)
|
||||
: rates
|
||||
? { ...model.costs, ...rates }
|
||||
: undefined;
|
||||
};
|
||||
|
||||
export interface ServiceSpeedSelection {
|
||||
model: IChatModel;
|
||||
args: ICompleteArguments;
|
||||
speed: ServiceSpeed;
|
||||
}
|
||||
|
||||
/** Select only an equivalent model from the same provider's loaded catalog. */
|
||||
export const selectServiceSpeed = (
|
||||
model: IChatModel,
|
||||
args: ICompleteArguments,
|
||||
findVariant: (id: string) => IChatModel | undefined,
|
||||
offers: Partial<Record<'fast' | 'ultrafast', IChatModel>> = {},
|
||||
): ServiceSpeedSelection => {
|
||||
if (args.serviceSpeed === undefined)
|
||||
return { model, args, speed: 'standard' };
|
||||
const provider = model.provider;
|
||||
const fast =
|
||||
offers.fast?.costs ??
|
||||
(provider === 'gemini' &&
|
||||
(requiredInputModalities(args.messages).some(
|
||||
(modality) => !['text', 'image'].includes(modality),
|
||||
) ||
|
||||
args.messages.some(
|
||||
(message) =>
|
||||
Array.isArray(message.content) &&
|
||||
message.content.some(
|
||||
(part: Record<string, unknown>) =>
|
||||
part &&
|
||||
[
|
||||
'input_audio',
|
||||
'audio_url',
|
||||
'document',
|
||||
'file',
|
||||
'input_file',
|
||||
].includes(String(part.type)),
|
||||
),
|
||||
))
|
||||
? undefined
|
||||
: serviceSpeedCosts(model, 'fast'));
|
||||
const ultra =
|
||||
offers.ultrafast?.costs ?? serviceSpeedCosts(model, 'ultrafast');
|
||||
let speed = args.serviceSpeed;
|
||||
const variantId = model.serviceSpeedVariants?.fast;
|
||||
const variant =
|
||||
offers.fast && offers.fast.id !== model.id
|
||||
? offers.fast
|
||||
: variantId
|
||||
? findVariant(variantId)
|
||||
: undefined;
|
||||
while (speed !== 'standard') {
|
||||
if (speed === 'ultrafast' && ultra) break;
|
||||
if (speed === 'fast' && (fast || variant)) break;
|
||||
speed = lowerServiceSpeed(speed);
|
||||
}
|
||||
const request: ICompleteArguments = {
|
||||
...args,
|
||||
serviceSpeed: speed,
|
||||
speed: undefined,
|
||||
service_tier: undefined,
|
||||
};
|
||||
if (speed === 'fast' && variant) {
|
||||
return {
|
||||
model: { ...variant, release_date: model.release_date },
|
||||
args: request,
|
||||
speed,
|
||||
};
|
||||
}
|
||||
if (provider === 'gemini' && fast)
|
||||
request.service_tier = speed === 'fast' ? 'priority' : 'standard';
|
||||
else if (provider === 'claude')
|
||||
request.speed = speed === 'fast' ? 'fast' : 'standard';
|
||||
else if (provider === 'mistral')
|
||||
request.service_tier = speed === 'fast' ? 'auto' : 'default';
|
||||
else if (
|
||||
provider === 'minimax' &&
|
||||
model.serviceSpeedRates?.fast !== undefined
|
||||
)
|
||||
request.service_tier = speed === 'fast' ? 'priority' : 'standard';
|
||||
else if (fast || ultra)
|
||||
request.service_tier =
|
||||
speed === 'ultrafast'
|
||||
? 'ultrafast'
|
||||
: speed === 'fast'
|
||||
? 'priority'
|
||||
: 'default';
|
||||
if (
|
||||
speed === 'standard' &&
|
||||
[
|
||||
'openai-completion',
|
||||
'openai-responses',
|
||||
'azure-openai',
|
||||
'azure-openai-responses',
|
||||
'openrouter',
|
||||
].includes(provider ?? '')
|
||||
)
|
||||
request.service_tier = 'default';
|
||||
const selectedCosts =
|
||||
speed === 'ultrafast' ? ultra : speed === 'fast' ? fast : undefined;
|
||||
return {
|
||||
args: request,
|
||||
speed,
|
||||
model: selectedCosts
|
||||
? {
|
||||
...model,
|
||||
costs: selectedCosts,
|
||||
pricingOverrides: model.pricingOverrides?.map((override) => ({
|
||||
...override,
|
||||
costs: Object.fromEntries(
|
||||
Object.entries(override.costs).map(([key, value]) => [
|
||||
key,
|
||||
model.costs[key] > 0
|
||||
? (value *
|
||||
(selectedCosts[key] ??
|
||||
model.costs[key])) /
|
||||
model.costs[key]
|
||||
: value,
|
||||
]),
|
||||
),
|
||||
})),
|
||||
serviceSpeedPricing: {
|
||||
speed,
|
||||
standardCosts: model.costs,
|
||||
standardOverrides: model.pricingOverrides,
|
||||
fastCosts: fast,
|
||||
ultrafastCosts: ultra,
|
||||
},
|
||||
}
|
||||
: model,
|
||||
};
|
||||
};
|
||||
|
||||
/** The response is authoritative; an absent tier never earns a premium. */
|
||||
export const servedServiceSpeed = (
|
||||
source: Record<string, unknown>,
|
||||
): ServiceSpeed => {
|
||||
const usage = source.usage as Record<string, unknown> | undefined;
|
||||
const tier =
|
||||
usage?.speed ??
|
||||
usage?.serviceTier ??
|
||||
usage?.service_tier ??
|
||||
source.service_tier;
|
||||
return tier === 'ultrafast'
|
||||
? 'ultrafast'
|
||||
: tier === 'fast' || tier === 'priority'
|
||||
? 'fast'
|
||||
: 'standard';
|
||||
};
|
||||
|
||||
export const serviceSpeedBillingModel = (
|
||||
model: IChatModel,
|
||||
speed: ServiceSpeed,
|
||||
): IChatModel => {
|
||||
const pricing = model.serviceSpeedPricing;
|
||||
if (!pricing || pricing.speed === speed) return model;
|
||||
const costs =
|
||||
speed === 'fast'
|
||||
? (pricing.fastCosts ?? pricing.standardCosts)
|
||||
: pricing.standardCosts;
|
||||
return {
|
||||
...model,
|
||||
costs,
|
||||
pricingOverrides: pricing.standardOverrides?.map((override) => ({
|
||||
...override,
|
||||
costs: Object.fromEntries(
|
||||
Object.entries(override.costs).map(([key, value]) => [
|
||||
key,
|
||||
pricing.standardCosts[key] > 0
|
||||
? (value * (costs[key] ?? pricing.standardCosts[key])) /
|
||||
pricing.standardCosts[key]
|
||||
: value,
|
||||
]),
|
||||
),
|
||||
})),
|
||||
};
|
||||
};
|
||||
|
||||
/** Retry only failures that explicitly concern admission to the faster tier. */
|
||||
export const isServiceSpeedUnavailable = (
|
||||
error: unknown,
|
||||
variant = false,
|
||||
): boolean => {
|
||||
const e = error as {
|
||||
status?: number;
|
||||
statusCode?: number;
|
||||
message?: unknown;
|
||||
error?: { message?: unknown };
|
||||
};
|
||||
const status = e?.status ?? e?.statusCode;
|
||||
const message = e?.error?.message ?? e?.message ?? '';
|
||||
if (typeof message !== 'string') return false;
|
||||
if (
|
||||
variant &&
|
||||
[400, 404].includes(status ?? 0) &&
|
||||
/model.*(?:not found|not exist|unavailable)/i.test(message)
|
||||
)
|
||||
return true;
|
||||
return (
|
||||
[400, 403, 429, 503].includes(status ?? 0) &&
|
||||
/fast mode|ultrafast|service[_ ]?tier|priority (?:tier|processing|capacity)|speed parameter/i.test(
|
||||
message,
|
||||
) &&
|
||||
/unavailable|unsupported|not support|rate.limit|quota|capacity|not enabled|not allowed|access|exceed|invalid value/i.test(
|
||||
message,
|
||||
)
|
||||
);
|
||||
};
|
||||
@@ -463,26 +463,10 @@ describe('ImageGenerationDriver.generate provider routing', () => {
|
||||
driver.generate({ prompt: 'a landscape' }),
|
||||
);
|
||||
expect(openaiImagesGenerateMock).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ model: 'gpt-image-2.6-flare' }),
|
||||
expect.objectContaining({ model: 'gpt-image-2' }),
|
||||
);
|
||||
});
|
||||
|
||||
it.each(['gpt-image-2.6-flare', 'gpt-image-2.6-sunburst'])(
|
||||
'routes %s and its OpenAI alias', async (model) => {
|
||||
for (const id of [model, `openai/${model}`]) {
|
||||
openaiImagesGenerateMock.mockResolvedValueOnce({
|
||||
data: [{ url: 'https://oai/image.png' }],
|
||||
});
|
||||
await withDriverName('ai-image', () => driver.generate({
|
||||
model: id, prompt: 'a landscape',
|
||||
}));
|
||||
expect(openaiImagesGenerateMock).toHaveBeenLastCalledWith(
|
||||
expect.objectContaining({ model }),
|
||||
);
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it('rejects an excluded model regardless of caller casing', async () => {
|
||||
await expect(withActor(() =>
|
||||
driver.generate({
|
||||
@@ -758,10 +742,7 @@ describe('ImageGenerationDriver.generate provider routing', () => {
|
||||
}),
|
||||
);
|
||||
|
||||
const reserveSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'reserveCreditsIfAffordable',
|
||||
);
|
||||
const reserveSpy = vi.spyOn(server.services.metering, 'reserveCredits');
|
||||
const usageSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'batchIncrementUsages',
|
||||
@@ -855,7 +836,7 @@ describe('ImageGenerationDriver.generate provider routing', () => {
|
||||
});
|
||||
const reserveSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'reserveCreditsIfAffordable',
|
||||
'reserveCredits',
|
||||
);
|
||||
|
||||
// Cloudflare answers the scheduled refresh with new rates at once;
|
||||
@@ -1165,8 +1146,8 @@ describe('ImageGenerationDriver.generate Puter file inputs', () => {
|
||||
);
|
||||
|
||||
beforeEach(() => {
|
||||
vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable').mockImplementation((who, amount, opts) =>
|
||||
server.services.metering.reserveCredits(who, amount, opts),
|
||||
vi.spyOn(server.services.metering, 'hasEnoughCredits').mockResolvedValue(
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
@@ -1668,10 +1649,7 @@ it.each([undefined, null, '', ' ', 42, {}])(
|
||||
|
||||
describe('ImageGenerationDriver cost factor', () => {
|
||||
it('holds and meters at the AI cost factor', async () => {
|
||||
const reserveSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'reserveCreditsIfAffordable',
|
||||
);
|
||||
const reserveSpy = vi.spyOn(server.services.metering, 'reserveCredits');
|
||||
const usageSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'batchIncrementUsages',
|
||||
|
||||
@@ -72,7 +72,7 @@ vi.mock('openai', () => {
|
||||
// -- Test harness ----------------------------------------------------
|
||||
|
||||
let server: PuterServer;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let batchIncrementUsagesSpy: MockInstance<
|
||||
MeteringService['batchIncrementUsages']
|
||||
>;
|
||||
@@ -107,7 +107,7 @@ const makeProvider = (
|
||||
beforeEach(() => {
|
||||
generateMock.mockReset();
|
||||
openAICtor.mockReset();
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
batchIncrementUsagesSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'batchIncrementUsages',
|
||||
@@ -177,7 +177,7 @@ describe('BytePlusImageProvider.generate gates', () => {
|
||||
expect(result).toBe(
|
||||
'https://puter-sample-data.puter.site/image_example.png',
|
||||
);
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(generateMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
@@ -196,7 +196,7 @@ describe('BytePlusImageProvider.generate gates', () => {
|
||||
});
|
||||
|
||||
it('throws 402 BEFORE hitting ModelArk when the actor lacks credits', async () => {
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
await expect(
|
||||
withTestActor(() => makeProvider().generate({ prompt: 'hi' })),
|
||||
).rejects.toMatchObject({ statusCode: 402 });
|
||||
@@ -713,7 +713,7 @@ it.each([
|
||||
}),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400, legacyCode: 'bad_request' });
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(generateMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
|
||||
@@ -73,7 +73,7 @@ let server: PuterServer;
|
||||
let globalFetchSpy: MockInstance<typeof fetch>;
|
||||
let fetchSpy: Mock<typeof fetch>;
|
||||
let modelSearchFetch: Mock<typeof fetch>;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let batchIncrementUsagesSpy: MockInstance<
|
||||
MeteringService['batchIncrementUsages']
|
||||
>;
|
||||
@@ -123,7 +123,7 @@ beforeEach(() => {
|
||||
? modelSearchFetch(input, init)
|
||||
: fetchSpy(input, init),
|
||||
);
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
batchIncrementUsagesSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'batchIncrementUsages',
|
||||
@@ -188,7 +188,7 @@ describe('CloudflareImageProvider.generate test_mode', () => {
|
||||
expect(result).toBe(
|
||||
'https://puter-sample-data.puter.site/image_example.png',
|
||||
);
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -210,7 +210,7 @@ describe('CloudflareImageProvider.generate argument validation', () => {
|
||||
describe('CloudflareImageProvider.generate credit gate', () => {
|
||||
it('throws 402 BEFORE hitting Cloudflare when actor lacks credits', async () => {
|
||||
const provider = makeProvider();
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
|
||||
await expect(
|
||||
withTestActor(() => provider.generate({ prompt: 'hi' })),
|
||||
@@ -240,7 +240,7 @@ describe('CloudflareImageProvider.generate credit gate', () => {
|
||||
actor,
|
||||
);
|
||||
|
||||
const [, estimate] = admitSpy.mock.calls[0]!;
|
||||
const [, estimate] = hasCreditsSpy.mock.calls[0]!;
|
||||
expect(estimate).toBeGreaterThan(0);
|
||||
expect(heldDuringCall).toBe(Math.ceil(estimate));
|
||||
// The hold stays until the usage write lands.
|
||||
@@ -708,7 +708,7 @@ describe('Cloudflare legacy image option', () => {
|
||||
legacyCode: 'bad_request',
|
||||
});
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
|
||||
@@ -726,7 +726,7 @@ describe('Cloudflare legacy image option', () => {
|
||||
message: 'Invalid input image (empty or not base64)',
|
||||
});
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -823,12 +823,12 @@ describe('Cloudflare priced legacy models', () => {
|
||||
imageSize: { kind: 'pixels', w: 512, h: 512 },
|
||||
}));
|
||||
expect(JSON.parse(fetchSpy.mock.calls[0]![1]!.body as string)).toMatchObject({ image: [1, 2, 3], mask: [4, 5, 6], num_steps: 20, width: 512, height: 512 });
|
||||
expect(admitSpy).toHaveBeenCalledWith(expect.anything(), 0, undefined);
|
||||
expect(hasCreditsSpy).toHaveBeenCalledWith(expect.anything(), 0);
|
||||
});
|
||||
|
||||
it('rejects missing inpainting inputs before credit checks', async () => {
|
||||
await expect(withTestActor(() => makeProvider().generate({ model: '@cf/runwayml/stable-diffusion-v1-5-inpainting', prompt: 'A cup' }))).rejects.toMatchObject({ statusCode: 400 });
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
@@ -844,7 +844,7 @@ describe('Cloudflare priced legacy models', () => {
|
||||
it('charges Lucid at the price returned by the model API', async () => {
|
||||
fetchSpy.mockResolvedValueOnce(new Response(new Uint8Array([1, 2, 3]), { headers: { 'content-type': 'image/png' } }));
|
||||
await withTestActor(() => makeProvider().generate({ model: '@cf/leonardo/lucid-origin', prompt: 'A cup', imageSize: { kind: 'pixels', w: 512, h: 512 }, steps: 1 }));
|
||||
expect(admitSpy).toHaveBeenCalledWith(expect.anything(), 713200, undefined);
|
||||
expect(hasCreditsSpy).toHaveBeenCalledWith(expect.anything(), 713200);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -897,10 +897,9 @@ describe('Cloudflare model-search pricing', () => {
|
||||
await runPhoenix(provider);
|
||||
|
||||
expect(listed?.costs).toEqual({ tile_512: 1_000_000, step: 20_200 });
|
||||
expect(admitSpy).toHaveBeenCalledWith(
|
||||
expect(hasCreditsSpy).toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
4 * 1_000_000 + 10 * 20_200,
|
||||
undefined,
|
||||
);
|
||||
expect(batchIncrementUsagesSpy.mock.calls[0]![1]).toEqual([
|
||||
expect.objectContaining({
|
||||
@@ -931,10 +930,9 @@ describe('Cloudflare model-search pricing', () => {
|
||||
await runPhoenix(makeProvider());
|
||||
|
||||
expect(modelSearchFetch).not.toHaveBeenCalled();
|
||||
expect(admitSpy).toHaveBeenCalledWith(
|
||||
expect(hasCreditsSpy).toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
4 * 583_000 + 10 * 11_000,
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
|
||||
@@ -957,7 +955,7 @@ describe('Cloudflare model-search pricing', () => {
|
||||
await refresh;
|
||||
await runPhoenix(provider);
|
||||
|
||||
expect(admitSpy.mock.calls.map(([, cost]) => cost)).toEqual([
|
||||
expect(hasCreditsSpy.mock.calls.map(([, cost]) => cost)).toEqual([
|
||||
2_442_000, 4_202_000,
|
||||
]);
|
||||
});
|
||||
@@ -979,10 +977,9 @@ describe('Cloudflare model-search pricing', () => {
|
||||
await runPhoenix(provider);
|
||||
|
||||
expect(modelSearchFetch).toHaveBeenCalledTimes(2);
|
||||
expect(admitSpy).toHaveBeenCalledWith(
|
||||
expect(hasCreditsSpy).toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
4_202_000,
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
|
||||
@@ -1018,7 +1015,7 @@ describe('Cloudflare model-search pricing', () => {
|
||||
steps: 10,
|
||||
} as never),
|
||||
);
|
||||
return admitSpy.mock.lastCall![1];
|
||||
return hasCreditsSpy.mock.lastCall![1];
|
||||
};
|
||||
const settle = () => new Promise((resolve) => setImmediate(resolve));
|
||||
const OLD = { tile_512: 583_000, step: 11_000 };
|
||||
@@ -1072,10 +1069,9 @@ describe('Cloudflare model-search pricing', () => {
|
||||
}),
|
||||
);
|
||||
expect(modelSearchFetch).toHaveBeenCalledOnce();
|
||||
expect(admitSpy).toHaveBeenCalledWith(
|
||||
expect(hasCreditsSpy).toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
4 * 5280 + 4 * 10560,
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
@@ -82,7 +82,7 @@ vi.mock('../../inputImage.js', async (orig) => ({
|
||||
// ── Test harness ────────────────────────────────────────────────────
|
||||
|
||||
let server: PuterServer;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
|
||||
let batchIncrementUsagesSpy: MockInstance<
|
||||
MeteringService['batchIncrementUsages']
|
||||
@@ -111,7 +111,7 @@ beforeEach(() => {
|
||||
generateContentMock.mockReset();
|
||||
toBase64DataUriMock.mockReset();
|
||||
googleAICtor.mockReset();
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
batchIncrementUsagesSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
@@ -167,7 +167,7 @@ describe('GeminiImageProvider.generate test_mode', () => {
|
||||
expect(result).toBe(
|
||||
'https://puter-sample-data.puter.site/image_example.png',
|
||||
);
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(generateContentMock).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -219,7 +219,7 @@ describe('GeminiImageProvider.generate argument validation', () => {
|
||||
legacyCode: 'bad_request',
|
||||
message: expect.stringContaining('Unsupported quality tier'),
|
||||
});
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(generateContentMock).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
@@ -427,7 +427,7 @@ describe('GeminiImageProvider.generate Flash path (generateContent)', () => {
|
||||
|
||||
it('throws 402 BEFORE hitting Gemini when actor lacks credits', async () => {
|
||||
const provider = makeProvider();
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
@@ -466,7 +466,7 @@ describe('GeminiImageProvider.generate Flash path (generateContent)', () => {
|
||||
actor,
|
||||
);
|
||||
|
||||
const [, estimate] = admitSpy.mock.calls[0]!;
|
||||
const [, estimate] = hasCreditsSpy.mock.calls[0]!;
|
||||
expect(estimate).toBeGreaterThan(0);
|
||||
expect(heldDuringCall).toBe(Math.ceil(estimate));
|
||||
// The hold stays until the usage write lands.
|
||||
@@ -643,7 +643,7 @@ describe('Gemini Nano Banana image generation', () => {
|
||||
const request = generateContentMock.mock.calls[0][0];
|
||||
expect(request.config.imageConfig.imageSize).toBe('1K');
|
||||
// Input + 1120 image tokens at $30/1M + 1000 text tokens at $7.50/1M.
|
||||
expect(admitSpy).toHaveBeenCalledWith(expect.anything(), 4_194_450, undefined);
|
||||
expect(hasCreditsSpy).toHaveBeenCalledWith(expect.anything(), 4_194_450);
|
||||
expect(batchIncrementUsagesSpy).toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
expect.arrayContaining([
|
||||
|
||||
@@ -50,7 +50,7 @@ import { KlingImageProvider } from './KlingImageProvider.js';
|
||||
|
||||
let server: PuterServer;
|
||||
let fetchSpy: MockInstance<typeof fetch>;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
|
||||
|
||||
beforeAll(async () => {
|
||||
@@ -73,7 +73,7 @@ const makeProvider = () =>
|
||||
|
||||
beforeEach(() => {
|
||||
fetchSpy = vi.spyOn(globalThis, 'fetch') as MockInstance<typeof fetch>;
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
incrementUsageSpy.mockResolvedValue({} as never);
|
||||
});
|
||||
@@ -318,7 +318,7 @@ describe('KlingImageProvider billing', () => {
|
||||
});
|
||||
|
||||
it('rejects before calling Kling when credits run short', async () => {
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
await expect(generate({})).rejects.toMatchObject({ statusCode: 402 });
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
@@ -97,7 +97,7 @@ vi.mock('../../inputImage.js', async (orig) => ({
|
||||
// ── Test harness ────────────────────────────────────────────────────
|
||||
|
||||
let server: PuterServer;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let batchIncrementUsagesSpy: MockInstance<
|
||||
MeteringService['batchIncrementUsages']
|
||||
>;
|
||||
@@ -126,7 +126,7 @@ beforeEach(() => {
|
||||
editMock.mockReset();
|
||||
fetchImageAsBase64Mock.mockReset();
|
||||
openAICtor.mockReset();
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
batchIncrementUsagesSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'batchIncrementUsages',
|
||||
@@ -150,9 +150,9 @@ describe('OpenAiImageProvider construction', () => {
|
||||
// ── Model catalog ───────────────────────────────────────────────────
|
||||
|
||||
describe('OpenAiImageProvider model catalog', () => {
|
||||
it('returns gpt-image-2.6-flare as the default', () => {
|
||||
it('returns gpt-image-2 as the default', () => {
|
||||
const provider = makeProvider();
|
||||
expect(provider.getDefaultModel()).toBe('gpt-image-2.6-flare');
|
||||
expect(provider.getDefaultModel()).toBe('gpt-image-2');
|
||||
});
|
||||
|
||||
it('drops the deprecated gpt-image-1 family and chatgpt-image-latest', () => {
|
||||
@@ -195,7 +195,7 @@ describe('OpenAiImageProvider.generate test_mode', () => {
|
||||
expect(result).toBe(
|
||||
'https://puter-sample-data.puter.site/image_example.png',
|
||||
);
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(generateMock).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -253,8 +253,7 @@ describe('OpenAiImageProvider.generate credit gate', () => {
|
||||
});
|
||||
let blocked = false;
|
||||
vi.spyOn(redis, 'meterIncr').mockImplementation(async (...args) => {
|
||||
const actorKey = `:actor:${actor.user!.uuid}:`;
|
||||
if (!blocked && args.some((arg) => arg.includes(actorKey))) {
|
||||
if (!blocked && args[5].includes(`:actor:${actor.user!.uuid}:`)) {
|
||||
blocked = true;
|
||||
await held;
|
||||
}
|
||||
@@ -288,7 +287,7 @@ describe('OpenAiImageProvider.generate credit gate', () => {
|
||||
|
||||
it('throws 402 BEFORE hitting OpenAI when actor lacks credits', async () => {
|
||||
const provider = makeProvider();
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
provider.generate({
|
||||
@@ -324,7 +323,7 @@ describe('OpenAiImageProvider.generate credit gate', () => {
|
||||
actor,
|
||||
);
|
||||
|
||||
const [, estimate] = admitSpy.mock.calls[0]!;
|
||||
const [, estimate] = hasCreditsSpy.mock.calls[0]!;
|
||||
expect(estimate).toBeGreaterThan(0);
|
||||
expect(heldDuringCall).toBe(Math.ceil(estimate));
|
||||
// The hold stays until the usage write lands.
|
||||
@@ -728,8 +727,8 @@ describe('OpenAiImageProvider gpt-image-2.5 models', () => {
|
||||
);
|
||||
|
||||
// Latent factor 48 vs 24 at the medium tier.
|
||||
const gptImage2Estimate = admitSpy.mock.calls[0]![1];
|
||||
const gptImage25Estimate = admitSpy.mock.calls[1]![1];
|
||||
const gptImage2Estimate = hasCreditsSpy.mock.calls[0]![1];
|
||||
const gptImage25Estimate = hasCreditsSpy.mock.calls[1]![1];
|
||||
expect(gptImage25Estimate).toBeLessThan(gptImage2Estimate);
|
||||
});
|
||||
|
||||
@@ -754,8 +753,8 @@ describe('OpenAiImageProvider gpt-image-2.5 models', () => {
|
||||
}),
|
||||
);
|
||||
|
||||
const highEstimate = admitSpy.mock.calls[0]![1];
|
||||
const maxEstimate = admitSpy.mock.calls[1]![1];
|
||||
const highEstimate = hasCreditsSpy.mock.calls[0]![1];
|
||||
const maxEstimate = hasCreditsSpy.mock.calls[1]![1];
|
||||
expect(maxEstimate).toBeGreaterThan(highEstimate);
|
||||
});
|
||||
});
|
||||
@@ -908,64 +907,3 @@ it('keeps a landscape aspect hint landscape at gpt-image-2 sizes', async () => {
|
||||
expect(w).toBeGreaterThan(h);
|
||||
expect(Math.abs(w / h - 1.7)).toBeLessThan(0.1);
|
||||
});
|
||||
|
||||
|
||||
describe('OpenAiImageProvider GPT Image 2.6', () => {
|
||||
it.each(['gpt-image-2.6-flare', 'gpt-image-2.6-sunburst'])(
|
||||
'%s generates and edits with token-based billing', async (model) => {
|
||||
const provider = makeProvider();
|
||||
const response = {
|
||||
data: [{ b64_json: 'AAAA' }],
|
||||
usage: {
|
||||
input_tokens: 100,
|
||||
output_tokens: 800,
|
||||
input_tokens_details: { text_tokens: 100, image_tokens: 0 },
|
||||
},
|
||||
};
|
||||
generateMock.mockResolvedValue(response);
|
||||
editMock.mockResolvedValue(response);
|
||||
for (const input_image of [undefined, 'data:image/png;base64,AAAA']) {
|
||||
const result = await withTestActor(() => provider.generate({
|
||||
model, prompt: 'hi', quality: 'xhigh',
|
||||
ratio: { w: 1024, h: 1024 }, input_image,
|
||||
}));
|
||||
expect(result).toBe('data:image/png;base64,AAAA');
|
||||
}
|
||||
for (const mock of [generateMock, editMock]) {
|
||||
expect(mock.mock.calls[0]![0]).toMatchObject({
|
||||
model, quality: 'xhigh', size: '1024x1024',
|
||||
});
|
||||
}
|
||||
const [, entries] = batchIncrementUsagesSpy.mock.calls[0]!;
|
||||
expect(entries).toEqual(expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
usageType: `openai:${model}:xhigh:1024x1024:output`,
|
||||
costOverride: 2_400_000,
|
||||
}),
|
||||
]));
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
['low', 1024, 1024, 823],
|
||||
['medium', 1024, 1024, 1449],
|
||||
['high', 1024, 1024, 1756],
|
||||
['xhigh', 1024, 1024, 3122],
|
||||
['max', 1024, 1024, 7024],
|
||||
['auto', 1024, 1024, 1449],
|
||||
['low', 2048, 1088, 571],
|
||||
])('estimates %s at %sx%s from the 2.6 calculator', async (quality, w, h, tokens) => {
|
||||
generateMock.mockResolvedValue({ data: [{ b64_json: 'AAAA' }] });
|
||||
await withTestActor(() => makeProvider().generate({
|
||||
model: 'gpt-image-2.6-flare', prompt: 'hi', quality: String(quality),
|
||||
ratio: { w: Number(w), h: Number(h) },
|
||||
}));
|
||||
const [, entries] = batchIncrementUsagesSpy.mock.calls[0]!;
|
||||
const output = (entries as Array<{ usageType: string; costOverride: number }>)
|
||||
.find((entry) => entry.usageType.endsWith(':output'))!;
|
||||
expect(output.usageType).toBe(`openai:gpt-image-2.6-flare:${quality}:${w}x${h}:output`);
|
||||
// The metering conversion rounds fractional microcents upward.
|
||||
expect(output.costOverride).toBeGreaterThanOrEqual(Number(tokens) * 3000);
|
||||
expect(output.costOverride).toBeLessThanOrEqual(Number(tokens) * 3000 + 1);
|
||||
});
|
||||
});
|
||||
@@ -56,8 +56,8 @@ interface OpenAIImageUsage {
|
||||
|
||||
/**
|
||||
* OpenAI image generation provider for v2. Supports the GPT Image models
|
||||
* (gpt-image-2 and the 2.5/2.6 Sunburst and Flare models), including
|
||||
* image-to-image editing via `input_images` (the `images.edit` endpoint).
|
||||
* (gpt-image-2, -2.5-sunburst, -2.5-flare), including image-to-image editing
|
||||
* via `input_images` (the `images.edit` endpoint).
|
||||
*/
|
||||
export class OpenAiImageProvider implements IImageProvider {
|
||||
#meteringService: AiMeteringService;
|
||||
@@ -109,7 +109,7 @@ export class OpenAiImageProvider implements IImageProvider {
|
||||
}
|
||||
|
||||
getDefaultModel(): string {
|
||||
return 'gpt-image-2.6-flare';
|
||||
return 'gpt-image-2';
|
||||
}
|
||||
|
||||
async generate({
|
||||
@@ -604,36 +604,17 @@ export class OpenAiImageProvider implements IImageProvider {
|
||||
height: number,
|
||||
quality?: string,
|
||||
): number {
|
||||
const isGptImage26 = model.startsWith('gpt-image-2.6');
|
||||
const factors =
|
||||
isGptImage26 || model.startsWith('gpt-image-2.5')
|
||||
? OpenAiImageProvider.#GPT_IMAGE_2_5_LATENT_FACTORS
|
||||
: OpenAiImageProvider.#GPT_IMAGE_2_LATENT_FACTORS;
|
||||
const factors = model.startsWith('gpt-image-2.5')
|
||||
? OpenAiImageProvider.#GPT_IMAGE_2_5_LATENT_FACTORS
|
||||
: OpenAiImageProvider.#GPT_IMAGE_2_LATENT_FACTORS;
|
||||
const factor = factors[quality ?? ''] ?? factors.medium;
|
||||
const longEdge = Math.max(width, height);
|
||||
const shortEdge = Math.min(width, height);
|
||||
const shortGrid = (factor * shortEdge) / longEdge;
|
||||
const floorGrid = Math.floor(shortGrid);
|
||||
const shortLatent =
|
||||
isGptImage26 && shortGrid - floorGrid === 0.5
|
||||
? floorGrid + (floorGrid % 2)
|
||||
: Math.round(shortGrid);
|
||||
const shortLatent = Math.round((factor * shortEdge) / longEdge);
|
||||
const latentW = width >= height ? factor : shortLatent;
|
||||
const latentH = width >= height ? shortLatent : factor;
|
||||
const baseArea = latentW * latentH;
|
||||
const tokens = Math.ceil(
|
||||
(baseArea * (2_000_000 + width * height)) / 4_000_000,
|
||||
);
|
||||
// GPT Image 2.6 bills more tokens at low and medium quality.
|
||||
const estimatedQuality =
|
||||
quality && quality in factors ? quality : 'medium';
|
||||
const multiplier =
|
||||
estimatedQuality === 'low'
|
||||
? 4.2
|
||||
: estimatedQuality === 'medium'
|
||||
? 3.3
|
||||
: 1;
|
||||
return isGptImage26 ? Math.round(tokens * multiplier) : tokens;
|
||||
return Math.ceil((baseArea * (2_000_000 + width * height)) / 4_000_000);
|
||||
}
|
||||
|
||||
#estimateOutputCostFromTokens(
|
||||
|
||||
@@ -20,46 +20,6 @@
|
||||
import type { IImageModel } from '../../types.js';
|
||||
|
||||
export const OPEN_AI_IMAGE_GENERATION_MODELS: IImageModel[] = [
|
||||
{
|
||||
puterId: 'openai:openai/gpt-image-2.6-sunburst',
|
||||
id: 'gpt-image-2.6-sunburst',
|
||||
aliases: ['openai/gpt-image-2.6-sunburst'],
|
||||
name: 'GPT Image 2.6 Sunburst',
|
||||
version: '2.6',
|
||||
costs_currency: 'usd-cents',
|
||||
index_cost_key: 'low:1024x1024',
|
||||
costs: {
|
||||
// Text tokens (per 1M tokens)
|
||||
text_input: 500, // $5.00
|
||||
text_cached_input: 125, // $1.25
|
||||
// Image tokens (per 1M tokens)
|
||||
image_input: 800, // $8.00
|
||||
image_cached_input: 200, // $2.00
|
||||
image_output: 3000, // $30.00
|
||||
'low:1024x1024': 2.469,
|
||||
},
|
||||
allowedQualityLevels: ['low', 'medium', 'high', 'xhigh', 'max', 'auto'],
|
||||
},
|
||||
{
|
||||
puterId: 'openai:openai/gpt-image-2.6-flare',
|
||||
id: 'gpt-image-2.6-flare',
|
||||
aliases: ['openai/gpt-image-2.6-flare'],
|
||||
name: 'GPT Image 2.6 Flare',
|
||||
version: '2.6',
|
||||
costs_currency: 'usd-cents',
|
||||
index_cost_key: 'low:1024x1024',
|
||||
costs: {
|
||||
// Text tokens (per 1M tokens)
|
||||
text_input: 500, // $5.00
|
||||
text_cached_input: 125, // $1.25
|
||||
// Image tokens (per 1M tokens)
|
||||
image_input: 800, // $8.00
|
||||
image_cached_input: 200, // $2.00
|
||||
image_output: 3000, // $30.00
|
||||
'low:1024x1024': 2.469,
|
||||
},
|
||||
allowedQualityLevels: ['low', 'medium', 'high', 'xhigh', 'max', 'auto'],
|
||||
},
|
||||
{
|
||||
puterId: 'openai:openai/gpt-image-2.5-sunburst',
|
||||
id: 'gpt-image-2.5-sunburst',
|
||||
|
||||
+9
-14
@@ -84,7 +84,7 @@ vi.mock('../../../../util/secureHttp.js', () => ({
|
||||
// ── Test harness ────────────────────────────────────────────────────
|
||||
|
||||
let server: PuterServer;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
|
||||
let batchIncrementUsagesSpy: MockInstance<
|
||||
MeteringService['batchIncrementUsages']
|
||||
@@ -115,7 +115,7 @@ beforeEach(() => {
|
||||
cancelMock.mockReset();
|
||||
replicateCtor.mockReset();
|
||||
secureFetchMock.mockReset();
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
batchIncrementUsagesSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
@@ -177,7 +177,7 @@ describe('ReplicateImageGenerationProvider.generate test_mode', () => {
|
||||
expect(result).toBe(
|
||||
'https://puter-sample-data.puter.site/image_example.png',
|
||||
);
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(createPredictionMock).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -199,7 +199,7 @@ describe('ReplicateImageGenerationProvider.generate argument validation', () =>
|
||||
describe('ReplicateImageGenerationProvider.generate credit gate', () => {
|
||||
it('throws 402 BEFORE hitting Replicate when actor lacks credits', async () => {
|
||||
const provider = makeProvider();
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
await expect(
|
||||
withTestActor(() => provider.generate({ prompt: 'hi' })),
|
||||
).rejects.toMatchObject({ statusCode: 402 });
|
||||
@@ -255,10 +255,7 @@ describe('ReplicateImageGenerationProvider.generate credit gate', () => {
|
||||
it('holds credits for longer than a prediction can run', async () => {
|
||||
const provider = makeProvider();
|
||||
const actor = holdUser('ttl');
|
||||
const reserveSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'reserveCreditsIfAffordable',
|
||||
);
|
||||
const reserveSpy = vi.spyOn(server.services.metering, 'reserveCredits');
|
||||
createPredictionMock.mockResolvedValueOnce({
|
||||
id: 'ttl',
|
||||
status: 'succeeded',
|
||||
@@ -277,9 +274,7 @@ describe('ReplicateImageGenerationProvider.generate credit gate', () => {
|
||||
const provider = makeProvider();
|
||||
const actor = holdUser('surcharge');
|
||||
secureFetchMock.mockResolvedValueOnce(new Response('not-an-image'));
|
||||
admitSpy.mockImplementationOnce((who, amount, opts) =>
|
||||
server.services.metering.reserveCredits(who, amount, opts),
|
||||
).mockResolvedValueOnce(false);
|
||||
hasCreditsSpy.mockResolvedValueOnce(true).mockResolvedValueOnce(false);
|
||||
|
||||
await expect(
|
||||
withTestActor(
|
||||
@@ -955,7 +950,7 @@ it('does not start a prediction for an already-aborted request', async () => {
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400, legacyCode: 'client_aborted' });
|
||||
expect(createPredictionMock).not.toHaveBeenCalled();
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('stops polling and cancels at the prediction deadline', async () => {
|
||||
@@ -1197,12 +1192,12 @@ it('rejects more than ten input images before fetching or pricing any', async ()
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400, legacyCode: 'bad_request' });
|
||||
expect(secureFetchMock).not.toHaveBeenCalled();
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(createPredictionMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('checks credits on the output-only estimate before fetching input images', async () => {
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
makeProvider().generate({
|
||||
|
||||
@@ -71,7 +71,7 @@ vi.mock('together-ai', () => {
|
||||
// ── Test harness ────────────────────────────────────────────────────
|
||||
|
||||
let server: PuterServer;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
|
||||
|
||||
beforeAll(async () => {
|
||||
@@ -96,7 +96,7 @@ const makeProvider = () =>
|
||||
beforeEach(() => {
|
||||
generateMock.mockReset();
|
||||
togetherCtor.mockReset();
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
});
|
||||
|
||||
@@ -158,7 +158,7 @@ describe('TogetherImageProvider.generate test_mode', () => {
|
||||
expect(result).toBe(
|
||||
'https://puter-sample-data.puter.site/image_example.png',
|
||||
);
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(generateMock).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -189,7 +189,7 @@ describe('TogetherImageProvider.generate argument validation', () => {
|
||||
describe('TogetherImageProvider.generate credit gate', () => {
|
||||
it('throws 402 BEFORE hitting Together when actor lacks credits', async () => {
|
||||
const provider = makeProvider();
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
@@ -760,7 +760,7 @@ describe('Together supported sizes and references', () => {
|
||||
width: 1024,
|
||||
height: 1024,
|
||||
});
|
||||
expect(Number.isFinite(admitSpy.mock.calls[0][1])).toBe(true);
|
||||
expect(Number.isFinite(hasCreditsSpy.mock.calls[0][1])).toBe(true);
|
||||
});
|
||||
|
||||
it('resolves provider aliases without changing the selected model', async () => {
|
||||
@@ -790,7 +790,7 @@ describe('Together supported sizes and references', () => {
|
||||
}),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(generateMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
@@ -859,7 +859,7 @@ it.each(['3K', 'unknown', ' 5k '])(
|
||||
}),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400, legacyCode: 'bad_request' });
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(generateMock).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
@@ -895,7 +895,7 @@ describe('Together current API prices', () => {
|
||||
input_image: 'https://example.com/reference.png',
|
||||
}),
|
||||
);
|
||||
expect(admitSpy).toHaveBeenCalledWith(expect.anything(), 4685000, undefined);
|
||||
expect(hasCreditsSpy).toHaveBeenCalledWith(expect.anything(), 4685000);
|
||||
expect(incrementUsageSpy).toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
'togetherai:google/flash-image-3.1:0.5K',
|
||||
|
||||
@@ -85,7 +85,7 @@ vi.mock('openai', () => {
|
||||
// ── Test harness ────────────────────────────────────────────────────
|
||||
|
||||
let server: PuterServer;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let batchIncrementUsagesSpy: MockInstance<
|
||||
MeteringService['batchIncrementUsages']
|
||||
>;
|
||||
@@ -115,7 +115,7 @@ beforeEach(() => {
|
||||
openAICtor.mockReset();
|
||||
axiosGetMock.mockReset();
|
||||
axiosGetMock.mockRejectedValue(new Error('Catalog unavailable'));
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
batchIncrementUsagesSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'batchIncrementUsages',
|
||||
@@ -182,7 +182,7 @@ describe('XAIImageProvider.generate test_mode', () => {
|
||||
expect(result).toBe(
|
||||
'https://puter-sample-data.puter.site/image_example.png',
|
||||
);
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(generateMock).not.toHaveBeenCalled();
|
||||
expect(batchIncrementUsagesSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
@@ -231,7 +231,7 @@ describe('XAIImageProvider.generate argument validation', () => {
|
||||
describe('XAIImageProvider.generate credit gate', () => {
|
||||
it('throws 402 BEFORE hitting xAI when actor lacks credits', async () => {
|
||||
const provider = makeProvider();
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
@@ -718,7 +718,7 @@ describe('XAIImageProvider quality and reference limits', () => {
|
||||
}),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(postMock).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -746,7 +746,7 @@ describe('xAI option validation', () => {
|
||||
makeProvider().generate({ prompt: 'hi', ...options } as never),
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 400 });
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(generateMock).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -167,7 +167,7 @@ const sentRegions = () =>
|
||||
|
||||
let server: PuterServer;
|
||||
let driver: OCRDriver;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
|
||||
|
||||
beforeAll(async () => {
|
||||
@@ -199,7 +199,7 @@ beforeEach(() => {
|
||||
mistralCtor.mockReset();
|
||||
// Spy on metering — keep the real impl so its recording side runs,
|
||||
// but capture calls so per-test assertions can inspect them.
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
});
|
||||
|
||||
@@ -335,7 +335,7 @@ describe('OCRDriver.recognize (aws-textract)', () => {
|
||||
};
|
||||
|
||||
it('throws 402 when the actor does not have enough credits', async () => {
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
const { actor } = await makeUser();
|
||||
|
||||
await expect(
|
||||
@@ -546,7 +546,7 @@ describe('OCRDriver.recognize (aws-textract)', () => {
|
||||
describe('regional pricing', () => {
|
||||
const onePage = { Blocks: [{ BlockType: 'PAGE' }] };
|
||||
const checkedCosts = () =>
|
||||
admitSpy.mock.calls.map(([, cost]) => cost);
|
||||
hasCreditsSpy.mock.calls.map(([, cost]) => cost);
|
||||
const textractCharges = () =>
|
||||
incrementUsageSpy.mock.calls
|
||||
.filter(([, type]) => type === TEXTRACT_USAGE)
|
||||
@@ -719,7 +719,7 @@ describe('OCRDriver.recognize (mistral)', () => {
|
||||
};
|
||||
|
||||
it('throws 402 when the actor does not have enough credits', async () => {
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
const { actor } = await makeUser();
|
||||
|
||||
await expect(
|
||||
@@ -1033,7 +1033,7 @@ describe('OCRDriver.recognize (mistral)', () => {
|
||||
const pageType = 'mistral-ocr:mistral-ocr-4-1:page';
|
||||
const annotationType = 'mistral-ocr:mistral-ocr-4-1:annotations:page';
|
||||
// The pre-flight covers the page plus its annotation.
|
||||
expect(admitSpy.mock.calls[0]![1]).toBe(
|
||||
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(
|
||||
OCR_COSTS[pageType] + OCR_COSTS[annotationType],
|
||||
);
|
||||
const ocrCalls = incrementUsageSpy.mock.calls.filter(
|
||||
@@ -1080,7 +1080,7 @@ describe('OCRDriver credit pre-flight', () => {
|
||||
const pageType = 'mistral-ocr:mistral-ocr-4-1:page';
|
||||
const perPage = OCR_COSTS[pageType];
|
||||
const pdfSource = (buffer: Buffer) => dataUrl(buffer, 'application/pdf');
|
||||
const checkedCosts = () => admitSpy.mock.calls.map(([, cost]) => cost);
|
||||
const checkedCosts = () => hasCreditsSpy.mock.calls.map(([, cost]) => cost);
|
||||
const outstandingHolds = (actor: Actor) =>
|
||||
server.stores.creditHold.outstanding(actor.user.uuid!);
|
||||
|
||||
@@ -1171,8 +1171,8 @@ describe('OCRDriver credit pre-flight', () => {
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 402 });
|
||||
|
||||
// The estimate/hold path never ran.
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
// The estimate/hold path (which calls hasEnoughCredits) never ran.
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(mistralOcrProcessMock).not.toHaveBeenCalled();
|
||||
noUsage.mockRestore();
|
||||
});
|
||||
|
||||
@@ -72,7 +72,7 @@ vi.mock('undici', async (importOriginal) => ({
|
||||
let server: PuterServer;
|
||||
let driver: VoiceChangerDriver;
|
||||
let fetchSpy: MockInstance<typeof fetch>;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
|
||||
|
||||
beforeAll(async () => {
|
||||
@@ -91,7 +91,7 @@ afterAll(async () => {
|
||||
beforeEach(() => {
|
||||
webFetchMock.mockReset();
|
||||
fetchSpy = vi.spyOn(globalThis, 'fetch') as MockInstance<typeof fetch>;
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
});
|
||||
|
||||
@@ -197,7 +197,7 @@ describe('VoiceChangerDriver.convert argument validation', () => {
|
||||
describe('VoiceChangerDriver.convert credit gate', () => {
|
||||
it('throws 402 BEFORE hitting ElevenLabs when the actor lacks credits', async () => {
|
||||
const { actor } = await makeUser();
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
|
||||
await expect(
|
||||
withActor(actor, () =>
|
||||
@@ -387,7 +387,7 @@ describe('VoiceChangerDriver.convert success path', () => {
|
||||
|
||||
const [base, factored] = metered();
|
||||
expect(factored).toBe(base! * 2);
|
||||
const held = admitSpy.mock.calls.map(([, amount]) => amount);
|
||||
const held = hasCreditsSpy.mock.calls.map(([, amount]) => amount);
|
||||
expect(held).toEqual([base, base! * 2]);
|
||||
});
|
||||
|
||||
|
||||
@@ -122,7 +122,7 @@ vi.mock('openai', () => {
|
||||
|
||||
let server: PuterServer;
|
||||
let driver: SpeechToTextDriver;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
|
||||
|
||||
beforeAll(async () => {
|
||||
@@ -145,7 +145,7 @@ beforeEach(() => {
|
||||
toFileMock.mockClear();
|
||||
// Spy on metering — keep the real impl so its recording side runs,
|
||||
// but capture calls so per-test assertions can inspect them.
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
});
|
||||
|
||||
@@ -360,7 +360,7 @@ describe('SpeechToTextDriver argument validation', () => {
|
||||
|
||||
describe('SpeechToTextDriver credit gate', () => {
|
||||
it('throws 402 BEFORE hitting OpenAI when actor lacks credits', async () => {
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
const { actor } = await makeUser();
|
||||
|
||||
await expect(
|
||||
@@ -686,7 +686,7 @@ describe('SpeechToTextDriver metering', () => {
|
||||
expect(cost).toBe(SPEECH_TO_TEXT_COSTS[usageType]);
|
||||
});
|
||||
|
||||
it('reserves the same total it later meters', async () => {
|
||||
it('asks hasEnoughCredits for the same total it later meters', async () => {
|
||||
const { actor } = await makeUser();
|
||||
transcriptionsCreateMock.mockResolvedValueOnce({ text: 'x' });
|
||||
|
||||
@@ -700,7 +700,7 @@ describe('SpeechToTextDriver metering', () => {
|
||||
|
||||
const usageType = 'openai:gpt-transcribe:second';
|
||||
const expected = SPEECH_TO_TEXT_COSTS[usageType] * 2;
|
||||
const creditCall = admitSpy.mock.calls[0]!;
|
||||
const creditCall = hasCreditsSpy.mock.calls[0]!;
|
||||
expect(creditCall[1]).toBe(expected);
|
||||
});
|
||||
|
||||
@@ -794,7 +794,7 @@ describe('SpeechToTextDriver metering', () => {
|
||||
|
||||
const [base, factored] = metered();
|
||||
expect(factored).toBe(base! * 2);
|
||||
const held = admitSpy.mock.calls.map(([, amount]) => amount);
|
||||
const held = hasCreditsSpy.mock.calls.map(([, amount]) => amount);
|
||||
expect(held).toEqual([base, base! * 2]);
|
||||
});
|
||||
});
|
||||
|
||||
+7
-10
@@ -72,7 +72,7 @@ const HOLD = COUNTED_TOKENS * AUDIO_RATE + 8192 * OUTPUT_RATE;
|
||||
let server: PuterServer;
|
||||
let sttDriver: SpeechToTextDriver;
|
||||
let fetchSpy: MockInstance<typeof fetch>;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let batchSpy: MockInstance<MeteringService['batchIncrementUsages']>;
|
||||
|
||||
const driver = {
|
||||
@@ -102,7 +102,7 @@ beforeEach(() => {
|
||||
.mockRejectedValue(new Error('unexpected fetch')) as MockInstance<
|
||||
typeof fetch
|
||||
>;
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
batchSpy = vi.spyOn(server.services.metering, 'batchIncrementUsages');
|
||||
});
|
||||
|
||||
@@ -220,16 +220,13 @@ const metered = (): UsageInput[] =>
|
||||
/** Records each credit hold taken, its options, and its release. */
|
||||
const trackHolds = (order: string[] = []) => {
|
||||
const metering = server.services.metering;
|
||||
const realReserve = Object.getPrototypeOf(
|
||||
metering,
|
||||
).reserveCreditsIfAffordable.bind(metering) as typeof metering.reserveCreditsIfAffordable;
|
||||
const realReserve = metering.reserveCredits.bind(metering);
|
||||
const held: { amount: number; ttlMs?: number }[] = [];
|
||||
const released = vi.fn();
|
||||
vi.spyOn(metering, 'reserveCreditsIfAffordable').mockImplementation(
|
||||
vi.spyOn(metering, 'reserveCredits').mockImplementation(
|
||||
async (actor, amount, opts) => {
|
||||
held.push({ amount, ttlMs: opts?.ttlMs });
|
||||
const hold = await realReserve(actor, amount, opts);
|
||||
if (!hold) return null;
|
||||
return {
|
||||
...hold,
|
||||
release: async () => {
|
||||
@@ -428,7 +425,7 @@ describe('GeminiSpeechToTextProvider test_mode', () => {
|
||||
model: MODEL,
|
||||
});
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(batchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
@@ -1266,12 +1263,12 @@ describe('GeminiSpeechToTextProvider countTokens preflight', () => {
|
||||
describe('GeminiSpeechToTextProvider credit hold', () => {
|
||||
it('throws 402 after counting tokens, before transcribing', async () => {
|
||||
const { actor } = await makeUser();
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
fetchSpy.mockResolvedValueOnce(Response.json({ totalTokens: 91 }));
|
||||
await expect(
|
||||
withActor(actor, () => driver.transcribe({ file: dataUrl() })),
|
||||
).rejects.toMatchObject({ statusCode: 402 });
|
||||
expect(admitSpy.mock.calls[0]![1]).toBe(HOLD);
|
||||
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(HOLD);
|
||||
expect(fetchSpy).toHaveBeenCalledTimes(1);
|
||||
expect(batchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
@@ -70,7 +70,7 @@ const driver = {
|
||||
getReportedCosts: () => sttDriver.getReportedCosts(),
|
||||
};
|
||||
let fetchSpy: MockInstance<typeof fetch>;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
|
||||
|
||||
beforeAll(async () => {
|
||||
@@ -88,7 +88,7 @@ afterAll(async () => {
|
||||
|
||||
beforeEach(() => {
|
||||
fetchSpy = vi.spyOn(globalThis, 'fetch') as MockInstance<typeof fetch>;
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
});
|
||||
|
||||
@@ -193,7 +193,7 @@ describe('XAISpeechToTextProvider test_mode', () => {
|
||||
expect(Array.isArray(result.words)).toBe(true);
|
||||
// No file required, no actor required, no fetch / metering hit.
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(incrementUsageSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
@@ -258,7 +258,7 @@ describe('XAISpeechToTextProvider missing API key', () => {
|
||||
|
||||
describe('XAISpeechToTextProvider credit gate', () => {
|
||||
it('throws 402 BEFORE hitting xAI when actor lacks credits', async () => {
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
const { actor } = await makeUser();
|
||||
|
||||
await expect(
|
||||
@@ -273,7 +273,7 @@ describe('XAISpeechToTextProvider credit gate', () => {
|
||||
expect(incrementUsageSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('reserves estimated seconds × per-second ucents', async () => {
|
||||
it('asks hasEnoughCredits for estimated seconds × per-second ucents', async () => {
|
||||
const { actor } = await makeUser();
|
||||
fetchSpy.mockResolvedValueOnce(
|
||||
sttResponse({ text: 'ok', duration: 1 }),
|
||||
@@ -285,7 +285,7 @@ describe('XAISpeechToTextProvider credit gate', () => {
|
||||
driver.transcribe({ file: dataUrl(audio, 'audio/mp3') }),
|
||||
);
|
||||
|
||||
expect(admitSpy.mock.calls[0]![1]).toBe(UCENTS_PER_SECOND * 2);
|
||||
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(UCENTS_PER_SECOND * 2);
|
||||
});
|
||||
|
||||
it('estimates 60 seconds for URL inputs (we cannot inspect size locally)', async () => {
|
||||
@@ -298,7 +298,7 @@ describe('XAISpeechToTextProvider credit gate', () => {
|
||||
driver.transcribe({ file: 'https://example.com/clip.mp3' }),
|
||||
);
|
||||
|
||||
expect(admitSpy.mock.calls[0]![1]).toBe(UCENTS_PER_SECOND * 60);
|
||||
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(UCENTS_PER_SECOND * 60);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -699,9 +699,9 @@ describe('TTSDriver cost factor', () => {
|
||||
server.services.metering,
|
||||
'incrementUsage',
|
||||
);
|
||||
const admitSpy = vi.spyOn(
|
||||
const hasCreditsSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'reserveCreditsIfAffordable',
|
||||
'hasEnoughCredits',
|
||||
);
|
||||
const metered = () =>
|
||||
incrementUsageSpy.mock.calls.map(([, , , cost]) => cost!);
|
||||
@@ -725,7 +725,7 @@ describe('TTSDriver cost factor', () => {
|
||||
const [base, factored] = metered();
|
||||
expect(base).toBeGreaterThan(0);
|
||||
expect(factored).toBe(Math.round(base! * 2));
|
||||
const held = admitSpy.mock.calls.map(([, amount]) => amount);
|
||||
const held = hasCreditsSpy.mock.calls.map(([, amount]) => amount);
|
||||
expect(held).toEqual([base, Math.round(base! * 2)]);
|
||||
});
|
||||
});
|
||||
@@ -84,7 +84,7 @@ vi.mock('@aws-sdk/client-polly', async () => {
|
||||
|
||||
let server: PuterServer;
|
||||
let priceFetch: MockInstance<typeof fetch>;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
|
||||
|
||||
beforeAll(async () => {
|
||||
@@ -180,7 +180,7 @@ beforeEach(() => {
|
||||
pollyCtor.mockReset();
|
||||
priceFetch = vi.fn(servePriceLists());
|
||||
vi.stubGlobal('fetch', priceFetch);
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
});
|
||||
|
||||
@@ -314,7 +314,7 @@ describe('AWSPollyTTSProvider.synthesize test_mode', () => {
|
||||
url: 'https://puter-sample-data.puter.site/tts_example.mp3',
|
||||
content_type: 'audio',
|
||||
});
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(pollySendMock).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -349,7 +349,7 @@ describe('AWSPollyTTSProvider.synthesize argument validation', () => {
|
||||
describe('AWSPollyTTSProvider.synthesize credit gate', () => {
|
||||
it('throws 402 BEFORE hitting Polly when actor lacks credits', async () => {
|
||||
const provider = makeProvider();
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
sendDispatch();
|
||||
|
||||
await expect(
|
||||
@@ -481,7 +481,7 @@ describe('AWSPollyTTSProvider.synthesize metering', () => {
|
||||
expect(cost).toBe(expectedCost);
|
||||
});
|
||||
|
||||
it('reserves the same total it later meters', async () => {
|
||||
it('asks for hasEnoughCredits with the same total it later meters', async () => {
|
||||
const provider = makeProvider();
|
||||
sendDispatch();
|
||||
|
||||
@@ -495,7 +495,7 @@ describe('AWSPollyTTSProvider.synthesize metering', () => {
|
||||
);
|
||||
|
||||
const expectedCost = AWS_POLLY_COSTS['generative'] * text.length;
|
||||
expect(admitSpy.mock.calls[0]![1]).toBe(expectedCost);
|
||||
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(expectedCost);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -599,7 +599,7 @@ describe('AWSPollyTTSProvider published pricing', () => {
|
||||
engine: 'generative',
|
||||
});
|
||||
|
||||
expect(admitSpy.mock.calls.map((c) => c[1])).toEqual([
|
||||
expect(hasCreditsSpy.mock.calls.map((c) => c[1])).toEqual([
|
||||
1700 * 5,
|
||||
3000 * 5,
|
||||
]);
|
||||
@@ -648,7 +648,7 @@ describe('AWSPollyTTSProvider published pricing', () => {
|
||||
await readStreamResult(result);
|
||||
|
||||
expect(refreshedNeural).toBe(19);
|
||||
expect(admitSpy.mock.calls[0]![1]).toBe(1700 * 5);
|
||||
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(1700 * 5);
|
||||
expect(incrementUsageSpy.mock.calls[0]![3]).toBe(1700 * 5);
|
||||
});
|
||||
|
||||
|
||||
@@ -86,10 +86,8 @@ describe('meterPerCharacter', () => {
|
||||
|
||||
it('holds and then meters exactly text length × price', async () => {
|
||||
const check = vi
|
||||
.spyOn(server.services.metering, 'reserveCreditsIfAffordable')
|
||||
.mockImplementation((who, amount, opts) =>
|
||||
server.services.metering.reserveCredits(who, amount, opts),
|
||||
);
|
||||
.spyOn(server.services.metering, 'hasEnoughCredits')
|
||||
.mockResolvedValue(true);
|
||||
const record = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
|
||||
await drain(
|
||||
@@ -103,7 +101,7 @@ describe('meterPerCharacter', () => {
|
||||
),
|
||||
);
|
||||
|
||||
expect(check).toHaveBeenCalledWith(actor, 7500, undefined);
|
||||
expect(check).toHaveBeenCalledWith(actor, 7500);
|
||||
expect(record).toHaveBeenCalledWith(
|
||||
actor,
|
||||
'openai:tts-1:character',
|
||||
@@ -120,10 +118,8 @@ describe('meterPerCharacter', () => {
|
||||
it('meters nothing when the upstream breaks before any audio', async () => {
|
||||
vi.spyOn(
|
||||
server.services.metering,
|
||||
'reserveCreditsIfAffordable',
|
||||
).mockImplementation((who, amount, opts) =>
|
||||
server.services.metering.reserveCredits(who, amount, opts),
|
||||
);
|
||||
'hasEnoughCredits',
|
||||
).mockResolvedValue(true);
|
||||
const record = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
|
||||
await drain(
|
||||
@@ -143,10 +139,8 @@ describe('meterPerCharacter', () => {
|
||||
it('meters the full text once audio has started, even if the upstream then breaks', async () => {
|
||||
vi.spyOn(
|
||||
server.services.metering,
|
||||
'reserveCreditsIfAffordable',
|
||||
).mockImplementation((who, amount, opts) =>
|
||||
server.services.metering.reserveCredits(who, amount, opts),
|
||||
);
|
||||
'hasEnoughCredits',
|
||||
).mockResolvedValue(true);
|
||||
const record = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
|
||||
await drain(
|
||||
@@ -171,10 +165,8 @@ describe('meterPerCharacter', () => {
|
||||
it('meters the full text when the caller hangs up before any audio', async () => {
|
||||
vi.spyOn(
|
||||
server.services.metering,
|
||||
'reserveCreditsIfAffordable',
|
||||
).mockImplementation((who, amount, opts) =>
|
||||
server.services.metering.reserveCredits(who, amount, opts),
|
||||
);
|
||||
'hasEnoughCredits',
|
||||
).mockResolvedValue(true);
|
||||
const record = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
|
||||
const result = await meterPerCharacter(
|
||||
@@ -200,10 +192,8 @@ describe('meterPerCharacter', () => {
|
||||
it('meters nothing when synthesis fails', async () => {
|
||||
vi.spyOn(
|
||||
server.services.metering,
|
||||
'reserveCreditsIfAffordable',
|
||||
).mockImplementation((who, amount, opts) =>
|
||||
server.services.metering.reserveCredits(who, amount, opts),
|
||||
);
|
||||
'hasEnoughCredits',
|
||||
).mockResolvedValue(true);
|
||||
const record = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
const boom = new Error('vendor down');
|
||||
|
||||
@@ -223,8 +213,8 @@ describe('meterPerCharacter', () => {
|
||||
it('refuses with a 402 before synthesizing when unaffordable', async () => {
|
||||
vi.spyOn(
|
||||
server.services.metering,
|
||||
'reserveCreditsIfAffordable',
|
||||
).mockResolvedValue(null);
|
||||
'hasEnoughCredits',
|
||||
).mockResolvedValue(false);
|
||||
const synthesize = vi.fn(audio);
|
||||
|
||||
await expect(
|
||||
|
||||
@@ -57,7 +57,7 @@ import { ELEVENLABS_TTS_COSTS } from './costs.js';
|
||||
|
||||
let server: PuterServer;
|
||||
let fetchSpy: MockInstance<typeof fetch>;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
|
||||
|
||||
beforeAll(async () => {
|
||||
@@ -88,7 +88,7 @@ const audioResponse = (body = 'audio-bytes', contentType = 'audio/mpeg') =>
|
||||
|
||||
beforeEach(() => {
|
||||
fetchSpy = vi.spyOn(globalThis, 'fetch') as MockInstance<typeof fetch>;
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
});
|
||||
|
||||
@@ -283,7 +283,7 @@ describe('ElevenLabsTTSProvider.synthesize test_mode', () => {
|
||||
url: 'https://puter-sample-data.puter.site/tts_example.mp3',
|
||||
content_type: 'audio',
|
||||
});
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -323,7 +323,7 @@ describe('ElevenLabsTTSProvider.synthesize argument validation', () => {
|
||||
});
|
||||
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(incrementUsageSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
@@ -358,7 +358,7 @@ describe('ElevenLabsTTSProvider.synthesize argument validation', () => {
|
||||
describe('ElevenLabsTTSProvider.synthesize credit gate', () => {
|
||||
it('throws 402 BEFORE hitting ElevenLabs when actor lacks credits', async () => {
|
||||
const provider = makeProvider();
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
|
||||
await expect(
|
||||
withTestActor(() => provider.synthesize({ text: 'hi' })),
|
||||
@@ -510,7 +510,7 @@ describe('ElevenLabsTTSProvider.synthesize request shape', () => {
|
||||
fields: { key, got: value },
|
||||
});
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(incrementUsageSpy).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
@@ -893,7 +893,7 @@ describe('ElevenLabsTTSProvider.synthesize metering', () => {
|
||||
expect(cost).toBe(expectedCost);
|
||||
});
|
||||
|
||||
it('reserves the same total it later meters', async () => {
|
||||
it('asks for hasEnoughCredits with the same total it later meters', async () => {
|
||||
const provider = makeProvider();
|
||||
fetchSpy.mockResolvedValueOnce(audioResponse());
|
||||
|
||||
@@ -904,7 +904,7 @@ describe('ElevenLabsTTSProvider.synthesize metering', () => {
|
||||
|
||||
const expectedCost =
|
||||
ELEVENLABS_TTS_COSTS['eleven_multilingual_v2'] * text.length;
|
||||
expect(admitSpy.mock.calls[0]![1]).toBe(expectedCost);
|
||||
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(expectedCost);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -61,7 +61,7 @@ const INTERACTIONS_URL =
|
||||
|
||||
let server: PuterServer;
|
||||
let fetchSpy: MockInstance<typeof fetch>;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let batchIncrementUsagesSpy: MockInstance<
|
||||
MeteringService['batchIncrementUsages']
|
||||
>;
|
||||
@@ -183,16 +183,13 @@ const payingUser = () =>
|
||||
/** Spies on the credit holds taken, recording the amount and each release. */
|
||||
const trackHolds = () => {
|
||||
const metering = server.services.metering;
|
||||
const realReserve = Object.getPrototypeOf(
|
||||
metering,
|
||||
).reserveCreditsIfAffordable.bind(metering) as typeof metering.reserveCreditsIfAffordable;
|
||||
const realReserve = metering.reserveCredits.bind(metering);
|
||||
const held: number[] = [];
|
||||
const released = vi.fn();
|
||||
vi.spyOn(metering, 'reserveCreditsIfAffordable').mockImplementation(
|
||||
vi.spyOn(metering, 'reserveCredits').mockImplementation(
|
||||
async (actor, amount, opts) => {
|
||||
held.push(amount);
|
||||
const hold = await realReserve(actor, amount, opts);
|
||||
if (!hold) return null;
|
||||
return {
|
||||
...hold,
|
||||
release: async () => {
|
||||
@@ -217,7 +214,7 @@ beforeEach(() => {
|
||||
.mockRejectedValue(new Error('unexpected fetch')) as MockInstance<
|
||||
typeof fetch
|
||||
>;
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
batchIncrementUsagesSpy = vi.spyOn(
|
||||
server.services.metering,
|
||||
'batchIncrementUsages',
|
||||
@@ -335,7 +332,7 @@ describe('GeminiTTSProvider.synthesize test_mode', () => {
|
||||
url: 'https://puter-sample-data.puter.site/tts_example.mp3',
|
||||
content_type: 'audio',
|
||||
});
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -409,7 +406,7 @@ describe('GeminiTTSProvider.synthesize argument validation', () => {
|
||||
legacyCode: 'input_too_large',
|
||||
fields: { key: 'text', max: 3000 },
|
||||
});
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
@@ -437,7 +434,7 @@ describe('GeminiTTSProvider.synthesize argument validation', () => {
|
||||
legacyCode: 'input_too_large',
|
||||
fields: { key: 'instructions', max: 3000 },
|
||||
});
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -446,7 +443,7 @@ describe('GeminiTTSProvider.synthesize argument validation', () => {
|
||||
|
||||
describe('GeminiTTSProvider.synthesize credit gate', () => {
|
||||
it('throws 402 before calling Gemini when the actor lacks credits', async () => {
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
await expect(
|
||||
withTestActor(() => makeProvider().synthesize({ text: 'hi' })),
|
||||
).rejects.toMatchObject({ statusCode: 402 });
|
||||
@@ -476,10 +473,9 @@ describe('GeminiTTSProvider.synthesize credit gate', () => {
|
||||
actor,
|
||||
),
|
||||
).rejects.toMatchObject({ statusCode: 402 });
|
||||
expect(admitSpy).toHaveBeenCalledWith(
|
||||
expect(hasCreditsSpy).toHaveBeenCalledWith(
|
||||
actor,
|
||||
FLASH_RESERVATION,
|
||||
undefined,
|
||||
);
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
expect(batchIncrementUsagesSpy).not.toHaveBeenCalled();
|
||||
@@ -1369,10 +1365,9 @@ describe('GeminiTTSProvider.synthesize scheduled rates', () => {
|
||||
server.clients.event.off('ai.cost.factor.*', factor);
|
||||
}
|
||||
|
||||
expect(admitSpy).toHaveBeenCalledWith(
|
||||
expect(hasCreditsSpy).toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
reserved * 1.5,
|
||||
undefined,
|
||||
);
|
||||
expect(held).toEqual([reserved * 1.5]);
|
||||
expect(metered(':input').costOverride).toBe(7 * inputRate * 1.5);
|
||||
|
||||
@@ -84,7 +84,7 @@ vi.mock('openai', () => {
|
||||
// ── Test harness ────────────────────────────────────────────────────
|
||||
|
||||
let server: PuterServer;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
|
||||
|
||||
beforeAll(async () => {
|
||||
@@ -108,7 +108,7 @@ const mockAudioResponse = (bytes = 'opus-audio') => new Response(bytes);
|
||||
beforeEach(() => {
|
||||
speechCreateMock.mockReset();
|
||||
openAICtor.mockReset();
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
});
|
||||
|
||||
@@ -205,7 +205,7 @@ describe('OpenAITTSProvider.synthesize test_mode', () => {
|
||||
url: 'https://puter-sample-data.puter.site/tts_example.mp3',
|
||||
content_type: 'audio',
|
||||
});
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(speechCreateMock).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -283,7 +283,7 @@ describe('OpenAITTSProvider.synthesize argument validation', () => {
|
||||
describe('OpenAITTSProvider.synthesize credit gate', () => {
|
||||
it('throws 402 BEFORE hitting OpenAI when actor lacks credits', async () => {
|
||||
const provider = makeProvider();
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
|
||||
await expect(
|
||||
withTestActor(() => provider.synthesize({ text: 'hi' })),
|
||||
@@ -298,9 +298,7 @@ describe('OpenAITTSProvider.synthesize credit gate', () => {
|
||||
user: { id: 7, uuid: `tts-hold-${Date.now()}`, username: 'hold' },
|
||||
});
|
||||
const userId = actor.user!.uuid;
|
||||
admitSpy.mockImplementation((who, amount, opts) =>
|
||||
server.services.metering.reserveCredits(who, amount, opts),
|
||||
);
|
||||
hasCreditsSpy.mockResolvedValue(true);
|
||||
let heldDuringCall = -1;
|
||||
speechCreateMock.mockImplementationOnce(async () => {
|
||||
heldDuringCall = await server.stores.creditHold.outstanding(userId);
|
||||
@@ -326,9 +324,7 @@ describe('OpenAITTSProvider.synthesize credit gate', () => {
|
||||
const actor = makeActor({
|
||||
user: { id: 8, uuid: `tts-fail-${Date.now()}`, username: 'fail' },
|
||||
});
|
||||
admitSpy.mockImplementation((who, amount, opts) =>
|
||||
server.services.metering.reserveCredits(who, amount, opts),
|
||||
);
|
||||
hasCreditsSpy.mockResolvedValue(true);
|
||||
speechCreateMock.mockRejectedValueOnce(new Error('upstream down'));
|
||||
|
||||
await expect(
|
||||
@@ -467,7 +463,7 @@ describe('OpenAITTSProvider.synthesize metering', () => {
|
||||
expect(cost).toBe(expectedCost);
|
||||
});
|
||||
|
||||
it('reserves the same total it later meters', async () => {
|
||||
it('asks for hasEnoughCredits with the same total it later meters', async () => {
|
||||
const provider = makeProvider();
|
||||
speechCreateMock.mockResolvedValueOnce(mockAudioResponse());
|
||||
|
||||
@@ -477,8 +473,8 @@ describe('OpenAITTSProvider.synthesize metering', () => {
|
||||
);
|
||||
|
||||
const expectedCost = OPENAI_TTS_COSTS['tts-1'] * text.length;
|
||||
// The first reservation should match the metered cost.
|
||||
const creditCall = admitSpy.mock.calls[0]!;
|
||||
// First call to hasEnoughCredits should match the metered cost.
|
||||
const creditCall = hasCreditsSpy.mock.calls[0]!;
|
||||
expect(creditCall[1]).toBe(expectedCost);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -56,7 +56,7 @@ import { SPEECHIFY_TTS_COSTS } from './costs.js';
|
||||
|
||||
let server: PuterServer;
|
||||
let fetchSpy: MockInstance<typeof fetch>;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
|
||||
|
||||
beforeAll(async () => {
|
||||
@@ -82,7 +82,7 @@ const audioResponse = (audioData = Buffer.from('audio-bytes').toString('base64')
|
||||
|
||||
beforeEach(() => {
|
||||
fetchSpy = vi.spyOn(globalThis, 'fetch') as MockInstance<typeof fetch>;
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
});
|
||||
|
||||
@@ -159,7 +159,7 @@ describe('SpeechifyTTSProvider.synthesize test_mode', () => {
|
||||
url: 'https://puter-sample-data.puter.site/tts_example.mp3',
|
||||
content_type: 'audio',
|
||||
});
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -206,7 +206,7 @@ describe('SpeechifyTTSProvider.synthesize argument validation', () => {
|
||||
describe('SpeechifyTTSProvider.synthesize credit gate', () => {
|
||||
it('throws 402 BEFORE hitting Speechify when actor lacks credits', async () => {
|
||||
const provider = makeProvider();
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
|
||||
await expect(
|
||||
withTestActor(() => provider.synthesize({ text: 'hi' })),
|
||||
@@ -324,7 +324,7 @@ describe('SpeechifyTTSProvider.synthesize metering', () => {
|
||||
expect(cost).toBe(expectedCost);
|
||||
});
|
||||
|
||||
it('reserves the same total it later meters', async () => {
|
||||
it('asks for hasEnoughCredits with the same total it later meters', async () => {
|
||||
const provider = makeProvider();
|
||||
fetchSpy.mockResolvedValueOnce(audioResponse());
|
||||
|
||||
@@ -332,7 +332,7 @@ describe('SpeechifyTTSProvider.synthesize metering', () => {
|
||||
await withTestActor(() => provider.synthesize({ text }));
|
||||
|
||||
const expectedCost = SPEECHIFY_TTS_COSTS['simba-3.2'] * text.length;
|
||||
expect(admitSpy.mock.calls[0]![1]).toBe(expectedCost);
|
||||
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(expectedCost);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -56,7 +56,7 @@ import { XAI_TTS_COSTS } from './costs.js';
|
||||
|
||||
let server: PuterServer;
|
||||
let fetchSpy: MockInstance<typeof fetch>;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
|
||||
|
||||
beforeAll(async () => {
|
||||
@@ -79,7 +79,7 @@ const audioResponse = (body = 'audio-bytes', contentType = 'audio/mpeg') =>
|
||||
|
||||
beforeEach(() => {
|
||||
fetchSpy = vi.spyOn(globalThis, 'fetch') as MockInstance<typeof fetch>;
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
});
|
||||
|
||||
@@ -160,7 +160,7 @@ describe('XAITTSProvider.synthesize test_mode', () => {
|
||||
url: 'https://puter-sample-data.puter.site/tts_example.mp3',
|
||||
content_type: 'audio',
|
||||
});
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -195,7 +195,7 @@ describe('XAITTSProvider.synthesize argument validation', () => {
|
||||
describe('XAITTSProvider.synthesize credit gate', () => {
|
||||
it('throws 402 BEFORE hitting xAI when actor lacks credits', async () => {
|
||||
const provider = makeProvider();
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
|
||||
await expect(
|
||||
withTestActor(() => provider.synthesize({ text: 'hi' })),
|
||||
@@ -327,7 +327,7 @@ describe('XAITTSProvider.synthesize metering', () => {
|
||||
expect(cost).toBe(expectedCost);
|
||||
});
|
||||
|
||||
it('reserves the same total it later meters', async () => {
|
||||
it('asks for hasEnoughCredits with the same total it later meters', async () => {
|
||||
const provider = makeProvider();
|
||||
fetchSpy.mockResolvedValueOnce(audioResponse());
|
||||
|
||||
@@ -335,7 +335,7 @@ describe('XAITTSProvider.synthesize metering', () => {
|
||||
await withTestActor(() => provider.synthesize({ text }));
|
||||
|
||||
const expectedCost = XAI_TTS_COSTS['xai-tts'] * text.length;
|
||||
expect(admitSpy.mock.calls[0]![1]).toBe(expectedCost);
|
||||
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(expectedCost);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -94,7 +94,7 @@ vi.mock('../../util/secureHttp.js', async (importOriginal) => ({
|
||||
|
||||
let server: PuterServer;
|
||||
let driver: VideoGenerationDriver;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
|
||||
beforeAll(async () => {
|
||||
server = await setupTestServer({
|
||||
@@ -122,10 +122,8 @@ beforeEach(() => {
|
||||
togetherVideosCreateMock.mockReset();
|
||||
togetherVideosRetrieveMock.mockReset();
|
||||
secureFetchMock.mockReset();
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
admitSpy.mockImplementation((who, amount, opts) =>
|
||||
server.services.metering.reserveCredits(who, amount, opts),
|
||||
);
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
hasCreditsSpy.mockResolvedValue(true);
|
||||
vi.spyOn(server.services.metering, 'getRemainingUsage').mockResolvedValue(
|
||||
100_000_000_000,
|
||||
);
|
||||
|
||||
@@ -106,7 +106,7 @@ const MP4 = Buffer.concat([
|
||||
let server: PuterServer;
|
||||
let videoDriver: VideoGenerationDriver;
|
||||
let fetchSpy: MockInstance<typeof fetch>;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let batchSpy: MockInstance<MeteringService['batchIncrementUsages']>;
|
||||
|
||||
const generate = (args: Record<string, unknown> = {}) =>
|
||||
@@ -139,11 +139,9 @@ beforeEach(() => {
|
||||
secureFetchMock.mockReset();
|
||||
secureFetchMock.mockRejectedValue(new Error('unexpected secureFetch'));
|
||||
// The full hold is more than a new account's allowance.
|
||||
admitSpy = vi
|
||||
.spyOn(server.services.metering, 'reserveCreditsIfAffordable')
|
||||
.mockImplementation((who, amount, opts) =>
|
||||
server.services.metering.reserveCredits(who, amount, opts),
|
||||
);
|
||||
hasCreditsSpy = vi
|
||||
.spyOn(server.services.metering, 'hasEnoughCredits')
|
||||
.mockResolvedValue(true);
|
||||
batchSpy = vi.spyOn(server.services.metering, 'batchIncrementUsages');
|
||||
});
|
||||
|
||||
@@ -591,7 +589,7 @@ describe('GeminiVideoProvider test_mode', () => {
|
||||
it('returns the sample clip without network, credits or metering', async () => {
|
||||
expect(await testMode()).toBe(SAMPLE_URL);
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(batchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
@@ -1030,7 +1028,7 @@ describe('GeminiVideoProvider source video', () => {
|
||||
|
||||
const path = await writeVideo(user, 'dog.mp4', MP4);
|
||||
const read = vi.spyOn(server.stores.s3Object, 'getObjectStream');
|
||||
admitSpy.mockResolvedValue(null);
|
||||
hasCreditsSpy.mockResolvedValue(false);
|
||||
countOnly();
|
||||
await expect(
|
||||
withActor(user.actor, () => generate({ input_video: path })),
|
||||
@@ -1575,12 +1573,12 @@ describe('GeminiVideoProvider countTokens preflight', () => {
|
||||
describe('GeminiVideoProvider credit hold', () => {
|
||||
it('throws 402 after counting tokens, before generating', async () => {
|
||||
const { actor } = await makeUser();
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
fetchSpy.mockResolvedValueOnce(Response.json({ totalTokens: 18 }));
|
||||
await expect(withActor(actor, () => generate())).rejects.toMatchObject({
|
||||
statusCode: 402,
|
||||
});
|
||||
expect(admitSpy.mock.calls[0]![1]).toBe(HOLD);
|
||||
expect(hasCreditsSpy.mock.calls[0]![1]).toBe(HOLD);
|
||||
expect(fetchSpy).toHaveBeenCalledTimes(1);
|
||||
expect(batchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
@@ -85,7 +85,7 @@ vi.mock('together-ai', () => {
|
||||
// ── Test harness ────────────────────────────────────────────────────
|
||||
|
||||
let server: PuterServer;
|
||||
let admitSpy: MockInstance<MeteringService['reserveCreditsIfAffordable']>;
|
||||
let hasCreditsSpy: MockInstance<MeteringService['hasEnoughCredits']>;
|
||||
let remainingUsageSpy: MockInstance<MeteringService['getRemainingUsage']>;
|
||||
let incrementUsageSpy: MockInstance<MeteringService['incrementUsage']>;
|
||||
|
||||
@@ -117,13 +117,11 @@ beforeEach(() => {
|
||||
videosCreateMock.mockReset();
|
||||
videosRetrieveMock.mockReset();
|
||||
togetherCtor.mockReset();
|
||||
admitSpy = vi.spyOn(server.services.metering, 'reserveCreditsIfAffordable');
|
||||
hasCreditsSpy = vi.spyOn(server.services.metering, 'hasEnoughCredits');
|
||||
// Default to "has credits" so per-test setup only needs to override
|
||||
// for the explicit credit-gate scenarios. SYSTEM_ACTOR has no uuid,
|
||||
// so the live `getRemainingUsage` path can short-circuit to 0.
|
||||
admitSpy.mockImplementation((who, amount, opts) =>
|
||||
server.services.metering.reserveCredits(who, amount, opts),
|
||||
);
|
||||
hasCreditsSpy.mockResolvedValue(true);
|
||||
remainingUsageSpy = vi.spyOn(server.services.metering, 'getRemainingUsage');
|
||||
remainingUsageSpy.mockResolvedValue(AMPLE_CREDIT);
|
||||
incrementUsageSpy = vi.spyOn(server.services.metering, 'incrementUsage');
|
||||
@@ -199,7 +197,7 @@ describe('TogetherVideoProvider.generate test_mode', () => {
|
||||
provider.generate({ prompt: 'hi', test_mode: true }),
|
||||
);
|
||||
expect(result).toBe('https://assets.puter.site/txt2vid.mp4');
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
expect(videosCreateMock).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -240,7 +238,7 @@ const DIRECTOR = 'togetherai:minimax/video-01-director';
|
||||
describe('TogetherVideoProvider.generate credit gate', () => {
|
||||
it('throws 402 BEFORE hitting Together when actor lacks credits', async () => {
|
||||
const provider = makeProvider();
|
||||
admitSpy.mockResolvedValueOnce(null);
|
||||
hasCreditsSpy.mockResolvedValueOnce(false);
|
||||
|
||||
await expect(
|
||||
withTestActor(() =>
|
||||
@@ -717,7 +715,7 @@ describe('TogetherVideoProvider.generate per-second models', () => {
|
||||
expect('height' in sent).toBe(false);
|
||||
expect('ratio' in sent).toBe(false);
|
||||
// The flat-rate gate is not consulted for per-second models.
|
||||
expect(admitSpy).not.toHaveBeenCalled();
|
||||
expect(hasCreditsSpy).not.toHaveBeenCalled();
|
||||
|
||||
const seedance = TOGETHER_VIDEO_GENERATION_MODELS.find(
|
||||
(m) => m.model === 'ByteDance/Seedance-2.5',
|
||||
|
||||
Loaded 100 of 609 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user