Compare commits

..
220 Commits
Author SHA1 Message Date
Daniel Salazar 97ec5d115c fix: double counting purchased credit usage (#3359) 2026-07-07 21:43:09 -07:00
Daniel Salazar 722a6cf50f fix: phone retry limit messagin (#3358) 2026-07-07 21:05:41 -07:00
Neal Shah d039dcfee5 fix appTelemetry (#3357) 2026-07-07 20:10:13 -04:00
Daniel Salazar d09aa11f2e feat: add whatsapp support? (#3356) 2026-07-07 16:13:18 -07:00
jelveh 8a2264546d Always use hash for dashboard tabs
Previously the dashboard used a clean root URL for the 'apps' tab and hashes for other tabs. This change makes tab links and history entries consistently use a hash (e.g. #apps, #home) by always generating href as `#<id>` and pushing `#<section>` to history. Simplifies URL handling and ensures the Apps tab also reflects its hash in the address bar.
2026-07-07 15:16:59 -07:00
0801e1dc44 feat: add signup disable config and fix telemetry startup (#3319)
* feat: add signup disable config and fix telemetry startup

* fix: nits for config spread and 403 checks

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Daniel Salazar <[email protected]>
Co-authored-by: Claude Fable 5 <[email protected]>
2026-07-07 13:52:29 -07:00
Daniel Salazar a04d5fa5e0 fix: package-lock (#3355) 2026-07-07 13:43:15 -07:00
e1280e8cc7 Add custom icon support for web links (#3325)
* feat: add custom web link icon support and context menu option

Introduce the ability to change and display custom icons for .weblink files. Added a "Change Icon" context menu item that invokes the changeWeblinkIcon helper, and updated the icon rendering logic to use getWeblinkIcon instead of the default link icon. This allows users to personalize web link appearance in the file manager.

* Fix security and robustness issues in web link custom icons

 # UIWindowSearch and require the icon data URL body to be pure base64 in
  #isValidWeblinkIcon. A shared/downloaded .weblink is untrusted input and
 # the previous prefix-only check let a crafted icon break out of the
 # <img src> attribute.
 # byte-signature MIME sniffing and capping stored icon size (was storing
 # full-resolution images as base64).
 # without picking a file.
 # weblinks no longer fetches every file's contents on each render.
 # path instead of the target weblink).
 # label, add the AGPL license header, and drop redundant stored icon
 # copies.

* Keep SVG weblink icons as vectors instead of rasterizing

Rasterizing a picked SVG to a 256px PNG threw away its scalability for no
real benefit: an SVG rendered in <img> runs in script-free static mode, and
the base64-body validation already prevents attribute injection. Store SVGs
as data:image/svg+xml;base64 (content-sniffed, size-capped), and keep
rasterizing raster formats and oversized SVGs to bound stored size.

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Daniel Salazar <[email protected]>
Co-authored-by: Claude Fable 5 <[email protected]>
2026-07-07 13:41:37 -07:00
Aman Rai 70884c2e30 fix: rename start-gui script to start:gui (#3346) 2026-07-07 12:09:49 -07:00
Reynaldi Chernando 917e95e3d7 Add nano banana 2 lite (#3336) 2026-07-07 12:08:36 -07:00
jelvehandClaude Opus 4.8 27222bbb0d Let ctrl/cmd+f use browser find while dashboard is open
The global ctrl/cmd+f shortcut opened UIWindowSearch and preventDefault'd
the keystroke, blocking the browser's native find. Skip that interception
while a dashboard window is open so ctrl/cmd+f falls through to the browser.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-07-07 09:43:34 -07:00
jelveh 3720034464 Make Apps the default dashboard tab
Treat the Apps tab as the default (root URL) instead of Home. UIDashboard now reads an initial route from window.dashboard_initial_route?.tab and falls back to 'apps' for unknown/absent routes. Sidebar and section active class logic and tab href/hash generation were updated so 'apps' uses the clean root URL while all other tabs (including home) use #tab. The history.pushState logic was adjusted accordingly. parseDashboardRoute in initgui.js was updated to default to 'apps' and its comment clarified the new behavior.
2026-07-07 09:31:43 -07:00
jelveh de6cab1f9a Reduce myapps tile icon size to 56px
Update .myapps-tile-icon in src/gui/src/css/dashboard.css to change width and height from 60px to 56px, adjusting tile icon sizing for improved layout and spacing.
2026-07-06 21:04:32 -07:00
jelveh 2d1fbc2611 Auto-focus My Apps search on activation
Call focusSearch when the Apps dashboard is activated and add a focusSearch helper. The helper imports isTouchPrimaryDevice and skips focusing on touch-primary devices to avoid showing the on-screen keyboard. It retries up to 12 times (30ms interval) until the search input is visible to handle cases where the dashboard is briefly hidden while entering full-page mode.
2026-07-06 20:10:44 -07:00
jelveh 0afc70218f Use input[type=search] and normalize styling 2026-07-06 20:03:48 -07:00
jelveh 8d3abbc5f0 Increase myapps tile icon size to 60px
Update .myapps-tile-icon width and height from 52px to 60px to improve icon visibility and better match the updated UI design. Other styles (overflow, background, transitions) are left unchanged.
2026-07-06 19:47:17 -07:00
jelveh 7ec14bc7db Add audio-editor and ai-image-project to recommended apps 2026-07-06 19:42:20 -07:00
jelveh 2843010a97 Increase myapps tile size to 100px 2026-07-06 19:37:46 -07:00
jelveh ad4e21ee61 Open external apps via index_url from Apps tab
When an app is external and provides an index_url, use its hostname as the tile title and store the index_url in a new data-target-link attribute. Update the click handler to open the target link in a new tab for external apps, and fall back to opening the internal /app/{name} page for others. This aligns Apps tab behavior with the Home tab and ensures external apps navigate to their website.
2026-07-06 19:21:34 -07:00
Nariman Jelveh 4de39ab6f2 Mark external apps and use the flag for dashboard titles (#3354)
* Mark external apps and use the flag for dashboard titles

An app with no owner_user_id (null/empty) isn't owned by a Puter user —
it's an external, origin-bootstrapped app whose uuid/name/title are all the
opaque app-… id. The dashboard used to detect these client-side by comparing
uuid/name/title and a name.startsWith('app-') check.

Expose an authoritative `external` flag from the API instead:

- /installedApps and /get-launch-apps now return `external`, derived from
  owner_user_id, and no longer leak the raw owner id.
- The dashboard (Home + Apps tabs) shows the index_url hostname for external
  apps based on `external` rather than the uuid/name/title heuristic.

Add/extend extension tests to cover the `external` flag and non-leak.

* Use hostname for opaque external app titles

Detect opaque external app IDs (when name === title === uid/uuid) and replace the displayed title with the hostname from index_url. Adds a uid/uuid fallback and tightens the external-app check in both the Apps and Home dashboard tabs so they behave consistently; preserves target_link for Home entries.

* Add external flag for apps without owner

Expose an `external` property in AppSummary (toAppSummary in RecommendedAppsService.ts) to mark apps that are not owned by a Puter user. The flag is set when `owner_user_id` is null or an empty string, allowing clients to identify origin-bootstrapped/external apps.
2026-07-06 18:00:56 -07:00
Nariman Jelveh fbb646b8eb Return index_url from /installedApps for dashboard app titles (#3352)
The dashboard derives a hostname title for anonymous (app-…) apps whose
uuid/name/title are all identical, falling back to the host of index_url.
That worked for launch/recent apps but not for apps that come only from
/installedApps, because the endpoint didn't return index_url.

Add apps.index_url to the SELECT (alongside apps.icon, relying on the
functional dependency on the apps.id primary key) and assert it in the
extension test.
2026-07-06 16:46:31 -07:00
Neal Shah 46f75ecd6f add type for puter.app.authenticated (#3353) 2026-07-06 18:38:30 -04:00
Daniel Salazar f68caa3bbd feat: allow optional card fallback for failed phone verifications (#3351) 2026-07-06 14:59:55 -07:00
jelveh 0109bf5b72 Use index_url hostname for anonymous app title
Detect anonymous apps whose name/title/uid are identical opaque ids (starting with 'app-') and, when index_url is available, replace the displayed title with the index_url hostname (matching the Home tab behavior). Also persist index_url into the mapped app object and make title a mutable variable to support the change.
2026-07-06 14:43:39 -07:00
Nariman Jelveh c69fb52d5c Make the dashboard the default interface at root (#3345)
* feat: make the dashboard the default interface at root

The dashboard now loads at / (with /dashboard kept as an alias), and the
desktop moves to a new /desktop route. An "Open Desktop" button pinned at
the bottom of the dashboard sidebar switches to the desktop.

Root URLs that carry a desktop-only flow keep booting the desktop:
auth popups (?embedded_in_popup=), app deep links (?app=), direct
downloads (?download=), fullpage mode (?puter.fullpage=), and iframe
embeds.

Desktop-context redirects are retargeted so desktop users stay on the
desktop: logout, the QR "continue on phone" link, the auth cover-page
logo click, and window close/hide URL resets now use /desktop. OIDC
login/signup gains a strictly whitelisted return_to param (/desktop or
/dashboard) so social login returns to the interface it started from.

Also: the dashboard strips ?ref= like the desktop does, ?auth_token=
cleanup preserves the path and tab hash, the logged-out session list
gets the wallpaper back, ?download= also works at /desktop, and the GUI
dev server serves /desktop and /dashboard.

* Add Open Desktop bento card and CSS

Replace the previous desktop-switch UI with a full-width "Open Desktop" bento card that links to /desktop. Removes the pinned sidebar Open Desktop button and the old desktop-switch button/handler, and adds new CSS variables and styles for the desktop card (icon background, shadow, surface/header colors), hover shadow and arrow transition, and layout/padding adjustments.

Changed files: TabHome.js (removed old switch, added anchor card), UIDashboard.js (removed sidebar button), dashboard.css (new desktop color vars and styles; removed legacy desktop-switch/button styles).

* Remove box-shadow from myapps search input

Delete the box-shadow declarations from input.myapps-search::placeholder in dashboard.css. This simplifies the placeholder styling (removes subtle and light shadow layers), likely to reduce visual noise or fix rendering inconsistency.

* Show index_url hostname for opaque app IDs

Only replace an app's title with the hostname from index_url when the app appears to use an opaque id. Add a check for names starting with 'app-' so the hostname is used only if title, name and uuid are identical and the name looks like an opaque app id. This avoids overwriting meaningful user-facing names that may coincidentally match the uuid and clarifies the intent in the comment.

* Update dashboard.css
2026-07-06 13:22:27 -07:00
Neal Shah 3a26f1f1c2 add admin moderation events table (#3348) 2026-07-06 11:49:09 -04:00
Daniel Salazar f1b94a5b69 fix: otel upgrade bump (#3328) 2026-07-06 08:42:25 -07:00
Neal Shah e3fce9191a emit puter.app.authenticated event (#3337) 2026-07-06 11:23:27 -04:00
Nariman Jelveh 2c0a1b9bbb Revert "feat: add sign-in flow for crossOriginIsolated contexts (#3253)" (#3335)
This reverts commit 6fff1fd75f.
2026-06-30 23:32:27 -07:00
jelveh 0d7417f858 Add 'contacts' to recommended apps
Include 'contacts' in the RECOMMENDED_APP_NAMES array so the Contacts app appears in the recommended apps list. Updated src/backend/services/apps/RecommendedAppsService.ts.
2026-06-30 19:08:11 -07:00
Neal Shah 2eee098c66 Add suspension reasons (#3331) 2026-06-30 18:09:00 -04:00
Neal Shah 890d4b3c16 remove temperature from sonnet-5 (#3330) 2026-06-30 17:18:29 -04:00
Neal Shah af9a7c6345 add claude sonnet 5 (#3329) 2026-06-30 17:08:42 -04:00
velzie 6fff1fd75f feat: add sign-in flow for crossOriginIsolated contexts (#3253)
* poc

* route login key through broadcastservice

* cleanup

* security validation and prevent additional webhook sends

* block double-broadcast from redis
2026-06-30 20:17:53 +01:00
velzie e6b7b08bda Add forceRelay option to Peer API and document raw peerconnection property (#3313)
* add forceRelay options

* document peerconnection
2026-06-30 12:28:49 -04:00
Daniel Salazar f4c3cbd53d feat: add logs for failed sms (#3318) 2026-06-30 06:50:47 -07:00
Reynaldi ChernandoandCopilot Autofix powered by AI c0c2f35266 Update docs model drift (#3326)
* Update docs model drift

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

* standardize

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-06-30 16:55:42 +07:00
Nikhil Kumar SinghandDevblaze14 ccea980753 feat: add Ideogram 4.0 Together AI image model (#3309)
Co-authored-by: Devblaze14 <[email protected]>
2026-06-29 15:33:01 -04:00
Neal Shah 2bddebd576 add ability to suspend apps (#3322) 2026-06-29 13:06:08 -04:00
Neal Shah b9866ddbeb add suspended_at col (#3321) 2026-06-29 09:51:00 -04:00
Mochammad Fadhlan Al-Ghiffari 6fc79a7184 fix(docs): highlight code blocks after client-side navigation (#3287) (#3288)
* fix(docs): highlight all code blocks after client-side navigation

The docs site highlights code on full page load via an inline
hljs.highlightAll(), which targets the 'pre code' selector and so
highlights every fenced block, including those without a language tag.

The client-side navigation path (pathchange handler) instead selected
only code[class^="language"], so fenced blocks written without a
language hint were left unhighlighted after in-app navigation while
appearing correctly on a hard reload.

Align the pathchange handler with highlightAll() by selecting 'pre code'
and skipping blocks hljs has already processed (data-highlighted=yes),
keeping the pass idempotent across repeated pathchange events.

Fixes #3287

* fix(docs): hoist hljs.configure and scope highlight to docs content

Address review nits on the pathchange highlight pass:
- Move hljs.configure({ ignoreUnescapedHTML: true }) out of the per-element
  loop since it is global config and only needs to run once.
- Scope the selector to '.docs-content pre code' so the re-highlight pass
  only touches the swapped-in docs body and leaves code blocks elsewhere
  on the page untouched.
2026-06-28 23:27:07 +07:00
Nariman Jelveh 50039571b6 Reorder recommended apps and add vault
Adjust RECOMMENDED_APP_NAMES ordering: place 'editor', 'camera', and 'recorder' immediately after 'builder' and move 'app-center' accordingly. Also add 'vault' to the recommended apps list. This changes the resolved ordering of recommended apps at call time.
2026-06-27 17:55:52 -07:00
Daniel Salazar 1216f20567 fix: package-lock issue (#3316) 2026-06-27 14:33:11 -07:00
Daniel Salazar 0752ea761c feat: remove stych, add prelude client (#3312) 2026-06-27 14:07:52 -07:00
Daniel Salazar 98461088f8 fix: decrease puts for unused stamps (#3310) 2026-06-26 12:33:04 -07:00
Neal Shah d4a3224aa3 force locale en-US (#3311) 2026-06-26 15:31:58 -04:00
Daniel Salazar 197ce6faff fix: LegacyFSController (#3308) 2026-06-25 14:47:29 -07:00
MRB Labs c98e293879 fix: unlink OIDC providers when user updates email address (#3307) 2026-06-25 14:46:41 -07:00
Miika Kuisma ebbe382596 tweak context menu's text labels (#3306) 2026-06-25 09:46:05 -07:00
Neal Shah 01d28eb2c5 Await phone verification event (#3305) 2026-06-24 20:08:30 -04:00
Daniel Salazar 7d1c2e7f0a fix: fsService copy (#3304) 2026-06-24 16:32:34 -07:00
Daniel Salazar 22b499af20 fix: cleanup fs responses (#3303) 2026-06-24 14:35:58 -07:00
Neal Shah 7208f86723 bind btoa and atob in emulated worker globalspace (#3302)
* bind btoa and atob in emulated worker globalspace

* Create working directory for workers
2026-06-24 15:56:26 -04:00
Neal Shah 2974a8b01c store app owner ID even when worker is deployed from root ctx (#3301) 2026-06-24 15:21:48 -04:00
Daniel Salazar 7f05a707bd feat: improve kv store incr/decr costs (#3300) 2026-06-24 11:57:36 -07:00
Miika Kuismaandjelveh 1d056253ef contextmenu design tweak (#3298)
* contextmenu design tweak

* icon gap fix (mobile context menu web component)

* fix submenu item padding

* add dark theme support for puter.ui.contextMenu and puter.ui.setMenubar (works only with puter.env === 'web')

* Add theme test & refactor context menu items

Extract shared mediaMenuItems to remove duplication and simplify context menu calls. Add a light-theme context menu handler and a "Run theme test" button that executes runContextMenuThemeTest(), which programmatically renders menus to verify the theme attribute and .puter-theme-dark class for dark, light, and default cases (mirrors existing Playwright assertions). Update testContextMenu and testContextMenuDark to use the new item factory and clean up rendered menus after checks. UI text is updated to show pass/fail results.

---------

Co-authored-by: jelveh <[email protected]>
2026-06-24 11:30:16 -07:00
Neal Shah a22dc5facb fix whoami user token check (#3299) 2026-06-24 14:28:02 -04:00
Neal Shah aeafd4ac79 txt2img megaupdate (#3294)
* Update OpenAIImageProvider with txt2img with input source

* Grok image provider mega update

* make input_images universal
2026-06-24 10:24:09 -04:00
Daniel Salazar 14861df6b2 fix: bad boot error (#3295) 2026-06-24 00:33:38 -07:00
Daniel Salazar 34a7595a6d fix: bad empty file error handling (#3293) 2026-06-23 16:13:41 -07:00
Daniel Salazar b2108c0216 fix: update email for supprot (#3292) 2026-06-23 15:43:12 -07:00
Philip Ifeanyi 5a5a449210 feat: enhance playground UI with page title and style adjustments (#3289) 2026-06-23 22:45:18 +07:00
Daniel Salazar 0405de30bd update taskbar apps (#3286) 2026-06-22 14:26:35 -07:00
Nariman Jelveh 40c2799b06 Add 'builder' to recommended apps list
Include 'builder' in the RECOMMENDED_APP_NAMES array in src/backend/services/apps/RecommendedAppsService.ts so the RecommendedAppsService treats the builder app as a recommended application and surfaces it accordingly.
2026-06-22 14:22:27 -07:00
Daniel Salazar 4bcdeefe68 chore: bump up deps (#3285) 2026-06-22 13:58:45 -07:00
Daniel Salazar b6ba6dd90e fix: [PUT-1199] more prelude signals (#3282) 2026-06-22 08:16:59 -07:00
Daniel Salazar 954f8b7b8d fix: puter self host guides + default bug (#3281)
* fix: oss dev center html_encode hardening

* fix: puter self host guides + default bug
2026-06-21 22:32:43 -07:00
Neal Shah 24d700673c Compaction support for OpenAI and Anthropic (#3279)
* Alpha: compaction support for OpenAI and Anthropic

* update lock

* fix billing for anthropic compactions

* Fix max_tokens bug in together provider

* Fix responses compaction
2026-06-21 21:57:56 -04:00
c6d64029e0 feat: meter Gemini thinking tokens and grounding requests (#3178)
* feat: meter Gemini thinking tokens and grounding requests

- Thinking tokens: Extracted from standard completion tokens to ensure they are billed accurately at the correct model-specific rate.
- Grounding requests: Added flat-fee metering for Google Search by tracking grounding_metadata across both streaming and non-streaming responses.
- Pricing updates: Corrected stale rates for Gemini 2.5 Flash output, cached tokens, thinking tokens, and grounding requests.

* fix: correct Gemini 2.x metering rates and harden grounding capture

Pricing corrections (verified against ai.google.dev/gemini-api/docs/pricing):
- gemini-2.5-flash output is $2.50/M, not $1.00/M: restore
  completion_tokens to 250 and bill thinking_tokens at the same output
  rate (250). The previous 100 under-billed output ~60%, and this is the
  provider's default model.
- gemini-2.5-flash cache read is $0.03/M: restore cached_tokens to 3
  (the 7.5 value over-billed).
- Grounding with Google Search is $35 / 1,000 requests for Gemini 2.x
  models and $14 / 1,000 for 3.x. Set grounding_requests to 3_500_000
  for gemini-2.0-flash, gemini-2.5-flash, gemini-2.5-flash-lite and
  gemini-2.5-pro; 3.x models keep 1_400_000.

Streaming robustness:
- In create_chat_stream_handler, don't let a later extra_content chunk
  without grounding_metadata overwrite an earlier one that carried it,
  so grounding requests are still metered.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

---------

Co-authored-by: Daniel Salazar <[email protected]>
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-20 15:08:38 -07:00
jelveh 40ff866598 Add 'butler' to recommended apps
Include 'butler' in the RECOMMENDED_APP_NAMES array in RecommendedAppsService so it appears in the recommended apps list. Updates src/backend/services/apps/RecommendedAppsService.ts.
2026-06-19 15:50:57 -07:00
Nariman Jelveh 36888e35d3 Add phone verification UI improvements and helpers (#3274)
Revamps the phone verification dialog and adds reusable phone helper modules.

- Heavily refactors UIWindowPhoneVerificationRequired.js to add a searchable inline country picker, national-number live formatting, country autodetect for "+" input, WebOTP auto-fill support, resend cooldown timer, accessibility improvements, proper teardown/cleanup, and last-used country persistence.
- Introduces helpers/src/helpers/country_codes.js (get_country_list, flag emoji) which builds a localized list of dialable countries from libphonenumber-js and Intl.DisplayNames.
- Introduces helpers/src/helpers/phone.js with thin wrappers around libphonenumber-js: format_phone_as_you_type, inspect_phone, phone_example_for, detect_country_from_input.
- Adds English i18n strings for the new UI in src/gui/src/i18n/translations/en.js.

These changes keep submission canonicalization using libphonenumber-js (E.164), improve UX for selecting/formatting numbers, and make the country list and phone utilities reusable across the app.
2026-06-19 10:28:30 -07:00
Reynaldi Chernando 30432b931a Add PUT and DELETE to preflight allowed methods (#3278) 2026-06-19 12:35:29 -04:00
Reynaldi Chernando e2174a14cf Update router cors preflight docs (#3277) 2026-06-19 20:50:41 +07:00
Daniel Salazar d3cb89d8c9 fix: import in extensions (#3276) 2026-06-18 21:30:00 -07:00
Reynaldi Chernando c8625b3396 Release puterjs npm (#3275)
* Release puterjs npm

* add package lock
2026-06-18 18:26:39 -07:00
Reynaldi Chernando b576f01d12 Sync Puter.js types and docs (#3194)
* AI chat types

* ai voice types

* update tts docs and jsdoc

* stt types and docs

* txt2img docs and types

* txt2vid and img2txt

* apps

* signin

* perms

* fs

* kv

* hosting

* workers

* net

* ui

* jsdoc
2026-06-18 09:34:13 -07:00
Vyasdev a87cafee3c Revise bug report and feature request instructions (#3220)
Updated the wording for reporting bugs and feature requests in Japanese.
2026-06-18 09:27:55 -07:00
Reynaldi Chernando 1c51fde163 Add minimax m3 direct integration + cleanup model list (#3263) 2026-06-18 10:12:44 -04:00
Daniel Salazar b05699db77 fix: AGENTS.md cleanup (#3273) 2026-06-17 17:07:20 -07:00
Daniel Salazar 163e48ab27 feat: viz over user verification types (#3271) 2026-06-17 14:21:55 -07:00
Reynaldi Chernando 0b41d992a4 Add glm 5.2 direct integration (#3269) 2026-06-17 09:40:10 -04:00
Miika Kuisma c2ddfd6867 Merge pull request #3270 from HeyPuter/fix-file-dialog
puter.js - fix issue where content of file dialog wasn't visible
2026-06-17 13:08:43 +03:00
Miika Kuisma 81b9dc1f06 fix issue where content of file dialog wasn't visible 2026-06-17 13:07:13 +03:00
Daniel Salazar 5506ddb2c2 feat: fp cleanup and better devex (#3268) 2026-06-16 22:48:54 -07:00
Neal Shah c74469b01a prelude RCS (#3267) 2026-06-15 19:08:30 -04:00
Daniel Salazar b706693f82 wip: hardening (#3266) 2026-06-15 16:07:44 -07:00
Daniel SalazarandNeal Shah cd800148d5 feat: optional additional verification gates (#3262)
* phone number verificiation initial

* finish phone verification

* feat: add card gate for signups

* chore: npm

* fix: modal order and wording

* fix:wording

---------

Co-authored-by: Neal Shah <[email protected]>
2026-06-15 09:16:00 -07:00
Ricardo Sawir 900b0d7aa6 fix: correct puter container healthcheck URL (#3255)
The healthcheck was using 'puter.localhost:4100/test' which fails for
two reasons:
- 'puter.localhost' does not resolve inside the container; the service
  listens on all interfaces at port 4100, so 'localhost' works
- '/test' endpoint does not exist; use '/' which serves the shell

This caused the container to be perpetually marked as unhealthy.

Fixes #3191
2026-06-13 23:17:44 -07:00
Daniel Salazar 7e21c1f888 fix: fs metadata sanitation (#3257) 2026-06-12 22:56:06 -07:00
Daniel Salazar 00d6f9c230 fix: disallow unverified outlook emails (#3256) 2026-06-12 18:52:24 -07:00
jelveh 4c03080efa Update mcp.md 2026-06-12 16:55:37 +02:00
jelveh 432a11d289 Add Cursor MCP docs and icons 2026-06-12 16:37:53 +02:00
jelveh e9e233404b Add icons for Claude, Codex and OpenCode 2026-06-12 13:20:15 +02:00
jelveh 8101d42604 Capitalize OpenCode in MCP docs 2026-06-12 13:16:41 +02:00
jelveh 88cedd43bc Add opencode MCP example to docs 2026-06-12 13:11:24 +02:00
Daniel Salazar d3a2934d9b feat: fingerprinting for signup checks (#3254) 2026-06-12 00:59:21 -07:00
Daniel Salazar 13a36b6199 fix: secure http calls (#3252) 2026-06-11 13:43:58 -07:00
Daniel Salazar a155464d46 fix: more data for puter site serve event (#3251) 2026-06-10 23:27:13 -07:00
Daniel Salazar 40d1c998bb feat: pass args to all events (#3248)
* feat: pass args to all events

* fix: alias app joining

* fix: actor in event
2026-06-10 14:13:04 -07:00
Daniel Salazar b4837d49b6 perf: some perf gains from fable (#3249) 2026-06-10 13:26:35 -07:00
ProgrammerIn-wonderland e23042b42e add preperation for reputation score persistence (#3250) 2026-06-10 16:09:42 -04:00
Daniel Salazar 240a733285 sec: misc fable hardening (#3244)
* sec: misc fable hardening

* more fixes

* more fixes

* fix: cors issue
2026-06-10 11:19:41 -07:00
Reynaldi Chernando bb202be12c Minor clarification local webserver in deployment (#3247) 2026-06-10 23:04:42 +07:00
Reynaldi Chernando 8c4cb601c9 Add cli docs (#3246) 2026-06-10 20:53:53 +07:00
Reynaldi Chernando 64a7c7b7c4 Add mcp docs (#3245) 2026-06-10 20:31:31 +07:00
wwb1942 2dd2887ff7 ci: add docs build workflow (#3235)
* ci: add docs build workflow

Add CI workflow that runs npm ci and npm run build in src/docs

whenever files under src/docs change in a pull request.

Closes #3225

* ci: also trigger docs build when workflow file changes

* ci: address docs build workflow review
2026-06-10 10:53:40 +07:00
ProgrammerIn-wonderland 4bbe8a2789 add protections for PUT-1094 (#3243) 2026-06-09 22:29:57 -04:00
Dilan Melvin T 3e9ceb673b fix: coerce Mistral image_url parts from object to string (#3177)
* fix: coerce Mistral image_url parts from object to string

Mistral's API expects image_url content parts to be a plain string URL,
not the OpenAI-style { url: string } object that process_input_messages
produces. Without this coercion, sending an image to any Mistral model
that supports vision (mistral-small, mistral-medium, mistral-large,
ministral-*) results in a request error from the Mistral SDK.

Add #coerceImageUrls() as a private method on MistralAIProvider that
maps { type: 'image_url', image_url: { url } } -> { type: 'image_url',
image_url: url } for every content part in every message. Messages with
plain string content are left untouched, as are parts whose image_url is
already a string.

Add four unit tests covering: object-to-string coercion, already-flat
strings, plain string message content, and mixed text+image content.

* style: use plain ASCII dashes in comment section divider
2026-06-09 20:12:44 -04:00
Daniel Salazar 9c4d1ef535 fix: misc bugs + new middleware for before/after routes (#3242) 2026-06-09 17:05:56 -07:00
ProgrammerIn-wonderland bf57518262 MCP: assert worker_exec ends with puter.work (#3241) 2026-06-09 17:43:34 -04:00
Reynaldi Chernando d48fb11de6 Add claude fable 5 (#3238) 2026-06-09 14:05:31 -04:00
ProgrammerIn-wonderland 7ed87f760b make max tokens less lenient (#3240) 2026-06-09 14:03:13 -04:00
Reynaldi Chernando ee37be16f0 Add cli deployment docs (#3239) 2026-06-10 00:40:19 +07:00
Reynaldi Chernando a8cfaeefbd Add subdomain and workers github actions deployment (#3226)
* Add subdomain and workers github actions deployment

* minor adjust
2026-06-09 23:35:34 +07:00
Badri467 58cb8ee048 skip github actions backend for docs only changes (#3236) 2026-06-09 23:28:48 +07:00
Daniel Salazar a72fbd1d47 fix: inform acl of full access flag (#3237) 2026-06-09 08:04:36 -07:00
Daniel Salazar f6408db74e fix: access tokens with full accesses allowed through with new middleware gate (#3234) 2026-06-08 18:47:07 -07:00
ProgrammerIn-wonderland df20993d8b set azure openai as the default (#3233) 2026-06-08 20:33:41 -04:00
ProgrammerIn-wonderland 708fe1bc4e azure ai provider (#3232) 2026-06-08 18:31:49 -04:00
Daniel Salazar e782fa832e fix: kv costs (#3231) 2026-06-08 13:32:36 -07:00
ProgrammerIn-wonderland c947c2c989 fix mcp server cloudfront cache bug (#3230) 2026-06-08 15:42:49 -04:00
Daniel Salazar 3de4613b7d Revert "feat (PUT-1091): have gui tokens labeled and be different than reques…" (#3229)
This reverts commit 1335b46404.
2026-06-08 11:15:24 -07:00
Daniel Salazar def7290c48 feat: emit user.deleted event on user deletion (#3228) 2026-06-08 10:30:24 -07:00
Daniel Salazar 1335b46404 feat (PUT-1091): have gui tokens labeled and be different than requestor ones (#3222)
* feat (PUT-1091): have gui tokens labeled and be different than creator ones

* fix: errors with auth token
2026-06-08 09:03:02 -07:00
Reynaldi Chernando 0e15ac98a9 Update cli readme and set version (#3227) 2026-06-08 22:23:32 +07:00
Reynaldi ChernandoandCopilot Autofix powered by AI f127b2509d Add docs fixes for deployment and workers pages (#3223)
* Add docs fixes for deployment and workers pages

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

* handle router edge case

* explicit info about wildcard requiring name

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-06-08 17:43:00 +07:00
Sourabh Sharma 15134c99bf Fix mkdir dedupeName for existing directories (#3215) 2026-06-07 12:49:00 -07:00
Daniel Salazar 8262e3193e perf: don't batch get app stats (#3219) 2026-06-06 18:28:59 -07:00
Reynaldi Chernando 98a21c4ac0 More workers docs improvement (#3218)
* More workers docs improvement

* Add mention of workers alongside mention of user scope
2026-06-06 01:27:24 +07:00
ProgrammerIn-wonderland c52eb1b903 Ns/bug fixes 060426 (#3214)
* Harden backend auth

* Puter.js auth hardening
2026-06-05 13:55:23 -04:00
mohit_parmar_007 d6a71e50bb fix: remove unnecessary cache console.log statements (#3216) 2026-06-05 10:01:59 -07:00
Reynaldi Chernando a4b6ee0dd2 [PUT-1090] Improve worker docs (#3217)
* Improve worker docs

* refine
2026-06-05 21:54:56 +07:00
Daniel Salazar bb61103080 feat: add apps api into mcp (#3213) 2026-06-04 17:19:53 -07:00
Daniel Salazar 9bbc76b2dd drop metering alerts to low severity (#3212) 2026-06-04 14:01:44 -07:00
Daniel Salazar 918aba9c80 fix: reduce alarms further (#3211)
* feat; update bug-bounty

* fix: reduce alarms further
2026-06-04 10:43:50 -07:00
Reynaldi ChernandoandCopilot Autofix powered by AI 3ceba9e3b3 Minor fix deployment docs (#3210)
* Minor fix deployment docs

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-06-04 23:48:25 +07:00
Daniel Salazar a480679a2d vis: make metering service alarm only go off on actual abuse (#3208) 2026-06-03 22:45:18 -07:00
Daniel Salazar e99ae40e93 chore: bump puter-js (#3207) 2026-06-03 18:59:07 -07:00
Daniel Salazar 3cd50fbaf3 move the token migration to root (#3206) 2026-06-03 18:54:22 -07:00
ProgrammerIn-wonderland 60dd1453d4 Fix issues (#3205) 2026-06-03 21:19:33 -04:00
ProgrammerIn-wonderland 07781cd681 allow overriding origin from workers.dev on mcp server (#3204) 2026-06-03 17:48:37 -04:00
ProgrammerIn-wonderland 2387d8dcc9 MCP whoami (#3203) 2026-06-03 17:13:13 -04:00
Reynaldi Chernando 723608f4db Add new puter cli (#3199)
* Add new cli

* update package lock

* fix readme and license
2026-06-04 01:51:31 +07:00
Reynaldi Chernando bd1dcde7b0 Add info for puterjs deployments (#3202)
* Add info for puterjs deployments

* update external links

* hide github actions method
2026-06-04 01:25:57 +07:00
ProgrammerIn-wonderland 5b8e41e66d MCP Server (#3197)
* beginnings of mcp

* oauth support

* add workers and better prompt engineering
2026-06-03 12:23:06 -04:00
Nariman Jelveh acc4796b99 Allow Cache-Control and Pragma headers (#3198) 2026-06-03 14:30:33 +02:00
jelveh 106e85d6ca Force revalidation on FileSystem reads 2026-06-03 14:15:22 +02:00
ProgrammerIn-wonderland 0ffc32f19e add whitelist for signed batch uploads (#3196) 2026-06-02 21:30:34 -04:00
Daniel Salazar d897cc5dd0 refactor: move subscriptions to webhook system (#3193)
* fix: dont sign write urls if they don't have write perm

* wip: new subscriptions

* refactor: move subscriptions to
2026-06-02 17:58:56 -07:00
Daniel Salazar 6d2f277ce2 fix: some error handling (#3190) 2026-05-31 18:58:27 -07:00
ProgrammerIn-wonderland 86f80b29c0 tighten RAO policy (#3188) 2026-05-29 20:52:21 -04:00
Daniel SalazarandProgrammerIn-wonderland 39243735e8 feat (put-1049): worker types (#3185)
* feat (put-1049): worket types

* Implement changes to worker types

---------

Co-authored-by: ProgrammerIn-wonderland <[email protected]>
2026-05-29 20:51:57 -04:00
ProgrammerIn-wonderland b3ff9d74b7 add browserjs to taskbar and RecommendedAppService (#3189) 2026-05-29 20:32:53 -04:00
ProgrammerIn-wonderland 7b7604440f Add serverless only filtering for togetherai chat models (#3187) 2026-05-29 14:33:55 -04:00
ProgrammerIn-wonderland 382d2b3c14 add minimax provider (#3171)
* add minimax provider
2026-05-29 13:36:10 -04:00
Anurag Pappula eda5379c17 fix(docs/search): handle search index loading and failure states (#3184)
* fix(docs/search): show loading state and replay query when index isn't ready

Previously performSearch rendered "No results found" while the
MiniSearch index was still loading, which is misleading on slow
networks. Track the last query as pendingQuery and replay it from
fetchSearchIndex once the index is ready; render a distinct
"Failed to load search index" message on fetch error so the
loading message doesn't persist indefinitely.

Fixes #3180.

* fix(docs/search): close stale-replay race and consolidate status messages

Drop pendingQuery synchronously in the input handler when the query
falls below the minimum length, so a fetchSearchIndex that resolves
within the 300ms debounce window after the user clears the field can
no longer replay the cleared query into an empty input.

Move the idle, loading, and failed copy into a single
renderSearchStatus(kind) helper so the four .search-no-results
renderings can only drift in one place.
2026-05-29 16:00:59 +07:00
Ron HernausandDaniel Salazar d2fee51844 Support PostgreSQL database backend (#3167)
* feat(database): add postgres database client

Adds the PostgreSQL database client, native bootstrap migration, SQL preparation helpers, and database config/factory wiring.\n\nRefs #3165.

* feat(database): make backend queries postgres-aware

Updates runtime SQL call sites for database-specific booleans, identifiers, insert-ignore, upserts, JSON extraction, intervals, and Postgres insert ids.\n\nRefs #3165.

* test(database): cover postgres client behavior

Adds unit coverage for SQL preparation, factory selection, write-result mapping, and transaction rollback/commit ordering, plus an env-gated PostgreSQL integration flow.\n\nRefs #3165.

* docs(self-hosting): document postgres database setup

Adds PostgreSQL configuration examples and migration path guidance for self-hosted deployments.\n\nRefs #3165.

* fix: harden postgres oidc tests

* fix(postgres): normalize query results and SQL prep

* fix(user): preserve normalized cache booleans

* test(postgres): run integration coverage with pgmock

* tests: add way to run all tests with postgres though slow

Also adding note that postgres is not in active use so might not work out the box

---------

Co-authored-by: Daniel Salazar <[email protected]>
2026-05-28 14:12:17 -07:00
Daniel Salazar 8ac4bad196 fix: readUrl perm check (#3183) 2026-05-28 11:44:16 -07:00
Reynaldi Chernando f140d7221c Add opus 4.8 (#3182)
* Add opus 4.8

* update alias

* handling opus 48 omit temperature
2026-05-28 13:27:46 -04:00
Daniel Salazar 631f0bda1b chore:deploy latest puter.js (#3181) 2026-05-28 09:58:33 -07:00
Reynaldi Chernando 1121314582 Docs search improvement (#3179)
* Init search improvement

* Add fuzzy implementation

* update package lock

* package lock matched with runner 24.16.0
2026-05-28 23:11:34 +07:00
Daniel Salazar dda6b82f0f fix: missing english i18n (#3176) 2026-05-27 16:31:40 -07:00
ProgrammerIn-wonderland 6c757c3fc8 Don't use passed in authorization in workers (#3175) 2026-05-27 18:09:59 -04:00
Daniel Salazar eba194936e tests: more tests for session and auth (#3174) 2026-05-27 12:32:15 -07:00
Daniel Salazar a7bdac16da tests: Add unit tests for XAISpeechToTextDriver (#3173)
closes #3002
2026-05-27 11:49:16 -07:00
Daniel Salazar 716a8d06e2 fix: session store kind column + duplicate error handling (#3172) 2026-05-27 11:44:08 -07:00
Daniel Salazar e95cf44fec fix: small fixes for perf and username checks (#3169) 2026-05-27 03:10:56 -07:00
Daniel Salazar 8f8efcb349 feat (PUT-1025 PUT-1026 and PUT-1017): better handling old token auth required in gui (#3166)
* feat (PUT-1025 PUT-1026 and PUT-1017): better handling old token forcing in gui

* fix: misc issues with token inval project
2026-05-27 02:26:35 -07:00
Daniel Salazar b188942436 feat (PUT-1016 & PUT-1020) (#3164)
* feat (PUT-1016 & PUT-1020)
temp account preservation on forced relogin
hosted asset cookies to v2 token too

* fix: remove llm dashes and ugly comments

* update agents
2026-05-26 23:35:16 -07:00
Daniel Salazar 9b7e9a98c8 fix: session revocation (#3163) 2026-05-26 22:13:12 -07:00
Daniel Salazar d6f7fb6155 chore: package-lock update (#3161) 2026-05-26 20:33:11 -07:00
Daniel Salazar bd91f5e192 feat: worker sessions get their own kind + per-(user, app, worker_name) row (#3160)
* feat: worker sessions get their own kind + per-(user, app, worker_name) row

Schema
------
- mysql_mig_11.sql + sqlite 0054: add idx_sessions_user_worker_active,
  a partial unique index over (user_id, app_uid, meta.worker_name) for
  kind='worker' rows. Active worker sessions are deduped by that triple
  so each named worker gets its own session row and they don't fight
  the existing idx_sessions_user_app_active (which still constrains
  kind='app' only). app_uid is allowed NULL for user-scoped workers
  with no app binding.

SessionStore
------------
- getOrCreateWorker(userId, { appUid, workerName, ... }): mirrors the
  getOrCreateApp pattern — cache lookup, partial-unique re-SELECT on
  insert-ignore, all keyed on the worker triple. expires_at lands at
  WORKER_WINDOW_SECONDS (~99y) so the worker doesn't have to re-mint
  on any cadence.
- #cacheKeyWorker + #allCacheKeysForRow worker branch so revoke /
  update invalidates the worker cache view alongside the by-uuid one.

AuthService
-----------
- createWorkerSessionToken(user, workerName, meta?) now takes the
  workerName explicitly and routes through getOrCreateWorker. Emits
  the same { session, token, gui_token } shape but both JWTs carry
  { worker: true, worker_name }.
- createWorkerAppToken(actor, appUid, workerName) likewise — JWT
  carries the worker_name claim so a verifier can tell two workers
  under the same app apart without a DB round-trip.
- Both methods 400 on empty workerName.

WorkerDriver
------------
- Five auth-mint call sites swapped over: app-bound deploy (3x:
  appId branch, actor.app fallback, hot-reload redeploy), user-bound
  fallback (2x: cold deploy, hot-reload). All pass `workerName` so
  the worker's session row is naturally idempotent across redeploys.

GUI manage-sessions
-------------------
- sessionTitle adds a kind='worker' branch ("name (app)" for
  app-scoped workers, just "name" for user-scoped), pulling worker_name
  from the meta-spread that listSessions already surfaces.
- en.js adds ui_session_kind_worker.

* fix(workers): MySQL JSON_EXTRACT quoting + revoke cache invalidation +
SQLite NULL-distinct in worker index

Three real bugs in the worker session plumbing from the prior commit,
plus a misleading comment. Schema design kept (worker_name lives in
`meta` rather than a dedicated column) per offline review:

1. `#selectWorkerRow` compared `JSON_EXTRACT(meta, '$.worker_name')`
   directly to a bind parameter. MySQL's `JSON_EXTRACT` returns a
   JSON-typed value with embedded quotes (`"name"`, not `name`), so
   the comparison never matched. After the first INSERT, every
   follow-up getOrCreateWorker call missed the existing row in the
   SELECT, hit INSERT-IGNORE, then missed again in the re-SELECT —
   the caller would receive whatever the INSERT-IGNORE returned (a
   no-op row in conflict cases). Wrap with `JSON_UNQUOTE` on MySQL
   via `db.case`; SQLite's `json_extract` already returns the
   unwrapped scalar so it keeps the literal form.

2. mig_11's generated `worker_unique_key` had the same JSON-quoting
   bug. Mirror the fix: `IFNULL(JSON_UNQUOTE(JSON_EXTRACT(...)), '')`
   so the concatenated unique key is a plain string that lines up
   with what `#selectWorkerRow` now binds against.

3. SQLite UNIQUE indexes treat NULL columns as distinct (per the SQL
   standard), so two user-scoped workers (app_uid NULL) with the same
   worker_name would both insert. Wrap the index expression with
   `IFNULL(app_uid, '')` so they correctly conflict — matches the
   MySQL side's `IFNULL` in the generated column.

4. `removeByUuid` / `revokeCascade` SELECTed only the identity
   columns (no `meta`), so `#allCacheKeysForRow`'s worker branch
   couldn't read `meta.worker_name` and the composite
   `sessions:v2:worker:<user>:<app>:<name>` cache key survived
   revocation. Up to CACHE_TTL_SECONDS (15min) afterwards,
   getOrCreateWorker would short-circuit to the cached (revoked) row.
   Add `meta` to both SELECTs; the existing meta-parsing logic in
   `#allCacheKeysForRow` handles the rest.
2026-05-26 20:05:31 -07:00
Daniel SalazarandClaude Opus 4.7 971a1b5071 chore: drop linear/issue ticket references from source comments (#3159)
54 references across 18 files (PUT-1010, PUT-1014, PUT-1019, PUT-1021,
PUT-1022, PUT-1023, PUT-1024 + sub-tags AUTH-2/4/5, SDK-1, PJS-1/2,
GUI-1/2, ROLLOUT-1) removed from inline comments, doc comments,
test describe blocks, and SQL migration headers. The substantive
explanations stay; only the ticket pointers go.

No behavior change. Full backend test suite: 2172 passed / 16 skipped.

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-26 18:50:43 -07:00
Daniel SalazarandClaude Opus 4.7 ac5eecb7f3 feat (put-1019 put-1021): v2 auth revoke endpoints + silent v1->v2 to… (#3158)
* feat (put-1019 put-1021): v2 auth revoke endpoints + silent v1->v2 token migration

PUT-1019 (AUTH-5): full revoke-endpoint coverage
- /logout: soft-revoke web session + its asset cookies via revokeCascade
  (app sessions and access tokens survive)
- POST /auth/revoke-session: cascade per row kind (web/app/access_token/asset)
- POST /auth/revoke-all-sessions: revoke all web rows for user; optional
  include_apps=true nuclear option; gated by userProtected (cookie-only)
- revokeAccessToken: soft-revoke matching access_token row in addition to
  removing access_token_permissions
- All revokes are UPDATE revoked_at = now(); no DELETE statements remain

PUT-1021 (SDK-1): backend POST /auth/migrate-token
- v1 access_token/app -> mint matching-kind v2 token, idempotent on
  (auth_id, kind, token_uid)
- v1 web/session -> 409 { code: "reauth_required" } (interactive relogin only)
- Same-origin / signed-referer hardening; rate-limited per IP and auth_id
- Gated by auth.allow_v1_tokens; emits puter_token_v2 cookie for app-in-browser

DB migrations: mysql_mig_10, sqlite 0053 (sessions.access_token_uid column)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(put-1019): reject self-revoke; enrich list-sessions response

- handleRevokeSession refuses uuid === req.actor.session.uid (use /logout
  instead). The cookie that authenticated the call should never be the
  target of a self-revoke — the response can't write fresh auth state
  and the client ends up with an ambiguous identity. revoke-all-sessions
  still has the explicit include_current opt-in for the nuclear case.

- AuthService.listSessions now joins the apps table for kind='app' rows
  (returning { uid, name, title, icon } so the manage-sessions UI can
  render the authorizing app without a second round trip), surfaces
  kind / expires_at / label / last_ip / created_via, and filters out
  asset rows (per-cookie children of web rows, revoked transitively via
  cascade — surfacing them as standalone entries would be confusing).

- Sort order: current session first, then most-recently-active. UI
  relies on this to anchor "you are here" at the top.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(put-1019 put-1021): review nits — origin normalization, cookie fallback, types, migration order

B1: createAccessToken.options.expiresIn widened to string | number.
    The impl (#hardExpiryFromExpiresIn) and existing callers/tests use
    jsonwebtoken-style strings ('1h', '30d'); narrowing to number forced
    unsafe casts at every call site. Cast at the single sign() boundary
    where jsonwebtoken's typed template-literal SignOptions clashes
    with the wider runtime contract.

B2: Inline comment on the DELETE in access_token_permissions. The
    AUTH-5 "no DELETE on revoke" rule scoped to the `sessions` table
    (where the cascade graph + audit trail matter). Permissions rows
    are the grant manifest for an active token — once its session is
    soft-revoked they're dead-weight cache entries. A future audit
    requirement would land as a `revoked_at` column on this table,
    not a behavior change in this PR.

B3: handleMigrateToken now sets the puter_token_v2 cookie (with the
    shared sessionCookieFlags + httpOnly) when the migration result
    is kind='app'. The endpoint is already gated on Origin so the
    caller is by definition in a browser; access tokens deliberately
    skip the cookie since they're programmatic.

B4: #isMigrateTokenOriginAllowed normalizes both incoming origin and
    config.origin / allowlist entries (trim + strip trailing slash +
    lowercase) before equality. A misconfigured `config.origin =
    "https://puter.com/"` would otherwise reject every same-origin
    browser call.

B5: Replaced 4x `this.config.cookie_name!` non-null assertions in
    AuthController with `(this.config.cookie_name ?? 'puter_token')`.
    IConfig is `Partial<IConfigOptional>` so cookie_name is undefined
    at runtime in some deployments / test setups; the fallback matches
    the pattern in userProtected / OIDCController / puterSite.

B6: MySQLDatabaseClient sorts migrations numerically by trailing
    integer instead of lexically. Existing files use unpadded names
    (`mysql_mig_<N>.sql`), so plain `.sort()` ran mysql_mig_10 before
    mysql_mig_2 — a future migration that depended on _2..9 running
    first would break. Non-numeric filenames fall through to
    localeCompare for determinism.

B7: Restored the docstring for SessionStore.getOrCreateApp's
    `opts.auth_id` ("Stable per-user identity (survives re-login);
    carried on every v2 JWT so manage-sessions can group by identity")
    — the previous edit truncated it to a fragment.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* test + feat: backend test coverage for PUT-1019/1021 review fixes + worker session methods

Tests
-----
- compareMigrationFilenames (new) covering B6 numeric-sort: confirms
  mysql_mig_10.sql lands after mysql_mig_9.sql; non-numeric files sort
  after numbered ones; stable for already-ordered input.
- listSessions (AuthService.test.ts): excludes kind="asset" rows;
  enriches with kind/expires_at/last_ip/created_via; joins kind="app"
  rows with the apps table; sorts current first then by last_activity
  desc.
- handleRevokeSession (AuthController.test.ts): refuses self-revoke
  with 400; still allows revoking a sibling session.
- handleMigrateToken (AuthController.test.ts): rejects missing/
  disallowed Origin; tolerates trailing slash and uppercase Origin
  (B4 normalization); returns 409 reauth_required for v1 web tokens;
  does NOT set the cookie for access-token migration; DOES set the
  puter_token_v2 cookie (httpOnly + sessionCookieFlags) for
  app-under-user migration.
- SessionStore tests updated to import APP_WINDOW_SECONDS /
  WEB_WINDOW_SECONDS rather than hardcoded 30/90 day values — the
  windows just got bumped to 1y and the assertions need to follow
  the constant.

Refactor
--------
- MySQLDatabaseClient exports compareMigrationFilenames so the sort
  logic is unit-testable in isolation.

Worker tokens
-------------
- AuthService.createWorkerSessionToken(user, meta?): mints a new
  kind="web" row tagged meta.worker=true, expires_at =
  WORKER_WINDOW_SECONDS, returns { session, token, gui_token }
  with worker: true on each JWT.
- AuthService.createWorkerAppToken(actor, appUid): mints a new
  kind="app" row tagged meta.worker=true, expires_at =
  WORKER_WINDOW_SECONDS, returns an app-under-user JWT with
  worker: true. Note the existing idx_sessions_user_app_active
  unique index will collide with an existing non-worker app
  session for the same (user, app) — future schema work can
  carve workers out of that uniqueness.

SessionStore.js: WEB/APP_WINDOW_SECONDS now 1y;
WORKER_WINDOW_SECONDS = 99y added for the worker path.

Full backend suite: 2172 passed / 16 skipped / 0 failed.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-26 18:49:36 -07:00
Daniel SalazarandClaude Opus 4.7 3ae076b73e feat (put-1022 put-1023 put-1024): puter.js reauth handling + GUI v2 modal + silent v1 migration (#3157)
* feat (put-1022 put-1023 put-1024): puter.js reauth handling + GUI v2 modal + silent v1 migration

PUT-1022 (PJS-1): puter.js handles 401 reauth_required
- Single apiCall wrapper intercepts 401 { code: "reauth_required" }
- Clears local token, emits puter.auth.reauth_required { reason, auth_id }
- Queues in-flight requests, re-issues after re-auth completes
- web/app: opens puter.com login popup forwarding auth_id
- gui: no-op (handled by GUI-1 modal)
- workers: surfaces as structured exception
- All token writes go through setAuthToken() (future-proofed for PJS-2)

PUT-1023 (GUI-1): web GUI reauth modal + v2 token storage
- Detects 401 reauth_required across http + websocket connect paths
- Soft modal preserves URL/window state; auth_id forwarded into login form
- Storage key migrated: auth_token -> auth_token_v2
- Cookie cleared on logout: puter_token (legacy) cleared, backend writes puter_token_v2
- Cross-tab propagation via localStorage storage events
- WebSocket revocation surfaces as same modal (no silent failure)

PUT-1024 (PJS-2): puter.js storage versioning + silent v1->v2 migration
- New storage key puter.auth.token.v2
- On SDK init: prefer v2 key; else legacy v1 -> silent POST /auth/migrate-token (SDK-1)
- On success: store v2, clear v1. On failure: fall back to PJS-1 reauth flow
- URL-param tokens (?puter.auth.token=, ?auth_token=) also run through silent migration
- Logout clears both keys; setAuthToken writes v2 only

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(put-1023): manage-sessions polish — hide self-revoke, fix modal stack, richer rows

Three issues surfaced during local testing of the manage-sessions UI:

1. The session row representing the caller's own cookie had a Revoke
   button that, if used, left the client in an ambiguous identity state
   (backend now rejects it too — see put-1019 backend fix). The button
   is now omitted entirely when session.current=true; /logout remains
   the right path for ending the active session.

2. The confirm-revoke prompt rendered behind the manage-sessions
   window because both share the dominant z-index pool. UIAlert calls
   now pass parent_uuid (the manage-sessions window's data-element_uuid)
   plus stay_on_top, so the prompt stacks above its parent.

3. Each row only showed the bare uuid. Now renders: title (app
   title for app sessions, label / "Browser session" / "Access token"
   otherwise), app icon when applicable, kind / current badges,
   created / last-active / expires (timeago, with absolute on hover),
   and last_ip. App metadata is joined server-side per the backend
   listSessions change.

Adds en.js strings for the new labels (ui_session_*).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(put-1022): tighten reauth replay + iframe postMessage hygiene

driverCall_ replay (G1): on 401 reauth_required, re-enter driverCall_
instead of calling replayXhrAfterReauth. The generic helper wires the
retried XHR through setupXhrEventHandlers which resolves with the
parsed response and silently skips driverCall_'s streaming detection,
usage-limit / email-confirmation handling, settings.transform, and
resp.result unwrapping — i.e. it would change the driver call API
contract on retry. One-shot via settings._reauthReplayed so a
fresh-token rejection bubbles up instead of looping.

responseType drift (G2): driverCall_ sets xhr.responseType from
settings AFTER initXhr (which captured the stale value into
xhr._puterReq). Mirror the mutation onto _puterReq so any replay path
builds the retry with the live config rather than the snapshot.

targetOrigin lockdown (G3): triggerReauth's parent.postMessage now
targets this.defaultGUIOrigin instead of '*'. The payload carries
reauth metadata + auth_id and is only meaningful to the GUI; '*'
would leak the signal to whatever frame happened to be embedding us.

event.source pinning (G4): the reauth wait listener also matches
event.source against globalThis.parent, not just event.origin —
origin alone admits any same-origin frame on the GUI domain.

event name alignment (G5): the SDK now emits and listens for
'puter.auth.reauth_required' (matches the documented name in the
PR / commit description). The old 'auth.reauth_required' key isn't
yet consumed externally so this is a safe rename.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-26 18:49:17 -07:00
Daniel Salazar 5394ccc45c feat: add verification for v2 auth (#3155) 2026-05-26 15:34:54 -07:00
Daniel Salazar 3649109f40 fix: add back AGENTS.md (#3154) 2026-05-25 16:01:35 -07:00
Daniel Salazar 0ff0ea7743 fix: fs event emissions (#3153) 2026-05-25 14:25:27 -07:00
Daniel Salazar 8be115096f feat (put-1012 put-1014): new tokens version signature and logic (#3152) 2026-05-25 14:19:23 -07:00
Daniel Salazar e66fd2373f feat: start adding support for token expiry and invalidation (#3151)
* feat: new expirable + revokable session

* fix: msql migration
2026-05-25 12:30:20 -07:00
jelveh 0e2cce152d Update PuterAlert.js 2026-05-24 17:03:34 -07:00
Daniel Salazar 101e0bc930 fix: search leaking metadata (#3150) 2026-05-24 16:21:33 -07:00
jelveh ea078250bf Strip non-essential media from docs' markdown 2026-05-24 16:15:24 -07:00
jelveh db79f8e484 Add note: serve Puter.js via HTTP server 2026-05-24 15:48:21 -07:00
Daniel Salazar 6422f2d513 fix: associated app ids as input (#3149) 2026-05-23 13:01:03 -07:00
ProgrammerIn-wonderland c60626f772 Add preamble versioning (#3147)
* Add preamble versioning

* update migrations index
2026-05-23 01:15:58 -04:00
ProgrammerIn-wonderland cafdaf7b50 fix bug with signed url directory listings (#3146) 2026-05-22 18:48:41 -04:00
Daniel Salazar bb0f56d758 fix: bring back puter site config logic (#3145) 2026-05-22 11:15:45 -07:00
NighttandReynaldi Chernando 7438db7a90 Add OpenRouter model release dates (#3144)
* Add OpenRouter model release dates

* simplify implementation

---------

Co-authored-by: Reynaldi Chernando <[email protected]>
2026-05-22 12:58:59 -04:00
Daniel Salazar 948f771838 feat: ai limits tightning and add severity alert (#3142) 2026-05-21 20:46:19 -07:00
Nariman Jelveh 82e34f014d Warn on file:// loads using PuterDialog 2026-05-21 19:15:21 -07:00
Nariman Jelveh 9e4c3f8ba3 Merge branch 'main' of https://github.com/HeyPuter/puter 2026-05-21 14:39:44 -07:00
Nariman JelvehandClaude Opus 4.7 e6acd7386a Bump @heyputer/puter.js to 2.4.2
Record the 2.4.2 release in the repo. The version was published to npm
(via prepublishOnly's `npm version patch`) but the bump was never
committed, leaving package.json stuck at 2.4.1 on GitHub.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-21 14:39:18 -07:00
Daniel Salazar b04e8b046a fix: fixing misc issues around alerting and bugs (#3141)
- decrease ai related alerts and make errors not 500
- decreased bad fs controller checks to be 404s instead of 500s
- force bucket regions for now
- validate webdav perms for locking/unlocking
2026-05-21 14:26:47 -07:00
Nariman Jelveh 3262b984b7 Unify Email/Usage dialogs with Puter dialog style 2026-05-21 14:26:26 -07:00
Nariman Jelveh 71af88c73c Bump @heyputer/puter.js to 2.4.1 2026-05-21 14:06:48 -07:00
Nariman Jelveh 2275f65169 Centralize auth popup handling and consent dialog 2026-05-21 12:06:47 -07:00
ProgrammerIn-wonderland 71b7d6367f fix together provider issues (#3140) 2026-05-21 08:58:48 -04:00
Nariman Jelveh 08369d840b Add cssstyle to docs dependencies (#3139) 2026-05-20 17:41:02 -07:00
Daniel Salazar 94236133b5 fix: event types (#3138)
* fix: agents.md

* fix weird names

* fix: weird names again

* fix: event types

* cleanup types
2026-05-20 16:26:16 -07:00
Daniel Salazar 84b48541bd fix: event types + emit subdomain serve event (#3137)
* fix: agents.md

* fix weird names

* fix: weird names again
2026-05-20 12:47:56 -07:00
Daniel Salazar a33d02f522 fix: agents.md name (#3136) 2026-05-20 12:18:35 -07:00
Reynaldi Chernando f94d4aa7d9 Update gemini 3.5 flash release date (#3135) 2026-05-20 13:25:46 -04:00
Daniel Salazar 89cfb0c634 fix: subdomain htmls (#3134) 2026-05-19 22:04:57 -07:00
ProgrammerIn-wonderland c9096e9eed more validation for signed URLs (#3133) 2026-05-19 19:04:42 -04:00
Daniel Salazar 859ca207c5 feat: new gemini 3.5 flash model (#3131) 2026-05-19 15:16:07 -07:00
Daniel Salazar 58fc122f0d fix: app creation dupped names limit (#3130)
* fix: app creation dupped names limit

* fix: app dupped names
2026-05-19 12:18:59 -07:00
ProgrammerIn-wonderland 0e990da961 import ESM by URL instead of path (#3129) 2026-05-18 19:31:03 -04:00
db8236cc97 update Indonesian section (Indonesian language) (#3117)
* Update id.js

* Update id.js

* Update id.js

* Update id.js

* Update README.id.md

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

* refine indonesia i18n

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
Co-authored-by: Reynaldi Chernando <[email protected]>
2026-05-18 17:15:01 +07:00
jelveh d731ccf873 Bump puter.js version to 2.3.4 2026-05-17 23:20:41 -07:00
Daniel Salazar 6493d66901 fix: misc write issues with fs + openrouter refresh (#3125)
* feat: refresh openrouter models every 15 min

* fix: if no extensions, skip build step

* fix: misc write issues with fs
2026-05-17 12:34:08 -07:00
Daniel Salazar 9e7f38f473 fix: validate private apps token too (#3124) 2026-05-17 12:28:32 -07:00
Daniel Salazar a568ca0f99 fix: expose upstream ai errors as not 500s (#3123) 2026-05-17 12:01:22 -07:00
Nariman Jelveh 41e7228a9f Update PuterDialog.js 2026-05-16 17:01:40 -07:00
Nariman Jelveh 4568639718 Adjust PuterDialog footer and button spacing 2026-05-16 16:57:35 -07:00
Nariman Jelveh 73ecff0bd7 Tweak PuterDialog spacing and button widths 2026-05-16 16:50:31 -07:00
Nariman Jelveh f2f023880f Remove mobile column-reverse for dialog buttons 2026-05-16 16:42:27 -07:00
Nariman Jelveh f2093e9204 PuterDialog: UI/style tweaks and button reorder 2026-05-16 16:36:53 -07:00
Nariman Jelveh 23497dd371 Add dark mode for puter.js login prompt 2026-05-16 16:04:29 -07:00
Nariman Jelveh 37619fb2a3 Remove embedded background image from PuterDialog 2026-05-16 15:43:15 -07:00
499 changed files with 49794 additions and 8639 deletions

No files matched your search

+2
View File
@@ -3,6 +3,8 @@ name: Backend Tests
on:
pull_request:
types: [opened, synchronize, reopened]
paths-ignore:
- 'src/docs/**'
permissions:
contents: read
+27
View File
@@ -0,0 +1,27 @@
name: Docs Build
on:
pull_request:
paths:
- 'src/docs/**'
jobs:
build-docs:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '24'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Build docs
run: npm run build
working-directory: src/docs
+2
View File
@@ -70,3 +70,5 @@ coverage/
undefined
servers.json
config.*.json
volatile/
+5 -1
View File
@@ -64,7 +64,11 @@ Keep comments light. Prefer self-documenting code — clear names, small functio
- The _why_ is non-obvious (a hidden constraint, a workaround, a subtle invariant).
- A non-trivial usage detail would otherwise trip the next reader.
Don't restate what the code already says. Don't write comments that reference the current task or PR — those rot.
Don't restate what the code already says. Don't write comments that reference the current task or PR or version of something — those rot.
**No ticket references in code or test comments.** Don't write `PUT-1234`, `AUTH-3`, `GUI-2`, `// fix for FOO-99`, `// per ROLLOUT-1`, or section headers like `// -- PUT-1020: ... --`. The ticket exists in git history and the issue tracker; the code should describe the _why_ in domain terms ("logout cascade transparently invalidates...") not project-management terms ("the cascade from AUTH-3..."). Same rule for test-block names and section dividers — describe what's being tested, not which ticket asked for it.
**Use plain ASCII `-` in comment section dividers, not box-drawing characters.** Write `// -- Section name --`, not `// ── Section name ──`. The `─` (U+2500) character renders inconsistently in some editors and diff tools and adds nothing over `-`.
### Tests
+34 -1
View File
@@ -17,18 +17,51 @@ Out-of-scope:
* Denial of Service (DoS), spam, or volumetric attacks.
* Physical security issues.
## Known Non-Issues (Please Check Before Submitting)
The following have already been reviewed and determined **not to be vulnerabilities**. Reports that only re-describe one of these are **not eligible for a reward** and will be closed as non-issues — even if they include new code references. Please review this list before submitting:
* **XSS / CORS / token issues scoped to `api.puter.com`.** The primary user session cookie lives on `puter.com`, not on the API origin, so reflected/stored XSS, CORS, or token handling on `api.puter.com` is generally out of scope. Two exceptions we *do* evaluate on their merits — please report these: (a) attacker-controlled content served **inline** (e.g. an HTML `Content-Type`) from API file/response endpoints, and (b) anything that abuses the app-scoped `puter_token_v2` companion cookie set on the API host.
* **SSRF via `secureFetch`.** Production routes outbound requests through an isolated proxy that has no access to internal/SSRF-sensitive resources.
* **Attacks that depend on guessing an `appInstanceID` or app UID.** These are random 128-bit secret values and are not considered guessable.
* **Apps invoking drivers, creating workers, or using KV.** Applications are intended to do this; worker permissions are scoped to the owning app. This is by design.
* **App metadata or app user-count "leaks".** This information is currently public by design.
* **General "token in a URL" / token-lifetime designs** — signed directory URLs exposing children, a write signature implying read, or app tokens outliving a web session. These are current intended behaviors.
* **Missing PKCE, unverified `id_token` signatures, or other OIDC hardening.** Provider tokens are obtained through a server-to-server authorization-code exchange with the provider's token endpoint over TLS, so the resulting `id_token` / userinfo claims are trusted from that channel rather than from local JWKS signature verification — the callback never accepts a caller-supplied `id_token`. Adding local signature / `aud` / `iss` / `exp` checks is welcome defense-in-depth (please open a GitHub issue/PR), but their absence is not an account-takeover vector on its own.
* **JWT "algorithm confusion" / unpinned `algorithms` in `jwt.verify`.** Puter's session tokens are HMAC-signed (HS256) with a server-side secret, and there is no asymmetric public key anywhere in the verification path, so `alg`-substitution attacks do not apply. Explicitly pinning `algorithms` is a fine hardening PR, but it is not a vulnerability.
* **Static-source "SQL injection" in internal pagination.** Findings such as `LIMIT ${limit}` in list/notification queries: the limit is numerically coerced and clamped before it reaches the query, so it is not reachable with attacker-controlled string input. Hardening PRs to the internal stores are welcome, but these are not exploitable as reported.
* **Deprecated `saveTo*` GUI app messages.** The legacy `saveToDesktop` / `saveToDocuments` / etc. app-IPC handlers can create — never overwrite — new files in standard user folders. The behavior is non-destructive, path-traversal-safe, and deprecated (slated for removal); it is not treated as a sandbox escape.
* **Best-practice suggestions** such as login/registration username enumeration (kept intentionally for UX) or unauthenticated unsubscribe links (industry norm).
* **Rate-limiting suggestions for TURN credential issuance** (intentional; not billed per tunnel).
If you believe you have a **genuinely new** exploit chain that defeats one of these rationales (for example, demonstrating a sensitive credential that really is reachable on `api.puter.com`), say so explicitly and show why the reasoning above does not apply.
## Rules of Engagement
To participate, you must:
1. **Report responsibly**: Provide detailed steps to reproduce the issue, including proof-of-concept code or screenshots where applicable.
1. **Report responsibly**: Provide detailed steps to reproduce the issue, including proof-of-concept code, screenshots, or a screen recording (see *Proof of Reproduction* below).
2. **Do no harm**: Do not exfiltrate, modify, or delete data. Only access your own account or test data.
3. **Respect availability**: Do not perform denial-of-service attacks or automated scans that degrade service.
4. **Follow disclosure policy**: Do not publicly disclose vulnerabilities until we have confirmed and patched the issue.
5. **Act in good faith**: Make every effort to avoid privacy violations, destruction of data, and interruption or degradation of services.
6. **Check the Known Non-Issues list**: Reports matching an item in the "Known Non-Issues" section above are not eligible and will be closed.
Reports that do not meet these guidelines may not be eligible for a reward.
## Proof of Reproduction
Reports must demonstrate a **working, reproducible exploit with real impact** — not a theoretical or static-source-review finding. Please include:
* Exact steps to reproduce, the relevant request/response or code path, and the commit or version you tested.
* The **observed** result versus the **expected** result.
* For client-side, UI, or authentication-flow bugs: a short screen recording (≤ 2 minutes) showing the exploit working end-to-end on a real Puter instance.
* For server-side bugs: a runnable proof-of-concept.
Reports based solely on reading the source ("source review only, not tested") or unverified AI/LLM-generated reports are the **lowest triage priority and are generally not eligible**. If you used an AI tool to help find an issue, you must personally verify that it actually reproduces before submitting.
Please submit **one issue per report**. Bundled "audit packs" of many speculative findings will be declined; send each confirmed issue separately.
## Reporting Process
To report a vulnerability, email us at: **[[email protected]](mailto:[email protected])**.
+1
View File
@@ -0,0 +1 @@
FOLLOW ./AGENTS.md
+3
View File
@@ -6,6 +6,8 @@
"domain": "puter.localhost",
"cookie_name": "puter_auth_token",
"jwt_secret": "dev-jwt-secret-change-me",
"jwt_secret_v2": "dev-jwt-secret-v2-change-me",
"allow_v1_tokens": true,
"url_signature_secret": "dev-url-signature-secret-change-me",
"allow_all_host_values": true,
"allow_no_host_header": true,
@@ -21,6 +23,7 @@
"default_user_group": "78b1b1dd-c959-44d2-b02c-8735671f9997",
"default_temp_group": "b7220104-7905-4985-b996-649fdcdb3c8f",
"storage_capacity": 104857600,
"disable_user_signup": false,
"strict_email_verification_required": false,
"gui_assets_root": "./src/gui",
"puterjs_root": "./src/puter-js/dist",
+33 -4
View File
@@ -67,10 +67,17 @@
// ── Auth / session ──────────────────────────────────────────────────
// ALWAYS replace these for any public install — `openssl rand -hex 64`.
// `jwt_secret` is legacy verify-only; new tokens sign with `jwt_secret_v2`.
"jwt_secret": "change-me",
"jwt_secret_v2": "change-me",
// Set false to retire v1 tokens entirely (ROLLOUT-1).
"allow_v1_tokens": true,
"url_signature_secret": "change-me",
"cookie_name": "puter_auth_token",
"min_pass_length": 6,
// When true, anonymous users must log in instead of creating temp or
// permanent accounts.
"disable_user_signup": false,
"allow_system_login": false,
"strict_email_verification_required": false,
"captcha": {
@@ -155,17 +162,30 @@
// ── Database ────────────────────────────────────────────────────────
"database": {
// `sqlite` for single-node / dev; `mysql` for self-host with MariaDB.
// `sqlite` for single-node/dev, `mysql` for MariaDB/MySQL,
// or `postgres` for PostgreSQL.
//
// NOTE: `postgres` support is a community contribution and is not
// exercised by Puter.com production. It boots, passes its own
// integration tests against pgmock, and runs the common user/app/
// fsentry/session/permission/OIDC flows — but less-traveled code
// paths may surface MySQL/SQLite-isms that haven't been ported yet.
// Expect rough edges and please file issues if you hit one. For
// production self-hosting today, `mysql` (MariaDB) and `sqlite` are
// the supported defaults.
"engine": "sqlite",
// sqlite — file path on disk
"path": "volatile/runtime/puter-database.sqlite",
"targetVersion": 0,
// mysql — connection details
// mysql/postgres — connection details. PostgreSQL defaults to port
// 5432 when `engine` is `postgres`; MySQL/MariaDB normally use 3306.
"host": "",
"port": 3306,
"user": "",
"password": "",
"database": "",
// postgres may also use a URL instead of host/user/password fields:
// "connectionString": "postgres://puter:secret@localhost:5432/puter",
// Optional read-replica pool. Reads route here when populated.
"replica": {
"host": "",
@@ -174,9 +194,10 @@
"password": "",
"database": ""
}
// mysql self-host bootstrap: set `migrationPaths` to apply the bundled
// schema on first boot. Idempotent — safe to leave on.
// mysql/postgres self-host bootstrap: set `migrationPaths` to apply the
// bundled schema on first boot. Idempotent — safe to leave on.
// "migrationPaths": ["./src/backend/clients/database/migrations/mysql"]
// "migrationPaths": ["./src/backend/clients/database/migrations/postgres"]
},
// ── DynamoDB (KV store) ─────────────────────────────────────────────
@@ -242,12 +263,20 @@
// ─ Chat / completion ─
"claude": { "apiKey": "" },
"openai-completion": { "apiKey": "" },
"azure-openai": {
"apiKey": "",
"apiURL": ""
},
"gemini": { "apiKey": "" },
"groq": { "apiKey": "" },
"deepseek": { "apiKey": "" },
"mistral": { "apiKey": "" },
"xai": { "apiKey": "" },
"moonshot": { "apiKey": "" },
"minimax": {
"apiKey": "",
"apiBaseUrl": "https://api.minimax.io/v1"
},
"openrouter": {
"apiKey": "",
"apiBaseUrl": "https://openrouter.ai/api/v1"
+1 -1
View File
@@ -113,7 +113,7 @@ Terhubung dengan maintainer dan komunitas melalui saluran-saluran berikut:
- Reddit: [reddit.com/r/puter/](https://www.reddit.com/r/puter/)
- Mastodon: [mastodon.social/@puter](https://mastodon.social/@puter)
- Isu keamanan? [[email protected]](mailto:[email protected])
- Email maintainers di [[email protected]](mailto:[email protected])
- Email pengelola di [[email protected]](mailto:[email protected])
Kami selalu senang membantu Anda dengan pertanyaan apa pun yang Anda miliki. Jangan ragu untuk bertanya!
+1 -1
View File
@@ -104,7 +104,7 @@ Puterは[**puter.com**](https://puter.com)でホストサービスとして利
メンテナーやコミュニティと以下のチャンネルを通じてつながりましょう:
- バグ報告や機能リクエストがありますか? [issueを開く](https://github.com/HeyPuter/puter/issues/new/choose) してください。
- バグ報告や機能リクエストは [新規Issue](https://github.com/HeyPuter/puter/issues/new/choose) まで
- Discord: [discord.com/invite/PQcx7Teh8u](https://discord.com/invite/PQcx7Teh8u)
- X (Twitter): [x.com/HeyPuter](https://x.com/HeyPuter)
- Reddit: [reddit.com/r/puter/](https://www.reddit.com/r/puter/)
+61 -2
View File
@@ -48,6 +48,7 @@ MARIADB_ROOT_PASSWORD=$(openssl rand -hex 32)
MARIADB_PASSWORD=$(openssl rand -hex 32)
S3_SECRET_KEY=$(openssl rand -hex 32)
JWT_SECRET=$(openssl rand -hex 64)
JWT_SECRET_V2=$(openssl rand -hex 64)
URL_SIGNATURE_SECRET=$(openssl rand -hex 64)
cat > .env <<EOF
@@ -78,6 +79,8 @@ cat > puter/config/config.json <<EOF
"private_app_hosting_domain_alt": "dev.puter.localhost",
"jwt_secret": "$JWT_SECRET",
"jwt_secret_v2": "$JWT_SECRET_V2",
"allow_v1_tokens": true,
"url_signature_secret": "$URL_SIGNATURE_SECRET",
"database": {
@@ -131,8 +134,9 @@ Replace `puter.localhost`, `site.puter.localhost`, `host.puter.localhost`, `dev.
Why these knobs:
- `env: "prod"` — the bundled `config.default.json` ships with `env: "dev"` (matches the source-tree `npm run start=gui` workflow, which expects webpack-dev-server emitting a CSS manifest). Self-host runs against pre-built static bundles, so `env: "prod"` makes the homepage emit the `/dist/bundle.min.css` `<link>` tag instead of waiting on a manifest that doesn't exist.
- `database.migrationPaths` — Puter applies the bundled MySQL schema on boot. `mysql_mig_1.sql` (tables) and `mysql_mig_2.sql` (default apps: editor, viewer, pdf, camera, player, recorder, git, dev-center, puter-linux). Idempotent — safe to re-run.
- `jwt_secret` + `jwt_secret_v2` — Puter signs new auth tokens with `jwt_secret_v2` (v2 token format, `kid: 'v2'` JWT header). `jwt_secret` is verify-only and lets tokens minted before this version (cookies still in users' browsers, stored API tokens) keep working. `allow_v1_tokens: true` keeps that fallback enabled. Fresh installs can leave it as-is; future versions will flip the flag to `false` and retire `jwt_secret` entirely once legacy tokens have drained.
- `env: "prod"` — the bundled `config.default.json` ships with `env: "dev"` (matches the source-tree `npm run start:gui` workflow, which expects webpack-dev-server emitting a CSS manifest). Self-host runs against pre-built static bundles, so `env: "prod"` makes the homepage emit the `/dist/bundle.min.css` `<link>` tag instead of waiting on a manifest that doesn't exist.
- `database.migrationPaths` — Puter applies the bundled MySQL/MariaDB schema on boot. The migration files are idempotent, so it is safe to leave this configured across restarts.
- `dynamo.bootstrapTables: true` — Puter creates its KV table on boot. **Only set against a local emulator**, never real AWS.
- `dynamo.aws` keys are dummies; DynamoDB-local doesn't validate them but the AWS SDK requires _something_. **Note:** DynamoDB uses `access_key` / `secret_key` (snake_case); S3 below uses `accessKeyId` / `secretAccessKey` (camelCase). Not interchangeable.
- `providers.ollama.enabled: false` — Puter auto-probes a local Ollama at `127.0.0.1:11434` by default; without one running you'd see `ECONNREFUSED` on every boot. To run a bundled Ollama, see [Optional: local LLM (Ollama)](#optional-local-llm-ollama) below.
@@ -195,6 +199,32 @@ Drop the resulting `fullchain.pem` and `privkey.pem` into `./puter/tls/`.
"s3": { "s3Config": { "publicEndpoint": "https://s3.puter.local", ... } }
```
## Running behind your own reverse proxy
The bundled `puter-nginx` mirrors production and is the supported default — it terminates TLS (Step 3) and forwards every Host to Puter. But plenty of self-hosters already run their own edge proxy (Caddy, Traefik, HAProxy, a cloud load balancer, another nginx). You can put Puter behind it instead; Puter doesn't terminate TLS itself in any setup, so "your proxy does TLS" is just a matter of pointing it at the Puter container and getting a handful of details right.
You can keep the bundled nginx as the single hop your proxy talks to, or bypass it and forward straight to the Puter container on port `4100` (uncomment the `4100:4100` mapping under the `puter` service in [docker-compose.yml](../docker-compose.yml), or attach your proxy to the compose network). Either works — what matters is the rules below.
**The rules that actually matter** (getting any of these wrong is what causes the redirect loops and "Invalid Host header" failures people hit):
1. **Don't rewrite the `Host` header.** Puter routes entirely on Host — `api.<domain>`, `site.<domain>`, `app.<domain>` are all distinguished by the incoming host. Forward the original host through unchanged. Do **not** rewrite external hostnames to an internal name like `puter.local`; that forces you to remap every subdomain by hand and still breaks signed-URL and CORS checks. Most proxies preserve Host by default — just don't override it.
2. **The external domain must equal `domain` in your `config.json`.** If users reach the box at `puter.example.com`, then `domain` must be `puter.example.com` and the hosting domains must be its real subdomains (`site.puter.example.com`, `app.puter.example.com`, …) — exactly as the installer/`config.json` lays them out. Puter rejects hosts it doesn't recognize with `Invalid Host header`.
3. **Set `protocol` to match the public scheme.** When your proxy terminates TLS, set `"protocol": "https"` (installer: `PUTER_PROTOCOL=https`). Puter builds its origins, redirects, signed S3 URLs, and OIDC callback URLs from this — leave it `http` behind an HTTPS proxy and you get redirect loops, broken logins, and mixed-content errors.
4. **Forward the standard proxy headers.** Puter needs:
- `Host` — the original request host (rule 1).
- `X-Forwarded-Proto` — the **external** scheme (`https`), so Puter knows TLS was terminated upstream.
- `X-Forwarded-For` — the real client IP (rate limiting + audit logs).
- `Upgrade` / `Connection` — passed through for WebSocket / socket.io upgrades, or the realtime connection silently fails.
5. **Set `trust_proxy` to the number of hops in front of Puter.** One external proxy talking directly to Puter → `"trust_proxy": 1` (installer: `PUTER_TRUST_PROXY=1`). A second proxy in front (e.g. Cloudflare → your proxy → Puter) → `2`. This is the count of proxies between the client and Puter, **including** the bundled nginx if you keep it. Too low and `req.ip` becomes a proxy address (breaks rate limiting); never set `true` (it trusts every hop and makes `X-Forwarded-For` forgeable).
6. **Route the wildcard to Puter.** Your proxy must forward `*.<domain>` (covering `api.*`, `app.*`, and `s3.<domain>`) and `*.site.<domain>` (and any other hosting domains) to Puter — same wildcard DNS as Step 2. Puter does the per-subdomain routing internally; the proxy just needs to hand it the traffic with the Host intact.
With those in place the rest of the stack is unchanged — `docker compose up -d` as below.
## Step 4 — Bring it up
```bash
@@ -228,6 +258,26 @@ Change it in Settings after first login.
All optional. Drop any of the blocks below into `puter/config/config.json` and `docker compose restart puter`. See [config.template.jsonc](../config.template.jsonc) for the full list. Per-key documentation lives in [src/backend/types.ts](../src/backend/types.ts).
### PostgreSQL database
> **Community contribution — expect rough edges.** PostgreSQL support was contributed by the community and is not run by Puter.com production. It boots, applies the bundled schema, and exercises the common user/app/fsentry/session/permission/OIDC flows in the integration tests, but less-traveled SQL call sites may still need porting. If you hit a query that doesn't work on Postgres, please open an issue. For production self-hosting today, MariaDB/MySQL and SQLite are the supported defaults.
The bundled Docker Compose stack still defaults to MariaDB. To use PostgreSQL instead, run a PostgreSQL service yourself, point Puter at it, and use the PostgreSQL migration path:
```json
"database": {
"engine": "postgres",
"host": "postgres",
"port": 5432,
"user": "puter",
"password": "...",
"database": "puter",
"migrationPaths": ["/opt/puter/dist/src/backend/clients/database/migrations/postgres"]
}
```
You may use `"connectionString": "postgres://puter:...@postgres:5432/puter"` instead of the host/user/password fields. Puter only needs normal PostgreSQL connection details and the migration path; CloudNativePG is one possible Kubernetes operator, but no operator-specific manifests are required by Puter.
### Email (SMTP)
Used for password resets, email confirmation, and notifications. Without it those flows silently fail.
@@ -329,6 +379,15 @@ Built-in proof-of-work captcha — no external service needed.
`difficulty` is one of `easy` / `medium` / `hard`.
### Disable new signups
Force visitors to log in with an existing account instead of creating a
temporary or permanent one.
```json
"disable_user_signup": true
```
### Block disposable email TLDs
Only enforced when `env: "prod"`.
+1 -1
View File
@@ -264,7 +264,7 @@ services:
# Persistent runtime data (anything your config points at /var/puter).
- ./puter/data/puter:/var/puter:z
healthcheck:
test: wget --no-verbose --tries=1 --spider http://puter.localhost:4100/test || exit 1
test: wget --no-verbose --tries=1 --spider http://localhost:4100/ || exit 1
interval: 30s
timeout: 3s
retries: 3
+59 -141
View File
@@ -1,75 +1,66 @@
import type { Request, Response } from 'express';
import { v4 as uuidv4 } from 'uuid';
import {
afterAll,
beforeAll,
describe,
expect,
it,
vi,
} from 'vitest';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { runWithContext } from '../src/backend/core/context.ts';
import { PuterServer } from '../src/backend/server.ts';
import { setupTestServer } from '../src/backend/testUtil.ts';
import {
handleAppTelemetryUserCount,
handleAppTelemetryUsers,
} from './appTelemetry.ts';
interface CapturedResponse {
body: unknown;
}
const makeReq = (query: Record<string, unknown> = {}): Request =>
({ query }) as unknown as Request;
const makeRes = () => {
const captured: CapturedResponse = { body: undefined };
const res = {
json: vi.fn((value: unknown) => {
captured.body = value;
return res;
}),
status: vi.fn(() => res),
};
return { res: res as unknown as Response, captured };
};
// `handleAppTelemetryUsers` calls `Context.get('actor')` for the permission
// gate. Wrap test calls in `runWithContext` so the ALS lookup resolves.
const callWithActor = async (
actor: { user: { uuid: string; id?: number }; app?: null } | undefined,
fn: () => Promise<void>,
) => runWithContext({ actor }, fn);
// Importing the module registers the `appTelemetry` driver into the shared
// extensionStore, so `setupTestServer` instantiates it on `server.drivers`.
import { AppTelemetryDriver } from './appTelemetry.ts';
let server: PuterServer;
let driver: AppTelemetryDriver;
// `get_users` reads `Context.get('actor')` for the ownership gate. Wrap
// actor-dependent calls in `runWithContext` so the ALS lookup resolves.
const callWithActor = <T>(
actor: { user: { uuid: string; id?: number } } | undefined,
fn: () => Promise<T>,
) => runWithContext({ actor }, fn);
beforeAll(async () => {
server = await setupTestServer();
driver = (server.drivers as unknown as Record<string, AppTelemetryDriver>)
.appTelemetry;
// Guard: the driver must have been wired onto the server. If this is
// undefined the extension didn't register, and every test below would
// fail with a confusing "cannot read property of undefined".
expect(driver).toBeInstanceOf(AppTelemetryDriver);
});
afterAll(async () => {
await server.shutdown();
});
describe('appTelemetry extension — handleAppTelemetryUsers', () => {
const seedOwnedApp = async (prefix: string) => {
const owner = await server.stores.user.create({
username: `${prefix}_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'x',
email: null,
});
const slug = Math.random().toString(36).slice(2, 8);
const app = await server.stores.app.create(
{
name: `${prefix}_${slug}`,
title: `${prefix} ${slug}`,
index_url: `https://example.com/${slug}`,
},
{ ownerUserId: owner.id as number },
);
return { owner, app: app! };
};
describe('appTelemetry driver — get_users', () => {
it('throws HttpError(400) when app_uuid is missing', async () => {
const { res } = makeRes();
await expect(
handleAppTelemetryUsers(makeReq({}), res),
).rejects.toMatchObject({
await expect(driver.get_users({})).rejects.toMatchObject({
statusCode: 400,
message: expect.stringContaining('app_uuid'),
});
});
it('throws HttpError(400) for a non-numeric limit', async () => {
const { res } = makeRes();
await expect(
handleAppTelemetryUsers(
makeReq({ app_uuid: 'app-anything', limit: 'banana' }),
res,
),
driver.get_users({ app_uuid: 'app-anything', limit: 'banana' }),
).rejects.toMatchObject({
statusCode: 400,
message: expect.stringContaining('limit'),
@@ -77,22 +68,14 @@ describe('appTelemetry extension — handleAppTelemetryUsers', () => {
});
it('throws HttpError(400) when offset exceeds the allowed maximum', async () => {
const { res } = makeRes();
await expect(
handleAppTelemetryUsers(
makeReq({ app_uuid: 'app-anything', offset: 1_000_001 }),
res,
),
driver.get_users({ app_uuid: 'app-anything', offset: 1_000_001 }),
).rejects.toMatchObject({ statusCode: 400 });
});
it('throws HttpError(404) when the app cannot be found', async () => {
const { res } = makeRes();
await expect(
handleAppTelemetryUsers(
makeReq({ app_uuid: 'app-does-not-exist' }),
res,
),
driver.get_users({ app_uuid: 'app-does-not-exist' }),
).rejects.toMatchObject({
statusCode: 404,
message: 'App not found',
@@ -100,25 +83,10 @@ describe('appTelemetry extension — handleAppTelemetryUsers', () => {
});
it('throws HttpError(403) when the caller does not own the app', async () => {
// Seed an owner user + an app belonging to them, then call the
// handler as a *different* user. The real permission service should
// reject since the caller has no `apps-of-user:<owner>:write` perm.
const owner = await server.stores.user.create({
username: `owner_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'x',
email: null,
});
const slug = Math.random().toString(36).slice(2, 8);
const app = await server.stores.app.create(
{
name: `app_${slug}`,
title: `App ${slug}`,
index_url: `https://example.com/${slug}`,
},
{ ownerUserId: owner.id as number },
);
// Seed an owner + their app, then call as a *different* user. The
// real permission service should reject: the caller has no
// `apps-of-user:<owner>:write`.
const { app } = await seedOwnedApp('owner');
const intruder = await server.stores.user.create({
username: `intruder_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
@@ -126,15 +94,11 @@ describe('appTelemetry extension — handleAppTelemetryUsers', () => {
email: null,
});
const { res } = makeRes();
await callWithActor(
{ user: { uuid: intruder.uuid, id: intruder.id as number } },
async () => {
await expect(
handleAppTelemetryUsers(
makeReq({ app_uuid: app!.uid }),
res,
),
driver.get_users({ app_uuid: app.uid }),
).rejects.toMatchObject({
statusCode: 403,
message: 'Permission denied',
@@ -144,78 +108,32 @@ describe('appTelemetry extension — handleAppTelemetryUsers', () => {
});
it('returns an empty list for an owned app with no authenticated users', async () => {
const owner = await server.stores.user.create({
username: `owner2_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'x',
email: null,
});
const slug = Math.random().toString(36).slice(2, 8);
const app = await server.stores.app.create(
{
name: `app2_${slug}`,
title: `App2 ${slug}`,
index_url: `https://example.com/${slug}`,
},
{ ownerUserId: owner.id as number },
);
const { owner, app } = await seedOwnedApp('owner2');
const { res, captured } = makeRes();
await callWithActor(
const result = await callWithActor(
{ user: { uuid: owner.uuid, id: owner.id as number } },
async () => {
await handleAppTelemetryUsers(
makeReq({ app_uuid: app!.uid }),
res,
);
},
() => driver.get_users({ app_uuid: app.uid }),
);
expect(captured.body).toEqual([]);
expect(result).toEqual([]);
});
});
describe('appTelemetry extension — handleAppTelemetryUserCount', () => {
describe('appTelemetry driver — user_count', () => {
it('throws HttpError(400) when app_uuid is missing', async () => {
const { res } = makeRes();
await expect(
handleAppTelemetryUserCount(makeReq({}), res),
).rejects.toMatchObject({ statusCode: 400 });
await expect(driver.user_count({})).rejects.toMatchObject({
statusCode: 400,
});
});
it('throws HttpError(404) for an unknown app_uuid', async () => {
const { res } = makeRes();
await expect(
handleAppTelemetryUserCount(
makeReq({ app_uuid: 'app-not-here' }),
res,
),
driver.user_count({ app_uuid: 'app-not-here' }),
).rejects.toMatchObject({ statusCode: 404 });
});
it('returns { count: 0 } for an app with no authenticated users', async () => {
const owner = await server.stores.user.create({
username: `counto_${Math.random().toString(36).slice(2, 8)}`,
uuid: uuidv4(),
password: 'x',
email: null,
});
const slug = Math.random().toString(36).slice(2, 8);
const app = await server.stores.app.create(
{
name: `appcount_${slug}`,
title: `AppCount ${slug}`,
index_url: `https://example.com/${slug}`,
},
{ ownerUserId: owner.id as number },
);
const { res, captured } = makeRes();
await handleAppTelemetryUserCount(
makeReq({ app_uuid: app!.uid }),
res,
);
expect(captured.body).toEqual({ count: 0 });
it('returns 0 for an app with no authenticated users', async () => {
const { app } = await seedOwnedApp('appcount');
await expect(driver.user_count({ app_uuid: app.uid })).resolves.toBe(0);
});
});
+88 -80
View File
@@ -1,11 +1,16 @@
import type { Request, Response } from 'express';
import { Context } from '@heyputer/backend/src/core';
import type { Actor } from '@heyputer/backend/src/core/actor';
import { HttpError } from '@heyputer/backend/src/core/http';
import { PuterDriver } from '@heyputer/backend/src/drivers/types';
import { extension } from '@heyputer/backend/src/extensions';
const clients = extension.import('client');
const stores = extension.import('store');
const services = extension.import('service');
// App-telemetry lets an app owner enumerate the users who have
// authenticated into their app. v1 shipped this as a driver on the
// `app-telemetry` interface (methods `get_users` / `user_count`) and
// puter-js's `puter.apps(...).getUsers()` still calls it that way
// (`puter.drivers.call('app-telemetry', 'app-telemetry', 'get_users', …)`).
// This is the v2 port of that driver — same interface/method/return shapes
// so existing puter-js callers work unchanged.
const DEFAULT_LIMIT = 100;
const MAX_LIMIT = 1000;
@@ -41,91 +46,94 @@ const parseIntParam = (
return parsed;
};
export const handleAppTelemetryUsers = async (
req: Request,
res: Response,
): Promise<void> => {
const { app_uuid } = req.query as Record<string, string>;
if (!app_uuid) throw new HttpError(400, 'Missing `app_uuid`');
/**
* Driver exposing the `app-telemetry` interface.
*
* The `/drivers/call` permission gate checks
* `service:app-telemetry:ii:app-telemetry`, which every actor already holds
* via the blanket `service` grant (hardcoded-permissions.js +
* `default_implicit_user_app_permissions`). The real authorization — "is the
* caller the app owner?" — is enforced inside `get_users` below, exactly as
* v1 did.
*/
export class AppTelemetryDriver extends PuterDriver {
readonly driverInterface = 'app-telemetry';
readonly driverName = 'app-telemetry';
readonly isDefault = true;
const safeLimit = parseIntParam(req.query.limit, {
key: 'limit',
min: 1,
max: MAX_LIMIT,
fallback: DEFAULT_LIMIT,
});
const safeOffset = parseIntParam(req.query.offset, {
key: 'offset',
min: 0,
max: MAX_OFFSET,
fallback: 0,
});
/** Users who have authenticated into the given app (owner-only). */
async get_users({
app_uuid,
limit,
offset,
}: {
app_uuid?: string;
limit?: unknown;
offset?: unknown;
} = {}): Promise<Array<{ user: string; user_uuid: string }>> {
if (!app_uuid) throw new HttpError(400, 'Missing `app_uuid`');
const app = await stores.app.getByUid(app_uuid);
if (!app) throw new HttpError(404, 'App not found');
const safeLimit = parseIntParam(limit, {
key: 'limit',
min: 1,
max: MAX_LIMIT,
fallback: DEFAULT_LIMIT,
});
const safeOffset = parseIntParam(offset, {
key: 'offset',
min: 0,
max: MAX_OFFSET,
fallback: 0,
});
// `apps-of-user:<uuid>:write` — the implicator keys on the owner's
// UUID, not the numeric id. Look up the owner explicitly. v1 got
// this for free because its entity-storage layer eager-joined the
// owner row; v2's AppStore.getByUid returns the raw row with only
// `owner_user_id` populated.
const ownerId = (app as { owner_user_id?: number }).owner_user_id;
if (!ownerId) throw new HttpError(404, 'App owner not found');
const owner = (await stores.user.getById(ownerId)) as {
uuid?: string;
} | null;
if (!owner?.uuid) throw new HttpError(404, 'App owner not found');
const app = await this.stores.app.getByUid(app_uuid);
if (!app) throw new HttpError(404, 'App not found');
const actor = Context.get('actor');
const ownsApp = await services.permission
.check(actor!, `apps-of-user:${owner.uuid}:write`)
.catch(() => false);
if (!ownsApp) throw new HttpError(403, 'Permission denied');
// The `apps-of-user:<uuid>:write` implicator keys on the owner's
// UUID, not the numeric id. Look up the owner explicitly — the raw
// app row only carries `owner_user_id`. (v1 got the owner for free
// because its entity-storage layer eager-joined the owner row.)
const ownerId = (app as { owner_user_id?: number }).owner_user_id;
if (!ownerId) throw new HttpError(404, 'App owner not found');
const owner = await this.stores.user.getById(ownerId);
if (!owner?.uuid) throw new HttpError(404, 'App owner not found');
const users = await clients.db.read(
`SELECT u.username, u.uuid FROM user_to_app_permissions p
INNER JOIN user u ON p.user_id = u.id
WHERE p.permission = 'flag:app-is-authenticated' AND p.app_id = ?
ORDER BY (p.dt IS NOT NULL), p.dt, p.user_id
LIMIT ? OFFSET ?`,
[(app as Record<string, unknown>).id, safeLimit, safeOffset],
);
const actor = Context.get('actor');
if (!actor) throw new HttpError(401, 'Authentication required');
const ownsApp = await this.services.permission
.check(actor as Actor, `apps-of-user:${owner.uuid}:write`)
.catch(() => false);
if (!ownsApp) throw new HttpError(403, 'Permission denied');
res.json(
(users as Array<{ username: string; uuid: string }>).map((e) => ({
user: e.username,
user_uuid: e.uuid,
})),
);
};
const users = (await this.clients.db.read(
`SELECT u.username, u.uuid FROM user_to_app_permissions p
INNER JOIN ${this.clients.db.quoteIdentifier('user')} u ON p.user_id = u.id
WHERE p.permission = 'flag:app-is-authenticated' AND p.app_id = ?
ORDER BY (p.dt IS NOT NULL), p.dt, p.user_id
LIMIT ? OFFSET ?`,
[(app as { id: number }).id, safeLimit, safeOffset],
)) as Array<{ username: string; uuid: string }>;
export const handleAppTelemetryUserCount = async (
req: Request,
res: Response,
): Promise<void> => {
const { app_uuid } = req.query as Record<string, string>;
if (!app_uuid) throw new HttpError(400, 'Missing `app_uuid`');
return users.map((e) => ({ user: e.username, user_uuid: e.uuid }));
}
const app = await stores.app.getByUid(app_uuid);
if (!app) throw new HttpError(404, 'App not found');
/** Count of users who have authenticated into the given app. */
async user_count({
app_uuid,
}: { app_uuid?: string } = {}): Promise<number> {
if (!app_uuid) throw new HttpError(400, 'Missing `app_uuid`');
const [row] = (await clients.db.read(
`SELECT COUNT(*) AS n FROM user_to_app_permissions
WHERE permission = 'flag:app-is-authenticated' AND app_id = ?`,
[(app as Record<string, unknown>).id],
)) as Array<{ n: number }>;
const app = await this.stores.app.getByUid(app_uuid);
if (!app) throw new HttpError(404, 'App not found');
res.json({ count: row?.n ?? 0 });
};
const [row] = (await this.clients.db.read(
`SELECT COUNT(*) AS n FROM user_to_app_permissions
WHERE permission = 'flag:app-is-authenticated' AND app_id = ?`,
[(app as { id: number }).id],
)) as Array<{ n: number }>;
extension.get(
'/app-telemetry/users',
{ subdomain: 'api', requireAuth: true },
handleAppTelemetryUsers,
);
return row?.n ?? 0;
}
}
extension.get(
'/app-telemetry/user-count',
{ subdomain: 'api', requireAuth: true },
handleAppTelemetryUserCount,
);
extension.registerDriver('appTelemetry', AppTelemetryDriver);
+32
View File
@@ -132,9 +132,41 @@ describe('installedApps extension — handleInstalledApps', () => {
expect(list[0].uid).toBe(app!.uid);
expect(list[0].name).toBe(app!.name);
expect(list[0].title).toBe(app!.title);
// index_url is required so the dashboard can derive a hostname title
// for anonymous (app-…) apps.
expect(list[0].index_url).toBe(app!.index_url);
expect(Object.prototype.hasOwnProperty.call(list[0], 'iconUrl')).toBe(
true,
);
// An owned app is not external, and the raw owner id must not leak.
expect(list[0].external).toBe(false);
expect(
Object.prototype.hasOwnProperty.call(list[0], 'owner_user_id'),
).toBe(false);
});
it('flags apps with no owner_user_id as external', async () => {
const user = await seedUser();
const slug = Math.random().toString(36).slice(2, 8);
// createFromOrigin bootstraps an app with owner_user_id = null.
const app = await server.stores.app.createFromOrigin(
`app-${slug}`,
`https://external-${slug}.example.com`,
);
await grantInstalled(app!.id as number, user.id as number);
const { res, captured } = makeRes();
await runWithContext(
{ actor: { user: { uuid: user.uuid, id: user.id as number } } },
() => handleInstalledApps(makeReq({}), res),
);
const list = captured.body as Array<Record<string, unknown>>;
expect(list).toHaveLength(1);
expect(list[0].external).toBe(true);
expect(
Object.prototype.hasOwnProperty.call(list[0], 'owner_user_id'),
).toBe(false);
});
it('clamps page/limit to safe ranges (page>=1, 1<=limit<=100)', async () => {
+15 -5
View File
@@ -49,6 +49,8 @@ export const handleInstalledApps = async (
apps.title,
apps.description,
apps.icon,
apps.index_url,
apps.owner_user_id,
MIN(perm.dt) AS installed_at
FROM apps
LEFT JOIN user_to_app_permissions AS perm ON apps.id = perm.app_id
@@ -62,15 +64,23 @@ export const handleInstalledApps = async (
const apiBaseUrl = extension.config.api_base_url as string | undefined;
res.json(
installedApps.map((app) => ({
...app,
iconUrl: getAppIconUrl(app, { apiBaseUrl }),
})),
installedApps.map((app) => {
// An app with no owner_user_id (null/empty) isn't owned by a Puter
// user — it's an "external" app. Derive a flag and don't leak the
// raw owner id to the client.
const { owner_user_id, ...rest } = app;
const external = owner_user_id == null || owner_user_id === '';
return {
...rest,
iconUrl: getAppIconUrl(app, { apiBaseUrl }),
external,
};
}),
);
};
extension.get(
'/installedApps',
{ subdomain: 'api', requireUserActor: true },
{ subdomain: 'api', requireUserActor: true, allowFullAccessToken: true },
handleInstalledApps,
);
+1 -1
View File
@@ -1,4 +1,3 @@
import type { Request, Response } from 'express';
import { Context } from '@heyputer/backend/src/core';
import { HttpError } from '@heyputer/backend/src/core/http';
import {
@@ -6,6 +5,7 @@ import {
driversContainers,
} from '@heyputer/backend/src/exports';
import { extension } from '@heyputer/backend/src/extensions';
import type { Request, Response } from 'express';
const services = extension.import('service');
const clients = extension.import('client');
+4 -4
View File
@@ -95,7 +95,7 @@ async function decodeAndValidateThumbnail(
return { mimeType, data };
}
// ── thumbnail.created ───────────────────────────────────────────────
// -- thumbnail.created -----------------------------------------------
// Intercept data-URL thumbnails before they hit the DB: upload to S3
// and replace the URL with an s3:// pointer.
@@ -253,7 +253,7 @@ extension.on(
},
);
// ── thumbnail.upload.prepare ────────────────────────────────────────
// -- thumbnail.upload.prepare ----------------------------------------
// Generate pre-signed upload URLs so the client can PUT directly to S3.
extension.on(
@@ -266,7 +266,7 @@ extension.on(
},
);
// ── thumbnail.read ──────────────────────────────────────────────────
// -- thumbnail.read --------------------------------------------------
// Convert s3:// or legacy https:// thumbnails to signed URLs.
extension.on('thumbnail.read', async (_key, entry: Record<string, unknown>) => {
@@ -279,7 +279,7 @@ extension.on('thumbnail.read', async (_key, entry: Record<string, unknown>) => {
});
});
// ── fs.remove.node ──────────────────────────────────────────────────
// -- fs.remove.node --------------------------------------------------
// Delete S3 thumbnail when the file is removed.
extension.on(
+4
View File
@@ -107,6 +107,8 @@ describe('whoami extension — handleWhoami', () => {
expect(body.email).toBe(user.email);
expect(body.is_temp).toBe(false);
expect(body.oidc_only).toBe(false);
// is_user_token is present (true) for user actors.
expect(body.is_user_token).toBe(true);
// `directories` is only sent to user actors — confirm it’s present.
expect(body.directories).toBeDefined();
// taskbar_items is only sent to user actors.
@@ -146,6 +148,8 @@ describe('whoami extension — handleWhoami', () => {
const body = captured.body as Record<string, unknown>;
expect(body.app_name).toBe('app-test-actor');
// is_user_token is stripped for app actors.
expect(body.is_user_token).toBeUndefined();
expect(body.desktop_bg_url).toBeUndefined();
expect(body.desktop_bg_color).toBeUndefined();
expect(body.desktop_bg_fit).toBeUndefined();
+5
View File
@@ -72,10 +72,14 @@ export const handleWhoami = async (
unconfirmed_email: user.email,
email_confirmed: user.email_confirmed || user.username === 'admin',
requires_email_confirmation: user.requires_email_confirmation,
phone: user.phone,
requires_phone_verification: user.requires_phone_verification,
requires_card_verification: user.requires_card_verification,
desktop_bg_url: user.desktop_bg_url,
desktop_bg_color: user.desktop_bg_color,
desktop_bg_fit: user.desktop_bg_fit,
is_temp: user.password === null && user.email === null,
is_user_token: true,
oidc_only: oidcOnly,
taskbar_items: isUser
? await getTaskbarItems(
@@ -157,6 +161,7 @@ export const handleWhoami = async (
delete details.desktop_bg_color;
delete details.desktop_bg_fit;
delete details.human_readable_age;
delete details.is_user_token;
}
if (actor.app) {
+5
View File
@@ -113,7 +113,10 @@ if ($writeConfig) {
$mariadbRootPw = New-HexSecret 32
$mariadbPw = New-HexSecret 32
$s3SecretKey = New-HexSecret 32
# Two JWT secrets: $jwtSecret is verify-only for legacy v1 tokens
# already in circulation; $jwtSecretV2 signs every new token.
$jwtSecret = New-HexSecret 64
$jwtSecretV2 = New-HexSecret 64
$urlSigSecret = New-HexSecret 64
$envContent = @"
@@ -142,6 +145,8 @@ S3_BUCKET=puter-local
private_app_hosting_domain = "app.$PuterDomain"
private_app_hosting_domain_alt = "dev.$PuterDomain"
jwt_secret = $jwtSecret
jwt_secret_v2 = $jwtSecretV2
allow_v1_tokens = $true
url_signature_secret = $urlSigSecret
database = [ordered]@{
engine = 'mysql'
+28 -10
View File
@@ -16,11 +16,20 @@
# the compose file + brings the stack up. Set PUTER_FORCE=1 to overwrite.
#
# Tunable env vars:
# PUTER_DIR install directory (default: ./puter-selfhosted)
# PUTER_URL base URL to fetch docker-compose.yml (default: GitHub raw, main branch)
# PUTER_DOMAIN domain Puter will serve on (default: puter.localhost)
# PUTER_PORT HTTP port for nginx (default: 80)
# PUTER_FORCE set to 1 to overwrite existing .env / config.json
# PUTER_DIR install directory (default: ./puter-selfhosted)
# PUTER_URL base URL to fetch docker-compose.yml (default: GitHub raw, main branch)
# PUTER_DOMAIN domain Puter will serve on (default: puter.localhost)
# PUTER_PORT HTTP port for nginx (default: 80)
# PUTER_PROTOCOL public scheme: http | https (default: http)
# Set https when a TLS-terminating reverse proxy (Caddy,
# Traefik, nginx, a cloud LB) sits in front — Puter then
# builds https:// origins, S3 URLs, and redirects. See
# "Running behind your own reverse proxy" in doc/self-hosting.md.
# PUTER_TRUST_PROXY number of reverse-proxy hops in front (default: 1)
# 1 = the bundled nginx (or a single external proxy);
# 2 = two hops (e.g. Cloudflare → your proxy → Puter).
# PUTER_ENV prod | dev (default: prod)
# PUTER_FORCE set to 1 to overwrite existing .env / config.json
set -eu
@@ -28,6 +37,9 @@ PUTER_DIR="${PUTER_DIR:-puter-selfhosted}"
PUTER_URL="${PUTER_URL:-https://raw.githubusercontent.com/HeyPuter/puter/main}"
PUTER_DOMAIN="${PUTER_DOMAIN:-puter.localhost}"
PUTER_PORT="${PUTER_PORT:-80}"
PUTER_PROTOCOL="${PUTER_PROTOCOL:-http}"
PUTER_TRUST_PROXY="${PUTER_TRUST_PROXY:-1}"
PUTER_ENV="${PUTER_ENV:-prod}"
PUTER_FORCE="${PUTER_FORCE:-0}"
log() { printf '\033[1;36m[puter-install]\033[0m %s\n' "$*"; }
@@ -92,7 +104,11 @@ if [ "$write_config" = "1" ]; then
MARIADB_ROOT_PASSWORD=$(openssl rand -hex 32)
MARIADB_PASSWORD=$(openssl rand -hex 32)
S3_SECRET_KEY=$(openssl rand -hex 32)
# Two JWT secrets: `jwt_secret` is verify-only for legacy v1 tokens
# already in circulation; `jwt_secret_v2` signs every new token.
# Both are required at boot (`jwt_secret` only when verifying v1).
JWT_SECRET=$(openssl rand -hex 64)
JWT_SECRET_V2=$(openssl rand -hex 64)
URL_SIGNATURE_SECRET=$(openssl rand -hex 64)
cat > .env <<EOF
@@ -113,9 +129,9 @@ EOF
cat > puter/config/config.json <<EOF
{
"domain": "$PUTER_DOMAIN",
"protocol": "http",
"protocol": "$PUTER_PROTOCOL",
"pub_port": $PUTER_PORT,
"env": "prod",
"env": "$PUTER_ENV",
"static_hosting_domain": "site.$PUTER_DOMAIN",
"static_hosting_domain_alt": "host.$PUTER_DOMAIN",
@@ -123,6 +139,8 @@ EOF
"private_app_hosting_domain_alt": "dev.$PUTER_DOMAIN",
"jwt_secret": "$JWT_SECRET",
"jwt_secret_v2": "$JWT_SECRET_V2",
"allow_v1_tokens": true,
"url_signature_secret": "$URL_SIGNATURE_SECRET",
"database": {
@@ -153,7 +171,7 @@ EOF
"s3": {
"s3Config": {
"endpoint": "http://s3:9000",
"publicEndpoint": "http://s3.$PUTER_DOMAIN",
"publicEndpoint": "$PUTER_PROTOCOL://s3.$PUTER_DOMAIN",
"accessKeyId": "puter",
"secretAccessKey": "$S3_SECRET_KEY",
"region": "us-east-1",
@@ -167,7 +185,7 @@ EOF
"ollama": { "enabled": false }
},
"trust_proxy": 1
"trust_proxy": $PUTER_TRUST_PROXY
}
EOF
fi
@@ -181,6 +199,6 @@ log "stack starting. first boot takes ~30s while MariaDB initialises."
log "follow puter logs:"
log " cd $PUTER_DIR && docker compose logs -f puter"
log ""
log "open http://$PUTER_DOMAIN:$PUTER_PORT once the puter container is healthy."
log "open $PUTER_PROTOCOL://$PUTER_DOMAIN:$PUTER_PORT once the puter container is healthy."
log "first-boot admin password is logged once — grab it with:"
log " cd $PUTER_DIR && docker compose logs puter | grep password"
+3961 -4182
View File
File diff suppressed because it is too large. Load diff
+17 -5
View File
@@ -44,17 +44,18 @@
"yaml": "^2.8.1"
},
"scripts": {
"test:backend": "vitest run --config src/backend/vitest.config.ts ",
"start=gui": "nodemon --exec \"node dev-server.js\" ",
"test:backend": "npm run setupExtensions && vitest run --config src/backend/vitest.config.ts ",
"test:backend:postgres": "npm run setupExtensions && PUTER_TEST_DB_ENGINE=postgres vitest run --config src/backend/vitest.config.ts ",
"start:gui": "nodemon --exec \"node dev-server.js\" ",
"start": "node --enable-source-maps -r ./dist/src/backend/telemetry.js ./dist/src/backend/index.js",
"prestart": "npm run build:ts",
"prestart": "npm run setupExtensions && npm run build:ts",
"dev": "npm start",
"build": "npm run build:ts && cd src/gui && node ./build.js && cd ../puter-js && npm run build",
"build": "npm run setupExtensions && npm run build:ts && cd src/gui && node ./build.js && cd ../puter-js && npm run build",
"build:workerLib": "cd src/puter-js && npm run build && cd ../worker && npm run build",
"check-translations": "node tools/check-translations.js",
"prepare": "husky",
"build:ts": "tsc -p tsconfig.build.json && node ./tools/write-dist-package-json.mjs",
"postinstall": "node ./tools/extensionSetup.mjs"
"setupExtensions": "node ./tools/extensionSetup.mjs"
},
"workspaces": [
"src/*",
@@ -76,9 +77,20 @@
"ai": "^6.0.73",
"dedent": "^1.5.3",
"javascript-time-ago": "^2.5.11",
"libphonenumber-js": "1.13.6",
"open": "^10.1.0"
},
"engines": {
"node": ">=24.0.0"
},
"allowScripts": {
"@google/genai": false,
"[email protected]": true,
"[email protected]": true,
"[email protected]": true,
"[email protected]": true,
"[email protected]": true,
"[email protected]": true,
"[email protected]": true
}
}
+19 -8
View File
@@ -23,7 +23,7 @@ import { createHash } from 'node:crypto';
import type { IConfig } from '../../types';
import { PuterClient } from '../types';
// ── Types ────────────────────────────────────────────────────────────
// -- Types ------------------------------------------------------------
export interface AlarmFields {
error?: Error;
@@ -49,7 +49,7 @@ interface Alarm {
noAlert?: boolean;
}
type PagerSeverity = 'critical' | 'error' | 'warning' | 'info';
export type PagerSeverity = 'critical' | 'error' | 'warning' | 'info';
export interface AlertPayload {
id: string;
@@ -73,7 +73,7 @@ interface KnownErrorRule {
};
}
// ── Helpers ──────────────────────────────────────────────────────────
// -- Helpers ----------------------------------------------------------
/**
* Deterministic short identifier derived from an alarm ID.
@@ -166,7 +166,7 @@ function cleanFields(fields: AlarmFields): Record<string, string> {
return out;
}
// ── AlarmClient ──────────────────────────────────────────────────────
// -- AlarmClient ------------------------------------------------------
/**
* Manages system alarms and dispatches alerts to external paging
@@ -184,7 +184,7 @@ export class AlarmClient extends PuterClient {
super(config);
}
// ── Lifecycle ────────────────────────────────────────────────────
// -- Lifecycle ----------------------------------------------------
override async onServerStart(): Promise<void> {
const pagerConf = this.config.pager;
@@ -228,13 +228,23 @@ export class AlarmClient extends PuterClient {
console.log('[alarm] entering drain mode — suppressing new alarms');
}
// ── Public API ───────────────────────────────────────────────────
// -- Public API ---------------------------------------------------
/**
* Create or update an alarm. If the alarm ID already exists, the
* occurrence count is incremented and a repeat alert is dispatched.
*
* `severity` is the PagerDuty severity for this alarm; omit for the
* default 'critical'. Use 'info' / 'warning' for expected-but-worth-
* tracking signals (e.g. a user hitting a rate limit) so they record
* and de-dupe like any other alarm but don't page on-call.
*/
create(id: string, message: string, fields: AlarmFields = {}): void {
create(
id: string,
message: string,
fields: AlarmFields = {},
severity?: PagerSeverity,
): void {
if (this.draining) {
if (!this.drainLogged) {
this.drainLogged = true;
@@ -256,6 +266,7 @@ export class AlarmClient extends PuterClient {
shortId: shortId(id),
message,
fields,
severity,
started: Date.now(),
timestamps: [Date.now()],
occurrences: [],
@@ -298,7 +309,7 @@ export class AlarmClient extends PuterClient {
this.knownErrors = rules;
}
// ── Internals ────────────────────────────────────────────────────
// -- Internals ----------------------------------------------------
private recordOccurrence(
alarm: Alarm,
@@ -0,0 +1,50 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Optional ClickHouse client.
*
* ClickHouse is NOT a hard dependency. Core Puter runs without it — the app
* stats read path (open / unique-user counts) falls back to the primary SQL
* database. `this.clients.clickhouse` is therefore typed as optional and is
* `undefined` in default and self-hosted setups; callers MUST branch on its
* presence and fall back to SQL.
*
* A production deployment can register a real ClickHouse client via an
* extension (`extension.registerClient('clickhouse', client)`) to offload the
* analytics queries off the primary database and keep the stats path fast at
* scale. When registered, the instance flows into `this.clients.clickhouse`
* everywhere it's typed.
*
* Only the surface the stats path actually consumes is declared here. Extend
* this interface (don't widen to `any`) when a new query shape is needed.
*/
export interface ClickhouseQueryResult {
json<T = Record<string, unknown>>(): Promise<T[]>;
}
export interface ClickhouseQueryParams {
query: string;
query_params?: Record<string, unknown>;
format?: string;
}
export interface ClickhouseClient {
query(params: ClickhouseQueryParams): Promise<ClickhouseQueryResult>;
}
+90 -3
View File
@@ -31,6 +31,8 @@ export interface BatchEntry {
values: unknown[];
}
type SqlJsonPath = readonly [string, ...string[]];
/**
* Base database client. Subclasses must override every method that throws here.
*
@@ -101,9 +103,10 @@ export class AbstractDatabaseClient extends PuterClient {
const cols = Object.keys(data);
const values = Object.values(data);
const sql =
`INSERT INTO \`${tableName}\` ` +
`(${cols.map((c) => `\`${c}\``).join(', ')}) ` +
`VALUES (${cols.map(() => '?').join(', ')})`;
`INSERT INTO ${this.quoteIdentifier(tableName)} ` +
`(${cols.map((c) => this.quoteIdentifier(c)).join(', ')}) ` +
`VALUES (${cols.map(() => '?').join(', ')})` +
this.returningIdClause();
return this.write(sql, values);
}
@@ -161,4 +164,88 @@ export class AbstractDatabaseClient extends PuterClient {
}
return choices.otherwise as T;
}
quoteIdentifier(identifier: string): string {
return identifier
.split('.')
.map((part) => {
if (part === '*') return part;
return `\`${part.replaceAll('`', '``')}\``;
})
.join('.');
}
booleanLiteral(value: boolean): string {
return value ? '1' : '0';
}
booleanValue(value: boolean): boolean | 0 | 1 {
return value ? 1 : 0;
}
insertIgnoreInto(tableName: string): string {
const table = this.quoteIdentifier(tableName);
return this.case({
sqlite: `INSERT OR IGNORE INTO ${table}`,
postgres: `INSERT INTO ${table}`,
otherwise: `INSERT IGNORE INTO ${table}`,
});
}
insertIgnoreSuffix(): string {
return this.case({
postgres: ' ON CONFLICT DO NOTHING',
otherwise: '',
});
}
upsertClause(
conflictColumns: readonly string[],
updateColumns: readonly string[],
): string {
if (updateColumns.length === 0) {
throw new Error('upsertClause requires at least one update column');
}
const updateList = updateColumns
.map((column) => `${this.quoteIdentifier(column)} = ?`)
.join(', ');
return this.case({
mysql: `ON DUPLICATE KEY UPDATE ${updateList}`,
otherwise: `ON CONFLICT(${conflictColumns
.map((column) => this.quoteIdentifier(column))
.join(', ')}) DO UPDATE SET ${updateList}`,
});
}
jsonTextExtract(jsonExpression: string, path: SqlJsonPath): string {
const sqlitePath = `$${path.map((part) => `.${part}`).join('')}`;
return this.case({
sqlite: `json_extract(${jsonExpression}, ${this.sqlStringLiteral(sqlitePath)})`,
mysql: `JSON_UNQUOTE(JSON_EXTRACT(${jsonExpression}, ${this.sqlStringLiteral(sqlitePath)}))`,
postgres: `${jsonExpression} #>> ARRAY[${path
.map((part) => this.sqlStringLiteral(part))
.join(', ')}]`,
otherwise: `JSON_UNQUOTE(JSON_EXTRACT(${jsonExpression}, ${this.sqlStringLiteral(sqlitePath)}))`,
});
}
nullCoalesce(...expressions: readonly string[]): string {
if (expressions.length === 0) {
throw new Error('nullCoalesce requires at least one expression');
}
return `COALESCE(${expressions.join(', ')})`;
}
returningIdClause(): string {
return this.case({
postgres: ' RETURNING id',
otherwise: '',
});
}
protected sqlStringLiteral(value: string): string {
return `'${value.replaceAll("'", "''")}'`;
}
}
@@ -0,0 +1,95 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import { compareMigrationFilenames } from './MySQLDatabaseClient.js';
describe('compareMigrationFilenames', () => {
it('orders numbered migrations numerically, not lexically', () => {
const input = [
'mysql_mig_10.sql',
'mysql_mig_2.sql',
'mysql_mig_1.sql',
'mysql_mig_9.sql',
'mysql_mig_3.sql',
];
const sorted = [...input].sort(compareMigrationFilenames);
expect(sorted).toEqual([
'mysql_mig_1.sql',
'mysql_mig_2.sql',
'mysql_mig_3.sql',
'mysql_mig_9.sql',
'mysql_mig_10.sql',
]);
});
it('keeps mig_10 after mig_9 when the real prod set is shuffled', () => {
// Reflects the current migrations/mysql/ listing — guards against
// a future "let's just rename to padded" suggestion accidentally
// re-introducing the lex bug if the rename is incomplete.
const real = [
'mysql_mig_9.sql',
'mysql_mig_3.sql',
'mysql_mig_10.sql',
'mysql_mig_1.sql',
'mysql_mig_7.sql',
'mysql_mig_5.sql',
'mysql_mig_2.sql',
'mysql_mig_4.sql',
'mysql_mig_8.sql',
'mysql_mig_6.sql',
];
const sorted = [...real].sort(compareMigrationFilenames);
for (let i = 1; i <= sorted.length; i += 1) {
expect(sorted[i - 1]).toBe(`mysql_mig_${i}.sql`);
}
});
it('sorts non-numeric filenames after numbered ones, lexically among themselves', () => {
// Numbered files always run first (they're the canonical history);
// unmatched names follow in localeCompare order. Mixing the two
// sets prevents a vendor dump from accidentally wedging itself
// between mig_4 and mig_5 if it happened to lex-sort there.
const mixed = [
'mysql_mig_2.sql',
'mysql_vendor_dump.sql',
'mysql_mig_10.sql',
'mysql_bootstrap.sql',
'mysql_mig_1.sql',
];
const sorted = [...mixed].sort(compareMigrationFilenames);
expect(sorted).toEqual([
'mysql_mig_1.sql',
'mysql_mig_2.sql',
'mysql_mig_10.sql',
'mysql_bootstrap.sql',
'mysql_vendor_dump.sql',
]);
});
it('is stable for already-sorted input', () => {
const sorted = [
'mysql_mig_1.sql',
'mysql_mig_2.sql',
'mysql_mig_10.sql',
'mysql_mig_11.sql',
];
expect([...sorted].sort(compareMigrationFilenames)).toEqual(sorted);
});
});
@@ -23,6 +23,7 @@ import { createPool, type Pool } from 'mysql2';
import { AbstractDatabaseClient, type WriteResult } from './DatabaseClient';
import { SQLBatcher } from './SQLBatcher.js';
import { splitMysqlStatements } from './splitMysqlStatements.js';
import { compareMigrationFilenames } from './migrationFilenames.js';
import type { IConfig } from '../../types';
const RETRIABLE_ERROR_CODES = new Set([
@@ -44,6 +45,8 @@ const RETRIABLE_ERROR_MESSAGES = [
'ETIMEDOUT',
];
export { compareMigrationFilenames };
type PoolConfig = Parameters<typeof createPool>[0];
enum Configuration {
@@ -247,7 +250,7 @@ export class MySQLDatabaseClient extends AbstractDatabaseClient {
.filter(
(f) => f.endsWith('.sql') && f.startsWith('mysql'),
)
.sort();
.sort(compareMigrationFilenames);
} catch (e) {
throw new Error(
`[mysql] migration path is unreadable: ${dir}`,
@@ -0,0 +1,328 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { Pool } from 'pg';
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { IConfig } from '../../types';
import type { PuterServer } from '../../server';
import { generateDefaultFsentries } from '../../util/userProvisioning.js';
import {
createPgMockPostgresDatabaseClient,
POSTGRES_TEST_MIGRATIONS_PATH,
setupTestServer,
} from '../../testUtil.js';
import { PostgresDatabaseClient } from './PostgresDatabaseClient.js';
const postgresUrl = process.env.PUTER_TEST_POSTGRES_URL;
const postgresMigrationsPath = POSTGRES_TEST_MIGRATIONS_PATH;
const postgresTestSchemaPattern = /^puter_test_[a-f0-9]{32}$/u;
const postgresIntegrationTimeoutMs = 180_000;
let postgresTestSchema: string | undefined;
let postgresTestUrl: string | undefined;
const postgresConfig = (overrides: Partial<IConfig> = {}): IConfig => {
if (postgresUrl && !postgresTestUrl) {
throw new Error('Postgres test schema was not initialized');
}
const { database: databaseOverrides, ...rootOverrides } = overrides;
const database = postgresUrl
? {
engine: 'postgres' as const,
inMemory: false,
connectionString: postgresTestUrl,
migrationPaths: [postgresMigrationsPath],
}
: {
engine: 'postgres' as const,
inMemory: true,
migrationPaths: [postgresMigrationsPath],
};
return {
port: 0,
extensions: [],
database: { ...database, ...(databaseOverrides ?? {}) },
...rootOverrides,
};
};
const quoteTestSchemaIdentifier = (schema: string): string => {
if (!postgresTestSchemaPattern.test(schema)) {
throw new Error(`Unsafe Postgres test schema name: ${schema}`);
}
return `"${schema}"`;
};
const postgresConnectionStringForSchema = (
connectionString: string,
schema: string,
): string => {
if (!postgresTestSchemaPattern.test(schema)) {
throw new Error(`Unsafe Postgres test schema name: ${schema}`);
}
const url = new URL(connectionString);
url.searchParams.set('options', `-c search_path=${schema}`);
return url.toString();
};
const createPostgresTestSchema = async (): Promise<void> => {
if (!postgresUrl) return;
const schema = `puter_test_${uuidv4().replaceAll('-', '')}`;
const pool = new Pool({ connectionString: postgresUrl });
try {
await pool.query(`CREATE SCHEMA ${quoteTestSchemaIdentifier(schema)}`);
postgresTestSchema = schema;
postgresTestUrl = postgresConnectionStringForSchema(
postgresUrl,
schema,
);
} finally {
await pool.end();
}
};
const dropPostgresTestSchema = async (): Promise<void> => {
if (!postgresUrl || !postgresTestSchema) return;
const schema = postgresTestSchema;
postgresTestSchema = undefined;
postgresTestUrl = undefined;
const pool = new Pool({ connectionString: postgresUrl });
try {
await pool.query(
`DROP SCHEMA IF EXISTS ${quoteTestSchemaIdentifier(schema)} CASCADE`,
);
} finally {
await pool.end();
}
};
describe('PostgresDatabaseClient integration', () => {
let server: PuterServer | undefined;
beforeEach(async () => {
await createPostgresTestSchema();
});
afterEach(async () => {
try {
await server?.shutdown();
} finally {
server = undefined;
await dropPostgresTestSchema();
}
});
it(
'applies the native migrations idempotently to an empty database',
async () => {
const config = postgresConfig();
const pgMockClient = postgresUrl
? undefined
: await createPgMockPostgresDatabaseClient(config);
try {
const firstClient =
pgMockClient?.client ?? new PostgresDatabaseClient(config);
try {
await firstClient.onServerStart();
} finally {
await firstClient.onServerShutdown();
}
const secondClient =
pgMockClient?.createClient() ??
new PostgresDatabaseClient(config);
try {
await secondClient.onServerStart();
const [systemUser] = await secondClient.read(
'SELECT `id`, `username` FROM `user` WHERE `username` = ?',
['system'],
);
const [devCenter] = await secondClient.read(
'SELECT `name`, `index_url` FROM `apps` WHERE `name` = ?',
['dev-center'],
);
expect(systemUser).toMatchObject({
id: 1,
username: 'system',
});
expect(devCenter).toMatchObject({
name: 'dev-center',
index_url:
'https://builtins.namespaces.puter.com/dev-center',
});
} finally {
await secondClient.onServerShutdown();
}
} finally {
pgMockClient?.destroy();
}
},
postgresIntegrationTimeoutMs,
);
it('starts the server and exercises user, app, fsentry, permission, and session flows', async () => {
const userStorageAllowance = 123_456_789;
server = await setupTestServer(
postgresConfig({
no_default_user: false,
is_storage_limited: true,
}),
);
const admin = await server.stores.user.getByUsername('admin');
expect(admin?.username).toBe('admin');
const username = `pg-${uuidv4().slice(0, 8)}`;
const createdUser = await server.stores.user.create({
username,
uuid: uuidv4(),
password: null,
email: `${username}@test.local`,
free_storage: userStorageAllowance,
requires_email_confirmation: false,
});
await generateDefaultFsentries(
server.clients.db,
server.stores.user,
createdUser,
);
const user = await server.stores.user.getById(createdUser.id);
if (!user) throw new Error('created user was not readable');
expect(user.username).toBe(username);
await expect(
server.stores.fsEntry.getUserStorageAllowance(user.id),
).resolves.toMatchObject({
max: userStorageAllowance,
});
const otherUsername = `pg-other-${uuidv4().slice(0, 8)}`;
const otherUser = await server.stores.user.create({
username: otherUsername,
uuid: uuidv4(),
password: null,
email: `${otherUsername}@test.local`,
free_storage: 100 * 1024 * 1024,
requires_email_confirmation: false,
});
const authResult = await server.services.auth.createSessionToken(
user,
{
ip: '127.0.0.1',
user_agent: 'postgres-integration-test',
},
);
const authenticated =
await server.services.auth.authenticateFromToken(authResult.token);
expect(authenticated?.user?.id).toBe(user?.id);
const devCenter = await server.stores.app.getByName('dev-center');
if (!devCenter) throw new Error('dev-center app was not seeded');
expect(devCenter.name).toBe('dev-center');
const documents = await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents`,
);
if (!documents) throw new Error('Documents directory was not created');
expect(documents.isDir).toBe(true);
const folder = await server.stores.fsEntry.createNonFileEntry({
userId: user.id,
parent: documents,
name: 'postgres-folder',
kind: 'directory',
});
const renamedFolder = await server.stores.fsEntry.updateEntry(
folder.uuid,
{
name: 'postgres-folder-renamed',
path: `/${username}/Documents/postgres-folder-renamed`,
},
);
expect(renamedFolder.name).toBe('postgres-folder-renamed');
await server.stores.permission.upsertUserAppPerm(
user.id,
Number(devCenter.id),
'driver:postgres-integration',
{ ok: true },
);
await expect(
server.stores.permission.hasUserAppPerm(
user.id,
Number(devCenter.id),
'driver:postgres-integration',
),
).resolves.toBe(true);
await server.stores.oidc.link(
user.id,
'postgres-integration-test',
'subject-1',
null,
);
await expect(
server.stores.oidc.link(
user.id,
'postgres-integration-test',
'subject-1',
null,
),
).resolves.toBeUndefined();
await expect(
server.stores.oidc.link(
otherUser.id,
'postgres-integration-test',
'subject-1',
null,
),
).rejects.toMatchObject({
statusCode: 409,
legacyCode: 'conflict',
});
const session = await server.stores.session.create(user.id, {
meta: { source: 'postgres-integration-test' },
});
const activeSession = await server.stores.session.getByUuid(
session.uuid,
);
expect(activeSession?.uuid).toBe(session.uuid);
const workerName = `worker-${uuidv4().slice(0, 8)}`;
const workerSession = await server.stores.session.getOrCreateWorker(
user.id,
{ workerName },
);
expect(workerSession?.kind).toBe('worker');
expect(workerSession?.meta?.worker_name).toBe(workerName);
await server.stores.session.removeByUuid(session.uuid);
await expect(
server.stores.session.getByUuid(session.uuid),
).resolves.toBeNull();
}, postgresIntegrationTimeoutMs);
});
@@ -0,0 +1,360 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { FieldDef, QueryResult } from 'pg';
import { describe, expect, it } from 'vitest';
import type { IConfig } from '../../types';
import { DatabaseClientFactory } from './index.js';
import {
mapPostgresWriteResult,
PostgresDatabaseClient,
type PostgresPool,
type PostgresPoolClient,
} from './PostgresDatabaseClient.js';
type QueryCall = {
text: string;
values?: unknown[];
};
const postgresConfig = (): IConfig => ({
port: 0,
extensions: [],
database: {
engine: 'postgres',
migrationPaths: [],
},
});
const postgresReplicaConfig = (): IConfig => ({
port: 0,
extensions: [],
database: {
engine: 'postgres',
migrationPaths: [],
replica: {},
},
});
const field = (name: string, dataTypeID: number): FieldDef => ({
name,
tableID: 0,
columnID: 0,
dataTypeID,
dataTypeSize: -1,
dataTypeModifier: -1,
format: 'text',
});
const int8Field = (name: string): FieldDef => field(name, 20);
const textField = (name: string): FieldDef => field(name, 25);
const queryResult = (
rows: Record<string, unknown>[] = [],
rowCount = rows.length,
fields: FieldDef[] = [],
): QueryResult<Record<string, unknown>> => ({
command: '',
fields,
oid: 0,
rowCount,
rows,
});
class RecordingPoolClient implements PostgresPoolClient {
readonly calls: QueryCall[] = [];
released = false;
constructor(private readonly failOnText?: string) {}
async query(
text: string,
values?: unknown[],
): Promise<QueryResult<Record<string, unknown>>> {
this.calls.push({ text, values });
if (this.failOnText && text.includes(this.failOnText)) {
throw new Error(`forced query failure: ${text}`);
}
return queryResult([], text === 'ROLLBACK' ? 0 : 1);
}
release(): void {
this.released = true;
}
}
class RecordingPool implements PostgresPool {
readonly calls: QueryCall[] = [];
constructor(
private readonly client: RecordingPoolClient =
new RecordingPoolClient(),
private readonly nextResult: QueryResult<Record<string, unknown>> =
queryResult(),
) {}
async query(
text: string,
values?: unknown[],
): Promise<QueryResult<Record<string, unknown>>> {
this.calls.push({ text, values });
if (text === 'SELECT 1') return queryResult([{ ok: 1 }], 1);
return this.nextResult;
}
async connect(): Promise<PostgresPoolClient> {
return this.client;
}
async end(): Promise<void> {}
}
describe('PostgresDatabaseClient', () => {
it('is selected by the database factory', () => {
const client = new DatabaseClientFactory(postgresConfig());
expect(client).toBeInstanceOf(PostgresDatabaseClient);
});
it('maps pg write results to the shared WriteResult shape', () => {
expect(
mapPostgresWriteResult(queryResult([{ id: '42' }], 1)),
).toEqual({
insertId: 42,
affectedRows: 1,
anyRowsAffected: true,
});
expect(mapPostgresWriteResult(queryResult([], 0))).toEqual({
insertId: 0,
affectedRows: 0,
anyRowsAffected: false,
});
});
it('prepares write SQL at the pg boundary', async () => {
const pool = new RecordingPool(
new RecordingPoolClient(),
queryResult([{ id: '7' }], 1),
);
const client = new PostgresDatabaseClient(postgresConfig(), () => pool);
await client.onServerStart();
const result = await client.write(
'INSERT INTO `apps` (`name`) VALUES (?) RETURNING id',
['editor'],
);
expect(result.insertId).toBe(7);
expect(pool.calls.at(-1)).toEqual({
text: 'INSERT INTO "apps" ("name") VALUES ($1) RETURNING id',
values: ['editor'],
});
});
it('normalizes int8 fields on read and primary read rows', async () => {
const pool = new RecordingPool(
new RecordingPoolClient(),
queryResult(
[
{
uuid: 'session-1',
created_at: '1710000000',
last_activity: '1710000001',
expires_at: '1710000002',
revoked_at: null,
},
],
1,
[
textField('uuid'),
int8Field('created_at'),
int8Field('last_activity'),
int8Field('expires_at'),
int8Field('revoked_at'),
],
),
);
const client = new PostgresDatabaseClient(postgresConfig(), () => pool);
await client.onServerStart();
await expect(client.read('SELECT * FROM `sessions`')).resolves.toEqual(
[
{
uuid: 'session-1',
created_at: 1710000000,
last_activity: 1710000001,
expires_at: 1710000002,
revoked_at: null,
},
],
);
await expect(client.pread('SELECT * FROM `sessions`')).resolves.toEqual(
[
{
uuid: 'session-1',
created_at: 1710000000,
last_activity: 1710000001,
expires_at: 1710000002,
revoked_at: null,
},
],
);
});
it('rejects unsafe int8 values instead of losing precision', async () => {
const pool = new RecordingPool(
new RecordingPoolClient(),
queryResult(
[{ id: '9007199254740992' }],
1,
[int8Field('id')],
),
);
const client = new PostgresDatabaseClient(postgresConfig(), () => pool);
await client.onServerStart();
await expect(client.read('SELECT `id` FROM `sessions`')).rejects.toThrow(
'safe integer',
);
});
it('normalizes tryHardRead rows returned by a replica', async () => {
const primaryPool = new RecordingPool(
new RecordingPoolClient(),
queryResult([{ created_at: '1' }], 1, [
int8Field('created_at'),
]),
);
const replicaPool = new RecordingPool(
new RecordingPoolClient(),
queryResult([{ created_at: '2' }], 1, [
int8Field('created_at'),
]),
);
const pools = [primaryPool, replicaPool];
let nextPoolIndex = 0;
const client = new PostgresDatabaseClient(
postgresReplicaConfig(),
() => {
const pool = pools[nextPoolIndex];
nextPoolIndex += 1;
if (!pool) throw new Error('unexpected pool factory call');
return pool;
},
);
await client.onServerStart();
await expect(
client.tryHardRead('SELECT `created_at` FROM `sessions`'),
).resolves.toEqual([{ created_at: 2 }]);
});
it('normalizes tryHardRead rows returned by primary fallback', async () => {
const primaryPool = new RecordingPool(
new RecordingPoolClient(),
queryResult([{ created_at: '3' }], 1, [
int8Field('created_at'),
]),
);
const replicaPool = new RecordingPool(
new RecordingPoolClient(),
queryResult([], 0, [int8Field('created_at')]),
);
const pools = [primaryPool, replicaPool];
let nextPoolIndex = 0;
const client = new PostgresDatabaseClient(
postgresReplicaConfig(),
() => {
const pool = pools[nextPoolIndex];
nextPoolIndex += 1;
if (!pool) throw new Error('unexpected pool factory call');
return pool;
},
);
await client.onServerStart();
await expect(
client.tryHardRead('SELECT `created_at` FROM `sessions`'),
).resolves.toEqual([{ created_at: 3 }]);
});
it('runs batch writes in order and commits', async () => {
const conn = new RecordingPoolClient();
const client = new PostgresDatabaseClient(
postgresConfig(),
() => new RecordingPool(conn),
);
await client.onServerStart();
await client.batchWrite([
{
statement:
'UPDATE `user` SET `username` = ? WHERE `id` = ?',
values: ['ada', 1],
},
{
statement: 'DELETE FROM `sessions` WHERE `uuid` = ?',
values: ['session-1'],
},
]);
expect(conn.calls).toEqual([
{ text: 'BEGIN', values: undefined },
{
text: 'UPDATE "user" SET "username" = $1 WHERE "id" = $2',
values: ['ada', 1],
},
{
text: 'DELETE FROM "sessions" WHERE "uuid" = $1',
values: ['session-1'],
},
{ text: 'COMMIT', values: undefined },
]);
expect(conn.released).toBe(true);
});
it('rolls back and releases the pg connection when a batch write fails', async () => {
const conn = new RecordingPoolClient('UPDATE "user"');
const client = new PostgresDatabaseClient(
postgresConfig(),
() => new RecordingPool(conn),
);
await client.onServerStart();
await expect(
client.batchWrite([
{
statement:
'UPDATE `user` SET `username` = ? WHERE `id` = ?',
values: ['ada', 1],
},
]),
).rejects.toThrow('forced query failure');
expect(conn.calls).toEqual([
{ text: 'BEGIN', values: undefined },
{
text: 'UPDATE "user" SET "username" = $1 WHERE "id" = $2',
values: ['ada', 1],
},
{ text: 'ROLLBACK', values: undefined },
]);
expect(conn.released).toBe(true);
});
});
@@ -0,0 +1,390 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { readdirSync, readFileSync } from 'fs';
import { isAbsolute, resolve as resolvePath } from 'path';
import { Pool, type PoolConfig, type QueryResult } from 'pg';
import {
AbstractDatabaseClient,
type BatchEntry,
type WriteResult,
} from './DatabaseClient';
import { compareMigrationFilenames } from './migrationFilenames.js';
import { preparePostgresSql } from './preparePostgresSql.js';
import { splitPostgresStatements } from './splitPostgresStatements.js';
import type { IConfig } from '../../types';
type PostgresEndpointConfig = {
host?: string;
port?: number;
user?: string;
password?: string;
database?: string;
connectionString?: string;
url?: string;
};
export interface PostgresQueryable {
query(query: string, values?: unknown[]): Promise<QueryResult>;
}
export interface PostgresPoolClient extends PostgresQueryable {
release(): void;
}
export interface PostgresPool extends PostgresQueryable {
connect(): Promise<PostgresPoolClient>;
end(): Promise<void>;
}
type PostgresPoolFactory = (poolConfig: PoolConfig) => PostgresPool;
enum Configuration {
SINGLE,
REPLICA,
}
const POSTGRES_INT8_OID = 20;
const INTEGER_TEXT_PATTERN = /^-?\d+$/u;
const normalizePostgresInt8 = (
value: unknown,
columnName: string,
): number | null | undefined => {
if (value === null || value === undefined) return value;
const parsed =
typeof value === 'bigint'
? Number(value)
: typeof value === 'number'
? value
: typeof value === 'string' && INTEGER_TEXT_PATTERN.test(value)
? Number(value)
: Number.NaN;
if (!Number.isSafeInteger(parsed)) {
throw new Error(
`[postgres] int8 column ${columnName} is outside JavaScript's safe integer range`,
);
}
return parsed;
};
const normalizePostgresRows = (
result: QueryResult,
): Record<string, unknown>[] => {
const int8Fields = result.fields.filter(
(field) => field.dataTypeID === POSTGRES_INT8_OID,
);
if (int8Fields.length === 0) {
return result.rows as Record<string, unknown>[];
}
return result.rows.map((row) => {
const normalized: Record<string, unknown> = { ...row };
for (const field of int8Fields) {
normalized[field.name] = normalizePostgresInt8(
normalized[field.name],
field.name,
);
}
return normalized;
});
};
export const mapPostgresWriteResult = (result: QueryResult): WriteResult => {
const affectedRows = result.rowCount ?? 0;
const rowId = result.rows[0]?.id;
const insertId =
typeof rowId === 'bigint'
? rowId
: typeof rowId === 'number'
? rowId
: typeof rowId === 'string' && rowId !== ''
? Number(rowId)
: 0;
const normalizedInsertId =
typeof insertId === 'number' && Number.isNaN(insertId) ? 0 : insertId;
return {
insertId: normalizedInsertId,
affectedRows,
anyRowsAffected: affectedRows > 0,
};
};
export class PostgresDatabaseClient extends AbstractDatabaseClient {
override readonly engineName = 'postgres';
private primaryPool!: PostgresPool;
private replicaPool!: PostgresPool;
private configuration = Configuration.SINGLE;
private shutdownStarted = false;
private shutdownTimer: ReturnType<typeof setTimeout> | null = null;
constructor(
config: IConfig,
private readonly poolFactory: PostgresPoolFactory = (poolConfig) =>
new Pool(poolConfig) as unknown as PostgresPool,
) {
super(config);
}
override async onServerStart(): Promise<void> {
const dbConf = this.config.database!;
this.primaryPool = this.createPool(dbConf);
await this.primaryPool.query('SELECT 1');
console.log('[postgres] connected to primary');
if (dbConf.replica) {
this.replicaPool = this.createPool(dbConf.replica);
await this.replicaPool.query('SELECT 1');
this.configuration = Configuration.REPLICA;
console.log('[postgres] connected to read-replica');
} else {
this.replicaPool = this.primaryPool;
this.configuration = Configuration.SINGLE;
}
await this.runMigrations();
}
override async onServerPrepareShutdown(): Promise<void> {
if (this.shutdownStarted) return;
this.shutdownStarted = true;
const drainMs = 60_000;
console.log(
`[postgres] draining in-flight queries (${drainMs}ms) before closing pools`,
);
this.shutdownTimer = setTimeout(() => {
this.shutdownTimer = null;
this.closeCurrentPools().catch((e) =>
console.error('[postgres] error closing pools after drain', e),
);
}, drainMs);
if (typeof this.shutdownTimer.unref === 'function') {
this.shutdownTimer.unref();
}
}
override async onServerShutdown(): Promise<void> {
if (this.shutdownTimer) {
clearTimeout(this.shutdownTimer);
this.shutdownTimer = null;
}
await this.closeCurrentPools();
}
override quoteIdentifier(identifier: string): string {
return identifier
.split('.')
.map((part) => {
if (part === '*') return part;
return `"${part.replaceAll('"', '""')}"`;
})
.join('.');
}
override booleanLiteral(value: boolean): string {
return value ? 'TRUE' : 'FALSE';
}
override booleanValue(value: boolean): boolean {
return value;
}
override async read(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
const result = await this.query(this.replicaPool, query, params);
return normalizePostgresRows(result);
}
override async pread(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
const result = await this.query(this.primaryPool, query, params);
return normalizePostgresRows(result);
}
override async write(
query: string,
params: unknown[] = [],
): Promise<WriteResult> {
const result = await this.query(this.primaryPool, query, params);
return mapPostgresWriteResult(result);
}
override async batchWrite(entries: BatchEntry[]): Promise<void> {
if (entries.length === 0) return;
const conn = await this.primaryPool.connect();
try {
await conn.query('BEGIN');
try {
for (const { statement, values } of entries) {
await this.query(conn, statement, values);
}
await conn.query('COMMIT');
} catch (err) {
await conn.query('ROLLBACK').catch(() => {});
throw err;
}
} finally {
conn.release();
}
}
override async tryHardRead(
query: string,
params: unknown[] = [],
): Promise<Record<string, unknown>[]> {
if (this.configuration === Configuration.SINGLE) {
return this.read(query, params);
}
const primaryPromise = this.query(this.primaryPool, query, params);
try {
const replicaResult = await this.query(
this.replicaPool,
query,
params,
);
if (replicaResult.rows.length > 0) {
primaryPromise.catch(() => {});
return normalizePostgresRows(replicaResult);
}
} catch {
// fall through to primary
}
const primaryResult = await primaryPromise;
return normalizePostgresRows(primaryResult);
}
private async runMigrations(): Promise<void> {
const paths = this.config.database?.migrationPaths;
if (!paths || paths.length === 0) return;
const conn = await this.primaryPool.connect();
try {
for (const rawPath of paths) {
const dir = isAbsolute(rawPath)
? rawPath
: resolvePath(process.cwd(), rawPath);
let files: string[];
try {
files = readdirSync(dir)
.filter(
(f) =>
f.endsWith('.sql') && f.startsWith('postgres'),
)
.sort(compareMigrationFilenames);
} catch (e) {
throw new Error(
`[postgres] migration path is unreadable: ${dir}`,
{ cause: e },
);
}
if (files.length === 0) {
console.log(`[postgres] no migrations in ${dir}`);
continue;
}
console.log(
`[postgres] running migrations from ${dir}: ${files.length} file(s)`,
);
for (const file of files) {
const filePath = resolvePath(dir, file);
const contents = readFileSync(filePath, 'utf8');
const statements = splitPostgresStatements(contents);
await conn.query('BEGIN');
try {
for (let i = 0; i < statements.length; i++) {
try {
await conn.query(statements[i]);
} catch (e) {
throw new Error(
`[postgres] failed to apply ${file} at statement ${i}`,
{ cause: e },
);
}
}
await conn.query('COMMIT');
} catch (e) {
await conn.query('ROLLBACK').catch(() => {});
throw e;
}
console.log(
`[postgres] applied ${file} (${statements.length} statements)`,
);
}
}
} finally {
conn.release();
}
}
private createPool(dbConf: PostgresEndpointConfig): PostgresPool {
const connectionString = dbConf.connectionString ?? dbConf.url;
if (connectionString) {
return this.poolFactory({
connectionString,
max: 30,
});
}
return this.poolFactory({
host: dbConf.host ?? '127.0.0.1',
port: dbConf.port ?? 5432,
user: dbConf.user ?? 'postgres',
password: dbConf.password ?? '',
database: dbConf.database ?? 'puter',
max: 30,
});
}
private async query(
target: PostgresQueryable,
query: string,
params: unknown[] = [],
): Promise<QueryResult> {
const prepared = preparePostgresSql(query);
return target.query(prepared.text, params);
}
private async closeCurrentPools(): Promise<void> {
const tasks: Promise<void>[] = [];
if (this.primaryPool) tasks.push(this.primaryPool.end());
if (this.replicaPool && this.replicaPool !== this.primaryPool) {
tasks.push(this.replicaPool.end());
}
await Promise.all(tasks);
}
}
@@ -78,6 +78,21 @@ const AVAILABLE_MIGRATIONS: [number, string[]][] = [
[43, ['0047_app-url-updates.sql']],
[44, ['0048_old-app-names-unique-tuple.sql']],
[45, ['0049_music-player-pdf-player-updates.sql']],
[46, ['0050_add_preamble_version.sql']],
[47, ['0051_sessions_v2.sql']],
[48, ['0052_sessions_v2_lookups.sql']],
[49, ['0053_sessions_access_token_uid.sql']],
[50, ['0054_sessions_workers.sql']],
[50, ['0055_username_nocase_unique.sql']],
[51, ['0056_sessions_kind_worker.sql']],
[52, ['0057_add_user_reputation.sql']],
[53, ['0058_add_phone_verification.sql']],
[54, ['0059_add_card_verification.sql']],
[55, ['0060_add_card_fingerprint.sql']],
[56, ['0061_add_suspended_at.sql']],
[57, ['0062_blocked-app-origins.sql']],
[58, ['0063_add_suspended_reason.sql']],
[59, ['0064_abuse-moderation-events.sql']],
];
export class SqliteDatabaseClient extends AbstractDatabaseClient {
+4
View File
@@ -24,10 +24,12 @@ export {
} from './DatabaseClient';
export { SqliteDatabaseClient } from './SqliteDatabaseClient';
export { MySQLDatabaseClient } from './MySQLDatabaseClient';
export { PostgresDatabaseClient } from './PostgresDatabaseClient';
import type { IConfig } from '../../types';
import { AbstractDatabaseClient } from './DatabaseClient';
import { MySQLDatabaseClient } from './MySQLDatabaseClient';
import { PostgresDatabaseClient } from './PostgresDatabaseClient';
import { SqliteDatabaseClient } from './SqliteDatabaseClient';
/**
@@ -41,6 +43,8 @@ export const DatabaseClientFactory = class DatabaseClientFactory {
switch (engine) {
case 'mysql':
return new MySQLDatabaseClient(config);
case 'postgres':
return new PostgresDatabaseClient(config);
case 'sqlite':
return new SqliteDatabaseClient(config);
default:
@@ -0,0 +1,43 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/**
* Comparator for migration filenames named `<engine>_mig_<N>.sql`.
*
* Existing MySQL files use unpadded integers, so lexical sorting places
* `*_mig_10.sql` before `*_mig_2.sql`. Pull the trailing integer out and sort
* numerically. Anything that does not match the `_<digits>.sql` shape falls
* back to lexical comparison and sorts after numbered files.
*/
export const compareMigrationFilenames = (a: string, b: string): number => {
const numericIndex = (name: string): number => {
const m = /_(\d+)\.sql$/.exec(name);
return m ? Number.parseInt(m[1], 10) : Number.NaN;
};
const na = numericIndex(a);
const nb = numericIndex(b);
if (Number.isFinite(na) && Number.isFinite(nb)) {
if (na !== nb) return na - nb;
} else if (Number.isFinite(na)) {
return -1;
} else if (Number.isFinite(nb)) {
return 1;
}
return a.localeCompare(b);
};
@@ -24,7 +24,9 @@
--
--
-- Idempotent column-ensure helper (used by CALLs below; dropped at end of file)
-- Idempotent column-ensure helper. Used by the CALLs below AND by later
-- migration files, which CALL it as a one-line idempotent column add. Left
-- resident (not dropped at end of file) so it outlives this migration.
--
DROP PROCEDURE IF EXISTS _puter_add_col;
@@ -1333,7 +1335,10 @@ CALL _puter_add_col('user_update_audit', 'new_username', '`new_username` varchar
CALL _puter_add_col('user_update_audit', 'reason', '`reason` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_col;
-- _puter_add_col is intentionally NOT dropped here: later migration files CALL
-- it as a one-line idempotent column add, and migrations replay on every boot
-- with no applied-state tracking, so the helper must outlive this file. The
-- DROP-before-CREATE at the top keeps this migration itself replay-safe.
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;
/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
@@ -0,0 +1,53 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Mirrors SQLite migration 0053. Adds the
-- `access_token_uid` reverse-lookup column on `sessions` so raw-uuid
-- revoke can find the matching session row when only the v2 token_uid
-- (no JWT) is presented.
--
-- Idempotent: each ADD COLUMN / ADD INDEX is guarded so the migration
-- directory can be replayed safely.
DROP PROCEDURE IF EXISTS _puter_sessions_access_token_uid;
DELIMITER //
CREATE PROCEDURE _puter_sessions_access_token_uid()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'access_token_uid'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `access_token_uid` VARCHAR(64) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND INDEX_NAME = 'idx_sessions_access_token_uid'
) THEN
ALTER TABLE `sessions`
ADD INDEX `idx_sessions_access_token_uid` (`access_token_uid`);
END IF;
END//
DELIMITER ;
CALL _puter_sessions_access_token_uid();
DROP PROCEDURE IF EXISTS _puter_sessions_access_token_uid;
@@ -0,0 +1,70 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Worker session uniqueness. Each Puter worker is a separately-deployed
-- code unit (its own subdomain), so one user can have many workers under
-- the same app — distinguished by `meta.worker_name`. The natural unique
-- key for an active worker session is therefore
-- (user_id, app_uid, worker_name), and app_uid is allowed NULL for
-- user-scoped workers that aren't bound to any specific app.
--
-- Implemented the same way as `app_unique_key` from mig_9: a VIRTUAL
-- generated column that's non-NULL only for the rows under the rule,
-- then a UNIQUE INDEX on the column. NULLs don't conflict in MySQL
-- UNIQUE indexes, so soft-revoked / non-worker rows fall out
-- automatically. IFNULL normalises NULL `app_uid` so two user-scoped
-- workers with the same name still dedupe. JSON_UNQUOTE strips the
-- JSON value quoting from JSON_EXTRACT so the concatenated key is a
-- plain string that matches the SELECT path's binding.
--
-- Idempotent: each ADD COLUMN / ADD INDEX is guarded so the migration
-- directory can be replayed safely.
DROP PROCEDURE IF EXISTS _puter_sessions_worker_unique;
DELIMITER //
CREATE PROCEDURE _puter_sessions_worker_unique()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'worker_unique_key'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `worker_unique_key` VARCHAR(550)
GENERATED ALWAYS AS (
IF(`kind` = 'worker' AND `revoked_at` IS NULL,
CONCAT(`user_id`, '|', IFNULL(`app_uid`, ''), '|',
IFNULL(JSON_UNQUOTE(JSON_EXTRACT(`meta`, '$.worker_name')), '')),
NULL)
) VIRTUAL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND INDEX_NAME = 'idx_sessions_user_worker_active'
) THEN
ALTER TABLE `sessions`
ADD UNIQUE INDEX `idx_sessions_user_worker_active` (`worker_unique_key`);
END IF;
END//
DELIMITER ;
CALL _puter_sessions_worker_unique();
DROP PROCEDURE IF EXISTS _puter_sessions_worker_unique;
@@ -0,0 +1,50 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Add 'worker' to the `sessions.kind` ENUM. mig_11 already added the
-- worker uniqueness index and the application code in SessionStore
-- inserts rows with kind='worker', but the ENUM defined in mig_8 never
-- listed 'worker' as a permitted value. Under STRICT_TRANS_TABLES the
-- INSERT fails outright; under a relaxed sql_mode the value is coerced
-- to '' (and the worker SELECT-by-kind path then misses the row).
-- Mirrors SQLite migration 0056.
--
-- Idempotent: guarded against COLUMN_TYPE so re-running the directory is
-- a no-op once 'worker' is in the ENUM.
DROP PROCEDURE IF EXISTS _puter_sessions_kind_worker;
DELIMITER //
CREATE PROCEDURE _puter_sessions_kind_worker()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND COLUMN_NAME = 'kind'
AND FIND_IN_SET('worker', REPLACE(REPLACE(REPLACE(COLUMN_TYPE, 'enum(', ''), ')', ''), '''', '')) > 0
) THEN
ALTER TABLE `sessions`
MODIFY COLUMN `kind`
ENUM('web', 'app', 'access_token', 'asset', 'worker')
NOT NULL DEFAULT 'web';
END IF;
END//
DELIMITER ;
CALL _puter_sessions_kind_worker();
DROP PROCEDURE IF EXISTS _puter_sessions_kind_worker;
@@ -0,0 +1,47 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- SMS phone verification columns. Mirrors SQLite migration 0058. `phone` is the
-- E.164 number collected during verification (indexed like `email`);
-- `requires_phone_verification` gates account use for low-reputation signups
-- (not indexed, mirroring `requires_email_confirmation`).
--
-- Idempotent: column adds use _puter_add_col (defined in mig_1, which leaves it
-- resident for later migrations); the index add is guarded against
-- INFORMATION_SCHEMA.STATISTICS so the directory replays safely.
CALL _puter_add_col('user', 'phone', '`phone` varchar(20) DEFAULT NULL');
CALL _puter_add_col('user', 'requires_phone_verification', '`requires_phone_verification` tinyint(1) NOT NULL DEFAULT ''0''');
DROP PROCEDURE IF EXISTS _puter_add_user_phone_index;
DELIMITER //
CREATE PROCEDURE _puter_add_user_phone_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user'
AND INDEX_NAME = 'idx_user_phone'
) THEN
ALTER TABLE `user` ADD INDEX `idx_user_phone` (`phone`);
END IF;
END//
DELIMITER ;
CALL _puter_add_user_phone_index();
DROP PROCEDURE IF EXISTS _puter_add_user_phone_index;
@@ -0,0 +1,26 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Credit-card verification column. Mirrors SQLite migration 0059.
-- `requires_card_verification` gates account use for low-reputation signups;
-- the card itself never touches our DB, so this is the only column (not
-- indexed, mirroring `requires_phone_verification`).
--
-- Idempotent: the column add uses _puter_add_col (from mig_1) so the
-- directory replays safely.
CALL _puter_add_col('user', 'requires_card_verification', '`requires_card_verification` tinyint(1) NOT NULL DEFAULT ''0''');
@@ -0,0 +1,47 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Card fingerprint column. Mirrors SQLite migration 0060. `card_fingerprint`
-- is the Stripe card fingerprint (stable per card number) recorded when a user
-- clears card verification — the card sibling of `phone`, indexed like it so
-- admin tooling can find the accounts that verified with a given card. The card
-- itself never touches our DB, only Stripe's fingerprint for it.
--
-- Idempotent: the column add uses _puter_add_col (defined in mig_1, which
-- leaves it resident for later migrations); the index add is guarded against
-- INFORMATION_SCHEMA.STATISTICS so the directory replays safely.
CALL _puter_add_col('user', 'card_fingerprint', '`card_fingerprint` varchar(128) DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_user_card_fingerprint_index;
DELIMITER //
CREATE PROCEDURE _puter_add_user_card_fingerprint_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user'
AND INDEX_NAME = 'idx_user_card_fingerprint'
) THEN
ALTER TABLE `user` ADD INDEX `idx_user_card_fingerprint` (`card_fingerprint`);
END IF;
END//
DELIMITER ;
CALL _puter_add_user_card_fingerprint_index();
DROP PROCEDURE IF EXISTS _puter_add_user_card_fingerprint_index;
@@ -0,0 +1,46 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Suspended-at column. Mirrors SQLite migration 0061. `suspended_at` is when
-- the account was suspended, as unix seconds (NULL while not suspended) — the
-- timestamp sibling of the boolean `suspended` flag, indexed so the
-- signup-abuse harness can count an IP's recently-suspended accounts.
--
-- Idempotent: the column add uses _puter_add_col (defined in mig_1, which
-- leaves it resident for later migrations); the index add is guarded against
-- INFORMATION_SCHEMA.STATISTICS so the directory replays safely.
CALL _puter_add_col('user', 'suspended_at', '`suspended_at` bigint DEFAULT NULL');
DROP PROCEDURE IF EXISTS _puter_add_user_suspended_at_index;
DELIMITER //
CREATE PROCEDURE _puter_add_user_suspended_at_index()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'user'
AND INDEX_NAME = 'idx_user_suspended_at'
) THEN
ALTER TABLE `user` ADD INDEX `idx_user_suspended_at` (`suspended_at`);
END IF;
END//
DELIMITER ;
CALL _puter_add_user_suspended_at_index();
DROP PROCEDURE IF EXISTS _puter_add_user_suspended_at_index;
@@ -0,0 +1,36 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Admin-managed blocklist of app origins. Mirrors SQLite migration 0062.
-- An app whose `index_url` host (or a request origin) matches an entry is
-- denied access to Puter resources: it cannot obtain an app token and
-- already-issued app tokens are rejected on each request. `include_subdomains
-- = 1` also blocks every subdomain of `domain`. Enforced in AuthService via
-- AppOriginBlocklistService.
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` lets the directory replay safely.
CREATE TABLE IF NOT EXISTS `blocked_app_origins` (
`id` INT NOT NULL AUTO_INCREMENT,
`domain` VARCHAR(255) NOT NULL,
`include_subdomains` TINYINT(1) NOT NULL DEFAULT 0,
`reason` TEXT DEFAULT NULL,
`created_by` VARCHAR(255) DEFAULT NULL,
`created_at` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_blocked_app_origins_domain` (`domain`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,26 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Suspension reason column. Mirrors SQLite migration 0063. Why an account was
-- suspended (NULL while not suspended) — companion to the boolean `suspended`
-- flag and `suspended_at` timestamp. Constrained at the application layer to a
-- fixed set of reasons (see extensions/admin suspension_reasons.js).
--
-- Idempotent: the column add uses _puter_add_col (defined in mig_1, which
-- leaves it resident for later migrations).
CALL _puter_add_col('user', 'suspended_reason', '`suspended_reason` VARCHAR(64) DEFAULT NULL');
@@ -0,0 +1,38 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Append-only log of admin moderation actions. Mirrors SQLite migration 0064.
-- Used to measure the abuse system's false-positive rate: `unsuspend` (admin
-- unblock) and `admin_create_user` are the false-positive signals; `suspend`
-- gives the denominator. Preserves history the `user` suspension columns lose
-- on unsuspend. `created_at` is unix seconds (matching `user.suspended_at`).
--
-- Idempotent: `CREATE TABLE IF NOT EXISTS` lets the directory replay safely.
CREATE TABLE IF NOT EXISTS `abuse_moderation_events` (
`id` INT NOT NULL AUTO_INCREMENT,
`action` VARCHAR(32) NOT NULL,
`target_user_id` BIGINT DEFAULT NULL,
`target_username` VARCHAR(255) DEFAULT NULL,
`admin_username` VARCHAR(255) DEFAULT NULL,
`reason` TEXT DEFAULT NULL,
`source` VARCHAR(64) DEFAULT NULL,
`created_at` BIGINT NOT NULL,
PRIMARY KEY (`id`),
KEY `idx_abuse_moderation_events_created_at` (`created_at`),
KEY `idx_abuse_moderation_events_action` (`action`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
@@ -0,0 +1,39 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
--
-- Idempotent: the ADD COLUMN is guarded by an INFORMATION_SCHEMA check,
-- so re-running the migration directory is safe (required — the runner
-- has no per-file tracking).
DROP PROCEDURE IF EXISTS _puter_add_subdomains_preamble_version;
DELIMITER //
CREATE PROCEDURE _puter_add_subdomains_preamble_version()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'subdomains' AND COLUMN_NAME = 'preamble_version'
) THEN
ALTER TABLE `subdomains`
ADD COLUMN `preamble_version` varchar(64) DEFAULT NULL;
END IF;
END//
DELIMITER ;
CALL _puter_add_subdomains_preamble_version();
DROP PROCEDURE IF EXISTS _puter_add_subdomains_preamble_version;
@@ -0,0 +1,98 @@
-- Extend `sessions` so a single row can represent any token kind
-- (web/app/access_token/asset), carry display metadata for the
-- manage-sessions UI, and be soft-revoked with row-level expiry.
-- Mirrors SQLite migration 0050.
--
-- Idempotent: each ADD COLUMN / ADD INDEX is guarded by an
-- INFORMATION_SCHEMA check, so re-running the migration directory
-- is safe (required — the runner has no per-file tracking).
DROP PROCEDURE IF EXISTS _puter_extend_sessions_v2;
DELIMITER //
CREATE PROCEDURE _puter_extend_sessions_v2()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'kind'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `kind` ENUM('web', 'app', 'access_token', 'asset')
NOT NULL DEFAULT 'web';
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'label'
) THEN
ALTER TABLE `sessions` ADD COLUMN `label` VARCHAR(255) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'parent_session_id'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `parent_session_id` VARCHAR(64) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'last_ip'
) THEN
ALTER TABLE `sessions` ADD COLUMN `last_ip` VARCHAR(64) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'last_user_agent'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `last_user_agent` VARCHAR(512) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'revoked_at'
) THEN
ALTER TABLE `sessions` ADD COLUMN `revoked_at` BIGINT DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'expires_at'
) THEN
ALTER TABLE `sessions` ADD COLUMN `expires_at` BIGINT DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND INDEX_NAME = 'idx_sessions_user_revoked'
) THEN
ALTER TABLE `sessions`
ADD INDEX `idx_sessions_user_revoked` (`user_id`, `revoked_at`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND INDEX_NAME = 'idx_sessions_parent'
) THEN
ALTER TABLE `sessions`
ADD INDEX `idx_sessions_parent` (`parent_session_id`);
END IF;
END//
DELIMITER ;
CALL _puter_extend_sessions_v2();
DROP PROCEDURE IF EXISTS _puter_extend_sessions_v2;
@@ -0,0 +1,130 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Composite-key lookups + audit columns. Mirrors
-- SQLite migration 0052. MySQL has no partial unique indexes, so the
-- "at most one active row per (user_id, app_uid)" / "one active row
-- per legacy_token_uid" semantics are encoded via VIRTUAL generated
-- columns that are NULL when the row isn't subject to the rule —
-- MySQL allows multiple NULLs in a UNIQUE index, so non-applicable
-- rows don't conflict.
--
-- Idempotent: each ADD COLUMN / ADD INDEX is guarded so the migration
-- directory can be replayed safely.
DROP PROCEDURE IF EXISTS _puter_sessions_v2_lookups;
DELIMITER //
CREATE PROCEDURE _puter_sessions_v2_lookups()
BEGIN
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'app_uid'
) THEN
ALTER TABLE `sessions` ADD COLUMN `app_uid` VARCHAR(64) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'legacy_token_uid'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `legacy_token_uid` VARCHAR(64) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'created_via'
) THEN
ALTER TABLE `sessions` ADD COLUMN `created_via` VARCHAR(32) DEFAULT NULL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'auth_id'
) THEN
ALTER TABLE `sessions` ADD COLUMN `auth_id` VARCHAR(64) DEFAULT NULL;
END IF;
-- Generated discriminant: non-NULL only for active app-authorization rows,
-- so UNIQUE(app_unique_key) enforces "one active app session per
-- (user_id, app_uid)" while permitting any number of revoked rows.
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'app_unique_key'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `app_unique_key` VARCHAR(150)
GENERATED ALWAYS AS (
IF(`kind` = 'app' AND `revoked_at` IS NULL,
CONCAT(`user_id`, '|', `app_uid`),
NULL)
) VIRTUAL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions' AND COLUMN_NAME = 'legacy_token_unique_key'
) THEN
ALTER TABLE `sessions`
ADD COLUMN `legacy_token_unique_key` VARCHAR(64)
GENERATED ALWAYS AS (
IF(`revoked_at` IS NULL, `legacy_token_uid`, NULL)
) VIRTUAL;
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND INDEX_NAME = 'idx_sessions_user_app_active'
) THEN
ALTER TABLE `sessions`
ADD UNIQUE INDEX `idx_sessions_user_app_active` (`app_unique_key`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND INDEX_NAME = 'idx_sessions_legacy_token_active'
) THEN
ALTER TABLE `sessions`
ADD UNIQUE INDEX `idx_sessions_legacy_token_active`
(`legacy_token_unique_key`);
END IF;
IF NOT EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'sessions'
AND INDEX_NAME = 'idx_sessions_kind_user'
) THEN
ALTER TABLE `sessions`
ADD INDEX `idx_sessions_kind_user` (`kind`, `user_id`);
END IF;
END//
DELIMITER ;
CALL _puter_sessions_v2_lookups();
DROP PROCEDURE IF EXISTS _puter_sessions_v2_lookups;
@@ -0,0 +1,724 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
CREATE TABLE IF NOT EXISTS "user" (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uuid varchar(36) NOT NULL UNIQUE,
username varchar(50) UNIQUE,
email varchar(256),
password varchar(225),
free_storage bigint,
max_subdomains integer,
taskbar_items text,
desktop_uuid varchar(36),
appdata_uuid varchar(36),
documents_uuid varchar(36),
pictures_uuid varchar(36),
videos_uuid varchar(36),
trash_uuid varchar(36),
trash_id integer,
appdata_id integer,
desktop_id integer,
documents_id integer,
pictures_id integer,
videos_id integer,
referrer varchar(64),
desktop_bg_url text,
desktop_bg_color varchar(20),
desktop_bg_fit varchar(16),
pass_recovery_token varchar(36),
requires_email_confirmation boolean NOT NULL DEFAULT FALSE,
email_confirm_code varchar(8),
email_confirm_token varchar(36),
email_confirmed boolean NOT NULL DEFAULT FALSE,
dev_first_name varchar(100),
dev_last_name varchar(100),
dev_paypal varchar(100),
dev_approved_for_incentive_program boolean DEFAULT FALSE,
dev_joined_incentive_program boolean DEFAULT FALSE,
suspended boolean,
unsubscribed boolean NOT NULL DEFAULT FALSE,
"timestamp" timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_activity_ts timestamp,
referral_code varchar(16) UNIQUE,
referred_by integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
unconfirmed_change_email varchar(256),
change_email_confirm_token varchar(256),
otp_secret text,
otp_enabled boolean DEFAULT FALSE,
otp_recovery_codes text,
stripe_customer_id varchar(40),
public_uuid varchar(36),
public_id integer,
clean_email varchar(256),
audit_metadata jsonb,
signup_ip varchar(45),
signup_ip_forwarded varchar(45),
signup_user_agent varchar(512),
signup_origin varchar(255),
signup_server varchar(255),
metadata jsonb DEFAULT '{}'::jsonb,
reputation smallint DEFAULT 100
);
CREATE INDEX IF NOT EXISTS idx_user_email ON "user" (email);
CREATE INDEX IF NOT EXISTS idx_user_pass_recovery_token ON "user" (pass_recovery_token);
CREATE INDEX IF NOT EXISTS idx_user_referrer ON "user" (referrer);
CREATE INDEX IF NOT EXISTS idx_user_email_confirm_token ON "user" (email_confirm_token);
CREATE INDEX IF NOT EXISTS idx_user_last_activity_ts ON "user" (last_activity_ts);
CREATE INDEX IF NOT EXISTS idx_user_referred_by ON "user" (referred_by);
CREATE INDEX IF NOT EXISTS idx_user_stripe_customer_id ON "user" (stripe_customer_id);
CREATE INDEX IF NOT EXISTS idx_user_clean_email ON "user" (clean_email);
CREATE INDEX IF NOT EXISTS idx_user_signup_ip ON "user" (signup_ip);
CREATE INDEX IF NOT EXISTS idx_user_signup_ip_forwarded ON "user" (signup_ip_forwarded);
CREATE INDEX IF NOT EXISTS idx_user_signup_user_agent ON "user" (signup_user_agent);
CREATE INDEX IF NOT EXISTS idx_user_signup_origin ON "user" (signup_origin);
CREATE INDEX IF NOT EXISTS idx_user_signup_server ON "user" (signup_server);
CREATE TABLE IF NOT EXISTS apps (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uid varchar(40) NOT NULL UNIQUE,
owner_user_id integer REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
icon text,
name varchar(100) NOT NULL UNIQUE,
title varchar(100) NOT NULL,
description text,
godmode boolean DEFAULT FALSE,
maximize_on_start boolean DEFAULT FALSE,
index_url text NOT NULL,
approved_for_listing boolean DEFAULT FALSE,
approved_for_opening_items boolean DEFAULT FALSE,
approved_for_incentive_program boolean DEFAULT FALSE,
"timestamp" timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_review timestamp,
tags varchar(255),
app_owner integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
background boolean DEFAULT FALSE,
metadata jsonb,
protected boolean DEFAULT FALSE,
is_private boolean DEFAULT FALSE
);
CREATE INDEX IF NOT EXISTS idx_apps_owner_user_id ON apps (owner_user_id);
CREATE INDEX IF NOT EXISTS idx_apps_app_owner ON apps (app_owner);
CREATE INDEX IF NOT EXISTS idx_apps_owner_timestamp ON apps (owner_user_id, "timestamp" DESC);
CREATE INDEX IF NOT EXISTS idx_apps_listing_timestamp ON apps (approved_for_listing, "timestamp" DESC);
CREATE TABLE IF NOT EXISTS "group" (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uid varchar(40) UNIQUE,
owner_user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
extra jsonb,
metadata jsonb,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_group_owner_user_id ON "group" (owner_user_id);
CREATE TABLE IF NOT EXISTS app_filetype_association (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
app_id integer NOT NULL REFERENCES apps (id) ON DELETE CASCADE ON UPDATE CASCADE,
type varchar(60) NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_app_filetype_association_app_id ON app_filetype_association (app_id);
CREATE INDEX IF NOT EXISTS idx_app_filetype_association_type ON app_filetype_association (type);
CREATE TABLE IF NOT EXISTS app_opens (
_id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
app_uid varchar(40) NOT NULL REFERENCES apps (uid) ON DELETE CASCADE ON UPDATE CASCADE,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
ts integer NOT NULL,
human_ts timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_app_opens_user_id ON app_opens (user_id);
CREATE INDEX IF NOT EXISTS idx_app_opens_app_uid ON app_opens (app_uid);
CREATE INDEX IF NOT EXISTS idx_app_opens_uid_ts ON app_opens (app_uid, ts);
CREATE INDEX IF NOT EXISTS idx_app_opens_app_user ON app_opens (app_uid, user_id);
CREATE TABLE IF NOT EXISTS fsentries (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uuid varchar(36) NOT NULL UNIQUE,
bucket varchar(50),
bucket_region varchar(30),
public_token varchar(36) UNIQUE,
file_request_token varchar(36) UNIQUE,
is_shortcut boolean DEFAULT FALSE,
shortcut_to integer REFERENCES fsentries (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
parent_id integer REFERENCES fsentries (id) ON DELETE CASCADE ON UPDATE CASCADE,
associated_app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
is_dir boolean DEFAULT FALSE,
layout varchar(30),
sort_by varchar(20),
sort_order varchar(10),
is_public boolean,
thumbnail text,
immutable boolean NOT NULL DEFAULT FALSE,
name varchar(767) NOT NULL,
metadata text,
modified integer NOT NULL,
created integer,
accessed integer,
size bigint,
symlink_path varchar(260),
is_symlink boolean DEFAULT FALSE,
parent_uid varchar(36),
path varchar(4096),
CONSTRAINT fsentries_parent_name_unique UNIQUE (parent_id, name)
);
CREATE INDEX IF NOT EXISTS idx_fsentries_name ON fsentries (name);
CREATE INDEX IF NOT EXISTS idx_fsentries_modified ON fsentries (modified);
CREATE INDEX IF NOT EXISTS idx_fsentries_parent_id ON fsentries (parent_id);
CREATE INDEX IF NOT EXISTS idx_fsentries_is_dir ON fsentries (is_dir);
CREATE INDEX IF NOT EXISTS idx_fsentries_user_id ON fsentries (user_id);
CREATE INDEX IF NOT EXISTS idx_fsentries_shortcut_to ON fsentries (shortcut_to);
CREATE INDEX IF NOT EXISTS idx_fsentries_associated_app_id ON fsentries (associated_app_id);
CREATE INDEX IF NOT EXISTS idx_fsentries_bucket ON fsentries (bucket);
CREATE INDEX IF NOT EXISTS idx_fsentries_bucket_region ON fsentries (bucket_region);
CREATE INDEX IF NOT EXISTS idx_fsentries_parent_uid ON fsentries (parent_uid);
CREATE INDEX IF NOT EXISTS idx_fsentries_path ON fsentries (path);
CREATE INDEX IF NOT EXISTS idx_fsentries_accessed ON fsentries (accessed);
CREATE INDEX IF NOT EXISTS idx_fsentries_user_parent_name ON fsentries (user_id, parent_uid, name);
CREATE INDEX IF NOT EXISTS idx_fsentries_parent_uid_name ON fsentries (parent_uid, name);
CREATE TABLE IF NOT EXISTS fsentry_versions (
id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
fsentry_id integer NOT NULL REFERENCES fsentries (id) ON DELETE CASCADE ON UPDATE CASCADE,
fsentry_uuid varchar(36) NOT NULL,
version_id varchar(60) NOT NULL,
user_id integer REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
message text,
ts_epoch integer
);
CREATE INDEX IF NOT EXISTS idx_fsentry_versions_fsentry_id ON fsentry_versions (fsentry_id);
CREATE INDEX IF NOT EXISTS idx_fsentry_versions_fsentry_uuid ON fsentry_versions (fsentry_uuid);
CREATE INDEX IF NOT EXISTS idx_fsentry_versions_user_id ON fsentry_versions (user_id);
CREATE TABLE IF NOT EXISTS subdomains (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uuid varchar(40) UNIQUE,
subdomain varchar(64) NOT NULL UNIQUE,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
root_dir_id integer REFERENCES fsentries (id) ON DELETE SET NULL ON UPDATE CASCADE,
associated_app_id integer REFERENCES apps (id) ON DELETE CASCADE ON UPDATE CASCADE,
ts timestamp DEFAULT CURRENT_TIMESTAMP,
app_owner integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
protected boolean DEFAULT FALSE,
domain varchar(265),
database_id varchar(40),
preamble_version varchar(64)
);
CREATE INDEX IF NOT EXISTS idx_subdomains_user_id ON subdomains (user_id);
CREATE INDEX IF NOT EXISTS idx_subdomains_root_dir_id ON subdomains (root_dir_id);
CREATE INDEX IF NOT EXISTS idx_subdomains_associated_app_id ON subdomains (associated_app_id);
CREATE INDEX IF NOT EXISTS idx_subdomains_app_owner ON subdomains (app_owner);
CREATE INDEX IF NOT EXISTS idx_subdomains_domain ON subdomains (domain);
CREATE INDEX IF NOT EXISTS idx_subdomains_root_user ON subdomains (root_dir_id, user_id);
CREATE INDEX IF NOT EXISTS idx_subdomains_app_user ON subdomains (associated_app_id, user_id);
CREATE TABLE IF NOT EXISTS kv (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
app varchar(40),
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
kkey_hash bigint NOT NULL,
kkey text NOT NULL,
value text,
migrated boolean DEFAULT FALSE,
"expireAt" timestamp,
CONSTRAINT kv_app_user_kkey_hash_unique UNIQUE (app, user_id, kkey_hash)
);
CREATE INDEX IF NOT EXISTS idx_kv_app ON kv (app);
CREATE INDEX IF NOT EXISTS idx_kv_user_id ON kv (user_id);
CREATE INDEX IF NOT EXISTS idx_kv_kkey_hash ON kv (kkey_hash);
CREATE TABLE IF NOT EXISTS sessions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
uuid varchar(40) NOT NULL,
meta jsonb,
created_at bigint DEFAULT 0,
last_activity bigint DEFAULT 0,
kind varchar(32) NOT NULL DEFAULT 'web'
CHECK (kind IN ('web', 'app', 'access_token', 'asset', 'worker')),
label varchar(255),
parent_session_id varchar(64),
last_ip varchar(64),
last_user_agent varchar(512),
revoked_at bigint,
expires_at bigint,
app_uid varchar(64),
legacy_token_uid varchar(64),
created_via varchar(32),
auth_id varchar(64),
access_token_uid varchar(64)
);
CREATE INDEX IF NOT EXISTS idx_sessions_user_id ON sessions (user_id);
CREATE INDEX IF NOT EXISTS idx_sessions_uuid ON sessions (uuid);
CREATE INDEX IF NOT EXISTS idx_sessions_user_revoked ON sessions (user_id, revoked_at);
CREATE INDEX IF NOT EXISTS idx_sessions_parent ON sessions (parent_session_id);
CREATE UNIQUE INDEX IF NOT EXISTS idx_sessions_user_app_active
ON sessions (user_id, app_uid)
WHERE kind = 'app' AND revoked_at IS NULL;
CREATE UNIQUE INDEX IF NOT EXISTS idx_sessions_legacy_token_active
ON sessions (legacy_token_uid)
WHERE legacy_token_uid IS NOT NULL AND revoked_at IS NULL;
CREATE INDEX IF NOT EXISTS idx_sessions_kind_user ON sessions (kind, user_id);
CREATE INDEX IF NOT EXISTS idx_sessions_access_token_uid
ON sessions (access_token_uid)
WHERE access_token_uid IS NOT NULL;
CREATE UNIQUE INDEX IF NOT EXISTS idx_sessions_user_worker_active
ON sessions (user_id, COALESCE(app_uid, ''), (meta #>> ARRAY['worker_name']))
WHERE kind = 'worker' AND revoked_at IS NULL;
CREATE TABLE IF NOT EXISTS user_to_app_permissions (
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
app_id integer NOT NULL REFERENCES apps (id) ON DELETE CASCADE ON UPDATE CASCADE,
permission varchar(255) NOT NULL,
extra jsonb,
dt timestamp DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (user_id, app_id, permission)
);
CREATE INDEX IF NOT EXISTS idx_utap_user_permission ON user_to_app_permissions (user_id, permission);
CREATE INDEX IF NOT EXISTS idx_utap_app_permission ON user_to_app_permissions (app_id, permission);
CREATE TABLE IF NOT EXISTS user_to_group_permissions (
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
group_id integer NOT NULL REFERENCES "group" (id) ON DELETE CASCADE ON UPDATE CASCADE,
permission varchar(255) NOT NULL,
extra jsonb,
PRIMARY KEY (user_id, group_id, permission)
);
CREATE INDEX IF NOT EXISTS idx_user_to_group_permissions_group_id ON user_to_group_permissions (group_id);
CREATE TABLE IF NOT EXISTS user_to_user_permissions (
issuer_user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
holder_user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
permission varchar(255) NOT NULL,
extra jsonb,
PRIMARY KEY (issuer_user_id, holder_user_id, permission)
);
CREATE INDEX IF NOT EXISTS idx_user_to_user_permissions_holder_user_id ON user_to_user_permissions (holder_user_id);
CREATE TABLE IF NOT EXISTS dev_to_app_permissions (
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
app_id integer NOT NULL REFERENCES apps (id) ON DELETE CASCADE ON UPDATE CASCADE,
permission varchar(255) NOT NULL,
extra jsonb,
PRIMARY KEY (user_id, app_id, permission)
);
CREATE INDEX IF NOT EXISTS idx_dev_to_app_permissions_app_id ON dev_to_app_permissions (app_id);
CREATE INDEX IF NOT EXISTS idx_dev_to_app_permissions_permission ON dev_to_app_permissions (permission);
CREATE TABLE IF NOT EXISTS access_token_permissions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
token_uid varchar(40) NOT NULL,
authorizer_user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
authorizer_app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
permission varchar(255) NOT NULL,
extra jsonb,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_access_token_permissions_token_uid ON access_token_permissions (token_uid);
CREATE TABLE IF NOT EXISTS old_app_names (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
app_uid varchar(40) NOT NULL REFERENCES apps (uid) ON DELETE CASCADE,
name varchar(255) NOT NULL,
"timestamp" timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT unique_old_app_names_app_uid_name UNIQUE (app_uid, name)
);
CREATE INDEX IF NOT EXISTS idx_old_app_names_app_uid ON old_app_names (app_uid);
CREATE INDEX IF NOT EXISTS idx_old_app_names_name ON old_app_names (name);
CREATE TABLE IF NOT EXISTS jct_user_group (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
group_id integer NOT NULL REFERENCES "group" (id) ON DELETE CASCADE ON UPDATE CASCADE,
extra jsonb,
metadata jsonb,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_jct_user_group_user_id ON jct_user_group (user_id);
CREATE INDEX IF NOT EXISTS idx_jct_user_group_group_id ON jct_user_group (group_id);
CREATE TABLE IF NOT EXISTS notification (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
uid varchar(40) UNIQUE,
value jsonb NOT NULL,
acknowledged bigint,
shown bigint,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_notification_user_id ON notification (user_id);
CREATE TABLE IF NOT EXISTS share (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uid varchar(40) UNIQUE,
issuer_user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
recipient_email varchar(255) NOT NULL,
data jsonb,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_share_issuer_user_id ON share (issuer_user_id);
CREATE INDEX IF NOT EXISTS idx_share_recipient_email ON share (recipient_email);
CREATE TABLE IF NOT EXISTS feedback (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
message text,
ts timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_feedback_user_id ON feedback (user_id);
CREATE TABLE IF NOT EXISTS thread (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uid varchar(40) NOT NULL UNIQUE,
parent_uid varchar(40) REFERENCES thread (uid) ON DELETE CASCADE ON UPDATE CASCADE,
owner_user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
"schema" text,
text text NOT NULL,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_thread_parent_uid ON thread (parent_uid);
CREATE INDEX IF NOT EXISTS idx_thread_owner_user_id ON thread (owner_user_id);
CREATE TABLE IF NOT EXISTS user_comments (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
uid varchar(40) NOT NULL UNIQUE,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
metadata jsonb,
text text NOT NULL,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_user_comments_user_id ON user_comments (user_id);
CREATE INDEX IF NOT EXISTS idx_user_comments_uid ON user_comments (uid);
CREATE TABLE IF NOT EXISTS user_fsentry_comments (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_comment_id integer NOT NULL REFERENCES user_comments (id) ON DELETE CASCADE ON UPDATE CASCADE,
fsentry_id integer NOT NULL REFERENCES fsentries (id) ON DELETE CASCADE ON UPDATE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_user_fsentry_comments_user_comment_id ON user_fsentry_comments (user_comment_id);
CREATE INDEX IF NOT EXISTS idx_user_fsentry_comments_fsentry_id ON user_fsentry_comments (fsentry_id);
CREATE TABLE IF NOT EXISTS user_fsentry_version_comments (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_comment_id integer NOT NULL REFERENCES user_comments (id) ON DELETE CASCADE ON UPDATE CASCADE,
fsentry_version_id integer NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_user_fsentry_version_comments_user_comment_id ON user_fsentry_version_comments (user_comment_id);
CREATE INDEX IF NOT EXISTS idx_user_fsentry_version_comments_version_id ON user_fsentry_version_comments (fsentry_version_id);
CREATE TABLE IF NOT EXISTS user_group_comments (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_comment_id integer NOT NULL REFERENCES user_comments (id) ON DELETE CASCADE ON UPDATE CASCADE,
group_id integer NOT NULL REFERENCES "group" (id) ON DELETE CASCADE ON UPDATE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_user_group_comments_user_comment_id ON user_group_comments (user_comment_id);
CREATE INDEX IF NOT EXISTS idx_user_group_comments_group_id ON user_group_comments (group_id);
CREATE TABLE IF NOT EXISTS user_user_comments (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_comment_id integer NOT NULL REFERENCES user_comments (id) ON DELETE CASCADE ON UPDATE CASCADE,
commented_user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_user_user_comments_user_comment_id ON user_user_comments (user_comment_id);
CREATE INDEX IF NOT EXISTS idx_user_user_comments_commented_user_id ON user_user_comments (commented_user_id);
CREATE TABLE IF NOT EXISTS user_oidc_providers (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
provider varchar(64) NOT NULL,
provider_sub varchar(255) NOT NULL,
refresh_token text,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_user_oidc_providers_user_id ON user_oidc_providers (user_id);
CREATE INDEX IF NOT EXISTS idx_user_oidc_providers_provider ON user_oidc_providers (provider);
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_oidc_providers_provider_sub_unique ON user_oidc_providers (provider, provider_sub);
CREATE TABLE IF NOT EXISTS app_update_audit (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
app_id_keep integer NOT NULL,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
old_name varchar(50),
new_name varchar(50),
reason varchar(255)
);
CREATE INDEX IF NOT EXISTS idx_app_update_audit_app_id ON app_update_audit (app_id);
CREATE TABLE IF NOT EXISTS user_update_audit (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer NOT NULL,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
old_email varchar(256),
new_email varchar(256),
old_username varchar(50),
new_username varchar(50),
reason varchar(255)
);
CREATE INDEX IF NOT EXISTS idx_user_update_audit_user_id ON user_update_audit (user_id);
CREATE TABLE IF NOT EXISTS storage_audit (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer NOT NULL,
is_subtract boolean NOT NULL DEFAULT FALSE,
amount bigint NOT NULL,
field_a varchar(16),
field_b varchar(16),
reason varchar(255),
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_storage_audit_user_id ON storage_audit (user_id);
CREATE TABLE IF NOT EXISTS audit_user_to_app_permissions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer NOT NULL,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
app_id_keep integer NOT NULL,
permission varchar(255) NOT NULL,
extra jsonb,
action varchar(16),
reason varchar(255),
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_app_permissions_user_id ON audit_user_to_app_permissions (user_id);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_app_permissions_app_id ON audit_user_to_app_permissions (app_id);
CREATE TABLE IF NOT EXISTS audit_dev_to_app_permissions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer NOT NULL,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
app_id_keep integer NOT NULL,
permission varchar(255) NOT NULL,
extra jsonb,
action varchar(16),
reason varchar(255),
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_audit_dev_to_app_permissions_user_id ON audit_dev_to_app_permissions (user_id);
CREATE INDEX IF NOT EXISTS idx_audit_dev_to_app_permissions_app_id ON audit_dev_to_app_permissions (app_id);
CREATE TABLE IF NOT EXISTS audit_user_to_group_permissions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer NOT NULL,
group_id integer REFERENCES "group" (id) ON DELETE SET NULL ON UPDATE CASCADE,
group_id_keep integer NOT NULL,
permission varchar(255) NOT NULL,
extra jsonb,
action varchar(255),
reason varchar(255),
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_group_permissions_user_id ON audit_user_to_group_permissions (user_id);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_group_permissions_group_id ON audit_user_to_group_permissions (group_id);
CREATE TABLE IF NOT EXISTS audit_user_to_user_permissions (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
issuer_user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
issuer_user_id_keep integer NOT NULL,
holder_user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
holder_user_id_keep integer NOT NULL,
permission varchar(255) NOT NULL,
extra jsonb,
action varchar(16),
reason varchar(255),
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_user_permissions_issuer_user_id ON audit_user_to_user_permissions (issuer_user_id);
CREATE INDEX IF NOT EXISTS idx_audit_user_to_user_permissions_holder_user_id ON audit_user_to_user_permissions (holder_user_id);
CREATE TABLE IF NOT EXISTS ai_usage (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
service_name varchar(64),
model_name varchar(128),
price_modifier varchar(40),
cost integer,
value_uint_1 integer,
value_uint_2 integer,
value_uint_3 integer,
value_uint_4 integer,
value_uint_5 integer,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_ai_usage_app_id ON ai_usage (app_id);
CREATE INDEX IF NOT EXISTS idx_ai_usage_service_name ON ai_usage (service_name);
CREATE INDEX IF NOT EXISTS idx_ai_usage_model_name ON ai_usage (model_name);
CREATE INDEX IF NOT EXISTS idx_ai_usage_price_modifier ON ai_usage (price_modifier);
CREATE INDEX IF NOT EXISTS idx_ai_usage_created_at ON ai_usage (created_at);
CREATE INDEX IF NOT EXISTS idx_ai_usage_user_timestamp ON ai_usage (user_id, created_at);
CREATE TABLE IF NOT EXISTS general_analytics (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
uid varchar(40) NOT NULL,
trace_id varchar(40),
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
user_id_keep integer,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
app_id_keep integer,
server_id varchar(40),
actor_type varchar(40),
tags jsonb,
fields jsonb
);
CREATE INDEX IF NOT EXISTS idx_general_analytics_user_id ON general_analytics (user_id);
CREATE INDEX IF NOT EXISTS idx_general_analytics_app_id ON general_analytics (app_id);
CREATE TABLE IF NOT EXISTS monthly_usage_counts (
year integer NOT NULL,
month integer NOT NULL,
service_type varchar(40) NOT NULL,
service_name varchar(40) NOT NULL,
actor_key varchar(255) NOT NULL,
pricing_category jsonb NOT NULL,
pricing_category_hash bytea NOT NULL,
"count" integer DEFAULT 0,
value_uint_1 integer,
value_uint_2 integer,
value_uint_3 integer,
PRIMARY KEY (year, month, service_type, service_name, actor_key, pricing_category_hash)
);
CREATE TABLE IF NOT EXISTS service_usage_monthly (
"key" varchar(255) NOT NULL,
year integer NOT NULL,
month integer NOT NULL,
user_id integer REFERENCES "user" (id) ON DELETE SET NULL ON UPDATE CASCADE,
app_id integer REFERENCES apps (id) ON DELETE SET NULL ON UPDATE CASCADE,
"count" integer NOT NULL,
extra jsonb,
PRIMARY KEY ("key", year, month)
);
CREATE INDEX IF NOT EXISTS idx_service_usage_monthly_user_id ON service_usage_monthly (user_id);
CREATE INDEX IF NOT EXISTS idx_service_usage_monthly_app_id ON service_usage_monthly (app_id);
CREATE TABLE IF NOT EXISTS per_user_credit (
id integer GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
user_id integer NOT NULL UNIQUE REFERENCES "user" (id) ON DELETE CASCADE ON UPDATE CASCADE,
amount bigint NOT NULL,
last_updated_at bigint NOT NULL
);
INSERT INTO "user" (id, uuid, username, metadata)
VALUES (1, '5d4adce0-a381-4982-9c02-6e2540026238', 'system', '{}'::jsonb)
ON CONFLICT (uuid) DO UPDATE SET username = EXCLUDED.username;
SELECT setval(pg_get_serial_sequence('"user"', 'id'), COALESCE((SELECT MAX(id) FROM "user"), 1), true);
INSERT INTO "group" (uid, owner_user_id, extra, metadata)
VALUES
('26bfb1fb-421f-45bc-9aa4-d81ea569e7a5', 1, '{"critical": true, "type": "default", "name": "system"}'::jsonb, '{"title": "System", "color": "#000000"}'::jsonb),
('ca342a5e-b13d-4dee-9048-58b11a57cc55', 1, '{"critical": true, "type": "default", "name": "admin"}'::jsonb, '{"title": "Admin", "color": "#a83232"}'::jsonb),
('78b1b1dd-c959-44d2-b02c-8735671f9997', 1, '{"critical": true, "type": "default", "name": "user"}'::jsonb, '{"title": "User", "color": "#3254a8"}'::jsonb),
('b7220104-7905-4985-b996-649fdcdb3c8f', 1, '{"critical": true, "type": "default", "name": "temp"}'::jsonb, '{"title": "Temp", "color": "#888888"}'::jsonb),
('3c2dfff7-d22a-41aa-a193-59a61dac4b64', 1, '{"type": "default", "name": "moderator"}'::jsonb, '{"title": "Moderator", "color": "#a432a8"}'::jsonb),
('5e8f251d-3382-4b0d-932c-7bb82f48652f', 1, '{"type": "default", "name": "developer"}'::jsonb, '{"title": "Developer", "color": "#32a852"}'::jsonb)
ON CONFLICT (uid) DO UPDATE SET
owner_user_id = EXCLUDED.owner_user_id,
extra = EXCLUDED.extra,
metadata = EXCLUDED.metadata;
INSERT INTO apps (
uid, owner_user_id, icon, name, title, description, index_url,
godmode, maximize_on_start, background,
approved_for_listing, approved_for_opening_items,
approved_for_incentive_program, tags, "timestamp"
)
VALUES
('app-3920851d-bda8-479b-9407-8517293c7d44', 1, NULL, 'pdf', 'PDF', '', 'https://pdf.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-5584fbf7-ed69-41fc-99cd-85da21b1ef51', 1, NULL, 'camera', 'Camera', 'Camera in the browser.', 'https://online-camera.com', FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-11edfba2-1ed3-4e22-8573-47e88fb87d70', 1, NULL, 'player', 'Player', 'A free video player app in the browser.', 'https://simple-player.puter.com', FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-7bdca1a4-6373-4c98-ad97-03ff2d608ca1', 1, NULL, 'recorder', 'Recorder', 'Online voice recorder in the browser with cloud storage.', 'https://voice-recorder.com', FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-d7e9471f-e441-4d72-a5ab-75e96573b76b', 1, NULL, 'music-player', 'Music Player', 'A free music player app in the browser.', 'https://player.puter.com', FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, NULL, '2026-05-10 00:00:00'),
('app-e3ac5486-da8c-42ad-8377-8728086e0980', 1, NULL, 'git', 'Git', 'Puter Git client', 'https://builtins.namespaces.puter.com/git', FALSE, FALSE, TRUE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-0b37f054-07d4-4627-8765-11bd23e889d4', 1, NULL, 'dev-center', 'Dev Center', 'This is the app that makes apps', 'https://builtins.namespaces.puter.com/dev-center', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-fbbdb72b-ad08-4cb4-86a1-de0f27cf2e1e', 1, NULL, 'puter-linux', 'Puter Linux', 'Linux emulator for Puter', 'https://builtins.namespaces.puter.com/emulator', TRUE, FALSE, FALSE, TRUE, TRUE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-838dfbc4-bf8b-48c2-b47b-c4adc77fab58', 1, NULL, 'editor', 'Editor', 'Text editor', 'https://online-notepad.com', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-58282b08-990a-4906-95f7-fa37ff92452b', 1, NULL, 'draw', 'Draw', 'Image editor', 'https://draw.puter.com', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, 'graphics', '2020-01-01 00:00:00'),
('app-0bef044f-918f-4cbf-a0c0-b4a17ee81085', 1, NULL, 'about', 'About', 'About Puter', 'https://about.puter.com', TRUE, FALSE, FALSE, FALSE, TRUE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-a2ae72a4-1ba3-4a29-b5c0-6de1be5cf178', 1, NULL, 'app-center', 'App Center', 'Discover apps for Puter', 'https://app-center.puter.com', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, NULL, '2020-01-01 00:00:00'),
('app-93005ce0-80d1-50d9-9b1e-9c453c375d56', 1, NULL, 'markus', 'Markus', 'Markdown editor', 'https://markus.puter.com', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-6f79ef7b-52b7-4b31-91c6-fc07b62e9396', 1, NULL, 'code', 'Code', 'Code editor', 'https://code.puter.com', TRUE, TRUE, FALSE, TRUE, TRUE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-862fc09e-c7b8-4c30-b5b4-47b3cc5f5232', 1, NULL, 'memos', 'Memos', 'Notes and memos', 'https://memos.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-aeb8c03e-1144-4c57-bff5-3a1a7c17f9f0', 1, NULL, 'word-processor', 'Word Processor', 'Write documents', 'https://word-processor.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-c10a1999-22f5-4644-9960-9aaac0d4934e', 1, NULL, 'spreadsheet', 'Spreadsheet', 'Work with spreadsheets', 'https://spreadsheet.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-e9c1d58d-3d8d-4f0a-a85c-e383ef63bc29', 1, NULL, 'presentation', 'Presentation', 'Create presentations', 'https://presentation.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00'),
('app-fb7d9e42-8207-4fa0-b680-f0239327097f', 1, NULL, 'pdf-editor', 'PDF Editor', 'Edit PDF documents', 'https://pdf-editor.puter.com', FALSE, TRUE, FALSE, TRUE, FALSE, FALSE, 'productivity', '2020-01-01 00:00:00')
ON CONFLICT (uid) DO UPDATE SET
owner_user_id = EXCLUDED.owner_user_id,
name = EXCLUDED.name,
title = EXCLUDED.title,
description = EXCLUDED.description,
index_url = EXCLUDED.index_url,
godmode = EXCLUDED.godmode,
maximize_on_start = EXCLUDED.maximize_on_start,
background = EXCLUDED.background,
approved_for_listing = EXCLUDED.approved_for_listing,
approved_for_opening_items = EXCLUDED.approved_for_opening_items,
approved_for_incentive_program = EXCLUDED.approved_for_incentive_program,
tags = EXCLUDED.tags;
INSERT INTO user_to_group_permissions (user_id, group_id, permission, extra)
SELECT u.id, g.id, 'driver', '{}'::jsonb
FROM "user" u, "group" g
WHERE u.username = 'system'
AND g.uid = 'ca342a5e-b13d-4dee-9048-58b11a57cc55'
ON CONFLICT (user_id, group_id, permission) DO UPDATE SET extra = EXCLUDED.extra;
@@ -0,0 +1,26 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- SMS phone verification columns. Mirrors SQLite migration 0058. `phone` is the
-- E.164 number collected during verification (indexed like `email`);
-- `requires_phone_verification` gates account use for low-reputation signups
-- (not indexed, mirroring `requires_email_confirmation`).
-- Idempotent via IF NOT EXISTS.
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS phone varchar(20);
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS requires_phone_verification boolean NOT NULL DEFAULT FALSE;
CREATE INDEX IF NOT EXISTS idx_user_phone ON "user" (phone);
@@ -0,0 +1,24 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Credit-card verification column. Mirrors SQLite migration 0059.
-- `requires_card_verification` gates account use for low-reputation signups;
-- the card itself never touches our DB, so this is the only column (not
-- indexed, mirroring `requires_phone_verification`).
-- Idempotent via IF NOT EXISTS.
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS requires_card_verification boolean NOT NULL DEFAULT FALSE;
@@ -0,0 +1,26 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Card fingerprint column. Mirrors SQLite migration 0060. `card_fingerprint`
-- is the Stripe card fingerprint (stable per card number) recorded when a user
-- clears card verification — the card sibling of `phone`, indexed like it so
-- admin tooling can find the accounts that verified with a given card. The card
-- itself never touches our DB, only Stripe's fingerprint for it.
-- Idempotent via IF NOT EXISTS.
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS card_fingerprint varchar(128);
CREATE INDEX IF NOT EXISTS idx_user_card_fingerprint ON "user" (card_fingerprint);
@@ -0,0 +1,25 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Suspended-at column. Mirrors SQLite migration 0061. `suspended_at` is when
-- the account was suspended, as unix seconds (NULL while not suspended) — the
-- timestamp sibling of the boolean `suspended` flag, indexed so the
-- signup-abuse harness can count an IP's recently-suspended accounts.
-- Idempotent via IF NOT EXISTS.
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS suspended_at bigint;
CREATE INDEX IF NOT EXISTS idx_user_suspended_at ON "user" (suspended_at);
@@ -0,0 +1,36 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Admin-managed blocklist of app origins. Mirrors SQLite migration 0062 /
-- MySQL mysql_mig_17 (which Postgres was originally missed for). An app whose
-- `index_url` host (or a request origin) matches an entry is denied access to
-- Puter resources. `include_subdomains = 1` also blocks every subdomain of
-- `domain`. Enforced in AuthService via AppOriginBlocklistService.
--
-- Idempotent via IF NOT EXISTS.
CREATE TABLE IF NOT EXISTS blocked_app_origins (
id SERIAL PRIMARY KEY,
domain VARCHAR(255) NOT NULL,
include_subdomains SMALLINT NOT NULL DEFAULT 0,
reason TEXT DEFAULT NULL,
created_by VARCHAR(255) DEFAULT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_blocked_app_origins_domain
ON blocked_app_origins (domain);
@@ -0,0 +1,24 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Suspension reason column. Mirrors SQLite migration 0063. Why an account was
-- suspended (NULL while not suspended) — companion to the boolean `suspended`
-- flag and `suspended_at` timestamp. Constrained at the application layer to a
-- fixed set of reasons (see extensions/admin suspension_reasons.js).
-- Idempotent via IF NOT EXISTS.
ALTER TABLE "user" ADD COLUMN IF NOT EXISTS suspended_reason text;
@@ -0,0 +1,40 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Append-only log of admin moderation actions. Mirrors SQLite migration 0064 /
-- MySQL mysql_mig_19. Used to measure the abuse system's false-positive rate:
-- `unsuspend` (admin unblock) and `admin_create_user` are the false-positive
-- signals; `suspend` gives the denominator. Preserves history the `user`
-- suspension columns lose on unsuspend. `created_at` is unix seconds.
--
-- Idempotent via IF NOT EXISTS.
CREATE TABLE IF NOT EXISTS abuse_moderation_events (
id BIGSERIAL PRIMARY KEY,
action VARCHAR(32) NOT NULL,
target_user_id BIGINT DEFAULT NULL,
target_username VARCHAR(255) DEFAULT NULL,
admin_username VARCHAR(255) DEFAULT NULL,
reason TEXT DEFAULT NULL,
source VARCHAR(64) DEFAULT NULL,
created_at BIGINT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_abuse_moderation_events_created_at
ON abuse_moderation_events (created_at);
CREATE INDEX IF NOT EXISTS idx_abuse_moderation_events_action
ON abuse_moderation_events (action);
@@ -0,0 +1,18 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
ALTER TABLE `subdomains` ADD COLUMN `preamble_version` varchar(64) DEFAULT NULL;
@@ -0,0 +1,34 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Extend `sessions` so a single row can represent any token kind
-- (web/app/access_token/asset), carry display metadata for the
-- manage-sessions UI, and be soft-revoked.
ALTER TABLE `sessions` ADD COLUMN `kind` TEXT NOT NULL DEFAULT 'web'
CHECK (`kind` IN ('web', 'app', 'access_token', 'asset'));
ALTER TABLE `sessions` ADD COLUMN `label` TEXT;
ALTER TABLE `sessions` ADD COLUMN `parent_session_id` TEXT;
ALTER TABLE `sessions` ADD COLUMN `last_ip` TEXT;
ALTER TABLE `sessions` ADD COLUMN `last_user_agent` TEXT;
ALTER TABLE `sessions` ADD COLUMN `revoked_at` INTEGER;
ALTER TABLE `sessions` ADD COLUMN `expires_at` INTEGER;
CREATE INDEX IF NOT EXISTS `idx_sessions_user_revoked`
ON `sessions` (`user_id`, `revoked_at`);
CREATE INDEX IF NOT EXISTS `idx_sessions_parent`
ON `sessions` (`parent_session_id`);
@@ -0,0 +1,45 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Composite-key lookups + audit columns.
-- - `app_uid` : binds `kind='app'` rows to their app authorization
-- target. (user_id, app_uid) is the idempotency key.
-- - `legacy_token_uid` : keys lazy-backfilled rows to the v1 token_uid that
-- originally minted them.
-- - `created_via` : audit sentinel (e.g. 'legacy_backfill').
-- - `auth_id` : stable per-user identity that survives re-login;
-- lets manage-sessions group by identity.
ALTER TABLE `sessions` ADD COLUMN `app_uid` TEXT;
ALTER TABLE `sessions` ADD COLUMN `legacy_token_uid` TEXT;
ALTER TABLE `sessions` ADD COLUMN `created_via` TEXT;
ALTER TABLE `sessions` ADD COLUMN `auth_id` TEXT;
-- Partial unique indexes keep "at most one active row per key" without
-- breaking the soft-revoke pattern (revoked rows stay for audit).
CREATE UNIQUE INDEX IF NOT EXISTS `idx_sessions_user_app_active`
ON `sessions` (`user_id`, `app_uid`)
WHERE `kind` = 'app' AND `revoked_at` IS NULL;
CREATE UNIQUE INDEX IF NOT EXISTS `idx_sessions_legacy_token_active`
ON `sessions` (`legacy_token_uid`)
WHERE `legacy_token_uid` IS NOT NULL AND `revoked_at` IS NULL;
-- Supports manage-sessions list queries grouped by kind.
CREATE INDEX IF NOT EXISTS `idx_sessions_kind_user`
ON `sessions` (`kind`, `user_id`);
@@ -0,0 +1,29 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Let `kind='access_token'` rows be reverse-looked-up
-- from the `token_uid` claim that lives only in `access_token_permissions`.
-- Required so `POST /auth/revoke-access-token` with a raw token_uid input
-- (no JWT) can find and soft-revoke the session row, matching the JWT
-- input path's coverage. Without it, raw-uuid revoke would only drop the
-- permissions row, leaving the session-row kill switch un-flipped.
ALTER TABLE `sessions` ADD COLUMN `access_token_uid` TEXT;
CREATE INDEX IF NOT EXISTS `idx_sessions_access_token_uid`
ON `sessions` (`access_token_uid`)
WHERE `access_token_uid` IS NOT NULL;
@@ -0,0 +1,30 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Worker session uniqueness. One active kind='worker' row per
-- (user_id, app_uid, worker_name).
--
-- SQLite UNIQUE indexes treat NULLs as distinct (per the SQL standard),
-- so user-scoped workers (where `app_uid` is NULL) would otherwise be
-- allowed to duplicate. IFNULL collapses NULL `app_uid` to empty
-- string in the index expression so two user-scoped workers with the
-- same worker_name correctly conflict. MySQL handles the same case
-- via the IFNULL in mig_11's generated column.
CREATE UNIQUE INDEX IF NOT EXISTS `idx_sessions_user_worker_active`
ON `sessions` (`user_id`, IFNULL(`app_uid`, ''), json_extract(`meta`, '$.worker_name'))
WHERE `kind` = 'worker' AND `revoked_at` IS NULL;
@@ -0,0 +1,31 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Enforce case-insensitive uniqueness on user.username.
--
-- SQLite defaults varchar columns to BINARY collation, so without this
-- index `Admin` and `admin` can coexist as separate rows even though the
-- reserved-name check and adminOnly gate both treat usernames as
-- case-insensitive. Prod MySQL already uses ascii_general_ci on this
-- column; this brings self-hosted SQLite to the same invariant.
--
-- If this CREATE fails because the DB already contains case-collision
-- duplicates, resolve them manually before re-running the migration —
-- there is no safe automatic merge of two user accounts.
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_username_nocase
ON user(username COLLATE NOCASE)
WHERE username IS NOT NULL;
@@ -0,0 +1,90 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Add 'worker' to the `kind` CHECK constraint on `sessions`. Mirrors the
-- MySQL ENUM extension in mysql_mig_12.
--
-- SQLite cannot ALTER a CHECK constraint in place, so we follow the
-- standard 12-step rebuild: create `sessions_new` with the corrected
-- constraint, copy rows, drop the old table, rename, then recreate every
-- index that lived on the original (indexes are auto-dropped with their
-- table).
CREATE TABLE `sessions_new` (
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
"user_id" INTEGER NOT NULL,
"uuid" TEXT NOT NULL,
"meta" JSON DEFAULT NULL,
"created_at" INTEGER DEFAULT 0,
"last_activity" INTEGER DEFAULT 0,
"kind" TEXT NOT NULL DEFAULT 'web'
CHECK (`kind` IN ('web', 'app', 'access_token', 'asset', 'worker')),
"label" TEXT,
"parent_session_id" TEXT,
"last_ip" TEXT,
"last_user_agent" TEXT,
"revoked_at" INTEGER,
"expires_at" INTEGER,
"app_uid" TEXT,
"legacy_token_uid" TEXT,
"created_via" TEXT,
"auth_id" TEXT,
"access_token_uid" TEXT,
FOREIGN KEY("user_id") REFERENCES "user" ("id") ON DELETE CASCADE ON UPDATE CASCADE
);
INSERT INTO `sessions_new` (
`id`, `user_id`, `uuid`, `meta`, `created_at`, `last_activity`,
`kind`, `label`, `parent_session_id`, `last_ip`, `last_user_agent`,
`revoked_at`, `expires_at`, `app_uid`, `legacy_token_uid`,
`created_via`, `auth_id`, `access_token_uid`
)
SELECT
`id`, `user_id`, `uuid`, `meta`, `created_at`, `last_activity`,
`kind`, `label`, `parent_session_id`, `last_ip`, `last_user_agent`,
`revoked_at`, `expires_at`, `app_uid`, `legacy_token_uid`,
`created_via`, `auth_id`, `access_token_uid`
FROM `sessions`;
DROP TABLE `sessions`;
ALTER TABLE `sessions_new` RENAME TO `sessions`;
CREATE INDEX IF NOT EXISTS `idx_sessions_user_revoked`
ON `sessions` (`user_id`, `revoked_at`);
CREATE INDEX IF NOT EXISTS `idx_sessions_parent`
ON `sessions` (`parent_session_id`);
CREATE UNIQUE INDEX IF NOT EXISTS `idx_sessions_user_app_active`
ON `sessions` (`user_id`, `app_uid`)
WHERE `kind` = 'app' AND `revoked_at` IS NULL;
CREATE UNIQUE INDEX IF NOT EXISTS `idx_sessions_legacy_token_active`
ON `sessions` (`legacy_token_uid`)
WHERE `legacy_token_uid` IS NOT NULL AND `revoked_at` IS NULL;
CREATE INDEX IF NOT EXISTS `idx_sessions_kind_user`
ON `sessions` (`kind`, `user_id`);
CREATE INDEX IF NOT EXISTS `idx_sessions_access_token_uid`
ON `sessions` (`access_token_uid`)
WHERE `access_token_uid` IS NOT NULL;
CREATE UNIQUE INDEX IF NOT EXISTS `idx_sessions_user_worker_active`
ON `sessions` (`user_id`, IFNULL(`app_uid`, ''), json_extract(`meta`, '$.worker_name'))
WHERE `kind` = 'worker' AND `revoked_at` IS NULL;
@@ -0,0 +1,21 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Abuse v2 reputation score (0–100+) recorded at signup. Mirrors the
-- `reputation smallint DEFAULT 100` column already present in the mysql and
-- postgres baseline migrations.
ALTER TABLE `user` ADD COLUMN `reputation` SMALLINT DEFAULT 100;
@@ -0,0 +1,25 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- SMS phone verification. `phone` holds the E.164 number collected during
-- verification (indexed like `email`). `requires_phone_verification` gates
-- account use until verified — the abuse v2 harness sets it for low-reputation
-- signups instead of blocking them. (Not indexed, mirroring
-- `requires_email_confirmation`.)
ALTER TABLE `user` ADD COLUMN `phone` varchar(20) DEFAULT NULL;
ALTER TABLE `user` ADD COLUMN `requires_phone_verification` tinyint(1) NOT NULL DEFAULT 0;
CREATE INDEX IF NOT EXISTS idx_user_phone ON `user` (`phone`);
@@ -0,0 +1,23 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Credit-card verification ($0 authorization handled by a payments
-- extension). `requires_card_verification` gates account use until verified —
-- the abuse v2 harness sets it for low-reputation signups instead of blocking
-- them. The card itself never touches our DB, so this is the only column.
-- (Not indexed, mirroring `requires_phone_verification`.)
ALTER TABLE `user` ADD COLUMN `requires_card_verification` tinyint(1) NOT NULL DEFAULT 0;
@@ -0,0 +1,25 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Card fingerprint column. `card_fingerprint` is the Stripe card fingerprint
-- (stable per card number) recorded when a user clears card verification — the
-- card sibling of `phone`. Indexed like `phone` so admin tooling can find the
-- accounts that verified with a given card. The card itself never touches our
-- DB, only Stripe's fingerprint for it. Server-only (not on the whoami
-- allowlist), so it never reaches clients.
ALTER TABLE `user` ADD COLUMN `card_fingerprint` varchar(128) DEFAULT NULL;
CREATE INDEX IF NOT EXISTS idx_user_card_fingerprint ON `user` (`card_fingerprint`);
@@ -0,0 +1,23 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- When the account was suspended, as unix seconds; NULL while not suspended.
-- The timestamp sibling of the existing boolean `suspended` flag, stamped by
-- every suspension site. Indexed so the signup-abuse harness can count an IP's
-- recently-suspended accounts (see extensions/abuse/v2 suspendedAccountsByIp).
ALTER TABLE `user` ADD COLUMN `suspended_at` INTEGER DEFAULT NULL;
CREATE INDEX IF NOT EXISTS idx_user_suspended_at ON `user` (`suspended_at`);
@@ -0,0 +1,34 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Admin-managed blocklist of app origins. An app whose `index_url` host (or
-- a request origin) matches an entry is denied access to Puter resources:
-- it cannot obtain an app token and already-issued app tokens are rejected
-- on each request. `include_subdomains = 1` also blocks every subdomain of
-- `domain`. Enforced in AuthService via AppOriginBlocklistService.
CREATE TABLE IF NOT EXISTS `blocked_app_origins` (
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
"domain" TEXT NOT NULL,
"include_subdomains" INTEGER NOT NULL DEFAULT 0,
"reason" TEXT DEFAULT NULL,
"created_by" TEXT DEFAULT NULL,
"created_at" TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE UNIQUE INDEX IF NOT EXISTS `idx_blocked_app_origins_domain`
ON `blocked_app_origins` (`domain`);
@@ -0,0 +1,23 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Why an account was suspended, NULL while not suspended. Companion to the
-- boolean `suspended` flag and `suspended_at` timestamp. Constrained at the
-- application layer to a fixed set of reasons (see extensions/admin
-- suspension_reasons.js) — kept as free TEXT here so adding a reason later
-- needs no migration.
ALTER TABLE `user` ADD COLUMN `suspended_reason` TEXT DEFAULT NULL;
@@ -0,0 +1,43 @@
-- Copyright (C) 2024-present Puter Technologies Inc.
--
-- This file is part of Puter.
--
-- Puter is free software: you can redistribute it and/or modify
-- it under the terms of the GNU Affero General Public License as published
-- by the Free Software Foundation, either version 3 of the License, or
-- (at your option) any later version.
--
-- This program is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU Affero General Public License for more details.
--
-- You should have received a copy of the GNU Affero General Public License
-- along with this program. If not, see <https://www.gnu.org/licenses/>.
-- Append-only log of admin moderation actions, used to measure the abuse
-- system's false-positive rate: `unsuspend` (admin unblock) and
-- `admin_create_user` (an account minted by an admin, typically to recover a
-- legitimately-blocked signup) are the false-positive signals; `suspend` gives
-- the denominator. Unlike the `suspended`/`suspended_at`/`suspended_reason`
-- columns on `user` — which are cleared on unsuspend and so keep no history —
-- this table preserves every event. `created_at` is unix seconds (matching
-- `user.suspended_at`), stamped by the recorder in
-- extensions/admin/moderation_events.js. Written best-effort at the admin
-- suspend/unsuspend/create-user sites; surfaced on the /admin/abuse dashboard.
CREATE TABLE IF NOT EXISTS `abuse_moderation_events` (
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
"action" TEXT NOT NULL, -- 'suspend' | 'unsuspend' | 'admin_create_user'
"target_user_id" INTEGER DEFAULT NULL,
"target_username" TEXT DEFAULT NULL,
"admin_username" TEXT DEFAULT NULL, -- the acting admin (req.actor.user.username)
"reason" TEXT DEFAULT NULL,
"source" TEXT DEFAULT NULL, -- 'user_page' | 'email_hostname_blacklist' | 'create_user'
"created_at" INTEGER NOT NULL -- unix seconds
);
CREATE INDEX IF NOT EXISTS idx_abuse_moderation_events_created_at
ON `abuse_moderation_events` (`created_at`);
CREATE INDEX IF NOT EXISTS idx_abuse_moderation_events_action
ON `abuse_moderation_events` (`action`);
@@ -0,0 +1,85 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import { preparePostgresSql } from './preparePostgresSql.js';
describe('preparePostgresSql', () => {
it('converts placeholders and backtick identifiers outside SQL literals', () => {
const prepared = preparePostgresSql(
'SELECT `user`.`id` FROM `user` WHERE `email` = ? AND `username` = ?',
);
expect(prepared).toEqual({
text: 'SELECT "user"."id" FROM "user" WHERE "email" = $1 AND "username" = $2',
parameterCount: 2,
});
});
it('leaves question marks inside strings and comments untouched', () => {
const prepared = preparePostgresSql(`
SELECT '?' AS literal, ?
-- ? in a comment
FROM \`apps\`
WHERE \`description\` = 'is this ok?'
/* and ? in a block comment */
AND \`name\` = ?
`);
expect(prepared.text).toContain("SELECT '?' AS literal, $1");
expect(prepared.text).toContain('-- ? in a comment');
expect(prepared.text).toContain('"description" = \'is this ok?\'');
expect(prepared.text).toContain('/* and ? in a block comment */');
expect(prepared.text).toContain('"name" = $2');
expect(prepared.parameterCount).toBe(2);
});
it('escapes double quotes inside converted identifiers', () => {
const prepared = preparePostgresSql(
'SELECT `odd"name` FROM `odd``table` WHERE `id` = ?',
);
expect(prepared).toEqual({
text: 'SELECT "odd""name" FROM "odd`table" WHERE "id" = $1',
parameterCount: 1,
});
});
it('does not scan placeholders inside dollar-quoted strings', () => {
const prepared = preparePostgresSql(
"SELECT $$?$$, $tag$`not_ident` ?$tag$, `id` FROM `user` WHERE `id` = ?",
);
expect(prepared).toEqual({
text: 'SELECT $$?$$, $tag$`not_ident` ?$tag$, "id" FROM "user" WHERE "id" = $1',
parameterCount: 1,
});
});
it('does not treat Postgres JSON operators as comments', () => {
const prepared = preparePostgresSql(
"SELECT * FROM `sessions` WHERE `user_id` = ? AND `meta` #>> ARRAY['worker_name'] = ? AND `expires_at` > ?",
);
expect(prepared).toEqual({
text: 'SELECT * FROM "sessions" WHERE "user_id" = $1 AND "meta" #>> ARRAY[\'worker_name\'] = $2 AND "expires_at" > $3',
parameterCount: 3,
});
});
});
@@ -0,0 +1,169 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
export interface PreparedPostgresSql {
text: string;
parameterCount: number;
}
type ScannerState =
| { kind: 'normal' }
| { kind: 'singleQuote' }
| { kind: 'doubleQuote' }
| { kind: 'backtickIdentifier' }
| { kind: 'lineComment' }
| { kind: 'blockComment' }
| { kind: 'dollarQuote'; tag: string };
const isLineCommentStart = (sql: string, index: number): boolean => {
if (sql[index] !== '-' || sql[index + 1] !== '-') return false;
const after = sql[index + 2];
return after === undefined || /\s/u.test(after);
};
const readDollarQuoteTag = (sql: string, index: number): string | null => {
const match = /^\$[A-Za-z_][A-Za-z0-9_]*\$|^\$\$/u.exec(sql.slice(index));
return match?.[0] ?? null;
};
export const preparePostgresSql = (sql: string): PreparedPostgresSql => {
let state: ScannerState = { kind: 'normal' };
let parameterCount = 0;
let out = '';
for (let i = 0; i < sql.length; i++) {
const char = sql[i]!;
const next = sql[i + 1];
switch (state.kind) {
case 'normal': {
const dollarTag =
char === '$' ? readDollarQuoteTag(sql, i) : null;
if (dollarTag) {
out += dollarTag;
i += dollarTag.length - 1;
state = { kind: 'dollarQuote', tag: dollarTag };
break;
}
if (char === "'") {
out += char;
state = { kind: 'singleQuote' };
break;
}
if (char === '"') {
out += char;
state = { kind: 'doubleQuote' };
break;
}
if (char === '`') {
out += '"';
state = { kind: 'backtickIdentifier' };
break;
}
if (isLineCommentStart(sql, i)) {
out += char;
if (char === '-') {
out += next ?? '';
i += 1;
}
state = { kind: 'lineComment' };
break;
}
if (char === '/' && next === '*') {
out += '/*';
i += 1;
state = { kind: 'blockComment' };
break;
}
if (char === '?') {
parameterCount += 1;
out += `$${parameterCount}`;
break;
}
out += char;
break;
}
case 'singleQuote':
out += char;
if (char === "'" && next === "'") {
out += next;
i += 1;
} else if (char === '\\' && next !== undefined) {
out += next;
i += 1;
} else if (char === "'") {
state = { kind: 'normal' };
}
break;
case 'doubleQuote':
out += char;
if (char === '"' && next === '"') {
out += next;
i += 1;
} else if (char === '"') {
state = { kind: 'normal' };
}
break;
case 'backtickIdentifier':
if (char === '`' && next === '`') {
out += '`';
i += 1;
} else if (char === '`') {
out += '"';
state = { kind: 'normal' };
} else if (char === '"') {
out += '""';
} else {
out += char;
}
break;
case 'lineComment':
out += char;
if (char === '\n') {
state = { kind: 'normal' };
}
break;
case 'blockComment':
out += char;
if (char === '*' && next === '/') {
out += '/';
i += 1;
state = { kind: 'normal' };
}
break;
case 'dollarQuote':
if (sql.startsWith(state.tag, i)) {
out += state.tag;
i += state.tag.length - 1;
state = { kind: 'normal' };
} else {
out += char;
}
break;
}
}
return { text: out, parameterCount };
};
@@ -0,0 +1,120 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
type ScannerState =
| { kind: 'normal' }
| { kind: 'singleQuote' }
| { kind: 'doubleQuote' }
| { kind: 'lineComment' }
| { kind: 'blockComment' }
| { kind: 'dollarQuote'; tag: string };
const readDollarQuoteTag = (sql: string, index: number): string | null => {
const match = /^\$[A-Za-z_][A-Za-z0-9_]*\$|^\$\$/u.exec(sql.slice(index));
return match?.[0] ?? null;
};
export const splitPostgresStatements = (sql: string): string[] => {
const statements: string[] = [];
let state: ScannerState = { kind: 'normal' };
let start = 0;
for (let i = 0; i < sql.length; i++) {
const char = sql[i]!;
const next = sql[i + 1];
switch (state.kind) {
case 'normal': {
const dollarTag =
char === '$' ? readDollarQuoteTag(sql, i) : null;
if (dollarTag) {
i += dollarTag.length - 1;
state = { kind: 'dollarQuote', tag: dollarTag };
break;
}
if (char === "'") {
state = { kind: 'singleQuote' };
break;
}
if (char === '"') {
state = { kind: 'doubleQuote' };
break;
}
if (char === '-' && next === '-') {
i += 1;
state = { kind: 'lineComment' };
break;
}
if (char === '/' && next === '*') {
i += 1;
state = { kind: 'blockComment' };
break;
}
if (char === ';') {
const statement = sql.slice(start, i).trim();
if (statement !== '') statements.push(statement);
start = i + 1;
}
break;
}
case 'singleQuote':
if (char === "'" && next === "'") {
i += 1;
} else if (char === '\\' && next !== undefined) {
i += 1;
} else if (char === "'") {
state = { kind: 'normal' };
}
break;
case 'doubleQuote':
if (char === '"' && next === '"') {
i += 1;
} else if (char === '"') {
state = { kind: 'normal' };
}
break;
case 'lineComment':
if (char === '\n') {
state = { kind: 'normal' };
}
break;
case 'blockComment':
if (char === '*' && next === '/') {
i += 1;
state = { kind: 'normal' };
}
break;
case 'dollarQuote':
if (sql.startsWith(state.tag, i)) {
i += state.tag.length - 1;
state = { kind: 'normal' };
}
break;
}
}
const finalStatement = sql.slice(start).trim();
if (finalStatement !== '') statements.push(finalStatement);
return statements;
};
+7 -7
View File
@@ -24,7 +24,7 @@ import type { IConfig } from '../../types';
import { PuterClient } from '../types';
import { EMAIL_TEMPLATES, type EmailTemplateName } from './templates';
// ── Types ────────────────────────────────────────────────────────────
// -- Types ------------------------------------------------------------
// nodemailer doesn't ship TS types, so declare the subset we use.
interface NodemailerTransport {
@@ -50,7 +50,7 @@ interface CompiledTemplate {
html: ReturnType<typeof template>;
}
// ── Clean-email rules ────────────────────────────────────────────────
// -- Clean-email rules ------------------------------------------------
type CleanRule = (parts: { local: string; domain: string }) => {
local: string;
@@ -90,7 +90,7 @@ const DOMAIN_ALIASES: Record<string, string> = {
'googlemail.com': 'gmail.com',
};
// ── EmailClient ──────────────────────────────────────────────────────
// -- EmailClient ------------------------------------------------------
/**
* Unified email client. Handles:
@@ -112,7 +112,7 @@ export class EmailClient extends PuterClient {
this.compileTemplates();
}
// ── Lifecycle ────────────────────────────────────────────────────
// -- Lifecycle ----------------------------------------------------
override onServerStart(): void {
const emailConf = this.config.email;
@@ -133,7 +133,7 @@ export class EmailClient extends PuterClient {
this.transport = null;
}
// ── Public API: sending ──────────────────────────────────────────
// -- Public API: sending ------------------------------------------
/**
* Render a template and send it to `to`.
@@ -174,7 +174,7 @@ export class EmailClient extends PuterClient {
});
}
// ── Public API: clean / validate ─────────────────────────────────
// -- Public API: clean / validate ---------------------------------
/**
* Normalize an email to its canonical form for dedup comparisons.
@@ -241,7 +241,7 @@ export class EmailClient extends PuterClient {
this.validators.push(fn);
}
// ── Internals ────────────────────────────────────────────────────
// -- Internals ----------------------------------------------------
private defaultFrom(): string {
return this.config.email?.from ?? '"Puter" [email protected]';
+15 -5
View File
@@ -19,7 +19,13 @@
import { extensionStore } from '../../extensions';
import { PuterClient } from '../types';
import { EventListener, EventMap, ListenKey, MatchingEvents } from './types';
import {
EventListener,
EventMap,
EventMetadata,
ListenKey,
MatchingEvents,
} from './types';
export class EventClient extends PuterClient {
#eventListeners: Partial<Record<ListenKey, EventListener[]>> = {};
@@ -52,7 +58,11 @@ export class EventClient extends PuterClient {
* scan of every listener.
*
*/
emit<T extends keyof EventMap>(key: T, data: EventMap[T], meta: unknown) {
emit<T extends keyof EventMap>(
key: T,
data: EventMap[T],
meta: EventMetadata,
) {
const parts = key.split('.');
for (let i = 0; i < parts.length; i++) {
const matchKey = (
@@ -87,7 +97,7 @@ export class EventClient extends PuterClient {
async emitAndWait<T extends keyof EventMap>(
key: T,
data: EventMap[T],
meta: unknown,
meta: EventMetadata,
) {
const parts = key.split('.');
for (let i = 0; i < parts.length; i++) {
@@ -129,7 +139,7 @@ export class EventClient extends PuterClient {
callback: (
key: MatchingEvents<P>,
data: EventMap[MatchingEvents<P>],
meta: unknown,
meta: EventMetadata,
) => Promise<void> | void,
) {
const listeners: EventListener[] =
@@ -140,7 +150,7 @@ export class EventClient extends PuterClient {
listener: EventListener,
key: T,
data: EventMap[T],
meta: unknown,
meta: EventMetadata,
) {
try {
await listener(key, data, meta);
+226 -6
View File
@@ -14,6 +14,13 @@
* the SocketService fans out to user-scoped channels.
*/
import type {
Request as ExpressRequest,
Response as ExpressResponse,
} from 'express';
import { Actor } from '../../core';
import { FSEntry } from '../../stores/fs/FSEntry';
// GUI write events spread an entry plus per-event metadata into `response`.
// The exact field set varies by emit site (FSController / LegacyFSController /
// WebDAVController each project a slightly different shape) so the envelope
@@ -96,8 +103,11 @@ export type EventMap = {
};
'app.from-origin': { origin: string };
'app.privateAccess.check': {
app: unknown;
actor: unknown;
appUid: string;
userUid: string;
requestHost: string;
requestPath: string;
actor?: Actor;
result: {
allowed: boolean;
reason?: string;
@@ -110,6 +120,21 @@ export type EventMap = {
actor: unknown;
result: { allowed: boolean; reason?: string; error?: string };
};
// Emitted after an app token is granted (get-user-app-token). The app-side
// analogue of `site.htmlServed` — the external-app abuse scanner
// (extensions/subdomainAbuse/appScan.ts) listens to screenshot + classify
// external `index_url`s that Puter never serves itself.
'puter.app.authenticated': {
app_uid?: string;
user_id?: number | null;
app?: {
id?: number;
uid?: string;
index_url?: string | null;
owner_user_id?: number | null;
name?: string | null;
};
};
// ---- Auth / signup ----
'puter.signup.validate': {
@@ -120,7 +145,19 @@ export type EventMap = {
req?: unknown;
data?: unknown;
abuse?: unknown;
custom?: unknown;
trail?: Array<string>;
/**
* Set by the abuse harness for flagged signups — the id under which the
* decision trail is persisted to KV (`abuse:trail:<id>`), shared back on
* the request for log / support correlation.
*/
trail_id?: string;
/** Device signal forwarded verbatim from the signup request body. */
fingerprint?: string | null;
/** Set by the abuse harness — require SMS phone verification post-signup. */
requires_phone_verification?: boolean;
/** Set by the abuse harness — require card verification post-signup. */
requires_card_verification?: boolean;
[key: string]: unknown;
};
'puter.signup.success': {
@@ -129,6 +166,9 @@ export type EventMap = {
email: string;
username: string;
ip?: string | null;
fingerprint?: string | null;
/** True when the created account is a temp user (no email/password). */
is_temp?: boolean;
[key: string]: unknown;
};
'email.validate': {
@@ -148,12 +188,93 @@ export type EventMap = {
user_uid: string;
email: string;
};
// Phone-reuse cap is pure mechanism here — the abuse extension answers
// (emitted via `emitAndWait`) by counting how many OTHER accounts have
// already verified this number and flipping `allowed` to false when the
// cross-account limit is hit. No extension listening → `allowed` stays
// true and the send proceeds.
'puter.phone-verification.check': {
user_id: number;
user_uid: string;
phone: string;
// Client-supplied device fingerprint for this request, or null. Lets
// the abuse extension cap sends per device (across accounts) so a
// device farm can't dodge the per-account cap with fresh signups.
device_fingerprint: string | null;
allowed: boolean;
reason: string | null;
[key: string]: unknown;
};
// Fire-and-forget signal that a code was actually sent — the abuse
// extension bumps its per-number / per-account / per-device send-velocity
// counters off this. No-op with no extension listening.
'puter.phone-verification.sent': {
user_id: number;
user_uid: string;
phone: string;
device_fingerprint: string | null;
};
'user.phone-verified': {
user_id: number;
user_uid: string;
phone: string;
};
// Card verification is pure mechanism here — a payments extension fills
// these in (emitted via `emitAndWait`). `enabled` stays null when no
// extension is installed; the extension always sets it (true/false) so
// the endpoints can distinguish "disabled" from "not installed".
'puter.card-verification.setup': {
user_id: number;
user_uid: string;
ip?: string | null;
// Client-supplied device fingerprint for this request, or null. Lets
// the abuse extension cap card-verification setups per device (across
// accounts) before any Stripe SetupIntent is created.
device_fingerprint: string | null;
enabled: boolean | null;
// Set false by the extension to refuse this setup (e.g. the per-device
// setup-velocity cap); `reason` carries the opaque code. Stays true
// when allowed or with no extension listening.
allowed: boolean;
reason: string | null;
client_secret: string | null;
publishable_key: string | null;
[key: string]: unknown;
};
'puter.card-verification.confirm': {
user_id: number;
user_uid: string;
setup_intent_id: string;
enabled: boolean | null;
verified: boolean;
reason: string | null;
fingerprint: string | null;
funding: string | null;
country: string | null;
[key: string]: unknown;
};
'user.card-verified': {
user_id: number;
user_uid: string;
fingerprint: string | null;
funding: string | null;
country: string | null;
};
'user.username-changed': {
user_id: number;
old_username: string;
new_username: string;
};
'user.email-changed': { user_id: number; new_email: string };
// Fired after an account is torn down (self-serve, admin, or temp-user
// logout cleanup). Listeners purge external state tied to the account —
// e.g. the marketplace extension cancels the user's Stripe subscriptions.
// The row is already gone by emit time, so identifiers ride the payload.
'user.delete': {
user_id: number;
user_uuid?: string;
stripe_customer_id?: string | null;
};
// ---- Filesystem ----
'fs.copy.node': {
@@ -162,8 +283,9 @@ export type EventMap = {
sourceObjectKey: string;
copyObjectKey: string;
};
'fs.move.node': { node: unknown; fromPath: string; toPath: string };
'fs.remove.node': { node: unknown; entry: unknown; target: unknown };
'fs.move.node': { node: FSEntry; fromPath: string; toPath: string };
'fs.remove.node': { node: FSEntry; entry: FSEntry; target: FSEntry };
'fs.write.file': { node: FSEntry; entry: FSEntry; target: FSEntry };
'fs.storage.upload-progress': {
upload_tracker: unknown;
context: unknown;
@@ -199,6 +321,15 @@ export type EventMap = {
// ---- Subdomains ----
'subdomain.delete': { subdomain: string };
'subdomain.update': { subdomain: string };
'site.htmlServed': {
subdomain: string;
entry: unknown;
host: string;
requestPath: string;
requestUrl?: string;
requestHash?: string;
mime: string;
};
// ---- Thumbnails ----
'thumbnail.read': {
@@ -243,6 +374,94 @@ export type EventMap = {
// SocketService re-emits each fanout-eligible event under
// `sent-to-user.<wireName>` so per-user channels can subscribe by wire name.
[K in `sent-to-user.${string}`]: { user_id: number; response: unknown };
} & {
// Generic per-driver-method lifecycle, emitted by DriverController for
// EVERY driver call under a key scoped to the interface + method:
// `driver.<iface>.<method>.before|after|error|reject`. Wildcard
// subscribers can listen to `driver.*` for everything, `driver.<iface>.*`
// for one interface, or the exact key for one method. The `.before` phase
// is emitted via `emitAndWait`, so a listener may set `allow = false`
// (with an optional `rejectReason`) to veto the call before it runs — a
// vetoed call emits `.reject` instead of running.
[K in `driver.${string}`]: DriverMethodLifecycleEvent;
} & {
// Generic per-route-endpoint lifecycle, emitted by the route materializer
// for EVERY non-middleware route under a key scoped to the HTTP method +
// normalized path: `route.<method>.<path>.before|after|error|reject`. Same
// wildcard + veto semantics as the driver lifecycle above.
[K in `route.${string}`]: RouteLifecycleEvent;
};
/**
* Phase of a request/method lifecycle. `reject` is emitted when a `before`
* listener vetoes the call (sets `allow = false`); the call never runs and no
* `after`/`error` follows.
*/
export type LifecyclePhase = 'before' | 'after' | 'error' | 'reject';
/**
* Payload for `driver.<iface>.<method>.<phase>` events.
*
* One shape across all phases; read `phase` (or the key suffix) to
* branch. `allow`/`rejectReason` are only meaningful on the `before` phase
* (emitted via `emitAndWait`).
*/
export type DriverMethodLifecycleEvent = {
phase: LifecyclePhase;
iface: string;
method: string;
/** Resolved concrete driver name. */
driver: string;
/** Full actor object, if the request is authenticated. */
actor?: Actor;
/** Stable actor id (see `actorUid`), if the request is authenticated. */
actorUid?: string;
/** Call arguments. Present on every phase. */
args?: unknown;
/** Return value. Present on `after`. */
result?: unknown;
/** Thrown error. Present on `error`. */
error?: unknown;
/** Wall-clock duration of the invocation. Present on `after`/`error`. */
durationMs?: number;
/** Veto channel for `before`: set `false` to block the call. */
allow?: boolean;
/** Optional human-readable reason surfaced to the caller when vetoed. */
rejectReason?: string;
};
/**
* Payload for `route.<method>.<path>.<phase>` events. Mirrors
* {@link DriverMethodLifecycleEvent} for HTTP endpoints.
*/
export type RouteLifecycleEvent = {
phase: LifecyclePhase;
/** HTTP method, lowercased (`get`, `post`, ...). */
method: string;
/** Full route path including the controller prefix. */
path: string;
/**
* The live express request/response for this call. Present on every phase.
* On `before` a listener can read the parsed body/headers or write its own
* response; on terminal phases they're useful for logging. These are real
* in-process objects — never serialize or forward them across nodes.
*/
req: ExpressRequest;
res: ExpressResponse;
/** Full actor object, if the request is authenticated. */
actor?: Actor;
/** Stable actor id (see `actorUid`), if the request is authenticated. */
actorUid?: string;
/** Response status code. Present on `after`/`error`. */
statusCode?: number;
/** Wall-clock duration from `before` to terminal phase. */
durationMs?: number;
/** Set when the request did not complete normally (abort / >=500). */
error?: unknown;
/** Veto channel for `before`: set `false` to block the request. */
allow?: boolean;
/** Optional human-readable reason surfaced to the caller when vetoed. */
rejectReason?: string;
};
export type EventKey = keyof EventMap & string;
@@ -264,8 +483,9 @@ export type MatchingEvents<P extends ListenKey> = P extends `${infer Prefix}.*`
? Extract<EventKey, `${Prefix}.${string}`>
: P & EventKey;
export type EventMetadata = { from_outside?: boolean };
export type EventListener<K extends EventKey = EventKey> = (
key: K,
data: EventMap[K],
meta: unknown,
meta: EventMetadata,
) => Promise<void> | void;
+2
View File
@@ -24,6 +24,7 @@ import { EventClient } from './event/EventClient';
import { DDBClient } from './dynamodb/DDBClient';
import { RedisClient } from './redis/RedisClient';
import { S3Client } from './s3/S3Client';
import { PreludeClient } from './prelude/PreludeClient';
import type { IPuterClientRegistry } from './types';
export const puterClients = {
@@ -34,4 +35,5 @@ export const puterClients = {
dynamo: DDBClient,
redis: RedisClient,
s3: S3Client,
prelude: PreludeClient,
} satisfies IPuterClientRegistry;
@@ -0,0 +1,216 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { PreludeClient } from './PreludeClient';
import type { IConfig } from '../../types';
const makeClient = (apiKey?: string) =>
new PreludeClient({
prelude: apiKey ? { apiKey } : undefined,
} as unknown as IConfig);
const okJson = (body: unknown) =>
({
ok: true,
status: 200,
json: async () => body,
}) as unknown as Response;
describe('PreludeClient', () => {
let fetchMock: ReturnType<typeof vi.fn>;
beforeEach(() => {
fetchMock = vi.fn();
vi.stubGlobal('fetch', fetchMock);
});
afterEach(() => {
vi.unstubAllGlobals();
});
it('isConfigured reflects whether an apiKey is set', () => {
expect(makeClient('sk_test').isConfigured()).toBe(true);
expect(makeClient().isConfigured()).toBe(false);
});
describe('isCountrySupported (€0.07 cap)', () => {
const client = makeClient('sk_test');
it('allows revenue markets up to the cap (incl. the priciest)', () => {
expect(client.isCountrySupported('US')).toBe(true); // €0.0043
expect(client.isCountrySupported('DE')).toBe(true); // €0.0598
expect(client.isCountrySupported('SA')).toBe(true); // €0.0638
expect(client.isCountrySupported('us')).toBe(true); // case-insensitive
});
it('rejects countries above the cap, with no SMS, or unknown', () => {
expect(client.isCountrySupported('PK')).toBe(false); // €0.3548
expect(client.isCountrySupported('ID')).toBe(false); // €0.2430
expect(client.isCountrySupported('LI')).toBe(false); // null (no SMS)
expect(client.isCountrySupported('ZZ')).toBe(false); // unknown
expect(client.isCountrySupported(undefined)).toBe(false);
});
it('honors a configured maxSmsCostEur override', () => {
const strict = new PreludeClient({
prelude: { apiKey: 'sk', maxSmsCostEur: 0.01 },
} as unknown as IConfig);
expect(strict.isCountrySupported('US')).toBe(true); // €0.0043
expect(strict.isCountrySupported('DE')).toBe(false); // €0.0598 > 0.01
const loose = new PreludeClient({
prelude: { apiKey: 'sk', maxSmsCostEur: 0.5 },
} as unknown as IConfig);
expect(loose.isCountrySupported('PK')).toBe(true); // €0.3548 <= 0.5
});
});
it('createVerification POSTs the phone target + ip signal with bearer auth', async () => {
fetchMock.mockResolvedValue(
okJson({ id: 'vrf_1', status: 'success' }),
);
const client = makeClient('sk_test');
const res = await client.createVerification('+14155550123', {
ip: '203.0.113.7',
});
expect(res).toEqual({ id: 'vrf_1', status: 'success' });
const [url, init] = fetchMock.mock.calls[0];
expect(url).toBe('https://api.prelude.dev/v2/verification');
expect(init.method).toBe('POST');
expect(init.headers.Authorization).toBe('Bearer sk_test');
expect(JSON.parse(init.body)).toEqual({
target: { type: 'phone_number', value: '+14155550123' },
// Defaults to RCS (cheaper); Prelude falls back to SMS. locale is
// hardcoded to en-US so the message text is always English.
options: { code_size: 6, preferred_channel: 'rcs', locale: 'en-US' },
signals: { ip: '203.0.113.7' },
});
});
it('returns the delivery channel sequence Prelude reports', async () => {
fetchMock.mockResolvedValue(
okJson({
id: 'vrf_1',
status: 'success',
channels: ['whatsapp', 'sms'],
}),
);
const client = makeClient('sk_test');
const res = await client.createVerification('+14155550123');
expect(res.channels).toEqual(['whatsapp', 'sms']);
});
it('forwards device_id and user_agent signals when supplied', async () => {
fetchMock.mockResolvedValue(okJson({ id: 'v', status: 'success' }));
const client = makeClient('sk_test');
await client.createVerification('+14155550123', {
ip: '203.0.113.7',
device_id: 'thumb_abc123',
user_agent: 'Mozilla/5.0',
});
const [, init] = fetchMock.mock.calls[0];
expect(JSON.parse(init.body).signals).toEqual({
ip: '203.0.113.7',
device_id: 'thumb_abc123',
user_agent: 'Mozilla/5.0',
});
});
it('omits the signals object entirely when none are supplied', async () => {
fetchMock.mockResolvedValue(okJson({ id: 'v', status: 'success' }));
const client = makeClient('sk_test');
await client.createVerification('+14155550123');
expect(JSON.parse(fetchMock.mock.calls[0][1].body)).not.toHaveProperty(
'signals',
);
});
it('forwards dispatch_id as a top-level field, not inside signals', async () => {
fetchMock.mockResolvedValue(okJson({ id: 'v', status: 'success' }));
const client = makeClient('sk_test');
await client.createVerification('+14155550123', {
ip: '203.0.113.7',
dispatch_id: 'd1f5e9a0-0000-4000-8000-000000000000',
});
const body = JSON.parse(fetchMock.mock.calls[0][1].body);
expect(body.dispatch_id).toBe('d1f5e9a0-0000-4000-8000-000000000000');
expect(body.signals).toEqual({ ip: '203.0.113.7' });
expect(body.signals).not.toHaveProperty('dispatch_id');
});
it('omits dispatch_id when not supplied', async () => {
fetchMock.mockResolvedValue(okJson({ id: 'v', status: 'success' }));
const client = makeClient('sk_test');
await client.createVerification('+14155550123', { ip: '203.0.113.7' });
expect(JSON.parse(fetchMock.mock.calls[0][1].body)).not.toHaveProperty(
'dispatch_id',
);
});
it('includes a configured template_id + sender_id + preferred channel', async () => {
fetchMock.mockResolvedValue(okJson({ id: 'v', status: 'success' }));
const client = new PreludeClient({
prelude: {
apiKey: 'sk_test',
templateId: 'tmpl_puter',
senderId: 'Puter',
preferredChannel: 'sms',
},
} as unknown as IConfig);
await client.createVerification('+14155550123');
const [, init] = fetchMock.mock.calls[0];
expect(JSON.parse(init.body).options).toEqual({
code_size: 6,
preferred_channel: 'sms', // config override beats the rcs default
locale: 'en-US',
template_id: 'tmpl_puter',
sender_id: 'Puter',
});
});
it('checkVerification POSTs target + code and returns the status', async () => {
fetchMock.mockResolvedValue(okJson({ status: 'success' }));
const client = makeClient('sk_test');
const res = await client.checkVerification('+14155550123', '123456');
expect(res).toEqual({ status: 'success' });
const [url, init] = fetchMock.mock.calls[0];
expect(url).toBe('https://api.prelude.dev/v2/verification/check');
expect(JSON.parse(init.body)).toEqual({
target: { type: 'phone_number', value: '+14155550123' },
code: '123456',
});
});
it('throws (does not call fetch) when not configured', async () => {
const client = makeClient();
await expect(
client.createVerification('+14155550123'),
).rejects.toThrow(/not configured/i);
expect(fetchMock).not.toHaveBeenCalled();
});
it('throws on a non-2xx Prelude response', async () => {
fetchMock.mockResolvedValue({
ok: false,
status: 400,
json: async () => ({ message: 'bad target' }),
} as unknown as Response);
await expect(
makeClient('sk_test').checkVerification('+1', 'x'),
).rejects.toThrow(/Prelude .* failed: 400/);
});
});
@@ -0,0 +1,226 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import type { IConfig } from '../../types';
import { PuterClient } from '../types';
import { COUNTRY_SMS_PRICES } from './countries.js';
const PRELUDE_API_BASE = 'https://api.prelude.dev/v2';
const REQUEST_TIMEOUT_MS = 8000;
/** OTP length — matches the 6-box code UI. Prelude allows 4–8. */
const PRELUDE_CODE_SIZE = 6;
/**
* Channel Prelude prioritizes for delivery. RCS is far cheaper than SMS, so we
* prefer it by default; Prelude routes to the next reachable channel (SMS) when
* RCS isn't available on the destination. (Requires an RCS agent provisioned in
* the Prelude account to actually use RCS — otherwise it just falls through to
* SMS.) Override with `config.prelude.preferredChannel`.
*/
const DEFAULT_PREFERRED_CHANNEL: PreludeChannel = 'rcs';
/** Delivery channels Prelude can prioritize via `options.preferred_channel`. */
export type PreludeChannel =
| 'sms'
| 'rcs'
| 'whatsapp'
| 'viber'
| 'zalo'
| 'telegram';
/**
* Channels Prelude reports in a verification's `channels` array — the ordered
* delivery sequence, first entry first. Superset of `PreludeChannel`: 'silent'
* and 'voice' can appear as delivery methods but can't be preferred.
*/
export type PreludeDeliveryChannel = PreludeChannel | 'silent' | 'voice';
/**
* Default per-SMS cost ceiling (EUR). Countries whose Prelude SMS rate exceeds
* this — or that have no SMS channel — are not offered phone verification. The
* cap covers every realistic revenue market (priciest are Germany €0.0598 and
* Saudi Arabia €0.0638) while excluding the expensive, high-fraud long tail.
* Override per-deployment with `config.prelude.maxSmsCostEur`.
*/
const DEFAULT_MAX_SMS_COST_EUR = 0.07;
/** Status returned by Prelude when creating/retrying a verification. */
export type PreludeCreateStatus =
| 'success'
| 'retry'
| 'challenged'
| 'blocked'
| 'shadow_blocked';
/** Status returned by Prelude when checking a code. */
export type PreludeCheckStatus =
| 'success'
| 'failure'
| 'expired_or_not_found'
| 'transaction_missing'
| 'transaction_mismatch';
/**
* Prelude Verify v2 client (https://docs.prelude.so/verify/v2). Sends and checks
* SMS one-time codes — Prelude generates, delivers, and validates the code, so
* we never store one ourselves. No-ops with a warning when no API key is
* configured (so dev environments don't crash); `isConfigured()` lets callers
* surface a clean "phone verification unavailable" error instead.
*/
export class PreludeClient extends PuterClient {
private apiKey: string | null = null;
constructor(config: IConfig) {
super(config);
this.apiKey = config.prelude?.apiKey ?? null;
}
override onServerStart(): void {
if (!this.apiKey) {
console.warn(
'[prelude] no apiKey configured — SMS phone verification is disabled',
);
}
}
/** True when an API key is configured and verification can be attempted. */
isConfigured(): boolean {
return !!this.apiKey;
}
/** ISO region used to parse local-format numbers (config-driven). */
get defaultCountry(): string | undefined {
return this.config.prelude?.defaultCountry;
}
/** Per-SMS cost ceiling in EUR (config override or the default cap). */
get maxSmsCostEur(): number {
return this.config.prelude?.maxSmsCostEur ?? DEFAULT_MAX_SMS_COST_EUR;
}
/**
* Whether SMS verification should be offered for a country. False when the
* country is unknown, has no SMS channel, or its rate exceeds the cost cap.
* @param iso ISO-3166 alpha-2 (e.g. 'US') — from the parsed phone number.
*/
isCountrySupported(iso: string | undefined): boolean {
if (!iso) return false; // couldn't determine country → can't price it
const price = COUNTRY_SMS_PRICES[iso.toUpperCase()];
if (!price || price.sms == null) return false;
return price.sms <= this.maxSmsCostEur;
}
async createVerification(
target: string,
signals: {
ip?: string;
device_id?: string;
user_agent?: string;
dispatch_id?: string;
} = {},
): Promise<{
id?: string;
status: PreludeCreateStatus;
channels?: PreludeDeliveryChannel[];
}> {
// Match the 6-box code UI (UIWindowPhoneVerificationRequired). Without
// code_size Prelude uses the dashboard default (4). preferred_channel
// prioritizes RCS (cheaper); Prelude falls back to SMS when unavailable.
const options: Record<string, unknown> = {
code_size: PRELUDE_CODE_SIZE,
preferred_channel:
this.config.prelude?.preferredChannel ??
DEFAULT_PREFERRED_CHANNEL,
locale: 'en-US',
};
// Branding lives in the Prelude dashboard (the message text is a
// template); these just select a Puter-branded template / sender when
// configured. See IPreludeConfig.
const { templateId, senderId } = this.config.prelude ?? {};
if (templateId) options.template_id = templateId;
if (senderId) options.sender_id = senderId;
const body: Record<string, unknown> = {
target: { type: 'phone_number', value: target },
options,
};
// Only attach signals we actually have — Prelude treats the object as
// optional and an empty one adds nothing.
const sig: Record<string, string> = {};
if (signals.ip) sig.ip = signals.ip;
if (signals.device_id) sig.device_id = signals.device_id;
if (signals.user_agent) sig.user_agent = signals.user_agent;
if (Object.keys(sig).length > 0) body.signals = sig;
// dispatch_id is a top-level field, not a member of `signals`.
if (signals.dispatch_id) body.dispatch_id = signals.dispatch_id;
return this.#post('/verification', body) as Promise<{
id?: string;
status: PreludeCreateStatus;
channels?: PreludeDeliveryChannel[];
}>;
}
/**
* Check a code the user entered against the active verification for `target`.
* @returns `{ status }` — `'success'` means verified.
*/
async checkVerification(
target: string,
code: string,
): Promise<{ status: PreludeCheckStatus }> {
return this.#post('/verification/check', {
target: { type: 'phone_number', value: target },
code,
}) as Promise<{ status: PreludeCheckStatus }>;
}
async #post(
path: string,
body: Record<string, unknown>,
): Promise<Record<string, unknown>> {
if (!this.apiKey) {
throw new Error('Prelude is not configured');
}
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
try {
const res = await fetch(`${PRELUDE_API_BASE}${path}`, {
method: 'POST',
headers: {
Authorization: `Bearer ${this.apiKey}`,
'Content-Type': 'application/json',
Accept: 'application/json',
},
body: JSON.stringify(body),
signal: controller.signal,
});
const json = (await res.json().catch(() => ({}))) as Record<
string,
unknown
>;
if (!res.ok) {
throw new Error(
`Prelude ${path} failed: ${res.status} ${
(json as { message?: string })?.message ?? ''
}`.trim(),
);
}
return json;
} finally {
clearTimeout(timer);
}
}
}
+952
View File
@@ -0,0 +1,952 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/** Per-country Prelude SMS pricing, hardcoded from the Prelude global price
* sheet. `sms` is the average cost in EUR, or null when SMS is unavailable
* there. Used to cap phone verification to affordable countries (see
* PreludeClient.isCountrySupported). Regenerate from the price sheet when
* Prelude updates rates. */
export interface CountryPrice {
name: string;
/** Prelude SMS average in EUR, or null when SMS is not available. */
sms: number | null;
}
export const COUNTRY_SMS_PRICES: Record<string, CountryPrice> = {
AD: {
name: 'Andorra',
sms: 0.0515,
},
AE: {
name: 'United Arab Emirates',
sms: 0.0145,
},
AF: {
name: 'Afghanistan',
sms: 0.2409,
},
AG: {
name: 'Antigua and Barbuda',
sms: 0.0755,
},
AI: {
name: 'Anguilla',
sms: 0.0765,
},
AL: {
name: 'Albania',
sms: 0.0345,
},
AM: {
name: 'Armenia',
sms: 0.118,
},
AN: {
name: 'Netherlands Antilles',
sms: null,
},
AO: {
name: 'Angola',
sms: 0.0347,
},
AR: {
name: 'Argentina',
sms: 0.0614,
},
AS: {
name: 'American Samoa',
sms: 0.077,
},
AT: {
name: 'Austria',
sms: 0.014,
},
AU: {
name: 'Australia',
sms: 0.0091,
},
AW: {
name: 'Aruba',
sms: 0.0617,
},
AZ: {
name: 'Azerbaijan',
sms: 0.0969,
},
BA: {
name: 'Bosnia and Herzegovina',
sms: 0.0801,
},
BB: {
name: 'Barbados',
sms: 0.2021,
},
BD: {
name: 'Bangladesh',
sms: 0.1844,
},
BE: {
name: 'Belgium',
sms: 0.0297,
},
BF: {
name: 'Burkina Faso',
sms: 0.063,
},
BG: {
name: 'Bulgaria',
sms: 0.0799,
},
BH: {
name: 'Bahrain',
sms: 0.014,
},
BI: {
name: 'Burundi',
sms: 0.217,
},
BJ: {
name: 'Benin',
sms: 0.13,
},
BM: {
name: 'Bermuda',
sms: 0.21,
},
BN: {
name: 'Brunei',
sms: 0.035,
},
BO: {
name: 'Bolivia',
sms: 0.0708,
},
BQ: {
name: 'Caribbean Netherlands',
sms: 0.0565,
},
BR: {
name: 'Brazil',
sms: 0.0086,
},
BS: {
name: 'Bahamas',
sms: 0.0672,
},
BT: {
name: 'Bhutan',
sms: null,
},
BW: {
name: 'Botswana',
sms: 0.0235,
},
BY: {
name: 'Belarus',
sms: 0.118,
},
BZ: {
name: 'Belize',
sms: 0.1529,
},
CA: {
name: 'Canada',
sms: 0.0052,
},
CD: {
name: 'Congo RDC',
sms: 0.1112,
},
CF: {
name: 'Central African Republic',
sms: 0.2,
},
CG: {
name: 'Congo',
sms: 0.2507,
},
CH: {
name: 'Switzerland',
sms: 0.0163,
},
CI: {
name: "Cote d'Ivoire",
sms: 0.19,
},
CK: {
name: 'Cook Islands',
sms: 0.1041,
},
CL: {
name: 'Chile',
sms: 0.0027,
},
CM: {
name: 'Cameroon',
sms: 0.1504,
},
CN: {
name: 'China',
sms: 0.0053,
},
CO: {
name: 'Colombia',
sms: 0.0008,
},
CR: {
name: 'Costa Rica',
sms: 0.0045,
},
CU: {
name: 'Cuba',
sms: 0.0512,
},
CV: {
name: 'Cabo Verde',
sms: 0.2031,
},
CW: {
name: 'Curacao',
sms: 0.0138,
},
CY: {
name: 'Northern Cyprus',
sms: 0.0065,
},
CZ: {
name: 'Czech Republic',
sms: 0.0299,
},
DE: {
name: 'Germany',
sms: 0.0598,
},
DJ: {
name: 'Djibouti',
sms: 0.0897,
},
DK: {
name: 'Denmark',
sms: 0.0301,
},
DM: {
name: 'Dominica',
sms: 0.0824,
},
DO: {
name: 'Dominican Republic',
sms: 0.0351,
},
DZ: {
name: 'Algeria',
sms: 0.196,
},
EC: {
name: 'Ecuador',
sms: 0.1041,
},
EE: {
name: 'Estonia',
sms: 0.0234,
},
EG: {
name: 'Egypt',
sms: 0.1561,
},
ER: {
name: 'Eritrea',
sms: 0.0659,
},
ES: {
name: 'Spain',
sms: 0.0193,
},
ET: {
name: 'Ethiopia',
sms: 0.2741,
},
FI: {
name: 'Finland',
sms: 0.043,
},
FJ: {
name: 'Fiji',
sms: 0.069,
},
FK: {
name: 'Falkland Islands',
sms: 0.0713,
},
FM: {
name: 'Micronesia',
sms: 0.0118,
},
FO: {
name: 'Faroe Islands',
sms: 0.0319,
},
FR: {
name: 'France',
sms: 0.03,
},
GA: {
name: 'Gabon',
sms: 0.15,
},
GB: {
name: 'United Kingdom',
sms: 0.026,
},
GD: {
name: 'Grenada',
sms: null,
},
GE: {
name: 'Georgia',
sms: 0.0788,
},
GF: {
name: 'French Guiana',
sms: 0.05,
},
GG: {
name: 'Guernsey',
sms: 0.025,
},
GH: {
name: 'Ghana',
sms: 0.15,
},
GI: {
name: 'Gibraltar',
sms: 0.0134,
},
GL: {
name: 'Greenland',
sms: 0.0048,
},
GM: {
name: 'Gambia',
sms: 0.1283,
},
GN: {
name: 'Guinea',
sms: 0.2,
},
GP: {
name: 'Guadeloupe',
sms: 0.0455,
},
GQ: {
name: 'Equatorial Guinea',
sms: 0.099,
},
GR: {
name: 'Greece',
sms: 0.0296,
},
GT: {
name: 'Guatemala',
sms: 0.1202,
},
GU: {
name: 'Guam',
sms: 0.02,
},
GW: {
name: 'Guinea-Bissau',
sms: 0.1566,
},
GY: {
name: 'Guyana',
sms: 0.2178,
},
HK: {
name: 'Hong Kong',
sms: 0.038,
},
HN: {
name: 'Honduras',
sms: 0.162,
},
HR: {
name: 'Croatia',
sms: 0.03,
},
HT: {
name: 'Haiti',
sms: 0.09,
},
HU: {
name: 'Hungary',
sms: 0.029,
},
ID: {
name: 'Indonesia',
sms: 0.243,
},
IE: {
name: 'Ireland',
sms: 0.0305,
},
IL: {
name: 'Israel',
sms: 0.01,
},
IM: {
name: 'Isle of Man',
sms: 0.0385,
},
IN: {
name: 'India',
sms: 0.0375,
},
IQ: {
name: 'Iraq',
sms: 0.151,
},
IR: {
name: 'Iran',
sms: 0.14,
},
IS: {
name: 'Iceland',
sms: 0.0493,
},
IT: {
name: 'Italy',
sms: 0.0245,
},
JE: {
name: 'Jersey',
sms: 0.025,
},
JM: {
name: 'Jamaica',
sms: 0.1525,
},
JO: {
name: 'Jordan',
sms: 0.2094,
},
JP: {
name: 'Japan',
sms: 0.017,
},
KE: {
name: 'Kenya',
sms: 0.125,
},
KG: {
name: 'Kyrgyzstan',
sms: 0.15,
},
KH: {
name: 'Cambodia',
sms: 0.1529,
},
KI: {
name: 'Kiribati',
sms: 0.025,
},
KM: {
name: 'Comoros',
sms: 0.15,
},
KN: {
name: 'Saint Kitts and Nevis',
sms: 0.1295,
},
KR: {
name: 'South Korea',
sms: 0.006,
},
KW: {
name: 'Kuwait',
sms: 0.145,
},
KY: {
name: 'Cayman Islands',
sms: 0.2172,
},
KZ: {
name: 'Kazakhstan',
sms: 0.202,
},
LA: {
name: 'Laos',
sms: 0.15,
},
LB: {
name: 'Lebanon',
sms: 0.153,
},
LC: {
name: 'Saint Lucia',
sms: 0.0892,
},
LI: {
name: 'Liechtenstein',
sms: null,
},
LK: {
name: 'Sri Lanka',
sms: 0.3593,
},
LR: {
name: 'Liberia',
sms: 0.1448,
},
LS: {
name: 'Lesotho',
sms: 0.0301,
},
LT: {
name: 'Lithuania',
sms: 0.0261,
},
LU: {
name: 'Luxembourg',
sms: 0.034,
},
LV: {
name: 'Latvia',
sms: 0.028,
},
LY: {
name: 'Libya',
sms: 0.1898,
},
MA: {
name: 'Morocco',
sms: 0.105,
},
MC: {
name: 'Monaco',
sms: 0.1219,
},
MD: {
name: 'Moldova',
sms: 0.065,
},
ME: {
name: 'Montenegro',
sms: 0.07,
},
MG: {
name: 'Madagascar',
sms: 0.24,
},
MH: {
name: '',
sms: 0.032,
},
MK: {
name: 'Macedonia',
sms: 0.0096,
},
ML: {
name: 'Mali',
sms: 0.136,
},
MM: {
name: 'Myanmar',
sms: 0.3037,
},
MN: {
name: 'Mongolia',
sms: 0.183,
},
MO: {
name: 'Macao',
sms: 0.005,
},
MP: {
name: 'Northern Mariana Islands',
sms: 0.0669,
},
MQ: {
name: 'Martinique',
sms: 0.0455,
},
MR: {
name: 'Mauritania',
sms: 0.161,
},
MS: {
name: 'Montserrat',
sms: 0.0725,
},
MT: {
name: 'Malta',
sms: 0.0343,
},
MU: {
name: 'Mauritius',
sms: 0.1431,
},
MV: {
name: 'Maldives',
sms: 0.145,
},
MW: {
name: 'Malawi',
sms: 0.19,
},
MX: {
name: 'Mexico',
sms: 0.0021,
},
MY: {
name: 'Malaysia',
sms: 0.08,
},
MZ: {
name: 'Mozambique',
sms: 0.2266,
},
NA: {
name: 'Namibia',
sms: 0.0173,
},
NC: {
name: 'New Caledonia',
sms: 0.052,
},
NE: {
name: 'Niger',
sms: 0.143,
},
NG: {
name: 'Nigeria',
sms: 0.198,
},
NI: {
name: 'Nicaragua',
sms: 0.1033,
},
NL: {
name: 'Netherlands',
sms: 0.046,
},
NO: {
name: 'Norway',
sms: 0.03,
},
NP: {
name: 'Nepal',
sms: 0.1645,
},
NR: {
name: 'Nauru',
sms: null,
},
NU: {
name: 'Niue',
sms: null,
},
NZ: {
name: 'New Zealand',
sms: 0.0372,
},
OM: {
name: 'Oman',
sms: 0.0713,
},
PA: {
name: 'Panama',
sms: 0.07,
},
PE: {
name: 'Peru',
sms: 0.13,
},
PF: {
name: 'French Polynesia',
sms: 0.0518,
},
PG: {
name: 'Papua New Guinea',
sms: 0.15,
},
PH: {
name: 'Philippines',
sms: 0.1237,
},
PK: {
name: 'Pakistan',
sms: 0.3548,
},
PL: {
name: 'Poland',
sms: 0.011,
},
PM: {
name: 'Saint Pierre and Miquelon',
sms: 0.0905,
},
PR: {
name: 'Puerto Rico',
sms: 0.02,
},
PS: {
name: 'Palestine',
sms: 0.2541,
},
PT: {
name: 'Portugal',
sms: 0.009,
},
PW: {
name: 'Palau',
sms: null,
},
PY: {
name: 'Paraguay',
sms: 0.0236,
},
QA: {
name: 'Qatar',
sms: 0.1393,
},
RE: {
name: 'Reunion',
sms: 0.034,
},
RO: {
name: 'Romania',
sms: 0.0255,
},
RS: {
name: 'Serbia',
sms: 0.1935,
},
RU: {
name: 'Russia',
sms: 0.2028,
},
RW: {
name: 'Rwanda',
sms: 0.1177,
},
SA: {
name: 'Saudi Arabia',
sms: 0.0638,
},
SB: {
name: 'Solomon Islands',
sms: 0.0357,
},
SC: {
name: 'Seychelles',
sms: 0.0404,
},
SD: {
name: 'Sudan',
sms: 0.224,
},
SE: {
name: 'Sweden',
sms: 0.026,
},
SG: {
name: 'Singapore',
sms: 0.0256,
},
SI: {
name: 'Slovenia',
sms: 0.1,
},
SK: {
name: 'Slovakia',
sms: 0.0245,
},
SL: {
name: 'Sierra Leone',
sms: 0.2322,
},
SM: {
name: 'San Marino',
sms: null,
},
SN: {
name: 'Senegal',
sms: 0.1151,
},
SO: {
name: 'Somalia',
sms: 0.06,
},
SR: {
name: 'Suriname',
sms: 0.119,
},
SS: {
name: 'South Sudan',
sms: 0.15,
},
ST: {
name: 'Sao Tome and Principe',
sms: 0.0133,
},
SV: {
name: 'El Salvador',
sms: 0.06,
},
SX: {
name: 'Sint Maarten',
sms: 0.0623,
},
SY: {
name: 'Syria',
sms: 0.221,
},
SZ: {
name: 'Eswatini',
sms: 0.12,
},
TC: {
name: 'Turks and Caicos Islands',
sms: null,
},
TD: {
name: 'Chad',
sms: 0.1711,
},
TG: {
name: 'Togo',
sms: 0.1806,
},
TH: {
name: 'Thailand',
sms: 0.003,
},
TJ: {
name: 'Tajikistan',
sms: 0.2626,
},
TL: {
name: 'Timor-Leste',
sms: 0.0675,
},
TM: {
name: 'Turkmenistan',
sms: 0.1677,
},
TN: {
name: 'Tunisia',
sms: 0.225,
},
TO: {
name: 'Tonga',
sms: 0.0902,
},
TR: {
name: 'Turkey',
sms: 0.0008,
},
TT: {
name: 'Trinidad and Tobago',
sms: 0.1556,
},
TW: {
name: 'Taiwan',
sms: 0.0165,
},
TZ: {
name: 'Tanzania',
sms: 0.2753,
},
UA: {
name: 'Ukraine',
sms: 0.094,
},
UG: {
name: 'Uganda',
sms: 0.1707,
},
US: {
name: 'United States',
sms: 0.0043,
},
UY: {
name: 'Uruguay',
sms: 0.0174,
},
UZ: {
name: 'Uzbekistan',
sms: 0.292,
},
VC: {
name: 'Saint Vincent and The Grenadines',
sms: 0.1216,
},
VE: {
name: 'Venezuela',
sms: 0.0427,
},
VG: {
name: 'British Virgin Islands',
sms: 0.083,
},
VI: {
name: 'Virgin Island',
sms: 0.0017,
},
VN: {
name: 'Vietnam',
sms: 0.0885,
},
VU: {
name: 'Vanuatu',
sms: 0.1347,
},
WF: {
name: 'Wallis and Futuna',
sms: 0.09,
},
WS: {
name: 'Samoa',
sms: 0.2,
},
XK: {
name: 'Kosovo',
sms: 0.1394,
},
YE: {
name: 'Yemen',
sms: 0.1486,
},
YT: {
name: 'Mayotte',
sms: 0.06,
},
ZA: {
name: 'South Africa',
sms: 0.039,
},
ZM: {
name: 'Zambia',
sms: 0.21,
},
ZW: {
name: 'Zimbabwe',
sms: 0.13,
},
};
+9
View File
@@ -18,6 +18,7 @@
*/
import type { IConfig, WithLifecycle } from '../types';
import type { ClickhouseClient } from './clickhouse/ClickhouseClient';
/**
* Extension-augmentable client registry. Extensions add their own client
@@ -40,6 +41,14 @@ export interface IExtensionClientInstances {
* override this for that name.
*/
[key: string]: unknown;
/**
* Optional ClickHouse analytics client. Absent by default — a production
* deployment registers it via an extension to speed up the app-stats path
* at scale (see {@link ClickhouseClient}). Always branch on its presence
* and fall back to SQL.
*/
clickhouse?: ClickhouseClient;
}
export interface IPuterClient<T extends WithLifecycle = WithLifecycle> {
+38 -3
View File
@@ -17,6 +17,7 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { isAccessTokenActor, isAppActor } from '../../core/actor.js';
import { HttpError } from '../../core/http/HttpError.js';
import { driversContainers } from '../../exports.js';
import {
@@ -55,6 +56,7 @@ export class AppController extends PuterController {
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
},
async (req, res) => {
const apps = await this.appDriver.select({
@@ -70,6 +72,7 @@ export class AppController extends PuterController {
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
},
async (req, res) => {
const name = req.query?.name;
@@ -89,6 +92,16 @@ export class AppController extends PuterController {
// actor calls this, the app id is already on the token — clients
// don't re-send it in the body. Fall back to `actor.app.uid`
// before 400-ing for a missing body field.
//
// Authorization: only two callers are trusted to report opens —
// 1. a root user actor (plain session, no `.app` and no access
// token), e.g. the GUI launching apps on behalf of the user;
// 2. the app-under-user actor for the app being reported, i.e.
// `actor.app.uid === app_uid`.
// Everything else — access tokens (regardless of issuer), asset
// tokens, app actors reporting for a *different* app — is denied,
// otherwise any authenticated party could inflate another app's
// open count.
router.post(
'/rao',
{
@@ -96,8 +109,9 @@ export class AppController extends PuterController {
requireAuth: true,
},
async (req, res) => {
const actor = req.actor;
const bodyAppUid = req.body?.app_uid;
const actorAppUid = req.actor?.app?.uid;
const actorAppUid = actor?.app?.uid;
const app_uid =
typeof bodyAppUid === 'string' && bodyAppUid.length > 0
? bodyAppUid
@@ -108,6 +122,26 @@ export class AppController extends PuterController {
});
}
// Access tokens (and any other non-user/non-app identity,
// e.g. asset tokens) are not allowed to report opens —
// they're shared / scoped credentials and shouldn't drive
// analytics counters.
if (isAccessTokenActor(actor)) {
throw new HttpError(
403,
'Access tokens cannot report app opens',
{ legacyCode: 'forbidden' },
);
}
if (isAppActor(actor) && app_uid !== actorAppUid) {
throw new HttpError(
403,
'App actors can only report opens for their own app',
{ legacyCode: 'forbidden' },
);
}
const app = await this.appStore.getByUid(app_uid);
if (!app)
throw new HttpError(404, 'App not found', {
@@ -153,6 +187,7 @@ export class AppController extends PuterController {
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
},
async (req, res) => {
const raw = req.params.name;
@@ -196,7 +231,7 @@ export class AppController extends PuterController {
},
);
// ── POST /query/app ────────────────────────────────────────
// -- POST /query/app ----------------------------------------
// Batch marketplace-style lookup by name or UID.
//
// Access rules: only apps the caller has a legitimate reason to
@@ -293,7 +328,7 @@ export class AppController extends PuterController {
},
);
// ── GET /app-icon/:app_uid(/:size) ─────────────────────────
// -- GET /app-icon/:app_uid(/:size) -------------------------
// Serve app icon — data URL decoded inline, HTTP URL redirected.
//
// ⚠ FLAG: Missing sharp-based resize pipeline; serves the original.
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -44,13 +44,14 @@ export class DesktopController extends PuterController {
}
registerRoutes(router) {
// ── Desktop background ──────────────────────────────────────
// -- Desktop background --------------------------------------
router.post(
'/set-desktop-bg',
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
},
async (req, res) => {
const { url, color, fit } = req.body ?? {};
@@ -98,13 +99,14 @@ export class DesktopController extends PuterController {
},
);
// ── Taskbar items ───────────────────────────────────────────
// -- Taskbar items -------------------------------------------
router.post(
'/update-taskbar-items',
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
},
async (req, res) => {
const { items } = req.body ?? {};
@@ -123,13 +125,14 @@ export class DesktopController extends PuterController {
},
);
// ── Folder layout ───────────────────────────────────────────
// -- Folder layout -------------------------------------------
router.post(
'/set_layout',
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
},
async (req, res) => {
const { item_uid, item_path, layout } = req.body ?? {};
@@ -149,13 +152,14 @@ export class DesktopController extends PuterController {
},
);
// ── Folder sort ─────────────────────────────────────────────
// -- Folder sort ---------------------------------------------
router.post(
'/set_sort_by',
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
},
async (req, res) => {
const { item_uid, item_path, sort_by, sort_order } =
@@ -188,7 +192,7 @@ export class DesktopController extends PuterController {
);
}
// ── Helpers ──────────────────────────────────────────────────────
// -- Helpers ------------------------------------------------------
/**
* Update columns on an actor-owned fsentry. Accepts either `item_uid`
@@ -124,11 +124,13 @@ const makeSyntheticDriver = () => ({
const buildController = (driver: ReturnType<typeof makeSyntheticDriver>) => {
// The controller reads `this.services?.permission` only when an actor
// is on the request; otherwise the services bag is unused. Empty
// objects are sufficient for this path.
// is on the request; otherwise the services bag is unused. The
// alarm client is invoked on the rate-limit / concurrency rejection
// paths, so stub a no-op `create`.
const clients = { alarm: { create: () => {} } };
return new DriverController(
{} as any,
{} as any,
clients as any,
{} as any,
{} as any,
{ syntheticDriver: driver } as any,
@@ -21,7 +21,8 @@ import { Readable } from 'node:stream';
import type { Request, RequestHandler, Response } from 'express';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
import { actorUid, type Actor } from '../../core/actor.js';
import type { DriverMethodLifecycleEvent } from '../../clients/event/types.js';
import { runWithContext } from '../../core/context.js';
import { PuterServer } from '../../server.js';
import { setupTestServer } from '../../testUtil.js';
@@ -33,7 +34,7 @@ import type { DriverController } from './DriverController.js';
// The DriverController under test is the same instance the live request
// pipeline uses, so its iface→driver registry is populated from real
// drivers (puter-kvstore, puter-apps, puter-subdomains, …). The HTTP
// handlers (`#handleCall`, `#handleListInterfaces`, `#handleXd`) are
// handlers (`#handleCall`, `#handleListInterfaces`) are
// private — we exercise the public lookup API (`resolve` / list / get
// default) which the handlers themselves delegate to.
@@ -116,7 +117,7 @@ describe('DriverController.resolve', () => {
});
});
// ── Route handlers (#handleCall, #handleListInterfaces, #handleXd) ─
// ── Route handlers (#handleCall, #handleListInterfaces) ─────────────
// The handlers are private class fields. We capture references to them by
// invoking `registerRoutes` with a fake router whose `post`/`get` save the
@@ -189,10 +190,7 @@ const makeRes = (): MockRes => {
return res;
};
const makeReq = (
body: Record<string, unknown> = {},
actor?: Actor,
): Request =>
const makeReq = (body: Record<string, unknown> = {}, actor?: Actor): Request =>
({
body,
actor,
@@ -338,11 +336,7 @@ describe('DriverController.#handleCall (via captured router)', () => {
actor,
);
await runWithContext({ actor }, () =>
routes['POST /call'](
req,
res as unknown as Response,
() => {},
),
routes['POST /call'](req, res as unknown as Response, () => {}),
);
const body = res.body as {
success: boolean;
@@ -374,11 +368,9 @@ describe('DriverController.#handleCall (via captured router)', () => {
// Register it through the controller's private map by re-running
// its #buildIfaceMap path: easier to just stash it into the
// existing iface map directly via TypeScript-defeating cast.
const internalDrivers = (
controller as unknown as {
['#drivers']: Map<string, Map<string, unknown>>;
}
);
const internalDrivers = controller as unknown as {
['#drivers']: Map<string, Map<string, unknown>>;
};
// Access the actual private slot via the well-known getter
// pattern doesn't work for `#`-private fields; instead, re-run
// registerRoutes after stashing on the bag — but that's already
@@ -413,13 +405,12 @@ describe('DriverController.#handleCall (via captured router)', () => {
>;
const original = kv.set;
kv.set = async function (...args: unknown[]) {
const { Context } = await import(
'../../core/context.js'
);
const { Context } = await import('../../core/context.js');
Context.set('driverMetadata', { providerUsed: 'kv-direct' });
return (
original as (...x: unknown[]) => Promise<unknown>
).apply(this, args);
return (original as (...x: unknown[]) => Promise<unknown>).apply(
this,
args,
);
};
try {
const res = makeRes();
@@ -432,11 +423,7 @@ describe('DriverController.#handleCall (via captured router)', () => {
actor,
);
await runWithContext({ actor }, () =>
routes['POST /call'](
req,
res as unknown as Response,
() => {},
),
routes['POST /call'](req, res as unknown as Response, () => {}),
);
const body = res.body as {
metadata?: Record<string, unknown>;
@@ -474,6 +461,129 @@ describe('DriverController.#handleCall (via captured router)', () => {
});
});
describe('DriverController driver-method lifecycle events', () => {
let routes: Captured;
beforeAll(() => {
routes = captureRoutes(controller);
});
const grantKvPerm = async (actor: Actor) => {
await server.stores.permission.setFlatUserPerm(
actor.user!.id!,
'service:puter-kvstore:ii:puter-kvstore',
{
permission: 'service:puter-kvstore:ii:puter-kvstore',
deleted: false,
issuer_user_id: actor.user!.id!,
} as never,
);
};
it('emits before then after around a successful driver method', async () => {
const actor = await makeUserActor();
await grantKvPerm(actor);
// Correlate on the actor: each test creates a fresh user, so this
// isolates our call from any other traffic on the shared event bus.
const who = actorUid(actor);
const before: DriverMethodLifecycleEvent[] = [];
const after: DriverMethodLifecycleEvent[] = [];
server.clients.event.on(
'driver.puter-kvstore.set.before',
(_k, data) => {
if (data.actorUid === who) before.push(data);
},
);
server.clients.event.on(
'driver.puter-kvstore.set.after',
(_k, data) => {
if (data.actorUid === who) after.push(data);
},
);
const key = `lc-${uuidv4()}`;
const req = makeReq(
{
interface: 'puter-kvstore',
method: 'set',
args: { key, value: 'v' },
},
actor,
);
await runWithContext({ actor }, () =>
routes['POST /call'](
req,
makeRes() as unknown as Response,
() => {},
),
);
expect(before).toHaveLength(1);
expect(before[0]).toMatchObject({
phase: 'before',
iface: 'puter-kvstore',
method: 'set',
actor,
actorUid: who,
});
expect(after).toHaveLength(1);
expect(after[0].phase).toBe('after');
expect(after[0].args).toMatchObject({ key, value: 'v' });
expect(typeof after[0].durationMs).toBe('number');
});
it('emits reject and answers 403 when a before listener vetoes', async () => {
const actor = await makeUserActor();
await grantKvPerm(actor);
// Scope the veto to a sentinel key so it doesn't affect other calls
// sharing this server's event bus.
const key = `veto-${uuidv4()}`;
const reject: DriverMethodLifecycleEvent[] = [];
server.clients.event.on(
'driver.puter-kvstore.set.before',
(_k, data) => {
if ((data.args as { key?: string })?.key === key) {
data.allow = false;
data.rejectReason = 'blocked in test';
}
},
);
server.clients.event.on(
'driver.puter-kvstore.set.reject',
(_k, data) => {
if (data.rejectReason === 'blocked in test') reject.push(data);
},
);
const req = makeReq(
{
interface: 'puter-kvstore',
method: 'set',
args: { key, value: 'v' },
},
actor,
);
await expect(
runWithContext({ actor }, () =>
routes['POST /call'](
req,
makeRes() as unknown as Response,
() => {},
),
),
).rejects.toMatchObject({ statusCode: 403 });
expect(reject).toHaveLength(1);
expect(reject[0]).toMatchObject({
phase: 'reject',
method: 'set',
args: { key, value: 'v' },
rejectReason: 'blocked in test',
});
});
});
describe('DriverController.#handleListInterfaces', () => {
let routes: Captured;
beforeAll(() => {
@@ -497,31 +607,11 @@ describe('DriverController.#handleListInterfaces', () => {
});
});
describe('DriverController.#handleXd', () => {
let routes: Captured;
beforeAll(() => {
routes = captureRoutes(controller);
});
it('serves the iframe-bridge HTML with text/html content-type', () => {
const res = makeRes();
routes['GET /xd'](
makeReq(),
res as unknown as Response,
() => {},
);
expect(res.contentType).toBe('text/html');
expect(res.sentBody).toMatch(/<!DOCTYPE html>/);
expect(res.sentBody).toMatch(/\/drivers\/call/);
});
});
describe('DriverController.registerRoutes', () => {
it('registers POST /call, GET /list-interfaces, and GET /xd', () => {
it('registers POST /call and GET /list-interfaces', () => {
const routes = captureRoutes(controller);
expect(routes['POST /call']).toBeInstanceOf(Function);
expect(routes['GET /list-interfaces']).toBeInstanceOf(Function);
expect(routes['GET /xd']).toBeInstanceOf(Function);
});
});
@@ -18,16 +18,15 @@
*/
import type { Request, Response } from 'express';
import { actorUid } from '../../core/actor.js';
import { Context } from '../../core/context.js';
import { Controller } from '../../core/http/decorators.js';
import { HttpError } from '../../core/http/HttpError.js';
import { HttpError, isHttpError } from '../../core/http/HttpError.js';
import {
acquireDriverConcurrent,
checkDriverRateLimit,
} from '../../core/http/middleware/rateLimit.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import type { PermissionService } from '../../services/permission/PermissionService.js';
import type { WithLifecycle } from '../../types';
import type { DriverMeta } from '../../drivers/meta.js';
import {
isDriverStreamResult,
@@ -35,100 +34,83 @@ import {
resolveDriverMethodConcurrent,
resolveDriverMethodRateLimit,
} from '../../drivers/meta.js';
import type { PermissionService } from '../../services/permission/PermissionService.js';
import { PermissionUtil } from '../../services/permission/permissionUtil.js';
import type { WithLifecycle } from '../../types';
import { PuterController } from '../types.js';
type DriverInstance = WithLifecycle & Record<string, unknown>;
// ── /drivers/xd payload ─────────────────────────────────────────────
//
// Self-contained HTML/JS shipped to the iframe consumer. Listens for
// postMessage events shaped as `{ id, interface, method, params }`,
// forwards to `/drivers/call`, and posts `{ id, result }` back to the
// originating window.
//
// Wire-shape note: the postMessage uses `params` (puter-js's historical
// name) but `/drivers/call` expects `args`; this bridge translates.
const extractUpstreamStatus = (e: {
status?: number;
statusCode?: number;
response?: { status?: number };
$metadata?: { httpStatusCode?: number };
message?: string;
}): number | undefined => {
const direct = e.status ?? e.statusCode;
if (typeof direct === 'number') return direct;
const fromResponse = e.response?.status;
if (typeof fromResponse === 'number') return fromResponse;
const fromAws = e.$metadata?.httpStatusCode;
if (typeof fromAws === 'number') return fromAws;
// Message sniff (e.g. "... failed with status 422 ...").
// Only trust if it's adjacent to a status-indicating word to
// avoid matching random 4xx/5xx-looking numbers in payloads.
const msg = e.message;
if (typeof msg === 'string') {
const m = msg.match(/\bstatus(?:\s+code)?\s*[:=]?\s*(4\d\d|5\d\d)\b/i);
if (m) return Number(m[1]);
}
return undefined;
};
const XD_SCRIPT = /* js */ `
(function () {
const call = async ({ interface_name, method_name, params }) => {
const response = await fetch('/drivers/call', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
interface: interface_name,
method: method_name,
args: params,
}),
});
return await response.json();
const translateProviderError = (err: unknown): unknown => {
if (isHttpError(err)) return err;
if (!err || typeof err !== 'object') return err;
const e = err as {
status?: number;
statusCode?: number;
response?: { status?: number };
$metadata?: { httpStatusCode?: number };
message?: string;
error?: { code?: string; type?: string; message?: string };
code?: string;
};
const status = extractUpstreamStatus(e);
if (typeof status !== 'number') return err;
const fcall = async ({ interface_name, method_name, params }) => {
const form = new FormData();
form.append('interface', interface_name);
form.append('method', method_name);
for (const k in params) {
form.append(k, params[k]);
}
const response = await fetch('/drivers/call', {
method: 'POST',
body: form,
const msg = e.error?.message ?? e.message ?? 'Upstream provider error';
const upstreamCode = e.error?.code ?? e.code;
const fields = { upstreamStatus: status, upstreamCode };
if (status === 429) {
return new HttpError(429, msg, {
legacyCode: 'upstream_rate_limited',
fields,
});
return await response.json();
};
}
if (status === 401 || status === 403) {
return new HttpError(500, msg, {
legacyCode: 'upstream_auth_failed',
fields,
});
}
if (status >= 500) {
return new HttpError(400, 'AI provider unavailable', {
legacyCode: 'upstream_provider_unavailable',
fields,
});
}
if (status >= 400) {
return new HttpError(400, msg, {
legacyCode: 'upstream_bad_request',
fields,
});
}
return err;
};
window.addEventListener('message', async (event) => {
const { id, interface: iface, method, params } = event.data || {};
let has_file = false;
for (const k in params) {
if (params[k] instanceof File) {
has_file = true;
break;
}
}
const result = has_file
? await fcall({ interface_name: iface, method_name: method, params })
: await call({ interface_name: iface, method_name: method, params });
if (event.source) {
event.source.postMessage({ id, result }, event.origin);
}
});
})();
`;
const XD_HTML = `<!DOCTYPE html>
<html>
<head>
<title>Puter Driver API</title>
<script>
document.addEventListener('DOMContentLoaded', function () {
${XD_SCRIPT}
});
</script>
</head>
<body></body>
</html>`;
// ── Controller ──────────────────────────────────────────────────────
/**
* Routes driver RPC calls through a unified HTTP surface.
*
* - `POST /drivers/call` — invoke `<iface>.<method>(args)` on the
* registered driver after a per-actor permission + rate-limit check.
* Stream-shaped results are piped directly; everything else is
* returned as JSON.
* - `GET /drivers/list-interfaces` — enumerate registered driver
* interfaces with their default + alternate implementations.
* - `GET /drivers/xd` — legacy iframe bridge; serves an HTML page that
* proxies `postMessage` RPCs to `/drivers/call` on the same origin.
*
* Holds an internal iface → driverName → instance map built at
* construction time from `this.drivers`. Extensions that register
* additional drivers end up in that bag before this controller is
* instantiated, so they show up here automatically.
*/
@Controller('/drivers')
export class DriverController extends PuterController {
/** iface → Map<driverName, driverInstance> */
@@ -146,7 +128,7 @@ export class DriverController extends PuterController {
this.#buildIfaceMap();
}
// ── Lookup API (used by tests / internals) ──────────────────────
// -- Lookup API (used by tests / internals) ----------------------
/** Resolve a driver by interface + optional name (default when omitted). */
resolve(interfaceName: string, driverName?: string): DriverInstance | null {
@@ -170,7 +152,7 @@ export class DriverController extends PuterController {
return this.#defaults.get(interfaceName);
}
// ── Route registration ──────────────────────────────────────────
// -- Route registration ------------------------------------------
registerRoutes(router: PuterRouter): void {
router.post(
@@ -183,14 +165,9 @@ export class DriverController extends PuterController {
{ subdomain: 'api', requireAuth: true },
this.#handleListInterfaces,
);
router.get(
'/xd',
{ subdomain: 'api', requireAuth: true },
this.#handleXd,
);
}
// ── Handlers ────────────────────────────────────────────────────
// -- Handlers ----------------------------------------------------
#handleCall = async (req: Request, res: Response): Promise<void> => {
const {
@@ -246,7 +223,16 @@ export class DriverController extends PuterController {
| PermissionService
| undefined;
if (permService) {
const permKey = `service:${resolvedDriverName}:ii:${ifaceName}`;
// Build via PermissionUtil.join so any `:` in a driver or
// interface name is escaped — raw interpolation would let a
// crafted name shift permission-segment boundaries and match
// a broader/narrower parent than intended in the scan logic.
const permKey = PermissionUtil.join(
'service',
String(resolvedDriverName),
'ii',
ifaceName,
);
const hasPermission = await permService.check(
req.actor,
permKey,
@@ -278,6 +264,19 @@ export class DriverController extends PuterController {
if (
!(await checkDriverRateLimit(req, ifaceName, method, rateLimitSpec))
) {
// De-dupe on (iface, method) so a hot loop across many users
// aggregates as occurrences on a single low-severity alarm
// instead of fanning out one per user.
this.clients.alarm.create(
`driver_rate_limit_hit:${ifaceName}:${method}`,
`Driver rate limit hit on ${ifaceName}:${method}`,
{
iface: ifaceName,
method,
userUuid: req.actor?.user?.uuid,
},
'info',
);
throw new HttpError(429, 'Too many requests.', {
legacyCode: 'too_many_requests',
});
@@ -300,6 +299,16 @@ export class DriverController extends PuterController {
concurrentSpec,
);
if (!handle.ok) {
this.clients.alarm.create(
`driver_concurrent_limit_hit:${ifaceName}:${method}`,
`Driver concurrency limit hit on ${ifaceName}:${method}`,
{
iface: ifaceName,
method,
userUuid: req.actor?.user?.uuid,
},
'info',
);
throw new HttpError(429, 'Too many concurrent requests.', {
legacyCode: 'too_many_requests',
});
@@ -326,11 +335,90 @@ export class DriverController extends PuterController {
// a stale value from a prior call.
Context.set('driverName', requestedDriver);
// Drivers read actor/context via the Context API — no drilled args.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const result = await (fn as (...x: unknown[]) => any).call(
driver,
// Per-method lifecycle events, scoped to `driver.<iface>.<method>`.
// Subscribers can listen on `driver.*`, `driver.<iface>.*`, or the
// exact key. `before` is emitted via `emitAndWait` so a listener may
// veto the call by setting `allow = false` (emits `reject`, throws
// 403); otherwise `after`/`error` carry the result/error + duration.
const actor = req.actor ? actorUid(req.actor) : undefined;
const resolved = String(resolvedDriverName);
const beforeEvent = {
phase: 'before' as const,
iface: ifaceName,
method,
driver: resolved,
actor: req.actor,
actorUid: actor,
args,
allow: true as boolean,
rejectReason: undefined as string | undefined,
};
await this.clients.event?.emitAndWait(
`driver.${ifaceName}.${method}.before`,
beforeEvent,
{},
);
if (beforeEvent.allow === false) {
this.clients.event?.emit(
`driver.${ifaceName}.${method}.reject`,
{
phase: 'reject',
iface: ifaceName,
method,
driver: resolved,
actor: req.actor,
actorUid: actor,
args,
rejectReason: beforeEvent.rejectReason,
},
{},
);
throw new HttpError(
403,
beforeEvent.rejectReason ??
`Blocked by policy: ${ifaceName}:${method}`,
{ legacyCode: 'forbidden' },
);
}
// Drivers read actor/context via the Context API — no drilled args.
const startedAt = Date.now();
let result;
try {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
result = await (fn as (...x: unknown[]) => any).call(driver, args);
} catch (e) {
this.clients.event?.emit(
`driver.${ifaceName}.${method}.error`,
{
phase: 'error',
iface: ifaceName,
method,
driver: resolved,
actor: req.actor,
actorUid: actor,
args,
error: e,
durationMs: Date.now() - startedAt,
},
{},
);
throw translateProviderError(e);
}
this.clients.event?.emit(
`driver.${ifaceName}.${method}.after`,
{
phase: 'after',
iface: ifaceName,
method,
driver: resolved,
actor: req.actor,
actorUid: actor,
args,
result,
durationMs: Date.now() - startedAt,
},
{},
);
if (isDriverStreamResult(result)) {
@@ -374,12 +462,7 @@ export class DriverController extends PuterController {
res.json(out);
};
#handleXd = (_req: Request, res: Response): void => {
res.type('text/html');
res.send(XD_HTML);
};
// ── Internals ───────────────────────────────────────────────────
// -- Internals ---------------------------------------------------
#buildIfaceMap(): void {
const bag = this.drivers as unknown as Record<string, DriverInstance>;
+605 -9
View File
@@ -18,6 +18,7 @@
*/
import type { Request, Response } from 'express';
import type { Readable } from 'node:stream';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { v4 as uuidv4 } from 'uuid';
import type { Actor } from '../../core/actor.js';
@@ -124,6 +125,14 @@ const makeRes = () => {
const withActor = async <T>(actor: Actor, fn: () => Promise<T>): Promise<T> =>
runWithContext({ actor }, fn);
const streamToString = async (stream: Readable): Promise<string> => {
const chunks: Buffer[] = [];
for await (const chunk of stream) {
chunks.push(Buffer.from(chunk as Buffer));
}
return Buffer.concat(chunks).toString('utf8');
};
// ── /startBatchWrite ────────────────────────────────────────────────
describe('FSController.startBatchWrites', () => {
@@ -434,6 +443,176 @@ describe('FSController.completeBatchWrites', () => {
expect(finalized?.wasOverwrite).toBe(true);
});
// Regression: a signed (direct-to-S3) upload could declare a tiny size
// and PUT far more — the presigned URL doesn't bound the body. The
// completion path must reconcile the recorded size against the object's
// true size, or storage accounting is permanently understated and the
// quota is bypassable.
it('reconciles the recorded size to the real uploaded bytes (ignores under-declared size)', async () => {
const { actor, userId } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/under-declared.bin`;
// Declare 1 byte.
const start = makeRes();
await withActor(actor, () =>
controller.startBatchWrites(
makeReq<SignedWriteRequest[]>({
body: [{ fileMetadata: { path: target, size: 1 } }],
actor,
}),
start.res,
),
);
const [started] = start.captured.body as SignedWriteResponse[];
// Actually upload 4096 bytes to the session's object key (simulating
// a client that PUTs more than it declared via the signed URL).
const realBytes = Buffer.alloc(4096, 0x41);
await server.stores.s3Object.uploadFromServer(
{
bucket: started!.bucket,
objectKey: started!.objectKey,
contentType: 'application/octet-stream',
body: realBytes,
contentLength: realBytes.byteLength,
},
started!.bucketRegion,
);
const complete = makeRes();
await withActor(actor, () =>
controller.completeBatchWrites(
makeReq<CompleteWriteRequest[]>({
body: [{ uploadId: started!.sessionId }],
actor,
}),
complete.res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(target);
expect(entry).not.toBeNull();
// Recorded size is the true 4096 bytes, not the declared 1.
expect(entry?.size).toBe(4096);
// And the user's accounted usage reflects the real bytes.
const allowance =
await server.stores.fsEntry.getUserStorageAllowance(userId);
expect(allowance.curr).toBeGreaterThanOrEqual(4096);
});
it('emits updated events with GUI metadata when overwriting via batch completion', async () => {
const { actor, userId } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/overwrite-event.js`;
const firstStart = makeRes();
await withActor(actor, () =>
controller.startBatchWrites(
makeReq<SignedWriteRequest[]>({
body: [{ fileMetadata: { path: target, size: 1 } }],
actor,
}),
firstStart.res,
),
);
const [firstResponse] = firstStart.captured
.body as SignedWriteResponse[];
await withActor(actor, () =>
controller.completeBatchWrites(
makeReq<CompleteWriteRequest[]>({
body: [{ uploadId: firstResponse!.sessionId }],
actor,
}),
makeRes().res,
),
);
const targetEntry = await server.stores.fsEntry.getEntryByPath(target);
expect(targetEntry).not.toBeNull();
await server.stores.subdomain.create({
userId,
subdomain: `workers.puter.${username}-worker`,
rootDirId: targetEntry!.id,
});
const secondStart = makeRes();
await withActor(actor, () =>
controller.startBatchWrites(
makeReq<SignedWriteRequest[]>({
body: [
{
fileMetadata: {
path: target,
size: 2,
overwrite: true,
},
},
],
actor,
}),
secondStart.res,
),
);
const [secondResponse] = secondStart.captured
.body as SignedWriteResponse[];
const emitSpy = vi.spyOn(server.clients.event, 'emit');
let updatedCall: (typeof emitSpy.mock.calls)[number] | undefined;
try {
await withActor(actor, () =>
controller.completeBatchWrites(
makeReq<CompleteWriteRequest[]>({
body: [
{
uploadId: secondResponse!.sessionId,
guiMetadata: {
operationId: 'op-123',
itemUploadId: 'item-456',
socketId: 'socket-789',
originalClientSocketId: 'socket-789',
},
},
],
actor,
}),
makeRes().res,
),
);
updatedCall = emitSpy.mock.calls.find(
([eventName]) => eventName === 'outer.gui.item.updated',
);
} finally {
emitSpy.mockRestore();
}
expect(updatedCall).toBeTruthy();
const payload = updatedCall?.[1] as {
user_id_list?: number[];
response?: Record<string, unknown>;
};
expect(payload.user_id_list).toEqual([userId]);
expect(payload.response).toMatchObject({
uid: expect.any(String),
uuid: expect.any(String),
id: expect.any(String),
path: target,
name: 'overwrite-event.js',
is_dir: false,
type: expect.stringMatching(/^application\/javascript/),
workers: [
expect.objectContaining({
subdomain: `workers.puter.${username}-worker`,
address: expect.stringContaining(`${username}-worker`),
}),
],
from_new_service: true,
operation_id: 'op-123',
item_upload_id: 'item-456',
socket_id: 'socket-789',
original_client_socket_id: 'socket-789',
});
});
it("rejects another user's session ids with a 4xx", async () => {
const a = await makeUser();
const b = await makeUser();
@@ -554,9 +733,9 @@ describe('FSController.statEntry', () => {
...makeReq({ body: { path: '/x' }, actor }),
actor: undefined,
} as unknown as Request;
await expect(
controller.statEntry(req, res),
).rejects.toMatchObject({ statusCode: 401 });
await expect(controller.statEntry(req, res)).rejects.toMatchObject({
statusCode: 401,
});
});
});
@@ -678,6 +857,85 @@ describe('FSController.searchEntries', () => {
expect(Array.isArray(results)).toBe(true);
expect(results.some((r) => r.name === needle)).toBe(true);
});
it('scopes app-under-user actors to their AppData root', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-search-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const needle = `appneedle-${Math.random().toString(36).slice(2, 8)}`;
// User-owned entry outside AppData — must NOT appear for the app.
await withActor(userActor, () =>
controller.mkdirEntry(
makeReq({
body: { path: `/${username}/Documents/${needle}` },
actor: userActor,
}),
makeRes().res,
),
);
// Entry under the app's own AppData — must appear.
await withActor(userActor, () =>
controller.mkdirEntry(
makeReq({
body: {
path: `/${username}/AppData/${appUid}/${needle}`,
create_missing_parents: true,
},
actor: userActor,
}),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(appActor, () =>
controller.searchEntries(
makeReq({ body: { query: needle }, actor: appActor }),
res,
),
);
const results = captured.body as Array<{ name: string; path: string }>;
expect(results.length).toBeGreaterThan(0);
for (const r of results) {
expect(
r.path === `/${username}/AppData/${appUid}` ||
r.path.startsWith(`/${username}/AppData/${appUid}/`),
).toBe(true);
}
expect(
results.some((r) => r.path === `/${username}/Documents/${needle}`),
).toBe(false);
});
it('returns nothing for an app actor when no AppData entries match', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-search-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const needle = `appneedle-${Math.random().toString(36).slice(2, 8)}`;
// Only seed outside AppData — the app must not be able to find it.
await withActor(userActor, () =>
controller.mkdirEntry(
makeReq({
body: { path: `/${username}/Documents/${needle}` },
actor: userActor,
}),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(appActor, () =>
controller.searchEntries(
makeReq({ body: { query: needle }, actor: appActor }),
res,
),
);
expect(captured.body).toEqual([]);
});
});
// ── /read (readEntry, validation paths) ─────────────────────────────
@@ -759,6 +1017,82 @@ describe('FSController.mkdirEntry', () => {
expect(body.isDir).toBe(true);
});
it('dedupes an existing directory when dedupe_name is true', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/hello`;
await withActor(actor, () =>
controller.mkdirEntry(
makeReq({
body: { path: target },
actor,
}),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.mkdirEntry(
makeReq({
body: { path: target, dedupe_name: true },
actor,
}),
res,
),
);
const body = captured.body as {
path: string;
name: string;
isDir: boolean;
};
expect(body.path).toBe(`/${username}/Documents/hello (1)`);
expect(body.name).toBe('hello (1)');
expect(body.isDir).toBe(true);
expect(
await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents/hello (1)`,
),
).toMatchObject({ isDir: true });
});
it('requires parent write when deduping an existing directory', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-mkdir-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const target = `/${username}/AppData/${appUid}`;
await withActor(userActor, () =>
controller.mkdirEntry(
makeReq({
body: { path: target },
actor: userActor,
}),
makeRes().res,
),
);
await expect(
withActor(appActor, () =>
controller.mkdirEntry(
makeReq({
body: { path: target, dedupe_name: true },
actor: appActor,
}),
makeRes().res,
),
),
).rejects.toMatchObject({ statusCode: 404 });
expect(
await server.stores.fsEntry.getEntryByPath(
`/${username}/AppData/${appUid} (1)`,
),
).toBeNull();
});
it('expands ~/ in the path to the user home', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
@@ -989,10 +1323,6 @@ describe('FSController.copyEntry', () => {
// ── /read (readEntry, full read) ────────────────────────────────────
describe('FSController.readEntry (file streaming)', () => {
// makeRes here adds a real Writable surface so that
// `pipeline(download.body, res)` inside the controller can pipe
// the in-memory S3 stream into the test's response and capture
// bytes for assertions.
const makeStreamingRes = () => {
const captured = {
statusCode: 200,
@@ -1000,7 +1330,8 @@ describe('FSController.readEntry (file streaming)', () => {
bodyChunks: [] as Buffer[],
};
// eslint-disable-next-line @typescript-eslint/no-require-imports
const { Writable } = require('node:stream') as typeof import('node:stream');
const { Writable } =
require('node:stream') as typeof import('node:stream');
const writable = new Writable({
write(chunk: Buffer, _enc, cb) {
captured.bodyChunks.push(chunk);
@@ -1060,7 +1391,9 @@ describe('FSController.readEntry (file streaming)', () => {
// Pipeline awaits the stream-end on success.
expect(captured.statusCode).toBe(200);
expect(captured.headers['Content-Type']).toMatch(/text\/plain/);
expect(captured.headers['Content-Length']).toBe(String(body.byteLength));
expect(captured.headers['Content-Length']).toBe(
String(body.byteLength),
);
expect(captured.headers['Content-Disposition']).toMatch(
/inline; filename=/,
);
@@ -1504,3 +1837,266 @@ describe('FSController.mkshortcutEntry', () => {
expect(body.isShortcut).toBe(true);
});
});
describe('FSController metadata.objectKey injection', () => {
const writeFile = async (
actor: Actor,
path: string,
content: string,
metadata?: Record<string, unknown>,
) => {
await withActor(actor, () =>
controller.write(
makeReq({
body: {
fileMetadata: {
path,
size: Buffer.byteLength(content),
contentType: 'text/plain',
overwrite: true,
...(metadata ? { metadata } : {}),
},
fileContent: content,
encoding: 'utf8',
},
actor,
}) as unknown as Request<
Record<string, never>,
null,
import('./requestTypes.js').WriteRequest
>,
makeRes().res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(path, {
skipCache: true,
});
if (!entry) throw new Error(`entry not found after write: ${path}`);
return entry;
};
it("does not stream another user's file when a client injects metadata.objectKey on write", async () => {
const victim = await makeUser();
const attacker = await makeUser();
const victimSecret = 'VICTIM-TOP-SECRET-PAYLOAD';
const attackerDecoy = 'attacker-own-decoy-bytes';
const victimEntry = await writeFile(
victim.actor,
`/${victim.actor.user!.username}/Documents/secret.txt`,
victimSecret,
);
const victimRead = await server.services.fs.readContent(victimEntry);
expect(await streamToString(victimRead.body)).toBe(victimSecret);
const attackerEntry = await writeFile(
attacker.actor,
`/${attacker.actor.user!.username}/Documents/loot.txt`,
attackerDecoy,
{ objectKey: victimEntry.uuid },
);
const persisted = attackerEntry.metadata
? (JSON.parse(attackerEntry.metadata) as Record<string, unknown>)
: {};
expect(persisted.objectKey).toBeUndefined();
const attackerRead =
await server.services.fs.readContent(attackerEntry);
const got = await streamToString(attackerRead.body);
expect(got).toBe(attackerDecoy);
expect(got).not.toBe(victimSecret);
});
it('read path ignores a divergent metadata.objectKey on an already-poisoned row', async () => {
const victim = await makeUser();
const attacker = await makeUser();
const victimSecret = 'VICTIM-SECRET-FOR-POISON-TEST';
const attackerDecoy = 'attacker-decoy-for-poison-test';
const victimEntry = await writeFile(
victim.actor,
`/${victim.actor.user!.username}/Documents/secret2.txt`,
victimSecret,
);
const attackerEntry = await writeFile(
attacker.actor,
`/${attacker.actor.user!.username}/Documents/loot2.txt`,
attackerDecoy,
);
await server.stores.fsEntry.updateEntry(attackerEntry.uuid, {
metadata: JSON.stringify({ objectKey: victimEntry.uuid }),
});
const poisoned = await server.stores.fsEntry.getEntryByPath(
attackerEntry.path,
{ skipCache: true },
);
if (!poisoned) throw new Error('poisoned entry not found');
expect(
(JSON.parse(poisoned.metadata!) as { objectKey: string }).objectKey,
).toBe(victimEntry.uuid);
const read = await server.services.fs.readContent(poisoned);
expect(await streamToString(read.body)).toBe(attackerDecoy);
});
it('scrubs objectKey from move newMetadata while preserving legit trash metadata', async () => {
const victim = await makeUser();
const attacker = await makeUser();
const victimSecret = 'VICTIM-SECRET-FOR-MOVE-TEST';
const attackerDecoy = 'attacker-decoy-for-move-test';
const username = attacker.actor.user!.username!;
const victimEntry = await writeFile(
victim.actor,
`/${victim.actor.user!.username}/Documents/secret3.txt`,
victimSecret,
);
const attackerEntry = await writeFile(
attacker.actor,
`/${username}/Documents/loot3.txt`,
attackerDecoy,
);
const documents = await server.stores.fsEntry.getEntryByPath(
`/${username}/Documents`,
{ skipCache: true },
);
if (!documents) throw new Error('Documents dir not found');
const moved = await withActor(attacker.actor, () =>
server.services.fs.move(attacker.userId, {
source: attackerEntry,
destinationParent: documents,
newName: 'loot3-moved.txt',
newMetadata: {
original_path: `/${username}/Documents/loot3.txt`,
trashed_ts: 1700000000,
objectKey: victimEntry.uuid,
},
}),
);
const persisted = JSON.parse(moved.metadata!) as Record<
string,
unknown
>;
expect(persisted.objectKey).toBeUndefined();
expect(persisted.original_path).toBe(
`/${username}/Documents/loot3.txt`,
);
expect(persisted.trashed_ts).toBe(1700000000);
const read = await server.services.fs.readContent(moved);
expect(await streamToString(read.body)).toBe(attackerDecoy);
});
});
// ── associatedAppId entitlement gate ────────────────────────────────
//
// `associatedAppId` is client-supplied write metadata that's echoed back in
// legacy FS responses. Binding a file to another tenant's private app would
// turn `/stat` into an app-row enumeration oracle, so the write path drops
// any association the actor isn't entitled to make.
describe('FSController associatedAppId entitlement gate', () => {
// Seed an app row with a direct insert. `create` treats is_private as a
// read-only column, and going through the store would prime the cache —
// a raw insert leaves nothing cached so the gate's getById reads the DB.
const makeApp = async (
ownerUserId: number,
opts: { is_private?: boolean } = {},
): Promise<{ id: number }> => {
const uid = `app-${uuidv4()}`;
await server.clients.db.write(
`INSERT INTO \`apps\` (\`uid\`, \`name\`, \`title\`, \`index_url\`, \`owner_user_id\`, \`is_private\`)
VALUES (?, ?, ?, ?, ?, ?)`,
[
uid,
uid,
'Gate App',
'https://gate-app.puter.site/',
ownerUserId,
opts.is_private ? 1 : 0,
],
);
const row = (
await server.clients.db.read(
'SELECT id FROM apps WHERE uid = ?',
[uid],
)
)[0] as { id: number };
return { id: row.id };
};
// Write a file via the signed-write flow with the given associatedAppId
// and return the committed entry's stored associatedAppId.
const writeWithAssociation = async (
actor: Actor,
path: string,
associatedAppId: number,
): Promise<number | null> => {
const startRes = makeRes();
await withActor(actor, () =>
controller.startBatchWrites(
makeReq<SignedWriteRequest[]>({
body: [{ fileMetadata: { path, size: 3, associatedAppId } }],
actor,
}),
startRes.res,
),
);
const [started] = startRes.captured.body as SignedWriteResponse[];
const completeRes = makeRes();
await withActor(actor, () =>
controller.completeBatchWrites(
makeReq<CompleteWriteRequest[]>({
body: [{ uploadId: started.sessionId }],
actor,
}),
completeRes.res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(path);
return entry?.associatedAppId ?? null;
};
it("drops an association to another tenant's private app", async () => {
const victim = await makeUser();
const attacker = await makeUser();
const victimApp = await makeApp(victim.userId, { is_private: true });
const stored = await writeWithAssociation(
attacker.actor,
`/${attacker.actor.user!.username}/Documents/probe.txt`,
victimApp.id,
);
expect(stored).toBeNull();
});
it('keeps an association to a public app the actor does not own', async () => {
const owner = await makeUser();
const other = await makeUser();
const publicApp = await makeApp(owner.userId, { is_private: false });
const stored = await writeWithAssociation(
other.actor,
`/${other.actor.user!.username}/Documents/public-assoc.txt`,
publicApp.id,
);
expect(stored).toBe(publicApp.id);
});
it('keeps an association to the actor’s own private app', async () => {
const owner = await makeUser();
const ownApp = await makeApp(owner.userId, { is_private: true });
const stored = await writeWithAssociation(
owner.actor,
`/${owner.actor.user!.username}/Documents/own-assoc.txt`,
ownApp.id,
);
expect(stored).toBe(ownApp.id);
});
});
+109 -88
View File
@@ -20,12 +20,13 @@
import Busboy from 'busboy';
import type { Request, Response } from 'express';
import { posix as pathPosix } from 'node:path';
import { assertNormalized } from '../../services/fs/resolveNode.js';
import { pipeline } from 'node:stream/promises';
import type { Actor } from '../../core/actor.js';
import { effectiveActorApp } from '../../core/actor.js';
import { Context } from '../../core/context.js';
import { HttpError } from '../../core/http/HttpError.js';
import { Controller, Get, Post } from '../../core/http/decorators.js';
import { assertNormalized } from '../../services/fs/resolveNode.js';
import type {
PreparedBatchWrite,
UploadedBatchWriteItem,
@@ -36,6 +37,7 @@ import {
runWithConcurrencyLimit,
runWithConcurrencyLimitSettled,
} from '../../util/concurrency.js';
import { applyInlineContentSecurity } from '../../util/inlineContentSecurity.js';
import { PuterController } from '../types.js';
import { FS_COSTS } from './costs.js';
import type {
@@ -58,6 +60,7 @@ import type {
ThumbnailUploadPrepareItem,
ThumbnailUploadPreparePayload,
} from './types.js';
import { toLegacyEntry } from './legacyFsHelpers.js';
class UploadProgressTracker implements UploadProgressTrackerLike {
total = 0;
progress = 0;
@@ -116,6 +119,8 @@ export class FSController extends PuterController {
requestBody.fileMetadata = await this.#resolveAssociatedAppMetadata(
requestBody.fileMetadata,
requestBody,
undefined,
userId,
);
await this.#assertWriteAccess(req, requestBody.fileMetadata, {
pathAlreadyNormalized: true,
@@ -180,6 +185,7 @@ export class FSController extends PuterController {
normalizedRequestBody.fileMetadata,
normalizedRequestBody,
appUidLookupCache,
userId,
);
return normalizedRequestBody;
}),
@@ -361,6 +367,8 @@ export class FSController extends PuterController {
requestBody.fileMetadata = await this.#resolveAssociatedAppMetadata(
requestBody.fileMetadata,
requestBody,
undefined,
userId,
);
await this.#assertWriteAccess(req, requestBody.fileMetadata, {
pathAlreadyNormalized: true,
@@ -421,7 +429,12 @@ export class FSController extends PuterController {
const busboy = Busboy({ headers: req.headers });
busboy.on('field', (fieldName, value, info) => {
if (info.fieldnameTruncated || info.valueTruncated) {
if (
(info as unknown as { filenameTruncated: string })
.filenameTruncated ||
info.nameTruncated ||
info.valueTruncated
) {
failParse(
new HttpError(
400,
@@ -489,6 +502,7 @@ export class FSController extends PuterController {
item.fileMetadata,
item,
appUidLookupCache,
userId,
),
})),
),
@@ -710,6 +724,7 @@ export class FSController extends PuterController {
normalizedRequestBody.fileMetadata,
normalizedRequestBody,
appUidLookupCache,
userId,
);
return normalizedRequestBody;
}),
@@ -810,7 +825,7 @@ export class FSController extends PuterController {
res.json(updatedResponses);
}
// ── Read-side routes ────────────────────────────────────────────────
// -- Read-side routes ------------------------------------------------
@Post('/stat', { subdomain: 'api', requireVerified: true })
async statEntry(req: Request, res: Response) {
@@ -910,7 +925,7 @@ export class FSController extends PuterController {
@Post('/search', { subdomain: 'api', requireVerified: true })
async searchEntries(req: Request, res: Response) {
this.#requireActor(req);
const actor = this.#requireActor(req);
const userId = this.#getActorUserId(req);
const body = this.#toObjectRecord(req.body);
const query =
@@ -929,6 +944,7 @@ export class FSController extends PuterController {
userId,
query,
limit ?? 200,
this.#appDataScopeForActor(actor),
);
res.json(results);
}
@@ -956,8 +972,13 @@ export class FSController extends PuterController {
range,
});
if (download.contentType)
if (download.contentType) {
res.setHeader('Content-Type', download.contentType);
// Stored type is uploader-controlled and served inline on the
// api origin — sandbox active-document types (HTML/SVG/XML) so
// embedded scripts can't run here. Inert types are untouched.
applyInlineContentSecurity(res, download.contentType);
}
if (download.contentLength !== null)
res.setHeader('Content-Length', String(download.contentLength));
if (download.contentRange)
@@ -1007,7 +1028,7 @@ export class FSController extends PuterController {
}
}
// ── Mutation routes ────────────────────────────────────────────────
// -- Mutation routes ------------------------------------------------
@Post('/mkdir', { subdomain: 'api', requireVerified: true })
async mkdirEntry(req: Request, res: Response) {
@@ -1031,13 +1052,15 @@ export class FSController extends PuterController {
legacyCode: 'bad_request',
});
const dedupeName =
this.#toBoolean(body.dedupe_name ?? body.dedupeName) ?? false;
await this.#assertCanCreate(actor, path);
if (dedupeName) await this.#assertCanDedupeCreate(actor, path);
const entry = await this.services.fs.mkdir(userId, {
path,
overwrite: this.#toBoolean(body.overwrite) ?? false,
dedupeName:
this.#toBoolean(body.dedupe_name ?? body.dedupeName) ?? false,
dedupeName,
createMissingParents:
this.#toBoolean(
body.create_missing_parents ??
@@ -1205,7 +1228,7 @@ export class FSController extends PuterController {
res.json(shortcut);
}
// ── Read-side helpers ───────────────────────────────────────────────
// -- Read-side helpers -----------------------------------------------
#requireActor(req: Request): Actor {
const actor = req.actor;
@@ -1302,6 +1325,17 @@ export class FSController extends PuterController {
await this.#assertAccess(actor, parentForCheck, 'write');
}
async #assertCanDedupeCreate(actor: Actor, targetPath: string) {
const existing = await this.stores.fsEntry.getEntryByPath(targetPath);
if (!existing) return;
const parent = pathPosix.dirname(targetPath);
await this.#assertAccess(
actor,
parent === '/' ? targetPath : parent,
'write',
);
}
async #assertAccess(
actor: Actor,
path: string,
@@ -1649,24 +1683,6 @@ export class FSController extends PuterController {
normalizedFileMetadata.multipartPartSize = multipartPartSize;
}
const bucket = this.#firstDefined(
metadataRecord.bucket,
fallbackRecord.bucket,
);
if (typeof bucket === 'string' && bucket.length > 0) {
normalizedFileMetadata.bucket = bucket;
}
const bucketRegion = this.#firstDefined(
metadataRecord.bucketRegion,
metadataRecord.bucket_region,
fallbackRecord.bucketRegion,
fallbackRecord.bucket_region,
);
if (typeof bucketRegion === 'string' && bucketRegion.length > 0) {
normalizedFileMetadata.bucketRegion = bucketRegion;
}
const associatedAppId = this.#toNumber(
this.#firstDefined(
metadataRecord.associatedAppId,
@@ -1686,6 +1702,7 @@ export class FSController extends PuterController {
fileMetadata: FSEntryWriteInput,
fallbackSource?: unknown,
appUidLookupCache?: Map<string, Promise<number | null>>,
actorUserId?: number,
): Promise<FSEntryWriteInput> {
const metadataRecord = this.#toObjectRecord(fileMetadata);
const fallbackRecord = this.#toObjectRecord(fallbackSource);
@@ -1699,10 +1716,11 @@ export class FSController extends PuterController {
),
);
if (associatedAppId !== undefined) {
return {
...fileMetadata,
return this.#withAssociatedAppId(
fileMetadata,
associatedAppId,
};
actorUserId,
);
}
const appUid = this.#firstDefined(
@@ -1736,9 +1754,46 @@ export class FSController extends PuterController {
if (resolvedAppId === null) {
return fileMetadata;
}
return this.#withAssociatedAppId(
fileMetadata,
resolvedAppId,
actorUserId,
);
}
/**
* Bind `associatedAppId` onto the write input only when the actor is
* entitled to reference that app. `associatedAppId` is client-supplied
* and never trusted for authz, but it's echoed back in legacy FS
* responses — so an attacker could plant another tenant's private app id
* to confirm the row exists (an enumeration oracle) and harvest its
* metadata. Allow binding to public apps (their existence isn't secret)
* or to apps the actor owns; drop the association otherwise so the file
* simply carries no associated app.
*
* `#resolveWriteFileMetadata` has already copied the raw client value onto
* `fileMetadata`, so a dropped association must be actively stripped — not
* merely left unset.
*/
async #withAssociatedAppId(
fileMetadata: FSEntryWriteInput,
appId: number,
actorUserId?: number,
): Promise<FSEntryWriteInput> {
const app = await this.stores.app.getById(appId);
const isPrivate =
!!app && (Boolean(app.is_private) || Boolean(app.protected));
const isOwner =
actorUserId !== undefined &&
this.#toNumber(app?.owner_user_id) === actorUserId;
if (!app || (isPrivate && !isOwner)) {
const { associatedAppId: _dropped, ...rest } =
fileMetadata as unknown as Record<string, unknown>;
return rest as unknown as FSEntryWriteInput;
}
return {
...fileMetadata,
associatedAppId: resolvedAppId,
associatedAppId: appId,
};
}
@@ -2034,56 +2089,7 @@ export class FSController extends PuterController {
}
async #toGuiFsEntry(entry: FSEntry): Promise<Record<string, unknown>> {
const dirpath = pathPosix.dirname(entry.path);
const extension = pathPosix.extname(entry.name).slice(1).toLowerCase();
const response = {
id: entry.uuid,
uid: entry.uuid,
uuid: entry.uuid,
parent_id: entry.parentUid,
parent_uid: entry.parentUid,
path: entry.path,
dirname: dirpath,
dirpath,
name: entry.name,
is_dir: entry.isDir,
is_shortcut: entry.isShortcut ? 1 : 0,
shortcut_to: entry.shortcutTo,
type: entry.isDir ? 'folder' : extension,
writable: true,
is_public: entry.isPublic,
thumbnail: entry.thumbnail,
immutable: entry.immutable,
metadata: entry.metadata,
modified: entry.modified,
created: entry.created,
accessed: entry.accessed,
size: entry.size,
};
if (
typeof response.thumbnail === 'string' &&
response.thumbnail.length > 0
) {
const thumbnailEntry = {
uuid: entry.uuid,
thumbnail: response.thumbnail,
};
// emitAndWait — listener rewrites s3:// / legacy URLs into
// time-limited signed URLs on the payload object.
await this.clients.event.emitAndWait(
'thumbnail.read',
thumbnailEntry,
{},
);
response.thumbnail =
typeof thumbnailEntry.thumbnail === 'string' &&
thumbnailEntry.thumbnail.length > 0
? thumbnailEntry.thumbnail
: null;
}
return response;
return toLegacyEntry(this.clients.event, entry);
}
async #emitGuiWriteEvent(
@@ -2093,7 +2099,7 @@ export class FSController extends PuterController {
): Promise<void> {
const response = {
...(await this.#toGuiFsEntry(fsEntry)),
...this.#toEventGuiMetadata(guiMetadata, false),
...this.#toEventGuiMetadata(guiMetadata),
from_new_service: true,
};
await this.clients.event.emit(
@@ -2144,6 +2150,20 @@ export class FSController extends PuterController {
return pathParts.length >= 2 && pathParts[1] === 'AppData';
}
// If `actor` is effectively scoped to an app (directly or through an
// access-token issuer chain), return that app's AppData root under the
// actor's user. Returns undefined for pure user actors. The store anchors
// search results to this path so app actors can't see entries outside
// their AppData via `/fs/search`.
#appDataScopeForActor(actor: Actor): string | undefined {
const app = effectiveActorApp(actor);
if (!app) return undefined;
const username = actor.user?.username;
if (typeof username !== 'string' || username.length === 0)
return undefined;
return `/${username}/AppData/${app.uid}`;
}
#estimateDataUrlSize(dataUrl: string): number {
const commaIndex = dataUrl.indexOf(',');
const base64 =
@@ -2237,7 +2257,7 @@ export class FSController extends PuterController {
return null;
}
return { index, contentType, size };
return { index, contentType, size } as ThumbnailUploadPrepareItem;
}
async #attachSignedThumbnailUploadTargets(
@@ -2257,11 +2277,12 @@ export class FSController extends PuterController {
const payload: ThumbnailUploadPreparePayload = {
items: prepareItems.map(
(item): ThumbnailUploadPrepareItem => ({
index: item.index,
contentType: item.contentType,
...(item.size !== undefined ? { size: item.size } : {}),
}),
(item): ThumbnailUploadPrepareItem =>
({
index: item.index,
contentType: item.contentType,
...(item.size !== undefined ? { size: item.size } : {}),
}) as ThumbnailUploadPrepareItem,
),
};
// emitAndWait — listeners populate `uploadUrl` / `thumbnailUrl` on
@@ -206,6 +206,78 @@ describe('LegacyFSController.mkdir', () => {
expect(fetched?.isDir).toBe(true);
});
it('dedupes an existing directory when dedupe_name is true', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const parent = `/${username}/Documents`;
await withActor(actor, () =>
controller.mkdir(
makeReq({
body: { parent, path: 'hello' },
actor,
}),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.mkdir(
makeReq({
body: { parent, path: 'hello', dedupe_name: true },
actor,
}),
res,
),
);
const body = captured.body as Record<string, unknown>;
expect(body).toMatchObject({
path: `${parent}/hello (1)`,
name: 'hello (1)',
is_dir: true,
});
expect(
await server.stores.fsEntry.getEntryByPath(`${parent}/hello (1)`),
).toMatchObject({ isDir: true });
});
it('requires parent write when deduping an existing directory', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-legacy-mkdir-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const parent = `/${username}/AppData`;
await withActor(userActor, () =>
controller.mkdir(
makeReq({
body: { parent, path: appUid },
actor: userActor,
}),
makeRes().res,
),
);
await expect(
withActor(appActor, () =>
controller.mkdir(
makeReq({
body: { parent, path: appUid, dedupe_name: true },
actor: appActor,
}),
makeRes().res,
),
),
).rejects.toMatchObject({ statusCode: 404 });
expect(
await server.stores.fsEntry.getEntryByPath(
`${parent}/${appUid} (1)`,
),
).toBeNull();
});
it("rejects writing into another user's home with a 4xx", async () => {
const a = await makeUser();
const b = await makeUser();
@@ -693,6 +765,121 @@ describe('LegacyFSController.copy', () => {
`/${username}/Pictures/renamed-copy`,
);
});
it('copies an empty file (no backing S3 object) without erroring', async () => {
// Empty files created via /touch have size 0 and no S3 object —
// bucket is null. Copy must clone them as empty-file entries rather
// than issuing a CopyObject, which would throw S3 NoSuchKey.
const { actor } = await makeUser();
const username = actor.user!.username!;
const src = `/${username}/Documents/empty.txt`;
await withActor(actor, () =>
controller.touch(
makeReq({ body: { path: src }, actor }),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.copy(
makeReq({
body: { source: src, destination: `/${username}/Pictures` },
actor,
}),
res,
),
);
const body = captured.body as Array<{ copied: { path: string } }>;
expect(body[0].copied.path).toBe(`/${username}/Pictures/empty.txt`);
const copied = await server.stores.fsEntry.getEntryByPath(
`/${username}/Pictures/empty.txt`,
);
expect(copied).not.toBeNull();
expect(copied!.size).toBe(0);
// Original is untouched.
expect(await server.stores.fsEntry.getEntryByPath(src)).not.toBeNull();
});
it('copying a ghost file (S3 object missing) 404s and cleans up the orphan', async () => {
// A real file whose backing S3 object has vanished keeps a non-null
// bucket, so it isn't an empty file. CopyObject would throw S3
// NoSuchKey; copy must surface a clean 404 and remove the orphan row
// rather than bubbling a 500. Mirrors readContent's ghost handling.
const { actor, userId } = await makeUser();
const username = actor.user!.username!;
const src = `/${username}/Documents/ghost.txt`;
const content = Buffer.from('i will be deleted from s3');
await server.services.fs.write(userId, {
fileMetadata: {
path: src,
size: content.byteLength,
contentType: 'text/plain',
},
fileContent: content,
});
// Delete the backing S3 object directly, leaving the DB row behind.
const entry = (await server.stores.fsEntry.getEntryByPath(src))!;
await server.stores.s3Object.deleteObject(
server.stores.s3Object.resolveBucket(entry.bucket),
entry.uuid,
server.stores.s3Object.resolveRegion(entry.bucketRegion),
);
const { res } = makeRes();
await expect(
withActor(actor, () =>
controller.copy(
makeReq({
body: {
source: src,
destination: `/${username}/Pictures`,
},
actor,
}),
res,
),
),
).rejects.toMatchObject({ statusCode: 404 });
// The ghost handler removed the orphaned source row.
expect(await server.stores.fsEntry.getEntryByPath(src)).toBeNull();
// No partial copy was left behind.
expect(
await server.stores.fsEntry.getEntryByPath(
`/${username}/Pictures/ghost.txt`,
),
).toBeNull();
});
it('reads an empty file as empty content without deleting it', async () => {
// Reading an empty file (no S3 object) must not throw NoSuchKey nor
// trip the ghost-file cleanup, which would delete the entry.
const { actor } = await makeUser();
const username = actor.user!.username!;
const src = `/${username}/Documents/readme-empty.txt`;
await withActor(actor, () =>
controller.touch(
makeReq({ body: { path: src }, actor }),
makeRes().res,
),
);
const entry = (await server.stores.fsEntry.getEntryByPath(src))!;
const download = await server.services.fs.readContent(entry);
const chunks: Buffer[] = [];
for await (const chunk of download.body) {
chunks.push(Buffer.from(chunk as Uint8Array));
}
expect(Buffer.concat(chunks).byteLength).toBe(0);
expect(download.contentLength).toBe(0);
// The entry must still exist — the ghost handler must NOT have run.
expect(
await server.stores.fsEntry.getEntryByPath(src),
).not.toBeNull();
});
});
// ── move ────────────────────────────────────────────────────────────
@@ -813,6 +1000,57 @@ describe('LegacyFSController.search', () => {
expect(Array.isArray(results)).toBe(true);
expect(results.some((r) => r.name === needle)).toBe(true);
});
it('scopes app-under-user actors to their AppData root', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appUid = `app-legacy-search-${uuidv4()}`;
const appActor: Actor = { ...userActor, app: { uid: appUid } };
const needle = `appneedle-${Math.random().toString(36).slice(2, 8)}`;
await withActor(userActor, () =>
controller.mkdir(
makeReq({
body: { path: `/${username}/Documents/${needle}` },
actor: userActor,
}),
makeRes().res,
),
);
await withActor(userActor, () =>
controller.mkdir(
makeReq({
body: {
path: `/${username}/AppData/${appUid}/${needle}`,
create_missing_parents: true,
},
actor: userActor,
}),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(appActor, () =>
controller.search(
makeReq({ body: { query: needle }, actor: appActor }),
res,
),
);
const results = captured.body as Array<{ path: string }>;
expect(results.length).toBeGreaterThan(0);
for (const r of results) {
expect(
r.path === `/${username}/AppData/${appUid}` ||
r.path.startsWith(`/${username}/AppData/${appUid}/`),
).toBe(true);
}
expect(
results.some(
(r) => r.path === `/${username}/Documents/${needle}`,
),
).toBe(false);
});
});
// ── read (validation paths only) ────────────────────────────────────
@@ -959,6 +1197,76 @@ describe('LegacyFSController.sign', () => {
),
).rejects.toMatchObject({ statusCode: 404 });
});
it('refuses an app actor signing for a different app (403)', async () => {
// An app-under-user actor may only mint a token for its own app;
// requesting a different app's UID is rejected.
const { actor: userActor } = await makeUser();
const targetApp = await (
server.stores.app.create as unknown as (
fields: Record<string, unknown>,
opts: { ownerUserId: number },
) => Promise<{ uid: string; id: number }>
)(
{
name: `victim-${uuidv4()}`,
title: 'Victim app',
index_url: 'https://example.test/victim.html',
},
{ ownerUserId: userActor.user!.id! },
);
const attackerActor: Actor = {
...userActor,
app: { uid: `attacker-${uuidv4()}` },
};
const { res } = makeRes();
await expect(
withActor(attackerActor, () =>
controller.sign(
makeReq({
body: {
items: [{}],
app_uid: targetApp.uid,
},
actor: attackerActor,
}),
res,
),
),
).rejects.toMatchObject({ statusCode: 403, legacyCode: 'forbidden' });
});
it('lets an app actor sign for its own app', async () => {
const { actor: userActor } = await makeUser();
const ownApp = await (
server.stores.app.create as unknown as (
fields: Record<string, unknown>,
opts: { ownerUserId: number },
) => Promise<{ uid: string; id: number }>
)(
{
name: `self-${uuidv4()}`,
title: 'Self app',
index_url: 'https://example.test/self.html',
},
{ ownerUserId: userActor.user!.id! },
);
const appActor: Actor = { ...userActor, app: { uid: ownApp.uid } };
const { res, captured } = makeRes();
await withActor(appActor, () =>
controller.sign(
makeReq({
body: { items: [{}], app_uid: ownApp.uid },
actor: appActor,
}),
res,
),
);
const body = captured.body as { token?: string };
expect(typeof body.token).toBe('string');
});
});
// ── writeFile (validation paths) ────────────────────────────────────
@@ -1038,6 +1346,85 @@ describe('LegacyFSController.openItem', () => {
});
});
// ── openItem: write_url stripping for read-only callers ─────────────
//
// Mirrors `/sign` and `/readdir`, which already strip `write_url` when the
// caller only proved read. Without these gates, /open_item handed out a
// valid write signature to read-only sharees — `/writeFile`'s own ACL
// re-check would still reject the write, but the leak shape (a signed
// write URL escaping the access boundary) is the same one those other
// endpoints already defend against.
describe('LegacyFSController.openItem (write_url stripping)', () => {
beforeAll(() => {
(
controller as unknown as { config: { api_base_url?: string } }
).config.api_base_url = 'http://api.test.local';
});
it('returns write_url for the owner (who has write)', async () => {
const { actor } = await makeUser();
const target = `/${actor.user!.username}/Documents/owned.txt`;
await withActor(actor, () =>
controller.touch(
makeReq({ body: { path: target }, actor }),
makeRes().res,
),
);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.openItem(
makeReq({ body: { path: target }, actor }),
res,
),
);
const body = captured.body as {
signature: { read_url?: string; write_url?: string };
};
expect(body.signature.read_url).toBeDefined();
expect(body.signature.write_url).toBeDefined();
expect(body.signature.write_url).toContain('writeFile');
});
it('strips write_url for a read-only sharee (the fix)', async () => {
const victim = await makeUser();
const attacker = await makeUser();
const target = `/${victim.actor.user!.username}/Documents/shared-ro.txt`;
await withActor(victim.actor, () =>
controller.touch(
makeReq({ body: { path: target }, actor: victim.actor }),
makeRes().res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(target);
await server.services.permission.grantUserUserPermission(
victim.actor,
attacker.actor.user!.username!,
`fs:${entry!.uuid}:read`,
{},
);
const { res, captured } = makeRes();
await withActor(attacker.actor, () =>
controller.openItem(
makeReq({
body: { uid: entry!.uuid },
actor: attacker.actor,
}),
res,
),
);
const body = captured.body as {
signature: { read_url?: string; write_url?: string };
};
// Sharee still gets read_url (they have read).
expect(body.signature.read_url).toBeDefined();
// …but write_url is stripped (they don't have write).
expect(body.signature.write_url).toBeUndefined();
});
});
// ── requestAppRootDir ───────────────────────────────────────────────
describe('LegacyFSController.requestAppRootDir', () => {
@@ -2032,3 +2419,197 @@ describe('LegacyFSController.readdir extras', () => {
).rejects.toMatchObject({ statusCode: 400 });
});
});
// ── suggestApps (ACL-gated) ─────────────────────────────────────────
describe('LegacyFSController.suggestApps', () => {
it('refuses to look up entries an app actor cannot see', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appActor: Actor = {
...userActor,
app: { uid: `app-suggest-${uuidv4()}` },
};
const target = `/${username}/Documents/probe.txt`;
await withActor(userActor, () =>
controller.touch(
makeReq({
body: { path: target, set_modified_to_now: true },
actor: userActor,
}),
makeRes().res,
),
);
const spy = vi.spyOn(
server.services.suggestedApps,
'getSuggestedApps',
);
try {
const { res } = makeRes();
await withActor(appActor, () =>
controller.suggestApps(
makeReq({ body: { path: target }, actor: appActor }),
res,
),
);
expect(spy).toHaveBeenLastCalledWith({
name: undefined,
path: undefined,
});
} finally {
spy.mockRestore();
}
});
it('forwards entry name/path to the suggester for the owning user', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/probe2.txt`;
await withActor(actor, () =>
controller.touch(
makeReq({
body: { path: target, set_modified_to_now: true },
actor,
}),
makeRes().res,
),
);
const spy = vi.spyOn(
server.services.suggestedApps,
'getSuggestedApps',
);
try {
const { res } = makeRes();
await withActor(actor, () =>
controller.suggestApps(
makeReq({ body: { path: target }, actor }),
res,
),
);
expect(spy).toHaveBeenLastCalledWith({
name: 'probe2.txt',
path: target,
});
} finally {
spy.mockRestore();
}
});
});
// ── readdirSubdomains ───────────────────────────────────────────────
describe('LegacyFSController.readdirSubdomains', () => {
it('returns an empty array for app-under-user actors', async () => {
const { actor: userActor, userId } = await makeUser();
const appActor: Actor = {
...userActor,
app: { uid: `app-subd-${uuidv4()}` },
};
await server.clients.db.write(
'INSERT INTO `subdomains` (`uuid`, `subdomain`, `user_id`) VALUES (?, ?, ?)',
[uuidv4(), `sd-${uuidv4().slice(0, 8)}`, userId],
);
const { res, captured } = makeRes();
await withActor(appActor, () =>
controller.readdirSubdomains(
makeReq({ body: {}, actor: appActor }),
res,
),
);
expect(captured.body).toEqual([]);
});
it("returns the user actor's own subdomain rows", async () => {
const { actor, userId } = await makeUser();
const sd = `sd-${uuidv4().slice(0, 8)}`;
await server.clients.db.write(
'INSERT INTO `subdomains` (`uuid`, `subdomain`, `user_id`) VALUES (?, ?, ?)',
[uuidv4(), sd, userId],
);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.readdirSubdomains(
makeReq({ body: {}, actor }),
res,
),
);
const rows = captured.body as Array<{ subdomain: string }>;
expect(rows.some((r) => r.subdomain === sd)).toBe(true);
});
});
// ── updateFsentryThumbnail ──────────────────────────────────────────
describe('LegacyFSController.updateFsentryThumbnail', () => {
it('rejects an app actor probing entries outside AppData with 404', async () => {
const { actor: userActor } = await makeUser();
const username = userActor.user!.username!;
const appActor: Actor = {
...userActor,
app: { uid: `app-thumb-${uuidv4()}` },
};
const target = `/${username}/Documents/thumbme.txt`;
await withActor(userActor, () =>
controller.touch(
makeReq({
body: { path: target, set_modified_to_now: true },
actor: userActor,
}),
makeRes().res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(target);
const { res } = makeRes();
await expect(
withActor(appActor, () =>
controller.updateFsentryThumbnail(
makeReq({
body: {
uid: entry!.uuid,
thumbnail: 'data:image/png;base64,AA==',
},
actor: appActor,
}),
res,
),
),
).rejects.toMatchObject({ statusCode: 404 });
});
it('updates the thumbnail for the owning user actor', async () => {
const { actor } = await makeUser();
const username = actor.user!.username!;
const target = `/${username}/Documents/thumbme2.txt`;
await withActor(actor, () =>
controller.touch(
makeReq({
body: { path: target, set_modified_to_now: true },
actor,
}),
makeRes().res,
),
);
const entry = await server.stores.fsEntry.getEntryByPath(target);
const { res, captured } = makeRes();
await withActor(actor, () =>
controller.updateFsentryThumbnail(
makeReq({
body: {
uid: entry!.uuid,
thumbnail: 'data:image/png;base64,AA==',
},
actor,
}),
res,
),
);
const body = captured.body as { thumbnail: string };
expect(typeof body.thumbnail).toBe('string');
});
});
+336 -254
View File
@@ -22,13 +22,14 @@ import type { Request, RequestHandler, Response } from 'express';
import { contentType as contentTypeFromMime } from 'mime-types';
import { posix as pathPosix } from 'node:path';
import type { Actor } from '../../core/actor.js';
import { isAccessTokenActor } from '../../core/actor.js';
import { effectiveActorApp, isAccessTokenActor } from '../../core/actor.js';
import { Context } from '../../core/context.js';
import { HttpError } from '../../core/http/HttpError.js';
import type { PuterRouter } from '../../core/http/PuterRouter.js';
import type { ACLService } from '../../services/acl/ACLService.js';
import type { SignedFile } from '../../util/fileSigning.js';
import { verifySignature } from '../../util/fileSigning.js';
import { applyInlineContentSecurity } from '../../util/inlineContentSecurity.js';
import { PuterController } from '../types.js';
import { FS_COSTS } from './costs.js';
import {
@@ -45,18 +46,20 @@ import {
} from './legacyFsHelpers.js';
import { RouteOptions } from '../../core/http/index.js';
/**
* Legacy FS routes, implemented as thin shims over `FSService`.
*
* Each shim parses the request shape (FSNodeParam-style `{ path, uid, id }`
* or `{ parent, name }`), invokes the service method, and returns the
* snake_case response clients expect.
*/
type RouterCache = Map<string, RequestHandler | null>;
const additionalRoutePaths: Record<string, string> = {};
// Legacy `/batch` multipart upload caps. Each file is buffered fully into
// memory before any quota / storage check runs, so without these limits an
// authenticated caller could grow the process heap proportional to whatever
// they sent. Streaming uploads go through `/writeFile`; this path is for
// pre-v2 clients only.
const BATCH_MAX_FILE_SIZE = 100 * 1024 * 1024; // 100 MiB per file
const BATCH_MAX_FILES = 64;
const BATCH_MAX_PARTS = 256;
const BATCH_MAX_FIELD_SIZE = 1 * 1024 * 1024; // 1 MiB per operation/fileinfo JSON
async function loadAdditionalRouter(
key: string,
): Promise<RequestHandler | null> {
@@ -74,7 +77,7 @@ async function loadAdditionalRouter(
}
}
// ── Controller ──────────────────────────────────────────────────────
// -- Controller ------------------------------------------------------
export class LegacyFSController extends PuterController {
#additionalCache: RouterCache = new Map();
@@ -136,6 +139,11 @@ export class LegacyFSController extends PuterController {
{
subdomain: ['api', ''],
requireUserActor: true,
// The user's own credential may download their files; apps and
// scoped tokens stay blocked. antiCsrf still protects the
// cookie-authed GUI path — a bearer-token PAT is exempt below
// (CSRF can't forge a header-credentialed request).
allowFullAccessToken: true,
requireVerified: true,
antiCsrf: true,
},
@@ -150,8 +158,7 @@ export class LegacyFSController extends PuterController {
router.get('/get-launch-apps', apiOptions, async (req, res) => {
const recommendedSvc = this.services.recommendedApps as unknown as
| { getRecommendedApps?: () => Promise<unknown[]> }
| undefined;
{ getRecommendedApps?: () => Promise<unknown[]> } | undefined;
const recommended = recommendedSvc?.getRecommendedApps
? await recommendedSvc.getRecommendedApps()
: [];
@@ -186,6 +193,11 @@ export class LegacyFSController extends PuterController {
godmode: Boolean(app.godmode),
maximize_on_start: Boolean(app.maximize_on_start),
index_url: app.index_url,
// An app with no owner isn't owned by a Puter user —
// it's an "external" (origin-bootstrapped) app.
external:
app.owner_user_id == null ||
app.owner_user_id === '',
});
}
}
@@ -195,35 +207,7 @@ export class LegacyFSController extends PuterController {
res.json({ recommended, recent });
});
router.post('/suggest_apps', apiOptions, async (req, res) => {
const suggestSvc = this.services.suggestedApps;
if (!suggestSvc?.getSuggestedApps) {
res.json([]);
return;
}
const body = req.body ?? {};
// Client sends { uid } or { path } identifying a file entry.
// Resolve the entry to get its name/path for extension detection.
let entryName: string | undefined;
let entryPath: string | undefined;
if (body.uid || body.path) {
try {
const entry = await resolveV1Selector(
this.stores.fsEntry,
body,
);
entryName = entry?.name;
entryPath = entry?.path;
} catch {
// If we can't resolve, fall back to empty
}
}
const suggestions = await suggestSvc.getSuggestedApps({
name: entryName,
path: entryPath,
});
res.json(suggestions);
});
router.post('/suggest_apps', apiOptions, this.suggestApps);
// puter-js polls this to decide whether to purge its in-memory FS
// cache. SocketService bumps a per-user Redis key on every
@@ -251,70 +235,11 @@ export class LegacyFSController extends PuterController {
},
);
// ── POST /readdir-subdomains ────────────────────────────────
router.post(
'/readdir-subdomains',
apiOptions,
async (req: Request, res: Response) => {
const userId = req.actor?.user?.id;
if (!userId)
throw new HttpError(401, 'Authentication required', {
legacyCode: 'unauthorized',
});
const rows = await this.clients.db.read(
'SELECT `subdomain`, `root_dir_id`, `uuid`, `ts` FROM `subdomains` WHERE `user_id` = ?',
[userId],
);
res.json(rows);
},
);
// ── POST /update-fsentry-thumbnail ──────────────────────────
router.post('/readdir-subdomains', apiOptions, this.readdirSubdomains);
router.post(
'/update-fsentry-thumbnail',
apiOptions,
async (req: Request, res: Response) => {
const userId = req.actor?.user?.id;
if (!userId)
throw new HttpError(401, 'Authentication required', {
legacyCode: 'unauthorized',
});
const { uid, thumbnail } = (req.body ?? {}) as {
uid?: string;
thumbnail?: string;
};
if (!uid)
throw new HttpError(400, 'Missing `uid`', {
legacyCode: 'bad_request',
});
if (!thumbnail)
throw new HttpError(400, 'Missing `thumbnail`', {
legacyCode: 'bad_request',
});
const entry = await this.stores.fsEntry.getEntryByUuid(uid);
if (!entry || entry.userId !== userId)
throw new HttpError(403, 'Access denied', {
legacyCode: 'forbidden',
});
// Emit thumbnail.created so the thumbnails extension can S3-upload.
// emitAndWait is required — the extension rewrites `event.url`
// from a data URL to an `s3://` pointer, and the DB write below
// needs to see that rewrite.
const event = { url: thumbnail };
await this.clients.event.emitAndWait(
'thumbnail.created',
event,
{},
);
await this.clients.db.write(
'UPDATE `fsentries` SET `thumbnail` = ? WHERE `uuid` = ?',
[event.url, uid],
);
res.json({ thumbnail: event.url });
},
this.updateFsentryThumbnail,
);
for (const key of Object.keys(additionalRoutePaths)) {
@@ -328,7 +253,7 @@ export class LegacyFSController extends PuterController {
}
}
// ── Route implementations ───────────────────────────────────────────
// -- Route implementations -------------------------------------------
//
// Handlers are public arrow class fields so they auto-bind `this` and can
// be passed directly to `router.post(...)` without `.bind(this)`. Express
@@ -504,17 +429,33 @@ export class LegacyFSController extends PuterController {
const normalizedTarget = targetPath.startsWith('/')
? targetPath
: `/${targetPath}`;
const dedupeName =
getBoolean(body, 'dedupe_name', 'change_name') ?? false;
await assertCanCreate(
this.services.acl,
this.services.fs,
actor,
normalizedTarget,
);
if (dedupeName) {
const existing =
await this.stores.fsEntry.getEntryByPath(normalizedTarget);
if (existing) {
const parent = pathPosix.dirname(normalizedTarget);
await assertAccess(
this.services.acl,
this.services.fs,
actor,
parent === '/' ? normalizedTarget : parent,
'write',
);
}
}
const entry = await this.services.fs.mkdir(userId, {
path: targetPath,
overwrite: getBoolean(body, 'overwrite') ?? false,
dedupeName: getBoolean(body, 'dedupe_name', 'change_name') ?? false,
dedupeName,
createMissingParents:
getBoolean(
body,
@@ -618,9 +559,7 @@ export class LegacyFSController extends PuterController {
// Trash, and `null`/`{}` when restoring. See
// `src/gui/src/helpers.js` → `window.move_items`.
newMetadata: (body.new_metadata ?? undefined) as
| Record<string, unknown>
| null
| undefined,
Record<string, unknown> | null | undefined,
});
const oldPath = source.path;
await this.#emitGuiEvent('outer.gui.item.moved', moved, {
@@ -756,8 +695,108 @@ export class LegacyFSController extends PuterController {
res.send('');
};
suggestApps = async (req: Request, res: Response): Promise<void> => {
const suggestSvc = this.services.suggestedApps;
if (!suggestSvc?.getSuggestedApps) {
res.json([]);
return;
}
const actor = this.#requireActor(req);
const body = asRecord(req.body);
let entryName: string | undefined;
let entryPath: string | undefined;
if (body.uid || body.path) {
try {
const entry = await resolveV1Selector(
this.stores.fsEntry,
body,
);
// Suggestions leak the entry's extension and existence;
// gate on `see` so app actors can't probe outside scope.
if (entry?.path) {
await assertAccess(
this.services.acl,
this.services.fs,
actor,
entry.path,
'see',
);
}
entryName = entry?.name;
entryPath = entry?.path;
} catch {
// Unresolvable or ACL-denied → empty suggestions.
}
}
const suggestions = await suggestSvc.getSuggestedApps({
name: entryName,
path: entryPath,
});
res.json(suggestions);
};
readdirSubdomains = async (req: Request, res: Response): Promise<void> => {
const actor = this.#requireActor(req);
// Subdomain enumeration is a user-level concern; app actors would
// otherwise see root_dir uids pointing outside their AppData scope.
if (effectiveActorApp(actor)) {
res.json([]);
return;
}
const userId = this.#getActorUserId(req);
const rows = await this.clients.db.read(
'SELECT `subdomain`, `root_dir_id`, `uuid`, `ts` FROM `subdomains` WHERE `user_id` = ?',
[userId],
);
res.json(rows);
};
updateFsentryThumbnail = async (
req: Request,
res: Response,
): Promise<void> => {
const actor = this.#requireActor(req);
const { uid, thumbnail } = asRecord(req.body) as {
uid?: string;
thumbnail?: string;
};
if (!uid)
throw new HttpError(400, 'Missing `uid`', {
legacyCode: 'bad_request',
});
if (!thumbnail)
throw new HttpError(400, 'Missing `thumbnail`', {
legacyCode: 'bad_request',
});
const entry = await this.stores.fsEntry.getEntryByUuid(uid);
if (!entry || !entry.path)
throw new HttpError(404, `Entry not found: uid=${uid}`, {
legacyCode: 'subject_does_not_exist',
});
await assertAccess(
this.services.acl,
this.services.fs,
actor,
entry.path,
'write',
);
// emitAndWait is required: the thumbnails extension rewrites
// `event.url` from a data URL to an `s3://` pointer, and the DB
// write below needs to see that rewrite.
const event = { url: thumbnail };
await this.clients.event.emitAndWait('thumbnail.created', event, {});
await this.clients.db.write(
'UPDATE `fsentries` SET `thumbnail` = ? WHERE `uuid` = ?',
[event.url, uid],
);
res.json({ thumbnail: event.url });
};
search = async (req: Request, res: Response): Promise<void> => {
this.#requireActor(req);
const actor = this.#requireActor(req);
const userId = this.#getActorUserId(req);
const body = asRecord(req.body);
const query = getString(body, 'query', 'text') ?? '';
@@ -766,7 +805,21 @@ export class LegacyFSController extends PuterController {
legacyCode: 'bad_request',
});
const results = await this.services.fs.searchByName(userId, query, 200);
// App-under-user actors only see entries within their AppData root;
// user actors are unscoped. Mirrors the ACL short-circuit in
// ACLService.check.
const app = effectiveActorApp(actor);
const username = actor.user?.username;
const pathScope =
app && typeof username === 'string' && username.length > 0
? `/${username}/AppData/${app.uid}`
: undefined;
const results = await this.services.fs.searchByName(
userId,
query,
200,
pathScope,
);
const shaped = await Promise.all(
results.map((r) => toLegacyEntry(this.clients.event, r)),
);
@@ -888,7 +941,7 @@ export class LegacyFSController extends PuterController {
return this.read(req, res, { realMime: true });
};
// ── Signed-URL + meta routes ────────────────────────────────────────
// -- Signed-URL + meta routes ----------------------------------------
/**
* POST /sign
@@ -922,8 +975,7 @@ export class LegacyFSController extends PuterController {
}
type SignedOrEmpty =
| (SignedFile & { path?: string })
| Record<string, never>;
(SignedFile & { path?: string }) | Record<string, never>;
const result: { signatures: SignedOrEmpty[]; token?: string } = {
signatures: [],
};
@@ -937,7 +989,10 @@ export class LegacyFSController extends PuterController {
legacyCode: 'not_found',
});
grantApp = { uid: app.uid };
result.token = this.services.auth.getUserAppToken(actor, app.uid);
result.token = await this.services.auth.getUserAppToken(
actor,
app.uid,
);
}
for (const rawItem of items) {
@@ -1001,7 +1056,12 @@ export class LegacyFSController extends PuterController {
}
const signed = signEntry(entry, signingCfg);
result.signatures.push({ ...signed, path: entry.path });
if (finalAction !== 'write') {
const { write_url: _, ...rest } = signed;
result.signatures.push({ ...rest, path: entry.path });
} else {
result.signatures.push({ ...signed, path: entry.path });
}
} catch {
// Silently skip unresolvable items.
result.signatures.push({});
@@ -1054,7 +1114,32 @@ export class LegacyFSController extends PuterController {
const operation =
typeof query.operation === 'string' ? query.operation : 'write';
// `write` — multipart upload, streamed directly to the v2 write path.
// A valid write signature authorises overwriting the file's bytes,
// not structural changes. Restrict copy/move/mkdir/rename/delete/trash
// to a caller authenticated as the owner — otherwise a recipient of a
// write-share could relocate or destroy the source via this endpoint.
if (operation !== 'write' && req.actor?.user?.id !== userId) {
throw new HttpError(
403,
`'${operation}' via signed URL requires owner authentication`,
{ legacyCode: 'forbidden' },
);
}
// ACL re-check: require an authenticated actor with write
// permission. A valid write signature alone is not sufficient —
// the caller must also pass ACL, preventing exploitation of
// leaked write URLs by read-only share recipients.
const callerActor = this.#requireActor(req);
if (operation === 'write') {
await assertAccess(
this.services.acl,
this.services.fs,
callerActor,
targetEntry.path,
'write',
);
}
if (operation === 'write') {
const body = asRecord(req.body);
const parentEntry = targetEntry.isDir
@@ -1087,8 +1172,18 @@ export class LegacyFSController extends PuterController {
}
// Non-write operations: route to existing service methods and sign the result.
// The signature alone authorises only byte writes to `targetEntry`. Structural
// ops (mkdir/rename/copy/move/delete) require a caller actor with explicit
// ACL on the affected paths — mirroring the unsigned counterparts above.
const record = asRecord(req.body);
if (operation === 'mkdir') {
await assertAccess(
this.services.acl,
this.services.fs,
callerActor,
targetEntry.path,
'write',
);
const folderName =
typeof record.name === 'string'
? record.name
@@ -1110,12 +1205,26 @@ export class LegacyFSController extends PuterController {
throw new HttpError(400, '`new_name` required', {
legacyCode: 'bad_request',
});
await assertAccess(
this.services.acl,
this.services.fs,
callerActor,
targetEntry.path,
'write',
);
const renamed = await this.services.fs.rename(targetEntry, newName);
await this.#emitGuiEvent('outer.gui.item.updated', renamed);
res.json({ ...signEntry(renamed, signingCfg), path: renamed.path });
return;
}
if (operation === 'delete' || operation === 'trash') {
await assertAccess(
this.services.acl,
this.services.fs,
callerActor,
targetEntry.path,
'write',
);
// Treat trash == delete (recursive). Most clients just call delete
// directly; if a trash folder becomes important we can revisit.
await this.services.fs.remove(userId, {
@@ -1139,6 +1248,20 @@ export class LegacyFSController extends PuterController {
this.stores.fsEntry,
destRef,
);
await assertAccess(
this.services.acl,
this.services.fs,
callerActor,
targetEntry.path,
operation === 'copy' ? 'read' : 'write',
);
await assertAccess(
this.services.acl,
this.services.fs,
callerActor,
destinationParent.path,
'write',
);
const method = operation === 'copy' ? 'copy' : 'move';
const result = await this.services.fs[method](userId, {
source: targetEntry,
@@ -1203,12 +1326,14 @@ export class LegacyFSController extends PuterController {
}
// Directory: return a signed listing of direct children.
// The caller only proved read access, so strip write_url from
// each child to prevent privilege escalation via /writeFile.
if (entry.isDir) {
const children = await this.services.fs.listDirectory(entry.uuid);
const signedChildren = children.map((child) => ({
...signEntry(child, signingCfg),
path: child.path,
}));
const signedChildren = children.map((child) => {
const { write_url: _, ...rest } = signEntry(child, signingCfg);
return { ...rest, path: child.path };
});
res.json(signedChildren);
return;
}
@@ -1226,8 +1351,15 @@ export class LegacyFSController extends PuterController {
query.download === '1' ||
query.download === true;
if (download.contentType)
if (download.contentType) {
res.setHeader('Content-Type', download.contentType);
// Uploader-controlled type served inline on the file origin —
// sandbox active-document types so an uploaded HTML/SVG can't
// execute scripts here. Mirrors FSController /fs/read.
if (!wantsAttachment) {
applyInlineContentSecurity(res, download.contentType);
}
}
if (download.contentLength !== null)
res.setHeader('Content-Length', String(download.contentLength));
if (download.contentRange)
@@ -1281,6 +1413,20 @@ export class LegacyFSController extends PuterController {
'read',
);
// Downgrade the envelope when the caller only proved read.
// `/writeFile`'s ACL re-check would still block the write, but
// returning `write_url` to a read-only caller is the same
// privilege-leak shape that `/sign` and `/readdir` strip.
const writeOk = await this.services.acl.check(
actor,
{
path: entry.path,
resolveAncestors: () =>
this.services.fs.getAncestorChain(entry.path),
},
'write',
);
const suggested =
(await this.services.suggestedApps?.getSuggestedApps({
name: entry.name,
@@ -1300,11 +1446,20 @@ export class LegacyFSController extends PuterController {
{},
{ reason: 'open_item' },
);
token = this.services.auth.getUserAppToken(actor, defaultAppUid);
token = await this.services.auth.getUserAppToken(
actor,
defaultAppUid,
);
}
const signingCfg = signingConfigFromAppConfig(this.config);
const signature = { ...signEntry(entry, signingCfg), path: entry.path };
const signed = signEntry(entry, signingCfg);
const signature = writeOk
? { ...signed, path: entry.path }
: (() => {
const { write_url: _, ...rest } = signed;
return { ...rest, path: entry.path };
})();
res.json({
signature,
token,
@@ -1361,10 +1516,7 @@ export class LegacyFSController extends PuterController {
const subjectRef = body.subject;
const appRef = body.app;
const mode = (getString(body, 'mode') ?? 'read') as
| 'see'
| 'list'
| 'read'
| 'write';
'see' | 'list' | 'read' | 'write';
if (!subjectRef || !appRef)
throw new HttpError(400, '`subject` and `app` are required', {
legacyCode: 'bad_request',
@@ -1469,17 +1621,6 @@ export class LegacyFSController extends PuterController {
download.body.pipe(res);
};
// Helpers for writeFile
// ── GUI event emission ───────────────────────────────────────────
//
// Fire-and-forget `outer.gui.item.*` events so SocketService,
// BroadcastService, WorkerDriver (hot-reload), and cache-invalidation
// listeners pick up mutations made through the legacy (bare-path) routes.
// FSController (v2-native /fs/* routes) emits these from its own handlers;
// LegacyFSController delegates to the same FSService but needs its
// own emissions because the service layer deliberately doesn't emit GUI
// events (that's a controller concern).
async #emitGuiEvent(
eventName:
| 'outer.gui.item.added'
@@ -1600,7 +1741,7 @@ export class LegacyFSController extends PuterController {
});
}
// ── Batch route ─────────────────────────────────────────────────────
// -- Batch route -----------------------------------------------------
//
// `/batch` interleaves multipart JSON operations with optional file
// uploads. puter-js uses it for `write`, `shortcut`, `mkdir`, `move`,
@@ -1909,7 +2050,46 @@ export class LegacyFSController extends PuterController {
}> = [];
const fileinfos: Array<Record<string, unknown>> = [];
let parseError: Error | null = null;
const bb = Busboy({ headers: req.headers });
const bb = Busboy({
headers: req.headers,
limits: {
fileSize: BATCH_MAX_FILE_SIZE,
files: BATCH_MAX_FILES,
parts: BATCH_MAX_PARTS,
fieldSize: BATCH_MAX_FIELD_SIZE,
},
});
// Busboy emits these `*Limit` events when a configured cap is
// hit. Capture the first one as a 413 so callers get a clean
// signal instead of a silently-truncated upload.
bb.on('filesLimit', () => {
if (!parseError) {
parseError = new HttpError(
413,
`Too many files in batch (max ${BATCH_MAX_FILES})`,
{ legacyCode: 'too_large' as never },
);
}
});
bb.on('partsLimit', () => {
if (!parseError) {
parseError = new HttpError(
413,
`Too many parts in batch (max ${BATCH_MAX_PARTS})`,
{ legacyCode: 'too_large' as never },
);
}
});
bb.on('fieldsLimit', () => {
if (!parseError) {
parseError = new HttpError(
413,
'Too many fields in batch',
{ legacyCode: 'too_large' as never },
);
}
});
bb.on('field', (fieldName, value) => {
try {
@@ -1936,8 +2116,24 @@ export class LegacyFSController extends PuterController {
// For streaming uploads use the signed `/writeFile` endpoint.
bb.on('file', (_fieldName, stream, info) => {
const chunks: Buffer[] = [];
let truncated = false;
stream.on('data', (chunk: Buffer) => chunks.push(chunk));
// Busboy emits `limit` after writing the first byte past
// `fileSize`. The stream continues being drained so the
// multipart parser stays in sync, but we discard the (now
// truncated) buffer and mark the batch as failed.
stream.on('limit', () => {
truncated = true;
if (!parseError) {
parseError = new HttpError(
413,
`File in batch exceeds ${BATCH_MAX_FILE_SIZE} bytes`,
{ legacyCode: 'too_large' as never },
);
}
});
stream.on('end', () => {
if (truncated) return;
files.push({
content: Buffer.concat(chunks),
mimeType:
@@ -1997,121 +2193,7 @@ export class LegacyFSController extends PuterController {
return { error: true, status: 500, message: 'Unknown batch error' };
}
// ── Helpers ─────────────────────────────────────────────────────────
#parsePositiveIntegerQuery(
query: Record<string, unknown>,
key: string,
message: string,
): number | undefined {
const value = query[key];
if (value === undefined || value === null || value === '') {
return undefined;
}
const parsed = Number.parseInt(String(value), 10);
if (!Number.isInteger(parsed) || parsed < 1) {
throw new HttpError(400, message, { legacyCode: 'bad_request' });
}
return parsed;
}
#parseNonNegativeIntegerQuery(
query: Record<string, unknown>,
key: string,
message: string,
): number | undefined {
const value = query[key];
if (value === undefined || value === null || value === '') {
return undefined;
}
const parsed = Number.parseInt(String(value), 10);
if (!Number.isInteger(parsed) || parsed < 0) {
throw new HttpError(400, message, { legacyCode: 'bad_request' });
}
return parsed;
}
#normalizeRangeHeader(rangeHeader: string): string | undefined {
const firstRange = rangeHeader.includes(',')
? rangeHeader.split(',')[0]?.trim()
: rangeHeader.trim();
if (!firstRange) return undefined;
const matches = firstRange.match(/^bytes=(\d+)-(\d*)$/);
if (!matches) return undefined;
const [, start, end] = matches;
return end ? `bytes=${start}-${end}` : `bytes=${start}-`;
}
#pipeLimitedLines(
source: NodeJS.ReadableStream,
res: Response,
lineCount: number,
): void {
let remainingLines = lineCount;
let isClosed = false;
const closeSource = () => {
if (isClosed) return;
isClosed = true;
if ('destroy' in source && typeof source.destroy === 'function') {
source.destroy();
}
};
source.on('error', (err) => {
if (!isClosed) {
isClosed = true;
res.destroy(err);
}
});
res.on('close', () => {
closeSource();
});
source.on('data', (chunk: Buffer | string) => {
if (isClosed) return;
const buffer = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);
let endIndex = buffer.length;
for (let index = 0; index < buffer.length; index++) {
if (buffer[index] !== 0x0a) continue;
remainingLines -= 1;
if (remainingLines === 0) {
endIndex = index + 1;
break;
}
}
if (endIndex > 0) {
const canContinue = res.write(buffer.subarray(0, endIndex));
if (!canContinue) {
source.pause();
res.once('drain', () => {
if (!isClosed) {
source.resume();
}
});
}
}
if (endIndex !== buffer.length) {
res.end();
closeSource();
}
});
source.on('end', () => {
if (!isClosed) {
isClosed = true;
res.end();
}
});
}
// -- Helpers ---------------------------------------------------------
#requireActor(req: Request) {
const actor = req.actor;
if (!actor) {
@@ -0,0 +1,102 @@
/**
* Copyright (C) 2024-present Puter Technologies Inc.
*
* This file is part of Puter.
*
* Puter is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
import { describe, expect, it } from 'vitest';
import { loadLegacyAssociatedApps } from './legacyFsHelpers.js';
import type { FSEntry } from '../../stores/fs/FSEntry.js';
const entryWithApp = (associatedAppId: number): FSEntry =>
({ associatedAppId }) as unknown as FSEntry;
const appRow = (overrides: Record<string, unknown>): Record<string, unknown> => ({
id: 1,
uid: 'app-1',
owner_user_id: 99,
app_owner: 99,
icon: 'icon',
name: 'an-app',
title: 'An App',
description: 'desc',
godmode: 1,
maximize_on_start: 1,
index_url: 'https://an-app.puter.site/',
background: 1,
metadata: { secret: true },
is_private: 0,
protected: 0,
...overrides,
});
const fakeStore = (row: Record<string, unknown>) => ({
getByIds: async (ids: number[]) =>
new Map(ids.map((id) => [id, { ...row, id }])),
});
describe('loadLegacyAssociatedApps associated_app redaction', () => {
it('never leaks owner identifiers, even for public apps', async () => {
const out = await loadLegacyAssociatedApps(
fakeStore(appRow({ is_private: 0, protected: 0 })),
[entryWithApp(1)],
);
const app = out.get(1)!;
expect(app).not.toHaveProperty('owner_user_id');
expect(app).not.toHaveProperty('app_owner');
});
it('passes through hosting + capability fields for public apps', async () => {
const out = await loadLegacyAssociatedApps(
fakeStore(appRow({ is_private: 0, protected: 0, godmode: 1 })),
[entryWithApp(1)],
);
const app = out.get(1)!;
expect(app.index_url).toBe('https://an-app.puter.site/');
expect(app.godmode).toBe(1);
expect(app.maximize_on_start).toBe(1);
});
it('redacts hosting + capability fields for private apps', async () => {
const out = await loadLegacyAssociatedApps(
fakeStore(appRow({ is_private: 1, godmode: 1 })),
[entryWithApp(1)],
);
const app = out.get(1)!;
// Existence + display fields still surface...
expect(app.uid).toBe('app-1');
expect(app.name).toBe('an-app');
expect(app.is_private).toBe(1);
// ...but the sensitive bits are stripped.
expect(app.index_url).toBeNull();
expect(app.godmode).toBe(0);
expect(app.maximize_on_start).toBe(0);
expect(app.background).toBe(0);
expect(app.metadata).toBeNull();
expect(app).not.toHaveProperty('owner_user_id');
});
it('redacts the same fields for protected apps', async () => {
const out = await loadLegacyAssociatedApps(
fakeStore(appRow({ is_private: 0, protected: 1, godmode: 1 })),
[entryWithApp(1)],
);
const app = out.get(1)!;
expect(app.protected).toBe(1);
expect(app.index_url).toBeNull();
expect(app.godmode).toBe(0);
});
});
+22 -11
View File
@@ -48,7 +48,7 @@ import type { IConfig } from '../../types.js';
* terse.
*/
// ── Body parsing ─────────────────────────────────────────────────────
// -- Body parsing -----------------------------------------------------
export function asRecord(value: unknown): Record<string, unknown> {
if (!value || typeof value !== 'object' || Array.isArray(value)) return {};
@@ -160,7 +160,7 @@ export async function resolveV1Selector(
return entry;
}
// ── ACL ──────────────────────────────────────────────────────────────
// -- ACL --------------------------------------------------------------
export async function assertAccess(
aclService: ACLService,
@@ -261,7 +261,7 @@ export async function assertCanCreate(
await assertAccess(aclService, fsService, actor, parentForCheck, 'write');
}
// ── Response shaping ────────────────────────────────────────────────
// -- Response shaping ------------------------------------------------
type AppRowLookup = {
getByIds: (ids: number[]) => Promise<Map<number, Record<string, unknown>>>;
@@ -274,21 +274,33 @@ const toIntBool = (v: unknown): number => (v ? 1 : 0);
* embedded in legacy FS entries. Booleans round-trip back to integers (0/1)
* because the v1 wire contract emits them that way and existing clients key
* off it. Other columns pass through as-is — `metadata` is already parsed.
*
* Two redactions, because `associatedAppId` can name an app the actor does
* not own (the column is client-writable display metadata, never trusted for
* authz — see FSController) and may point cross-tenant:
* - Owner identifiers (`owner_user_id`, `app_owner`) are never emitted —
* they're internal user references with no client-side display use.
* - For private or protected apps, the direct hosting URL and launch
* capability flags (`index_url`, `godmode`, `maximize_on_start`,
* `background`, `metadata`) are dropped. This mirrors AppDriver, which
* withholds `index_url` from callers without read entitlement. Display
* fields (name, icon, title) still pass through so the GUI can label the
* file's associated app.
*/
function mapAppForLegacyAssociatedApp(
app: Record<string, unknown>,
): Record<string, unknown> {
const gated = Boolean(app.is_private) || Boolean(app.protected);
return {
id: app.id,
uid: app.uid,
owner_user_id: app.owner_user_id,
icon: app.icon,
name: app.name,
title: app.title,
description: app.description,
godmode: toIntBool(app.godmode),
maximize_on_start: toIntBool(app.maximize_on_start),
index_url: app.index_url,
godmode: gated ? 0 : toIntBool(app.godmode),
maximize_on_start: gated ? 0 : toIntBool(app.maximize_on_start),
index_url: gated ? null : app.index_url,
approved_for_listing: toIntBool(app.approved_for_listing),
approved_for_opening_items: toIntBool(app.approved_for_opening_items),
approved_for_incentive_program: toIntBool(
@@ -297,9 +309,8 @@ function mapAppForLegacyAssociatedApp(
timestamp: app.timestamp ?? null,
last_review: app.last_review ?? null,
tags: app.tags ?? null,
app_owner: app.app_owner ?? null,
background: toIntBool(app.background),
metadata: app.metadata ?? null,
background: gated ? 0 : toIntBool(app.background),
metadata: gated ? null : (app.metadata ?? null),
protected: toIntBool(app.protected),
is_private: toIntBool(app.is_private),
};
@@ -461,7 +472,7 @@ export async function toLegacyEntry(
export { normalizeAbsolutePath };
// ── Signing ─────────────────────────────────────────────────────────
// -- Signing ---------------------------------------------------------
/**
* Pull the signing config off the app config. Throws if either value is
+1
View File
@@ -52,6 +52,7 @@ export interface ThumbnailUploadPrepareItem {
size?: number;
uploadUrl?: string;
thumbnailUrl?: string;
item_uid: string;
}
export interface ThumbnailUploadPreparePayload {
@@ -165,6 +165,43 @@ describe('HomepageController shell routes', () => {
expect(html).toContain('Puter');
});
it('exposes disable_temp_users to the GUI when signups are disabled', async () => {
const homepageConfig = server.controllers.homepage.config as {
disable_user_signup?: boolean;
};
const prev = homepageConfig.disable_user_signup;
homepageConfig.disable_user_signup = true;
const { res, captured } = makeRes();
try {
await callRoute('get', '/', makeReq({ path: '/' }), res);
} finally {
homepageConfig.disable_user_signup = prev;
}
expect(String(captured.body)).toContain('"disable_temp_users":true');
});
it('keeps an operator-set gui_params.disable_temp_users when the flag is off', async () => {
const homepageConfig = server.controllers.homepage.config as {
disable_user_signup?: boolean;
gui_params?: Record<string, unknown>;
};
const prevFlag = homepageConfig.disable_user_signup;
const prevGuiParams = homepageConfig.gui_params;
homepageConfig.disable_user_signup = false;
homepageConfig.gui_params = {
...prevGuiParams,
disable_temp_users: true,
};
const { res, captured } = makeRes();
try {
await callRoute('get', '/', makeReq({ path: '/' }), res);
} finally {
homepageConfig.disable_user_signup = prevFlag;
homepageConfig.gui_params = prevGuiParams;
}
expect(String(captured.body)).toContain('"disable_temp_users":true');
});
it('still serves the shell when an authenticated actor is present', async () => {
const { actor } = await makeUser();
const { res, captured } = makeRes();
@@ -173,12 +210,14 @@ describe('HomepageController shell routes', () => {
expect(String(captured.body)).toMatch(/<!DOCTYPE html>/i);
});
it('serves the shell on /settings, /dashboard, /action, /@:username', async () => {
it('serves the shell on /settings, /dashboard, /desktop, /action, /@:username', async () => {
for (const path of [
'/settings',
'/settings/*splat',
'/dashboard',
'/dashboard/',
'/desktop',
'/desktop/',
'/action/*splat',
'/@:username',
]) {
@@ -72,7 +72,7 @@ export class HomepageController extends PuterController {
await homepage.send({ req, res, actor }, meta, launch);
};
// ── Root + path-aliased shell routes ────────────────────────
// -- Root + path-aliased shell routes ------------------------
router.get('/', {}, (req, res) => sendShell(req, res));
@@ -82,11 +82,14 @@ export class HomepageController extends PuterController {
router.get('/dashboard', {}, (req, res) => sendShell(req, res));
router.get('/dashboard/', {}, (req, res) => sendShell(req, res));
router.get('/desktop', {}, (req, res) => sendShell(req, res));
router.get('/desktop/', {}, (req, res) => sendShell(req, res));
router.get('/action/*splat', {}, (req, res) => sendShell(req, res));
router.get('/@:username', {}, (req, res) => sendShell(req, res));
// ── /app/:name ─ app metadata baked into the shell ──────────
// -- /app/:name - app metadata baked into the shell ----------
router.get('/app/:name', {}, async (req, res) => {
const name = String(req.params.name ?? '');
@@ -122,7 +125,7 @@ export class HomepageController extends PuterController {
});
});
// ── /show/* ─ launch explorer with the requested file path ──
// -- /show/* - launch explorer with the requested file path --
router.get('/show/*splat', {}, (req, res) => {
const filePath = req.path.slice('/show'.length);
@@ -138,7 +141,7 @@ export class HomepageController extends PuterController {
return sendShell(req, res, {}, launch);
});
// ── Fallback static mount ───────────────────────────────────
// -- Fallback static mount -----------------------------------
// Serves lingering GUI assets (images, fonts, lib files, etc.)
// out of <gui_assets_root>/src. Falls through to the 404 handler
// when the file doesn't exist.
@@ -37,13 +37,14 @@ export class HostingController extends PuterController {
}
registerRoutes(router) {
// ── Delete site ─────────────────────────────────────────────
// -- Delete site ---------------------------------------------
router.post(
'/delete-site',
{
subdomain: 'api',
requireUserActor: true,
allowFullAccessToken: true,
requireVerified: true,
},
async (req, res) => {
Loaded 100 of 499 files, more files were not shown because too many files have changed in this diff. Show more