Compare commits

...
Author SHA1 Message Date
feder-cr 0e502db1c0 dot-browser: a link to nothing is no browser, PATH checked where the image is made, failures say what they did
The check that dot's PATH reaches the server is the image's: it runs at install, not after a switch has already moved
the link, so install.sh's message on a failed switch is true (the browser is either the one the Dot had or the staged
one). A link whose environment is gone is no current browser. newest's awk reads every line instead of exiting early
under pipefail, and the browser smoke's preparation fails at the resolution that failed.
2026-10-15 04:27:16 +02:00
feder-cr c16e144f89 browser: every Dot follows the newest release, and install.sh alone starts the Dot's units
A Dot kept the browser of the golden image it was made from. runtime/dot-browser.sh (the old build-browser-env.sh,
now on the runtime disk as well as the builder seed) is the one owner of where the Dot's browser lives: each release
in an environment of its own, browser/<mcp>-<playwright>-<core>, and the Dot's browser the one
~dot/.local/bin/invisible-playwright-mcp links to. At every start install.sh switches that link to a release an
earlier start staged, in one rename and before the Dot's units, and after them stages the newest release as uv
resolves it, in the background: its environment and its browser engine, through the Dot's own way out. A start never
waits for a release; without the network the Dot keeps its browser. provision.sh installs the golden image's browser
with the same script.

The switch has to come before the engine, and that ordering did not hold: the units were enabled, so on every boot
after the first systemd started them at multi-user.target, before cloud-init's final stage ran install.sh
(cloud-final.service is After=multi-user.target; a Dot's serial log shows dot-agentd and the engine up two seconds
before install.sh). A Dot with a VM proxy went out before its proxy was set. The units are no longer enabled:
install.sh starts them, after the proxy and the switch, and removes the links an earlier runtime made.
2026-10-15 04:22:28 +02:00
Federico Elia ef69f5e19c Merge pull request #1454 from feder-cr/libs-latest
browser: never pinned; the image takes the newest release and the engine reads the server it finds
2026-10-14 18:38:56 -07:00
feder-cr ed406afb14 smoke and dependencies: the stand-in is a server package, and the capture runs on the server's python
The engine smoke found its stand-in by fake_mcp_server.py on the command line; installed the way the real server is,
it runs as `python -I -B -m invisible_playwright_mcp`, and that command line is what the checks look for.
test_dependencies names server_capture.py as a module another interpreter runs: its invisible_playwright_mcp import
is the installed server's, not a dependency of the engine.
2026-10-15 02:18:52 +02:00
feder-cr 2968ca8372 browser: never pinned; the image takes the newest release and the engine reads the server it finds
A Dot keeps the golden image it was made from and boots every new runtime disk, so the engine already talked to
servers other than the one its pinned copy of the interface described. The engine now reads the interface from the
installed server when it starts (nanobot/dots/server_interface.py, server_capture.py run as dot by the server's own
python): the page tools are what that server serves, a page tool it does not serve is not offered (GET /tools says
so), and a Dot whose server cannot be read offers no page tool and its launches say why.

With nothing in the engine tied to a version, the image no longer pins one: each build asks PyPI for the newest
invisible-playwright-mcp, the newest invisible-playwright it accepts and the invisible-core that one pins
(browser-release.ts). The three enter the inputs digest, pins.env and the manifest, which records what was installed.
mcp-requirements.lock and capture-mcp-interface.py go; the copy of the interface becomes a test fixture. The owner's
no-other-browser rule is checked at build on what was actually installed (builder/other-browsers.txt), and the golden
image workflow runs every six hours so a new release is published as a prebuilt image.
2026-10-15 02:04:31 +02:00
Federico 4d20e947a1 Merge pull request #1450 from feder-cr/libs-0722
image: the browser is invisible-playwright-mcp 0.72.2 (invisible-playwright 0.35.7, invisible-core 45.39.0, firefox-45)
2026-10-13 14:33:32 -07:00
feder-cr 6712104ea5 image: the browser is invisible-playwright-mcp 0.72.2 (invisible-playwright 0.35.7, invisible-core 45.39.0, firefox-45)
The server's snapshot and read_html read again on the page that replaced the one they went to (page.evaluate_read),
and browser_evaluate, the model's code, runs once when its page navigates under it (B285). Under them, through
firefox-45: a page that replaces itself keeps its main world (B286), the real mouse no longer turns a click into a
drag, a session keeps focus when another starts beside it, iframes stay in the frame tree after a few navigations,
go_back/go_forward to a same-document entry answer, a browser that dies ends every wait at once, every input event
is born a few ms before the page sees it, and the first enumerateDevices of a session takes a time the seed
declares. The tools' schemas and descriptions are unchanged: the recaptured interface differs only in its version.
2026-10-13 22:25:33 +02:00
Federico a2584f2619 Merge pull request #1448 from feder-cr/no-char-caps
engine: no fixed cap on a tool's result; the cut and the routing follow the model's window, read from OpenRouter on every turn
2026-10-12 08:35:49 -07:00
feder-cr fe03ee6325 floors: the engine suite at the 2493 tests CI measured on this branch 2026-10-12 17:20:59 +02:00
feder-cr 5c920517e4 engine: a model's requests go only to the providers of its widest window; image: invisible-playwright-mcp 0.72.1
OpenRouter routes a request by the answer it asks for but not by the prompt it carries, so a request sized for
the model's widest window could land on a provider with a narrower one (glm-5.3-flash: Reka at 262144 among
providers at 1048576) and fail there. The providers read with the limits on every turn now route the model's
requests: provider.only is the slugs of the widest window's providers. When none of them can serve, OpenRouter
refuses with "No allowed providers are available" and the turn fails with those words.

invisible-playwright-mcp 0.72.1 drops its last caps: browser_type quotes the value the field kept whole, and an
error detail in its chat and routes comes whole. The tools' schemas are unchanged.
2026-10-12 16:35:10 +02:00
feder-cr 440e43a1f5 engine: no fixed cap on a tool's result; the cut is the model's window, read from OpenRouter on every turn
A result comes back whole. exec, glob/grep and read_file lose their fixed caps (10000/50000, 12000 and 2000
per line, 128000) and the general 12000-character cut is gone. A request that outgrows the model's window is
compacted first, as before; only if it still does not fit are the results of its latest round cut to what the
window leaves, measured with the estimate that judges the request, and the cut says how many characters came
through of how many and how to ask for less. A result is stored cut only beyond what no compaction could make
room for.

The window is read on every turn from the providers OpenRouter lists for the model
(GET /models/{model}/endpoints), not once per process from the whole list: of the providers serving it, the
widest window and the longest answer at that window.
2026-10-12 12:43:40 +02:00
Federico 00f63e88c1 Merge pull request #1447 from feder-cr/browser-libs
image: the browser is invisible-playwright-mcp 0.72.0 (invisible-playwright 0.35.0, invisible-core 41.36.0)
2026-10-12 07:53:06 +02:00
feder-cr 9610d12e29 image: the browser is invisible-playwright-mcp 0.72.0 (invisible-playwright 0.35.0, invisible-core 41.36.0)
The core keeps a frame's navigation history when Firefox re-announces the main frame on a cross-process
navigation, redoes a by-value call whose execution context a navigation destroyed, and answers a goto that
another navigation overtook with Playwright's "interrupted" error instead of waiting 45 s. The wrapper carries
invisible_playwright#321's fix. The server reads an element's text in one command and returns it whole:
max_chars is gone from browser_read_text and browser_snapshot. The interface is captured again from the lock.
2026-10-12 07:27:49 +02:00
Federico 81eb3ba067 Merge pull request #1446 from feder-cr/hero-from-photos
docs: the README animation is built from photos in the tree, with new characters
2026-10-12 03:01:05 +02:00
feder-cr 9151ce3386 docs: the README animation is built from photos in the tree, with new characters
The six hero GIFs were rendered from pages that lived only in a scratch folder, so
changing one character meant editing frames. docs/hero/ now holds every piece, in
three layers, each made from the one before:

  photos/  characters.png as an image model draws it, on white; cut_out.py turns
           it into one cut-out per theme (shadows kept on light, removed on dark,
           and only the white connected to the edge touched, so the eyes stay)
  html/    wide, mobile and small pages that compose the photos; renderLoop(t)
           draws the frame at second t, and opened in a browser each page plays
           its own loop
  build.mjs  photos -> frames (Chromium, 12 a second) -> docs/images/hero-*.gif,
           with the same palette and dither as before

A fourth page, html/social.html, builds docs/images/social-preview.png, the
repository's social preview, from the same photos.

The characters are new: the same four, colours inverted, each with a role it can
carry (a hard hat, sunglasses, a beret, a magnifying glass), all still angry. The
scenes, their words and their timing are unchanged.
2026-10-12 02:45:30 +02:00
Federico 96eee2ba8c Merge pull request #1445 from feder-cr/telegram-429-test
tests: the 429 poll test has a bot of its own, and the API test removes the channels it adds
2026-10-12 00:11:32 +02:00
feder-cr fc470314f2 api tests: the test of every Dot's channels removes the channels it added
It added two channels and left them running for the rest of the file and its teardown, the only channels there; on
one Windows run that teardown did not finish in 60 s (not reproduced in twelve runs under load here). A test stops
what it starts: the two channels are removed at its end, and the teardown is what it was before they came.
2026-10-12 00:01:03 +02:00
feder-cr 361b770285 channels tests: the 429 poll test has a bot of its own
It failed now and then in CI (four runs since 2026-10-08), always right after the test that polls TOKEN every few
milliseconds with a 409: a poll of that test still on its way when it ended was answered after the next test had
queued its 429 on the same token, as Telegram answers a request whose client has gone, so the stray poll took the
429 and the new poller connected without the error the test waits for. The 429 test now uses a token no other test
polls.
2026-10-11 23:45:40 +02:00
Federico 6a9cc8c113 Merge pull request #1444 from feder-cr/engine-path-and-shell
The engine runs nothing of the model's by name, reads every sentence of its browser server, and the health is its own context
2026-10-11 23:31:39 +02:00
feder-cr fb3b667c01 ci: the vitest floors are what this branch's CI measured: 2209 on Linux and 2206 on Windows 2026-10-11 23:17:59 +02:00
feder-cr c5d48ac017 smoke: the browser smoke takes dot's PATH from the desktop's unit
GUEST_PATH was one PATH for the engine and the desktop; with the engine's unit on system directories only, the browser
smoke started the desktop and looked for invisible-playwright-mcp with a variable that no longer existed. It reads
dot's PATH from dot-desktop.service, as lib.sh reads the engine's and the daemon's from theirs.
2026-10-11 23:03:02 +02:00
feder-cr b82ca890ff e2e: step j asks the model to leave the browser open, which the step goes on to need
The step keeps research open "as it would be left at night" so that the server restart and the stop find it open, but
its task never said so; a model closed the browser after reading the title, as a tidy one may, and the step failed on
a choice the prompt left open. The task now asks to leave the browser open.
2026-10-11 22:58:37 +02:00
feder-cr 780fb3b46a engine: a browser opened is known by invisible-playwright-mcp's OPENED, captured like its other sentences
invisible-playwright-mcp 0.71.2 exports OPENED, the first sentence of browser_open's answer, which the engine matched
by its words ("browser is open") because the library had no constant for it. The lock moves to 0.71.2 (only that
package and its hashes), the capture records OPENED beside NOT_OPEN and GONE, and the engine reads all three from
invisible_playwright_mcp.json: no sentence of the server is written in the engine any more.
2026-10-11 22:43:34 +02:00
feder-cr 3d0da4e090 web: the health answer is a context of its own
The shell's context held the health with everything else, so its poll every 30 seconds gave the whole value a new
identity and re-rendered every user of the shell. useHealth reads it from its own context; only the rail's status,
the offline banner, the setup checklist, the key form and the host settings follow the poll.
2026-10-11 19:29:55 +02:00
feder-cr bd1b5866cc guest: the engine runs nothing by name from a directory the model writes
The engine's unit put dot's ~/.local/bin first on its PATH, only so the guest check could find
invisible-playwright-mcp by name; but the engine holds the OpenRouter key, and a program a library ran by name would
have been dot's, run as dotengine. The unit's PATH is now system directories only, as dot-agentd's is, and it names
the browser's MCP server by its path (INVISIBLE_DOTS_MCP_COMMAND), which the engine checks and starts as dot through
the relay. The smoke reads both from the unit.
2026-10-11 19:29:54 +02:00
Federico 1935f47e5a Merge pull request #1443 from feder-cr/fixes-batch
Fixes, removals and one owner per fact, from a read of the whole repository
2026-10-11 19:02:23 +02:00
feder-cr 08d6786421 ci: the floors are what this branch's CI measured: vitest 2207 on Linux and 2204 on Windows, PostgreSQL 744 with 295 pg runs 2026-10-11 18:54:30 +02:00
feder-cr b50f09ed40 web tests: the fake control plane answers the two routes of every Dot, and the inbox says what one read cannot
The shell reads the failed tasks and the channels of every Dot with GET /api/tasks and GET /api/channels: the web
tests' control plane answers them as the real one does, and the inbox tests that faked one Dot's list failing now
fail the one read, which the inbox reports as such.
2026-10-11 18:39:45 +02:00
feder-cr 9ee1f32094 e2e: the journal check reads the journal as the Dot's user, without asking the model
A careful model refused to run a command that searches the system journal for sk-or-: it read like a probe for
secrets. The property the check proves is what the model's user can read, so it runs the same command through
dot-agentd's exec route, which the Dot's exec tool uses, as that user; the hash of the count and a nonce still proves
the whole journal was read.
2026-10-11 18:20:38 +02:00
feder-cr 373d837e29 e2e: step l checks that the interrupted command is not run again, not that the model runs nothing else
A careful model looks whether the marker is there after its command was interrupted (cat and pgrep): that is a
second exec call, but not the command again. The step counted every exec of the task and failed on it; it now
requires that no other call runs the interrupted command, and the marker check still proves the command never ran.
2026-10-11 18:12:04 +02:00
feder-cr 2e762e77d5 ci: the pytest floor is 2483, what the engine suite measures without the code no Dot runs
The engine's lean pass removed the MCP reconnect, the provider's reasoning options, the event sink, concurrent tool
batches, spilling results to files and the cron methods only tests used, each with its tests: the whole suite now
passes 2483 on Linux (measured in WSL, where CI runs it), down from 2553. The other floors only rise; they are set
from CI's measured counts once it has run.
2026-10-11 17:58:03 +02:00
feder-cr 528be98890 vm-manager, api: one logger and one pathExists, from shared
vm-manager's own Logger, silentLogger and stderrLogger repeated the shared ones; it now takes them from
packages/shared and logs as the rest of the process does (prefixedStderrLogger). vm-manager and the API's VM driver
each had their own exists(): one pathExists in shared replaces both.
2026-10-11 17:51:54 +02:00
feder-cr ac26cec1bf Merge branch 'guest-fixes' into fixes-batch 2026-10-11 17:50:35 +02:00
feder-cr 5316d43308 guest: dot-install only installs, the images are what their inputs say, and the Go copies of the protocol are held
- dot-install refused nothing apt reads as an action: "foo-" removed foo, as root. A name can no longer end in "-";
  a test runs the real script against stubs.
- The VM proxy's setup is runtime/vmproxy.py, staged on the runtime ISO: a bad value, a host that does not resolve and
  no default route each say so (exit 3, 4, 5) instead of all reading "not socks5"; the firewall still drops all. The
  smoke runs the file itself.
- dot-agentd is built one way everywhere (-trimpath -buildvcs=false -ldflags=-s -w): a commit no longer changes the
  binary and the runtime ISO. go.mod pins the toolchain, setup-go@v6 reads it, a repo test keeps every spelling equal.
- The golden image's apt packages come from an Ubuntu snapshot (pins.json apt_snapshot, in the inputs digest); the
  browser environment installs wheels only, with no cache left in the image.
- tests/repo/agentd-protocol.test.ts compares the Go routes and constants (and the engine's defaults) with
  packages/shared; a Linux test runs the real dot-agentd under the real GuestClient: proof, auth, exec, files, the
  agent proxy and its SSE.
- vm-manager: one assertImages, the VM bounds from CONFIG_BOUNDS, the pid file in its own module; one
  resolveImageVersion and publishImage for the golden, runtime and prebuilt images.
- Fixed-sleep tests wait for their signal; the docs no longer say the images are never published or speak of CD-ROMs.
2026-10-11 17:50:35 +02:00
feder-cr 5a65ada65d api, web: the shell reads the failed tasks and the channels of every Dot in one request each
The web's shell, on every page, read each Dot's task list and each Dot's channels, at load and again for every Dot on
each task.failed and channel event: 2N requests for N Dots, as its own comments said, because no route answered for
every Dot. GET /api/tasks?status=&finished_after= lists the tasks of every Dot that ended in a state after a time, and
GET /api/channels every channel with its Dot's id (never a credential); the SDK has endedTasks and everyChannel, and
the shell makes one request for each.
2026-10-11 17:49:03 +02:00
feder-cr b0514eac62 host: the pieces every package uses live in shared, so a package never depends on an app
Clock, Logger, silentLogger, errorMessage, ControlPlaneError, notFound and the stderr logger move from
apps/scheduler to packages/shared; packages/channels and apps/api import them from there, and channels keeps the
scheduler for its tests only. One abortableSleep replaces four copies (scheduler, CLI, SDK, WhatsApp); the one
shared had slept the whole time on a signal that had already aborted. The API reports its package's version, as the
CLI does, instead of a hard-coded 0.1.0.
2026-10-11 17:44:28 +02:00
feder-cr c8cb603941 Merge branch 'clients-fixes' into fixes-batch 2026-10-11 17:38:05 +02:00
feder-cr 825740351d Merge branch 'engine-lean' into fixes-batch 2026-10-11 17:38:05 +02:00
feder-cr 4e376c1927 clients: one wording for an approval everywhere, the CLI reads what it shows and can stop what it starts
- The tool and permission labels move from the web to packages/shared (labels.ts, still checked against
  permissions.py): Telegram, WhatsApp and the CLI ask "The Dot wants to run a command" with "Run commands (high
  risk)", as the web card does, and the CLI names the Dot.
- The event reader moves from the web to the SDK: `logs` reads only the newest events and follows on; `approvals
  --all` keeps the newest 500 and `status` reads the 10 it shows.
- New CLI commands: cancel, delete --yes, update with the config version (a concurrent save is a 409). The CLI's
  version comes from its package.
- The web's tool and approval steps are one model (steps.ts) for the chat, the task story and the activity; one
  outcome table and one duration format.
- The inbox keys answer only while the approvals list has the focus (WCAG 2.1.4); useResource and the Dot shell's
  context keep their identity across renders.
- setup --all no longer sends the person to a sign-in that does not exist; the MCP panel says the Dot can add a
  server; the e2e contract declares every CLI command run.ts runs, and a test holds it; apps/web's second vitest
  config is gone; docs follow.
2026-10-11 17:37:54 +02:00
feder-cr 1cbf4609d8 engine: a call written in the text runs nothing, a call runs once, and the code no Dot runs is gone
- The provider no longer turns <tool_call> text into calls (architecture 8.3 says such text runs nothing); a test
  holds it.
- The MCP client no longer retries a call after a connection error, which could run it twice on an HTTP server
  (section 8.7): it fails with its error and reports the server's end.
- Removed what the Dot never reaches (3342 lines, with their tests): the MCP reconnect and enabled_tools, the
  provider's reasoning tables and unused options, the event sink, concurrent tool batches, spilling results to files,
  the cron methods only tests used and the protected system jobs, and the identity state `archived`.
- The sentences of invisible-playwright-mcp the browser matches are captured from the pinned package into
  invisible_playwright_mcp.json, so a version bump that changes them fails a test.
- Fixed-sleep tests wait for their signal; architecture.md, UPSTREAM.md and main.py say what the code does.
2026-10-11 17:34:00 +02:00
feder-cr 8c6f3cb010 host: every task end is told, an approval adds one server once, and no state nobody sets
- A task whose task.created could not be delivered is failed with a task.failed event in the same transaction
  (the guest's type: a client follows one), so the web no longer shows it running for good; a commit point covers it.
- markDelivered marks the row and its run in one statement.
- add_mcp_server: the host uses up the approval that allowed the call (approvals.used_at, migration 0013), found in
  one statement with jsonb equality instead of the newest 500 and a hand-made canonical JSON; a call refused for its
  entry keeps its approval. The permission is typed.
- Dot DISABLED and VM IDLE, which nothing ever wrote, are gone from the states, the checks (migration 0014), the
  queries and the web.
- Unused exports removed (latestId, activeForDot, hasPending, isVmState, isDotState, API_TOKEN_ENV, MIN_TOKEN_LENGTH).
- One BackgroundWork for the scheduler and the lifecycle; syncGuest marks a Dot suspect itself when its push fails.
- The scheduler alone checks a message, a note and "always"; /api/stream resolves its Dot with historyDotId.
- The MCP secrets of a push are read in one query; EventLog.stream keeps one id instead of every replayed one.
- The event schemas and the declared event types are held equal by the typecheck, so the parsers need no cast.
- start.test waits for the three requests instead of 200 ms; the pre-push hook says what only CI runs; docs follow.
2026-10-11 17:32:20 +02:00
feder-cr 61e6dc21b4 bench: the tasks are written by their generators before every run, never stored
.gitignore's *.log kept the data of top-ips (access.log) and log-pipeline (app.log) out of the repository, so a clone
had both tasks without their input; the stored output could also drift from author.py and author_long.py, and did
(hand edits, zlib's bytes). run.sh now writes tests/bench/tasks and tasks-long afresh before each run.
2026-10-11 17:32:09 +02:00
Federico 21500338a3 Merge pull request #1442 from feder-cr/dot-mcp-add
A Dot adds an MCP server itself, with the person's approval
2026-10-11 11:16:27 +02:00
feder-cr 0060eee57a ci: the floors are what this branch's CI measured with add_mcp_server's tests: vitest 2148 on Linux and 2151 on Windows, PostgreSQL 738 with 293 pg runs, pytest 2553 2026-10-11 10:59:24 +02:00
feder-cr 0c51c42c60 tests: the real-VM run has the Dot add an MCP server itself and the person approve it; the smoke offers add_mcp_server
Step m queues a task that adds `clock` with add_mcp_server: the approval is under mcp.servers.add with the entry, and
once approved the server is in the config, with its permission left to ask, and connected. The engine smoke's tool
list for a Dot granted everything has add_mcp_server, and its check that a removed server's tools are gone looks at
the servers' tool names.
2026-10-11 10:44:27 +02:00
feder-cr e03edb06d4 docs: a Dot adds an MCP server itself, with the person's approval, and how the agent-hosts compare
Sections 5.4, 7 and 8.3, the guide, and the wiki article, with what NanoClaw, Hermes, OpenClaw, nanobot and Goose let
an agent do, from their source.
2026-10-11 10:44:27 +02:00
feder-cr 4f4cbca2f4 web: the approval of an MCP server the Dot adds shows what it runs, the secrets you will set and why
add_mcp_server's card lists the name, the command line (or the URL), the environment and headers written in it,
the secrets the person will set and the reason; the activity says what was added, or why it was not. "Add MCP
servers" is a row of the MCP servers group of the permissions.
2026-10-11 10:44:27 +02:00
feder-cr bd3cf77274 host: a server the Dot adds is written only when allowed or approved for exactly these arguments
On `mcp.server.added` the host checks, in the transaction that stores it: mcp.servers.add allows it, or an approved
approval of add_mcp_server has the same arguments (compared key by key, in any order); the name is free; the entry
passes the config's schema. Then it writes the server with its own permission left to ask, logs `dot.updated`, and
pushes the config; otherwise it logs `mcp.server.refused` with why.
2026-10-11 10:44:27 +02:00
feder-cr a33c706731 engine: add_mcp_server, a Dot adds an MCP server to its own config, as NanoClaw's agents do
The tool checks at once what it can (a name that is free in the stored config, a command or a URL, no user or
password in the URL) and writes `mcp.server.added` with the call's arguments; the host decides again and writes it.
Its permission is the table's mcp.servers.add, so the gate asks the person with the whole entry. Its description
says how to install a server's program first (uvx, `uv tool install git+https://...`, npx after Node.js) and that a
key is named in secrets, never written. The config check no longer reads mcp.servers.add as the permission of a
server named "servers.add", and the prompt says the Dot adds a server this way.
2026-10-11 10:44:27 +02:00
feder-cr 7067f0357b shared: the permission to add an MCP server, mcp.servers.add, and the events of a server the Dot adds
mcp.servers.add asks by default and is high risk; a server's name holds no `.`, so it is never a server's own
permission. `mcp.server.added {arguments}` is the guest's report of an add_mcp_server call that ran, and
`mcp.server.refused {name, reason}` the host's when it does not write the server. parseMcpServerEntry is the one
check of a server's entry, the config's own schema.
2026-10-11 10:44:26 +02:00
Federico 2c803ef5b5 Merge pull request #1441 from feder-cr/dot-mcp-servers
A Dot uses the MCP servers its person declares
2026-10-11 05:49:34 +02:00
feder-cr 72cd7d4df6 ci: the floors are what this branch's CI measured with the MCP servers' tests: vitest 2143 on Linux and 2146 on Windows, PostgreSQL 732 with 290 pg runs, pytest 2535, Playwright 88 2026-10-11 05:40:51 +02:00
feder-cr 33d82c57d7 Merge remote-tracking branch 'origin/main' into dot-mcp-servers 2026-10-11 05:33:26 +02:00
Federico 440f6837d7 Merge pull request #1440 from feder-cr/browser-host-mcp
The page tools are invisible-playwright-mcp's own, and its instructions are in the prompt
2026-10-11 05:33:21 +02:00
feder-cr 7e103cc636 Merge remote-tracking branch 'origin/browser-host-mcp' into dot-mcp-servers 2026-10-11 05:29:09 +02:00
feder-cr c5c8c07e66 ci: the pytest floor is 2507, what this branch's CI measured: the page tools are the server's now, so the 88 tests of the engine's own (its 16-tool fixture, browser_scroll, browser_screenshot, the browser argument) gave way to 84 that hold them to the server's capture, with the cases of browser_evaluate, read_html, take_screenshot and upload_files 2026-10-11 05:27:15 +02:00
feder-cr 803d03b0ac wiki: the 30-second Browser.newPage bound is reported as invisible_playwright#321 2026-10-11 05:21:22 +02:00
feder-cr 441c96d4e0 Merge remote-tracking branch 'origin/browser-host-mcp' into dot-mcp-servers 2026-10-11 05:20:27 +02:00
feder-cr 7378d76e19 Merge remote-tracking branch 'origin/main' into browser-host-mcp 2026-10-11 05:18:37 +02:00
Federico ba6227765c Merge pull request #1439 from feder-cr/browser-audit
Browser audit: back, forward and reload removed, and the calls that lead nowhere by cause
2026-10-11 05:18:23 +02:00
feder-cr a6c727a5f1 web test: the Tasks page waits for a new task by the list's length, not by a page-wide text query over two hundred cards that used up waitFor's second on a loaded machine 2026-10-11 05:14:52 +02:00
feder-cr cd7e36f190 ci: the pytest floor is 2511, what this branch's CI measured: the 33 tests that left are the cases of browser_back, browser_forward and browser_reload (24 of the approval matrix, 9 of the page tools), gone with the tools 2026-10-11 05:12:15 +02:00
feder-cr a4472f0394 wiki: MCP servers a person declares, how other hosts do it, and a live Dot that installed a server's program and used it 2026-10-11 04:47:39 +02:00
feder-cr a0c5aefb0e guide: the acceptance run passes its 16 steps, the MCP servers' step with them 2026-10-11 04:43:30 +02:00
feder-cr f542349ce7 smoke: the MCP server's secret compared by hash, per server process, never printed 2026-10-11 04:39:08 +02:00
feder-cr 46ba8330c8 smoke: the MCP server's user, environment and command lines in three checks, each saying what it found 2026-10-11 04:33:54 +02:00
feder-cr 97fea69aee smoke and e2e: look for the MCP secret on the command lines with a pattern that does not match the grep's own 2026-10-11 04:28:31 +02:00
feder-cr 4d5e59fd0d mcp servers: a start bound of their own, startup_timeout_s, and one that timed out holds up no turn after
A server that never answers would be started again at every turn, and every message would wait out the bound for
it. As Codex has `startup_timeout_sec` beside `tool_timeout_sec`, a server's entry has `startup_timeout_s` (1 to 600,
60 by default, MCP_STARTUP_TIMEOUT_BOUNDS) beside `timeout_s`; one that does not start within it is not started again
until its entry or its secrets change, and its error says so. The fork's client tests fake the SDK's stdio_client
and connect_mcp_servers with their real parameters (errlog, the failures and errlogs sinks).
2026-10-11 04:23:32 +02:00
feder-cr 6875851a43 smoke: the MCP checks read the server's environment as dot, give the missing server a permission so the prompt says why, and group the jq test 2026-10-11 04:10:19 +02:00
feder-cr c0cbb552db web e2e: an MCP server added in the settings through the real host, its secret pushed to the guest and never on the page 2026-10-11 04:09:31 +02:00
feder-cr e6ebfc79f8 tests: the MCP servers in the engine smoke and in the real-VM run, and the e2e contract that knows them
The engine smoke declares the engine's fake MCP server and a program that is not installed: through the image's
relay the first connects as dot with its secret in its environment and on no command line, its tool runs under its
permission and its instructions reach the prompt, the second fails saying what the relay found, and a config without
them stops the server. The real-VM run's new step m runs mcp-server-time through uvx: its tool asks, is approved
always and runs as dot; a second server waits for its secret, set through the CLI. Steps n and p look for that secret
as for the key.
2026-10-11 04:02:57 +02:00
feder-cr 2afd73b0fe docs: MCP servers in the architecture and the guide; no longer out of scope
Section 7 declares them and their permission, 8.3 says where they run, when they start and what the model gets, 4.3
how their secrets travel, 5.3 and 9.6 the routes, 8.8 the engine's manager. The guide has a section on them, the two
commands, and the README one row.
2026-10-11 04:02:57 +02:00
feder-cr 71bdf9db85 web: an MCP servers panel in a Dot's settings, and each server's permission among the others
The panel lists each declared server with where it is on the computer and why one is not connected, adds, edits and
removes one in the draft like any setting, and sets or clears its secrets write-only once the config that names them
is saved. Each server is a row of Permissions and tools, asked by default and high risk; a preset leaves the
servers' permissions alone, the review lists a server's change by what it runs, a change put on top of a newer config
keeps both sides' servers, and an approval of a server's tool is the kind to stop and think about.
2026-10-11 04:02:57 +02:00
feder-cr a60ee568e3 host: MCP secrets are set write-only, pushed with the key, and none outlives the config that names it
A secret is stored encrypted under the Dot's scope as mcp/<server>/<NAME>, set and cleared through
PUT and DELETE /api/dots/:id/mcp-secrets/:server/:name (the SDK, `invisible-dots secret mcp`), listed without its
value, and pushed with the key on every READY and agent.started. A config that drops a server or a name drops its
value in the same transaction. `GET /api/dots/:id/tools` answers the engine's whole answer, the servers' states with
the tools, and `invisible-dots mcp <dot>` shows them. Always allowing the permission of a server the config no
longer declares is refused (409 permission_gone).
2026-10-11 04:02:56 +02:00
feder-cr 2b354e2a29 engine: the MCP servers a person declares, started as dot through the relay, their tools behind their permission
nanobot/dots/mcp_servers.py keeps one MCPProvider per declared server, as the BrowserManager keeps one per identity.
A command runs through dot-agentd relay as dot, its secrets by the relay's environment (--env-from); a URL is reached
with its secrets as headers. A server starts when declared, restarts alone when its entry or a secret changes, and
one that failed starts again when the next turn begins, which waits for those starting (60 s each at most). Its
tools join the registry as mcp_<server>_<tool> under mcp.<server>; the prompt carries each server's instructions, or
why it is not connected, with what its stderr said and its secrets masked. The fork's client keeps a server's
initialize instructions, says why a connection failed and takes a stdio server's stderr (UPSTREAM.md). A config
stored by a release without servers declares none.
2026-10-11 04:02:56 +02:00
feder-cr 05bd036027 shared: a Dot's config declares MCP servers, each with its own permission and secrets kept apart
`mcp_servers` takes the entries Claude Code, Codex and nanobot take: a command with its args and env, or a URL with
its headers, plus `secrets`, the names of the variables or headers whose values are set apart from the config, and
`timeout_s`. Each server has a permission, `mcp.<server>`, that asks by default; one of a server the config does not
declare is refused, and denied. The engine's tool table, `approval.requested` and `GET /tools` name such a
permission, and `GET /tools` says where each server is (connecting, connected, failed and why). `POST /secrets`
carries the servers' secrets with the key; a secret's value is one rule, MCP_SECRET_PATTERN, as the key's is. Servers
are listed by name, since the database keeps a config as jsonb.
2026-10-11 04:02:56 +02:00
feder-cr 6791bd1e34 wiki: browser_evaluate measured on 0.71.1, on the same four tasks as on 0.71.0 2026-10-11 03:14:26 +02:00
feder-cr 5c9d964423 image: invisible-playwright-mcp 0.71.1, whose browser_evaluate says it comes last and names the tool for each read
In the four verification tasks on 0.71.0 browser_evaluate was the call the model made most (14 of 72), mostly to
read text, a heading or a link a reading tool returns. 0.71.1 changes only that tool's description and the rung of
the server's instructions that names it: last, and which tool reads text, markup or a control instead.
mcp-requirements.lock is regenerated with the command in its header, and nanobot/dots/invisible_playwright_mcp.json
captured again from the package installed from PyPI, so the engine offers the published wording.
2026-10-11 03:03:43 +02:00
feder-cr 26ed9228e7 smoke: the engine smoke's browser checks for a server started in host mode: no browser argument on any call, the reading tools the server has, and browser_take_screenshot under its own name 2026-10-10 23:07:36 +02:00
feder-cr 540f21b6c5 image: the engine's .json data travels with its source, so the browser server's capture reaches the runtime disk
The engine reads nanobot/dots/invisible_playwright_mcp.json at start, and the runtime disk, the wheel and the smoke's
staging took only the .py files and the .md templates: the browser smoke's engine stopped at import with no such
file. The three readers of what the package holds (pyproject's include list, runtime.ts, prepare-engine.sh) now
take the package's .json too, and runtime.test.ts holds a .json of the package on the disk and the capture among
the real engine's files.
2026-10-10 23:07:35 +02:00
feder-cr fc07373f9d image: invisible-playwright-mcp 0.71.0 and invisible-playwright 0.32.0, the versions that serve a host its page rules and main alone, with the engine's capture taken again from the published package
0.71.0 is the release the engine's page tools were built for (INVISIBLE_MCP_HOST_MANAGED, the page rules, a read
that resolves its selector through the engine); it requires invisible-playwright 0.32.0, which brings
invisible-core 39.35.0. mcp-requirements.lock is regenerated with the command in its header for both, and
nanobot/dots/invisible_playwright_mcp.json is captured again from the package installed from PyPI: only its
version changed from the capture taken from the repository, so the published instructions and tools are the ones
the engine was tested against, and the test that holds the capture to the lock passes. golden.test.ts builds from
the lock and names the new pins.
2026-10-10 23:07:35 +02:00
feder-cr 1d6827065b wiki: the browser server's own tools and words, how other hosts carry a server's tools and instructions, the host mode a Dot needs, and what it costs
The audit article said the consent rule was in the skill and that browser_read_text takes CSS; both moved to the
server (its page rules, and a read that resolves its selector through the engine), and the article says so and
links the new one.
2026-10-10 23:07:35 +02:00
feder-cr f9a21766ad engine: the page tools are invisible-playwright-mcp's own, offered as an MCP host offers a server's tools, and its instructions are in the prompt; the skill keeps what only a Dot knows, the identities
The engine rewrote every page tool of the browser server under names and descriptions of its own, and its skill
copied the server's instructions (the order to try things in, never act by script, say when a task is impossible),
then gained three rules of its own from the audit of the browser calls, all true of any page. Two copies of a rule
drift, and the copy the model read was not the one the server's authors keep. Claude Code and opencode offer a
server's tools with the server's descriptions and put its instructions in the system prompt; nanobot, the engine's
base, offers the descriptions and drops the instructions.

The server could not be passed as it was: it serves one person and a helper, `main` and `support`, and tells its
model to open them with browser_open, which a Dot's model does not have. invisible-playwright-mcp 0.71.0 reads
INVISIBLE_MCP_HOST_MANAGED=1, under which it serves `main` alone, no tool takes `browser`, and its instructions are
the page rules. The BrowserManager starts it that way (BROWSER_ENV["HOST_MANAGED"], in the shared table too) and
stops adding `browser: "main"` to every call; it still opens and closes the browser with browser_open and
browser_close, which a first launch (the engine's download, answered in steps) and a close (Firefox writing its
profile before the client ends the process) need.

The page tools are now built from nanobot/dots/invisible_playwright_mcp.json, the server's instructions and
tools/list as guest/image-builder/builder/capture-mcp-interface.py captures them from the real server started that
way: the server's name, description and input schema (normalized as nanobot's MCP client does), read-only from its
annotation, plus the identity_id of the open identity, described once on that argument. The permission table says
which the model is offered: browser_read_html, browser_evaluate and browser_upload_files come in, and
browser_take_screenshot under its own name; browser_open, browser_close, browser_list and browser_status are the
identity tools' and browser_watch the UI's frame. browser_scroll, which the engine invented over PageDown, goes, and
so does the 214-character cap on browser_type: the server answers within 45 s and goes on typing in the background.
An image the server answers with is shown, so browser_click_at returns the page after the click. The engine still
refuses a URL that is not http or https before it reaches the server. The system prompt carries the server's
instructions whenever a page tool is offered. The capture replaces tests/fixtures/mcp-tools-0.70.2.json: the fake
MCP server and the smoke serve the same file.

The server's click_at takes `number` coordinates, so store.canonical_arguments writes a whole number one way: 10
and 10.0 are one call for an approval, 10.5 stays itself (the test that forbade a number type said to do exactly
this before adding one). A null for an optional page argument is now refused by the server's schema with what it
should be, as nanobot refuses it for any MCP tool, instead of being dropped by the engine.

Measured with tiktoken, every permission allowed: the page tools' definitions went from 1632 tokens (10 tools) to
2885 (12), and the instructions add 765, about 2000 tokens more on each request that offers the browser. The fake
model of the tests published a 32000-token window, under the engine's cautious estimate of a turn with every
permission (about 34000); it now publishes 128000, the window of the smaller models Dots run on.

The capture is of the server at 0.71.0 from its repository; the lock still pins 0.70.2, so the test that holds the
capture to the lock fails until 0.71.0 is published and the lock moves to it, with the capture taken again from the
published package.
2026-10-10 23:07:35 +02:00
feder-cr 7788fbc068 smoke: GET /tools is checked against the permission table's own count, which the removal of back, forward and reload made 27, not the 30 written by hand 2026-10-10 23:04:53 +02:00
feder-cr 669a4f0350 wiki: browser calls that lead nowhere, the audit of 45 browser tasks of real Dots by cause, with what was changed for each and what was not
90 of 773 page calls led nowhere and took 961 of the 2823 seconds spent on pages. Addresses from memory are 58 of
them (27 of the 54 navigations dropped unread answer 404 today, against 1 of 40 that were read), which the
finding-a-page rule of PR #1438 is for, and its measure is written here too. Then two timeouts of the library (the
first page after a launch fails at the 30 seconds invisible_playwright gives Browser.newPage; pages over 45
seconds), reads that fail once right after a navigation, clicks under Google's consent dialog, and back, forward
and reload, which never worked. The snapshot with the navigation is measured and not adopted: a snapshot follows 7%
of the navigations that loaded, a text read 72%.
2026-10-10 18:23:12 +02:00
feder-cr b47df8936b engine: browser_back, browser_forward and browser_reload are gone, since a key pressed on the page never moves the browser's history: back and forward failed every time ("unknown key: 'Left'") and reload answered that it reloaded a page it did not; browser_navigate does both, and the skill says so
An audit of the 45 tasks of the saved runs that drove a page (773 page calls) counted browser_back three times and
three failures, and browser_reload five times and five "reloaded the page". The three tools pressed Alt+Left,
Alt+Right and F5 through browser_press_key. Run on invisible_playwright itself: Alt+Left and Alt+Right are refused
because the library's key names are ArrowLeft and ArrowRight; Alt+ArrowLeft and Alt+ArrowRight are pressed and the
page stays where it is; F5 is pressed and the page is not loaded again (a value set on the window survived it). A
key the server presses reaches the page, never the browser's own shortcuts. PageDown does scroll (0 to 913 pixels),
so browser_scroll stays, and a navigation to the page's own address loads it again (HTTP 200, the value gone).

So the three tools are removed from PAGE_TOOLS, the permission table, the web client's labels, the smoke's list of
offered tools and the architecture's table; an old event naming one is shown as the model called it. The skill says
that going back is a navigation to the address the page came from, and loading a page again a navigation to its own
address.

The same audit gave two more lines to the skill. browser_read_text takes a CSS selector, and its argument now says
so: the server reads with document.querySelector, so a snapshot selector in Playwright's syntax (:nth-match(...),
text=...) fails there, as one of the ten failed reads did. And a cookie or consent dialog is answered first: four
clicks on a Google result waited their 15 seconds under Google's dialog and failed before the model answered it
(66 seconds). On the same Google task, run once with the old skill and once with the new one on a fresh Dot
(z-ai/glm-5.3-flash), both answered the dialog before clicking, so that run shows no difference; the line is for
the four clicks of the runs.

wire_shapes.json is written again for the tool list and the skill's content.
2026-10-10 18:23:12 +02:00
Federico 37d58b616b Merge pull request #1438 from feder-cr/browser-finding
skill: the browser skill says how a page is found, never an address from memory
2026-10-10 14:46:58 +02:00
feder-cr 6815e3a253 skill: the browser skill says how a page is found: a link seen in a snapshot or a search on Brave, never an address from memory, and a 404 sends the Dot back to a page that loaded
The skill said how to act on a page and how to keep identities, and nothing about how to reach a page. A model
left alone fills that gap from memory: it writes an address it remembers, the site answers 404, it writes
another, and a task's steps go on guesses. The new section says where an address comes from (a link in
browser_snapshot, the site's own menu or search, or a search engine's results), that the status browser_navigate
answers is read before the page is trusted (the tool's own words), that after a 404 the next address comes from a
page that loaded or from a search, and that an address from memory is for a site's home page at most, once. The
skill's description names finding a page and searching the web, so a research task is one it covers.

Brave is the search engine named because it is the one that answered the Dot's own Firefox with real results: a
Dot asked to open the three engines reported Brave 200 with usable results, DuckDuckGo's html version 200 (its
result links wrapped in a redirect), and Bing 200 with an empty results area, then results unrelated to the query.
DuckDuckGo is the fallback the skill names.

Measured on one Dot (z-ai/glm-5.3-flash) asked for the exact URL and first heading of three documentation pages
whose addresses are not guessable (Tailscale SSH, Hetzner Cloud firewalls, Fly.io autostop): with the old skill
28 calls, 8 navigations, 2 addresses from memory that answered 404 before it recovered from the sites' indexes,
$0.019; with the new one 29 calls, 7 navigations, 0 addresses from memory (each page came from a Brave result or
a link of the site), $0.037, the difference being the snapshots of the result pages. Both answered correctly.

wire_shapes.json is written again for the new description and content; architecture.md's line on the skill says
what it now covers.
2026-10-10 14:36:50 +02:00
Federico 80f4709da6 Merge pull request #1437 from feder-cr/memory-incremental
engine: a memory pass sends only the messages it has not taken yet
2026-10-10 06:32:48 +02:00
feder-cr 1ece62d3e3 engine: a memory pass sends the summary model only the messages it has not taken yet: a conversation file is written again whole after every turn but only grows at its end, so a long day's chat was read again in full at every pass, at a cost that grew through the day; docs and the event's description said it read only what changed, which is now so 2026-10-10 04:38:59 +02:00
Federico e00898b9f0 Merge pull request #1436 from feder-cr/bench-lock
A LongMemEval run holds its directory; the engine drops the provider-side conversation state it never used
2026-10-10 04:38:55 +02:00
feder-cr 6a324467e7 tests: the exec relay tests no longer bet on a login shell printing within 200 to 1000 ms: a session is read until its command exits, and a command that ends is given the longest yield, which it answers within as soon as it ends (two full runs on an overloaded workstation failed them) 2026-10-10 03:41:10 +02:00
feder-cr a7d48bc94c ci: the pytest floor is 2540: the 22 tests removed with the provider-side conversation state tested only that code (test_conversation_state.py's 11, and 11 runner, consolidator and retry cases), none a live path 2026-10-10 03:07:47 +02:00
feder-cr 7406d05d0c engine: the provider-side conversation state and provider-native compaction are removed: no provider of a Dot keeps state or compacts on its side (OpenAICompatProvider overrides none of it), so every one of its paths was a no-op; request_pressure, which nothing called, goes with them 2026-10-10 03:07:47 +02:00
feder-cr 02a17d5b36 bench: a LongMemEval run or judge holds its directory, so a second one on it stops at once instead of answering and grading the same questions twice 2026-10-10 02:05:02 +02:00
617 changed files with 11208 additions and 14443 deletions

No files matched your search

+4
View File
@@ -9,6 +9,10 @@
# declared in pyproject.toml under [tool.invisible.hooks]. That pyproject
# is configuration only; nothing here is a Python package.
#
# CI alone runs the rest of tests.yml: the engine's pytest (Linux only, see
# invisible_engine_dots), the PostgreSQL half of the pg suites, the web
# client's Playwright tests, the two smokes and the claims gate.
#
# Install once with:
# git config core.hooksPath .githooks
# Bypass a known-broken WIP push with `git push --no-verify`; that is the only
+7 -7
View File
@@ -1,11 +1,11 @@
{
"//": "What each suite must reach on each host, read by .github/scripts/test-guard.mjs in CI (tests.yml) and in the pre-push hook. Raise a floor when tests are added; lowering one is a decision to write down in the commit that does it. Linux skips only the tests that drive a real Windows PowerShell; a Windows host compiles dot-agentd's unix-only test files out and skips the two Go tests that need a Linux guest. The playwright floor is the web client's browser tests (the web job of tests.yml; the same eighty-seven run on a Windows host, which is where they are developed). The pytest floor is the Python engine's (invisible_engine_dots, the engine job of tests.yml, Linux only, nothing may skip). The whatsapp floor is the suite that needs the real WhatsApp client, an opt-in install (npm run test:whatsapp, the whatsapp-client job of tests.yml, which installs the client and runs it with PostgreSQL, so the pg runs of the auth state count); a host without PostgreSQL reaches the win32 number.",
"//": "What each suite must reach on each host, read by .github/scripts/test-guard.mjs in CI (tests.yml) and in the pre-push hook. Raise a floor when tests are added; lowering one is a decision to write down in the commit that does it. Linux skips only the tests that drive a real Windows PowerShell; a Windows host compiles dot-agentd's unix-only test files out and skips the two Go tests that need a Linux guest. The playwright floor is the web client's browser tests (the web job of tests.yml; the same eighty-eight run on a Windows host, which is where they are developed). The pytest floor is the Python engine's (invisible_engine_dots, the engine job of tests.yml, Linux only, nothing may skip). The whatsapp floor is the suite that needs the real WhatsApp client, an opt-in install (npm run test:whatsapp, the whatsapp-client job of tests.yml, which installs the client and runs it with PostgreSQL, so the pg runs of the auth state count); a host without PostgreSQL reaches the win32 number.",
"vitest": {
"linux": { "min_passed": 2110, "allow_skipped": "^generated PowerShell on a real Windows PowerShell " },
"win32": { "min_passed": 2113, "allow_skipped": "" }
"linux": { "min_passed": 2209, "allow_skipped": "^generated PowerShell on a real Windows PowerShell " },
"win32": { "min_passed": 2206, "allow_skipped": "" }
},
"postgres": {
"linux": { "min_passed": 717, "allow_skipped": "", "require": { "\\bpg\\b": 283 } }
"linux": { "min_passed": 744, "allow_skipped": "", "require": { "\\bpg\\b": 295 } }
},
"whatsapp": {
"linux": { "min_passed": 25, "allow_skipped": "", "require": { "\\bpg\\b": 9 } },
@@ -16,10 +16,10 @@
"win32": { "min_passed": 70, "allow_skipped": "^agentd (TestSystemRoute|TestPowerOffStartsTheCommandAndAnswers202)$" }
},
"pytest": {
"linux": { "min_passed": 2562, "allow_skipped": "" }
"linux": { "min_passed": 2493, "allow_skipped": "" }
},
"playwright": {
"linux": { "min_passed": 87, "allow_skipped": "" },
"win32": { "min_passed": 87, "allow_skipped": "" }
"linux": { "min_passed": 88, "allow_skipped": "" },
"win32": { "min_passed": 88, "allow_skipped": "" }
}
}
+12 -5
View File
@@ -2,14 +2,20 @@ name: golden image
# Builds the golden image once, here, and publishes it as the release golden-<inputs digest>, which
# `invisible-dots image build` downloads instead of building (guest/image-builder/src/prebuilt.ts). The digest is the
# one every host computes from the pinned inputs, so a host only ever takes the image of its own inputs. A digest that
# already has its release is not built again.
# one every host computes from the pinned inputs and the newest release of the Dot's browser on the package index
# (browser-release.ts), so a host only ever takes the image of its own inputs. A digest that already has its release
# is not built again.
#
# The schedule is what makes a new release of the browser reach the hosts as a prebuilt image: no file of this
# repository changes when one is published, and the next run here finds a new digest and builds it.
on:
push:
branches: [main]
paths:
- guest/image-builder/**
- virtualization/images/**
schedule:
- cron: "17 */6 * * *"
workflow_dispatch:
permissions:
@@ -37,7 +43,7 @@ jobs:
with:
node-version: "24"
cache: npm
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: guest/dot-agentd/go.mod
- name: KVM for the builder VM, and QEMU
@@ -50,7 +56,8 @@ jobs:
- run: npm ci --no-audit --no-fund
- run: npm run build --workspace @invisible-dots/cli
# The runtime ISO, which `image build` makes first, carries dot-agentd.
- run: CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go -C guest/dot-agentd build -trimpath -o bin/dot-agentd ./cmd/dot-agentd
- working-directory: guest/dot-agentd
run: CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -buildvcs=false -ldflags="-s -w" -o bin/dot-agentd ./cmd/dot-agentd
- name: the image (downloaded instead when its release exists already)
id: build
run: |
@@ -66,4 +73,4 @@ jobs:
fi
gh release create "$tag" "$RUNNER_TEMP"/release/* \
--title "Golden image $tag" \
--notes "The golden image of the inputs ${tag#golden-}, built by the golden image workflow from ${GITHUB_SHA}. invisible-dots image build downloads it when a checkout's inputs have this digest. It holds Ubuntu 24.04 and the software guest/image-builder/pins.json and the locks name, each under its own license (THIRD_PARTY_NOTICES.md)."
--notes "The golden image of the inputs ${tag#golden-}, built by the golden image workflow from ${GITHUB_SHA}. invisible-dots image build downloads it when a checkout's inputs have this digest. It holds Ubuntu 24.04, the software guest/image-builder/pins.json and the engine's lock name, and the release of invisible-playwright-mcp its manifest records, each under its own license (THIRD_PARTY_NOTICES.md)."
+13 -5
View File
@@ -39,6 +39,14 @@ jobs:
with:
node-version: "24"
cache: npm
# Linux's vitest builds the real dot-agentd and drives it with the host's GuestClient
# (apps/vm-manager/test/agentd.linux.test.ts, which vitest.config.ts collects on Linux only).
- if: runner.os == 'Linux'
uses: actions/setup-go@v6
with:
go-version-file: guest/dot-agentd/go.mod
# The module has no dependencies, so there is no go.sum to key a cache on.
cache: false
- run: npm ci --no-audit --no-fund
- run: npm run typecheck
- name: the pre-push hook parses
@@ -169,7 +177,7 @@ jobs:
working-directory: guest/dot-agentd
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
- uses: actions/setup-go@v6
with:
go-version-file: guest/dot-agentd/go.mod
# The module has no dependencies, so there is no go.sum to key a cache on.
@@ -186,7 +194,7 @@ jobs:
- name: refuse a run with too few tests or an unexpected skip
if: always()
run: node ../../.github/scripts/test-guard.mjs go-test.json --suite go --floors ../../.github/test-floors.json
- run: CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags="-s -w" -o bin/dot-agentd ./cmd/dot-agentd
- run: CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -buildvcs=false -ldflags="-s -w" -o bin/dot-agentd ./cmd/dot-agentd
- run: file bin/dot-agentd | tee /dev/stderr | grep -q "ELF 64-bit LSB executable, x86-64.*statically linked"
engine:
@@ -217,7 +225,7 @@ jobs:
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
# Builds dot-agentd in golang:1.26, the engine's environment in ubuntu:24.04
# Builds dot-agentd in golang:1.26.8, the engine's environment in ubuntu:24.04
# the way provision.sh builds it, and runs the Dot. It exits non-zero on a
# failed or a skipped check and prints "SMOKE: N passed, F failed, S skipped".
- name: run the smoke on this checkout
@@ -229,8 +237,8 @@ jobs:
timeout-minutes: 45
steps:
- uses: actions/checkout@v4
# The golden image's browser built as provision.sh builds it (apt packages of pins.json, the hashed
# mcp-requirements.lock, the engine of the browser), a desktop on :0, and the Dot
# The golden image's browser built as provision.sh builds it (apt packages of pins.json, the newest
# invisible-playwright-mcp, the engine of the browser), a desktop on :0, and the Dot
# (dot-agentd and the engine under their two users) browsing pages of the container. It needs the
# network for those downloads and for the egress address of a launch. Same exit rule and summary
# line as the smoke job.
+7
View File
@@ -23,6 +23,13 @@ bin/
# vitest's JSON report, as CI and the pre-push hook write it
vitest-report.json
# The benchmark's tasks: tests/bench/run.sh writes them with author.py and author_long.py before every run
/tests/bench/tasks/
/tests/bench/tasks-long/
# Editor and Python leftovers of the gates
__pycache__/
.venv/
# The README animation's frames: docs/hero/build.mjs writes them on every run, from the pages and photos beside it
/docs/hero/frames/
+1
View File
@@ -60,6 +60,7 @@ Anything that needs a computer and a person's judgement, for as long as it takes
| **[A computer of its own](docs/guide.md#what-a-dot-can-do)** | A hardware-accelerated VM with a Linux desktop and a persistent disk. It sleeps when idle and wakes for the next message, task or automation. |
| **[A browser that is not blocked](docs/guide.md#the-browser)** | [invisible_playwright_mcp](https://github.com/feder-cr/invisible_playwright_mcp): Firefox patched in C++, the fingerprint set inside the engine. Each identity keeps its own cookies and logins. |
| **[Memory and skills](docs/guide.md#what-a-dot-can-do)** | It writes its own notes and how-tos in its home folder, as Claude Code does, and reads them on the next task. |
| **[Your MCP servers](docs/guide.md#mcp-servers)** | Any MCP server you declare, a program on its computer or a URL, as in Claude Code. Its keys stay out of the config. |
| **[Permissions you set](docs/guide.md#approvals)** | Every tool belongs to a permission: allow, ask or deny. An ask waits in your Inbox, survives a restart and runs the call once. |
| **[Tasks and automations](docs/guide.md#the-web-ui)** | A queue with priorities and start times, and its own recurring jobs, for which its computer is started on time. |
| **[Many ways to reach it](docs/guide.md#talk-to-it-from-your-phone)** | Web UI, command line, HTTP API with live events, Telegram, and WhatsApp as an opt-in. |
+1 -1
View File
@@ -266,7 +266,7 @@ SOFTWARE.
## OpenDots (CopilotKit)
`apps/web/src/lib/time.ts`, `apps/web/src/lib/use-visible-interval.ts`,
`apps/web/src/lib/events/tool-labels.ts` and
`packages/shared/src/labels.ts` and
`apps/web/src/components/tasks/task-status.tsx` derive from OpenDots by CopilotKit
(https://github.com/CopilotKit/OpenDots, commit 88f2a08); each file's first
comment says what was changed.
+1 -7
View File
@@ -6,13 +6,7 @@
*/
import { randomBytes } from "node:crypto";
import { MASTER_KEY_BYTES } from "@invisible-dots/database";
import { DEFAULT_LISTEN, ENV, MIN_API_TOKEN_LENGTH, readApiToken, readSecretFile, writeSecretFile, type HostPaths } from "@invisible-dots/shared";
/** The same variable the CLI and the web server read, so one value configures all three. */
export const API_TOKEN_ENV = ENV.TOKEN;
/** Tokens shorter than this are refused (the one rule of shared readApiToken). */
export const MIN_TOKEN_LENGTH = MIN_API_TOKEN_LENGTH;
import { DEFAULT_LISTEN, ENV, readApiToken, readSecretFile, writeSecretFile, type HostPaths } from "@invisible-dots/shared";
/** A generated token: 32 random bytes as hex, so it survives copy and paste and any shell quoting. */
const GENERATED_TOKEN_BYTES = 32;
+1 -1
View File
@@ -4,7 +4,7 @@
* migrated, and checked against a master key that was lost.
*/
import { Database, databaseTarget, describeDatabaseTarget } from "@invisible-dots/database";
import { errorMessage, type Logger } from "@invisible-dots/scheduler";
import { errorMessage, type Logger } from "@invisible-dots/shared";
export interface OpenDatabaseOptions {
env: Record<string, string | undefined>;
+31 -33
View File
@@ -6,13 +6,14 @@
import { createHash, timingSafeEqual } from "node:crypto";
import { TELEGRAM_TOKEN_SECRET, type ChannelHub } from "@invisible-dots/channels";
import { StreamOverflowError } from "@invisible-dots/events";
import { ControlPlaneError, errorMessage, silentLogger, type Logger, type Scheduler } from "@invisible-dots/scheduler";
import { type Scheduler } from "@invisible-dots/scheduler";
import { STREAM_ERROR_EVENT } from "@invisible-dots/sdk";
import type {
ApprovalsAnswer,
ChannelPairingAnswer,
ChannelRecord,
ChannelsAnswer,
EveryChannelAnswer,
DoctorAnswer,
DotsAnswer,
EventsAnswer,
@@ -24,26 +25,14 @@ import type {
TasksAnswer,
UsageAnswer,
} from "@invisible-dots/sdk/types";
import {
APPROVAL_LIST_LIMIT,
CONVERSATION_LIST_LIMIT,
APPROVAL_STATUSES,
CHANNEL_KINDS,
LIST_ORDERS,
MAX_EVENT_PAGE,
TASK_LIST_LIMIT,
type ApprovalStatus,
type ChannelKind,
type DoctorCheck,
type ListOrder,
type SkillListAnswer,
type ToolListAnswer,
} from "@invisible-dots/shared";
import { APPROVAL_LIST_LIMIT, APPROVAL_STATUSES, type ApprovalStatus, CHANNEL_KINDS, type ChannelKind, ControlPlaneError, CONVERSATION_LIST_LIMIT, type DoctorCheck, errorMessage, LIST_ORDERS, type ListOrder, type Logger, MAX_EVENT_PAGE, type McpSecretsAnswer, silentLogger, type SkillListAnswer, TASK_LIST_LIMIT, type ToolListAnswer } from "@invisible-dots/shared";
import { doctorAnswer } from "@invisible-dots/vm-manager";
import { serveFile } from "./file-types.js";
import Fastify, { type FastifyInstance, type FastifyReply, type FastifyRequest } from "fastify";
import apiPackage from "../package.json" with { type: "json" };
export const API_VERSION = "0.1.0";
/** The version `GET /api/health` reports: this package's, as the CLI's `--version` reports its own. */
export const API_VERSION: string = apiPackage.version;
/** A comment line on idle SSE connections, so proxies and clients see the connection is alive. */
export const SSE_HEARTBEAT_MS = 15_000;
@@ -223,9 +212,7 @@ export function buildServer(options: ServerOptions): FastifyInstance {
// Messages and tasks
app.post<{ Params: Params }>("/api/dots/:id/messages", async (request, reply) => {
const text = bodyOf(request).text;
if (typeof text !== "string") throw bad("text must be a string");
return reply.code(202).send(await scheduler.sendMessage(request.params.id, text));
return reply.code(202).send(await scheduler.sendMessage(request.params.id, bodyOf(request).text));
});
// The conversation pages like the event log: oldest first by default, `order=desc` the newest, `before` (the
@@ -260,6 +247,11 @@ export function buildServer(options: ServerOptions): FastifyInstance {
return { tasks: await scheduler.listTasks(request.params.id, { limit, before }) };
});
// The tasks of every Dot that ended in a state after a time, the latest end first (the web's failed tasks of the day).
app.get<{ Querystring: { status?: unknown; finished_after?: unknown } }>("/api/tasks", async (request): Promise<TasksAnswer> => ({
tasks: await scheduler.listEndedTasks(request.query.status, request.query.finished_after),
}));
app.get<{ Params: Params }>("/api/tasks/:id", async (request) => scheduler.getTask(request.params.id));
app.post<{ Params: Params }>("/api/tasks/:id/cancel", async (request) => scheduler.cancelTask(request.params.id));
@@ -327,7 +319,7 @@ export function buildServer(options: ServerOptions): FastifyInstance {
app.get<{ Params: Params }>(
"/api/dots/:id/tools",
async (request): Promise<ToolListAnswer> => ({ tools: await scheduler.listTools(request.params.id) }),
async (request): Promise<ToolListAnswer> => scheduler.listTools(request.params.id),
);
app.get<{ Params: Params }>(
@@ -342,6 +334,9 @@ export function buildServer(options: ServerOptions): FastifyInstance {
async (request): Promise<ChannelsAnswer> => ({ channels: await channels.list(request.params.id), available: channels.kinds }),
);
// Every channel of every Dot, each with its Dot's id (the web's shell watches the ones that need the person).
app.get("/api/channels", async (): Promise<EveryChannelAnswer> => ({ channels: await channels.listEvery() }));
// Link the Dot to a Telegram bot, or give the linked one a new token (a revoked token is the only way back from needs_relink).
app.put<{ Params: Params }>("/api/dots/:id/channels/telegram", async (request, reply): Promise<ChannelRecord> => {
const token = bodyOf(request).token;
@@ -412,12 +407,7 @@ export function buildServer(options: ServerOptions): FastifyInstance {
for (const decision of ["approve", "reject"] as const) {
app.post<{ Params: Params }>(`/api/approvals/:id/${decision}`, async (request) => {
const { note, always } = bodyOf(request);
if (note !== undefined && typeof note !== "string") throw bad("note must be a string");
if (always !== undefined && always !== true) throw bad("always must be true");
return scheduler.resolveApproval(request.params.id, decision, {
...(note !== undefined ? { note } : {}),
...(always !== undefined ? { always } : {}),
});
return scheduler.resolveApproval(request.params.id, decision, { note, always });
});
}
@@ -475,12 +465,7 @@ export function buildServer(options: ServerOptions): FastifyInstance {
});
app.get<{ Querystring: { dot_id?: string; after?: string } }>("/api/stream", async (request, reply) => {
let dotId: string | undefined;
if (request.query.dot_id) {
const dot = await scheduler.db.dots.resolve(request.query.dot_id);
if (!dot && !request.query.dot_id.includes("_")) throw new ControlPlaneError(404, "not_found", `Dot "${request.query.dot_id}" not found`);
dotId = dot?.id ?? request.query.dot_id;
}
const dotId = request.query.dot_id ? await scheduler.historyDotId(request.query.dot_id) : undefined;
// EventSource sends the last id it saw when it reconnects; it wins over the query.
const lastEventId = request.headers["last-event-id"];
const after = intParam(typeof lastEventId === "string" ? lastEventId : request.query.after, "after");
@@ -512,6 +497,19 @@ export function buildServer(options: ServerOptions): FastifyInstance {
app.put<{ Params: { id: string } }>("/api/dots/:id/proxy", async (request) => scheduler.setVmProxy(request.params.id, bodyOf(request).value));
app.delete<{ Params: { id: string } }>("/api/dots/:id/proxy", async (request) => scheduler.setVmProxy(request.params.id, null));
// The secrets of a Dot's MCP servers: write-only, the answers say only which are set.
type McpSecretParams = { id: string; server: string; name: string };
app.get<{ Params: Params }>("/api/dots/:id/mcp-secrets", async (request): Promise<McpSecretsAnswer> => scheduler.mcpSecrets(request.params.id));
app.put<{ Params: McpSecretParams }>(
"/api/dots/:id/mcp-secrets/:server/:name",
async (request): Promise<McpSecretsAnswer> => // A PUT sets: a null value is no value (a clear is the DELETE).
scheduler.setMcpSecret(request.params.id, request.params.server, request.params.name, bodyOf(request).value ?? undefined),
);
app.delete<{ Params: McpSecretParams }>(
"/api/dots/:id/mcp-secrets/:server/:name",
async (request): Promise<McpSecretsAnswer> => scheduler.setMcpSecret(request.params.id, request.params.server, request.params.name, null),
);
return app;
}
+2 -2
View File
@@ -16,8 +16,8 @@ import {
} from "@invisible-dots/channels";
import type { Database } from "@invisible-dots/database";
import type { HostFacts } from "@invisible-dots/sdk/types";
import { errorMessage, prefixedStderrLogger, Scheduler, type ComputerDriver, type Logger, type SchedulerOptions } from "@invisible-dots/scheduler";
import { ensureDir, ENV, hostPaths, type HostPaths } from "@invisible-dots/shared";
import { Scheduler, type ComputerDriver, type SchedulerOptions } from "@invisible-dots/scheduler";
import { ensureDir, ENV, errorMessage, hostPaths, type HostPaths, type Logger, prefixedStderrLogger } from "@invisible-dots/shared";
import { hostDoctorDeps, openRouterCheck, runDoctor, type DoctorDeps } from "@invisible-dots/vm-manager";
import { loadOrCreateApiToken, loadOrGenerateMasterKey, parseListen, saveMasterKey, type ListenAddress } from "./config.js";
import { assertNothingEncrypted, openControlPlaneDatabase } from "./database.js";
+4 -22
View File
@@ -9,20 +9,10 @@
* - every start uses the newest `runtime-<version>.iso`, so a new version of
* our code reaches a Dot with a restart (section 3.3).
*/
import { access, readdir } from "node:fs/promises";
import { readdir } from "node:fs/promises";
import { join } from "node:path";
import type {
ComputerDriver,
ComputerSpecInput,
ComputerState,
CreatedComputer,
GuestApi,
GuestEndpoint,
Logger,
StartedComputer,
WaitForHealthOptions,
} from "@invisible-dots/scheduler";
import { imageLabel, imageVersionOf, type HealthAnswer, type HostPaths, type ImageKind } from "@invisible-dots/shared";
import { type ComputerDriver, type ComputerSpecInput, type ComputerState, type CreatedComputer, type GuestApi, type GuestEndpoint, type StartedComputer, type WaitForHealthOptions } from "@invisible-dots/scheduler";
import { type HealthAnswer, type HostPaths, type ImageKind, imageLabel, imageVersionOf, type Logger, pathExists } from "@invisible-dots/shared";
import { VmManager, type StartVmResult, type VmSpec } from "@invisible-dots/vm-manager";
/** Compare versions such as "2026.10.1" and "2026.9.3" numerically, part by part. */
@@ -122,7 +112,7 @@ export class VmManagerDriver implements ComputerDriver {
}
async state(dotId: string): Promise<ComputerState> {
const [info, exists] = await Promise.all([this.#vm.state(dotId), fileExists(this.#vm.paths.diskPath(dotId))]);
const [info, exists] = await Promise.all([this.#vm.state(dotId), pathExists(this.#vm.paths.diskPath(dotId))]);
return {
exists,
state: info.state,
@@ -140,14 +130,6 @@ export class VmManagerDriver implements ComputerDriver {
async close(): Promise<void> {}
}
async function fileExists(path: string): Promise<boolean> {
try {
await access(path);
return true;
} catch {
return false;
}
}
/** The driver `invisible-dots server` uses: VmManager over this host's QEMU and INVISIBLE_DOTS_HOME. */
export function createVmDriver(options: { env: Record<string, string | undefined>; paths: HostPaths; logger: Logger }): VmManagerDriver {
+61 -3
View File
@@ -349,7 +349,8 @@ describe.each(testAdapters())("control-plane API (%s)", (kind) => {
expect((await fetch(`${base}/api/dots/${dot.id}/tools`)).status).toBe(401);
// Tools: the table, and what the model is offered follows the permissions the config pushed.
const none = await api.listTools(dot.id);
const { tools: none, mcp_servers } = await api.listTools(dot.id);
expect(mcp_servers).toEqual([]);
expect(none.map((t) => t.name)).toEqual([
"exec",
"read_file",
@@ -366,8 +367,8 @@ describe.each(testAdapters())("control-plane API (%s)", (kind) => {
files.write: deny
automations: deny
`);
await waitFor(async () => (await api.listTools(dot.id)).some((t) => !t.offered), "permissions pushed");
expect(Object.fromEntries((await api.listTools(dot.id)).map((t) => [t.name, t.offered]))).toEqual({
await waitFor(async () => (await api.listTools(dot.id)).tools.some((t) => !t.offered), "permissions pushed");
expect(Object.fromEntries((await api.listTools(dot.id)).tools.map((t) => [t.name, t.offered]))).toEqual({
exec: true,
read_file: true,
write_file: false,
@@ -546,6 +547,44 @@ describe.each(testAdapters())("control-plane API (%s)", (kind) => {
expect(await get("limit=200&before=nobody")).toBe(200);
});
it("every Dot in one read: the tasks that failed since a time, latest first, and every channel with its Dot", async () => {
const a = await readyDot("every-a");
const b = await readyDot("every-b");
const since = new Date(Date.now() - 1);
const failed = async (dotId: string, description: string) => {
const guest = driver.guestOf(dotId);
guest.onInbound = () => {};
const task = await api.createTask(dotId, { description });
await waitFor(async () => (await db.tasks.runs(task.id))[0]?.delivered_at, "delivered");
guest.emit("task.failed", { task_id: task.id, error: "boom" });
await waitFor(async () => (await api.getTask(task.id)).status === "FAILED", "failed");
return task.id;
};
const first = await failed(a.id, "first");
const second = await failed(b.id, "second");
const ours = (tasks: { id: string }[]) => tasks.map((t) => t.id).filter((id) => id === first || id === second);
expect(ours(await api.endedTasks("FAILED", since))).toEqual([second, first]);
expect(ours(await api.endedTasks("COMPLETED", since))).toEqual([]);
expect(await api.endedTasks("FAILED", new Date(Date.now() + 60_000))).toEqual([]);
const get = async (query: string) => (await fetch(`${base}/api/tasks?${query}`, { headers: { authorization: `Bearer ${TOKEN}` } })).status;
expect(await get("status=RUNNING&finished_after=2026-01-01T00:00:00Z")).toBe(400);
expect(await get("status=FAILED&finished_after=yesterday")).toBe(400);
expect(await get("status=FAILED")).toBe(400);
await channels.add(a.id, "telegram");
await channels.add(b.id, "telegram");
const every = (await api.everyChannel()).filter((c) => c.dot_id === a.id || c.dot_id === b.id);
expect(every.map((c) => [c.dot_id, c.kind])).toEqual([
[a.id, "telegram"],
[b.id, "telegram"],
]);
expect(JSON.stringify(every)).not.toMatch(/token/i);
// The channels run until they are removed: this test stops what it started, so no later test or teardown inherits them.
await channels.remove(a.id, "telegram");
await channels.remove(b.id, "telegram");
expect((await api.everyChannel()).filter((c) => c.dot_id === a.id || c.dot_id === b.id)).toEqual([]);
});
it("messages: the newest page of a long conversation, and a cursor that goes back through the rest", async () => {
const dot = await readyDot("chatty");
for (const [index, word] of ["one", "two", "three"].entries()) {
@@ -638,6 +677,25 @@ describe.each(testAdapters())("control-plane API (%s)", (kind) => {
expect((await create({ name: "Number", proxy: 8080 })).status).toBe(400);
});
it("an MCP server's secrets are set and cleared through the API and the SDK, and no answer carries a value", async () => {
const dot = await api.createDot(`${yaml("mcp-api")}mcp_servers:\n web:\n url: https://web.example/mcp\n secrets: [Authorization]\n`);
await waitUntilSettledReady(scheduler, driver, dot.id, "mcp-api");
const route = `${base}/api/dots/${dot.id}/mcp-secrets/web/Authorization`;
expect((await fetch(route, { method: "PUT", body: JSON.stringify({ value: "x" }), headers: { "content-type": "application/json" } })).status).toBe(401);
expect((await fetch(`${base}/api/dots/${dot.id}/mcp-secrets`)).status).toBe(401);
const set = await api.setMcpSecret(dot.id, "web", "Authorization", "Bearer api-1");
expect(set).toEqual({ dot_id: dot.id, secrets: [{ server: "web", name: "Authorization", set: true }] });
expect(driver.guestOf(dot.id).mcpSecrets).toEqual({ web: { Authorization: "Bearer api-1" } });
expect(JSON.stringify(await api.mcpSecrets(dot.id))).not.toContain("api-1");
// A PUT with no value is refused, not taken as a clear; the DELETE clears.
const empty = await fetch(route, { method: "PUT", headers: { authorization: `Bearer ${TOKEN}`, "content-type": "application/json" }, body: JSON.stringify({ value: null }) });
expect(empty.status).toBe(400);
expect((await api.setMcpSecret(dot.id, "web", "Authorization", null)).secrets[0]!.set).toBe(false);
await expect(api.setMcpSecret(dot.id, "nope", "Authorization", "x")).rejects.toMatchObject({ status: 404 });
});
it("an identity's frame is a JPEG of an open one, and close ends it, through the API and the SDK", async () => {
const dot = await readyDot("viewing");
const identity = await api.createIdentity(dot.id, { name: "Main account" });
+2 -2
View File
@@ -9,7 +9,7 @@ import { createHmac } from "node:crypto";
import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http";
import type { AddressInfo, Socket } from "node:net";
import type { FakeGuest } from "@invisible-dots/scheduler/testing";
import { GUEST_PROOF_CONTEXT } from "@invisible-dots/shared";
import { GUEST_PROOF_CONTEXT, type SecretsRequest } from "@invisible-dots/shared";
export interface FakeGuestServer {
port: number;
@@ -94,7 +94,7 @@ export async function serveFakeGuest(
response.writeHead(200, { "content-type": "image/png" }).end(Buffer.from(await current.screenshot()));
return;
case "POST /v1/agent/secrets":
await current.pushSecrets(((await readJson(request)) as { openrouter_api_key: string }).openrouter_api_key);
await current.pushSecrets((await readJson(request)) as SecretsRequest);
return send(response, 204);
case "PUT /v1/agent/config":
await current.putConfig((await readJson(request)) as never);
+8 -4
View File
@@ -166,11 +166,15 @@ describe("startServer on an empty INVISIBLE_DOTS_HOME", { timeout: 120_000 }, ()
};
const server = await start({ doctor: { ...base, acceleratorAccess } });
const api = new InvisibleDotsClient({ baseUrl: server.url, token: server.token });
let arrived = 0;
server.app.server.on("request", (request: { url?: string }) => {
if (request.url === "/api/doctor") arrived += 1;
});
const together = Promise.all([api.doctor(), api.doctor(), api.doctor()]);
await waitFor(() => probes === 1, "the first request to start the probe");
// Time for the other two requests to reach the server while the probe is held.
await new Promise((resolve) => setTimeout(resolve, 200));
await waitFor(() => probes === 1 && arrived === 3, "all three requests at the server while the probe is held");
// A request's hooks and handler run on promises once it has arrived: one turn of the event loop has them joined.
await new Promise((resolve) => setImmediate(resolve));
release();
const [first, second, third] = await together;
expect(probes).toBe(1);
@@ -200,7 +204,7 @@ describe("startServer on an empty INVISIBLE_DOTS_HOME", { timeout: 120_000 }, ()
await first.db.dots.insert({
id: dotId,
config: parseDotConfig("name: channeled\nmodel:\n provider: openrouter\n id: test/model\n"),
status: "DISABLED",
status: "IDLE",
});
await first.channels.add(dotId, "telegram");
await stopped(first);
+5 -4
View File
@@ -10,10 +10,10 @@ import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { createTestDatabase, type TestDatabase } from "@invisible-dots/database/testing";
import { prefixedStderrLogger, Scheduler } from "@invisible-dots/scheduler";
import { Scheduler } from "@invisible-dots/scheduler";
import { FAKE_MAX_IDENTITIES, FakeGuest, waitFor } from "@invisible-dots/scheduler/testing";
import { computerResources, hostPaths, parseDotConfig, type HostPaths } from "@invisible-dots/shared";
import { silentLogger, VmManager } from "@invisible-dots/vm-manager";
import { computerResources, hostPaths, type HostPaths, parseDotConfig, prefixedStderrLogger, silentLogger } from "@invisible-dots/shared";
import { VmManager } from "@invisible-dots/vm-manager";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { FakeQemuHost, FakeRunner, type FakeVm } from "../../vm-manager/test/fakes.js";
import { compareVersions, latestImage, VmManagerDriver } from "../src/index.js";
@@ -160,7 +160,8 @@ describe("VmManagerDriver over VmManager", () => {
const endpoint = { dotId: DOT, port: started.guestPort };
const health = await driver.waitForHealth(endpoint, TOKEN, { timeoutMs: 2_000, intervalMs: 5, requestTimeoutMs: 1_000 });
expect(health).toMatchObject({ agentd: "ok", agent: { status: "ok" } });
await driver.guest(endpoint, TOKEN).pushSecrets("sk-or-x");
await driver.guest(endpoint, TOKEN).pushSecrets({ openrouter_api_key: "sk-or-x", mcp_secrets: { time: { TOKEN: "t-1" } } });
expect(guest?.mcpSecrets).toEqual({ time: { TOKEN: "t-1" } });
expect(guest?.openrouterKey).toBe("sk-or-x");
// Another token cannot check the guest's proof, so it is never sent; waiting does not help.
await expect(driver.guest(endpoint, "wrong-token-0000").health()).rejects.toMatchObject({ code: "guest_unproven" });
+162 -49
View File
@@ -12,9 +12,25 @@ import { readFile, writeFile } from "node:fs/promises";
import { resolve } from "node:path";
import { parseArgs } from "node:util";
import QRCode from "qrcode";
import { ApiError, type DotSummary, type InvisibleDotsClient, type TaskRecord } from "@invisible-dots/sdk";
import { CHANNEL_KINDS, DEFAULT_WEB_LISTEN, ENV, type ChannelKind, type ChannelRecord, type ComputerAnswer, type StoredEvent } from "@invisible-dots/shared";
import { ApiError, readRecentEvents, type ApprovalRecord, type DotSummary, type InvisibleDotsClient, type TaskRecord } from "@invisible-dots/sdk";
import {
accountLabel,
CHANNEL_KINDS,
COMPUTER_STOPPED,
DEFAULT_WEB_LISTEN,
ENV,
MAX_EVENT_PAGE,
permissionPhrase,
toolLabel,
type ChannelKind,
type ChannelRecord,
type ComputerAnswer,
type McpSecretsAnswer,
type StoredEvent,
type ToolListAnswer,
} from "@invisible-dots/shared";
import { STORE_OPENROUTER_KEY } from "@invisible-dots/vm-manager";
import cliPackage from "../package.json" with { type: "json" };
import { apiUrl, AuthSetupError, connectApi, DEFAULT_URL } from "./api-client.js";
import { addTelegramChannel } from "./commands.js";
import { EXIT } from "./exit.js";
@@ -22,7 +38,11 @@ import { EXIT } from "./exit.js";
export { DEFAULT_URL } from "./api-client.js";
export { EXIT } from "./exit.js";
export const CLI_VERSION = "0.1.0";
/** The version of this package, which the bundle carries inside it (esbuild inlines the JSON). */
export const CLI_VERSION: string = cliPackage.version;
/** How many tasks `status` shows: the newest, which is all it asks the server for. */
const STATUS_TASKS = 10;
/**
* The commands that act on this host. The real ones (host.ts) load QEMU
@@ -83,15 +103,19 @@ Getting this host ready (the same commands on Linux and Windows):
Using the server:
invisible-dots init [file] [--force] write a sample Dot config (default dot.yaml), check the server
invisible-dots create <file.yaml> create a Dot from a YAML config
invisible-dots update <dot> <file.yaml> replace the Dot's config with a YAML config; refused if it was changed since this command read it
invisible-dots delete <dot> --yes delete the Dot: its computer, its disk with everything on it, and its channels (cannot be undone)
invisible-dots list list Dots
invisible-dots status <dot> show a Dot, its computer and its recent tasks
invisible-dots message <dot> <text...> send a chat message
invisible-dots task <dot> <description...> [--priority N] [--at ISO-8601]
queue a task
invisible-dots tasks <dot> list a Dot's tasks
invisible-dots cancel <task-id> cancel a task that has not finished
invisible-dots computer <dot> start|stop|reboot
invisible-dots browser <dot> identities list the Dot's browser identities
invisible-dots approvals [--all] list pending (or all) approvals
invisible-dots mcp <dot> list the MCP servers the Dot's config declares: where each is on its computer, and which secrets are set
invisible-dots approvals [--all] list pending approvals (--all: every approval, the newest first)
invisible-dots approve <approval-id> [--note text] [--always]
--always also allows that permission for the Dot from now on
invisible-dots reject <approval-id> [--note text]
@@ -99,19 +123,21 @@ Using the server:
store the OpenRouter key: asked for in a terminal, read from stdin when piped (never from arguments)
invisible-dots secret proxy --dot <dot> [--clear]
set the Dot's VM proxy (socks5://user:pass@host:port), used from its next start; asked for like the key; --clear removes it
invisible-dots secret mcp --dot <dot> <server> <name> [--clear]
set a secret an MCP server of the Dot names (an environment variable or a header, e.g. "Bearer <token>"); asked for like the key; --clear removes it
invisible-dots channel add telegram --dot <dot>
link the Dot to a Telegram bot: the token from @BotFather is asked for in a terminal, read from stdin when piped (never from arguments)
invisible-dots channel link whatsapp --dot <dot>
link a WhatsApp number (opt-in on the server, unofficial, a ban of the account is possible): shows a code to scan in WhatsApp > Linked devices
invisible-dots channel list [--dot <dot>] list channels (of every Dot without --dot) with their status and paired people
invisible-dots channel pair telegram --dot <dot>
print a one-time link (valid 10 minutes) that pairs your Telegram account to the Dot
invisible-dots channel remove telegram --dot <dot>
unlink: the token and the paired people are deleted
invisible-dots channel pair telegram|whatsapp --dot <dot>
print a one-time link (valid 10 minutes) that pairs your Telegram or WhatsApp account to the Dot
invisible-dots channel remove telegram|whatsapp --dot <dot>
unlink: the token (or the linked device's keys) and the paired people are deleted
invisible-dots logs <dot> [--tail N] [--no-follow]
print recent events, then follow new ones
<dot> is a Dot name or id. Add --json for machine-readable output.
<dot> is a Dot name or id. Add --json for machine-readable output. invisible-dots --version prints the version.
Environment:
${ENV.HOME} the data directory (default ~/.invisible-dots)
@@ -159,6 +185,7 @@ const OPTIONS = {
compress: { type: "boolean" },
all: { type: "boolean" },
clear: { type: "boolean" },
yes: { type: "boolean" },
help: { type: "boolean", short: "h" },
version: { type: "boolean", short: "v" },
} as const;
@@ -241,12 +268,6 @@ const PAIRING_WORDS: Record<ChannelKind, { action: string; to: string }> = {
whatsapp: { action: "press Send", to: "the number" },
};
/** How a channel's account reads: a Telegram bot as @name, the number linked to WhatsApp as +number. */
function accountLabel(channel: ChannelRecord): string {
if (channel.account === null) return "-";
return channel.kind === "whatsapp" ? `+${channel.account}` : `@${channel.account}`;
}
function channelRows(rows: { dot: string; channel: ChannelRecord }[]): string {
if (rows.length === 0) return "no channels: link one with invisible-dots channel add telegram --dot <dot>\n";
return pad([
@@ -255,22 +276,72 @@ function channelRows(rows: { dot: string; channel: ChannelRecord }[]): string {
dot,
c.kind,
c.enabled ? c.status : "paused",
accountLabel(c),
accountLabel(c.kind, c.account) ?? "-",
c.peers.length === 0 ? "-" : c.peers.map((p) => p.label).join(", "),
c.status_detail ? oneLine(c.status_detail, 80) : "",
]),
]);
}
/** The approvals in the words of the web client's approval card: the Dot by its name, what it wants to do, the permission with its risk. */
function approvalRows(approvals: ApprovalRecord[], dotNames: ReadonlyMap<string, string>): string {
return pad([
["ID", "STATUS", "DOT", "WANTS TO", "PERMISSION", "REASON"],
// A Dot that was deleted has no name any more: its id is what is left of it.
...approvals.map((a) => [a.id, a.status, dotNames.get(a.dot_id) ?? a.dot_id, toolLabel(a.tool).ask, permissionPhrase(a.permission), oneLine(a.reason)]),
]);
}
/** A Dot config read from a YAML file named on the command line. */
async function readConfigFile(io: CliIo, path: string): Promise<string> {
const file = resolve(io.cwd, path);
try {
return await readFile(file, "utf8");
} catch (error) {
throw new UsageError(`cannot read ${file}: ${(error as NodeJS.ErrnoException).code ?? (error as Error).message}`);
}
}
/** The newest `count` events of the Dot's log, oldest first: read from the end, a page at a time, never from its start. */
async function newestEvents(client: InvisibleDotsClient, dotId: string, count: number): Promise<StoredEvent[]> {
const found: StoredEvent[] = [];
let before: number | undefined;
while (found.length < count) {
const want = Math.min(count - found.length, MAX_EVENT_PAGE);
const page = await readRecentEvents(client, dotId, { count: want, before });
found.unshift(...page);
if (page.length < want) break;
before = page[0]!.id;
}
return found;
}
/** One line of a secret, from the terminal (echo off) or from stdin: never from the arguments, which end up in shell history and ps. */
async function secretFromInput(io: CliIo, what: { prompt: string; noun: string; command: string }): Promise<string> {
async function secretFromInput(io: CliIo, what: { prompt: string; noun: string; command: string; spaces?: true }): Promise<string> {
if (io.stdinIsTTY) io.stderr(`paste the ${what.prompt}, then press Enter:\n`);
const value = (io.stdinIsTTY ? await io.readSecret() : await io.readStdin()).trim();
if (!value) throw new UsageError(`no ${what.noun} given; run "${what.command}" in a terminal and paste it, or pipe it in`);
if (/\s/.test(value)) throw new UsageError(`the ${what.noun} on stdin contains whitespace; pass only the ${what.noun}`);
// A header's value may be "Bearer <token>"; a key or a proxy never has a space, and a line break is never part of one.
if (what.spaces ? /[\r\n]/.test(value) : /\s/.test(value)) {
throw new UsageError(`the ${what.noun} on stdin contains ${what.spaces ? "a line break" : "whitespace"}; pass only the ${what.noun}`);
}
return value;
}
function mcpRows(tools: ToolListAnswer | null, secrets: McpSecretsAnswer): string {
const servers = new Set([...(tools?.mcp_servers ?? []).map((server) => server.name), ...secrets.secrets.map((secret) => secret.server)]);
if (servers.size === 0 && tools !== null) return "no MCP servers\n";
const state = (name: string) => {
if (tools === null) return "computer stopped";
const server = tools.mcp_servers.find((s) => s.name === name);
if (server === undefined) return "-";
return server.state === "connected" ? `connected, ${server.tools} tools` : server.state === "failed" ? `failed: ${oneLine(server.error ?? "", 80)}` : server.state;
};
const secretsOf = (name: string) =>
secrets.secrets.filter((secret) => secret.server === name).map((secret) => `${secret.name}${secret.set ? "" : " (not set)"}`).join(", ") || "-";
return pad([["SERVER", "STATE", "SECRETS"], ...[...servers].map((name) => [name, state(name), secretsOf(name)])]);
}
function channelKind(value: string): ChannelKind {
if (!(CHANNEL_KINDS as readonly string[]).includes(value)) throw new UsageError(`unknown channel "${value}": use ${CHANNEL_KINDS.join(" or ")}`);
return value as ChannelKind;
@@ -336,17 +407,37 @@ export async function run(argv: string[], io: CliIo): Promise<number> {
return EXIT.ok;
}
case "create": {
const file = resolve(io.cwd, need(args, 0, "<file.yaml>"));
let yaml: string;
try {
yaml = await readFile(file, "utf8");
} catch (error) {
throw new UsageError(`cannot read ${file}: ${(error as NodeJS.ErrnoException).code ?? (error as Error).message}`);
}
const yaml = await readConfigFile(io, need(args, 0, "<file.yaml>"));
const dot = await (await api()).createDot(yaml);
out(dot, `created Dot ${dot.name} (${dot.id}); its computer is being provisioned.\nfollow it with: invisible-dots logs ${dot.name}\n`);
return EXIT.ok;
}
case "update": {
const name = need(args, 0, "<dot>");
const yaml = await readConfigFile(io, need(args, 1, "<file.yaml>"));
if (args.length > 2) throw new UsageError("update takes a Dot and one YAML file");
const c = await api();
// The version read here makes the save conditional: a config changed meanwhile (in the web client, by an
// "always allow") is a 409 and nothing is saved, instead of being overwritten unseen.
const current = await c.getDot(name);
const dot = await c.updateDot(current.id, yaml, current.config_version);
out(dot, `updated Dot ${dot.name} (config version ${dot.config_version})\n`);
return EXIT.ok;
}
case "delete": {
const name = need(args, 0, "<dot>");
noArguments(args.slice(1), "delete <dot>");
if (values.yes !== true) {
throw new UsageError(
`delete destroys Dot ${name}'s computer and its disk, with every file, memory note and browser identity on them, and its channels; it cannot be undone. To do it: invisible-dots delete ${name} --yes`,
);
}
const c = await api();
const dot = await c.getDot(name);
const answer = await c.deleteDot(dot.id);
out(answer, `deleting Dot ${dot.name}; what it did stays in its log: invisible-dots logs ${dot.id}\n`);
return EXIT.ok;
}
case "list": {
const dots = await (await api()).listDots();
out(dots, dotRows(dots));
@@ -355,7 +446,7 @@ export async function run(argv: string[], io: CliIo): Promise<number> {
case "status": {
const name = need(args, 0, "<dot>");
const c = await api();
const [dot, computer, tasks] = await Promise.all([c.getDot(name), c.computer(name), c.listTasks(name)]);
const [dot, computer, tasks] = await Promise.all([c.getDot(name), c.computer(name), c.listTasks(name, { limit: STATUS_TASKS })]);
const text =
pad([
["name", dot.name],
@@ -368,7 +459,7 @@ export async function run(argv: string[], io: CliIo): Promise<number> {
["next automation", nextAutomation(computer)],
]) +
"\n" +
taskRows(tasks.slice(0, 10));
taskRows(tasks);
out({ dot, computer, tasks }, text);
return EXIT.ok;
}
@@ -410,6 +501,13 @@ export async function run(argv: string[], io: CliIo): Promise<number> {
out(tasks, taskRows(tasks));
return EXIT.ok;
}
case "cancel": {
const id = need(args, 0, "<task-id>");
noArguments(args.slice(1), "cancel <task-id>");
const task = await (await api()).cancelTask(id);
out(task, `task ${task.id} ${task.status}\n`);
return EXIT.ok;
}
case "computer": {
const name = need(args, 0, "<dot>");
const action = need(args, 1, "start|stop|reboot");
@@ -439,17 +537,13 @@ export async function run(argv: string[], io: CliIo): Promise<number> {
return EXIT.ok;
}
case "approvals": {
const approvals = await (await api()).listApprovals(values.all ? undefined : "pending");
const c = await api();
// With --all the newest come first, as in the web client's history, so the cap of the list drops the oldest and
// never the newest; the pending ones come oldest first, the one that has waited longest on top.
const [approvals, dots] = await Promise.all([values.all ? c.listApprovals(undefined, { order: "desc" }) : c.listApprovals("pending"), c.listDots()]);
out(
approvals,
approvals.length === 0
? values.all
? "no approvals\n"
: "no pending approvals\n"
: pad([
["ID", "STATUS", "DOT", "TOOL", "REASON"],
...approvals.map((a) => [a.id, a.status, a.dot_id, a.tool, oneLine(a.reason)]),
]),
approvals.length === 0 ? (values.all ? "no approvals\n" : "no pending approvals\n") : approvalRows(approvals, new Map(dots.map((d) => [d.id, d.name]))),
);
return EXIT.ok;
}
@@ -466,8 +560,33 @@ export async function run(argv: string[], io: CliIo): Promise<number> {
out(approval, `approval ${approval.id} ${approval.status}\n`);
return EXIT.ok;
}
case "mcp": {
const name = need(args, 0, "<dot>");
const c = await api();
const secrets = await c.mcpSecrets(name);
let tools: ToolListAnswer | null = null;
try {
tools = await c.listTools(name);
} catch (error) {
// A stopped computer has no engine to ask where the servers are; their secrets are the host's.
if (!(error instanceof ApiError && error.code === COMPUTER_STOPPED)) throw error;
}
out({ mcp_servers: tools?.mcp_servers ?? null, secrets: secrets.secrets }, mcpRows(tools, secrets));
return EXIT.ok;
}
case "secret": {
const kind = need(args, 0, "openrouter|proxy");
const kind = need(args, 0, "openrouter|proxy|mcp");
if (kind === "mcp") {
if (values.dot === undefined || values.dot === "") throw new UsageError("missing --dot <dot>");
const server = need(args, 1, "<server>");
const secretName = need(args, 2, "<name>");
if (args.length > 3) throw new UsageError("the secret is read from stdin, never from arguments (they end up in shell history and ps)");
const command = `invisible-dots secret mcp --dot ${values.dot} ${server} ${secretName}`;
const value = values.clear ? null : await secretFromInput(io, { prompt: `value of ${secretName} for the MCP server ${server}`, noun: "secret", command, spaces: true });
const result = await (await api()).setMcpSecret(values.dot, server, secretName, value);
out(result, `${secretName} of the MCP server ${server} ${value === null ? "cleared" : "stored"} for Dot ${result.dot_id}; a running Dot starts the server again with it\n`);
return EXIT.ok;
}
if (kind === "proxy") {
if (values.dot === undefined || values.dot === "") throw new UsageError("missing --dot <dot>");
if (args.length > 1) throw new UsageError("the proxy is read from stdin, never from arguments (they end up in shell history and ps)");
@@ -476,7 +595,7 @@ export async function run(argv: string[], io: CliIo): Promise<number> {
out(result, result.proxy ? `VM proxy stored for Dot ${result.dot_id}; it is used from the Dot's next start\n` : `Dot ${result.dot_id} goes out directly from its next start\n`);
return EXIT.ok;
}
if (kind !== "openrouter") throw new UsageError(`unknown secret "${kind}": openrouter or proxy`);
if (kind !== "openrouter") throw new UsageError(`unknown secret "${kind}": openrouter, proxy or mcp`);
if (args.length > 1) {
throw new UsageError("the key is read from stdin, never from arguments (they end up in shell history and ps)");
}
@@ -529,7 +648,7 @@ export async function run(argv: string[], io: CliIo): Promise<number> {
if (!values.json) io.stdout(`${await QRCode.toString(frame.code, { type: "terminal", small: true })}\n`);
} else if (frame.state === "linked") {
if (!values.json) {
io.stdout(`linked WhatsApp${frame.account ? ` number +${frame.account}` : ""} to Dot ${name}\nnext: invisible-dots channel pair whatsapp --dot ${name}\n`);
io.stdout(`linked WhatsApp${frame.account ? ` number ${accountLabel("whatsapp", frame.account)}` : ""} to Dot ${name}\nnext: invisible-dots channel pair whatsapp --dot ${name}\n`);
}
return EXIT.ok;
} else if (frame.state === "failed") {
@@ -583,16 +702,10 @@ export async function run(argv: string[], io: CliIo): Promise<number> {
if (error instanceof ApiError && error.status === 404 && name.includes("_")) return { id: name };
throw error;
});
let recent: StoredEvent[] = [];
let after = 0;
for (;;) {
const page = await c.events(dot.id, { after, limit: 1000 });
recent = [...recent, ...page].slice(-Math.max(tail, 1));
if (page.length < 1000) break;
after = page.at(-1)!.id;
}
const last = recent.at(-1)?.id ?? after;
for (const event of tail === 0 ? [] : recent) io.stdout(values.json ? `${JSON.stringify(event)}\n` : `${formatEvent(event)}\n`);
// The newest `tail` events, and at least the newest one, which following goes on from.
const recent = await newestEvents(c, dot.id, Math.max(tail, 1));
const last = recent.at(-1)?.id ?? 0;
for (const event of recent.slice(recent.length - tail)) io.stdout(values.json ? `${JSON.stringify(event)}\n` : `${formatEvent(event)}\n`);
if (values["no-follow"]) return EXIT.ok;
for await (const event of c.stream({
dotId: dot.id,
-1
View File
@@ -172,7 +172,6 @@ export function realHostCommands(): HostCommands {
web: () => buildWeb(builds),
images: () => imageBuild(io),
env: io.env,
tokenPath: hostPaths(io.env).apiTokenPath,
...(io.signal ? { signal: io.signal } : {}),
});
},
+1 -2
View File
@@ -6,8 +6,7 @@
* and a stop closes the web client first, then the control plane.
*/
import type { ListenAddress, ListenSetting, StartServerOptions } from "@invisible-dots/api";
import { DEFAULT_WEB_LISTEN, ENV, type HostPaths } from "@invisible-dots/shared";
import type { Logger } from "@invisible-dots/vm-manager";
import { DEFAULT_WEB_LISTEN, ENV, type HostPaths, type Logger } from "@invisible-dots/shared";
import { locateWebBuild, type WebServer, type WebServerOptions } from "./web.js";
/** What `serve` needs from the packages that own the processes; tests pass fakes. */
+1 -5
View File
@@ -27,8 +27,6 @@ export interface SetupAllDeps {
/** `image build`: the exit code, or a rejection with the reason. */
images(): Promise<number>;
env: Record<string, string | undefined>;
/** Where `invisible-dots server` writes the API token, which the web client asks for. */
tokenPath: string;
/** Aborted by Ctrl-C. Checked between steps: only `image build` can stop in the middle of one. */
signal?: AbortSignal;
}
@@ -86,9 +84,7 @@ export async function runSetupAll(deps: SetupAllDeps): Promise<number> {
out(`
This computer is ready. To use it:
1. start the server and leave it running: invisible-dots server
2. open ${webAddress(deps.env)} in your browser and sign in with the first line of
${deps.tokenPath}
(the server creates that file the first time it starts)
2. open ${webAddress(deps.env)} in your browser
3. the page asks for your OpenRouter key, then lets you create your first Dot.
`);
return EXIT.ok;
+2 -14
View File
@@ -7,7 +7,7 @@
*/
import { stat } from "node:fs/promises";
import { dirname, join } from "node:path";
import { allowlistedEnvironment, BASE_CHILD_ENV_VARS, ENV } from "@invisible-dots/shared";
import { abortableSleep, allowlistedEnvironment, BASE_CHILD_ENV_VARS, ENV } from "@invisible-dots/shared";
import { startProcess, WEB_BUILD_COMMAND, type StartedProcess, type StartProcessOptions, type WebBuild } from "@invisible-dots/vm-manager";
/**
@@ -97,18 +97,6 @@ function answers(url: string): Promise<boolean> {
);
}
function sleep(ms: number, signal: AbortSignal | undefined): Promise<void> {
return new Promise((resolve) => {
const timer = setTimeout(done, ms);
function done() {
clearTimeout(timer);
signal?.removeEventListener("abort", done);
resolve();
}
signal?.addEventListener("abort", done, { once: true });
});
}
/**
* Start the web server and resolve once it answers a request. Throws
* WebStartError, with the child already gone, when the build is incomplete,
@@ -167,7 +155,7 @@ export async function startWebServer(options: WebServerOptions): Promise<WebServ
started = true;
return { url, stop };
}
await sleep(READY_POLL_MS, options.signal);
await abortableSleep(READY_POLL_MS, options.signal);
}
} catch (error) {
await stop();
+164 -7
View File
@@ -5,6 +5,7 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import { parseDotConfig } from "@invisible-dots/shared";
import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest";
import cliPackage from "../package.json" with { type: "json" };
import { commandOf, EXIT, interruptIsAsked, run, SAMPLE_DOT, type CliIo, type HostCommands } from "../src/index.js";
const TOKEN = "cli-test-token-0123456789";
@@ -71,7 +72,7 @@ const channel = {
peers: [{ peer_id: "10", role: "owner", label: "Ann (@ann)", created_at: now }],
created_at: now,
};
const events = Array.from({ length: 5 }, (_, i) => ({
let events = Array.from({ length: 5 }, (_, i) => ({
id: i + 1,
dot_id: dot.id,
type: i % 2 ? "agent.state" : "automation.next_run",
@@ -92,6 +93,10 @@ let noAutomationDue = false;
let whatsappLinked = false;
let whatsappOff = false;
let linkFrames: { state: string; [key: string]: unknown }[] = [];
/** The config_version the fake server holds: an update that expects another one is the 409 of a concurrent change. */
let configVersion = 1;
/** Another client saves the Dot's config between a read and the PATCH that follows it. */
let savedMeanwhile = false;
function send(res: ServerResponse, status: number, body?: unknown) {
res.writeHead(status, { "content-type": "application/json" });
@@ -123,7 +128,17 @@ async function handle(req: IncomingMessage, res: ServerResponse) {
case "GET /api/dots":
return send(res, 200, { dots: [dot] });
case `GET /api/dots/${dot.id}`:
return send(res, 200, dot);
return send(res, 200, { ...dot, config_version: configVersion });
case `PATCH /api/dots/${dot.id}`:
if (savedMeanwhile) configVersion++;
if (body?.expected_config_version !== configVersion) return send(res, 409, { error: "dot_changed", message: "the Dot's config changed since it was read" });
return send(res, 200, { ...dot, config_version: configVersion + 1 });
case `DELETE /api/dots/${dot.id}`:
return send(res, 202, { accepted: true });
case "POST /api/tasks/task_02/cancel":
return send(res, 200, { ...task, id: "task_02", status: "CANCELLED" });
case "POST /api/tasks/task_01/cancel":
return send(res, 409, { error: "task_finished", message: "task task_01 is already COMPLETED" });
case `GET /api/dots/${dot.id}/computer`:
return send(res, 200, { ...(stoppedByPerson ? { ...computer, state: "STOPPED", stop_reason: "user", ready: false } : computer), ...(noAutomationDue ? { next_automation_at: null } : {}) });
case `GET /api/dots/${dot.id}/tasks`:
@@ -154,6 +169,21 @@ async function handle(req: IncomingMessage, res: ServerResponse) {
return send(res, 200, { id: "apr_1", status: "rejected", note: body?.note ?? null });
case "PUT /api/secrets/openrouter":
return send(res, 200, { pushed: 2 });
case `GET /api/dots/${dot.id}/mcp-secrets`:
return send(res, 200, { dot_id: dot.id, secrets: [{ server: "web", name: "Authorization", set: false }] });
case `PUT /api/dots/${dot.id}/mcp-secrets/web/Authorization`:
return send(res, 200, { dot_id: dot.id, secrets: [{ server: "web", name: "Authorization", set: true }] });
case `DELETE /api/dots/${dot.id}/mcp-secrets/web/Authorization`:
return send(res, 200, { dot_id: dot.id, secrets: [{ server: "web", name: "Authorization", set: false }] });
case `GET /api/dots/${dot.id}/tools`:
if (stopped) return send(res, 409, { error: "computer_stopped", message: "the computer of Dot fare-watch is STOPPED; start it first" });
return send(res, 200, {
tools: [],
mcp_servers: [
{ name: "time", state: "connected", error: null, tools: 2 },
{ name: "web", state: "failed", error: "its secret Authorization is not set", tools: 0 },
],
});
case `GET /api/dots/${dot.id}/channels`:
return send(res, 200, { channels: whatsappLinked ? [channel, { ...channel, kind: "whatsapp", account: "15550001111", peers: [] }] : [channel] });
case `PUT /api/dots/${dot.id}/channels/telegram`:
@@ -181,8 +211,12 @@ async function handle(req: IncomingMessage, res: ServerResponse) {
case `DELETE /api/dots/${dot.id}/channels/whatsapp`:
return send(res, 204);
case `GET /api/dots/${dot.id}/events`: {
// The route's paging: after and before bound the ids, order=desc gives the newest first, limit keeps that many.
const after = Number(url.searchParams.get("after") ?? 0);
return send(res, 200, { events: events.filter((e) => e.id > after) });
const before = Number(url.searchParams.get("before") ?? Infinity);
const kept = events.filter((e) => e.id > after && e.id < before);
const ordered = url.searchParams.get("order") === "desc" ? [...kept].reverse() : kept;
return send(res, 200, { events: ordered.slice(0, Number(url.searchParams.get("limit") ?? 1000)) });
}
case "GET /api/stream": {
res.writeHead(200, { "content-type": "text/event-stream" });
@@ -220,6 +254,8 @@ beforeEach(() => {
whatsappLinked = false;
whatsappOff = false;
linkFrames = [];
configVersion = 1;
savedMeanwhile = false;
});
async function cli(
@@ -252,7 +288,7 @@ describe("usage", () => {
expect((await cli(["launch"])).stderr).toMatch(/unknown command "launch"/);
expect((await cli(["list", "--bogus"])).code).toBe(EXIT.usage);
expect((await cli(["status"])).stderr).toMatch(/missing <dot>/);
expect((await cli(["--version"])).stdout).toMatch(/^\d+\.\d+\.\d+\n$/);
expect((await cli(["--version"])).stdout).toBe(`${cliPackage.version}\n`);
});
});
@@ -320,6 +356,8 @@ describe("commands", () => {
expect(status.stdout).toMatch(/computer\s+RUNNING, ready/);
expect(status.stdout).toMatch(/next automation\s+2030-01-01T09:00:00.000Z/);
expect(status.stdout).toContain("task_01");
// It shows the newest ten tasks, and asks for ten.
expect(requests.find((r) => r.path === "/api/dots/fare-watch/tasks")?.query.get("limit")).toBe("10");
expect((await cli(["tasks", "fare-watch"])).stdout).toContain("COMPLETED");
});
@@ -373,10 +411,20 @@ describe("commands", () => {
});
it("approvals, approve and reject", async () => {
expect((await cli(["approvals"])).stdout).toContain("apr_1");
expect(requests.at(-1)?.query.get("status")).toBe("pending");
const pending = await cli(["approvals"]);
expect(pending.stdout).toContain("apr_1");
const listed = requests.find((r) => r.path === "/api/approvals");
expect(listed?.query.get("status")).toBe("pending");
expect(listed?.query.get("order")).toBeNull();
// The Dot by its name, and the question and the permission in the web client's words.
expect(pending.stdout.split("\n")[0]).toMatch(/^ID\s+STATUS\s+DOT\s+WANTS TO\s+PERMISSION\s+REASON$/);
expect(pending.stdout).toMatch(/apr_1\s+pending\s+fare-watch\s+delete a browser identity\s+Delete browser identities \(high risk\)\s+cleanup/);
requests.length = 0;
await cli(["approvals", "--all"]);
expect(requests.at(-1)?.query.get("status")).toBeNull();
const all = requests.find((r) => r.path === "/api/approvals");
expect(all?.query.get("status")).toBeNull();
// The newest first, so the cap of the list drops the oldest and never the newest.
expect(all?.query.get("order")).toBe("desc");
const approved = await cli(["approve", "apr_1", "--note", "fine"]);
expect(approved.stdout).toBe("approval apr_1 approved\n");
expect(requests.at(-1)?.body).toEqual({ note: "fine" });
@@ -404,6 +452,36 @@ describe("commands", () => {
expect((await cli(["secret", "openrouter"], { stdin: " " })).code).toBe(EXIT.usage);
});
it("secret mcp reads the value from stdin only, keeps a space a header needs, and --clear removes it", async () => {
const stored = await cli(["secret", "mcp", "--dot", "fare-watch", "web", "Authorization"], { stdin: "Bearer tok-1\n" });
expect(stored.code).toBe(EXIT.ok);
expect(stored.stdout).toContain("Authorization of the MCP server web stored");
expect(requests.at(-1)).toMatchObject({ method: "PUT", body: { value: "Bearer tok-1" } });
expect(stored.stdout).not.toContain("tok-1");
const cleared = await cli(["secret", "mcp", "--dot", "fare-watch", "web", "Authorization", "--clear"]);
expect(cleared.stdout).toContain("cleared");
expect(requests.at(-1)?.method).toBe("DELETE");
const inArgs = await cli(["secret", "mcp", "--dot", "fare-watch", "web", "Authorization", "tok-1"]);
expect(inArgs.code).toBe(EXIT.usage);
expect(inArgs.stderr).toMatch(/never from arguments/);
expect((await cli(["secret", "mcp", "web", "Authorization"], { stdin: "x" })).stderr).toContain("missing --dot");
});
it("mcp lists the declared servers with where each is and which secrets are set, and still the secrets of a stopped Dot", async () => {
const listed = await cli(["mcp", "fare-watch"]);
expect(listed.code).toBe(EXIT.ok);
expect(listed.stdout).toMatch(/time\s+connected, 2 tools\s+-/);
expect(listed.stdout).toMatch(/web\s+failed: its secret Authorization is not set\s+Authorization \(not set\)/);
stopped = true;
try {
const off = await cli(["mcp", "fare-watch"]);
expect(off.code).toBe(EXIT.ok);
expect(off.stdout).toMatch(/web\s+computer stopped\s+Authorization \(not set\)/);
} finally {
stopped = false;
}
});
it("secret openrouter in a terminal reads one line, which Enter ends on every host", async () => {
// A terminal never reaches end of input by itself, and the key that ends it differs by host:
// only the line is read, so the command never waits for it.
@@ -546,6 +624,9 @@ describe("commands", () => {
const lines = tail.stdout.trim().split("\n");
expect(lines).toHaveLength(2);
expect(lines[0]).toMatch(/#4 agent\.state \{"state":"IDLE"\}$/);
// The log is read from its end, the newest two in one request, not paged from its start.
const reads = requests.filter((r) => r.path === `/api/dots/${dot.id}/events`);
expect(reads.map((r) => [r.query.get("order"), r.query.get("limit"), r.query.get("after")])).toEqual([["desc", "2", null]]);
const controller = new AbortController();
let output = "";
@@ -569,6 +650,76 @@ describe("commands", () => {
expect(stream?.query.get("after")).toBe("5");
expect(stream?.query.get("dot_id")).toBe(dot.id);
});
it("logs --tail 0 prints nothing old and follows from the newest event, which it reads alone", async () => {
const quiet = await cli(["logs", "fare-watch", "--tail", "0", "--no-follow"]);
expect(quiet.code).toBe(EXIT.ok);
expect(quiet.stdout).toBe("");
expect(requests.filter((r) => r.path === `/api/dots/${dot.id}/events`).map((r) => r.query.get("limit"))).toEqual(["1"]);
});
it("logs --tail beyond a page reads the newest pages from the end, older one by one, and stops at the start of the log", async () => {
const saved = events;
events = Array.from({ length: 2500 }, (_, i) => ({ ...saved[0]!, id: i + 1 }));
try {
const tail = await cli(["logs", "fare-watch", "--tail", "2200", "--no-follow"]);
const lines = tail.stdout.trim().split("\n");
expect(lines).toHaveLength(2200);
expect(lines[0]).toContain("#301 ");
expect(lines.at(-1)).toContain("#2500 ");
const reads = requests.filter((r) => r.path === `/api/dots/${dot.id}/events`);
expect(reads.map((r) => [r.query.get("limit"), r.query.get("before")])).toEqual([
["1000", null],
["1000", "1501"],
["200", "501"],
]);
requests.length = 0;
expect((await cli(["logs", "fare-watch", "--tail", "9000", "--no-follow"])).stdout.trim().split("\n")).toHaveLength(2500);
expect(requests.filter((r) => r.path === `/api/dots/${dot.id}/events`)).toHaveLength(3);
} finally {
events = saved;
}
});
it("cancel cancels a task, and shows the server's 409 for one that has finished", async () => {
const cancelled = await cli(["cancel", "task_02"]);
expect(cancelled.code).toBe(EXIT.ok);
expect(cancelled.stdout).toBe("task task_02 CANCELLED\n");
expect(requests.at(-1)).toMatchObject({ method: "POST", path: "/api/tasks/task_02/cancel" });
const finished = await cli(["cancel", "task_01"]);
expect(finished.code).toBe(EXIT.failed);
expect(finished.stderr).toMatch(/already COMPLETED \[409 task_finished\]/);
expect((await cli(["cancel"])).stderr).toMatch(/missing <task-id>/);
});
it("delete refuses without --yes and says what it destroys, and deletes with it", async () => {
const refused = await cli(["delete", "fare-watch"]);
expect(refused.code).toBe(EXIT.usage);
expect(refused.stderr).toContain("cannot be undone");
expect(refused.stderr).toContain("invisible-dots delete fare-watch --yes");
expect(requests).toHaveLength(0);
const deleted = await cli(["delete", "fare-watch", "--yes"]);
expect(deleted.code).toBe(EXIT.ok);
expect(requests.at(-1)).toMatchObject({ method: "DELETE", path: `/api/dots/${dot.id}` });
expect(deleted.stdout).toBe(`deleting Dot fare-watch; what it did stays in its log: invisible-dots logs ${dot.id}\n`);
});
it("update sends the YAML with the config version it read, so a change made meanwhile is a 409 and nothing is saved", async () => {
await writeFile(join(configDir, "changed.yaml"), SAMPLE_DOT);
configVersion = 4;
const updated = await cli(["update", "fare-watch", "changed.yaml"]);
expect(updated.code).toBe(EXIT.ok);
expect(updated.stdout).toBe("updated Dot fare-watch (config version 5)\n");
expect(requests.at(-1)).toMatchObject({ method: "PATCH", path: `/api/dots/${dot.id}`, body: { config: SAMPLE_DOT, expected_config_version: 4 } });
// The web client saved between this command's read and its write.
savedMeanwhile = true;
const raced = await cli(["update", "fare-watch", "changed.yaml"]);
expect(raced.code).toBe(EXIT.failed);
expect(raced.stderr).toMatch(/\[409 dot_changed\]/);
expect((await cli(["update", "fare-watch", "missing.yaml"])).code).toBe(EXIT.usage);
expect((await cli(["update", "fare-watch"])).stderr).toMatch(/missing <file.yaml>/);
});
});
describe("host commands", () => {
@@ -632,6 +783,12 @@ describe("host commands", () => {
expect(failed.stderr).toBe("invisible-dots: another invisible-dots server (pid 42) is already running\n");
});
it("lists both channels where a command takes either", async () => {
const help = (await cli(["--help"])).stdout;
expect(help).toContain("invisible-dots channel pair telegram|whatsapp --dot <dot>");
expect(help).toContain("invisible-dots channel remove telegram|whatsapp --dot <dot>");
});
it("lists setup --all in the help, first among the host commands", async () => {
const help = (await cli(["--help"])).stdout;
expect(help).toMatch(/invisible-dots setup --all .*everything in one run/);
+2 -4
View File
@@ -5,7 +5,6 @@ import { runSetupAll, SETUP_ALL_COMMAND, webAddress, type SetupAllDeps } from ".
import type { StepResult } from "../src/setup/build.js";
import type { HostOutcome } from "../src/setup/setup.js";
const TOKEN_PATH = "/home/someone/.invisible-dots/config/api.token";
const DONE: StepResult = { ok: true, lines: ["done"] };
/** A run of `setup --all` against fake steps; `calls` is the order they ran in. */
@@ -19,7 +18,6 @@ async function all(overrides: Partial<SetupAllDeps> = {}) {
web: async () => (calls.push("web"), DONE),
images: async () => (calls.push("images"), EXIT.ok),
env: {},
tokenPath: TOKEN_PATH,
...overrides,
};
const code = await runSetupAll(deps);
@@ -34,8 +32,8 @@ describe("setup --all", () => {
expect(result.text).toMatch(/\[1\/4\] the guest daemon[\s\S]*\[2\/4\] QEMU[\s\S]*\[3\/4\] the web client[\s\S]*\[4\/4\] the guest images/);
expect(result.text).toContain("built /repo/guest/dot-agentd/bin/dot-agentd");
expect(result.text).toContain("1. start the server and leave it running: invisible-dots server");
expect(result.text).toContain(`open http://${DEFAULT_WEB_LISTEN} in your browser and sign in with the first line of\n ${TOKEN_PATH}`);
expect(result.text).toContain("the server creates that file the first time it starts");
expect(result.text).toContain(`2. open http://${DEFAULT_WEB_LISTEN} in your browser\n`);
expect(result.text).not.toMatch(/sign in|api\.token/);
expect(result.text).toContain("OpenRouter key");
});
+1 -1
View File
@@ -6,7 +6,7 @@
* inbound delivery's job (inbound.ts).
*/
import type { Database } from "@invisible-dots/database";
import { errorMessage, type Clock, type Logger } from "./support.js";
import { errorMessage, type Clock, type Logger } from "@invisible-dots/shared";
// A claim skips a row another transaction holds (SKIP LOCKED); such holds last as long as one guest event's
// transaction. A pass that claimed nothing while a task could be claimed comes back this soon, this many times;
+2 -1
View File
@@ -15,6 +15,7 @@ import type {
InboundEvent,
OutboundEvent,
RefusedEvent,
SecretsRequest,
SystemAnswer,
SkillListAnswer,
ToolListAnswer,
@@ -31,7 +32,7 @@ import type {
export interface GuestApi {
health(options?: { timeoutMs?: number; signal?: AbortSignal }): Promise<HealthAnswer>;
system(): Promise<SystemAnswer>;
pushSecrets(openrouterApiKey: string): Promise<void>;
pushSecrets(secrets: SecretsRequest): Promise<void>;
putConfig(config: DotRuntimeConfig): Promise<void>;
postEvent(event: InboundEvent): Promise<unknown>;
state(): Promise<AgentStateAnswer>;
+59 -3
View File
@@ -1,9 +1,21 @@
/**
* What a guest event changes on the host besides the event log: task rows (state, summary, error, spend),
* approval rows, the Dot's status and the time of its next automation.
* approval rows, the Dot's status, the time of its next automation, and the MCP servers of its config.
*/
import type { Repositories } from "@invisible-dots/database";
import { TERMINAL_TASK_STATES, type AgentState, type DotState, type OutboundEvent } from "@invisible-dots/shared";
import {
isMcpServerName,
parseDotConfig,
parseMcpServerEntry,
resolvePermission,
TERMINAL_TASK_STATES,
type AgentState,
type DotRecord,
type DotState,
type HostEventDataMap,
type OutboundEvent,
type PermissionName,
} from "@invisible-dots/shared";
/**
* The Dot status for an agent state. READY means the computer is up and the
@@ -22,11 +34,53 @@ export function dotStatusForAgent(state: AgentState): DotState {
}
/** Dot states the agent's own reports never override: they are set by the control plane. */
const STICKY_DOT_STATES: readonly DotState[] = ["CREATING", "ERROR", "DISABLED", "IDLE"];
const STICKY_DOT_STATES: readonly DotState[] = ["CREATING", "ERROR", "IDLE"];
export interface AppliedGuestEvent {
/** A task reached a terminal state, so the Dot may take the next one. */
taskSettled: boolean;
/** The Dot's config changed (an MCP server it added): it is pushed to the guest once the transaction commits. */
reconfigured?: DotRecord;
/** What the host logs of it, in the same transaction: `dot.updated`, or why it refused (`mcp.server.refused`). */
hostEvents?: HostEventOf[];
}
type HostEventOf = { [K in keyof HostEventDataMap]: { type: K; data: HostEventDataMap[K] } }[keyof HostEventDataMap];
/** The tool of the engine that asks to add an MCP server; its approvals are the person's consent to one. */
export const ADD_MCP_SERVER_TOOL = "add_mcp_server";
const ADD_PERMISSION: PermissionName = "mcp.servers.add";
/**
* A Dot added an MCP server to its own config. The config is the host's, so the host decides again what the engine
* decided: the permission allows it, or the person approved a call of exactly these arguments, and that approval is
* used up here, so it adds one server once. The entry is checked by the config's own schema first (a call refused for
* it keeps its approval), and a name already declared is not replaced. The server's own permission is left to its
* default, ask, so each of its tools still asks until the person allows it.
*/
async function addMcpServer(tx: Repositories, dotId: string, args: Record<string, unknown>): Promise<AppliedGuestEvent> {
const name = typeof args.name === "string" ? args.name : "";
const refuse = (reason: string): AppliedGuestEvent => ({ taskSettled: false, hostEvents: [{ type: "mcp.server.refused", data: { name, reason } }] });
const dot = await tx.dots.get(dotId);
if (!dot) return refuse("the Dot no longer exists");
const decision = resolvePermission(dot.config, ADD_PERMISSION);
if (decision === "deny") return refuse(`the Dot's config denies ${ADD_PERMISSION}`);
if (!isMcpServerName(name)) return refuse(`"${name}" is not an MCP server's name`);
if (name in dot.config.mcp_servers) return refuse(`the Dot's config already declares an MCP server "${name}"`);
const { name: _name, reason: _reason, ...entry } = args;
const parsed = parseMcpServerEntry(entry);
if (!parsed.ok) return refuse(parsed.problem);
const config = parseDotConfig({ ...dot.config, mcp_servers: { ...dot.config.mcp_servers, [name]: parsed.server } });
let approvalId: string | undefined;
if (decision === "ask") {
const approval = await tx.approvals.useApproved(dotId, ADD_MCP_SERVER_TOOL, args);
if (!approval) return refuse(`${ADD_PERMISSION} asks, and no unused approval of exactly these arguments was given`);
approvalId = approval.id;
}
const reconfigured = await tx.dots.updateConfig(dotId, config);
if (!reconfigured) return refuse("the Dot no longer exists");
const added = { name: dot.name, mcp_server_added: name, ...(approvalId && { approval_id: approvalId }) };
return { taskSettled: false, reconfigured, hostEvents: [{ type: "dot.updated", data: added }] };
}
/** The spend a task event reports (architecture section 5.4), onto the task's row; events without it change nothing. */
@@ -70,6 +124,8 @@ export async function applyGuestEvent(tx: Repositories, dotId: string, event: Ou
if (event.data.task_id && !ended) await tx.tasks.transition(event.data.task_id, "WAITING_APPROVAL", { dotId });
return { taskSettled: false };
}
case "mcp.server.added":
return addMcpServer(tx, dotId, event.data.arguments);
default:
return { taskSettled: false };
}
+33 -10
View File
@@ -13,9 +13,10 @@
* nothing waits for a timer once the guest is back.
*/
import { TASK_ENDED_BEFORE_DELIVERY, type Database, type InboundRecord } from "@invisible-dots/database";
import type { EventLog } from "@invisible-dots/events";
import { guestErrorCode, guestErrorStatus } from "./driver.js";
import type { Lifecycle } from "./lifecycle.js";
import { errorMessage, type Clock, type Logger } from "./support.js";
import { errorMessage, type Clock, type Logger } from "@invisible-dots/shared";
export interface InboundDeliveryOptions {
/** Failed sends with a known outcome before a task.created gives its task up as FAILED. */
@@ -31,6 +32,7 @@ export const DEFAULT_INBOUND_OPTIONS: InboundDeliveryOptions = {
export interface InboundDeliveryDeps {
db: Database;
events: EventLog;
lifecycle: Lifecycle;
clock: Clock;
logger: Logger;
@@ -51,8 +53,12 @@ function knownUndelivered(error: unknown): boolean {
return guestErrorStatus(error) >= 400 || (error as { code?: unknown })?.code === "ECONNREFUSED";
}
/** Rolls back a give-up whose task ended before it: the task keeps that end and logs no failure. */
class TaskAlreadyEnded extends Error {}
export class InboundDelivery {
readonly #db: Database;
readonly #events: EventLog;
readonly #lifecycle: Lifecycle;
readonly #clock: Clock;
readonly #log: Logger;
@@ -68,6 +74,7 @@ export class InboundDelivery {
constructor(deps: InboundDeliveryDeps) {
this.#db = deps.db;
this.#events = deps.events;
this.#lifecycle = deps.lifecycle;
this.#clock = deps.clock;
this.#log = deps.logger;
@@ -245,17 +252,33 @@ export class InboundDelivery {
this.#schedule(row.dot_id, this.#opts.retryDelayMs * failures);
}
/** Drop a row for good; a task.created takes its task with it. */
/**
* Drop a row for good; a task.created takes its task with it, logged as `task.failed` in the same transaction like
* every task state change. A task that ended meanwhile (cancelled) keeps its end: only the row is dropped.
*/
async #giveUp(row: InboundRecord, reason: string): Promise<void> {
let settled = false;
await this.#db.transaction(async (tx) => {
await tx.inbound.drop(row.event.id, reason);
if (row.event.type === "task.created" && row.task_id) {
settled = (await tx.tasks.transition(row.task_id, "FAILED", { error: reason })) !== null;
const taskId = row.event.type === "task.created" ? row.task_id : null;
if (taskId) {
const logged = await this.#db
.transaction(async (tx) => {
// The event first: its insert takes the event-order lock (database events.ts).
const event = await this.#events.appendHostIn(tx, row.dot_id, "task.failed", { task_id: taskId, error: reason });
if (!(await tx.tasks.transition(taskId, "FAILED", { error: reason }))) throw new TaskAlreadyEnded();
await tx.inbound.drop(row.event.id, reason);
return event;
})
.catch((error: unknown) => {
if (error instanceof TaskAlreadyEnded) return null;
throw error;
});
if (logged) {
this.#events.publish(logged);
this.#log.error("task delivery failed for good", { taskId, error: reason });
this.#onTaskSettled();
return;
}
});
if (row.event.type === "task.created") this.#log.error("task delivery failed for good", { taskId: row.task_id, error: reason });
if (settled) this.#onTaskSettled();
}
await this.#db.inbound.drop(row.event.id, reason);
}
/** scheduled_at style: compared with the database's now(), so a zero delay is "no date" rather than this host's now. */
+41 -37
View File
@@ -30,8 +30,9 @@ import {
type VmState,
} from "@invisible-dots/shared";
import { guestErrorCode, guestErrorStatus, type ComputerDriver, type ComputerSpecInput, type ComputerState, type GuestApi } from "./driver.js";
import { applyGuestEvent, dotStatusForAgent } from "./guest-events.js";
import { ControlPlaneError, errorMessage, KeyedMutex, sleep, type Clock, type Logger } from "./support.js";
import { applyGuestEvent, dotStatusForAgent, type AppliedGuestEvent } from "./guest-events.js";
import { abortableSleep, ControlPlaneError, errorMessage, type Clock, type Logger } from "@invisible-dots/shared";
import { BackgroundWork, KeyedMutex } from "./support.js";
export interface LifecycleOptions {
/** How long a started VM may take to pass the READY procedure. Default 10 minutes (a first boot runs cloud-init). */
@@ -123,7 +124,7 @@ export class Lifecycle {
readonly #ready = new Set<string>();
readonly #pumps = new Map<string, AbortController>();
readonly #pumpDone = new Map<string, Promise<void>>();
readonly #background = new Set<Promise<unknown>>();
readonly #background: BackgroundWork;
#closed = false;
constructor(deps: LifecycleDeps) {
@@ -132,6 +133,7 @@ export class Lifecycle {
this.#driver = deps.driver;
this.#clock = deps.clock;
this.#log = deps.logger;
this.#background = new BackgroundWork(this.#log);
this.#opts = { ...DEFAULT_LIFECYCLE_OPTIONS, ...deps.options };
this.#onWorkPossible = deps.onWorkPossible ?? (() => {});
this.#onReady = deps.onReady ?? (() => {});
@@ -244,7 +246,7 @@ export class Lifecycle {
const dot = await tx.dots.get(dotId);
await tx.computers.setProcess(dotId, null);
await tx.computers.setState(dotId, "STOPPED", null, reason);
if (dot && !(keepFailedDot && (dot.status === "DISABLED" || dot.status === "ERROR"))) await tx.dots.setStatus(dotId, "IDLE", null);
if (dot && !(keepFailedDot && dot.status === "ERROR")) await tx.dots.setStatus(dotId, "IDLE", null);
});
}
@@ -368,21 +370,23 @@ export class Lifecycle {
}
/**
* Push what the guest must hold: the OpenRouter key (memory only in the
* guest, so lost with every agent restart) and the runtime config, both
* read from the database now. The one place that does this, for READY,
* for a changed key or config, and after an agent restart.
* Push what the guest must hold: the OpenRouter key and the secrets the
* config's MCP servers name (memory only in the guest, so lost with every
* agent restart), then the runtime config, all read from the database now.
* The one place that does this, for READY, for a changed key, secret or
* config, and after an agent restart.
*/
async #push(dotId: string, guest: GuestApi): Promise<void> {
const key = await this.#db.secrets.openRouterKey(dotId);
if (!key) throw new NotReadyError(dotId, `${MISSING_KEY_MESSAGE}; set one with PUT /api/secrets/openrouter`);
const dot = await this.#db.dots.get(dotId);
if (!dot) throw new ControlPlaneError(404, "not_found", `Dot ${dotId} not found`);
const mcpSecrets = await this.#db.secrets.mcpSecrets(dotId, dot.config.mcp_servers);
try {
await guest.pushSecrets(key);
await guest.pushSecrets({ openrouter_api_key: key, mcp_secrets: mcpSecrets });
} catch (error) {
// Never the guest's own words here: whatever answered may have echoed the key.
throw new Error(`the guest did not take the OpenRouter key (${describeWithoutBody(error)})`);
// Never the guest's own words here: whatever answered may have echoed a secret.
throw new Error(`the guest did not take the OpenRouter key and the MCP secrets (${describeWithoutBody(error)})`);
}
await guest.putConfig(toRuntimeConfig(dot.config));
}
@@ -629,14 +633,21 @@ export class Lifecycle {
* an agent that restarted and lost its key): push both to a READY guest
* now. A Dot that is not READY gets them from its READY procedure, which
* sees the change through the generation and pushes again if it was
* already past its own push. True when this call pushed.
* already past its own push. True when this call pushed. A push that fails
* may mean the guest is gone, so the Dot is suspect (`markSuspect`): its next
* delivery checks READY again, which pushes both again.
*/
async syncGuest(dotId: string): Promise<boolean> {
this.#generation.set(dotId, this.#generationOf(dotId) + 1);
if (!this.#ready.has(dotId)) return false;
return this.#pushes.run(dotId, async () => {
if (!this.#ready.has(dotId)) return false;
await this.#push(dotId, await this.guest(dotId));
try {
await this.#push(dotId, await this.guest(dotId));
} catch (error) {
this.markSuspect(dotId);
throw error;
}
return true;
});
}
@@ -650,22 +661,14 @@ export class Lifecycle {
this.#ready.delete(dotId);
}
#runInBackground(what: string, dotId: string, work: () => Promise<unknown>): void {
if (this.#closed) return;
const tracked: Promise<unknown> = work()
.catch((error) => this.#log.error(`${what} failed`, { dotId, error: errorMessage(error) }))
.finally(() => this.#background.delete(tracked));
this.#background.add(tracked);
}
/** Whether work this class started in the background is still running. */
get busy(): boolean {
return this.#background.size > 0;
return this.#background.busy;
}
/** Wait for the work this class started in the background (tests and shutdown). */
async settle(): Promise<void> {
while (this.#background.size > 0) await Promise.allSettled([...this.#background]);
await this.#background.settle();
}
#startPump(dotId: string): void {
@@ -711,10 +714,10 @@ export class Lifecycle {
this.#log.warn("event pump failed, retrying", { dotId, error: errorMessage(error), retryMs: delay });
}
if (await this.#qemuGone(dotId)) {
this.#runInBackground("recording a VM that stopped by itself", dotId, () => this.#exited(dotId));
this.#background.run("recording a VM that stopped by itself", dotId, () => this.#exited(dotId));
return;
}
await sleep(delay, signal);
await abortableSleep(delay, signal);
delay = Math.min(delay * 2, this.#opts.pumpMaxRetryMs);
}
}
@@ -747,7 +750,7 @@ export class Lifecycle {
});
if (recorded && (await this.keepsAwake(dotId))) {
this.#log.info("the Dot still has work, starting its computer again", { dotId });
this.#runInBackground("restart after an unexpected stop", dotId, () => this.ensureReady(dotId));
this.#background.run("restart after an unexpected stop", dotId, () => this.ensureReady(dotId));
}
}
@@ -770,25 +773,25 @@ export class Lifecycle {
/** Store and apply one guest event; exported for recovery tools and tests. */
async handleGuestEvent(dotId: string, event: OutboundEvent): Promise<StoredEvent | null> {
const now = this.#clock.now();
const { stored, applied } = await this.#db.transaction(async (tx) => {
const { stored, applied, logged } = await this.#db.transaction(async (tx) => {
// The event first: its insert takes the event-order lock (events.ts).
const stored = await this.#events.appendGuest(tx, dotId, event);
await tx.computers.advanceCursor(dotId, event.seq, now);
const applied = stored ? await applyGuestEvent(tx, dotId, event) : { taskSettled: false };
return { stored, applied };
const applied: AppliedGuestEvent = stored ? await applyGuestEvent(tx, dotId, event) : { taskSettled: false };
const logged: StoredEvent[] = [];
for (const host of applied.hostEvents ?? []) logged.push(await this.#events.appendHostIn(tx, dotId, host.type, host.data));
return { stored, applied, logged };
});
if (stored) this.#events.publish(stored);
for (const event of logged) this.#events.publish(event);
if (applied.taskSettled) this.#onWorkPossible();
if (applied.reconfigured) {
// The Dot added an MCP server to its config: the guest gets the config (and the secrets it names) now.
this.#background.run("pushing the config with the MCP server the Dot added", dotId, () => this.syncGuest(dotId));
}
if (stored && event.type === "agent.started") {
// The agent process restarted inside a running VM: it lost the key it keeps in memory.
this.#runInBackground("pushing the key again after an agent restart", dotId, async () => {
try {
await this.syncGuest(dotId);
} catch (error) {
this.markSuspect(dotId);
throw error;
}
});
this.#background.run("pushing the key again after an agent restart", dotId, () => this.syncGuest(dotId));
}
return stored;
}
@@ -866,6 +869,7 @@ export class Lifecycle {
/** Stop every pump and release the driver's host resources. VMs keep running. */
async close(): Promise<void> {
this.#closed = true;
this.#background.close();
for (const controller of this.#pumps.values()) controller.abort();
await Promise.all(this.#pumpDone.values());
await this.settle();
+105 -64
View File
@@ -4,7 +4,7 @@
* database, the VM layer or a guest happens here.
*/
import { randomBytes } from "node:crypto";
import { DotChangedError, DotNameTakenError, GLOBAL_SCOPE, OPENROUTER_KEY_NAME, VM_PROXY_NAME, type Database } from "@invisible-dots/database";
import { DotChangedError, DotNameTakenError, GLOBAL_SCOPE, mcpSecretName, OPENROUTER_KEY_NAME, VM_PROXY_NAME, type Database } from "@invisible-dots/database";
import { EventLog, USER_MESSAGE_EVENT } from "@invisible-dots/events";
import type {
AcceptedAnswer,
@@ -21,48 +21,52 @@ import type {
import {
APPROVAL_NOTE_MAX,
checkHomePath,
checkMcpSecret,
checkOpenRouterKey,
COMPUTER_STOPPED,
ControlPlaneError,
computerIsUp,
computerResources,
CONVERSATION_LIST_LIMIT,
DotConfigError,
errorMessage,
isIdentityAnswer,
isStoredEventType,
MAX_HOST_FILE_BYTES,
newId,
notFound,
parseDotConfig,
parseMessageOrigin,
mcpServerNames,
parseSize,
permissionNames,
silentLogger,
systemClock,
TASK_CANCELLED_SYSTEM_EVENT,
TERMINAL_TASK_STATES,
vmName,
type ApprovalStatus,
type BrowserIdentity,
type Clock,
type CreateBrowserIdentityRequest,
type DotConfig,
type FilesListAnswer,
type InboundEvent,
type ListOrder,
type Logger,
type McpSecretsAnswer,
type MessageOrigin,
type StoredEvent,
type SystemAnswer,
type Skill,
type ToolInfo,
type TaskState,
type ToolListAnswer,
} from "@invisible-dots/shared";
import { Dispatcher } from "./dispatcher.js";
import { guestErrorCode, guestErrorStatus, type ComputerDriver, type GuestApi } from "./driver.js";
import { InboundDelivery, type InboundDeliveryOptions } from "./inbound.js";
import { Lifecycle, MISSING_KEY_MESSAGE, type LifecycleOptions } from "./lifecycle.js";
import {
ControlPlaneError,
errorMessage,
notFound,
silentLogger,
systemClock,
type Clock,
type Logger,
} from "./support.js";
import { BackgroundWork } from "./support.js";
export interface SchedulerOptions {
db: Database;
@@ -99,7 +103,7 @@ export class Scheduler {
readonly #log: Logger;
readonly #dispatchIntervalMs: number;
readonly #idleCheckIntervalMs: number;
readonly #background = new Set<Promise<unknown>>();
readonly #background: BackgroundWork;
readonly #provisioning = new Map<string, Promise<void>>();
readonly #timers: NodeJS.Timeout[] = [];
#started = false;
@@ -108,6 +112,7 @@ export class Scheduler {
this.db = options.db;
this.#clock = options.clock ?? systemClock;
this.#log = options.logger ?? silentLogger;
this.#background = new BackgroundWork(this.#log);
this.events =
options.events ?? new EventLog(options.db.events, (line) => this.#log.warn(line));
this.#dispatchIntervalMs = options.dispatchIntervalMs ?? 5_000;
@@ -124,6 +129,7 @@ export class Scheduler {
});
this.inbound = new InboundDelivery({
db: options.db,
events: this.events,
lifecycle: this.lifecycle,
clock: this.#clock,
logger: this.#log,
@@ -144,8 +150,8 @@ export class Scheduler {
async start(): Promise<void> {
if (this.#started) return;
this.#started = true;
for (const work of await this.lifecycle.recover()) this.#track(work);
this.#track(this.pass());
for (const work of await this.lifecycle.recover()) this.#background.track(work);
this.#background.track(this.pass());
const every = (ms: number, fn: () => Promise<unknown>) => {
const timer = setInterval(() => void fn().catch((e) => this.#log.error("timer failed", { error: errorMessage(e) })), ms);
timer.unref();
@@ -167,12 +173,12 @@ export class Scheduler {
for (const dotId of await this.db.tasks.stoppedDotsWithBlockedWork()) {
if (this.lifecycle.isBusy(dotId)) continue;
this.#log.info("waking a stopped Dot: new work waits behind its unfinished task", { dotId });
this.#runInBackground("wake for blocked work", dotId, () => this.lifecycle.ensureReady(dotId));
this.#background.run("wake for blocked work", dotId, () => this.lifecycle.ensureReady(dotId));
}
for (const dotId of await this.lifecycle.stoppedDotsWithAutomationDue()) {
if (this.lifecycle.isBusy(dotId)) continue;
this.#log.info("waking a stopped Dot: an automation is due", { dotId });
this.#runInBackground("wake for an automation", dotId, () => this.lifecycle.wakeForAutomation(dotId));
this.#background.run("wake for an automation", dotId, () => this.lifecycle.wakeForAutomation(dotId));
}
}
@@ -190,32 +196,15 @@ export class Scheduler {
await this.dispatcher.settle();
await this.inbound.settle();
await this.lifecycle.settle();
const pending = [...this.#background];
if (pending.length === 0) {
if (!this.#background.busy) {
// Each of the three may have started work in another while it settled.
if (!this.dispatcher.busy && !this.inbound.busy && !this.lifecycle.busy) return;
continue;
}
await Promise.allSettled(pending);
await this.#background.settle();
}
}
#track<T>(work: Promise<T>): Promise<T> {
const tracked = work.finally(() => this.#background.delete(tracked));
tracked.catch(() => {});
this.#background.add(tracked);
return work;
}
/** Run something in the background and log its failure. */
#runInBackground(what: string, dotId: string, work: () => Promise<unknown>): void {
this.#track(
work().catch((error) => {
this.#log.error(`${what} failed`, { dotId, error: errorMessage(error) });
}),
);
}
// Dots
async requireDot(idOrName: string): Promise<DotSummary> {
@@ -270,7 +259,7 @@ export class Scheduler {
this.#log.info("dot created", { dotId: id, name: config.name });
const provisioning = this.lifecycle.provision(id);
this.#provisioning.set(id, provisioning);
this.#runInBackground("provisioning", id, () =>
this.#background.run("provisioning", id, () =>
provisioning.finally(() => {
this.#provisioning.delete(id);
}),
@@ -304,6 +293,8 @@ export class Scheduler {
const logged = await this.events.appendHostIn(tx, current.id, "dot.updated", { name: config.name });
const updated = await tx.dots.updateConfig(current.id, config, expectedConfigVersion);
if (!updated) throw notFound("Dot", idOrName);
// A secret of a server the config no longer declares, or no longer names, goes with it.
await tx.secrets.deleteUndeclaredMcpSecrets(current.id, config.mcp_servers);
return { updated, logged };
});
} catch (error) {
@@ -314,29 +305,29 @@ export class Scheduler {
throw error;
}
this.events.publish(saved.logged);
await this.#pushConfig(saved.updated);
await this.#pushToGuest(saved.updated);
return saved.updated;
}
/**
* The Dot's saved config changed (a PATCH, an "always allow"; `dot.updated` was logged with the change): push it to
* the guest. A failed push does not fail the change: the config is pushed again on the next READY.
* The Dot's saved config changed (a PATCH, an "always allow"; `dot.updated` was logged with the change), or a secret of
* its MCP servers did: push the secrets and the config to the guest. A failed push does not fail the change: the Dot
* is suspect then (`syncGuest`), and both are pushed again on its next READY.
*/
async #pushConfig(dot: DotRecord): Promise<void> {
async #pushToGuest(dot: DotRecord): Promise<void> {
try {
await this.lifecycle.syncGuest(dot.id);
} catch (error) {
this.#log.warn("config saved but the push to the guest failed; it is pushed again on the next READY", {
this.#log.warn("saved, but the push to the guest failed; it is pushed again on the next READY", {
dotId: dot.id,
error: errorMessage(error),
});
this.lifecycle.markSuspect(dot.id);
}
}
async deleteDot(idOrName: string): Promise<AcceptedAnswer> {
const dot = await this.requireDot(idOrName);
this.#runInBackground("deletion", dot.id, () => this.lifecycle.remove(dot.id));
this.#background.run("deletion", dot.id, () => this.lifecycle.remove(dot.id));
return { accepted: true };
}
@@ -350,7 +341,7 @@ export class Scheduler {
* is routed back by) and never sent to the guest. Only code inside the
* control plane passes one: the HTTP route does not take it.
*/
async sendMessage(idOrName: string, text: string, origin?: MessageOrigin): Promise<MessageAnswer> {
async sendMessage(idOrName: string, text: unknown, origin?: MessageOrigin): Promise<MessageAnswer> {
if (typeof text !== "string" || text.trim() === "") {
throw new ControlPlaneError(400, "invalid_request", "text must be a non-empty string");
}
@@ -466,6 +457,16 @@ export class Scheduler {
return this.db.tasks.listByDot(dot.id, page);
}
/** The tasks of every Dot that ended in a terminal `status` after `after`, the latest end first: one read for all of them. */
async listEndedTasks(status: unknown, after: unknown): Promise<TaskRecord[]> {
if (!TERMINAL_TASK_STATES.includes(status as TaskState)) {
throw new ControlPlaneError(400, "invalid_request", `status must be one of ${TERMINAL_TASK_STATES.join(", ")}`);
}
const since = typeof after === "string" ? new Date(after) : new Date(Number.NaN);
if (Number.isNaN(since.getTime())) throw new ControlPlaneError(400, "invalid_request", "finished_after must be an ISO 8601 time");
return this.db.tasks.listEnded(status as TaskState, since);
}
async getTask(id: string): Promise<TaskRecord> {
const task = await this.db.tasks.get(id);
if (!task) throw notFound("task", id);
@@ -535,7 +536,7 @@ export class Scheduler {
if (dot.computer_state === "PROVISIONING" || dot.computer_state === "DELETING") {
throw new ControlPlaneError(409, "invalid_state", `the computer is ${dot.computer_state}`);
}
this.#runInBackground("start", dot.id, () => this.lifecycle.ensureReady(dot.id));
this.#background.run("start", dot.id, () => this.lifecycle.ensureReady(dot.id));
return { accepted: true };
}
@@ -544,7 +545,7 @@ export class Scheduler {
if (dot.computer_state === "PROVISIONING" || dot.computer_state === "DELETING") {
throw new ControlPlaneError(409, "invalid_state", `the computer is ${dot.computer_state}`);
}
this.#runInBackground("stop", dot.id, () => this.lifecycle.stop(dot.id, "user"));
this.#background.run("stop", dot.id, () => this.lifecycle.stop(dot.id, "user"));
return { accepted: true };
}
@@ -553,7 +554,7 @@ export class Scheduler {
if (!computerIsUp(dot.computer_state)) {
throw new ControlPlaneError(409, COMPUTER_STOPPED, `the computer is ${dot.computer_state ?? "missing"}`);
}
this.#runInBackground("reboot", dot.id, () => this.lifecycle.reboot(dot.id));
this.#background.run("reboot", dot.id, () => this.lifecycle.reboot(dot.id));
return { accepted: true };
}
@@ -650,10 +651,13 @@ export class Scheduler {
// The tools and the skills: the Dot's engine keeps both, read through the computer
/** The Dot's tools, each with the permission it exercises and whether the model is offered it now. */
async listTools(idOrName: string): Promise<ToolInfo[]> {
/**
* The Dot's tools, each with the permission it exercises and whether the model is offered it now, and the MCP servers
* its config declares with where each is (connecting, connected, failed and why).
*/
async listTools(idOrName: string): Promise<ToolListAnswer> {
const { dotId, guest } = await this.#runningGuest(idOrName);
return (await this.#guestCall(dotId, "list tools", () => guest.listTools())).tools;
return this.#guestCall(dotId, "list tools", () => guest.listTools());
}
/** The Dot's skills, the built-in ones and its own, each with its whole file. */
@@ -671,7 +675,7 @@ export class Scheduler {
): Promise<ApprovalRecord[]> {
const { dot, ...rest } = page;
// Of one Dot (by id or name; the history of a deleted Dot is read by id, as the event log's is), in the database.
return this.db.approvals.list({ status, ...rest, ...(dot === undefined ? {} : { dotId: await this.#historyDotId(dot) }) });
return this.db.approvals.list({ status, ...rest, ...(dot === undefined ? {} : { dotId: await this.historyDotId(dot) }) });
}
/**
@@ -692,18 +696,19 @@ export class Scheduler {
async resolveApproval(
id: string,
decision: "approve" | "reject",
answer: { note?: string; always?: true } = {},
answer: { note?: unknown; always?: unknown } = {},
): Promise<ApprovalRecord> {
const { note, always } = answer;
if (note !== undefined && typeof note !== "string") {
if (answer.note !== undefined && typeof answer.note !== "string") {
throw new ControlPlaneError(400, "invalid_request", "note must be a string");
}
if (answer.always !== undefined && answer.always !== true) {
throw new ControlPlaneError(400, "invalid_request", "always must be true");
}
const note = answer.note as string | undefined;
const always = answer.always as true | undefined;
if (note !== undefined && note.length > APPROVAL_NOTE_MAX) {
throw new ControlPlaneError(400, "invalid_request", `note must be at most ${APPROVAL_NOTE_MAX} characters`);
}
if (always !== undefined && always !== true) {
throw new ControlPlaneError(400, "invalid_request", "always must be true");
}
if (always && decision !== "approve") {
throw new ControlPlaneError(400, "invalid_request", "always applies to an approval, not to a rejection");
}
@@ -732,6 +737,10 @@ export class Scheduler {
// "Always allow" changes the Dot's config: that is told by the event of every config change, in the same commit.
const dot = always ? await tx.dots.get(existing.dot_id) : null;
if (always && !dot) throw notFound("Dot", existing.dot_id);
// The permission of an MCP server the config no longer declares has nothing left to allow.
if (dot && !permissionNames(dot.config).includes(existing.permission as never)) {
throw new ControlPlaneError(409, "permission_gone", `the Dot's config no longer has the permission ${existing.permission}, so it cannot be allowed always`);
}
const updated = dot ? await this.events.appendHostIn(tx, existing.dot_id, "dot.updated", { name: dot.name }) : null;
const resolved = await tx.approvals.resolve(id, decision === "approve" ? "approved" : "rejected", note ?? null);
if (!resolved) {
@@ -746,7 +755,7 @@ export class Scheduler {
});
resolved = stored.resolved;
for (const logged of stored.logged) this.events.publish(logged);
if (stored.reconfigured) await this.#pushConfig(stored.reconfigured);
if (stored.reconfigured) await this.#pushToGuest(stored.reconfigured);
} finally {
release?.();
}
@@ -757,7 +766,7 @@ export class Scheduler {
// Events and secrets
/** The id of a Dot whose history is read: a deleted Dot's history stays readable by id. */
async #historyDotId(idOrName: string): Promise<string> {
async historyDotId(idOrName: string): Promise<string> {
const dot = await this.db.dots.resolve(idOrName);
const dotId = dot?.id ?? (idOrName.includes("_") ? idOrName : undefined);
if (!dotId) throw notFound("Dot", idOrName);
@@ -777,12 +786,12 @@ export class Scheduler {
if (filter.taskId === "") throw new ControlPlaneError(400, "invalid_request", "task_id must not be empty");
const types = filter.types === undefined || filter.types.length === 0 ? undefined : filter.types;
const tools = filter.tools === undefined || filter.tools.length === 0 ? undefined : filter.tools;
return this.events.query({ dotId: await this.#historyDotId(idOrName), after: filter.after, before: filter.before, limit: filter.limit, types, tools, taskId: filter.taskId, order: filter.order });
return this.events.query({ dotId: await this.historyDotId(idOrName), after: filter.after, before: filter.before, limit: filter.limit, types, tools, taskId: filter.taskId, order: filter.order });
}
/** The model spend the Dot's guest reported since `since` (every event when omitted), from the event log. */
async usage(idOrName: string, since?: Date): Promise<UsageAnswer> {
const dotId = await this.#historyDotId(idOrName);
const dotId = await this.historyDotId(idOrName);
return { dot_id: dotId, since: since?.toISOString() ?? null, spent_usd: await this.db.events.spentUsd(dotId, since) };
}
@@ -809,6 +818,39 @@ export class Scheduler {
return { dot_id: dot.id, proxy: (await this.db.secrets.get(dot.id, VM_PROXY_NAME)) !== null };
}
/** Every secret the Dot's MCP servers name, and whether each is set; never a value. */
async mcpSecrets(idOrName: string): Promise<McpSecretsAnswer> {
const dot = await this.requireDot(idOrName);
const set = await this.db.secrets.mcpSecrets(dot.id, dot.config.mcp_servers);
const secrets = mcpServerNames(dot.config).flatMap((server) =>
dot.config.mcp_servers[server]!.secrets.map((name) => ({ server, name, set: set[server]?.[name] !== undefined })),
);
return { dot_id: dot.id, secrets };
}
/**
* Set the value of a secret an MCP server of the Dot's config names, or clear it with null, and push the secrets to
* the guest when it runs: the server starts again with it. A secret the config does not name is refused, so none is
* stored that the config would not keep. The answer never carries a value.
*/
async setMcpSecret(idOrName: string, server: string, name: string, value: unknown): Promise<McpSecretsAnswer> {
const dot = await this.requireDot(idOrName);
const declared = dot.config.mcp_servers[server];
if (!declared) throw new ControlPlaneError(404, "not_found", `Dot ${dot.name} declares no MCP server "${server}"`);
if (!declared.secrets.includes(name)) {
throw new ControlPlaneError(404, "not_found", `the MCP server "${server}" of Dot ${dot.name} names no secret "${name}"`);
}
if (value === null) {
await this.db.secrets.delete(dot.id, mcpSecretName(server, name));
} else {
const checked = checkMcpSecret(value);
if (!checked.ok) throw new ControlPlaneError(400, "invalid_request", checked.problem);
await this.db.secrets.put(dot.id, mcpSecretName(server, name), checked.value);
}
await this.#pushToGuest(dot);
return this.mcpSecrets(dot.id);
}
/** Store the OpenRouter key (global or per Dot) and push it to the READY guests it applies to. */
async setOpenRouterKey(value: unknown, dotIdOrName?: unknown): Promise<{ pushed: number }> {
const checked = checkOpenRouterKey(value);
@@ -827,14 +869,13 @@ export class Scheduler {
if (await this.lifecycle.syncGuest(dotId)) pushed++;
} catch (error) {
this.#log.warn("could not push the new OpenRouter key", { dotId, error: errorMessage(error) });
this.lifecycle.markSuspect(dotId);
}
}
// Dots that failed READY only for want of a key can be retried now.
for (const dot of await this.db.dots.list()) {
const missedKey = dot.error?.includes(MISSING_KEY_MESSAGE) ?? false;
if (dot.status === "ERROR" && missedKey && (scope === GLOBAL_SCOPE || scope === dot.id)) {
this.#runInBackground("READY after a new key", dot.id, () => this.lifecycle.ensureReady(dot.id));
this.#background.run("READY after a new key", dot.id, () => this.lifecycle.ensureReady(dot.id));
}
}
return { pushed };
@@ -863,7 +904,7 @@ export class Scheduler {
if (await this.lifecycle.keepsAwake(dotId)) continue;
this.#log.info("dot idle, going to sleep", { dotId, idleMs: now.getTime() - lastActive });
sleeping.push(dotId);
this.#runInBackground("idle sleep", dotId, () => this.lifecycle.stop(dotId, "idle"));
this.#background.run("idle sleep", dotId, () => this.lifecycle.stop(dotId, "idle"));
}
return sleeping;
}
+39 -59
View File
@@ -1,33 +1,5 @@
/** Small pieces the scheduler is built from: clock, logger, a per-key lock and the error types the API maps to HTTP. */
export interface Clock {
now(): Date;
}
export const systemClock: Clock = { now: () => new Date() };
/** The logger shape vm-manager uses too, so one logger serves the whole process. */
export interface Logger {
debug(message: string, fields?: Record<string, unknown>): void;
info(message: string, fields?: Record<string, unknown>): void;
warn(message: string, fields?: Record<string, unknown>): void;
error(message: string, fields?: Record<string, unknown>): void;
}
export function prefixedStderrLogger(prefix: string, debug = false): Logger {
const write = (level: string, message: string, fields?: Record<string, unknown>) => {
const suffix = fields && Object.keys(fields).length > 0 ? ` ${JSON.stringify(fields)}` : "";
process.stderr.write(`${new Date().toISOString()} [${prefix}] ${level} ${message}${suffix}\n`);
};
return {
debug: debug ? (m, f) => write("debug", m, f) : () => {},
info: (m, f) => write("info", m, f),
warn: (m, f) => write("warn", m, f),
error: (m, f) => write("error", m, f),
};
}
export const silentLogger: Logger = { debug() {}, info() {}, warn() {}, error() {} };
/** The scheduler's own building blocks: a per-key lock, and the work it starts without waiting for it. */
import { errorMessage, type Logger } from "@invisible-dots/shared";
/**
* Serializes work per key: lifecycle operations on one Dot (create, start,
@@ -53,34 +25,42 @@ export class KeyedMutex {
}
}
/** An error with the HTTP status and `error` code the API answers with (section 9.6). */
export class ControlPlaneError extends Error {
constructor(
readonly status: number,
readonly code: string,
message: string,
readonly details?: unknown,
) {
super(message);
this.name = "ControlPlaneError";
/**
* Work a caller starts without waiting for it (a provisioning, a stop, a push to a guest): its failure is logged,
* since nobody awaits it, and `settle` waits until all of it has finished (tests and shutdown). After `close` nothing
* new starts.
*/
export class BackgroundWork {
readonly #pending = new Set<Promise<unknown>>();
#closed = false;
constructor(private readonly log: Logger) {}
/** Start `work`; a failure is logged as "<what> failed" with the Dot. */
run(what: string, dotId: string, work: () => Promise<unknown>): void {
if (this.#closed) return;
this.track(work().catch((error) => this.log.error(`${what} failed`, { dotId, error: errorMessage(error) })));
}
/** Count a promise as background work until it settles; it is returned as it is, for a caller that awaits it. */
track<T>(work: Promise<T>): Promise<T> {
const tracked = work.finally(() => this.#pending.delete(tracked));
tracked.catch(() => {});
this.#pending.add(tracked);
return work;
}
/** Whether any of it is still running. */
get busy(): boolean {
return this.#pending.size > 0;
}
/** Wait until all of it has finished, including work started meanwhile. */
async settle(): Promise<void> {
while (this.#pending.size > 0) await Promise.allSettled([...this.#pending]);
}
close(): void {
this.#closed = true;
}
}
export const notFound = (what: string, id: string) => new ControlPlaneError(404, "not_found", `${what} "${id}" not found`);
export function sleep(ms: number, signal?: AbortSignal): Promise<void> {
return new Promise((resolve) => {
if (signal?.aborted) return resolve();
const done = () => {
clearTimeout(timer);
signal?.removeEventListener("abort", done);
resolve();
};
const timer = setTimeout(done, ms);
signal?.addEventListener("abort", done, { once: true });
});
}
export function errorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
+38 -6
View File
@@ -14,6 +14,8 @@ import {
checkIdentityRequest,
IDENTITY_ERROR_STATUS,
FILE_TOO_LARGE,
mcpPermission,
mcpServerNames,
GUEST_PATHS,
IdentityRequestError,
newId,
@@ -34,7 +36,9 @@ import {
type OutboundEvent,
type OutboundEventDataMap,
type OutboundEventType,
type PermissionName,
type RefusedEvent,
type SecretsRequest,
type SystemAnswer,
type ToolInfo,
type Skill,
@@ -157,6 +161,8 @@ export class FakeGuest implements GuestApi {
#bootedAt = 0;
agentState: AgentState = "IDLE";
openrouterKey: string | null = null;
/** The MCP servers' secrets the host pushed last, by server and name. */
mcpSecrets: SecretsRequest["mcp_secrets"] = {};
config: DotRuntimeConfig | null = null;
checks: GuestChecks = { filesystem_writable: true, network_reachable: true, browser_installed: true };
readonly outbox: OutboundEvent[] = [];
@@ -180,6 +186,12 @@ export class FakeGuest implements GuestApi {
{ name: "browser_identity_list", permission: "browser.identity.list", description: "List the browser identities." },
{ name: "browser_identity_create", permission: "browser.identity.create", description: "Create a browser identity." },
];
/**
* The tools each declared MCP server serves, by server; a server of the config with none here serves none. A server
* named in `mcpFailures` is failed with that error instead, as the engine reports a server it could not start.
*/
readonly mcpTools = new Map<string, Omit<ToolInfo, "offered" | "permission">[]>();
readonly mcpFailures = new Map<string, string>();
/** The skills the fake's engine shows: one built-in by default, as the real one ships; a test adds the Dot's own. */
skills: Skill[] = [
{
@@ -216,6 +228,7 @@ export class FakeGuest implements GuestApi {
this.#pollsSinceBoot = 0;
this.#bootedAt = Date.now();
this.openrouterKey = null;
this.mcpSecrets = {};
this.agentState = "IDLE";
this.emit("agent.started", {});
this.#catchUp();
@@ -249,6 +262,7 @@ export class FakeGuest implements GuestApi {
restartAgent(): void {
this.agentRestarts++;
this.openrouterKey = null;
this.mcpSecrets = {};
this.disconnectStreams();
this.emit("agent.started", {});
}
@@ -336,15 +350,20 @@ export class FakeGuest implements GuestApi {
}
/** Ask for an approval from inside a task, as the policy engine does on `ask`. */
requestApproval(taskId: string | undefined, tool = "browser_identity_delete"): string {
requestApproval(
taskId: string | undefined,
tool = "browser_identity_delete",
permission: PermissionName = "browser.identity.delete",
args: Record<string, unknown> = { identity_id: "shop-abc123" },
): string {
const approvalId = newId("apr");
this.emit("agent.state", { state: "WAITING_APPROVAL" });
this.emit("approval.requested", {
approval_id: approvalId,
...(taskId ? { task_id: taskId } : {}),
tool,
permission: "browser.identity.delete",
arguments: { identity_id: "shop-abc123" },
permission,
arguments: args,
reason: "the tool needs approval",
});
return approvalId;
@@ -384,11 +403,13 @@ export class FakeGuest implements GuestApi {
};
}
async pushSecrets(key: string): Promise<void> {
async pushSecrets(secrets: SecretsRequest): Promise<void> {
this.#reachable("pushSecrets");
this.openrouterKey = key;
this.openrouterKey = secrets.openrouter_api_key;
this.mcpSecrets = structuredClone(secrets.mcp_secrets);
}
/** While set, `putConfig` waits for it (a slow push), and the config is stored when it resolves. */
configPushGate: Promise<void> | null = null;
@@ -520,7 +541,18 @@ export class FakeGuest implements GuestApi {
const permissions: Record<string, string | undefined> = this.config?.permissions ?? {};
const offered = (tool: Omit<ToolInfo, "offered">): boolean =>
this.config !== null && (permissions[tool.permission] === "allow" || permissions[tool.permission] === "ask");
return { tools: this.tools.map((tool) => ({ ...tool, offered: offered(tool) })) };
const servers = this.config ? mcpServerNames(this.config) : [];
const connected = servers.filter((name) => !this.mcpFailures.has(name));
const mcpTools = connected.flatMap((name) => (this.mcpTools.get(name) ?? []).map((tool) => ({ ...tool, permission: mcpPermission(name) })));
return {
tools: [...this.tools, ...mcpTools].map((tool) => ({ ...tool, offered: offered(tool) })),
mcp_servers: servers.map((name) => {
const error = this.mcpFailures.get(name);
return error === undefined
? { name, state: "connected" as const, error: null, tools: this.mcpTools.get(name)?.length ?? 0 }
: { name, state: "failed" as const, error, tools: 0 };
}),
};
}
async listSkills(): Promise<SkillListAnswer> {
+25
View File
@@ -130,6 +130,31 @@ describe("the commit points of the host", () => {
expect((await db.events.list({ dotId: dot.id, types: ["task.created"] }))[0]!.data.task_id).toBe(task.id);
});
it("a task given up killed at its last write is neither FAILED nor told; given up whole, it is both", async () => {
const driver = new FakeDriver();
const killed = killedAt(db, "inbound", "drop");
const scheduler = make(driver, killed.db);
const dot = await scheduler.createDot(yaml("killed-give-up"));
await waitUntilSettledReady(scheduler, driver, dot.id, "killed-give-up");
await scheduler.lifecycle.stop(dot.id, "user");
driver.failNext("start", new Error("qemu-system-x86_64 exited: boom"), 3);
const failures = async (taskId: string) =>
(await db.query<{ failures: number }>("SELECT failures FROM inbound_events WHERE task_id = $1", [taskId])).rows[0]?.failures ?? 0;
killed.kill();
const task = await scheduler.createTask(dot.id, { description: "undeliverable" });
await waitFor(async () => (await failures(task.id)) >= 2, "its attempts used up");
await scheduler.settle();
expect((await db.tasks.get(task.id))?.status).toBe("RUNNING");
expect(await typesOf(dot.id, "task.failed")).toEqual([]);
// The next attempt gives it up whole: the state and its event in one commit.
killed.revive();
await scheduler.inbound.kick(dot.id);
expect((await db.tasks.get(task.id))?.status).toBe("FAILED");
expect(await typesOf(dot.id, "task.failed")).toEqual(["task.failed"]);
});
it("a config killed at its save is not announced, and a saved one is, in the same commit", async () => {
const driver = new FakeDriver();
const killed = killedAt(db, "dots", "updateConfig");
+6 -5
View File
@@ -6,9 +6,10 @@
*/
import type { Database } from "@invisible-dots/database";
import { createTestDatabase, testAdapters, type TestDatabase } from "@invisible-dots/database/testing";
import type { InboundEvent, OutboundEvent } from "@invisible-dots/shared";
import type { InboundEvent, OutboundEvent, SecretsRequest } from "@invisible-dots/shared";
import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest";
import { Scheduler, type Logger, type SchedulerOptions } from "../src/index.js";
import { Scheduler, type SchedulerOptions } from "../src/index.js";
import { type Logger } from "@invisible-dots/shared";
import type { GuestApi, GuestEndpoint } from "../src/index.js";
import { FakeDriver, FakeGuest, FakeGuestError, ManualClock, waitFor, waitUntilSettledReady } from "../src/testing.js";
@@ -304,8 +305,8 @@ describe.each(testAdapters())("delivery to guests (%s)", (kind) => {
const logger: Logger = { debug: record("debug"), info: record("info"), warn: record("warn"), error: record("error") };
const driver = new FakeDriver();
driver.configureGuest = (g) => {
g.pushSecrets = async (key: string) => {
throw new FakeGuestError(502, `upstream said: bad request body {"openrouter_api_key":"${key}"}`);
g.pushSecrets = async (secrets: SecretsRequest) => {
throw new FakeGuestError(502, `upstream said: bad request body {"openrouter_api_key":"${secrets.openrouter_api_key}"}`);
};
};
const { scheduler } = make(driver, { logger });
@@ -315,7 +316,7 @@ describe.each(testAdapters())("delivery to guests (%s)", (kind) => {
const dotRow = await db.dots.get(dot.id);
const computer = await db.computers.get(dot.id);
const events = await db.events.list({ dotId: dot.id });
expect(dotRow?.error).toMatch(/did not take the OpenRouter key \(status 502\)/);
expect(dotRow?.error).toMatch(/did not take the OpenRouter key and the MCP secrets \(status 502\)/);
for (const text of [dotRow?.error ?? "", computer?.last_error ?? "", JSON.stringify(events), lines.join("\n")]) {
expect(text).not.toContain(KEY);
expect(text).not.toContain(KEY.slice(0, 12));
+17 -1
View File
@@ -10,6 +10,7 @@ import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import {
MAX_RUN_AT_MS,
mcpServerStatusSchema,
OUTBOUND_EVENT_TYPES,
parseDotConfig,
parseOutboundEvent,
@@ -30,7 +31,13 @@ interface OfferingCase {
const shapes = JSON.parse(
readFileSync(fileURLToPath(new URL("../../../invisible_engine_dots/tests/dots/wire_shapes.json", import.meta.url)), "utf8"),
) as { limits: { max_run_at_ms: number }; outbound_events: { type: string; data: Record<string, unknown> }[]; tool_offering: OfferingCase[]; skills: unknown[] };
) as {
limits: { max_run_at_ms: number };
outbound_events: { type: string; data: Record<string, unknown> }[];
tool_offering: OfferingCase[];
mcp: { tools: unknown[]; mcp_servers: unknown[] };
skills: unknown[];
};
const baseConfig = toRuntimeConfig(parseDotConfig("name: shapes\nmodel:\n provider: openrouter\n id: test/model\n"));
@@ -119,6 +126,15 @@ describe("what the engine answers, as the host describes it", () => {
}
});
it("what the engine says of the MCP servers a config declares parses with the host's schemas: their tools under `mcp.<server>`, their states", () => {
expect(shapes.mcp.tools.length).toBeGreaterThan(0);
for (const row of shapes.mcp.tools) expect(toolInfoSchema.safeParse(row).error?.issues, JSON.stringify(row)).toBeUndefined();
expect(new Set((shapes.mcp.tools as { permission: string }[]).map((row) => row.permission))).toEqual(new Set(["mcp.tools"]));
for (const status of shapes.mcp.mcp_servers) expect(mcpServerStatusSchema.safeParse(status).error?.issues, JSON.stringify(status)).toBeUndefined();
expect((shapes.mcp.mcp_servers as { name: string; state: string }[]).map((status) => [status.name, status.state])).toEqual([["keyed", "failed"], ["tools", "connected"]]);
expect(toolInfoSchema.safeParse({ ...(shapes.mcp.tools[0] as object), permission: "mcp.Not_A_Server" }).success).toBe(false);
});
it("the fake guest holds the engine's tools to the same permissions, and offers what the engine offers for the same config", async () => {
const engineRows = new Map((shapes.tool_offering[0]!.tools as { name: string; permission: string }[]).map((row) => [row.name, row.permission]));
for (const offering of shapes.tool_offering) {
+24 -2
View File
@@ -35,6 +35,20 @@ computer:
permissions:
computer.exec: deny
files.write: ask
mcp.time: allow
mcp_servers:
time:
command: uvx
args: [mcp-server-time, --local-timezone, Europe/Rome]
env:
LOG_LEVEL: warning
secrets: [TIME_TOKEN]
search:
url: https://search.example/mcp
headers:
X-Client: dots
secrets: [Authorization]
timeout_s: 30
limits:
max_steps_per_task: 30
max_cost_per_task_usd: 2.5
@@ -76,6 +90,12 @@ describe("what the host sends the engine", () => {
const dot = await readyDot(yaml, name);
configs.push({ name, config: structuredClone(driver.guestOf(dot.id).config) });
}
// The secrets of the full config's servers, set as the person sets them: the push carries them with the key.
const full = await scheduler.requireDot("shapes-full");
await scheduler.setMcpSecret(full.id, "time", "TIME_TOKEN", "tt-1");
await scheduler.setMcpSecret(full.id, "search", "Authorization", "Bearer sk-search-1");
const fullGuest = driver.guestOf(full.id);
const secrets_requests = [{ openrouter_api_key: fullGuest.openrouterKey, mcp_secrets: structuredClone(fullGuest.mcpSecrets) }];
const dot = await readyDot("name: shapes-talk\nmodel:\n provider: openrouter\n id: test/model\n", "shapes-talk");
const guest = driver.guestOf(dot.id);
@@ -103,7 +123,7 @@ describe("what the host sends the engine", () => {
const inbound_events = (JSON.parse(
[...named].reduce((text, [id, name]) => text.replaceAll(id, name), JSON.stringify(guest.inbound.map((event) => ({ type: event.type, data: event.data })))),
) as { type: string; data: Record<string, unknown> }[]);
return { runtime_configs: configs, inbound_events };
return { runtime_configs: configs, secrets_requests, inbound_events };
}
it("writes what the host's own writers sent, as the file the engine's parsers check", async () => {
@@ -119,7 +139,9 @@ describe("what the host sends the engine", () => {
const priorities = written.inbound_events.filter((event) => event.type === "task.created").map((event) => event.data.priority);
expect(priorities).toEqual([5, 0]);
const [minimal, full] = written.runtime_configs.map((entry) => entry.config as Record<string, unknown>);
expect(Object.keys(full!).sort()).toEqual(["instructions", "limits", "model", "models", "name", "permissions"]);
expect(Object.keys(full!).sort()).toEqual(["instructions", "limits", "mcp_servers", "model", "models", "name", "permissions"]);
expect(Object.keys(full!.mcp_servers as object).sort()).toEqual(["search", "time"]);
expect(written.secrets_requests[0]!.mcp_secrets).toEqual({ time: { TIME_TOKEN: "tt-1" }, search: { Authorization: "Bearer sk-search-1" } });
expect(Object.keys(minimal!)).not.toContain("instructions");
expect(minimal).not.toHaveProperty("computer");
});
+181 -3
View File
@@ -906,6 +906,10 @@ describe.each(testAdapters())("Scheduler with a fake driver and a fake guest (%s
const doomed = await scheduler.createTask(dot.id, { description: "never" });
await waitFor(async () => (await db.tasks.get(doomed.id))?.status === "FAILED", "failed task", 8_000);
expect((await db.tasks.get(doomed.id))?.error).toMatch(/after 2 attempts/);
// The host ended the task, so the host logs its end: every view that follows task.failed sees it.
const failed = await db.events.list({ dotId: dot.id, types: ["task.failed"], taskId: doomed.id });
expect(failed).toHaveLength(1);
expect(failed[0]).toMatchObject({ source: "host", data: { task_id: doomed.id, error: expect.stringMatching(/after 2 attempts/) } });
});
it("cancel: a pending task never runs, a finished one answers 409", async () => {
@@ -1191,12 +1195,186 @@ describe.each(testAdapters())("Scheduler with a fake driver and a fake guest (%s
expect(await scheduler.listFiles(dot.id, "docs")).toMatchObject({ entries: [{ name: "note.txt", size: 4 }] });
});
describe("MCP servers", () => {
const withServers = (name: string, servers: string, permissions = "") =>
`${yaml(name)}${permissions ? `permissions:\n${permissions}` : ""}mcp_servers:\n${servers}`;
const TIME = " time:\n command: uvx\n args: [mcp-server-time]\n secrets: [TIME_TOKEN]\n";
const WEB = " web:\n url: https://web.example/mcp\n secrets: [Authorization]\n";
it("a secret is set write-only, pushed with the key, and the answer says only which are set", async () => {
const { scheduler, driver } = make();
const dot = await scheduler.createDot(withServers("mcp-push", TIME + WEB));
await waitUntilSettledReady(scheduler, driver, dot.id, "mcp-push");
const guest = driver.guestOf(dot.id);
expect(guest.mcpSecrets).toEqual({});
const answer = await scheduler.setMcpSecret(dot.id, "web", "Authorization", " Bearer w-1 ");
expect(answer.secrets).toEqual([
{ server: "time", name: "TIME_TOKEN", set: false },
{ server: "web", name: "Authorization", set: true },
]);
expect(JSON.stringify(answer)).not.toContain("w-1");
expect(guest.mcpSecrets).toEqual({ web: { Authorization: "Bearer w-1" } });
await scheduler.setMcpSecret(dot.id, "web", "Authorization", null);
expect(guest.mcpSecrets).toEqual({});
// The guest restarts with nothing in memory: the next READY pushes what is stored again.
await scheduler.setMcpSecret(dot.id, "time", "TIME_TOKEN", "tt-1");
guest.restartAgent();
await waitFor(() => guest.mcpSecrets.time?.TIME_TOKEN === "tt-1", "the secrets pushed again after the restart");
});
it("refuses a secret the config does not name, and a value that cannot travel, without storing anything", async () => {
const { scheduler, driver } = make();
const dot = await scheduler.createDot(withServers("mcp-refuse", TIME));
await waitUntilSettledReady(scheduler, driver, dot.id, "mcp-refuse");
await expect(scheduler.setMcpSecret(dot.id, "ghost", "TIME_TOKEN", "x")).rejects.toMatchObject({ status: 404 });
await expect(scheduler.setMcpSecret(dot.id, "time", "OTHER", "x")).rejects.toMatchObject({ status: 404 });
const refused = scheduler.setMcpSecret(dot.id, "time", "TIME_TOKEN", "tok-SECRET\nX");
await expect(refused).rejects.toMatchObject({ status: 400, code: "invalid_request" });
await refused.catch((error: Error) => expect(error.message).not.toContain("SECRET"));
expect((await scheduler.mcpSecrets(dot.id)).secrets).toEqual([{ server: "time", name: "TIME_TOKEN", set: false }]);
});
it("a config that drops a server, or a name of its secrets, drops their values with it", async () => {
const { scheduler, driver } = make();
const dot = await scheduler.createDot(withServers("mcp-drop", TIME + WEB));
await waitUntilSettledReady(scheduler, driver, dot.id, "mcp-drop");
await scheduler.setMcpSecret(dot.id, "time", "TIME_TOKEN", "tt-1");
await scheduler.setMcpSecret(dot.id, "web", "Authorization", "Bearer w-1");
await scheduler.updateDot(dot.id, withServers("mcp-drop", TIME));
expect(await db.secrets.get(dot.id, "mcp/web/Authorization")).toBeNull();
expect(driver.guestOf(dot.id).mcpSecrets).toEqual({ time: { TIME_TOKEN: "tt-1" } });
// Declared again, it starts with nothing set.
await scheduler.updateDot(dot.id, withServers("mcp-drop", TIME + WEB));
expect((await scheduler.mcpSecrets(dot.id)).secrets.find((s) => s.server === "web")?.set).toBe(false);
});
it("the tool table says where each declared server is, and its tools are under its permission", async () => {
const { scheduler, driver } = make();
const dot = await scheduler.createDot(withServers("mcp-table", TIME + WEB, ' mcp.time: allow\n'));
await waitUntilSettledReady(scheduler, driver, dot.id, "mcp-table");
const guest = driver.guestOf(dot.id);
guest.mcpTools.set("time", [{ name: "mcp_time_now", description: "Now." }]);
guest.mcpFailures.set("web", "its secret Authorization is not set");
const { tools, mcp_servers } = await scheduler.listTools(dot.id);
expect(tools.find((t) => t.name === "mcp_time_now")).toEqual({ name: "mcp_time_now", permission: "mcp.time", offered: true, description: "Now." });
expect(mcp_servers).toEqual([
{ name: "time", state: "connected", error: null, tools: 1 },
{ name: "web", state: "failed", error: "its secret Authorization is not set", tools: 0 },
]);
});
describe("a server the Dot adds itself (add_mcp_server)", () => {
const ARGS = { name: "clock", command: "uvx", args: ["mcp-server-time"], reason: "to tell the time" };
const refusals = async (dotId: string) => (await db.events.list({ dotId })).filter((e) => e.type === "mcp.server.refused").map((e) => e.data);
it("is written into the config when the person approved exactly these arguments, with its permission left to ask, and pushed", async () => {
const { scheduler, driver } = make();
const dot = await readyDot(scheduler, "mcp-self");
const guest = driver.guestOf(dot.id);
// The same arguments in another order: the person approved this entry, however the keys were written.
const id = guest.requestApproval(undefined, "add_mcp_server", "mcp.servers.add", { reason: "to tell the time", args: ["mcp-server-time"], command: "uvx", name: "clock" });
await waitFor(async () => (await db.approvals.get(id)) !== null, "the approval stored");
await scheduler.resolveApproval(id, "approve");
guest.emit("mcp.server.added", { arguments: ARGS });
await waitFor(async () => "clock" in ((await db.dots.get(dot.id))?.config.mcp_servers ?? {}), "the server in the config");
const config = (await db.dots.get(dot.id))!.config;
expect(config.mcp_servers.clock).toEqual({ command: "uvx", args: ["mcp-server-time"], env: {}, secrets: [], timeout_s: 120, startup_timeout_s: 60 });
expect(config.permissions).not.toHaveProperty("mcp.clock");
await waitFor(() => guest.config?.mcp_servers.clock !== undefined, "the config pushed to the guest");
expect((await db.events.list({ dotId: dot.id })).some((e) => e.type === "dot.updated" && e.data.mcp_server_added === "clock" && e.data.approval_id === id)).toBe(true);
});
it("uses an approval up: it adds one server once, and a call refused for its entry keeps it", async () => {
const { scheduler, driver } = make();
const dot = await scheduler.createDot(`${yaml("mcp-once")}mcp_servers:\n clock:\n command: npx\n`);
await waitUntilSettledReady(scheduler, driver, dot.id, "mcp-once");
const guest = driver.guestOf(dot.id);
const id = guest.requestApproval(undefined, "add_mcp_server", "mcp.servers.add", ARGS);
await waitFor(async () => (await db.approvals.get(id)) !== null, "the approval stored");
await scheduler.resolveApproval(id, "approve");
// "clock" is taken: refused before the approval is touched.
guest.emit("mcp.server.added", { arguments: ARGS });
await waitFor(async () => (await refusals(dot.id)).length === 1, "refused: the name is taken");
expect((await refusals(dot.id))[0]).toEqual({ name: "clock", reason: 'the Dot\'s config already declares an MCP server "clock"' });
// The person frees the name: the same approval adds it now.
await scheduler.updateDot(dot.id, yaml("mcp-once"));
guest.emit("mcp.server.added", { arguments: ARGS });
await waitFor(async () => {
const clock = (await db.dots.get(dot.id))?.config.mcp_servers.clock;
return clock !== undefined && "command" in clock && clock.command === "uvx";
}, "added with the approval");
// Removed again, the same call cannot bring it back on the approval it used.
await scheduler.updateDot(dot.id, yaml("mcp-once"));
guest.emit("mcp.server.added", { arguments: ARGS });
await waitFor(async () => (await refusals(dot.id)).length === 2, "refused: the approval is used");
expect((await refusals(dot.id))[1]).toEqual({ name: "clock", reason: "mcp.servers.add asks, and no unused approval of exactly these arguments was given" });
expect((await db.dots.get(dot.id))?.config.mcp_servers).toEqual({});
});
it("is written without an approval when the person allowed mcp.servers.add", async () => {
const { scheduler, driver } = make();
const dot = await scheduler.createDot(`${yaml("mcp-allowed")}permissions:\n mcp.servers.add: allow\n`);
await waitUntilSettledReady(scheduler, driver, dot.id, "mcp-allowed");
driver.guestOf(dot.id).emit("mcp.server.added", { arguments: { name: "web", url: "https://web.example/mcp", secrets: ["Authorization"], reason: "r" } });
await waitFor(async () => "web" in ((await db.dots.get(dot.id))?.config.mcp_servers ?? {}), "the server in the config");
expect((await scheduler.mcpSecrets(dot.id)).secrets).toEqual([{ server: "web", name: "Authorization", set: false }]);
});
it("is refused, and said, when nobody approved it, the permission is denied, the name is taken or the entry is not one", async () => {
const { scheduler, driver } = make();
const dot = await scheduler.createDot(`${yaml("mcp-refused")}mcp_servers:\n taken:\n command: uvx\n`);
await waitUntilSettledReady(scheduler, driver, dot.id, "mcp-refused");
const guest = driver.guestOf(dot.id);
guest.emit("mcp.server.added", { arguments: ARGS });
await waitFor(async () => (await refusals(dot.id)).length === 1, "the first refusal");
expect((await refusals(dot.id))[0]).toEqual({ name: "clock", reason: "mcp.servers.add asks, and no unused approval of exactly these arguments was given" });
await scheduler.updateDot(dot.id, `${yaml("mcp-refused")}permissions:\n mcp.servers.add: allow\nmcp_servers:\n taken:\n command: uvx\n`);
guest.emit("mcp.server.added", { arguments: { name: "taken", command: "npx" } });
guest.emit("mcp.server.added", { arguments: { name: "both", command: "uvx", url: "https://a.example/mcp" } });
await waitFor(async () => (await refusals(dot.id)).length === 3, "two more refusals");
const [, taken, both] = await refusals(dot.id);
expect(taken).toEqual({ name: "taken", reason: 'the Dot\'s config already declares an MCP server "taken"' });
expect((both as { reason: string }).reason).toMatch(/MCP server is either/);
await scheduler.updateDot(dot.id, `${yaml("mcp-refused")}permissions:\n mcp.servers.add: deny\nmcp_servers:\n taken:\n command: uvx\n`);
guest.emit("mcp.server.added", { arguments: ARGS });
await waitFor(async () => (await refusals(dot.id)).length === 4, "the denied one");
expect((await refusals(dot.id))[3]).toEqual({ name: "clock", reason: "the Dot's config denies mcp.servers.add" });
expect(Object.keys((await db.dots.get(dot.id))!.config.mcp_servers)).toEqual(["taken"]);
});
});
it("always allowing the permission of a server the config no longer declares is refused, and nothing changes", async () => {
const { scheduler, driver } = make();
const dot = await scheduler.createDot(withServers("mcp-always", TIME));
await waitUntilSettledReady(scheduler, driver, dot.id, "mcp-always");
const id = driver.guestOf(dot.id).requestApproval(undefined, "mcp_time_now", "mcp.time");
await waitFor(async () => (await scheduler.listApprovals("pending")).some((a) => a.id === id), "the approval");
await scheduler.updateDot(dot.id, yaml("mcp-always"));
await expect(scheduler.resolveApproval(id, "approve", { always: true })).rejects.toMatchObject({ status: 409, code: "permission_gone" });
expect((await db.approvals.get(id))?.status).toBe("pending");
expect((await db.dots.get(dot.id))?.config.permissions).toEqual({});
});
});
it("tools: the engine's table with what is offered now; an unreachable computer is not an empty table", async () => {
const { scheduler, driver } = make();
const dot = await readyDot(scheduler, "toolbox");
const guest = driver.guestOf(dot.id);
const tools = await scheduler.listTools("toolbox");
const { tools, mcp_servers } = await scheduler.listTools("toolbox");
expect(mcp_servers).toEqual([]);
expect(tools.map((t) => [t.name, t.permission])).toEqual([
["exec", "computer.exec"],
["read_file", "files.read"],
@@ -1210,8 +1388,8 @@ describe.each(testAdapters())("Scheduler with a fake driver and a fake guest (%s
await scheduler.updateDot(dot.id, `${yaml("toolbox")}permissions:
automations: deny
`);
await waitFor(async () => (await scheduler.listTools(dot.id)).some((t) => !t.offered), "config pushed");
expect((await scheduler.listTools(dot.id)).filter((t) => !t.offered).map((t) => t.name)).toEqual(["cron"]);
await waitFor(async () => (await scheduler.listTools(dot.id)).tools.some((t) => !t.offered), "config pushed");
expect((await scheduler.listTools(dot.id)).tools.filter((t) => !t.offered).map((t) => t.name)).toEqual(["cron"]);
guest.powerOff();
await expect(scheduler.listTools(dot.id)).rejects.toMatchObject({ status: 502, code: "guest_unavailable" });
+5 -7
View File
@@ -38,11 +38,13 @@ import {
type ProofAnswer,
type RefusedEvent,
type SystemAnswer,
type SecretsRequest,
type SkillListAnswer,
type ToolListAnswer,
silentLogger,
type Logger,
} from "@invisible-dots/shared";
import { GuestRequestError } from "./errors.js";
import { silentLogger, type Logger } from "./logger.js";
import { GUEST_PORT_HOST } from "./ports.js";
export interface GuestClientOptions {
@@ -307,12 +309,8 @@ export class GuestClient {
return this.json({ path: this.agentPath(AGENT_ROUTES.health) });
}
pushSecrets(openrouterApiKey: string): Promise<void> {
return this.noContent({
method: "POST",
path: this.agentPath(AGENT_ROUTES.secrets),
body: { openrouter_api_key: openrouterApiKey },
});
pushSecrets(secrets: SecretsRequest): Promise<void> {
return this.noContent({ method: "POST", path: this.agentPath(AGENT_ROUTES.secrets), body: secrets });
}
putConfig(config: DotRuntimeConfig): Promise<void> {
+1 -1
View File
@@ -76,9 +76,9 @@ export * from "./errors.js";
export * from "./guest-client.js";
export * from "./host-doctor.js";
export * from "./host.js";
export * from "./logger.js";
export * from "./ports.js";
export * from "./powershell.js";
export * from "./process-file.js";
export * from "./qemu-args.js";
export * from "./runner.js";
export * from "./seed.js";
-27
View File
@@ -1,27 +0,0 @@
/** A minimal structured logger; the API process passes its own (pino-compatible shape). */
export interface Logger {
debug(message: string, fields?: Record<string, unknown>): void;
info(message: string, fields?: Record<string, unknown>): void;
warn(message: string, fields?: Record<string, unknown>): void;
error(message: string, fields?: Record<string, unknown>): void;
}
function write(level: string, message: string, fields?: Record<string, unknown>): void {
const suffix = fields && Object.keys(fields).length > 0 ? ` ${JSON.stringify(fields)}` : "";
process.stderr.write(`[vm-manager] ${level} ${message}${suffix}\n`);
}
/** Logs info and above to stderr, one line each. */
export const stderrLogger: Logger = {
debug: () => {},
info: (message, fields) => write("info", message, fields),
warn: (message, fields) => write("warn", message, fields),
error: (message, fields) => write("error", message, fields),
};
export const silentLogger: Logger = {
debug: () => {},
info: () => {},
warn: () => {},
error: () => {},
};
+51
View File
@@ -0,0 +1,51 @@
/**
* The pid file of a Dot's VM (architecture section 3.2): what lets a control plane that restarted find the QEMU it
* left running. The VmManager decides from it whether that QEMU still runs and may be killed (`processIsOurs`).
*/
import { readFile, writeFile } from "node:fs/promises";
import { replaceFile } from "@invisible-dots/shared";
/**
* The QEMU process of a Dot, the guest port it forwards, and the host's uptime when it was spawned. A host uptime
* lower than that one means the host restarted since, so the process is gone whatever now runs under that pid.
*/
export interface ProcessFile {
pid: number;
guest_port: number;
host_uptime_s?: number;
}
export function parseProcessFile(text: string): ProcessFile | undefined {
try {
const value = JSON.parse(text) as Partial<ProcessFile>;
const pid = value.pid;
const port = value.guest_port;
if (Number.isSafeInteger(pid) && pid! > 0 && Number.isInteger(port) && port! >= 1 && port! <= 65535) {
const record: ProcessFile = { pid: pid!, guest_port: port! };
if (typeof value.host_uptime_s === "number" && Number.isFinite(value.host_uptime_s)) record.host_uptime_s = value.host_uptime_s;
return record;
}
} catch {
// Falls through: not JSON is as unreadable as JSON of the wrong shape.
}
return undefined;
}
/** The pid file at `path`: undefined when there is none, "unreadable" when it is not one this code wrote. */
export async function readProcessFile(path: string): Promise<ProcessFile | "unreadable" | undefined> {
let text: string;
try {
text = await readFile(path, "utf8");
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") return undefined;
throw error;
}
return parseProcessFile(text) ?? "unreadable";
}
/** Written whole or not at all (a temporary file renamed over it), so a reader never sees half a pid file. */
export async function writeProcessFile(path: string, record: ProcessFile): Promise<void> {
const temporary = `${path}.tmp`;
await writeFile(temporary, `${JSON.stringify(record)}\n`, { mode: 0o600 });
await replaceFile(temporary, path);
}
+19 -3
View File
@@ -3,7 +3,7 @@
* one function that builds it, on every host; only the accelerator differs,
* and it arrives here as a value from host.ts.
*/
import { GUEST_PORT, vmName } from "@invisible-dots/shared";
import { CONFIG_BOUNDS, GUEST_PORT, parseSizeMiB, vmName } from "@invisible-dots/shared";
import { VmManagerError } from "./errors.js";
import type { Accelerator } from "./host.js";
@@ -57,6 +57,21 @@ function integerInRange(label: string, value: number, min: number, max: number):
return value;
}
/**
* The cpus and memory every VM of this project may have, Dots and the image builder's VM alike: the range a Dot's
* config allows (CONFIG_BOUNDS of packages/shared, which owns it), in the units QEMU takes.
*/
export const VM_RESOURCE_BOUNDS = {
cpus: { min: CONFIG_BOUNDS.cpu.min, max: CONFIG_BOUNDS.cpu.max },
memoryMiB: { min: parseSizeMiB(CONFIG_BOUNDS.memory.min), max: parseSizeMiB(CONFIG_BOUNDS.memory.max) },
} as const;
/** Refuses cpus or memory outside VM_RESOURCE_BOUNDS, naming the value. */
export function checkVmResources(spec: { cpus: number; memoryMiB: number }): void {
integerInRange("cpus", spec.cpus, VM_RESOURCE_BOUNDS.cpus.min, VM_RESOURCE_BOUNDS.cpus.max);
integerInRange("memoryMiB", spec.memoryMiB, VM_RESOURCE_BOUNDS.memoryMiB.min, VM_RESOURCE_BOUNDS.memoryMiB.max);
}
/**
* The CPU model of every VM of this project (section 3.4): the host's own
* CPU, without the virtualization extensions. A Dot never runs a hypervisor,
@@ -79,6 +94,7 @@ export const CPU_MODEL = "host,-vmx,-svm";
* changes both.
*/
export function machineArgs(spec: { accelerator: Accelerator; cpus: number; memoryMiB: number }): string[] {
checkVmResources(spec);
return [
"-machine",
"q35",
@@ -87,9 +103,9 @@ export function machineArgs(spec: { accelerator: Accelerator; cpus: number; memo
"-cpu",
CPU_MODEL,
"-smp",
String(integerInRange("cpus", spec.cpus, 1, 16)),
String(spec.cpus),
"-m",
String(integerInRange("memoryMiB", spec.memoryMiB, 256, 64 * 1024)),
String(spec.memoryMiB),
];
}
+25 -86
View File
@@ -14,7 +14,7 @@
* restart the pid file stands in for it, checked as `processIsOurs` says.
*/
import { randomBytes } from "node:crypto";
import { access, appendFile, mkdir, open, readFile, rm, writeFile } from "node:fs/promises";
import { appendFile, mkdir, open, readFile, rm, writeFile } from "node:fs/promises";
import { isAbsolute, join, relative, resolve } from "node:path";
import {
allowlistedEnvironment,
@@ -23,9 +23,12 @@ import {
hostRestartedSince,
hostUptimeSeconds,
isFatalGuestError,
pathExists,
prefixedStderrLogger,
replaceFile,
type DotConfig,
type HostPaths,
type Logger,
type VmState,
} from "@invisible-dots/shared";
import {
@@ -40,9 +43,9 @@ import {
} from "./errors.js";
import { GuestClient, waitForGuestHealth, type GuestClientOptions, type WaitForGuestHealthOptions } from "./guest-client.js";
import { accelerator as hostAccelerator, findQemu, type Accelerator, type QemuInstallation } from "./host.js";
import { stderrLogger, type Logger } from "./logger.js";
import { pickFreePort, portListens } from "./ports.js";
import { CPU_MODEL, qemuArgs, qemuPathArg } from "./qemu-args.js";
import { readProcessFile, writeProcessFile, type ProcessFile } from "./process-file.js";
import { checkVmResources, CPU_MODEL, qemuArgs, qemuPathArg } from "./qemu-args.js";
import {
fileSize,
NodeCommandRunner,
@@ -182,21 +185,6 @@ function assertDotId(dotId: string): void {
}
}
function assertIntInRange(label: string, value: number, min: number, max: number): void {
if (!Number.isInteger(value) || value < min || value > max) {
throw new VmManagerError(`${label} must be an integer from ${min} to ${max}, got ${value}`);
}
}
async function exists(path: string): Promise<boolean> {
try {
await access(path);
return true;
} catch {
return false;
}
}
/** QEMU wrote this when it could not bind the host side of the port forward. */
const PORT_TAKEN = /Could not set up host forwarding rule/i;
@@ -218,18 +206,6 @@ const CPU_MODEL_FAILURES = [/CPU model 'host' requires/i, /unable to find CPU mo
/** How much of the end of a log goes into an error: enough for QEMU's last words and the guest's last boot lines. */
const LOG_TAIL_BYTES = 4 * 1024;
/**
* The pid file (section 3.2): the QEMU process of a Dot, the guest port it
* forwards, and the host's uptime when it was spawned. A host uptime lower
* than that one means the host restarted since, so the process is gone
* whatever now runs under that pid.
*/
export interface ProcessFile {
pid: number;
guest_port: number;
host_uptime_s?: number;
}
/** seed-instance.json, or undefined for one this code did not write: the next seed then takes the content's own id. */
function parsePreviousSeed(text: string): PreviousSeed | undefined {
try {
@@ -241,22 +217,6 @@ function parsePreviousSeed(text: string): PreviousSeed | undefined {
return undefined;
}
function parseProcessFile(text: string): ProcessFile | undefined {
try {
const value = JSON.parse(text) as Partial<ProcessFile>;
const pid = value.pid;
const port = value.guest_port;
if (Number.isSafeInteger(pid) && pid! > 0 && Number.isInteger(port) && port! >= 1 && port! <= 65535) {
const record: ProcessFile = { pid: pid!, guest_port: port! };
if (typeof value.host_uptime_s === "number" && Number.isFinite(value.host_uptime_s)) record.host_uptime_s = value.host_uptime_s;
return record;
}
} catch {
// Falls through: not JSON is as unreadable as JSON of the wrong shape.
}
return undefined;
}
/** A QEMU this process spawned: its exit is set the moment Node reports it. */
interface OwnedProcess {
pid: number;
@@ -290,7 +250,7 @@ export class VmManager {
this.options = options;
this.env = options.env ?? process.env;
this.paths = options.paths ?? hostPaths(this.env);
this.logger = options.logger ?? stderrLogger;
this.logger = options.logger ?? prefixedStderrLogger("vm-manager");
this.runner = options.runner ?? new NodeCommandRunner();
this.processes = options.processes ?? new NodeProcessControl();
this.sleep = options.sleep ?? ((ms) => new Promise((resolve) => setTimeout(resolve, ms)));
@@ -389,8 +349,7 @@ export class VmManager {
private validateSpec(spec: VmSpec): void {
assertDotId(spec.dotId);
assertIntInRange("cpus", spec.cpus, 1, 16);
assertIntInRange("memoryMiB", spec.memoryMiB, 256, 64 * 1024);
checkVmResources(spec);
if (!Number.isSafeInteger(spec.diskBytes) || spec.diskBytes <= 0) {
throw new VmManagerError(`diskBytes must be a positive integer, got ${spec.diskBytes}`);
}
@@ -400,6 +359,16 @@ export class VmManager {
}
}
/** Both images of the spec exist: a VM is never created or started on a path that leads nowhere. */
private async assertImages(spec: VmSpec): Promise<void> {
for (const [label, path] of [
["golden image", spec.goldenImage],
["runtime ISO", spec.runtimeImage],
] as const) {
if (!(await pathExists(path))) throw new VmManagerError(`${label} ${path} does not exist: build it with "invisible-dots image build" first`);
}
}
private async qemuImg(args: string[]) {
const { img } = await this.qemu();
return this.runner.run(img, args, { timeoutMs: this.options.qemuImgTimeoutMs ?? 300_000 });
@@ -426,12 +395,7 @@ export class VmManager {
async create(spec: VmSpec): Promise<CreateVmResult> {
this.validateSpec(spec);
return this.withLock(spec.dotId, async () => {
for (const [label, path] of [
["golden image", spec.goldenImage],
["runtime ISO", spec.runtimeImage],
] as const) {
if (!(await exists(path))) throw new VmManagerError(`${label} ${path} does not exist: build it with "invisible-dots image build" first`);
}
await this.assertImages(spec);
const current = await this.probe(spec.dotId);
if (current.state !== "STOPPED") {
throw new VmStateError(spec.dotId, `the VM is ${current.state}; stop it before creating its disk and seed again`);
@@ -440,7 +404,7 @@ export class VmManager {
await mkdir(dir, { recursive: true, mode: 0o700 });
const diskPath = this.paths.diskPath(spec.dotId);
let diskCreated = false;
if (await exists(diskPath)) {
if (await pathExists(diskPath)) {
this.logger.warn("overlay already exists, keeping it", { dotId: spec.dotId, disk: diskPath });
} else {
await this.qemuImg(["create", "-f", "qcow2", "-F", "qcow2", "-b", qemuPathArg("golden image", spec.goldenImage), qemuPathArg("disk", diskPath), String(spec.diskBytes)]);
@@ -452,26 +416,6 @@ export class VmManager {
});
}
/** The pid file: undefined when there is none, "unreadable" when it is not one this code wrote. */
private async readProcessFile(dotId: string): Promise<ProcessFile | "unreadable" | undefined> {
let text: string;
try {
text = await readFile(this.paths.processFilePath(dotId), "utf8");
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") return undefined;
throw error;
}
return parseProcessFile(text) ?? "unreadable";
}
/** Written whole or not at all (a temporary file renamed over it), so a reader never sees half a pid file. */
private async writeProcessFile(dotId: string, record: ProcessFile): Promise<void> {
const path = this.paths.processFilePath(dotId);
const temporary = `${path}.tmp`;
await writeFile(temporary, `${JSON.stringify(record)}\n`, { mode: 0o600 });
await replaceFile(temporary, path);
}
/** Forget a QEMU that is gone: its pid file, and its ChildProcess if this process held it. */
private async forget(dotId: string): Promise<void> {
this.owned.delete(dotId);
@@ -492,7 +436,7 @@ export class VmManager {
}
private async qemuRuns(dotId: string): Promise<boolean> {
const record = await this.readProcessFile(dotId);
const record = await readProcessFile(this.paths.processFilePath(dotId));
return record !== undefined && record !== "unreadable" && this.runs(dotId, record);
}
@@ -517,7 +461,7 @@ export class VmManager {
/** Read a VM's state from its pid file and its process (section 3.4). */
private async probe(dotId: string): Promise<Probe> {
const record = await this.readProcessFile(dotId);
const record = await readProcessFile(this.paths.processFilePath(dotId));
if (record === undefined) return STOPPED;
const file = this.paths.processFilePath(dotId);
if (record === "unreadable") {
@@ -589,13 +533,8 @@ export class VmManager {
}
await this.forget(dotId);
const diskPath = this.paths.diskPath(dotId);
if (!(await exists(diskPath))) throw new VmStateError(dotId, `${diskPath} does not exist; create the VM first`);
for (const [label, path] of [
["golden image", spec.goldenImage],
["runtime ISO", spec.runtimeImage],
] as const) {
if (!(await exists(path))) throw new VmManagerError(`${label} ${path} does not exist: build it with "invisible-dots image build" first`);
}
if (!(await pathExists(diskPath))) throw new VmStateError(dotId, `${diskPath} does not exist; create the VM first`);
await this.assertImages(spec);
await this.writeSeed(spec);
const qemu = await this.qemu();
const accel = this.accelerator;
@@ -641,7 +580,7 @@ export class VmManager {
const record: ProcessFile = { pid: child.pid, guest_port: guestPort, host_uptime_s: hostUptimeSeconds() };
try {
// Before anything can fail: from here on a restarted control plane finds this QEMU.
await this.writeProcessFile(dotId, record);
await writeProcessFile(this.paths.processFilePath(dotId), record);
} catch (error) {
await this.killOurs(dotId, record);
await this.forget(dotId);
+179
View File
@@ -0,0 +1,179 @@
/**
* The host's GuestClient against the real dot-agentd, not a stand-in: the daemon built with the image builder's one
* build, run as its README's development mode describes (`--run-as=`: the model's work as the daemon's own user;
* `--listen 127.0.0.1:0`), with a guest config holding the Dot's token, and a stand-in for the engine on the unix
* socket the daemon proxies `/v1/agent` to. What the two sides each test alone with a fake of the other, this tests
* across the seam: the proof, the token, the routes, the bodies, and an event stream passed on as it is written.
*
* Linux only (vitest.config.ts leaves `*.linux.test.ts` out elsewhere): the daemon is a Linux program, its exec route
* runs /bin/bash and the engine's socket is a unix socket. It needs Go, which CI's node job sets up from go.mod.
*/
import { execFile, spawn, type ChildProcess } from "node:child_process";
import { mkdir, mkdtemp, realpath, rm, writeFile } from "node:fs/promises";
import { createServer, get, type Server, type ServerResponse } from "node:http";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { AGENTD_ROUTES, GUEST_UNPROVEN, type OutboundEvent } from "@invisible-dots/shared";
import { agentdBuildCommand } from "@invisible-dots/image-builder";
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import { GuestClient, GuestRequestError } from "../src/index.js";
const REPO = resolve(fileURLToPath(new URL(".", import.meta.url)), "../../..");
const TOKEN = "integration-token-0123456789";
let dir: string;
let home: string;
let daemon: ChildProcess | undefined;
let agent: Server | undefined;
let port: number;
/** What reached the stand-in engine through the daemon's proxy, as `METHOD /path?query`. */
const agentSeen: string[] = [];
/** The open event streams the stand-in holds, to write the next event into. */
const streams: ServerResponse[] = [];
function event(seq: number): OutboundEvent {
return { seq, id: `evt_${seq}`, type: "automation.next_run", ts: "2026-10-11T10:00:00.000Z", data: { next_run_at_ms: seq } };
}
function sse(res: ServerResponse, seq: number): void {
res.write(`id: ${seq}\ndata: ${JSON.stringify(event(seq))}\n\n`);
}
/** The engine's side of the proxy: /health, and /events/stream, which sends one event and keeps the stream open. */
function startAgent(socket: string): Promise<Server> {
const server = createServer((req, res) => {
const url = new URL(req.url!, "http://agent");
agentSeen.push(`${req.method} ${url.pathname}${url.search}`);
if (url.pathname === "/health") {
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify({ status: "ok", state: "idle", openrouter_configured: true, browser: { identities: 0, open: 0 }, checks: {} }));
return;
}
if (url.pathname === "/events/stream") {
res.writeHead(200, { "content-type": "text/event-stream", "cache-control": "no-cache" });
sse(res, Number(url.searchParams.get("after") ?? 0) + 1);
streams.push(res);
return;
}
res.writeHead(404, { "content-type": "application/json" }).end(JSON.stringify({ error: "not_found" }));
});
return new Promise((done, fail) => {
server.once("error", fail);
server.listen(socket, () => done(server));
});
}
/** Starts the daemon and answers the TCP port it listens on, from its log line `listening on="tcp 127.0.0.1:<port>"`. */
function startDaemon(binary: string, args: string[]): Promise<{ child: ChildProcess; port: number }> {
const child = spawn(binary, args, { stdio: ["ignore", "ignore", "pipe"] });
return new Promise((done, fail) => {
let log = "";
child.stderr!.setEncoding("utf8");
child.stderr!.on("data", (chunk: string) => {
log += chunk;
const listening = /listening on="tcp 127\.0\.0\.1:(\d+)"/.exec(log);
if (listening) done({ child, port: Number(listening[1]) });
});
child.once("error", fail);
child.once("exit", (code, signal) => fail(new Error(`dot-agentd exited (${code ?? signal}) before it listened:\n${log}`)));
});
}
beforeAll(async () => {
dir = await mkdtemp(join(tmpdir(), "idots-agentd-"));
home = join(dir, "home");
await mkdir(home);
const binary = join(dir, "dot-agentd");
const build = agentdBuildCommand(REPO, binary);
await promisify(execFile)(build.command, build.args, { env: { ...process.env, ...build.env } });
const config = join(dir, "config.json");
await writeFile(config, JSON.stringify({ dotId: "dot_integration", token: TOKEN }), { mode: 0o600 });
const agentSocket = join(dir, "agent.sock");
agent = await startAgent(agentSocket);
const started = await startDaemon(binary, [
"--config", config,
"--home", home,
"--run-dir", join(dir, "run"),
"--agent-socket", agentSocket,
"--listen", "127.0.0.1:0",
"--run-as=",
]);
daemon = started.child;
port = started.port;
// The build: a cold Go cache compiles the standard library for linux/amd64 first, and a host whose Go is older
// than go.mod's toolchain downloads that one.
}, 600_000);
afterAll(async () => {
for (const res of streams) res.destroy();
if (daemon && daemon.exitCode === null && daemon.signalCode === null) {
const exited = new Promise((done) => daemon!.once("exit", done));
daemon.kill("SIGTERM");
await exited;
}
await new Promise<void>((done) => (agent ? agent.close(() => done()) : done()));
if (dir) await rm(dir, { recursive: true, force: true });
});
describe("GuestClient against dot-agentd", () => {
it("is given the proof of the token before it sends the token", async () => {
await expect(new GuestClient(port, TOKEN).prove()).resolves.toBeUndefined();
const impostor = new GuestClient(port, "another-dot-token");
await expect(impostor.health()).rejects.toMatchObject({ code: GUEST_UNPROVEN });
});
it("is refused without the token: 401 with the Bearer challenge", async () => {
const answer = await new Promise<{ status: number; challenge: string | undefined; body: string }>((done, fail) => {
get({ host: "127.0.0.1", port, path: AGENTD_ROUTES.health }, (res) => {
let body = "";
res.setEncoding("utf8");
res.on("data", (chunk: string) => (body += chunk));
res.on("end", () => done({ status: res.statusCode ?? 0, challenge: res.headers["www-authenticate"], body }));
}).once("error", fail);
});
expect(answer.status).toBe(401);
expect(answer.challenge).toBe('Bearer realm="dot-agentd"');
expect(JSON.parse(answer.body)).toMatchObject({ error: "unauthorized" });
});
it("reads health with the engine's own answer passed through", async () => {
const health = await new GuestClient(port, TOKEN).health();
expect(health).toMatchObject({ agentd: "ok", agent: { status: "ok", state: "idle", openrouter_configured: true } });
expect(agentSeen).toContain("GET /health");
});
it("runs a command in home and reads its exit code and both streams", async () => {
const client = new GuestClient(port, TOKEN);
expect(await client.exec({ command: "printf out; printf err >&2; exit 3" })).toMatchObject({ exit_code: 3, stdout: "out", stderr: "err", timed_out: false });
expect((await client.exec({ command: "pwd" })).stdout.trim()).toBe(await realpath(home));
});
it("writes, reads and lists a file under home, and refuses one outside it", async () => {
const client = new GuestClient(port, TOKEN);
await client.writeFile("notes/hello.txt", "hello from the host");
expect((await client.readFile("~/notes/hello.txt")).toString("utf8")).toBe("hello from the host");
const listing = await client.listFiles("notes");
expect(listing.entries).toEqual([expect.objectContaining({ name: "hello.txt", type: "file", size: 19 })]);
const outside = await client.readFile("/etc/hostname").catch((error: unknown) => error);
expect(outside).toBeInstanceOf(GuestRequestError);
expect(outside).toMatchObject({ status: 403, code: "outside_home" });
});
it("proxies the engine's routes, and its event stream as it is written", async () => {
const client = new GuestClient(port, TOKEN);
expect(await client.agentHealth()).toMatchObject({ status: "ok", state: "idle" });
const controller = new AbortController();
const got: number[] = [];
for await (const evt of client.events({ after: 4, signal: controller.signal })) {
got.push(evt.seq);
// The first event arrived while the engine still holds the stream open: the proxy did not wait for its end.
// The next one is written only now.
if (got.length === 1) sse(streams.at(-1)!, 6);
else controller.abort();
}
expect(got).toEqual([5, 6]);
expect(agentSeen).toContain("GET /events/stream?after=4");
});
});
+3 -3
View File
@@ -119,7 +119,7 @@ describe("GuestClient", () => {
}
});
const client = new GuestClient(port, TOKEN);
await client.pushSecrets("sk-or-test");
await client.pushSecrets({ openrouter_api_key: "sk-or-test", mcp_secrets: {} });
await client.putConfig({ name: "n" } as never);
expect(await client.postEvent({ id: "e1", type: "user.message", ts: "2026-10-02T10:00:00Z", data: { text: "hi" } })).toEqual({ accepted: true });
expect((await client.state()).state).toBe("IDLE");
@@ -127,7 +127,7 @@ describe("GuestClient", () => {
expect((await client.createBrowserIdentity({ name: "shop" })).id).toBe("shop-abc123");
await client.deleteBrowserIdentity("shop-abc123");
await client.prepareSleep();
expect(JSON.parse(seen[0]!.body)).toEqual({ openrouter_api_key: "sk-or-test" });
expect(JSON.parse(seen[0]!.body)).toEqual({ openrouter_api_key: "sk-or-test", mcp_secrets: {} });
expect(seen.map((s) => s.url)).toContain("/v1/agent/browser-identities/shop-abc123");
});
@@ -216,7 +216,7 @@ describe("GuestClient", () => {
// What takes over a stale guest port after a host restart or a QEMU that died.
const { port, seen, proofs } = await serve((_req, res) => json(res, 200, health("ok")), { impostor: true });
const client = new GuestClient(port, TOKEN);
await expect(client.pushSecrets("sk-or-must-not-leak")).rejects.toMatchObject({ code: GUEST_UNPROVEN });
await expect(client.pushSecrets({ openrouter_api_key: "sk-or-must-not-leak", mcp_secrets: {} })).rejects.toMatchObject({ code: GUEST_UNPROVEN });
await expect(waitForGuestHealth(client, { intervalMs: 5, timeoutMs: 5000 })).rejects.toMatchObject({ code: GUEST_UNPROVEN });
expect(proofs.length).toBeGreaterThan(0);
expect(proofs.every((p) => p.auth === undefined)).toBe(true);
+1 -2
View File
@@ -3,7 +3,7 @@ import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { createServer, type Server } from "node:net";
import { tmpdir, uptime } from "node:os";
import { join } from "node:path";
import { hostPaths, type HostPaths } from "@invisible-dots/shared";
import { hostPaths, type HostPaths, silentLogger } from "@invisible-dots/shared";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import {
AcceleratorUnavailableError,
@@ -18,7 +18,6 @@ import {
VmStateError,
generateToken,
qemuArgs,
silentLogger,
vmSpecFromConfig,
type VmManagerOptions,
type VmSpec,
+3 -3
View File
@@ -28,11 +28,11 @@ test("the Activity tab reads the log as lines, filters by family at the host, fo
const rows = page.getByRole("list", { name: "Events" }).getByRole("listitem");
await expect(rows.filter({ hasText: "Ran a command" })).toHaveCount(2);
const allowed = rows.filter({ hasText: "ls -la /home/dot" });
await expect(allowed).toContainText("ok in 40 ms | exec [computer.exec] allow");
// The refused call is an error line, and says it never ran.
await expect(allowed).toContainText("done in 40 ms | exec [computer.exec] allow");
// The refused call is a warning line (the config refused it; nothing failed), and says it never ran.
const refused = rows.filter({ hasText: "rm -rf /" });
await expect(refused).toContainText("denied in 0 ms | exec [computer.exec] deny");
await expect(refused).toHaveAttribute("data-tone", "error");
await expect(refused).toHaveAttribute("data-tone", "warn");
await expect(rows.filter({ hasText: "You sent a message" })).toContainText("hello there");
// The engine's own start, in words, not as empty data.
await expect(rows.filter({ hasText: "The engine started" })).toContainText("The key was sent again");
+34 -1
View File
@@ -37,7 +37,7 @@ test("allow, ask and deny a permission: the host saves one entry, the Dot's engi
await expect(page.getByText("Saved. The change applies from the Dot's next turn.")).toBeVisible();
expect((await harness.api.getDot(dot.id)).config.permissions).toEqual({ "computer.exec": "deny" });
await expect.poll(() => guest.config?.permissions["computer.exec"]).toBe("deny");
expect((await harness.api.listTools(dot.id)).find((tool) => tool.name === "exec")?.offered).toBe(false);
expect((await harness.api.listTools(dot.id)).tools.find((tool) => tool.name === "exec")?.offered).toBe(false);
await expect(exec.getByRole("listitem")).toHaveText(["execnot offered"]);
// Ask: the tool is offered again, and the Dot will wait for an answer.
@@ -59,6 +59,39 @@ test("allow, ask and deny a permission: the host saves one entry, the Dot's engi
await expect(row(page, "Automations").getByRole("radio", { name: "Ask" })).toBeChecked();
});
test("an MCP server is added in the settings, its secret reaches the Dot's engine and never the page, and its state shows", async ({ page, harness }) => {
const dot = await harness.createDot("config-mcp");
const guest = harness.driver.guestOf(dot.id);
guest.mcpTools.set("search", [{ name: "mcp_search_query", description: "Search." }]);
await page.goto(settings(harness.webUrl, dot.id));
const panel = page.locator("section", { has: page.getByRole("heading", { level: 2, name: "MCP servers" }) });
await panel.getByRole("button", { name: "Add an MCP server" }).click();
const form = page.getByRole("form", { name: "New MCP server" });
await form.getByLabel("Name").fill("search");
await form.getByText("A URL").click();
await form.getByRole("textbox", { name: "URL" }).fill("https://search.example/mcp");
await form.getByLabel(/^Secrets/).fill("Authorization");
await form.getByRole("button", { name: "Add" }).click();
await saveReviewed(page);
await expect.poll(async () => (await harness.api.getDot(dot.id)).config.mcp_servers).toEqual({
search: { url: "https://search.example/mcp", headers: {}, secrets: ["Authorization"], timeout_s: 120, startup_timeout_s: 60 },
});
// Saved, the server's secret can be set; the engine gets it, the page keeps nothing of it.
const secret = panel.getByLabel("Authorization");
await secret.fill("Bearer e2e-token");
await panel.getByRole("button", { name: "Set" }).click();
await expect(secret).toHaveValue("");
await expect.poll(() => guest.mcpSecrets).toEqual({ search: { Authorization: "Bearer e2e-token" } });
expect(await page.content()).not.toContain("e2e-token");
// Where it is, as the engine says, and its permission among the others, asked by default.
await expect(panel.getByText("Connected, 1 tool")).toBeVisible();
await expect(row(page, "MCP server search").getByRole("radio", { name: "Ask" })).toBeChecked();
await expect(page.getByRole("list", { name: "Tools of MCP server search" }).getByRole("listitem")).toHaveText(["mcp_search_query"]);
});
test("a preset sets the rows together, and the review lists every row that moved", async ({ page, harness }) => {
const dot = await harness.createDot("config-preset");
await page.goto(settings(harness.webUrl, dot.id));
+8 -1
View File
@@ -160,7 +160,7 @@ test("the old addresses lead to the Inbox, a Dot's approvals to the Inbox filter
await expect(page.getByRole("navigation", { name: "Dot sections" }).getByRole("link", { name: /Approvals/ })).toHaveCount(0);
});
test("the Inbox is answered from the keyboard alone: j and k choose, a allows once, d denies, and a command waits for its confirmation", async ({ page, fresh: harness }) => {
test("the Inbox is answered from the keyboard alone: Tab to the list, j and k choose, a allows once, d denies, and a command waits for its confirmation", async ({ page, fresh: harness }) => {
const dot = await harness.createDot("inbox-keys");
const guest = harness.driver.guestOf(dot.id);
const one = ask(guest, navigate("https://example.com/one"));
@@ -171,6 +171,13 @@ test("the Inbox is answered from the keyboard alone: j and k choose, a allows on
const selected = page.locator("article[aria-current=true]");
await expect(selected).toHaveAttribute("data-approval-id", one);
// A key typed while the focus is elsewhere on the page answers nothing (WCAG 2.1.4): the keys are the list's.
await page.keyboard.press("d");
await page.keyboard.press("a");
expect(await harness.api.listApprovals("pending")).toHaveLength(3);
const list = page.getByRole("list", { name: "Approvals waiting for your answer" });
while (!(await list.evaluate((element) => element === document.activeElement))) await page.keyboard.press("Tab");
await page.keyboard.press("a");
await expect(selected.getByRole("status")).toHaveText("Allowed");
await page.keyboard.press("j");
+1 -1
View File
@@ -11,7 +11,7 @@
"scripts": {
"dev": "next dev --webpack --hostname 127.0.0.1 --port 3000",
"build": "node scripts/build.mjs",
"test": "vitest run",
"test": "vitest run --root ../.. apps/web/",
"typecheck": "tsc -p tsconfig.json",
"test:e2e": "playwright test"
},
+2 -2
View File
@@ -4,8 +4,8 @@ import { AttentionProvider } from "../../components/shell/attention";
import { AppShell } from "../../components/shell/AppShell";
/**
* Everything behind the session: one live stream, what needs the person, and the frame with the rail. The login
* page lives outside this group on purpose, because it has no session and so nothing here may call the API.
* Every page of the app: one live stream, what needs the person, and the frame with the rail. The page for an address
* with nothing behind it lives outside this group, so it shows without them.
*/
export default function AppLayout({ children }: { children: ReactNode }) {
return (
+4 -2
View File
@@ -2,7 +2,7 @@
import Link from "next/link";
import { usePathname, useRouter } from "next/navigation";
import { createContext, useContext, useEffect, type ReactNode } from "react";
import { createContext, useContext, useEffect, useMemo, type ReactNode } from "react";
import { api, ApiError } from "../lib/api";
import type { Dot } from "../lib/types";
import { DotHeader } from "./dot/DotHeader";
@@ -49,6 +49,8 @@ export function DotShell({ dotId, children }: { dotId: string; children: ReactNo
if (address) router.replace(`/dots/${encodeURIComponent(canonical)}${pathname.slice(address[0].length)}${window.location.search}${window.location.hash}`);
}, [canonical, pathname, router]);
const value = useMemo(() => ({ dotId, dot }), [dotId, dot]);
// A Dot the control plane does not have (deleted, or never this host's): no header, tabs or composer for it.
if (dot.error instanceof ApiError && dot.error.status === 404) {
return (
@@ -65,7 +67,7 @@ export function DotShell({ dotId, children }: { dotId: string; children: ReactNo
}
return (
<DotContext.Provider value={{ dotId, dot }}>
<DotContext.Provider value={value}>
<div className="flex min-h-0 flex-1 flex-col">
<DotHeader dotId={dotId} dot={dot} />
<DotTabs dotId={dotId} />
@@ -1,7 +1,7 @@
"use client";
import { BrainIcon, CircleDotIcon, FlagIcon, GlobeIcon, HandIcon, MessageSquareIcon, MonitorIcon, SendIcon, SparklesIcon, type LucideIcon } from "lucide-react";
import { toolLabel } from "../../lib/events/tool-labels";
import { toolLabel } from "@invisible-dots/shared/browser";
import type { EventFamily, EventView } from "../../lib/events/view";
import { formatDate } from "../../lib/format";
import type { StoredEvent } from "../../lib/types";
@@ -1,9 +1,9 @@
"use client";
import { mergeEvents, readRecentEvents } from "@invisible-dots/sdk";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import { ACTIVITY_PAGE, typesFor } from "../../lib/activity";
import { api } from "../../lib/api";
import { mergeEvents, readRecentEvents } from "../../lib/event-log";
import type { EventFamily } from "../../lib/events/view";
import type { StoredEvent } from "../../lib/types";
import { useLiveEvents } from "../events";
@@ -2,8 +2,7 @@
import Link from "next/link";
import { useState } from "react";
import { approvalBody, askTitle, boundedJson, isDestructive, permissionInfo, RECEIPT_WORD, RISK_LABEL, type ApprovalAsk } from "../../lib/approval-view";
import { toolLabel } from "../../lib/events/tool-labels";
import { approvalBody, askTitle, boundedJson, isDestructive, permissionInfo, RECEIPT_WORD, type ApprovalAsk } from "../../lib/approval-view";
import { formatDate } from "../../lib/format";
import { relativeTime } from "../../lib/time";
import { cn } from "../../lib/utils";
@@ -11,7 +10,7 @@ import { ErrorAlert } from "../ErrorAlert";
import { ApprovalCard as ApprovalCardElement } from "../elements/approval-card";
import { DotAvatar } from "../shell/DotAvatar";
import { FAMILY_ICON } from "../tool-family-icon";
import { APPROVAL_NOTE_MAX } from "@invisible-dots/shared/browser";
import { APPROVAL_NOTE_MAX, permissionPhrase, toolLabel } from "@invisible-dots/shared/browser";
import { Button } from "../ui/button";
import { Input } from "../ui/input";
import { Label } from "../ui/label";
@@ -80,7 +79,7 @@ export function ApprovalCard({ ask, answers, dotName, selected = false, classNam
{" · "}
</>
) : null}
<span>{info ? `${info.label} (${RISK_LABEL[info.risk].toLowerCase()})` : ask.permission}</span>
<span>{permissionPhrase(ask.permission)}</span>
{" · "}
<Link href={where.href} className="underline underline-offset-2 hover:text-foreground">
{where.words}
@@ -1,7 +1,8 @@
"use client";
import { RISK_LABEL } from "@invisible-dots/shared/browser";
import { api } from "../../lib/api";
import { permissionInfo, RISK_LABEL, toolsCovered, type ApprovalAsk } from "../../lib/approval-view";
import { permissionInfo, toolsCovered, type ApprovalAsk } from "../../lib/approval-view";
import { useResource } from "../ui";
import { Button } from "../ui/button";
import { Dialog, DialogContent, DialogDescription, DialogFooter, DialogHeader, DialogTitle } from "../ui/dialog";
@@ -10,7 +11,7 @@ import { Dialog, DialogContent, DialogDescription, DialogFooter, DialogHeader, D
function useToolTable(dotId: string) {
return useResource(async () => {
try {
return await api.listTools(dotId);
return (await api.listTools(dotId)).tools;
} catch {
return null;
}
@@ -1,7 +1,7 @@
import type { ChannelKind, ChannelRecord } from "@invisible-dots/shared/browser";
import { accountLabel, type ChannelKind, type ChannelRecord } from "@invisible-dots/shared/browser";
import { MessageCircleIcon, SendIcon } from "lucide-react";
import type { ReactNode } from "react";
import { accountHref, accountLabel, CHANNEL_LABELS } from "../../lib/channels";
import { accountHref, CHANNEL_LABELS } from "../../lib/channels";
import { ChannelStateChip } from "./state-chip";
const ICONS = { telegram: SendIcon, whatsapp: MessageCircleIcon } as const satisfies Record<ChannelKind, unknown>;
@@ -1,14 +1,13 @@
// Derived from nanobot webui/src/components/settings/channels/ChannelQrConnectFlow.tsx at 9dc0aba, MIT; changed: the flow reads the host's stream of frames instead of polling a session (its states are the pure reducer in lib/channel-link.ts), the code is drawn as an SVG from the QR modules instead of a canvas data URL, there is no i18n and no client provider, and cancelling removes the channel the start created, or only pauses one that holds people.
"use client";
import type { ChannelRecord } from "@invisible-dots/shared/browser";
import { accountLabel, type ChannelRecord } from "@invisible-dots/shared/browser";
import { CheckIcon, Loader2Icon, RotateCcwIcon, SmartphoneIcon } from "lucide-react";
import QRCode from "qrcode";
import { useCallback, useEffect, useMemo, useReducer, useRef } from "react";
import { toast } from "sonner";
import { api } from "../../lib/api";
import { IDLE, linkEnded, reduceLink, STREAM_LOST, type LinkView } from "../../lib/channel-link";
import { accountLabel } from "../../lib/channels";
import { ErrorAlert } from "../ErrorAlert";
import { useAction } from "../ui";
import { Alert, AlertDescription, AlertTitle } from "../ui/alert";
@@ -1,9 +1,9 @@
"use client";
import { accountLabel } from "@invisible-dots/shared/browser";
import { useState, type FormEvent } from "react";
import { toast } from "sonner";
import { api } from "../../lib/api";
import { accountLabel } from "../../lib/channels";
import { ErrorAlert } from "../ErrorAlert";
import { Field } from "../new-dot/Field";
import { useAction } from "../ui";
+8 -6
View File
@@ -2,15 +2,18 @@
import { ChevronRightIcon, HandIcon } from "lucide-react";
import { useState } from "react";
import { clusterSummary, groupActivity, type ActivityItem, type ApprovalStep, type ToolStep } from "../../lib/chat-thread";
import { formatDate, formatMillis } from "../../lib/format";
import { clusterSummary, groupActivity, type ActivityItem } from "../../lib/chat-thread";
import { formatMillis } from "../../lib/format";
import { APPROVAL_WORD, TOOL_OUTCOME, type ApprovalStep, type ToolStep } from "../../lib/steps";
import type { Tone } from "../../lib/tone";
import { cn } from "../../lib/utils";
import { InlineApproval } from "../approvals/InlineApproval";
import { FAMILY_ICON } from "../tool-family-icon";
import { ActivityStep, type ActivityStepTone } from "./activity-step";
import { TOOL_STATUS, ToolStatusWord } from "./tool-status";
const STATE_TONE: Record<ToolStep["state"], ActivityStepTone> = { ok: "neutral", error: "error", denied: "warn", interrupted: "warn" };
/** A step's marker in the tone of how the call ended. */
const MARKER: Record<Tone, ActivityStepTone> = { neutral: "neutral", info: "active", ok: "success", warn: "warn", error: "error" };
/** One tool call as a quiet line: what it did, what it acted on, and how it ended when that was not well. */
export function ToolLine({ step }: { step: ToolStep }) {
@@ -19,7 +22,7 @@ export function ToolLine({ step }: { step: ToolStep }) {
return (
<ActivityStep
icon={FAMILY_ICON[step.family]}
tone={STATE_TONE[step.state]}
tone={MARKER[TOOL_OUTCOME[step.state].tone]}
title={full}
label={
<>
@@ -33,8 +36,6 @@ export function ToolLine({ step }: { step: ToolStep }) {
);
}
const APPROVAL_WORD: Record<Exclude<ApprovalStep["outcome"], "waiting">, string> = { approved: "Allowed", rejected: "Denied" };
/**
* An approval the Dot asked for in the conversation, in the place it was asked. While it waits it is the approval card,
* answerable here; once answered it is a receipt, which is what the log keeps.
@@ -58,6 +59,7 @@ export function ApprovalLine({ dotId, step }: { dotId: string; step: ApprovalSte
{APPROVAL_WORD[step.outcome]}
{approved && step.always ? " for good" : ""}: {step.label.toLowerCase()}
{step.reason ? <span className="font-normal"> ({step.reason})</span> : null}
{step.note ? <span className="block text-xs font-normal">Note: {step.note}</span> : null}
</span>
}
/>
+1 -1
View File
@@ -1,11 +1,11 @@
"use client";
import { readEventLog, readEventRange, readRecentEvents } from "@invisible-dots/sdk";
import { MAX_EVENT_PAGE } from "@invisible-dots/shared/browser";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import { api } from "../../lib/api";
import { buildThread, CHAT_ACTIVITY_EVENT_TYPES, isChatActivityEvent, mergeChatEvents, unsettled, type PendingMessage, type ThreadItem } from "../../lib/chat-thread";
import { CHAT_PAGE_SIZE, withEarlierPage, withNewestPage, type ChatWindow } from "../../lib/chat-window";
import { readEventLog, readEventRange, readRecentEvents } from "../../lib/event-log";
import { CHAT_EVENT_TYPES } from "../../lib/messages";
import type { ChatMessage, StoredEvent } from "../../lib/types";
import { useLiveEvents, useLiveRefresh } from "../events";
+5 -5
View File
@@ -1,7 +1,7 @@
// Derived from AI Elements (Vercel) packages/elements/src/tool.tsx at 6a9d5b1, Apache-2.0; changed: the states are this app's own (a call is reported when it has ended, so there is no running state; a call that waits for the person belongs to the approval step), and the map gives each a word, an icon and a tone from this app's tokens, not a badge in fixed colors.
import { BanIcon, CheckCircle2Icon, CircleAlertIcon, XCircleIcon, type LucideIcon } from "lucide-react";
import type { ToolState } from "../../lib/chat-thread";
import { TOOL_OUTCOME, type ToolState } from "../../lib/steps";
import { cn } from "../../lib/utils";
export interface ToolStatus {
@@ -15,10 +15,10 @@ export interface ToolStatus {
}
export const TOOL_STATUS: Readonly<Record<ToolState, ToolStatus>> = {
ok: { word: "done", icon: CheckCircle2Icon, tone: "text-ok", attention: false },
error: { word: "failed", icon: XCircleIcon, tone: "text-danger", attention: true },
denied: { word: "denied", icon: BanIcon, tone: "text-warn", attention: true },
interrupted: { word: "interrupted", icon: CircleAlertIcon, tone: "text-warn", attention: true },
ok: { word: TOOL_OUTCOME.ok.word, icon: CheckCircle2Icon, tone: "text-ok", attention: false },
error: { word: TOOL_OUTCOME.error.word, icon: XCircleIcon, tone: "text-danger", attention: true },
denied: { word: TOOL_OUTCOME.denied.word, icon: BanIcon, tone: "text-warn", attention: true },
interrupted: { word: TOOL_OUTCOME.interrupted.word, icon: CircleAlertIcon, tone: "text-warn", attention: true },
};
/** The word of a call's ending, shown beside it only when it is not an ordinary success. */
+1 -1
View File
@@ -1,7 +1,7 @@
"use client";
import { agentStateLabel, type AgentStateValue } from "../../lib/agent";
import type { ApprovalStep, ToolStep } from "../../lib/chat-thread";
import type { ApprovalStep, ToolStep } from "../../lib/steps";
import type { RingState } from "../../lib/attention";
import { DotAvatar } from "../shell/DotAvatar";
@@ -18,8 +18,10 @@ import { ConfigYamlEditor } from "../yaml-editor";
import { DeleteDot } from "./delete-dot";
import { ComputerPanel, GeneralPanel, LimitsPanel, ModelPanel } from "./panels";
import type { PanelProps } from "./panel";
import { McpServersPanel } from "./mcp-servers-panel";
import { PermissionsPanel } from "./permissions-panel";
import { ReviewDialog } from "./review-dialog";
import { useToolTable } from "./tool-table";
import { VmProxy } from "./vm-proxy";
type Mode = "form" | "yaml";
@@ -59,6 +61,8 @@ function Editor({
const [refusal, setRefusal] = useState<FormIssue[] | null>(null);
const [reviewing, setReviewing] = useState(false);
const save = useAction();
// The engine's tool table: the permissions panel lists each permission's tools, the MCP panel each server's state.
const table = useToolTable(dotId, computerState);
const issues = useMemo(() => configIssues(mode === "form" ? state.draft : yaml), [mode, state.draft, yaml]);
const valid = issues.length === 0;
@@ -159,7 +163,8 @@ function Editor({
<>
<GeneralPanel {...panel} />
<ModelPanel {...panel} />
<PermissionsPanel {...panel} dotId={dotId} computerState={computerState} />
<PermissionsPanel {...panel} computerState={computerState} table={table} />
<McpServersPanel {...panel} dotId={dotId} computerState={computerState} table={table} />
<ComputerPanel {...panel} />
<LimitsPanel {...panel} />
</>
@@ -0,0 +1,251 @@
"use client";
import { computerIsUp, isMcpServerName, MCP_STARTUP_TIMEOUT_BOUNDS, MCP_TIMEOUT_BOUNDS, mcpServerNames, type McpSecretState, type McpServerStatus } from "@invisible-dots/shared/browser";
import { useState, type FormEvent } from "react";
import { api } from "../../lib/api";
import { mcpServerSummary, setMcpServer } from "../../lib/config-fields";
import { EMPTY_MCP_SERVER_FORM, formOfServer, serverOfForm, type McpServerForm } from "../../lib/mcp-server-form";
import { cn } from "../../lib/utils";
import { TONE_CLASS } from "../dot/tone";
import { ErrorAlert } from "../ErrorAlert";
import { useLiveRefresh } from "../events";
import { Field, NumberField } from "../new-dot/Field";
import { useAction, useResource } from "../ui";
import { Button } from "../ui/button";
import { Input } from "../ui/input";
import { Textarea } from "../ui/textarea";
import { Panel, type PanelProps } from "./panel";
import type { ToolTable } from "./tool-table";
const STATE_LABEL: Record<McpServerStatus["state"], string> = { connecting: "Starting", connected: "Connected", failed: "Not connected" };
const STATE_TONE: Record<McpServerStatus["state"], keyof typeof TONE_CLASS> = { connecting: "info", connected: "ok", failed: "error" };
/** Where a declared server is on the Dot's computer, as its engine says; nothing while the computer does not answer. */
function ServerState({ status }: { status: McpServerStatus | undefined }) {
if (status === undefined) return null;
return (
<div className="space-y-1">
<span className={cn("rounded-full px-2 py-0.5 text-xs font-medium", TONE_CLASS[STATE_TONE[status.state]])}>
{STATE_LABEL[status.state]}
{status.state === "connected" ? `, ${status.tools === 1 ? "1 tool" : `${status.tools} tools`}` : ""}
</span>
{status.error ? <p className="text-xs text-danger">{status.error}</p> : null}
</div>
);
}
/** The fields of a server's entry; the name is asked only of a new one. */
function ServerFields({ id, form, onChange, errorOf }: { id: string; form: McpServerForm; onChange: (form: McpServerForm) => void; errorOf: (field: string) => string | null }) {
const set = (patch: Partial<McpServerForm>) => onChange({ ...form, ...patch });
return (
<>
<fieldset className="space-y-1.5">
<legend className="text-sm font-medium">How the Dot reaches it</legend>
<div className="inline-flex rounded-md border p-0.5">
{(["command", "url"] as const).map((kind) => (
<label key={kind} className={cn("cursor-pointer rounded px-3 py-1 text-sm text-muted-foreground transition-colors hover:text-foreground has-[:focus-visible]:ring-[3px] has-[:focus-visible]:ring-ring has-[:focus-visible]:outline-hidden", form.kind === kind && "bg-secondary font-medium text-foreground")}>
<input type="radio" className="sr-only" name={`${id}-kind`} value={kind} checked={form.kind === kind} onChange={() => set({ kind })} />
{kind === "command" ? "A program on its computer" : "A URL"}
</label>
))}
</div>
</fieldset>
{form.kind === "command" ? (
<>
<Field id={`${id}-command`} label="Command" hint="Run as the Dot's user in its home, e.g. uvx or npx. The Dot can install what it needs." error={errorOf("command")}>
{(control) => <Input {...control} value={form.command} autoComplete="off" spellCheck={false} placeholder="uvx" onChange={(event) => set({ command: event.target.value })} />}
</Field>
<Field id={`${id}-args`} label="Arguments" optional hint="One per line." error={errorOf("args")}>
{(control) => <Textarea {...control} rows={2} value={form.args} spellCheck={false} placeholder="mcp-server-time" onChange={(event) => set({ args: event.target.value })} />}
</Field>
<Field id={`${id}-env`} label="Environment" optional hint="NAME=value, one per line. A key or a token goes in Secrets instead." error={errorOf("env")}>
{(control) => <Textarea {...control} rows={2} value={form.env} spellCheck={false} onChange={(event) => set({ env: event.target.value })} />}
</Field>
</>
) : (
<>
<Field id={`${id}-url`} label="URL" hint="Streamable HTTP, or SSE for a URL that ends in /sse." error={errorOf("url")}>
{(control) => <Input {...control} value={form.url} autoComplete="off" spellCheck={false} placeholder="https://example.com/mcp" onChange={(event) => set({ url: event.target.value })} />}
</Field>
<Field id={`${id}-headers`} label="Headers" optional hint="Name: value, one per line. A key or a token goes in Secrets instead." error={errorOf("headers")}>
{(control) => <Textarea {...control} rows={2} value={form.headers} spellCheck={false} onChange={(event) => set({ headers: event.target.value })} />}
</Field>
</>
)}
<Field
id={`${id}-secrets`}
label="Secrets"
optional
hint={form.kind === "command" ? "The names of environment variables whose values you set below once saved, one per line." : "The names of headers whose values you set below once saved (Authorization), one per line."}
error={errorOf("secrets")}
>
{(control) => <Textarea {...control} rows={2} value={form.secrets} spellCheck={false} onChange={(event) => set({ secrets: event.target.value })} />}
</Field>
<NumberField
id={`${id}-timeout`}
label="Longest call"
unit="seconds"
min={MCP_TIMEOUT_BOUNDS.min}
max={MCP_TIMEOUT_BOUNDS.max}
step={1}
value={form.timeoutS}
onChange={(timeoutS) => set({ timeoutS })}
error={errorOf("timeout_s")}
/>
<NumberField
id={`${id}-startup-timeout`}
label="Longest start"
unit="seconds"
min={MCP_STARTUP_TIMEOUT_BOUNDS.min}
max={MCP_STARTUP_TIMEOUT_BOUNDS.max}
step={1}
value={form.startupTimeoutS}
onChange={(startupTimeoutS) => set({ startupTimeoutS })}
hint="A first start through uvx or npx downloads the server. One that does not start in time is tried again once its settings change."
error={errorOf("startup_timeout_s")}
/>
</>
);
}
/** One secret of a saved server: whether it is set, and a write-only control to set, replace or clear it. */
function SecretRow({ dotId, secret, onSaved }: { dotId: string; secret: McpSecretState; onSaved: () => void }) {
const [value, setValue] = useState("");
const save = useAction();
const id = `mcp-secret-${secret.server}-${secret.name}`;
async function apply(next: string | null) {
await save.run(async () => {
await api.setMcpSecret(dotId, secret.server, secret.name, next);
setValue("");
onSaved();
});
}
function submit(event: FormEvent) {
event.preventDefault();
if (value.trim() !== "") void apply(value.trim());
}
return (
<form onSubmit={submit} className="space-y-2">
<Field id={id} label={secret.name} hint={secret.set ? "Set. Stored encrypted and never shown again." : "Not set: the server does not start without it."}>
{(control) => <Input {...control} type="password" autoComplete="off" spellCheck={false} value={value} onChange={(event) => setValue(event.target.value)} />}
</Field>
<div className="flex flex-wrap gap-2">
<Button type="submit" size="sm" disabled={save.pending || value.trim() === ""}>
{secret.set ? "Replace" : "Set"}
</Button>
{secret.set ? (
<Button type="button" size="sm" variant="outline" disabled={save.pending} onClick={() => void apply(null)}>
Clear
</Button>
) : null}
</div>
{save.error ? <ErrorAlert error={save.error} title={`Could not save ${secret.name}`} /> : null}
</form>
);
}
/**
* The MCP servers the Dot may use: each one declared, where it is on the Dot's computer, its secrets; a server is added,
* changed or removed in the draft like any other setting, and its permission is in "Permissions and tools". The secrets
* are set apart from the config, write-only, for a server the saved config declares.
*/
export function McpServersPanel({ draft, saved, change, errorOf, dotId, computerState, table }: PanelProps & { dotId: string; computerState: string | null | undefined; table: ToolTable }) {
const [editing, setEditing] = useState<{ name: string; form: McpServerForm; isNew: boolean } | null>(null);
const [problem, setProblem] = useState<string | null>(null);
const secrets = useResource(() => api.mcpSecrets(dotId), `mcp-secrets:${dotId}`);
// A save that drops a server, or a name of its secrets, drops their values too.
useLiveRefresh(secrets.reload, ["dot.updated"]);
const statuses = new Map((computerIsUp(computerState) ? (table.data?.mcp_servers ?? []) : []).map((status) => [status.name, status]));
const names = mcpServerNames(draft);
function apply(event: FormEvent) {
event.preventDefault();
if (editing === null) return;
const name = editing.name.trim();
if (!isMcpServerName(name)) return setProblem("A name is 1 to 32 lowercase letters, digits and '-', starting with a letter or a digit.");
if (editing.isNew && name in draft.mcp_servers) return setProblem(`There is a server "${name}" already.`);
const parsed = serverOfForm(editing.form);
if (!parsed.ok) return setProblem(parsed.problem);
change(setMcpServer(draft, name, parsed.server));
setProblem(null);
setEditing(null);
}
return (
<Panel
id="mcp-servers"
title="MCP servers"
description="Programs and services whose tools the Dot may use, as Claude Code and Codex use them. You add one here, or the Dot adds one itself once it has installed the program the server needs, asking you first unless you allowed it. Only you change or remove one. Each server's permission is under Permissions and tools."
>
{names.length === 0 ? <p className="text-sm text-muted-foreground">No MCP server.</p> : null}
<ul className="divide-y rounded-lg border">
{names.map((name) => {
const server = draft.mcp_servers[name]!;
const savedServer = name in saved.mcp_servers;
const serverSecrets = (secrets.data?.secrets ?? []).filter((secret) => secret.server === name);
return (
<li key={name} className="space-y-3 p-4">
<div className="flex flex-wrap items-start justify-between gap-3">
<div className="min-w-0 flex-1 space-y-1">
<p className="font-medium">{name}</p>
<p className="truncate font-mono text-xs text-muted-foreground">{mcpServerSummary(server)}</p>
{savedServer ? <ServerState status={statuses.get(name)} /> : <p className="text-xs text-muted-foreground">Not saved yet.</p>}
{(["command", "url", "args", "env", "headers", "secrets", "timeout_s", "startup_timeout_s"] as const).map((field) => {
const error = errorOf(`mcp_servers.${name}.${field}`);
return error ? <p key={field} className="text-xs text-danger">{error}</p> : null;
})}
</div>
<div className="flex gap-2">
<Button type="button" size="sm" variant="outline" onClick={() => { setProblem(null); setEditing({ name, form: formOfServer(server), isNew: false }); }}>
Edit
</Button>
<Button type="button" size="sm" variant="outline" onClick={() => change(setMcpServer(draft, name, null))}>
Remove
</Button>
</div>
</div>
{serverSecrets.length > 0 ? (
<div className="space-y-3 border-t pt-3">
<p className="text-sm font-medium">Secrets</p>
{serverSecrets.map((secret) => (
<SecretRow key={secret.name} dotId={dotId} secret={secret} onSaved={secrets.reload} />
))}
</div>
) : server.secrets.length > 0 ? (
<p className="text-xs text-muted-foreground">Save the settings to set its secrets.</p>
) : null}
</li>
);
})}
</ul>
{secrets.error ? <ErrorAlert error={secrets.error} title="Could not read the MCP servers' secrets" /> : null}
{editing === null ? (
<Button type="button" variant="outline" onClick={() => { setProblem(null); setEditing({ name: "", form: EMPTY_MCP_SERVER_FORM, isNew: true }); }}>
Add an MCP server
</Button>
) : (
<form onSubmit={apply} aria-label={editing.isNew ? "New MCP server" : `MCP server ${editing.name}`} className="space-y-4 rounded-lg border p-4">
{editing.isNew ? (
<Field id="mcp-new-name" label="Name" hint="Its tools are named mcp_<name>_<tool>.">
{(control) => <Input {...control} value={editing.name} autoComplete="off" spellCheck={false} placeholder="time" onChange={(event) => setEditing({ ...editing, name: event.target.value })} />}
</Field>
) : (
<p className="font-medium">{editing.name}</p>
)}
<ServerFields id="mcp-edit" form={editing.form} onChange={(form) => setEditing({ ...editing, form })} errorOf={(field) => errorOf(`mcp_servers.${editing.name.trim()}.${field}`)} />
{problem ? <p className="text-sm text-danger">{problem}</p> : null}
<div className="flex gap-2">
<Button type="submit">{editing.isNew ? "Add" : "Apply"}</Button>
<Button type="button" variant="outline" onClick={() => setEditing(null)}>
Cancel
</Button>
</div>
</form>
)}
</Panel>
);
}
@@ -1,42 +1,17 @@
"use client";
import { computerIsUp, type PermissionDecision, type PermissionRisk } from "@invisible-dots/shared/browser";
import { api } from "../../lib/api";
import { RISK_LABEL } from "../../lib/approval-view";
import { computerIsUp, RISK_LABEL, type PermissionDecision, type PermissionRisk } from "@invisible-dots/shared/browser";
import { setPermission } from "../../lib/config-fields";
import { isComputerStopped } from "../../lib/computer";
import { DECISION_LABEL, DECISION_MEANING, DECISIONS, permissionGroups, type PermissionRow } from "../../lib/permission-table";
import { cn } from "../../lib/utils";
import { TONE_CLASS } from "../dot/tone";
import { ErrorAlert } from "../ErrorAlert";
import { useLiveRefresh } from "../events";
import { PresetPicker } from "../new-dot/PresetPicker";
import { useResource } from "../ui";
import { Panel, type PanelProps } from "./panel";
import type { ToolTable } from "./tool-table";
const RISK_TONE: Record<PermissionRisk, keyof typeof TONE_CLASS> = { low: "neutral", medium: "warn", high: "error" };
/**
* The Dot's tool table as its engine has it (`GET /tools`): every tool with the permission it uses and whether the
* model is offered it. Only a running computer has one to read, so a stopped one is not asked; the permissions
* can be edited either way. The offers change when a config is pushed and when the computer comes up.
*/
function useToolTable(dotId: string, computerState: string | null | undefined) {
const up = computerIsUp(computerState);
const table = useResource(async () => {
if (!up) return null;
try {
return await api.listTools(dotId);
} catch (error) {
// The computer went down between the state and the question: the same as not running.
if (isComputerStopped(error)) return null;
throw error;
}
}, `tools:${dotId}:${up}`);
useLiveRefresh(table.reload, ["dot.updated", "computer.state"]);
return table;
}
function DecisionControl({ row, onChange }: { row: PermissionRow; onChange: (decision: PermissionDecision) => void }) {
return (
<fieldset className="shrink-0">
@@ -83,10 +58,9 @@ function Tools({ row }: { row: PermissionRow }) {
* marked in words, and the tools that use it. The preset picker is the create page's, so the three presets mean the same
* in both. A permission set to what its default is leaves the config's `permissions` without an entry (`setPermission`).
*/
export function PermissionsPanel({ draft, saved, change, dotId, computerState }: PanelProps & { dotId: string; computerState: string | null | undefined }) {
const table = useToolTable(dotId, computerState);
export function PermissionsPanel({ draft, saved, change, computerState, table }: PanelProps & { computerState: string | null | undefined; table: ToolTable }) {
const up = computerIsUp(computerState);
const groups = permissionGroups(draft, saved, up ? (table.data ?? null) : null);
const groups = permissionGroups(draft, saved, up ? (table.data?.tools ?? null) : null);
// The computer is not running, or is said to run but its engine does not answer (it is just starting): there is no table to show.
const silent = computerState !== undefined && (!up || (table.data === null && !table.loading));
@@ -0,0 +1,30 @@
"use client";
import { computerIsUp } from "@invisible-dots/shared/browser";
import { api } from "../../lib/api";
import { isComputerStopped } from "../../lib/computer";
import { useLiveRefresh } from "../events";
import { useResource } from "../ui";
/**
* The Dot's tool table as its engine has it (`GET /tools`): every tool with the permission it uses and whether the
* model is offered it, and the MCP servers the config declares with where each is. Only a running computer has one to
* read, so a stopped one is not asked (null). The table changes when a config is pushed and when the computer comes up.
*/
export function useToolTable(dotId: string, computerState: string | null | undefined) {
const up = computerIsUp(computerState);
const table = useResource(async () => {
if (!up) return null;
try {
return await api.listTools(dotId);
} catch (error) {
// The computer went down between the state and the question: the same as not running.
if (isComputerStopped(error)) return null;
throw error;
}
}, `tools:${dotId}:${up}`);
useLiveRefresh(table.reload, ["dot.updated", "computer.state"]);
return table;
}
export type ToolTable = ReturnType<typeof useToolTable>;
+1 -1
View File
@@ -27,7 +27,7 @@ export function DotEventScope({ dotId, children }: { dotId: string; children: Re
return <ScopeContext.Provider value={dotId}>{children}</ScopeContext.Provider>;
}
/** One SSE connection for every Dot, kept for as long as the signed-in pages are open and shared by every component below it. */
/** One SSE connection for every Dot, kept for as long as the app's pages are open and shared by every component below it. */
export function EventStreamProvider({ children }: { children: ReactNode }) {
const listeners = useRef(new Set<Listener>());
const [status, setStatus] = useState<StreamStatus>("connecting");
+19 -12
View File
@@ -121,8 +121,8 @@ function Key({ children }: { children: string }) {
/**
* What needs the person (S6): the approvals that wait, oldest first, each answerable where it stands; Dots in an error
* state; tasks that failed in the last day; channels that need linking again. An approval that was answered here
* stays in its place as a receipt until the person leaves, so that nothing jumps away under the pointer. Keys: j and k
* move, a allows once, d denies.
* stays in its place as a receipt until the person leaves, so that nothing jumps away under the pointer. Keys, while
* the focus is in the list of the approvals: j and k move, a allows once, d denies.
*/
export function NeedsYou({ filter }: { filter: Filter }) {
const { dots, approvals, attention, failedTasks, relinks, dismissFailedTask } = useShell();
@@ -143,15 +143,16 @@ export function NeedsYou({ filter }: { filter: Filter }) {
const unlinked = otherThings ? relinks.items.filter((relink) => filter.dotId === null || relink.dot_id === filter.dotId) : [];
const waitingCount = asks.filter((ask) => !answers.settled.has(ask.id)).length;
const keys = useInboxKeys(asks, answers, true, (dotId) => names.get(dotId) ?? dotId);
const keys = useInboxKeys(asks, answers, (dotId) => names.get(dotId) ?? dotId);
const loading = (approvals.data === undefined && !approvals.error) || (dots.data === undefined && !dots.error);
// "Nothing needs you" is said only once everything has been read, failed tasks and channels included: the approvals do not wait for them.
const empty = !loading && !failedTasks.loading && !relinks.loading && asks.length === 0 && broken.length === 0 && failed.length === 0 && unlinked.length === 0;
return (
<div className="space-y-6">
<p className="text-xs text-muted-foreground">
Keys: <Key>j</Key> and <Key>k</Key> move between the cards, <Key>a</Key> allows the selected one once, <Key>d</Key> denies it.
<p id="inbox-keys" className="text-xs text-muted-foreground">
Keys, once the list of approvals has the focus (click it or Tab to it): <Key>j</Key> and <Key>k</Key> move between the cards, <Key>a</Key> allows the selected one
once, <Key>d</Key> denies it.
</p>
<p role="status" className="sr-only">
{keys.announcement}
@@ -167,7 +168,13 @@ export function NeedsYou({ filter }: { filter: Filter }) {
{!loading && asks.length > 0 ? (
<Section id="inbox-approvals" title="Waiting for your answer" count={waitingCount}>
<ul className="space-y-3">
<ul
aria-label="Approvals waiting for your answer"
aria-describedby="inbox-keys"
tabIndex={0}
className="space-y-3 rounded-lg focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-4 focus-visible:ring-offset-background focus-visible:outline-hidden"
{...keys.listProps}
>
{asks.map((ask) => (
<li key={ask.id}>
<ApprovalCard ask={ask} answers={answers} dotName={names.get(ask.dotId) ?? ask.dotId} selected={keys.selectedId === ask.id} />
@@ -213,22 +220,22 @@ export function NeedsYou({ filter }: { filter: Filter }) {
</Section>
) : null}
{failedTasks.unread > 0 && otherThings ? (
{failedTasks.unreadable && otherThings ? (
<Alert>
<AlertCircleIcon />
<AlertTitle>Some failed tasks may be missing</AlertTitle>
<AlertTitle>Failed tasks may be missing</AlertTitle>
<AlertDescription>
<p>The tasks of {failedTasks.unread === 1 ? "one Dot" : `${failedTasks.unread} Dots`} could not be read.</p>
<p>The tasks that failed today could not be read.</p>
</AlertDescription>
</Alert>
) : null}
{relinks.unread > 0 && otherThings ? (
{relinks.unreadable && otherThings ? (
<Alert>
<AlertCircleIcon />
<AlertTitle>Some channels may be missing</AlertTitle>
<AlertTitle>Channels may be missing</AlertTitle>
<AlertDescription>
<p>The channels of {relinks.unread === 1 ? "one Dot" : `${relinks.unread} Dots`} could not be read.</p>
<p>The channels could not be read.</p>
</AlertDescription>
</Alert>
) : null}
+42 -41
View File
@@ -1,10 +1,10 @@
"use client";
import { useEffect, useRef, useState } from "react";
import { useState, type FocusEvent, type KeyboardEvent } from "react";
import { askTitle, isDestructive, type ApprovalAsk } from "../../lib/approval-view";
import type { Answers } from "../approvals/use-answers";
/** Whether a key press is meant for a field (or a menu, a dialog) and not for the page. */
/** Whether a key press is meant for a field (or a menu, a dialog) and not for the list. */
function forTheFocus(target: EventTarget | null): boolean {
if (!(target instanceof HTMLElement)) return false;
if (target.isContentEditable || ["INPUT", "TEXTAREA", "SELECT"].includes(target.tagName)) return true;
@@ -21,54 +21,55 @@ export interface InboxKeys {
selectedId: string | null;
/** What to say to a person who cannot see the selection move or the card take the focus: which card is selected now, and that a destructive "allow" waits for a confirmation. */
announcement: string;
/** For the list of the cards: the keys act only while the focus is in it, and a card the focus enters is selected. */
listProps: { onKeyDown: (event: KeyboardEvent<HTMLElement>) => void; onFocus: (event: FocusEvent<HTMLElement>) => void };
}
/**
* The Inbox from the keyboard: `j` and `k` move between the cards, `a` allows the one selected once and `d` denies it.
* A card that is destructive (a command, a deleted identity, a file outside the workspace) is not allowed by a key
* press alone: `a` moves the focus to its "Allow once" button, and the press of that button is the confirmation. A key
* meant for a field, a menu or a dialog is left to it. A move says which card is selected now ("fares: Wants to run a
* command. 2 of 3."), since the selection is drawn and not focused: a screen reader would otherwise not hear it.
* The keys are the list's, not the page's: they act only while the focus is in the list of the cards (WCAG 2.1.4), so
* a "d" typed with the focus anywhere else denies nothing. A card the focus enters (a click on it, Tab to one of its
* buttons) is the one selected. A card that is destructive (a command, a deleted identity, a file outside the
* workspace) is not allowed by a key press alone: `a` moves the focus to its "Allow once" button, and the press of
* that button is the confirmation. A key meant for a field, a menu or a dialog is left to it. A move says which card
* is selected now ("fares: Wants to run a command. 2 of 3."), since the selection is drawn and not focused: a screen
* reader would otherwise not hear it.
*/
export function useInboxKeys(asks: readonly ApprovalAsk[], answers: Answers, enabled: boolean, nameOf: (dotId: string) => string): InboxKeys {
export function useInboxKeys(asks: readonly ApprovalAsk[], answers: Answers, nameOf: (dotId: string) => string): InboxKeys {
const [selected, setSelected] = useState<string | null>(null);
const [announcement, setAnnouncement] = useState("");
const selectedId = asks.some((ask) => ask.id === selected) ? selected : (asks[0]?.id ?? null);
const latest = useRef({ asks, answers, selectedId, nameOf });
useEffect(() => {
latest.current = { asks, answers, selectedId, nameOf };
});
useEffect(() => {
if (!enabled) return;
const onKey = (event: KeyboardEvent) => {
if (event.defaultPrevented || event.ctrlKey || event.metaKey || event.altKey || event.shiftKey) return;
if (!["j", "k", "a", "d"].includes(event.key) || forTheFocus(event.target) || document.querySelector("[role=dialog]") !== null) return;
const { asks: list, answers: current, selectedId: id, nameOf: nameFor } = latest.current;
const index = list.findIndex((ask) => ask.id === id);
if (index < 0) return;
const ask = list[index]!;
setAnnouncement("");
if (event.key === "j" || event.key === "k") {
const at = Math.min(list.length - 1, Math.max(0, index + (event.key === "j" ? 1 : -1)));
const next = list[at]!;
setSelected(next.id);
setAnnouncement(`${nameFor(next.dotId)}: ${askTitle(next)}. ${at + 1} of ${list.length}.`);
cardOf(next.id)?.scrollIntoView?.({ block: "nearest" });
return;
}
if (current.settled.has(ask.id) || current.sending.has(ask.id)) return;
const onFocus = (event: FocusEvent<HTMLElement>) => {
const id = (event.target as Element).closest("[data-approval-id]")?.getAttribute("data-approval-id");
if (id && asks.some((ask) => ask.id === id)) setSelected(id);
};
const onKeyDown = (event: KeyboardEvent<HTMLElement>) => {
if (event.defaultPrevented || event.ctrlKey || event.metaKey || event.altKey || event.shiftKey) return;
if (!["j", "k", "a", "d"].includes(event.key) || forTheFocus(event.target) || document.querySelector("[role=dialog]") !== null) return;
const index = asks.findIndex((ask) => ask.id === selectedId);
if (index < 0) return;
const ask = asks[index]!;
setAnnouncement("");
if (event.key === "j" || event.key === "k") {
event.preventDefault();
if (event.key === "d") void current.answer(ask, "reject");
else if (!isDestructive(ask)) void current.answer(ask, "approve");
else {
cardOf(ask.id)?.querySelector<HTMLElement>("[data-action=allow]")?.focus();
setAnnouncement("This can do real harm. Press Enter on Allow once to confirm.");
}
};
document.addEventListener("keydown", onKey);
return () => document.removeEventListener("keydown", onKey);
}, [enabled]);
const at = Math.min(asks.length - 1, Math.max(0, index + (event.key === "j" ? 1 : -1)));
const next = asks[at]!;
setSelected(next.id);
setAnnouncement(`${nameOf(next.dotId)}: ${askTitle(next)}. ${at + 1} of ${asks.length}.`);
cardOf(next.id)?.scrollIntoView?.({ block: "nearest" });
return;
}
if (answers.settled.has(ask.id) || answers.sending.has(ask.id)) return;
event.preventDefault();
if (event.key === "d") void answers.answer(ask, "reject");
else if (!isDestructive(ask)) void answers.answer(ask, "approve");
else {
cardOf(ask.id)?.querySelector<HTMLElement>("[data-action=allow]")?.focus();
setAnnouncement("This can do real harm. Press Enter on Allow once to confirm.");
}
};
return { selectedId, announcement };
return { selectedId, announcement, listProps: { onKeyDown, onFocus } };
}
@@ -25,7 +25,7 @@ export function PresetPicker({ permissions, onChange }: { permissions: Record<st
)}
>
<span className="flex items-center gap-2 font-medium">
<input type="radio" name="permission-preset" value={id} checked={chosen === id} onChange={() => onChange(presetPermissions(id))} className="accent-primary" />
<input type="radio" name="permission-preset" value={id} checked={chosen === id} onChange={() => onChange(presetPermissions(id, permissions))} className="accent-primary" />
{preset.label}
</span>
<span className="text-xs text-muted-foreground">{preset.description}</span>
@@ -6,7 +6,7 @@ import type { ThemePreference } from "../../lib/theme";
import { useThemePreference } from "../../lib/use-theme";
import { cn } from "../../lib/utils";
import { ErrorAlert } from "../ErrorAlert";
import { useShell } from "../shell/attention";
import { useHealth } from "../shell/attention";
import { CheckAgainButton, CheckList, CheckListSkeleton } from "../setup/CheckList";
import { KeyForm } from "../setup/KeyForm";
import { Section } from "../setup/Section";
@@ -84,7 +84,7 @@ function Appearance() {
}
function About() {
const { health } = useShell();
const health = useHealth();
return (
<Section id="about" title="About" busy={health.loading && health.data === undefined}>
{health.error !== null && health.data === undefined ? <ErrorAlert error={health.error} title="The control plane does not answer" /> : null}
+2 -2
View File
@@ -7,7 +7,7 @@ import { api } from "../../lib/api";
import { keySavedMessage } from "../../lib/host-settings";
import { ErrorAlert } from "../ErrorAlert";
import { Field } from "../new-dot/Field";
import { useShell } from "../shell/attention";
import { useHealth } from "../shell/attention";
import { Button } from "../ui/button";
import { Input } from "../ui/input";
import { useAction } from "../ui";
@@ -18,7 +18,7 @@ import { useAction } from "../ui";
* answer, and the answer to saving one says how many running Dots were given it at once.
*/
export function KeyForm() {
const { health } = useShell();
const health = useHealth();
const [value, setValue] = useState("");
const [problem, setProblem] = useState<string | null>(null);
const [saved, setSaved] = useState<string | null>(null);
@@ -2,7 +2,7 @@
import { useState } from "react";
import { isReady, notReadyCount } from "../../lib/preflight";
import { useShell } from "../shell/attention";
import { useHealth } from "../shell/attention";
import { CheckAgainButton, CheckList } from "./CheckList";
import { KeyForm } from "./KeyForm";
import { Section } from "./Section";
@@ -16,7 +16,7 @@ import { useHostChecks } from "./use-host-checks";
*/
export function SetupChecklist() {
const checks = useHostChecks();
const { health } = useShell();
const health = useHealth();
const [needed, setNeeded] = useState<{ key: boolean } | null>(null);
const items = checks.items;
const ready = items !== null && isReady(items);
@@ -4,7 +4,7 @@ import { useCallback, useEffect, useRef } from "react";
import { api } from "../../lib/api";
import { isReady, preflightItems, type HealthResult, type HostResult, type PreflightItem } from "../../lib/preflight";
import { useResource } from "../ui";
import { useShell } from "../shell/attention";
import { useHealth } from "../shell/attention";
export interface HostChecks {
/** Null until both answers are in (or have failed). */
@@ -27,7 +27,7 @@ function reason(error: unknown): string {
* still not ready, the checks run again by themselves.
*/
export function useHostChecks(): HostChecks {
const { health } = useShell();
const health = useHealth();
const doctor = useResource(() => api.doctor(), "host:doctor");
const healthResult: HealthResult | null = health.error !== null ? { error: reason(health.error) } : health.data ? { health: health.data } : null;
+4 -4
View File
@@ -3,19 +3,19 @@
import { ServerIcon } from "lucide-react";
import Link from "next/link";
import { cn } from "../../lib/utils";
import { useShell } from "./attention";
import { useHealth } from "./attention";
import { StatusIcon } from "./StatusIcon";
type ApiState = "checking" | "ok" | "down";
function useApiState(): ApiState {
const { health } = useShell();
const health = useHealth();
return health.error !== null ? "down" : health.data ? "ok" : "checking";
}
/** Whether the control plane answers through the proxy, with its version (the shell asks every 30 seconds): an icon, said in full on hover. */
export function ApiStatus() {
const { health } = useShell();
const health = useHealth();
const state = useApiState();
const label = state === "checking" ? "API: checking" : state === "ok" ? "API: ok" : "API: unreachable";
const version = state === "ok" && health.data ? `v${health.data.version}` : null;
@@ -30,7 +30,7 @@ export function ApiStatus() {
/** While the control plane has no OpenRouter key: a link to where it is entered, on a line of its own under the statuses. */
export function MissingKeyNotice({ onNavigate }: { onNavigate?: () => void }) {
const { health } = useShell();
const health = useHealth();
const state = useApiState();
if (state !== "ok" || !health.data || health.data.openrouter_configured) return null;
return (
+28 -18
View File
@@ -6,8 +6,8 @@ import type { HealthResponse } from "@invisible-dots/sdk";
import { api } from "../../lib/api";
import { attentionByDot, needsYouCount, NO_ATTENTION, withTitlePrefix, type DotAttention } from "../../lib/attention";
import { readDismissed, writeDismissed } from "../../lib/dismissed";
import { loadRelinks, type ChannelRelink, type RelinkRead } from "../../lib/channels";
import { loadFailedTasks, recentFailures, type FailedTasks } from "../../lib/failed-tasks";
import { loadRelinks, type ChannelRelink } from "../../lib/channels";
import { loadFailedTasks, recentFailures } from "../../lib/failed-tasks";
import { applyLiveEvent, dismissRestart, dotIdFromPath, liveOf, markRead, type LiveDot, type LiveDots } from "../../lib/dot-live";
import { faviconHref } from "../../lib/favicon";
import type { Approval, Dot, Task } from "../../lib/types";
@@ -26,15 +26,13 @@ const HEALTH_INTERVAL_MS = 30_000;
interface ShellData {
dots: Resource<Dot[]>;
approvals: Resource<Approval[]>;
/** The control plane's answer, asked every 30 seconds and again on `health.reload()` (after a key is saved). */
health: Resource<HealthResponse>;
attention: ReadonlyMap<string, DotAttention>;
/** What needs the person, across every Dot: waiting approvals, Dots in ERROR, failed tasks and channels to link again. The number on the rail's Inbox entry. */
needsYou: number;
/** Tasks that failed in the last 24 hours and are not dismissed, newest first; and how many Dots' tasks could not be read. */
failedTasks: { tasks: readonly Task[]; unread: number; loading: boolean };
/** Channels that need linking again, and how many Dots' channels could not be read. */
relinks: { items: readonly ChannelRelink[]; unread: number; loading: boolean };
/** Tasks that failed in the last 24 hours and are not dismissed, newest first; and whether they could not be read. */
failedTasks: { tasks: readonly Task[]; unreadable: boolean; loading: boolean };
/** Channels that need linking again; and whether they could not be read. */
relinks: { items: readonly ChannelRelink[]; unreadable: boolean; loading: boolean };
/** Take a failed task out of the Inbox (remembered in this browser). */
dismissFailedTask: (taskId: string) => void;
live: LiveDots;
@@ -42,9 +40,10 @@ interface ShellData {
}
const Context = createContext<ShellData | null>(null);
const HealthContext = createContext<Resource<HealthResponse> | null>(null);
/**
* What every signed-in page shares: the Dots, the approvals that wait, the tasks that failed lately, the channels that
* What every page of the app shares: the Dots, the approvals that wait, the tasks that failed lately, the channels that
* need linking again, what the live stream says about each Dot, and whether the control plane answers. The rail, the
* Dot header, the document title, the favicon and the setup pages
* all read it, so they agree and the control plane is asked once, not once per component.
@@ -53,9 +52,8 @@ export function AttentionProvider({ children }: { children: ReactNode }) {
const dots = useResource(() => api.listDots(), "shell:dots");
const approvals = useResource(() => api.listApprovals("pending"), "shell:approvals");
const health = useResource(() => api.health(), "shell:health");
const dotIds = (dots.data ?? []).map((dot) => dot.id).join(",");
const failed = useResource<FailedTasks>(() => loadFailedTasks(api, dots.data ?? []), `shell:failed:${dotIds}`);
const channels = useResource<RelinkRead>(() => loadRelinks(api, dots.data ?? []), `shell:channels:${dotIds}`);
const failed = useResource<Task[]>(() => loadFailedTasks(api), "shell:failed");
const channels = useResource<ChannelRelink[]>(() => loadRelinks(api), "shell:channels");
const reloadHealth = health.reload;
useEffect(() => {
const timer = setInterval(reloadHealth, HEALTH_INTERVAL_MS);
@@ -105,11 +103,11 @@ export function AttentionProvider({ children }: { children: ReactNode }) {
}, []);
// A failure leaves the Inbox a day after it happened, without anyone reloading anything.
const now = Math.floor(useNow(60_000) / 60_000);
const recent = useMemo(() => recentFailures(failed.data?.tasks ?? [], dismissed, now * 60_000), [failed.data, dismissed, now]);
const failedTasks = useMemo(() => ({ tasks: recent, unread: failed.data?.unread ?? 0, loading: failed.data === undefined && failed.error === null }), [recent, failed.data, failed.error]);
const recent = useMemo(() => recentFailures(failed.data ?? [], dismissed, now * 60_000), [failed.data, dismissed, now]);
const failedTasks = useMemo(() => ({ tasks: recent, unreadable: failed.error !== null, loading: failed.data === undefined && failed.error === null }), [recent, failed.data, failed.error]);
const relinks = useMemo(
() => ({ items: channels.data?.relinks ?? [], unread: channels.data?.unread ?? 0, loading: channels.data === undefined && channels.error === null }),
() => ({ items: channels.data ?? [], unreadable: channels.error !== null, loading: channels.data === undefined && channels.error === null }),
[channels.data, channels.error],
);
@@ -119,10 +117,22 @@ export function AttentionProvider({ children }: { children: ReactNode }) {
const dismiss = useCallback((dotId: string) => setLive((current) => dismissRestart(current, dotId)), []);
const value = useMemo<ShellData>(
() => ({ dots, approvals, health, attention, needsYou, failedTasks, relinks, dismissFailedTask, live, dismissRestart: dismiss }),
[dots, approvals, health, attention, needsYou, failedTasks, relinks, dismissFailedTask, live, dismiss],
() => ({ dots, approvals, attention, needsYou, failedTasks, relinks, dismissFailedTask, live, dismissRestart: dismiss }),
[dots, approvals, attention, needsYou, failedTasks, relinks, dismissFailedTask, live, dismiss],
);
return <Context.Provider value={value}>{children}</Context.Provider>;
// The health is a context of its own: its poll every 30 seconds re-renders what reads it, not every user of the shell.
return (
<HealthContext.Provider value={health}>
<Context.Provider value={value}>{children}</Context.Provider>
</HealthContext.Provider>
);
}
/** The control plane's answer, asked every 30 seconds and again on `reload()` (after a key is saved). */
export function useHealth(): Resource<HealthResponse> {
const value = useContext(HealthContext);
if (!value) throw new Error("useHealth must be used inside AttentionProvider");
return value;
}
export function useShell(): ShellData {
@@ -4,7 +4,7 @@ import { WifiOffIcon } from "lucide-react";
import { offlineOf } from "../../lib/offline";
import { useStreamStatus } from "../events";
import { Alert, AlertDescription, AlertTitle } from "../ui/alert";
import { useShell } from "./attention";
import { useHealth } from "./attention";
/**
* Says so, at the top of every page, when the control plane does not answer: the live stream is down or the health
@@ -12,7 +12,7 @@ import { useShell } from "./attention";
*/
export function OfflineBanner() {
const stream = useStreamStatus();
const { health } = useShell();
const health = useHealth();
const offline = offlineOf({ stream: stream.status, streamDetail: stream.detail, apiError: health.error });
if (offline === null) return null;
return (
+9 -15
View File
@@ -1,8 +1,8 @@
"use client";
import { BanIcon, CheckCircle2Icon, CircleDotIcon, FlagIcon, HandIcon, MessageSquareIcon, PlayIcon, TerminalIcon, XCircleIcon, type LucideIcon } from "lucide-react";
import { toolLabel } from "../../lib/events/tool-labels";
import { formatDate, formatDuration } from "../../lib/format";
import { formatDate, formatMillis } from "../../lib/format";
import { APPROVAL_WORD, TOOL_OUTCOME } from "../../lib/steps";
import type { StoryStep } from "../../lib/task-events";
import { priorityLabel } from "../../lib/task-view";
import { relativeTime } from "../../lib/time";
@@ -22,14 +22,8 @@ const ICON: Record<StoryStep["kind"], LucideIcon> = {
const ICON_TONE: Partial<Record<StoryStep["kind"], string>> = { completed: "text-ok", failed: "text-danger", approval: "text-warn" };
/** How a tool call ended, in a word: it ran, it failed, it was refused, or it was cut off. */
function toolOutcome(step: Extract<StoryStep, { kind: "tool" }>): { word: string; bad: boolean } {
if (step.interrupted) return { word: "interrupted", bad: true };
if (step.decision === "deny") return { word: "denied", bad: true };
return step.ok ? { word: "ok", bad: false } : { word: "failed", bad: true };
}
const APPROVAL_WORD = { waiting: "Waiting for an answer", approved: "Allowed", rejected: "Denied" } as const;
/** The color of how a call ended: only what is not an ordinary success has one. */
const OUTCOME_CLASS = { neutral: "text-muted-foreground", info: "text-muted-foreground", ok: "text-muted-foreground", warn: "text-warn", error: "text-danger" } as const;
function Body({ step }: { step: StoryStep }) {
switch (step.kind) {
@@ -44,16 +38,16 @@ function Body({ step }: { step: StoryStep }) {
case "progress":
return <span className="whitespace-pre-wrap">{step.text}</span>;
case "tool": {
const outcome = toolOutcome(step);
const outcome = TOOL_OUTCOME[step.state];
return (
<>
<span title={step.tool} className="font-medium">
{toolLabel(step.tool, step.tty).label}
{step.label}
</span>
{step.target ? <span className="break-all text-muted-foreground"> {step.target}</span> : null}{" "}
<span className={cn("text-xs", outcome.bad ? "text-danger" : "text-muted-foreground")}>
<span className={cn("text-xs", OUTCOME_CLASS[outcome.tone])}>
{outcome.word}
{step.durationMs > 0 ? `, ${formatDuration(step.durationMs / 1000)}` : ""}
{step.durationMs > 0 ? `, ${formatMillis(step.durationMs)}` : ""}
</span>
</>
);
@@ -65,7 +59,7 @@ function Body({ step }: { step: StoryStep }) {
{APPROVAL_WORD[step.outcome]}
{step.outcome === "approved" && step.always ? " for good" : ""}:
</span>{" "}
<span title={step.tool}>{toolLabel(step.tool).label.toLowerCase()}</span>
<span title={step.tool}>{step.label.toLowerCase()}</span>
{step.reason ? <span className="text-muted-foreground"> {step.reason}</span> : null}
{step.note ? <span className="block text-xs text-muted-foreground">Note: {step.note}</span> : null}
</>
+1 -1
View File
@@ -1,5 +1,5 @@
import { ClockIcon, FileTextIcon, FilePenIcon, FingerprintIcon, GlobeIcon, MonitorIcon, TerminalIcon, WrenchIcon, type LucideIcon } from "lucide-react";
import type { ToolFamily } from "../lib/events/tool-labels";
import type { ToolFamily } from "@invisible-dots/shared/browser";
/** One icon per family of the Dot's tools: the chat's steps and the approval cards draw the same one for the same kind of call. */
export const FAMILY_ICON: Record<ToolFamily, LucideIcon> = {
+4 -3
View File
@@ -1,6 +1,6 @@
"use client";
import { useCallback, useEffect, useRef, useState } from "react";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
export interface Resource<T> {
data: T | undefined;
@@ -12,7 +12,8 @@ export interface Resource<T> {
/**
* Load data on mount and whenever `key` changes. Answers that arrive after a
* newer load started are dropped, so a slow response never overwrites a
* fresh one.
* fresh one. The object returned is the same until one of its fields changes,
* so a memo or a context value built from it changes only with it.
*/
export function useResource<T>(load: () => Promise<T>, key: string): Resource<T> {
const [data, setData] = useState<T | undefined>(undefined);
@@ -50,7 +51,7 @@ export function useResource<T>(load: () => Promise<T>, key: string): Resource<T>
reload();
}, [key, reload]);
return { data, error, loading, reload };
return useMemo(() => ({ data, error, loading, reload }), [data, error, loading, reload]);
}
/** Run an action, tracking its pending state and error. */
-1
View File
@@ -36,7 +36,6 @@ export function isWorking(agent: AgentState | null): agent is "THINKING" | "PLAN
*/
export function statePill(status: string, agent: AgentState | null): StatePill {
if (status === "ERROR") return { label: "Error", tone: "error", working: false };
if (status === "DISABLED") return { label: "Disabled", tone: "neutral", working: false };
if (status === "CREATING") return { label: "Preparing the computer", tone: "info", working: true };
if (agent === "WAITING_APPROVAL" || status === "WAITING_APPROVAL") return { label: "Waiting for you", tone: "warn", working: false };
if (isWorking(agent)) return { label: agentStateLabel(agent), tone: "info", working: true };
+39 -7
View File
@@ -5,9 +5,17 @@
* The arguments come from the engine's table (`tool_arguments` in nanobot/dots/permissions.py): all of a call's
* arguments, bar a secret the table redacts. So what is shown here is what the call will do, not a summary of it.
*/
import { GUEST_PATHS, isPermission, PERMISSION_INFO, type Permission, type PermissionInfo, type PermissionRisk, type ToolInfo } from "@invisible-dots/shared/browser";
import {
GUEST_PATHS,
isPermissionName,
mcpServerOf,
permissionInfo as sharedPermissionInfo,
toolLabel,
type PermissionInfo,
type PermissionName,
type ToolInfo,
} from "@invisible-dots/shared/browser";
import { additionDiff, replacementDiff, type DiffLine } from "./diff";
import { toolLabel } from "./events/tool-labels";
import { formatDate } from "./format";
import type { Approval } from "./types";
@@ -79,20 +87,21 @@ function changedPaths(ask: ApprovalAsk): string[] {
/**
* Whether answering "allow" here is the kind of answer to stop and think about: a command, the deletion of a browser
* identity (its logins with it), or a change to a file outside the workspace.
* identity (its logins with it), a change to a file outside the workspace, or a tool of an MCP server, which does
* whatever that server does.
*/
export function isDestructive(ask: ApprovalAsk): boolean {
if (ask.permission === "computer.exec" || ask.permission === "browser.identity.delete") return true;
if (mcpServerOf(ask.permission) !== null) return true;
return ask.permission === "files.write" && changedPaths(ask).some(outsideWorkspace);
}
/** The words for a permission: its name and what it lets the Dot do; null for one a Dot's config can no longer name. */
export function permissionInfo(permission: string): (PermissionInfo & { permission: Permission }) | null {
return isPermission(permission) ? { ...PERMISSION_INFO[permission], permission } : null;
export function permissionInfo(permission: string): (PermissionInfo & { permission: PermissionName }) | null {
const info = sharedPermissionInfo(permission);
return info !== null && isPermissionName(permission) ? { ...info, permission } : null;
}
export const RISK_LABEL: Record<PermissionRisk, string> = { low: "Low risk", medium: "Medium risk", high: "High risk" };
/** The question: what the Dot wants to do, as the engine's table words it. */
export function askTitle(ask: ApprovalAsk): string {
return `Wants to ${toolLabel(ask.tool).ask}`;
@@ -228,12 +237,35 @@ export function approvalBody(ask: ApprovalAsk): ApprovalBody {
if (files.length > 0) return { kind: "patch", files, dryRun: args.dry_run === true };
break;
}
case "add_mcp_server":
return { kind: "facts", facts: mcpServerFacts(args) };
default:
break;
}
return { kind: "facts", facts: argumentFacts(args) };
}
/**
* The server a Dot asks to add, as the person must read it before it enters the config: what it runs (or where it
* is), with what, which secrets it will ask them for, and why. A value written in env or headers is shown as it is,
* since it would sit in the config for anyone to read; the secrets are names, never values.
*/
function mcpServerFacts(args: Record<string, unknown>): Fact[] {
const pairs = (value: unknown, separator: string) =>
typeof value === "object" && value !== null ? Object.entries(value as Record<string, unknown>).map(([k, v]) => `${k}${separator}${text(v)}`).join("\n") : "";
const list = (value: unknown) => (Array.isArray(value) ? value.map(text).filter(Boolean) : []);
const runs = [text(args.command), ...list(args.args)].filter(Boolean).join(" ");
const facts: Fact[] = [
{ label: "Name", value: text(args.name) },
runs !== "" ? { label: "Runs, as the Dot's user", value: runs } : { label: "URL", value: text(args.url) },
{ label: "Environment", value: pairs(args.env, "=") },
{ label: "Headers", value: pairs(args.headers, ": ") },
{ label: "Secrets you will set", value: list(args.secrets).join(", ") },
{ label: "Why", value: text(args.reason) },
];
return facts.filter((fact) => fact.value !== "");
}
/**
* The raw arguments, as JSON, cut when they are enormous (a file's whole content can be); `cut` says it was. The engine
* already shows an identity's proxy as `***`; it is hidden here too, so that no path to this page can show one (a
+3 -25
View File
@@ -6,7 +6,6 @@
import type { InvisibleDotsClient } from "@invisible-dots/sdk";
import { APPROVAL_PROMPT_TEXT_MAX, type ChannelKind, type ChannelPeerRecord, type ChannelRecord, type ChannelSettings, type ChannelStatus, type MessageOrigin } from "@invisible-dots/shared/browser";
import type { Tone } from "./tone";
import type { Dot } from "./types";
/** How a channel is named to the person. A `Record` over the contract's kinds, so a kind added there is a compile error here until it has a name. */
export const CHANNEL_LABELS: Record<ChannelKind, string> = {
@@ -36,13 +35,6 @@ export function channelState(record: Pick<ChannelRecord, "enabled" | "status">):
return record.enabled ? STATUS_VIEW[record.status] : { label: "Paused", tone: "neutral" };
}
/** The account of a channel as a person writes it: a bot's `@name`, a phone number with its plus; null until the channel reports one. */
export function accountLabel(kind: ChannelKind, account: string | null): string | null {
if (account === null || account === "") return null;
if (kind === "telegram") return `@${account}`;
return account.startsWith("+") ? account : `+${account}`;
}
/** Where a channel's account can be opened in a browser: a Telegram bot's page; WhatsApp has no page for a number. */
export function accountHref(kind: ChannelKind, account: string | null): string | null {
return kind === "telegram" && account !== null && /^[A-Za-z0-9_]+$/.test(account) ? `https://t.me/${account}` : null;
@@ -136,21 +128,7 @@ export function needsRelinking(record: Pick<ChannelRecord, "enabled" | "status">
return record.enabled && record.status === "needs_relink";
}
export interface RelinkRead {
/** The channels that need linking again, Dots in the order given, a Dot's channels in the order the host lists them. */
relinks: ChannelRelink[];
/** Dots whose channels could not be read. */
unread: number;
}
/**
* The channels of every Dot that need the person. The host has no route for "every Dot's channels", so each Dot's list
* is read; a Dot whose list cannot be read is counted, not hidden.
*/
export async function loadRelinks(client: Pick<InvisibleDotsClient, "channels">, dots: readonly Pick<Dot, "id">[]): Promise<RelinkRead> {
const answers = await Promise.allSettled(dots.map((dot) => client.channels(dot.id)));
const relinks = answers.flatMap((answer, index) =>
answer.status === "fulfilled" ? answer.value.filter(needsRelinking).map((record): ChannelRelink => ({ dot_id: dots[index]!.id, kind: record.kind, detail: record.status_detail })) : [],
);
return { relinks, unread: answers.filter((answer) => answer.status === "rejected").length };
/** The channels of every Dot that need the person, in the order the host lists them: one read of every Dot's (`GET /api/channels`). */
export async function loadRelinks(client: Pick<InvisibleDotsClient, "everyChannel">): Promise<ChannelRelink[]> {
return (await client.everyChannel()).filter(needsRelinking).map((record): ChannelRelink => ({ dot_id: record.dot_id, kind: record.kind, detail: record.status_detail }));
}
+7 -108
View File
@@ -6,47 +6,10 @@
* Only the chat's own turns are drawn here: a tool call or an approval that names a task belongs to that task's
* story (the Tasks page), not to the conversation.
*/
import { toolLabel, type ToolFamily } from "./events/tool-labels";
import type { ApprovalOutcome } from "./task-events";
import { mergeEvents } from "./event-log";
import { mergeEvents } from "@invisible-dots/sdk";
import { stepsOf, type ApprovalStep, type ToolStep } from "./steps";
import type { ChatMessage, StoredEvent } from "./types";
/**
* How a tool call ended. The engine reports a call when it has ended, so a call that is still running has no event
* yet (the chat's working row says the Dot is busy); a call that waits for the person is the approval's step.
*/
export type ToolState = "ok" | "error" | "denied" | "interrupted";
export interface ToolStep {
kind: "tool";
id: number;
at: string;
tool: string;
label: string;
family: ToolFamily;
/** What the call acted on, as the engine reported it (one line); "" when it named nothing. */
target: string;
state: ToolState;
durationMs: number;
}
export interface ApprovalStep {
kind: "approval";
id: number;
at: string;
approvalId: string;
tool: string;
label: string;
family: ToolFamily;
reason: string;
/** What the call needs the person's permission for, and with which arguments: what the approval card shows. */
permission: string;
arguments: Record<string, unknown>;
outcome: ApprovalOutcome;
/** The answer also allowed the permission for good. */
always: boolean;
}
export type ActivityItem = ToolStep | ApprovalStep;
export type ThreadItem =
@@ -64,79 +27,15 @@ export function isChatActivityEvent(event: Pick<StoredEvent, "type">): boolean {
/** Runs of tool steps longer than this fold into one line the person can open. */
export const CLUSTER_AFTER = 3;
function text(value: unknown): string {
return typeof value === "string" ? value : "";
}
function hasTask(event: Pick<StoredEvent, "data">): boolean {
return text(event.data?.task_id) !== "";
}
export function toolStateOf(data: Record<string, unknown>): ToolState {
if (data.interrupted === true) return "interrupted";
if (data.decision === "deny") return "denied";
return data.ok === true ? "ok" : "error";
/** Whether a call or an approval is the chat's: one that names a task is that task's. */
function isTheChats(event: Pick<StoredEvent, "data">): boolean {
const taskId = event.data?.task_id;
return typeof taskId !== "string" || taskId === "";
}
/** The steps of the chat's turns in the order they happened, from the events `isChatActivityEvent` accepts. */
export function activityOf(events: readonly StoredEvent[]): ActivityItem[] {
const items: ActivityItem[] = [];
const approvals = new Map<string, ApprovalStep>();
for (const event of [...events].sort((a, b) => a.id - b.id)) {
const d = event.data ?? {};
const base = { id: event.id, at: event.created_at };
switch (event.type) {
case "tool.called": {
if (hasTask(event)) break;
const tool = text(d.tool);
const { label, family } = toolLabel(tool, d.tty === true);
items.push({
...base,
kind: "tool",
tool,
label,
family,
target: text(d.target),
state: toolStateOf(d),
durationMs: typeof d.duration_ms === "number" ? d.duration_ms : 0,
});
break;
}
case "approval.requested": {
if (hasTask(event)) break;
const tool = text(d.tool);
const { label, family } = toolLabel(tool);
const step: ApprovalStep = {
...base,
kind: "approval",
approvalId: text(d.approval_id),
tool,
label,
family,
reason: text(d.reason),
permission: text(d.permission),
arguments: typeof d.arguments === "object" && d.arguments !== null && !Array.isArray(d.arguments) ? (d.arguments as Record<string, unknown>) : {},
outcome: "waiting",
always: false,
};
approvals.set(step.approvalId, step);
items.push(step);
break;
}
case "approval.resolved": {
// It names only the approval: one that was not the chat's is not in the map.
const step = approvals.get(text(d.approval_id));
if (step) {
step.outcome = d.decision === "approve" ? "approved" : "rejected";
step.always = d.always === true;
}
break;
}
default:
break;
}
}
return items;
return stepsOf([...events].sort((a, b) => a.id - b.id), isTheChats);
}
/** The thread: messages and activity merged by event id; an assistant message starts a group after anything but another assistant message. */
+2 -3
View File
@@ -10,13 +10,12 @@ export interface ComposerState {
/**
* A message is accepted whenever the Dot exists: the control plane stores it and delivers it once the computer is
* up (`delivery: "queued"`), waking it if it sleeps. So the person is stopped only for a Dot that will not answer
* (disabled) or no longer exists (being deleted), and told in a sentence when the answer will take a while.
* up (`delivery: "queued"`), waking it if it sleeps. So the person is stopped only for a Dot that is being deleted,
* and told in a sentence when the answer will take a while.
*/
export function composerState(dot: Pick<Dot, "status" | "computer_state"> | undefined): ComposerState {
if (dot === undefined) return { blocked: null, hint: null };
if (dot.computer_state === "DELETING") return { blocked: "This Dot is being deleted.", hint: null };
if (dot.status === "DISABLED") return { blocked: "This Dot is disabled, so it does not answer.", hint: null };
switch (dot.computer_state) {
case "PROVISIONING":
case "STARTING":
+2 -5
View File
@@ -1,5 +1,5 @@
/** What the Computer page shows: allocated resources from the config, live usage from the VM when it runs. */
import { COMPUTER_STOPPED, computerIsUp } from "@invisible-dots/shared/browser";
import { COMPUTER_STOPPED } from "@invisible-dots/shared/browser";
import { ApiError } from "./api";
import { relativeTime } from "./time";
import type { Computer, DotConfig, SystemAnswer, VmState } from "./types";
@@ -8,7 +8,6 @@ const COMPUTER_STATE_LABELS: Record<VmState, string> = {
PROVISIONING: "Provisioning",
STARTING: "Starting",
RUNNING: "Running",
IDLE: "Idle",
STOPPING: "Stopping",
STOPPED: "Stopped",
ERROR: "Error",
@@ -130,9 +129,7 @@ export function allowedActions(state: string): { start: boolean; stop: boolean;
case "ERROR":
return { start: true, stop: state === "ERROR", reboot: false };
case "RUNNING":
case "IDLE":
// The host reboots only a computer whose guest answers.
return { start: false, stop: true, reboot: computerIsUp(state) };
return { start: false, stop: true, reboot: true };
case "PROVISIONING":
case "STARTING":
case "STOPPING":
+80 -11
View File
@@ -6,7 +6,20 @@
* one (`rebase`). A permission is compared by what it resolves to, so an explicit entry that says what the default
* says is not a change.
*/
import { PERMISSION_INFO, PERMISSIONS, resolvePermission, type DotConfig, type Permission, type PermissionDecision } from "@invisible-dots/shared/browser";
import {
defaultPermission,
isMcpStdioServer,
mcpPermission,
mcpServerNames,
mcpServerOf,
PERMISSION_INFO,
PERMISSIONS,
resolvePermission,
type DotConfig,
type McpServerConfig,
type PermissionDecision,
type PermissionName,
} from "@invisible-dots/shared/browser";
/** What a field holds: a text, a number or a switch. */
export type ConfigValue = string | number | boolean;
@@ -34,15 +47,33 @@ function shownText(value: ConfigValue): string {
const shownPlain = (value: ConfigValue) => String(value);
/** The default decision of a permission: what it resolves to when the config says nothing about it. */
export function defaultDecision(permission: Permission): PermissionDecision {
return resolvePermission({ permissions: {} }, permission);
/** The default decision of a permission: what it resolves to when the config says nothing about it (an MCP server's asks). */
export function defaultDecision(permission: PermissionName): PermissionDecision {
return defaultPermission(permission);
}
/** The config with one permission set. A decision the default already makes is not written, so a config stays as short as what it changes. */
export function setPermission(config: DotConfig, permission: Permission, decision: PermissionDecision): DotConfig {
/**
* The config with one permission set. A decision the default already makes is not written, so a config stays as short
* as what it changes; nor is the permission of an MCP server the config does not declare, which it could not hold.
*/
export function setPermission(config: DotConfig, permission: PermissionName, decision: PermissionDecision): DotConfig {
const { [permission]: _replaced, ...rest } = config.permissions;
return { ...config, permissions: decision === defaultDecision(permission) ? rest : { ...rest, [permission]: decision } };
const server = mcpServerOf(permission);
const holds = server === null || server in config.mcp_servers;
return { ...config, permissions: !holds || decision === defaultDecision(permission) ? rest : { ...rest, [permission]: decision } };
}
/** The config with one MCP server declared as `server`, or no longer declared (null): its permission goes with it. */
export function setMcpServer(config: DotConfig, name: string, server: McpServerConfig | null): DotConfig {
const { [name]: _replaced, ...others } = config.mcp_servers;
if (server !== null) return { ...config, mcp_servers: { ...others, [name]: server } };
const { [mcpPermission(name)]: _dropped, ...permissions } = config.permissions;
return { ...config, mcp_servers: others, permissions };
}
/** A server as a change shows it: what it runs, or where it is. */
export function mcpServerSummary(server: McpServerConfig): string {
return isMcpStdioServer(server) ? [server.command, ...server.args].join(" ") : server.url;
}
const SCALARS = {
@@ -106,8 +137,46 @@ const PERMISSION_FIELDS: readonly ConfigField[] = PERMISSIONS.map((permission) =
show: shownPlain,
}));
/** Every field, in the order the page lists them: the settings first, then the permissions. */
const FIELDS: readonly ConfigField[] = [...Object.entries(SCALARS).map(([key, f]): ConfigField => ({ key, ...f })), ...PERMISSION_FIELDS];
/** A value as JSON with the keys of every object in order, so two configs that say the same compare equal. */
function canonicalJson(value: unknown): string {
return JSON.stringify(value, (_key, item: unknown) =>
item !== null && typeof item === "object" && !Array.isArray(item)
? Object.fromEntries(Object.entries(item as Record<string, unknown>).sort(([a], [b]) => (a < b ? -1 : a > b ? 1 : 0)))
: item,
);
}
/** The fields of one MCP server: its entry (compared whole), then its permission. A server absent from a config is "". */
function mcpServerFields(name: string): ConfigField[] {
const permission = mcpPermission(name);
return [
{
key: `mcp_servers.${name}`,
label: `MCP server ${name}`,
applies: "turn",
get: (c) => (name in c.mcp_servers ? canonicalJson(c.mcp_servers[name]) : ""),
set: (c, v) => setMcpServer(c, name, v === "" ? null : (JSON.parse(String(v)) as McpServerConfig)),
show: (v) => (v === "" ? "(not declared)" : mcpServerSummary(JSON.parse(String(v)) as McpServerConfig)),
},
{
key: `permissions.${permission}`,
label: `Permission of MCP server ${name}`,
applies: "turn",
get: (c) => resolvePermission(c, permission),
set: (c, v) => setPermission(c, permission, v as PermissionDecision),
show: shownPlain,
},
];
}
/**
* Every field of the configs given, in the order the page lists them: the settings, the permissions, then each MCP
* server any of them declares, by name.
*/
function fieldsOf(...configs: DotConfig[]): ConfigField[] {
const servers = [...new Set(configs.flatMap((c) => mcpServerNames(c)))].sort();
return [...Object.entries(SCALARS).map(([key, f]): ConfigField => ({ key, ...f })), ...PERMISSION_FIELDS, ...servers.flatMap(mcpServerFields)];
}
/** The config with one setting set (a permission is set with `setPermission`). */
export function setField(config: DotConfig, key: FieldKey, value: ConfigValue): DotConfig {
@@ -125,7 +194,7 @@ export interface ConfigChange {
/** What differs between two configs, field by field. Empty when they amount to the same Dot. */
export function configChanges(before: DotConfig, after: DotConfig): ConfigChange[] {
return FIELDS.flatMap((f) => {
return fieldsOf(before, after).flatMap((f) => {
const was = f.get(before);
const now = f.get(after);
return was === now ? [] : [{ key: f.key, label: f.label, before: f.show(was), after: f.show(now), applies: f.applies }];
@@ -139,7 +208,7 @@ export function configChanges(before: DotConfig, after: DotConfig): ConfigChange
* `latest`.
*/
export function rebase(base: DotConfig, mine: DotConfig, latest: DotConfig): DotConfig {
return FIELDS.reduce((merged, f) => {
return fieldsOf(base, mine, latest).reduce((merged, f) => {
const value = f.get(mine);
return value === f.get(base) ? merged : f.set(merged, value);
}, latest);
+8 -7
View File
@@ -4,6 +4,7 @@
* so the Dot's name is asked of the control plane there, with the web server's own credential, and only for a request
* that reaches this server (it listens where the Dots' VMs cannot reach it, section 9.7). Anything that goes wrong leaves the title the tab alone.
*/
import { InvisibleDotsClient } from "@invisible-dots/sdk";
import type { Metadata } from "next";
import { apiBaseUrl, loadApiToken } from "./proxy";
@@ -12,14 +13,14 @@ const NAME_TIMEOUT_MS = 2000;
export async function dotPageTitle(dotIdOrName: string, tab: string): Promise<Metadata> {
try {
const token = await loadApiToken();
const answer = await fetch(`${apiBaseUrl()}/api/dots/${encodeURIComponent(dotIdOrName)}`, {
headers: { authorization: `Bearer ${token}`, accept: "application/json" },
cache: "no-store",
signal: AbortSignal.timeout(NAME_TIMEOUT_MS),
const client = new InvisibleDotsClient({
baseUrl: apiBaseUrl(),
token: await loadApiToken(),
timeoutMs: NAME_TIMEOUT_MS,
// A Dot can be renamed: an answer Next kept would show the old name.
fetch: (url, init) => fetch(url, { ...init, cache: "no-store" }),
});
if (!answer.ok) return { title: tab };
const { name } = (await answer.json()) as { name?: unknown };
const { name } = await client.getDot(dotIdOrName);
return { title: typeof name === "string" && name !== "" ? `${tab} - ${name}` : tab };
} catch {
return { title: tab };
+14 -11
View File
@@ -9,14 +9,16 @@
*/
import {
parseMessageOrigin,
toolLabel,
type HostEventDataMap,
type InboundEventDataMap,
type OutboundEventDataMap,
type StoredEvent,
} from "@invisible-dots/shared/browser";
import { CHANNEL_LABELS, viaChannel } from "../channels";
import { formatMillis } from "../format";
import { APPROVAL_WORD, TOOL_OUTCOME, toolStateOf } from "../steps";
import type { Tone } from "../tone";
import { toolLabel } from "./tool-labels";
/** What the Activity page filters by: one family per kind of thing that happens to a Dot. */
export const EVENT_FAMILIES = ["chat", "tasks", "tools", "approvals", "memory", "browser", "computer", "channels", "dot"] as const;
@@ -63,6 +65,8 @@ const FAMILY_OF: Record<KnownType, EventFamily> = {
"agent.state": "computer",
"automation.next_run": "computer",
"guest.event.refused": "computer",
"mcp.server.added": "dot",
"mcp.server.refused": "dot",
"channel.status": "channels",
"channel.peer.paired": "channels",
"channel.changed": "channels",
@@ -135,13 +139,6 @@ interface Draft {
tool?: string;
}
/** How a tool call ended, in the words of the row. A denied call never ran; an interrupted one has no known outcome. */
function outcome(data: Partial<OutboundEventDataMap["tool.called"]>): { word: string; tone: Tone } {
if (data.interrupted === true) return { word: "interrupted", tone: "warn" };
if (data.decision === "deny") return { word: "denied", tone: "error" };
return data.ok === true ? { word: "ok", tone: "neutral" } : { word: "failed", tone: "error" };
}
/** What a task event says, after the task it is about. */
const withTask = (taskId: unknown, rest: string): string => joined(text(taskId) && `${text(taskId)}:`, rest);
@@ -161,8 +158,8 @@ const DESCRIBE: { [K in KnownType]: (data: Partial<EventData[K]>) => Draft } = {
"task.failed": (d) => ({ title: "Task failed", detail: withTask(d.task_id, text(d.error)), tone: "error" }),
"task.cancelled": (d) => ({ title: "Task cancelled", detail: text(d.task_id), tone: "neutral" }),
"tool.called": (d) => {
const { word, tone } = outcome(d);
const duration = typeof d.duration_ms === "number" ? ` in ${Math.round(d.duration_ms)} ms` : "";
const { word, tone } = TOOL_OUTCOME[toolStateOf(d)];
const duration = typeof d.duration_ms === "number" ? ` in ${formatMillis(d.duration_ms)}` : "";
const tool = text(d.tool);
// What it acted on, how it ended, and which tool under which permission the policy decided on.
const policy = joined(tool, text(d.permission) && `[${text(d.permission)}]`, text(d.decision));
@@ -179,7 +176,7 @@ const DESCRIBE: { [K in KnownType]: (data: Partial<EventData[K]>) => Draft } = {
tone: "warn",
}),
"approval.resolved": (d) => ({
title: d.decision === "approve" ? (d.always === true ? "Approved, and always allowed" : "Approved") : d.decision === "reject" ? "Rejected" : "Approval resolved",
title: d.decision === "approve" ? `${APPROVAL_WORD.approved}${d.always === true ? " for good" : ""}` : d.decision === "reject" ? APPROVAL_WORD.rejected : "Approval resolved",
detail: joined(text(d.approval_id), text(d.note) && `- ${text(d.note)}`),
tone: d.decision === "reject" ? "warn" : "ok",
}),
@@ -223,6 +220,12 @@ const DESCRIBE: { [K in KnownType]: (data: Partial<EventData[K]>) => Draft } = {
tone: d.changed === false ? "neutral" : "info",
}),
// The engine sent something the control plane could not read, so it is not in this log: this row is all there is of it.
"mcp.server.added": (d) => {
const call = (d.arguments ?? {}) as Record<string, unknown>;
const runs = [text(call.command), ...(Array.isArray(call.args) ? call.args.map(text) : [])].filter(Boolean).join(" ") || text(call.url);
return { title: "Added an MCP server", detail: joined(text(call.name), runs && `- ${runs}`), tone: "info" };
},
"mcp.server.refused": (d) => ({ title: "An MCP server was not added", detail: joined(text(d.name), text(d.reason) && `- ${text(d.reason)}`), tone: "warn" }),
"guest.event.refused": (d) => ({
title: "The computer sent an event that was not read",
detail: joined(text(d.type), typeof d.seq === "number" ? `(#${d.seq})` : "", text(d.problem) && `- ${text(d.problem)}`),
+5 -18
View File
@@ -1,21 +1,13 @@
/**
* Tasks that failed in the last 24 hours, which the Inbox puts under "Needs you" until the person has seen them and
* says so. The host has no route for "failed tasks of every Dot", so each Dot's list (the newest tasks, which is where
* a recent failure is) is read and the failed ones kept; a Dot whose list cannot be read is counted, not hidden.
* says so: one read of every Dot's (`GET /api/tasks?status=FAILED`).
*/
import type { InvisibleDotsClient } from "@invisible-dots/sdk";
import type { Dot, Task } from "./types";
import type { Task } from "./types";
/** How long a failure stays in the Inbox. */
export const FAILED_WINDOW_MS = 24 * 3_600_000;
export interface FailedTasks {
/** Failed tasks, newest first, as of when they were read; `recentFailures` applies the window and the dismissals. */
tasks: Task[];
/** Dots whose tasks could not be read. */
unread: number;
}
/** Whether a task failed less than a day before `now`. */
export function failedRecently(task: Pick<Task, "status" | "finished_at">, now: number): boolean {
if (task.status !== "FAILED" || task.finished_at === null) return false;
@@ -23,14 +15,9 @@ export function failedRecently(task: Pick<Task, "status" | "finished_at">, now:
return Number.isFinite(finished) && now - finished < FAILED_WINDOW_MS && finished <= now + 60_000;
}
function newestFirst(a: Task, b: Task): number {
return Date.parse(b.finished_at ?? "") - Date.parse(a.finished_at ?? "");
}
export async function loadFailedTasks(client: Pick<InvisibleDotsClient, "listTasks">, dots: readonly Pick<Dot, "id">[], now = Date.now()): Promise<FailedTasks> {
const answers = await Promise.allSettled(dots.map((dot) => client.listTasks(dot.id)));
const tasks = answers.flatMap((answer) => (answer.status === "fulfilled" ? answer.value : [])).filter((task) => failedRecently(task, now));
return { tasks: tasks.sort(newestFirst), unread: answers.filter((answer) => answer.status === "rejected").length };
/** The tasks of every Dot that failed within the window, newest first, as of when they were read; `recentFailures` applies the window and the dismissals as time goes on. */
export function loadFailedTasks(client: Pick<InvisibleDotsClient, "endedTasks">, now = Date.now()): Promise<Task[]> {
return client.endedTasks("FAILED", new Date(now - FAILED_WINDOW_MS));
}
/** The failures the Inbox shows: within the window, and not dismissed. */
-1
View File
@@ -48,7 +48,6 @@ const TONES: Record<string, Tone> = {
READY: "ok",
CREATING: "info",
WAITING_APPROVAL: "warn",
DISABLED: "neutral",
// VM states
PROVISIONING: "info",
STARTING: "info",
+92
View File
@@ -0,0 +1,92 @@
/**
* The form of one MCP server in the settings: what a person types (a command line or a URL, one variable, header or
* secret name per line) and the config entry it is (`mcp_servers.<name>`). The schema of the shared package checks the
* entry once it is in the draft; this only turns lines into the entry's shape and says which line it cannot read.
*/
import { isMcpStdioServer, MCP_STARTUP_TIMEOUT_BOUNDS, MCP_TIMEOUT_BOUNDS, type McpServerConfig } from "@invisible-dots/shared/browser";
export type McpServerKind = "command" | "url";
export interface McpServerForm {
kind: McpServerKind;
command: string;
/** One argument per line, as typed: an argument may hold spaces. */
args: string;
/** `NAME=value`, one per line. */
env: string;
url: string;
/** `Name: value`, one per line. */
headers: string;
/** One secret name per line: an environment variable of a command, a header of a URL. */
secrets: string;
timeoutS: number;
startupTimeoutS: number;
}
export const EMPTY_MCP_SERVER_FORM: McpServerForm = {
kind: "command",
command: "",
args: "",
env: "",
url: "",
headers: "",
secrets: "",
timeoutS: MCP_TIMEOUT_BOUNDS.default,
startupTimeoutS: MCP_STARTUP_TIMEOUT_BOUNDS.default,
};
function lines(text: string): string[] {
return text
.split("\n")
.map((line) => line.trim())
.filter((line) => line !== "");
}
/** `NAME<separator>value` lines as a record, or the first line that has no separator. */
function pairs(text: string, separator: string): { ok: true; value: Record<string, string> } | { ok: false; line: string } {
const value: Record<string, string> = {};
for (const line of lines(text)) {
const at = line.indexOf(separator);
if (at <= 0) return { ok: false, line };
value[line.slice(0, at).trim()] = line.slice(at + separator.length).trim();
}
return { ok: true, value };
}
/** The form of a declared server, to edit it. */
export function formOfServer(server: McpServerConfig): McpServerForm {
if (isMcpStdioServer(server)) {
return {
...EMPTY_MCP_SERVER_FORM,
kind: "command",
command: server.command,
args: server.args.join("\n"),
env: Object.entries(server.env).map(([name, value]) => `${name}=${value}`).join("\n"),
secrets: server.secrets.join("\n"),
timeoutS: server.timeout_s,
startupTimeoutS: server.startup_timeout_s,
};
}
return {
...EMPTY_MCP_SERVER_FORM,
kind: "url",
url: server.url,
headers: Object.entries(server.headers).map(([name, value]) => `${name}: ${value}`).join("\n"),
secrets: server.secrets.join("\n"),
timeoutS: server.timeout_s,
startupTimeoutS: server.startup_timeout_s,
};
}
/** The config entry the form says, or the line it cannot read. */
export function serverOfForm(form: McpServerForm): { ok: true; server: McpServerConfig } | { ok: false; problem: string } {
const secrets = lines(form.secrets);
if (form.kind === "command") {
const env = pairs(form.env, "=");
if (!env.ok) return { ok: false, problem: `"${env.line}" is not NAME=value` };
return { ok: true, server: { command: form.command.trim(), args: lines(form.args), env: env.value, secrets, timeout_s: form.timeoutS, startup_timeout_s: form.startupTimeoutS } };
}
const headers = pairs(form.headers, ":");
if (!headers.ok) return { ok: false, problem: `"${headers.line}" is not Name: value` };
return { ok: true, server: { url: form.url.trim(), headers: headers.value, secrets, timeout_s: form.timeoutS, startup_timeout_s: form.startupTimeoutS } };
}
+8 -4
View File
@@ -4,7 +4,7 @@
* permission it leaves out keeps the shipped default (`resolvePermission` in the shared package), so a preset can
* never disagree with the defaults about a permission it does not name.
*/
import { PERMISSIONS, resolvePermission, type Permission, type PermissionDecision } from "@invisible-dots/shared/browser";
import { mcpServerOf, PERMISSIONS, resolvePermission, type Permission, type PermissionDecision } from "@invisible-dots/shared/browser";
export type PermissionMap = Partial<Record<string, PermissionDecision>>;
@@ -41,9 +41,13 @@ export const PRESETS: Readonly<Record<PresetId, Preset>> = {
},
};
/** The permissions a preset writes into a config: a copy, so nobody edits the preset by accident. */
export function presetPermissions(id: PresetId): Record<string, PermissionDecision> {
return { ...PRESETS[id].permissions } as Record<string, PermissionDecision>;
/**
* The permissions a config has once a preset is chosen: the preset's (a copy, so nobody edits the preset by accident),
* and the permissions of its MCP servers as they were, which no preset is about: what a server's tools do is the server's.
*/
export function presetPermissions(id: PresetId, current: PermissionMap = {}): Record<string, PermissionDecision> {
const servers = Object.entries(current).filter(([permission, decision]) => mcpServerOf(permission) !== null && decision !== undefined);
return { ...(Object.fromEntries(servers) as Record<string, PermissionDecision>), ...PRESETS[id].permissions } as Record<string, PermissionDecision>;
}
function decisions(permissions: PermissionMap): PermissionDecision[] {
+10 -6
View File
@@ -4,7 +4,7 @@
* whether the model is offered each one right now. The tools come from the Dot's own table (`GET /tools`), which
* the engine owns, so a tool added there shows up here with no change in this file.
*/
import { PERMISSION_INFO, PERMISSIONS, resolvePermission, type DotConfig, type Permission, type PermissionDecision, type PermissionRisk, type ToolInfo } from "@invisible-dots/shared/browser";
import { mcpPermission, mcpServerNames, permissionInfo, PERMISSIONS, resolvePermission, type DotConfig, type PermissionDecision, type PermissionName, type PermissionRisk, type ToolInfo } from "@invisible-dots/shared/browser";
import { defaultDecision } from "./config-fields";
export const DECISIONS: readonly PermissionDecision[] = ["allow", "ask", "deny"];
@@ -25,7 +25,7 @@ export interface ToolState {
}
export interface PermissionRow {
permission: Permission;
permission: PermissionName;
label: string;
description: string;
risk: PermissionRisk;
@@ -49,23 +49,27 @@ const GROUP_LABELS: Readonly<Record<string, string>> = {
files: "Files",
browser: "Browser",
automations: "Automations",
mcp: "MCP servers",
};
function groupOf(permission: Permission): string {
function groupOf(permission: PermissionName): string {
return permission.split(".")[0]!;
}
/** The rows of the editor, grouped in the order of `PERMISSIONS`. `saved` is the config as the host has it; `draft` what the page holds. */
/**
* The rows of the editor, grouped in the order of `PERMISSIONS`, then one row per MCP server the draft declares, by name. `saved`
* is the config as the host has it; `draft` what the page holds.
*/
export function permissionGroups(draft: DotConfig, saved: DotConfig, tools: readonly ToolInfo[] | null): PermissionGroup[] {
const groups: PermissionGroup[] = [];
for (const permission of PERMISSIONS) {
for (const permission of [...PERMISSIONS, ...mcpServerNames(draft).map(mcpPermission)]) {
const id = groupOf(permission);
let group = groups.find((g) => g.id === id);
if (!group) {
group = { id, label: GROUP_LABELS[id] ?? id, rows: [] };
groups.push(group);
}
const info = PERMISSION_INFO[permission];
const info = permissionInfo(permission)!;
const decision = resolvePermission(draft, permission);
group.rows.push({
permission,
+143
View File
@@ -0,0 +1,143 @@
/**
* A tool call and an approval as a step of what the Dot did: the one reading of `tool.called`, `approval.requested`
* and `approval.resolved` that the chat (`chat-thread.ts`), a task's story (`task-events.ts`) and the Activity page
* (`events/view.ts`) draw, with one set of words for how a call ended and how an approval was answered. How long a
* call took is `formatMillis` (format.ts) in all three.
*/
import { toolLabel, type ToolFamily } from "@invisible-dots/shared/browser";
import type { Tone } from "./tone";
import type { StoredEvent } from "./types";
function text(value: unknown): string {
return typeof value === "string" ? value : "";
}
/**
* How a tool call ended. The engine reports a call when it has ended, so a call that is still running has no event
* yet; a call that waits for the person is the approval's step.
*/
export type ToolState = "ok" | "error" | "denied" | "interrupted";
export function toolStateOf(data: Record<string, unknown>): ToolState {
if (data.interrupted === true) return "interrupted";
if (data.decision === "deny") return "denied";
return data.ok === true ? "ok" : "error";
}
/**
* How a call ended, in a word and a tone. A denied call never ran: the Dot's config refused it, which is worth noticing
* but is not a failure; an interrupted one has no known outcome.
*/
export const TOOL_OUTCOME: Readonly<Record<ToolState, { word: string; tone: Tone }>> = {
ok: { word: "done", tone: "neutral" },
error: { word: "failed", tone: "error" },
denied: { word: "denied", tone: "warn" },
interrupted: { word: "interrupted", tone: "warn" },
};
export interface ToolStep {
kind: "tool";
id: number;
at: string;
tool: string;
label: string;
family: ToolFamily;
/** What the call acted on, as the engine reported it (one line); "" when it named nothing. */
target: string;
state: ToolState;
durationMs: number;
}
export type ApprovalOutcome = "waiting" | "approved" | "rejected";
/** How an approval stands, in the words of a step. */
export const APPROVAL_WORD: Readonly<Record<ApprovalOutcome, string>> = { waiting: "Waiting for an answer", approved: "Allowed", rejected: "Denied" };
export interface ApprovalStep {
kind: "approval";
id: number;
at: string;
approvalId: string;
tool: string;
label: string;
family: ToolFamily;
reason: string;
/** What the call needs the person's permission for, and with which arguments: what the approval card shows. */
permission: string;
arguments: Record<string, unknown>;
outcome: ApprovalOutcome;
/** What the person wrote to the Dot with the answer; "" for none. */
note: string;
/** The answer also allowed the permission for good. */
always: boolean;
}
/** A `tool.called` event as a step. */
export function toolStep(event: Pick<StoredEvent, "id" | "created_at" | "data">): ToolStep {
const d = event.data ?? {};
const tool = text(d.tool);
const { label, family } = toolLabel(tool, d.tty === true);
return {
kind: "tool",
id: event.id,
at: event.created_at,
tool,
label,
family,
target: text(d.target),
state: toolStateOf(d),
durationMs: typeof d.duration_ms === "number" ? d.duration_ms : 0,
};
}
/** An `approval.requested` event as a step that waits; `answerApproval` records its answer. */
export function approvalStep(event: Pick<StoredEvent, "id" | "created_at" | "data">): ApprovalStep {
const d = event.data ?? {};
const tool = text(d.tool);
const { label, family } = toolLabel(tool);
return {
kind: "approval",
id: event.id,
at: event.created_at,
approvalId: text(d.approval_id),
tool,
label,
family,
reason: text(d.reason),
permission: text(d.permission),
arguments: typeof d.arguments === "object" && d.arguments !== null && !Array.isArray(d.arguments) ? (d.arguments as Record<string, unknown>) : {},
outcome: "waiting",
note: "",
always: false,
};
}
/** An `approval.resolved` event, written into the step of the approval it answers. */
export function answerApproval(step: ApprovalStep, resolved: Pick<StoredEvent, "data">): void {
const d = resolved.data ?? {};
step.outcome = d.decision === "approve" ? "approved" : "rejected";
step.note = text(d.note);
step.always = d.always === true;
}
/**
* The steps of the tool calls and approvals among `events`, in the order of the events, and how each approval was
* answered. `keep` says which `tool.called` and `approval.requested` events are the caller's (the chat's turns, one
* task); an answer is found by its approval's id, so one to an approval that was not kept changes nothing.
*/
export function stepsOf(events: readonly StoredEvent[], keep: (event: StoredEvent) => boolean): Array<ToolStep | ApprovalStep> {
const steps: Array<ToolStep | ApprovalStep> = [];
const approvals = new Map<string, ApprovalStep>();
for (const event of events) {
if (event.type === "tool.called" && keep(event)) steps.push(toolStep(event));
else if (event.type === "approval.requested" && keep(event)) {
const step = approvalStep(event);
approvals.set(step.approvalId, step);
steps.push(step);
} else if (event.type === "approval.resolved") {
const step = approvals.get(text(event.data?.approval_id));
if (step) answerApproval(step, event);
}
}
return steps;
}
+29 -73
View File
@@ -3,8 +3,8 @@
* drawer. The log is the one record of both (the task row keeps only its state, result and spend). Both are read by
* the task (`task_id`, indexed in the database), so what it costs does not grow with the age of the Dot.
*/
import type { InvisibleDotsClient } from "@invisible-dots/sdk";
import { mergeEvents, readEventLog, readRecentEvents } from "./event-log";
import { mergeEvents, readEventLog, readRecentEvents, type InvisibleDotsClient } from "@invisible-dots/sdk";
import { stepsOf, type ApprovalStep, type ToolStep } from "./steps";
import type { StoredEvent } from "./types";
function text(value: unknown): string {
@@ -59,85 +59,41 @@ export async function readTaskProgress(client: Pick<InvisibleDotsClient, "events
return newest === undefined ? null : progressOfEvent(newest);
}
export type ApprovalOutcome = "waiting" | "approved" | "rejected";
export type StoryStep =
| { kind: "created"; id: number; at: string; description: string; priority: number | null }
| { kind: "started"; id: number; at: string }
| { kind: "progress"; id: number; at: string; text: string }
| { kind: "tool"; id: number; at: string; tool: string; target: string; decision: string; ok: boolean; interrupted: boolean; tty: boolean; durationMs: number }
| { kind: "approval"; id: number; at: string; approvalId: string; tool: string; reason: string; outcome: ApprovalOutcome; note: string; always: boolean }
| ToolStep
| ApprovalStep
| { kind: "completed"; id: number; at: string; summary: string }
| { kind: "failed"; id: number; at: string; error: string }
| { kind: "cancelled"; id: number; at: string };
/** A step of the task's own life (created, started, said something, ended); null for an event that is none. */
function lifeStep(event: StoredEvent): StoryStep | null {
const base = { id: event.id, at: event.created_at };
const d = event.data ?? {};
switch (event.type) {
case "task.created":
return { ...base, kind: "created", description: text(d.description), priority: typeof d.priority === "number" ? d.priority : null };
case "task.started":
return { ...base, kind: "started" };
case "task.progress":
return { ...base, kind: "progress", text: text(d.text) };
case "task.completed":
return { ...base, kind: "completed", summary: text(d.summary) };
case "task.failed":
return { ...base, kind: "failed", error: text(d.error) };
case "task.cancelled":
return { ...base, kind: "cancelled" };
default:
return null;
}
}
/** The steps of one task, in the order they happened. An approval is one step: asked, and then how it was answered. */
export function storyOf(events: readonly StoredEvent[], taskId: string): StoryStep[] {
const steps: StoryStep[] = [];
const approvals = new Map<string, Extract<StoryStep, { kind: "approval" }>>();
for (const event of events) {
const base = { id: event.id, at: event.created_at };
const d = event.data ?? {};
if (taskIdOf(event) !== taskId) continue;
switch (event.type) {
case "task.created":
steps.push({ ...base, kind: "created", description: text(d.description), priority: typeof d.priority === "number" ? d.priority : null });
break;
case "task.started":
steps.push({ ...base, kind: "started" });
break;
case "task.progress":
steps.push({ ...base, kind: "progress", text: text(d.text) });
break;
case "tool.called":
steps.push({
...base,
kind: "tool",
tool: text(d.tool),
target: text(d.target),
decision: text(d.decision),
ok: d.ok === true,
interrupted: d.interrupted === true,
tty: d.tty === true,
durationMs: typeof d.duration_ms === "number" ? d.duration_ms : 0,
});
break;
case "approval.requested": {
const step: Extract<StoryStep, { kind: "approval" }> = {
...base,
kind: "approval",
approvalId: text(d.approval_id),
tool: text(d.tool),
reason: text(d.reason),
outcome: "waiting",
note: "",
always: false,
};
approvals.set(step.approvalId, step);
steps.push(step);
break;
}
case "approval.resolved": {
const step = approvals.get(text(d.approval_id));
if (step) {
step.outcome = d.decision === "approve" ? "approved" : "rejected";
step.note = text(d.note);
step.always = d.always === true;
}
break;
}
case "task.completed":
steps.push({ ...base, kind: "completed", summary: text(d.summary) });
break;
case "task.failed":
steps.push({ ...base, kind: "failed", error: text(d.error) });
break;
case "task.cancelled":
steps.push({ ...base, kind: "cancelled" });
break;
default:
break;
}
}
return steps;
const own = events.filter((event) => taskIdOf(event) === taskId);
const life = own.flatMap((event) => lifeStep(event) ?? []);
return [...life, ...stepsOf(own, () => true)].sort((a, b) => a.id - b.id);
}
+1 -1
View File
@@ -70,7 +70,7 @@ describe("the Activity page", () => {
expect(titles()).toEqual(["Assistant replied", "Ran a command", "You sent a message", "The engine started"]);
const [reply, tool, message, started] = rows();
expect(within(tool!).getByText("ls -la | ok in 12 ms | exec [computer.exec] allow")).toBeTruthy();
expect(within(tool!).getByText("ls -la | done in 12 ms | exec [computer.exec] allow")).toBeTruthy();
expect(within(tool!).getByText("tool.called")).toBeTruthy();
expect(within(message!).getByText("via Telegram")).toBeTruthy();
expect(within(message!).getByText("find fares")).toBeTruthy();
+1 -2
View File
@@ -21,9 +21,8 @@ describe("the state pill of a Dot", () => {
expect(statePill("WAITING_APPROVAL", null)).toMatchObject({ label: "Waiting for you", tone: "warn" });
});
it("lets an error, a disabled Dot and a Dot being created override the agent's last word", () => {
it("lets an error and a Dot being created override the agent's last word", () => {
expect(statePill("ERROR", "THINKING")).toEqual({ label: "Error", tone: "error", working: false });
expect(statePill("DISABLED", "IDLE")).toMatchObject({ label: "Disabled", tone: "neutral" });
expect(statePill("CREATING", null)).toMatchObject({ label: "Preparing the computer", working: true });
});
});
-3
View File
@@ -86,10 +86,7 @@ describe("the avatar ring", () => {
it("is grey for a computer that is stopped or missing, green for one that runs quietly", () => {
expect(ring({ computerState: "STOPPED" })).toBe("stopped");
expect(ring({ computerState: null })).toBe("stopped");
expect(ring({ status: "DISABLED", computerState: "STOPPED" })).toBe("stopped");
expect(ring({})).toBe("ready");
// The host's guest routes answer only a RUNNING computer, so an IDLE one is not up.
expect(ring({ computerState: "IDLE", agentState: "IDLE" })).toBe("stopped");
});
it("breathes while the agent thinks, plans or runs a tool, and while the computer changes state", () => {
+16 -28
View File
@@ -2,7 +2,6 @@ import { CHANNEL_KINDS, CHANNEL_STATUSES, type ChannelKind, type ChannelPeerReco
import { describe, expect, it } from "vitest";
import {
accountHref,
accountLabel,
CHANNEL_LABELS,
CHANNEL_NOTES,
channelState,
@@ -48,14 +47,6 @@ describe("the chip of a channel", () => {
});
describe("the account of a channel", () => {
it("is a bot's @name and a number with its plus, and nothing before the channel reports one", () => {
expect(accountLabel("telegram", "dot_helper_bot")).toBe("@dot_helper_bot");
expect(accountLabel("whatsapp", "15550001111")).toBe("+15550001111");
expect(accountLabel("whatsapp", "+15550001111")).toBe("+15550001111");
expect(accountLabel("telegram", null)).toBeNull();
expect(accountLabel("whatsapp", "")).toBeNull();
});
it("links a Telegram bot's page, and nothing for a number or for a name that is not a bot's", () => {
expect(accountHref("telegram", "dot_helper_bot")).toBe("https://t.me/dot_helper_bot");
expect(accountHref("telegram", "../evil")).toBeNull();
@@ -139,27 +130,24 @@ describe("the channels that need the person", () => {
expect(needsRelinking({ enabled: true, status: "connecting" })).toBe(false);
});
it("reads every Dot's channels, keeps those to link again with the host's words, and counts the Dots it could not read", async () => {
const answers: Record<string, ChannelRecord[] | Error> = {
a: [record("telegram", { status: "needs_relink", status_detail: "Telegram refused the token" }), record("whatsapp")],
b: [record("telegram"), record("whatsapp", { status: "needs_relink", status_detail: null })],
c: new Error("down"),
d: [record("telegram", { enabled: false, status: "needs_relink" })],
};
it("reads every Dot's channels in one request and keeps those to link again, with the host's words", async () => {
let reads = 0;
const client = {
channels: async (id: string) => {
const answer = answers[id]!;
if (answer instanceof Error) throw answer;
return answer;
everyChannel: async () => {
reads += 1;
return [
{ dot_id: "a", ...record("telegram", { status: "needs_relink", status_detail: "Telegram refused the token" }) },
{ dot_id: "a", ...record("whatsapp") },
{ dot_id: "b", ...record("telegram") },
{ dot_id: "b", ...record("whatsapp", { status: "needs_relink", status_detail: null }) },
{ dot_id: "d", ...record("telegram", { enabled: false, status: "needs_relink" }) },
];
},
};
expect(await loadRelinks(client, [{ id: "a" }, { id: "b" }, { id: "c" }, { id: "d" }])).toEqual({
relinks: [
{ dot_id: "a", kind: "telegram", detail: "Telegram refused the token" },
{ dot_id: "b", kind: "whatsapp", detail: null },
],
unread: 1,
});
expect(await loadRelinks(client, [])).toEqual({ relinks: [], unread: 0 });
expect(await loadRelinks(client)).toEqual([
{ dot_id: "a", kind: "telegram", detail: "Telegram refused the token" },
{ dot_id: "b", kind: "whatsapp", detail: null },
]);
expect(reads).toBe(1);
});
});
Loaded 100 of 617 files, more files were not shown because too many files have changed in this diff. Show more