Compare commits

...
2 Commits
Author SHA1 Message Date
feder-cr 0e502db1c0 dot-browser: a link to nothing is no browser, PATH checked where the image is made, failures say what they did
The check that dot's PATH reaches the server is the image's: it runs at install, not after a switch has already moved
the link, so install.sh's message on a failed switch is true (the browser is either the one the Dot had or the staged
one). A link whose environment is gone is no current browser. newest's awk reads every line instead of exiting early
under pipefail, and the browser smoke's preparation fails at the resolution that failed.
2026-10-15 04:27:16 +02:00
feder-cr c16e144f89 browser: every Dot follows the newest release, and install.sh alone starts the Dot's units
A Dot kept the browser of the golden image it was made from. runtime/dot-browser.sh (the old build-browser-env.sh,
now on the runtime disk as well as the builder seed) is the one owner of where the Dot's browser lives: each release
in an environment of its own, browser/<mcp>-<playwright>-<core>, and the Dot's browser the one
~dot/.local/bin/invisible-playwright-mcp links to. At every start install.sh switches that link to a release an
earlier start staged, in one rename and before the Dot's units, and after them stages the newest release as uv
resolves it, in the background: its environment and its browser engine, through the Dot's own way out. A start never
waits for a release; without the network the Dot keeps its browser. provision.sh installs the golden image's browser
with the same script.

The switch has to come before the engine, and that ordering did not hold: the units were enabled, so on every boot
after the first systemd started them at multi-user.target, before cloud-init's final stage ran install.sh
(cloud-final.service is After=multi-user.target; a Dot's serial log shows dot-agentd and the engine up two seconds
before install.sh). A Dot with a VM proxy went out before its proxy was set. The units are no longer enabled:
install.sh starts them, after the proxy and the switch, and removes the links an earlier runtime made.
2026-10-15 04:22:28 +02:00
22 changed files with 385 additions and 143 deletions

No files matched your search

+26 -2
View File
@@ -285,10 +285,11 @@ QEMU on the same disk would corrupt it.
part of the inputs digest, so a new release is a new image, and the image's
manifest records the ones installed; the scheduled run of
`.github/workflows/golden-image.yml` publishes that image without any change
to this repository. The builder installs them with
to this repository. The builder installs them with the runtime disk's own
`guest/image-builder/runtime/dot-browser.sh` (below), which runs
`uv pip install --only-binary :all:`, so no package's build script runs, and
refuses an environment that holds another browser (the names in
`guest/image-builder/builder/other-browsers.txt`). The apt packages come from the Ubuntu archive as it
`guest/image-builder/runtime/other-browsers.txt`). The apt packages come from the Ubuntu archive as it
was at the moment `apt_snapshot` of `pins.json` names (snapshot.ubuntu.com,
`apt-get --snapshot`), so they and their dependencies are the same on any day
of the build; apt checks their signatures.
@@ -297,6 +298,29 @@ QEMU on the same disk would corrupt it.
the `dot-agentd` binary and the systemd units. A new
version of our code is a new ISO and a VM restart, not a new golden image and
not a rebuilt overlay.
- A Dot keeps the golden image it was made from, and its **browser follows the
newest release** all the same: `dot-browser` (on the runtime disk, at
`browser/dot-browser`) keeps each release in an environment of its own,
`~dot/.local/share/invisible-dots/browser/<mcp>-<playwright>-<core>` (a venv
cannot move), and the Dot's browser is the one
`~dot/.local/bin/invisible-playwright-mcp` links to. At every start,
`install.sh` switches that link to a release an earlier start staged, in one
rename and before the Dot's units start, and removes the other environments;
after the units, it stages in the background (`dot-browser-stage`, a transient
unit) the newest release as uv resolves `invisible-playwright-mcp`, when it is
not the Dot's already: its environment and the browser engine its
`invisible-core` seals, through the Dot's own way out. A start never waits for
a new release and the Dot uses it from its next start; without the network a
stage fails and the Dot keeps its browser. The engine reads what the server
serves when it starts (section 6), so it always reads the release the switch
left. The library keeps every engine it downloaded in its cache.
- `install.sh` (cloud-init's per-boot script) is what starts the Dot's units,
on every boot, after the VM proxy and the browser switch: they are copied into
`/etc/systemd/system` and never enabled. Enabled, systemd would start them at
`multi-user.target`, and cloud-init's final stage, which runs `install.sh`
and writes the seed's `config.json` first, comes after that target
(`cloud-final.service` is `After=multi-user.target`): on every boot after the
first, a Dot would have gone out before its proxy was set.
The host builds the runtime disk from this checkout, and the golden image from
public sources (`invisible-dots image build`, code in `guest/image-builder/`)
+10 -8
View File
@@ -28,7 +28,7 @@ image's SHA-256; `invisible-dots doctor` checks an image against it with
3. Writes the builder seed with `@invisible-dots/iso`: one ISO labelled
`cidata` holding `user-data`, `meta-data`, `provision.sh`, `pins.env`,
the pinned downloads, `engine-requirements.lock`, `build-engine-env.sh`,
`build-browser-env.sh` and `other-browsers.txt`. The engine's own source is
and the runtime disk's `dot-browser.sh` with `other-browsers.txt`. The engine's own source is
not on it: it is ours, so it travels on the runtime ISO.
4. Boots it once with the QEMU the host runs Dots with, on the accelerator
the vm-manager chose (`-accel kvm` or `-accel whpx`, never emulation), the
@@ -39,9 +39,9 @@ image's SHA-256; `invisible-dots doctor` checks an image against it with
`pins.json`'s `apt_snapshot` (`apt-get --snapshot`), uv, then removes the
cloud image's services a Dot never uses (snapd, unattended-upgrades, apport, the LXD stubs, Ubuntu Pro,
the release upgrader, the SSH server, pollinate, which would tell Canonical of
every new Dot), then builds the Dot's browser with
`build-browser-env.sh ~dot/.local/share/invisible-dots/mcp <the three releases>`, as user
`dot`: a virtual environment filled with
every new Dot), then installs the Dot's browser with
`dot-browser.sh install <the three releases>`, as user
`dot`: a virtual environment of its own under `~/.local/share/invisible-dots/browser/`, filled with
`uv pip install --only-binary :all:` of the versions `pins.env` names, its
`invisible-playwright-mcp` linked into `~/.local/bin`, and
`invisible-playwright fetch` run from that environment, so the cached browser
@@ -49,7 +49,7 @@ image's SHA-256; `invisible-dots doctor` checks an image against it with
database, fetched from its release at a launch). The
browser smoke runs the same script. The only
browser a Dot has is that server: the build refuses an environment holding a
package `builder/other-browsers.txt` names, and a test refuses one among the apt
package `runtime/other-browsers.txt` names, and a test refuses one among the apt
packages and the build scripts. `provision.sh`
then builds the engine's Python
environment with `build-engine-env.sh <lock> /opt/invisible-dots-engine
@@ -77,7 +77,9 @@ manifest, so a new release is a new image, and the scheduled run of
`.github/workflows/golden-image.yml` publishes it. The engine does not assume a
version: at start it reads the tools and instructions of the server its Dot has
(`invisible_engine_dots/nanobot/dots/server_interface.py`), so a Dot on an older
golden image keeps working with every new runtime disk.
golden image keeps working with every new runtime disk. Its browser follows the newest release all the same:
at every start the runtime disk's `install.sh` switches to a release an earlier start staged, before the Dot's units
start, and stages the newest one in the background after them (`runtime/dot-browser.sh`).
`builder/engine-requirements.lock` is the same for the engine
(`invisible_engine_dots/pyproject.toml`): the one place its dependency
@@ -160,8 +162,8 @@ from Linux or WSL.
## The browser smoke
`test/smoke/run.sh --suite browser` runs the same two daemons with the Dot's real browser:
the apt packages of `pins.json`, the newest browser built by `builder/build-browser-env.sh`,
the runtime disk's `dot-desktop` script (Xvfb and XFCE on
the apt packages of `pins.json`, the browser installed by `runtime/dot-browser.sh` on the release
before the newest and brought to the newest as a Dot's starts bring it, the runtime disk's `dot-desktop` script (Xvfb and XFCE on
`:0`), and the stand-in model navigating, reading and taking screenshots of pages served
from the container. The `browser-smoke` job of `.github/workflows/tests.yml` runs it, and
the `gate` job needs it. It needs the network for the downloads of the build and for the
@@ -1,48 +0,0 @@
#!/usr/bin/env bash
# Builds the Dot's browser (architecture sections 3.3 and 6): the Python environment of invisible-playwright-mcp
# (as user dot), the command on dot's PATH, the browser engine the wrapper's seal expects. provision.sh runs it as
# root inside the golden image's builder VM, and the browser smoke runs it on a plain ubuntu:24.04 container, so both
# build the same thing. The Dot's browser is this server and nothing else (owner's rule).
#
# build-browser-env.sh <environment directory> [<requirement>...]
#
# The requirements are what to install; with none, the newest invisible-playwright-mcp and what it needs. The golden
# image passes the release it resolved from the index (browser-release.ts), so the image is what its digest names;
# the browser smoke installs the newest. No version of the browser is written in the repository (owner's rule).
#
# The command is linked into ~dot/.local/bin, where the model's commands find it; the engine's unit names it by that
# path (INVISIBLE_DOTS_MCP_COMMAND), starts it as dot, and reads what it serves from it when it starts. The engine is
# fetched with the invisible-playwright of that environment, so the cached engine is the one the server's seal
# expects, in ~dot/.cache/invisible-playwright.
set -Eeuo pipefail
usage="usage: build-browser-env.sh <environment directory> [<requirement>...]"
env_dir="${1:?$usage}"
shift
requirements=("$@")
[ "${#requirements[@]}" -gt 0 ] || requirements=(invisible-playwright-mcp)
bin_dir=/home/dot/.local/bin
other_browsers="$(dirname "${BASH_SOURCE[0]}")/other-browsers.txt"
[ -f "$other_browsers" ] || { echo "build-browser-env: no $other_browsers" >&2; exit 1; }
id dot >/dev/null 2>&1 || { echo "build-browser-env: user dot does not exist" >&2; exit 1; }
as_dot() { sudo -u dot -H env PATH="/home/dot/.local/bin:/usr/local/bin:/usr/bin:/bin" "$@"; }
as_dot mkdir -p "$bin_dir" "$(dirname "$env_dir")"
as_dot uv venv --quiet --python /usr/bin/python3 "$env_dir"
# Wheels only, as for the engine: no build script of any package runs. --no-cache: the image does not need uv's
# download cache in dot's home.
as_dot uv pip install --no-cache --python "$env_dir/bin/python" --only-binary :all: "${requirements[@]}"
# What was installed, not what was asked for: no package of it may be another browser.
installed="$(as_dot uv pip freeze --python "$env_dir/bin/python" | sed -n 's/^\([A-Za-z0-9._-]*\)==.*/\1/p' | tr 'A-Z_.' 'a-z--')"
refused=""
while read -r name; do
case "$name" in ''|'#'*) continue ;; esac
refused="$refused$(printf '%s\n' "$installed" | grep -E "^${name}(-.*)?$" || true)"
done < "$other_browsers"
[ -z "$refused" ] || { echo "build-browser-env: another browser would enter the image (owner's rule): $refused" >&2; exit 1; }
[ -x "$env_dir/bin/invisible-playwright" ] || { echo "build-browser-env: no invisible-playwright in $env_dir/bin" >&2; exit 1; }
[ -x "$env_dir/bin/invisible-playwright-mcp" ] || { echo "build-browser-env: no invisible-playwright-mcp in $env_dir/bin" >&2; exit 1; }
as_dot ln -sfn "$env_dir/bin/invisible-playwright-mcp" "$bin_dir/invisible-playwright-mcp"
as_dot "$env_dir/bin/invisible-playwright" fetch
as_dot sh -c 'command -v invisible-playwright-mcp' >/dev/null || { echo "build-browser-env: invisible-playwright-mcp is not on the PATH of dot" >&2; exit 1; }
+7 -7
View File
@@ -78,14 +78,14 @@ id dotagentd >/dev/null 2>&1 || { console "the builder seed made no user dotagen
step "installing invisible-playwright-mcp $MCP_VERSION (invisible-playwright $PLAYWRIGHT_VERSION, invisible-core $CORE_VERSION), fetching the browser engine"
# The release is the newest on the index when the host started the build (browser-release.ts): the three versions
# that make it up are installed exactly, and uv resolves what they need. The environment is built as dot by the
# script the seed carries, the one the browser smoke runs too (section 3.3), which refuses any other browser among
# what it installed. The engine is fetched with the invisible-playwright of that environment, so the cached engine is
# the one the server's seal expects, in ~dot/.cache/invisible-playwright.
# that make it up are installed exactly, and uv resolves what they need. The seed carries the runtime disk's
# dot-browser, the one owner of where the Dot's browser lives (section 3.3), which every Dot runs at its starts to
# bring its browser to the newest release; it builds the environment as dot, refuses any other browser among what it
# installed, and fetches the engine with the invisible-playwright of that environment, so the cached engine is the
# one the server's seal expects, in ~dot/.cache/invisible-playwright.
as_dot() { sudo -u dot -H env PATH="/home/dot/.local/bin:/usr/local/bin:/usr/bin:/bin" "$@"; }
mcp_env=/home/dot/.local/share/invisible-dots/mcp
bash "$payload/$BROWSER_BUILD" "$mcp_env" \
"invisible-playwright-mcp==$MCP_VERSION" "invisible-playwright==$PLAYWRIGHT_VERSION" "invisible-core==$CORE_VERSION"
bash "$payload/$BROWSER" install "$MCP_VERSION" "$PLAYWRIGHT_VERSION" "$CORE_VERSION"
mcp_env="$(dirname "$(dirname "$(readlink -f /home/dot/.local/bin/invisible-playwright-mcp)")")"
# What the environment holds, read from it: the manifest records what was installed, not what was asked for.
installed_version() { as_dot "$mcp_env/bin/python" -c 'import sys, importlib.metadata as m; print(m.version(sys.argv[1]))' "$1"; }
# The engine the image carries is read from the library that decides it (the seal of invisible_core: tag,
+168
View File
@@ -0,0 +1,168 @@
#!/usr/bin/env bash
# dot-browser: the Dot's browser (architecture sections 3.3 and 6), invisible-playwright-mcp, and the one owner of where
# it lives in the guest. The Dot's browser is this server and nothing else (owner's rule), and it is the newest release:
# no version of it is written in the repository.
#
# dot-browser install <mcp> <playwright> <core> build that release and make it the Dot's browser (provision.sh)
# dot-browser newest [<requirement>...] the newest release as uv resolves invisible-playwright-mcp (with
# the requirements given, if any): "<mcp> <playwright> <core>"
# dot-browser stage build the newest release beside the Dot's browser, when it is not
# that already; it becomes the Dot's browser at the next switch
# dot-browser switch make the staged release the Dot's browser, and remove every other
#
# Each release is an environment of its own, browser/<mcp>-<playwright>-<core> under ~dot/.local/share/invisible-dots
# (a venv cannot move), with the browser engine its invisible-core seals, fetched into ~dot/.cache/invisible-playwright.
# The Dot's browser is the environment ~dot/.local/bin/invisible-playwright-mcp links to: the engine's unit names that
# link, starts it as dot, and reads what it serves through it when it starts. The runtime disk's install.sh runs
# `switch` at every boot before it starts the Dot's units, so the browser changes only while nothing of it runs, and
# `stage` after them, in the background, through the Dot's own way out (its proxy, when it has one): a newer release
# costs a start nothing, and the Dot uses it from the next one. Without the network a stage fails and the Dot keeps
# its browser.
#
# Run as root (provision.sh, install.sh); everything of the browser is dot's and is done as dot.
set -Eeuo pipefail
usage="usage: dot-browser install <mcp> <playwright> <core> | newest [<requirement>...] | stage | switch"
packages=(invisible-playwright-mcp invisible-playwright invisible-core)
share=/home/dot/.local/share/invisible-dots
releases=$share/browser
staged=$releases/next
# The environment the golden images built before this layout, in place of a release directory.
legacy_env=$share/mcp
command_link=/home/dot/.local/bin/invisible-playwright-mcp
other_browsers="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")/other-browsers.txt"
log() { printf 'dot-browser: %s\n' "$*" >&2; }
die() { log "$*"; exit 1; }
as_dot() { sudo -u dot -H env PATH="/home/dot/.local/bin:/usr/local/bin:/usr/bin:/bin" "$@"; }
# newest [<requirement>...]: what uv installs for invisible-playwright-mcp now, wheels only, for the guest's python.
newest() {
local resolved
resolved="$(printf '%s\n' "${packages[0]}" "$@" | as_dot uv pip compile - --quiet --no-cache --no-header --no-annotate \
--only-binary :all: --python /usr/bin/python3)" || die "uv could not resolve ${packages[0]} (is the index reachable?)"
local versions=() package version
for package in "${packages[@]}"; do
# uv writes each name normalized (PEP 503); normalized here too, so the match does not depend on it.
version="$(printf '%s\n' "$resolved" | awk -F'==' -v want="$package" \
'{ name = tolower($1); gsub(/[_.]+/, "-", name); if (name == want && !found) { print $2; found = 1 } }')"
[ -n "$version" ] || die "uv's resolution of ${packages[0]} has no $package"
versions+=("$version")
done
printf '%s %s %s\n' "${versions[@]}"
}
# The environment of the Dot's browser, from its command's link; empty when it has none (no link, or a link to nothing).
current_env() {
[ -L "$command_link" ] && [ -e "$command_link" ] || return 0
dirname "$(dirname "$(readlink -f "$command_link")")"
}
# The release an environment holds, read from it: "<mcp> <playwright> <core>".
release_of() {
as_dot "$1/bin/python" -c 'import sys, importlib.metadata as m; print(" ".join(m.version(p) for p in sys.argv[1:]))' \
"${packages[@]}"
}
# build <directory> <mcp> <playwright> <core>: that release's environment, refused when it holds another browser, with
# the browser engine its invisible-core seals. uv resolves what the three need; wheels only, so no build script of any
# package runs, and --no-cache, so dot's home keeps no download cache.
build() {
local env_dir=$1
local requirements=("${packages[0]}==$2" "${packages[1]}==$3" "${packages[2]}==$4")
[ -f "$other_browsers" ] || die "no $other_browsers"
[ ! -e "$env_dir" ] || die "$env_dir exists already"
as_dot mkdir -p "$(dirname "$env_dir")"
as_dot uv venv --quiet --python /usr/bin/python3 "$env_dir"
as_dot uv pip install --no-cache --python "$env_dir/bin/python" --only-binary :all: "${requirements[@]}"
# What was installed, not what was asked for: no package of it may be another browser.
local installed refused="" name
installed="$(as_dot uv pip freeze --python "$env_dir/bin/python" | sed -n 's/^\([A-Za-z0-9._-]*\)==.*/\1/p' | tr 'A-Z_.' 'a-z--')"
while read -r name; do
case "$name" in ''|'#'*) continue ;; esac
refused="$refused$(printf '%s\n' "$installed" | grep -E "^${name}(-.*)?$" || true)"
done < "$other_browsers"
[ -z "$refused" ] || die "another browser would enter the Dot (owner's rule): $refused"
[ -x "$env_dir/bin/invisible-playwright" ] || die "no invisible-playwright in $env_dir/bin"
[ -x "$env_dir/bin/invisible-playwright-mcp" ] || die "no invisible-playwright-mcp in $env_dir/bin"
as_dot "$env_dir/bin/invisible-playwright" fetch
}
# use <directory>: that environment's server becomes the Dot's browser, in one rename.
use() {
as_dot mkdir -p "$(dirname "$command_link")"
as_dot ln -sfn "$1/bin/invisible-playwright-mcp" "$command_link.new"
as_dot mv -T "$command_link.new" "$command_link"
}
# prune <directory>...: every release environment but these is removed, the golden image's former one included.
prune() {
local dir keep
for dir in "$releases"/*/ "$legacy_env/"; do
dir=${dir%/}
[ -d "$dir" ] && [ ! -L "$dir" ] || continue
for keep in "$@"; do [ "$dir" != "$keep" ] || continue 2; done
as_dot rm -rf -- "$dir"
log "removed $dir"
done
}
install_release() {
local env_dir="$releases/$1-$2-$3"
build "$env_dir" "$@"
use "$env_dir"
# The image's PATH of dot, where the model's commands find the server, reaches the link.
as_dot sh -c 'command -v invisible-playwright-mcp' >/dev/null || die "invisible-playwright-mcp is not on the PATH of dot"
prune "$env_dir"
log "the Dot's browser is invisible-playwright-mcp $1 (invisible-playwright $2, invisible-core $3)"
}
stage() {
local release current next="" mcp playwright core env_dir
release="$(newest)"
read -r mcp playwright core <<< "$release"
current="$(current_env)"
[ -L "$staged" ] && next="$(readlink -f "$staged")"
if [ -n "$current" ] && [ "$(release_of "$current")" = "$release" ]; then
# Nothing newer: a release staged before is not newer either.
[ -z "$next" ] || as_dot rm -f "$staged"
prune "$current"
log "the Dot's browser is the newest release, invisible-playwright-mcp $mcp"
return
fi
env_dir="$releases/$mcp-$playwright-$core"
if [ "$next" = "$env_dir" ]; then
log "invisible-playwright-mcp $mcp is staged already; the Dot uses it from its next start"
return
fi
# What an earlier stage left (a release no longer the newest, a build that failed or stopped halfway) makes room
# for this one; a build that fails here is removed the same way by the next stage.
[ -z "$next" ] || as_dot rm -f "$staged"
prune "$current"
build "$env_dir" "$mcp" "$playwright" "$core"
as_dot ln -sfn "$env_dir" "$staged.new"
as_dot mv -T "$staged.new" "$staged"
log "staged invisible-playwright-mcp $mcp (invisible-playwright $playwright, invisible-core $core); the Dot uses it from its next start"
}
switch() {
local next
[ -L "$staged" ] || return 0
next="$(readlink -f "$staged")"
# A stage links its release only once the release is whole, so this holds unless the link was made by hand.
[ -x "$next/bin/invisible-playwright-mcp" ] || die "the staged release $next has no invisible-playwright-mcp"
use "$next"
as_dot rm -f "$staged"
prune "$next"
log "the Dot's browser is now $(basename "$next")"
}
[ "$(id -u)" -eq 0 ] || die "run it as root ($usage)"
id dot >/dev/null 2>&1 || die "user dot does not exist"
case "${1:-}" in
install) [ $# -eq 4 ] || die "$usage"; shift; install_release "$@" ;;
newest) shift; newest "$@" ;;
stage) [ $# -eq 1 ] || die "$usage"; stage ;;
switch) [ $# -eq 1 ] || die "$usage"; switch ;;
*) die "$usage" ;;
esac
+24 -3
View File
@@ -1,11 +1,20 @@
#!/usr/bin/env bash
# The runtime ISO's install hook. cloud-init runs it as root on every boot
# (a per-boot script written by the seed), after the ISO is mounted at
# /opt/invisible-dots. It installs the guest units and starts them.
# /opt/invisible-dots. It sets up the Dot's way out, brings in the browser a
# previous start staged, installs the guest units and starts them, then stages
# a newer browser in the background.
#
# The units are COPIED into /etc/systemd/system rather than linked: a linked
# unit lives on the ISO, which is not mounted yet when systemd loads units at
# early boot, so it would be "not found" on every boot after the first.
# early boot, so it would be "not found" on every boot after the first. They
# are not enabled: this script starts them, on every boot, after what has to
# come first. Enabled, systemd would start them at multi-user.target, and
# cloud-init's final stage, which runs this script, comes after that target
# (cloud-final.service: After=multi-user.target), so on every boot after the
# first the Dot would go out before its proxy is set and its engine would read
# a browser about to be switched. This script is also the first point of the
# boot that has the seed's config.json (written by the same final stage).
set -euo pipefail
runtime="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)"
@@ -85,6 +94,15 @@ visudo -cqf /etc/sudoers.d/dot-install.new && mv -f /etc/sudoers.d/dot-install.n
# A /run/user/<uid> for the desktop session without anyone logging in.
loginctl enable-linger dot || log "warning: could not enable lingering for dot; the desktop runs without XDG_RUNTIME_DIR"
# The Dot's browser (browser/dot-browser on this disk): the release an earlier start staged becomes the Dot's browser
# now, while nothing of it runs, and the engine started below reads what it serves. The switch is one rename, so the
# Dot's browser is either the one it had or the staged one; a switch that fails says why, and the next start tries
# again while the release stays staged.
"$runtime/browser/dot-browser" switch || log "error: the browser switch did not finish (above); the Dot starts with the browser its link names"
# A runtime before this one enabled the units: they would still start ahead of this script.
for unit in "${units[@]}"; do rm -f "/etc/systemd/system/multi-user.target.wants/$unit"; done
changed=()
for unit in "${units[@]}"; do
src="$runtime/units/$unit"
@@ -98,7 +116,6 @@ for unit in "${units[@]}"; do
done
systemctl daemon-reload
systemctl enable "${units[@]}" >/dev/null 2>&1
for unit in "${units[@]}"; do
if [[ " ${changed[*]-} " == *" $unit "* ]] && systemctl is-active --quiet "$unit"; then
log "restarting $unit (its unit file changed)"
@@ -109,3 +126,7 @@ for unit in "${units[@]}"; do
fi
done
log "runtime $version installed; units: ${units[*]}"
# A newer browser, when one is published, is built beside the Dot's in the background, through the Dot's own way out
# (its proxy, when it has one, is set above): a start never waits for it, and the next start switches to it.
systemd-run --quiet --no-block --collect --unit=dot-browser-stage "$runtime/browser/dot-browser" stage
@@ -1,7 +1,7 @@
# Owner's rule: the only browser of a Dot is invisible-playwright-mcp. No other browser, headless browser or browser
# library enters the guest image. One name per line; a package or apt package is refused when its name is one of
# these or starts with one of them followed by "-". Read by build-browser-env.sh, on what it actually installed, and
# by the tests, on the apt packages of pins.json.
# these or starts with one of them followed by "-". Read by dot-browser.sh, beside it, on what it actually installed,
# and by the tests, on the apt packages of pins.json.
chromium
chrome
google-chrome
+10 -6
View File
@@ -23,16 +23,20 @@ export const BUILDER_PROVISION = "builder/provision.sh";
export const BUILDER_ENGINE_LOCK = "builder/engine-requirements.lock";
/** Builds the engine's Python environment inside the builder VM; the same script builds it in a Linux container. */
export const BUILDER_ENGINE_BUILD = "builder/build-engine-env.sh";
/** Builds the Dot's browser (the MCP server's environment and the engine) inside the builder VM; the browser smoke runs the same script. */
export const BUILDER_BROWSER_BUILD = "builder/build-browser-env.sh";
/** The browsers that never enter the image (owner's rule): build-browser-env.sh refuses them among what it installs. */
export const BUILDER_OTHER_BROWSERS = "builder/other-browsers.txt";
export const RUNTIME_INSTALL = "runtime/install.sh";
export const RUNTIME_DESKTOP = "runtime/dot-desktop.sh";
export const RUNTIME_DOT_INSTALL = "runtime/dot-install.sh";
/** Writes the VM proxy's tunnel settings, routes and firewall; install.sh runs it when the Dot has a proxy. */
export const RUNTIME_VMPROXY = "runtime/vmproxy.py";
/** The guest units, in the order install.sh enables them. */
/**
* The Dot's browser and where it lives (the MCP server's environment and the engine): the runtime disk carries it for
* install.sh, which brings every Dot's browser to the newest release at its starts, and the builder seed for
* provision.sh, which installs the golden image's with it. The browser smoke runs it too.
*/
export const RUNTIME_BROWSER = "runtime/dot-browser.sh";
/** The browsers that never enter a Dot (owner's rule): dot-browser.sh refuses them among what it installs, reading them beside it. */
export const RUNTIME_OTHER_BROWSERS = "runtime/other-browsers.txt";
/** The guest units, in the order install.sh starts them. */
export const GUEST_UNITS = ["dot-desktop.service", "dot-agentd.service", "invisible-dots-agent.service"] as const;
export function unitAsset(name: (typeof GUEST_UNITS)[number]): string {
@@ -40,7 +44,7 @@ export function unitAsset(name: (typeof GUEST_UNITS)[number]): string {
}
/** Every guest file, for checks that apply to all of them. */
export const GUEST_ASSETS: readonly string[] = [BUILDER_USER_DATA, BUILDER_PROVISION, BUILDER_ENGINE_LOCK, BUILDER_ENGINE_BUILD, BUILDER_BROWSER_BUILD, BUILDER_OTHER_BROWSERS, RUNTIME_INSTALL, RUNTIME_DESKTOP, RUNTIME_DOT_INSTALL, RUNTIME_VMPROXY, ...GUEST_UNITS.map(unitAsset)];
export const GUEST_ASSETS: readonly string[] = [BUILDER_USER_DATA, BUILDER_PROVISION, BUILDER_ENGINE_LOCK, BUILDER_ENGINE_BUILD, RUNTIME_INSTALL, RUNTIME_DESKTOP, RUNTIME_DOT_INSTALL, RUNTIME_VMPROXY, RUNTIME_BROWSER, RUNTIME_OTHER_BROWSERS, ...GUEST_UNITS.map(unitAsset)];
/**
* Reads a guest file and refuses one a Windows checkout turned into CRLF:
+7 -7
View File
@@ -9,7 +9,7 @@ import { chmod, copyFile, mkdir, readFile, rm, stat } from "node:fs/promises";
import { basename, join } from "node:path";
import { writeIso } from "@invisible-dots/iso";
import { hostPaths, type HostPaths } from "@invisible-dots/shared";
import { BUILDER_BROWSER_BUILD, BUILDER_ENGINE_BUILD, BUILDER_ENGINE_LOCK, BUILDER_OTHER_BROWSERS, BUILDER_PROVISION, BUILDER_USER_DATA, defaultAssetRoot, readGuestAsset } from "./assets.js";
import { BUILDER_ENGINE_BUILD, BUILDER_ENGINE_LOCK, BUILDER_PROVISION, BUILDER_USER_DATA, defaultAssetRoot, readGuestAsset, RUNTIME_BROWSER, RUNTIME_OTHER_BROWSERS } from "./assets.js";
import { latestBrowserRelease, type BrowserRelease } from "./browser-release.js";
import { fetchVerified, sha256File, type Fetch, type FetchVerifiedOptions } from "./download.js";
import { acquireLock, type Lock } from "./lock.js";
@@ -105,8 +105,8 @@ export async function buildGoldenImage(options: GoldenBuildOptions): Promise<Gol
const userData = await readGuestAsset(assetRoot, BUILDER_USER_DATA);
const provision = await readGuestAsset(assetRoot, BUILDER_PROVISION);
const engineBuild = await readGuestAsset(assetRoot, BUILDER_ENGINE_BUILD);
const browserBuild = await readGuestAsset(assetRoot, BUILDER_BROWSER_BUILD);
const otherBrowsers = await readGuestAsset(assetRoot, BUILDER_OTHER_BROWSERS);
const browserScript = await readGuestAsset(assetRoot, RUNTIME_BROWSER);
const otherBrowsers = await readGuestAsset(assetRoot, RUNTIME_OTHER_BROWSERS);
const engineLock = await readGuestAsset(assetRoot, BUILDER_ENGINE_LOCK);
parseHashedLock(engineLock.toString("utf8"), BUILDER_ENGINE_LOCK);
// The browser is the newest release of invisible-playwright-mcp, asked of the index now: it is an input, so a new
@@ -117,7 +117,7 @@ export async function buildGoldenImage(options: GoldenBuildOptions): Promise<Gol
// The engine's lock is too, so a changed transitive dependency is another image, and
// so are the scripts that build the engine's environment and the browser. The engine's own
// source is not: it travels on the runtime disk (section 3.3).
const digest = inputsDigest([JSON.stringify(base), JSON.stringify(pins), userData, provision, JSON.stringify(browser), diskSize, engineLock, engineBuild, browserBuild, otherBrowsers]);
const digest = inputsDigest([JSON.stringify(base), JSON.stringify(pins), userData, provision, JSON.stringify(browser), diskSize, engineLock, engineBuild, browserScript, otherBrowsers]);
const { version, image, manifest, existing } = await resolveImageVersion({
kind: "golden",
@@ -148,7 +148,7 @@ export async function buildGoldenImage(options: GoldenBuildOptions): Promise<Gol
}
return await buildLocked(
{ ...options, log, paths, base, pins, diskSize, now, lock },
{ version, image, manifest, digest, userData, provision, browser, engineLock, engineBuild, browserBuild, otherBrowsers },
{ version, image, manifest, digest, userData, provision, browser, engineLock, engineBuild, browserScript, otherBrowsers },
);
} finally {
await lock.release();
@@ -176,7 +176,7 @@ interface Target {
browser: BrowserRelease;
engineLock: Buffer;
engineBuild: Buffer;
browserBuild: Buffer;
browserScript: Buffer;
otherBrowsers: Buffer;
}
@@ -225,7 +225,7 @@ async function buildLocked(options: GoldenBuildOptions & Resolved, target: Targe
tunnelBinary,
engineLock: target.engineLock,
engineBuild: target.engineBuild,
browserBuild: target.browserBuild,
browserScript: target.browserScript,
otherBrowsers: target.otherBrowsers,
}),
{ volumeId: SEED_VOLUME_ID, timestamp: options.now() },
+5
View File
@@ -33,9 +33,11 @@ import {
defaultAssetRoot,
GUEST_UNITS,
readGuestAsset,
RUNTIME_BROWSER,
RUNTIME_DESKTOP,
RUNTIME_DOT_INSTALL,
RUNTIME_INSTALL,
RUNTIME_OTHER_BROWSERS,
RUNTIME_VMPROXY,
unitAsset,
} from "./assets.js";
@@ -216,6 +218,9 @@ export async function runtimeFiles(inputs: RuntimeInputs, assetRoot: string = de
stageBytes("vmproxy.py", await readGuestAsset(assetRoot, RUNTIME_VMPROXY)),
stageBytes("bin/dot-desktop", await readGuestAsset(assetRoot, RUNTIME_DESKTOP)),
stageBytes("bin/dot-install", await readGuestAsset(assetRoot, RUNTIME_DOT_INSTALL)),
// The browser's script and, beside it where it reads it, the list of the browsers it refuses.
stageBytes("browser/dot-browser", await readGuestAsset(assetRoot, RUNTIME_BROWSER)),
stageBytes("browser/other-browsers.txt", await readGuestAsset(assetRoot, RUNTIME_OTHER_BROWSERS)),
await stageFile("bin/dot-agentd", inputs.agentdBinary),
...(await engineFiles(inputs.engineRoot)),
];
+7 -7
View File
@@ -12,9 +12,9 @@ import type { BrowserRelease } from "./browser-release.js";
/** The engine's lock and the script that builds its Python environment from it, on the builder seed. */
export const SEED_ENGINE_LOCK = "engine-requirements.lock";
export const SEED_ENGINE_BUILD = "build-engine-env.sh";
/** The script that builds the Dot's browser, on the builder seed. */
export const SEED_BROWSER_BUILD = "build-browser-env.sh";
/** The browsers build-browser-env.sh refuses, beside it on the seed, where it reads them. */
/** The runtime disk's script that installs the Dot's browser, on the builder seed (provision.sh runs it). */
export const SEED_BROWSER = "dot-browser.sh";
/** The browsers dot-browser.sh refuses, beside it on the seed, where it reads them. */
export const SEED_OTHER_BROWSERS = "other-browsers.txt";
export interface BuilderSeedInput {
@@ -31,8 +31,8 @@ export interface BuilderSeedInput {
/** builder/engine-requirements.lock and builder/build-engine-env.sh, as read. */
engineLock: Uint8Array;
engineBuild: Uint8Array;
/** builder/build-browser-env.sh and builder/other-browsers.txt, as read. */
browserBuild: Uint8Array;
/** runtime/dot-browser.sh and runtime/other-browsers.txt, as read. */
browserScript: Uint8Array;
otherBrowsers: Uint8Array;
}
@@ -53,7 +53,7 @@ export function pinsEnv(pins: GuestPins, browser: BrowserRelease): string {
["CORE_VERSION", browser.core],
["ENGINE_LOCK", SEED_ENGINE_LOCK],
["ENGINE_BUILD", SEED_ENGINE_BUILD],
["BROWSER_BUILD", SEED_BROWSER_BUILD],
["BROWSER", SEED_BROWSER],
["APT_SNAPSHOT", pins.apt_snapshot],
["APT_PACKAGES", pins.apt_packages.join(" ")],
];
@@ -80,7 +80,7 @@ export function builderSeedEntries(input: BuilderSeedInput): IsoEntry[] {
{ path: downloadFileName(input.pins.tunnel), file: input.tunnelBinary },
{ path: SEED_ENGINE_LOCK, data: input.engineLock },
{ path: SEED_ENGINE_BUILD, data: input.engineBuild },
{ path: SEED_BROWSER_BUILD, data: input.browserBuild },
{ path: SEED_BROWSER, data: input.browserScript },
{ path: SEED_OTHER_BROWSERS, data: input.otherBrowsers },
];
}
+5 -5
View File
@@ -4,7 +4,7 @@ import { tmpdir } from "node:os";
import { dirname, join } from "node:path";
import { hostPaths, type HostPaths } from "@invisible-dots/shared";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { BUILDER_BROWSER_BUILD, BUILDER_ENGINE_BUILD, BUILDER_ENGINE_LOCK, BUILDER_OTHER_BROWSERS, BUILDER_PROVISION, BUILDER_USER_DATA, defaultAssetRoot } from "../src/assets.js";
import { RUNTIME_BROWSER, BUILDER_ENGINE_BUILD, BUILDER_ENGINE_LOCK, RUNTIME_OTHER_BROWSERS, BUILDER_PROVISION, BUILDER_USER_DATA, defaultAssetRoot } from "../src/assets.js";
import type { BrowserRelease } from "../src/browser-release.js";
import { buildGoldenImage, GOLDEN_DEFAULTS, GoldenBuildError, type GoldenBuildOptions } from "../src/golden.js";
import { readManifest, verifyImage, type GoldenManifest } from "../src/manifest.js";
@@ -177,7 +177,7 @@ describe("buildGoldenImage", () => {
/** A copy of the guest files the digest reads, for a test that changes one of them. */
async function assetCopy(): Promise<string> {
const assetRoot = join(home, "assets");
for (const relative of [BUILDER_USER_DATA, BUILDER_PROVISION, BUILDER_ENGINE_LOCK, BUILDER_ENGINE_BUILD, BUILDER_BROWSER_BUILD, BUILDER_OTHER_BROWSERS]) {
for (const relative of [BUILDER_USER_DATA, BUILDER_PROVISION, BUILDER_ENGINE_LOCK, BUILDER_ENGINE_BUILD, RUNTIME_BROWSER, RUNTIME_OTHER_BROWSERS]) {
await mkdir(join(assetRoot, dirname(relative)), { recursive: true });
await copyFile(join(defaultAssetRoot(), relative), join(assetRoot, relative));
}
@@ -210,10 +210,10 @@ describe("buildGoldenImage", () => {
expect(second.version).not.toBe(first.version);
});
it("builds a new version when only the script that builds the browser changes", async () => {
it("builds a new version when only the script that installs the browser changes", async () => {
const first = await buildGoldenImage(options({ console: OK_CONSOLE, exit: 0 }).opts);
const assetRoot = await assetCopy();
const scriptPath = join(assetRoot, BUILDER_BROWSER_BUILD);
const scriptPath = join(assetRoot, RUNTIME_BROWSER);
await writeFile(scriptPath, `${await readFile(scriptPath, "utf8")}# changed\n`);
const second = await buildGoldenImage(options({ console: OK_CONSOLE, exit: 0 }, { assetRoot }).opts);
@@ -224,7 +224,7 @@ describe("buildGoldenImage", () => {
it("builds a new version when only the list of browsers the image refuses changes", async () => {
const first = await buildGoldenImage(options({ console: OK_CONSOLE, exit: 0 }).opts);
const assetRoot = await assetCopy();
const listPath = join(assetRoot, BUILDER_OTHER_BROWSERS);
const listPath = join(assetRoot, RUNTIME_OTHER_BROWSERS);
await writeFile(listPath, `${await readFile(listPath, "utf8")}servo\n`);
const second = await buildGoldenImage(options({ console: OK_CONSOLE, exit: 0 }, { assetRoot }).opts);
+50 -18
View File
@@ -7,7 +7,7 @@ import { tmpdir } from "node:os";
import { delimiter, join } from "node:path";
import { GUEST_PATHS } from "@invisible-dots/shared";
import { describe, expect, it } from "vitest";
import { BUILDER_OTHER_BROWSERS, defaultAssetRoot, GUEST_ASSETS, GUEST_UNITS, unitAsset } from "../src/assets.js";
import { defaultAssetRoot, GUEST_ASSETS, GUEST_UNITS, RUNTIME_OTHER_BROWSERS, unitAsset } from "../src/assets.js";
import { GUEST_PINS } from "../src/pins.js";
const root = defaultAssetRoot();
@@ -31,7 +31,7 @@ const bashUsable = bashCanRead(join(root, "runtime", "dot-desktop.sh"));
describe("guest files", () => {
it("lists every guest script", () => {
expect([...scripts].sort()).toEqual(["builder/build-browser-env.sh", "builder/build-engine-env.sh", "builder/provision.sh", "runtime/dot-desktop.sh", "runtime/dot-install.sh", "runtime/install.sh"]);
expect([...scripts].sort()).toEqual(["builder/build-engine-env.sh", "builder/provision.sh", "runtime/dot-browser.sh", "runtime/dot-desktop.sh", "runtime/dot-install.sh", "runtime/install.sh"]);
});
it.each(GUEST_ASSETS)("%s is LF-only ASCII and says nothing about vsock or libvirt", (path) => {
@@ -172,21 +172,43 @@ describe.skipIf(!bashUsable)("dot-install, the one command dot may run as root",
describe("the Dot's browser", () => {
const provision = text("builder/provision.sh");
const build = text("builder/build-browser-env.sh");
const build = text("runtime/dot-browser.sh");
const install = text("runtime/install.sh");
it("is the release the build resolved, installed exactly by the script the seed carries", () => {
expect(provision).toContain('bash "$payload/$BROWSER_BUILD" "$mcp_env" \\\n');
expect(provision).toContain('"invisible-playwright-mcp==$MCP_VERSION" "invisible-playwright==$PLAYWRIGHT_VERSION" "invisible-core==$CORE_VERSION"');
expect(provision).toContain("mcp_env=/home/dot/.local/share/invisible-dots/mcp\n");
it("is the release the build resolved, installed exactly by the runtime disk's own script, which the seed carries", () => {
expect(provision).toContain('bash "$payload/$BROWSER" install "$MCP_VERSION" "$PLAYWRIGHT_VERSION" "$CORE_VERSION"\n');
// What provision.sh records is read from the environment the server's link names.
expect(provision).toContain('mcp_env="$(dirname "$(dirname "$(readlink -f /home/dot/.local/bin/invisible-playwright-mcp)")")"\n');
expect(build).toContain('local requirements=("${packages[0]}==$2" "${packages[1]}==$3" "${packages[2]}==$4")');
expect(build).toContain('as_dot uv venv --quiet --python /usr/bin/python3 "$env_dir"');
// Wheels only, the requirements given or the newest server.
// Wheels only, the release given.
expect(build).toContain('as_dot uv pip install --no-cache --python "$env_dir/bin/python" --only-binary :all: "${requirements[@]}"');
expect(build).toContain('[ "${#requirements[@]}" -gt 0 ] || requirements=(invisible-playwright-mcp)');
expect(build).not.toMatch(/uv tool install/);
expect(build).toContain('as_dot ln -sfn "$env_dir/bin/invisible-playwright-mcp" "$bin_dir/invisible-playwright-mcp"');
expect(build).toContain('as_dot "$env_dir/bin/invisible-playwright" fetch');
});
it("keeps each release in an environment of its own and switches the server's link in one rename", () => {
expect(build).toContain("releases=$share/browser\n");
expect(build).toContain('local env_dir="$releases/$1-$2-$3"');
expect(build).toContain('env_dir="$releases/$mcp-$playwright-$core"');
expect(build).toContain('as_dot ln -sfn "$1/bin/invisible-playwright-mcp" "$command_link.new"\n as_dot mv -T "$command_link.new" "$command_link"');
// A release is staged only once it is whole: the link to it is the last step of a stage.
expect(build).toMatch(/build "\$env_dir" "\$mcp" "\$playwright" "\$core"\n as_dot ln -sfn "\$env_dir" "\$staged\.new"\n as_dot mv -T "\$staged\.new" "\$staged"/);
// The newest is uv's own resolution of the server, not a version written anywhere.
expect(build).toContain("as_dot uv pip compile - --quiet --no-cache --no-header --no-annotate");
});
it("is brought to the newest release at a Dot's starts: switched before the units start, staged after them", () => {
const proxy = install.indexOf('nft -f "$proxy_dir/allow.nft"');
const switched = install.indexOf('"$runtime/browser/dot-browser" switch');
const started = install.indexOf('systemctl start "$unit"');
const staged = install.indexOf('systemd-run --quiet --no-block --collect --unit=dot-browser-stage "$runtime/browser/dot-browser" stage');
expect(proxy).toBeGreaterThan(0);
expect(switched).toBeGreaterThan(proxy);
expect(started).toBeGreaterThan(switched);
expect(staged).toBeGreaterThan(started);
});
it("records the versions the environment holds, read from it, not the ones it was asked for", () => {
expect(provision).toContain("installed_version() { as_dot \"$mcp_env/bin/python\" -c 'import sys, importlib.metadata as m; print(m.version(sys.argv[1]))' \"$1\"; }");
for (const name of ["invisible-playwright-mcp", "invisible-playwright", "invisible-core"]) {
@@ -197,9 +219,9 @@ describe("the Dot's browser", () => {
/**
* Owner's rule: the only browser of a Dot is invisible-playwright-mcp. No other browser, headless browser or browser
* library enters the guest image: not as an apt package, not among the Python packages the browser's environment
* installs (build-browser-env.sh refuses them, reading the same list), not as a command of the scripts that build it.
* installs (dot-browser.sh refuses them, reading the same list), not as a command of the scripts that build it.
*/
const otherBrowsers = text(BUILDER_OTHER_BROWSERS)
const otherBrowsers = text(RUNTIME_OTHER_BROWSERS)
.split("\n")
.map((line) => line.trim())
.filter((line) => line && !line.startsWith("#"));
@@ -207,10 +229,10 @@ describe("the Dot's browser", () => {
it("installs no other browser, headless browser or browser library", () => {
expect(GUEST_PINS.apt_packages.filter(isOtherBrowser)).toEqual([]);
expect(build).toContain('other_browsers="$(dirname "${BASH_SOURCE[0]}")/other-browsers.txt"');
expect(build).toContain('other_browsers="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")/other-browsers.txt"');
expect(build).toContain('installed="$(as_dot uv pip freeze --python "$env_dir/bin/python"');
expect(build).toContain("another browser would enter the image (owner's rule)");
for (const [name, content] of [["provision.sh", provision], ["build-browser-env.sh", build]] as const) {
expect(build).toContain("another browser would enter the Dot (owner's rule)");
for (const [name, content] of [["provision.sh", provision], ["dot-browser.sh", build]] as const) {
expect(content, name).not.toMatch(/\b(chromium|google-chrome|selenium|puppeteer|patchright|chromedriver|geckodriver)\b/i);
expect(content, name).not.toMatch(/\bplaywright install\b/);
}
@@ -247,7 +269,16 @@ describe("guest units", () => {
expect(unit).toMatch(/^Environment=DISPLAY=:0$/m);
expect(unit).toMatch(/^RequiresMountsFor=\/opt\/invisible-dots$/m);
expect(unit).toMatch(/^Restart=on-failure$/m);
expect(unit).toMatch(/^WantedBy=multi-user\.target$/m);
// Not enabled: install.sh starts it, after the Dot's proxy and browser (the test below).
expect(unit).not.toMatch(/^\[Install\]$/m);
});
it("are started by install.sh alone, which also takes back what an earlier runtime enabled", () => {
// Enabled, systemd would start them at multi-user.target on every boot after the first, ahead of cloud-init's final
// stage that runs install.sh (cloud-final.service is After=multi-user.target): before the proxy and the browser.
const install = text("runtime/install.sh");
expect(install).not.toMatch(/systemctl enable/);
expect(install).toContain('for unit in "${units[@]}"; do rm -f "/etc/systemd/system/multi-user.target.wants/$unit"; done');
});
it("runs the desktop, which is dot's, with uv's bin dir first", () => {
@@ -256,7 +287,7 @@ describe("guest units", () => {
it("runs the engine, which holds the key, with a PATH of system directories only, and names the browser's server by its path", () => {
// Nothing the engine runs by name may come from a directory dot (the model) can write. The browser's MCP server
// is dot's (build-browser-env.sh links it into ~dot/.local/bin): the unit names it, and the engine starts it as dot.
// is dot's (dot-browser.sh links it into ~dot/.local/bin): the unit names it, and the engine starts it as dot.
for (const dir of unitPath("invisible-dots-agent.service")) expect(dir, `PATH entry ${dir}`).toMatch(/^\/(usr\/local\/|usr\/)?s?bin$/);
expect(text(unitAsset("invisible-dots-agent.service"))).toMatch(/^Environment=INVISIBLE_DOTS_MCP_COMMAND=\/home\/dot\/\.local\/bin\/invisible-playwright-mcp$/m);
});
@@ -267,7 +298,8 @@ describe("guest units", () => {
expect(unit).toMatch(/^Environment=DISPLAY=:0$/m);
expect(unit).toMatch(/^RequiresMountsFor=\/opt\/invisible-dots$/m);
expect(unit).toMatch(/^Restart=on-failure$/m);
expect(unit).toMatch(/^WantedBy=multi-user\.target$/m);
// Not enabled: install.sh starts it, after the Dot's proxy and browser (the test below).
expect(unit).not.toMatch(/^\[Install\]$/m);
// The daemon starts the poweroff as itself, and may become root through it: nothing it finds may be in a
// directory dot (the model) can write, so its PATH has no entry of /home (dot's ~/.local/bin included) and
// none that is relative or empty. The PATH of the model's commands is the daemon's to build.
+5
View File
@@ -115,6 +115,8 @@ describe("buildRuntimeIso", () => {
"bin/dot-agentd",
"bin/dot-desktop",
"bin/dot-install",
"browser/dot-browser",
"browser/other-browsers.txt",
"engine/LICENSE",
"engine/UPSTREAM.md",
"engine/nanobot/__init__.py",
@@ -132,6 +134,9 @@ describe("buildRuntimeIso", () => {
expect(read("install.sh")).toEqual(await readFile(join(defaultAssetRoot(), "runtime", "install.sh")));
expect(read("vmproxy.py")).toEqual(await readFile(join(defaultAssetRoot(), "runtime", "vmproxy.py")));
expect(read("bin/dot-desktop")).toEqual(await readFile(join(defaultAssetRoot(), "runtime", "dot-desktop.sh")));
// The browser's script with its list beside it, where it reads it: install.sh switches and stages with them.
expect(read("browser/dot-browser")).toEqual(await readFile(join(defaultAssetRoot(), "runtime", "dot-browser.sh")));
expect(read("browser/other-browsers.txt")).toEqual(await readFile(join(defaultAssetRoot(), "runtime", "other-browsers.txt")));
expect(read("units/dot-agentd.service")).toEqual(await readFile(join(defaultAssetRoot(), "units", "dot-agentd.service")));
// The engine's source as it is.
expect(read("engine/nanobot/dots/main.py").toString()).toBe("def main(): ...\n");
+9 -8
View File
@@ -3,7 +3,7 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import { writeIso } from "@invisible-dots/iso";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { BUILDER_BROWSER_BUILD, BUILDER_ENGINE_BUILD, BUILDER_ENGINE_LOCK, BUILDER_OTHER_BROWSERS, BUILDER_PROVISION, BUILDER_USER_DATA, defaultAssetRoot, readGuestAsset } from "../src/assets.js";
import { RUNTIME_BROWSER, BUILDER_ENGINE_BUILD, BUILDER_ENGINE_LOCK, RUNTIME_OTHER_BROWSERS, BUILDER_PROVISION, BUILDER_USER_DATA, defaultAssetRoot, readGuestAsset } from "../src/assets.js";
import type { BrowserRelease } from "../src/browser-release.js";
import { GUEST_PINS } from "../src/pins.js";
import { SEED_VOLUME_ID } from "@invisible-dots/vm-manager";
@@ -31,7 +31,7 @@ describe("pins.env", () => {
expect(text).toContain("CORE_VERSION='47.39.0'\n");
expect(text).toContain("ENGINE_LOCK='engine-requirements.lock'\n");
expect(text).toContain("ENGINE_BUILD='build-engine-env.sh'\n");
expect(text).toContain("BROWSER_BUILD='build-browser-env.sh'\n");
expect(text).toContain("BROWSER='dot-browser.sh'\n");
expect(text).toContain(`APT_SNAPSHOT='${GUEST_PINS.apt_snapshot}'\n`);
expect(text).toContain(`APT_PACKAGES='${GUEST_PINS.apt_packages.join(" ")}'\n`);
@@ -55,8 +55,8 @@ describe("the builder seed", () => {
await writeFile(tunnelBinary, "hev bytes");
const engineLock = await readGuestAsset(defaultAssetRoot(), BUILDER_ENGINE_LOCK);
const engineBuild = await readGuestAsset(defaultAssetRoot(), BUILDER_ENGINE_BUILD);
const browserBuild = await readGuestAsset(defaultAssetRoot(), BUILDER_BROWSER_BUILD);
const otherBrowsers = await readGuestAsset(defaultAssetRoot(), BUILDER_OTHER_BROWSERS);
const browserScript = await readGuestAsset(defaultAssetRoot(), RUNTIME_BROWSER);
const otherBrowsers = await readGuestAsset(defaultAssetRoot(), RUNTIME_OTHER_BROWSERS);
const userData = await readGuestAsset(defaultAssetRoot(), BUILDER_USER_DATA);
const provision = await readGuestAsset(defaultAssetRoot(), BUILDER_PROVISION);
const entries = builderSeedEntries({
@@ -69,7 +69,7 @@ describe("the builder seed", () => {
tunnelBinary,
engineLock,
engineBuild,
browserBuild,
browserScript,
otherBrowsers,
});
@@ -82,7 +82,7 @@ describe("the builder seed", () => {
"hev-socks5-tunnel-linux-x86_64",
"engine-requirements.lock",
"build-engine-env.sh",
"build-browser-env.sh",
"dot-browser.sh",
"other-browsers.txt",
]);
expect(entries[0]).toEqual({ path: "user-data", data: userData });
@@ -90,8 +90,9 @@ describe("the builder seed", () => {
expect(entries[5]).toEqual({ path: "hev-socks5-tunnel-linux-x86_64", file: tunnelBinary });
expect(entries[6]).toEqual({ path: "engine-requirements.lock", data: engineLock });
expect(entries[7]).toEqual({ path: "build-engine-env.sh", data: engineBuild });
expect(entries[8]).toEqual({ path: "build-browser-env.sh", data: browserBuild });
// Beside the script on the seed, where it reads the list (build-browser-env.sh).
// The runtime disk's script: provision.sh installs the golden image's browser with the one every Dot updates its own with.
expect(entries[8]).toEqual({ path: "dot-browser.sh", data: browserScript });
// Beside the script on the seed, where it reads the list (dot-browser.sh).
expect(entries[9]).toEqual({ path: "other-browsers.txt", data: otherBrowsers });
// The real writer accepts it under the label NoCloud looks for.
+7 -3
View File
@@ -85,15 +85,19 @@ The real server and a real Firefox are not run by this smoke: they are the brows
## The browser smoke
`browser/smoke.sh` runs the Dot's real browser. The golden image's apt packages (`pins.json`:
the desktop, Firefox's libraries, ImageMagick) are installed, the browser is built by
`builder/build-browser-env.sh` at the newest release (the MCP server's environment and the
browser engine: the script `provision.sh` runs), the
the desktop, Firefox's libraries, ImageMagick) are installed, the browser is installed by
`runtime/dot-browser.sh` (the script `provision.sh` runs and `install.sh` updates every Dot with) on
the release before the newest, and the first checks bring it to the newest as a Dot's starts do:
a stage builds the newest beside it, a switch makes it the Dot's browser and removes the old one; the
runtime disk's `dot-desktop` script starts Xvfb and an XFCE session on `:0` as dot, and the Dot
runs as in the engine smoke with no stand-in for the browser: the engine finds
`invisible-playwright-mcp` on its PATH. Only the model is a stand-in, and the pages the browser
opens are served from the container. The browser needs the network twice, at the build (the
engine) and at a launch (the egress address, for the timezone, and the library's GeoIP database). What it pins:
- a Dot on the release before the newest is brought to the newest as its starts bring it: a stage builds the newest
beside its browser and changes nothing else, a second stage builds nothing, the switch makes the newest the Dot's
browser and removes the old one, and on the newest a stage and a switch change nothing;
- the model launches an identity and the real server answers that its browser is open; Firefox
and the server run as `dot`, the server with the profile, its home, a real window on `:0`, no
proxy variable (an identity with no proxy of its own, the default, inherits the VM's egress) and
@@ -9,9 +9,10 @@
# only the real server can show is here: a page read, a screenshot that is a real picture and
# crosses the relay whole, a profile that keeps its seed, a profile lock left by kill -9.
#
# prepare-engine.sh has built the engine's environment and, for this suite, the browser
# (builder/build-browser-env.sh) and the golden image's apt packages. Exits non-zero when a check
# failed or was skipped, and always prints
# prepare-engine.sh has built the engine's environment and, for this suite, the golden image's apt
# packages and the release of the browser before the newest (runtime/dot-browser.sh), which the
# first checks bring to the newest as a Dot's starts do. Exits non-zero when a check failed or was
# skipped, and always prints
# SMOKE: <passed> passed, <failed> failed, <skipped> skipped
HERE=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
set -uo pipefail
@@ -20,6 +21,25 @@ RUNTIME=$HERE/../../../runtime
# A real launch downloads nothing but may wait for the network: events are waited for longer here.
WAIT_EVENT_S=240
# --- the Dot's browser brought to the newest release, as install.sh does at a Dot's starts ---
# One start stages the newest release beside the Dot's (dot-browser stage, in the background there), the next one
# switches to it before the units start (dot-browser switch). Everything below runs on what the switch left.
DOT_BROWSER=$RUNTIME/dot-browser.sh
BROWSERS=/home/dot/.local/share/invisible-dots/browser
server_env() { dirname "$(dirname "$(readlink -f /home/dot/.local/bin/invisible-playwright-mcp)")"; }
release_in() { su -s /bin/sh dot -c "$1/bin/python -c 'import importlib.metadata as m; print(m.version(\"invisible-playwright-mcp\"), m.version(\"invisible-playwright\"), m.version(\"invisible-core\"))'"; }
NEWEST=$(bash "$DOT_BROWSER" newest)
BEFORE_ENV=$(server_env)
check "the Dot starts on the release before the newest" "[ \"\$(release_in \"$BEFORE_ENV\")\" != \"$NEWEST\" ]"
check "a stage builds the newest beside it and leaves the Dot's browser as it was until a switch" \
"bash $DOT_BROWSER stage && [ -L $BROWSERS/next ] && [ \"\$(server_env)\" = \"$BEFORE_ENV\" ] && [ \"\$(release_in \$(readlink -f $BROWSERS/next))\" = \"$NEWEST\" ]"
check "a second stage finds the newest staged already and builds nothing" \
"bash $DOT_BROWSER stage 2>&1 | grep -q 'staged already' && [ \$(ls -d $BROWSERS/*/ | wc -l) = 3 ]"
check "the switch makes the newest the Dot's browser, removes the release before it and leaves nothing staged" \
"bash $DOT_BROWSER switch && [ \"\$(release_in \$(server_env))\" = \"$NEWEST\" ] && [ ! -e $BROWSERS/next ] && [ ! -e \"$BEFORE_ENV\" ] && [ \$(ls -d $BROWSERS/*/ | wc -l) = 1 ]"
check "a stage on the newest builds nothing, and a switch with nothing staged changes nothing" \
"bash $DOT_BROWSER stage 2>&1 | grep -q 'is the newest release' && bash $DOT_BROWSER switch && [ ! -e $BROWSERS/next ] && [ \"\$(release_in \$(server_env))\" = \"$NEWEST\" ]"
lay_out_guest
write_host_token
start_fake_openrouter
@@ -16,8 +16,8 @@
# LICENSE, UPSTREAM.md), world-readable as on the ISO
# The browser suite builds one more thing, as the golden image does it: the apt packages of
# pins.json (the desktop, Firefox's libraries, ImageMagick) and the Dot's browser
# (builder/build-browser-env.sh: the newest invisible-playwright-mcp's environment and
# the engine of the browser).
# (runtime/dot-browser.sh: the release before the newest, which browser/smoke.sh brings to
# the newest as a Dot's start does).
set -euo pipefail
: "${TREE:?run.sh sets TREE}" "${AGENTD_BIN:?run.sh sets AGENTD_BIN}"
suite=${SMOKE_SUITE:-engine}
@@ -79,9 +79,16 @@ if [ "$suite" = browser ]; then
snapshot=$(jq -r '.apt_snapshot' "$TREE/guest/image-builder/pins.json")
apt-get --snapshot "$snapshot" --error-on=any update -qq >/dev/null
apt-get --snapshot "$snapshot" install -y -qq --no-install-recommends $(jq -r '.apt_packages[]' "$TREE/guest/image-builder/pins.json") >/dev/null
# The Dot's browser, with the script provision.sh runs: the newest release, as the next image installs it. dot
# The Dot's browser, with the runtime disk's script, which provision.sh runs too: the release before the newest, on
# the newest invisible-playwright and invisible-core (so the browser engine is downloaded once). browser/smoke.sh then
# stages the newest and switches to it, as a Dot's starts do, and its checks run on what the switch left. dot
# reaches its home, not the tree.
bash "$builder/build-browser-env.sh" /home/dot/.local/share/invisible-dots/mcp
browser=$TREE/guest/image-builder/runtime/dot-browser.sh
newest=$(bash "$browser" newest)
read -r mcp playwright core <<< "$newest"
before=$(bash "$browser" newest "invisible-playwright-mcp<$mcp" "invisible-playwright==$playwright" "invisible-core==$core")
read -r before_mcp before_playwright before_core <<< "$before"
bash "$browser" install "$before_mcp" "$before_playwright" "$before_core"
fi
exec bash "$checks"
@@ -32,6 +32,3 @@ Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ExecStart=/opt/invisible-dots/bin/dot-agentd
Restart=on-failure
RestartSec=2
[Install]
WantedBy=multi-user.target
@@ -17,6 +17,3 @@ Restart=on-failure
RestartSec=2
KillMode=control-group
TimeoutStopSec=15
[Install]
WantedBy=multi-user.target
@@ -44,6 +44,3 @@ ExecStart=/opt/invisible-dots-engine/bin/python -I -B -m nanobot
Restart=on-failure
RestartSec=2
TimeoutStopSec=30
[Install]
WantedBy=multi-user.target
+9 -3
View File
@@ -105,14 +105,20 @@ describe("the engine smoke", () => {
expect(read(join(smoke, "prepare-engine.sh"))).toContain("browser/smoke.sh");
});
it("builds the browser smoke's browser with the image builder's own script and apt packages, the newest release, and runs the real server", () => {
it("builds the browser smoke's browser with the runtime disk's own script and the image's apt packages, updates it to the newest as a Dot does, and runs the real server", () => {
const prepare = read(join(smoke, "prepare-engine.sh"));
expect(prepare).toContain('bash "$builder/build-browser-env.sh" /home/dot/.local/share/invisible-dots/mcp\n');
// The script provision.sh runs too, and install.sh at every start of a Dot.
expect(prepare).toContain("browser=$TREE/guest/image-builder/runtime/dot-browser.sh\n");
expect(prepare).toContain('bash "$browser" install "$before_mcp" "$before_playwright" "$before_core"');
expect(prepare).toContain(".apt_packages[]");
const provision = read(join(repo, "guest/image-builder/builder/provision.sh"));
expect(provision).toContain("BROWSER_BUILD");
expect(provision).toContain('bash "$payload/$BROWSER" install');
const checks = code(read(join(smoke, "browser/smoke.sh")));
// The update a Dot's starts make: a stage, then a switch, before anything else runs.
expect(checks.indexOf("bash $DOT_BROWSER stage")).toBeGreaterThan(0);
expect(checks.indexOf("bash $DOT_BROWSER switch")).toBeGreaterThan(checks.indexOf("bash $DOT_BROWSER stage"));
expect(checks.indexOf("bash $DOT_BROWSER switch")).toBeLessThan(checks.indexOf("lay_out_guest"));
// No stand-in for the browser: the engine runs what is on its PATH, in front of the real desktop script.
expect(checks).toContain("unset MCP_COMMAND");
expect(checks).not.toMatch(/fake_mcp|MCP_COMMAND=/);