mirror of
https://github.com/feder-cr/Jobs_Applier_AI_Agent.git
synced 2026-11-07 17:10:50 -10:00
engine, shared, image-builder: the MCP server's home is outside /home/dot, so no file route serves its session file (integ1)
Finding (medium), confirmed and fixed: the pinned invisible-playwright-mcp saves who `main` is, the proxy with its password included, in <INVISIBLE_MCP_HOME>/sessions/<session id>.json on every browser_open. The engine set INVISIBLE_MCP_HOME to /home/dot/browsers/<id>/mcp, so with the host file routes of lane-host (GET /api/dots/:id/files and /files/list, limited to /home/dot, with no exclusion for that directory) the password was one request away from the API, the web proxy, the SDK and the Files tab. Fix at the origin: the home is /var/lib/invisible-dots/mcp/<identity_id>, owned by dot, made by provision.sh and install.sh (the parent is root's). dot-agentd's TCP listener already refuses a real path outside home with 403 outside_home, so nothing new is filtered. One owner for the path: GUEST_PATHS.mcpHomes in packages/shared and MCP_HOMES_DIR in the engine's protocol.py, kept equal by tests/repo/vendored-nanobot.test.ts. The BrowserManager makes the subdirectory with the profile and removes it with the identity (before, it went with the identity's directory). Tests that fail before the fix: the manager tests (the home is not under browsers, the environment names it, a delete removes it, and the session file with the password is under the home and nowhere in the browsers directory), the shared test that checkHomePath refuses the home, the image-builder test that both scripts make it. The engine's fake MCP server now writes the session file as the real server does. The engine smoke has the stand-in's session file refused by the TCP port (the file, its directory, a link to it, a ../ path) and removed by a delete; the browser smoke creates an identity with a proxy (a small authenticating proxy, browser/proxy.py), launches it on the real server, and checks the real session file is under the new home, that no file of /home/dot holds the password, that the TCP port refuses it and that the identity's directory has no MCP home. files_get, files_list and refuses_outside_home move to lib.sh, which both smokes use. docs/architecture.md: sections 4.2, 6 and 9.6 name the session file as the second copy of the proxy on disk, outside /home/dot, and the new directory. Floors raised: vitest +2 (linux 1118, win32 1121), pytest +2 (2439); none lowered. No test of kept behavior was weakened or removed: the existing tests that read the server's record now read it from its new home.
This commit is contained in:
1 parent
1932f16069
commit
a258d512bb
20 files changed
+296
-54
No files matched your search
@@ -1,8 +1,8 @@
|
||||
{
|
||||
"//": "What each suite must reach on each host, read by .github/scripts/test-guard.mjs in CI (tests.yml) and in the pre-push hook. Raise a floor when tests are added; lowering one is a decision to write down in the commit that does it. Linux skips only the tests that drive a real Windows PowerShell; a Windows host compiles dot-agentd's unix-only test files out and skips the two Go tests that need a Linux guest. The pytest floor is the Python engine's (invisible_engine_dots, the engine job of tests.yml, Linux only, nothing may skip).",
|
||||
"vitest": {
|
||||
"linux": { "min_passed": 1116, "allow_skipped": "^generated PowerShell on a real Windows PowerShell " },
|
||||
"win32": { "min_passed": 1119, "allow_skipped": "" }
|
||||
"linux": { "min_passed": 1118, "allow_skipped": "^generated PowerShell on a real Windows PowerShell " },
|
||||
"win32": { "min_passed": 1121, "allow_skipped": "" }
|
||||
},
|
||||
"postgres": {
|
||||
"linux": { "min_passed": 633, "allow_skipped": "", "require": { "\\bpg\\b": 253 } }
|
||||
@@ -12,6 +12,6 @@
|
||||
"win32": { "min_passed": 64, "allow_skipped": "^agentd (TestSystemRoute|TestPowerOffStartsTheCommandAndAnswers202)$" }
|
||||
},
|
||||
"pytest": {
|
||||
"linux": { "min_passed": 2437, "allow_skipped": "" }
|
||||
"linux": { "min_passed": 2439, "allow_skipped": "" }
|
||||
}
|
||||
}
|
||||
+23
-5
@@ -476,13 +476,23 @@ because cloud-init adds no group to a user that exists already.
|
||||
memory/ long-term memory notes the Dot writes itself (files; section 8.6)
|
||||
browsers/<identity_id>/
|
||||
profile/ the browser profile
|
||||
mcp/ INVISIBLE_MCP_HOME for that identity's server
|
||||
/var/lib/invisible-dots/ root, 0755
|
||||
mcp/<identity_id>/ INVISIBLE_MCP_HOME for that identity's server (dot, 0700; outside /home/dot on purpose)
|
||||
/run/invisible-dots/ dot:dotengine 2750
|
||||
agentd.sock dot-agentd's local API for the engine (dot:dotengine 0660)
|
||||
/run/invisible-dots-agent/ dotengine:dot 2750
|
||||
agent.sock the engine's API, reached by dot-agentd's proxy (dotengine:dot 0660)
|
||||
```
|
||||
|
||||
The home of an identity's MCP server is not under `/home/dot` because the
|
||||
server saves who its browser is, the proxy with its password included, in
|
||||
`<home>/sessions/<identity_id>.json` on every `browser_open`: a file under
|
||||
`/home/dot` would be served by the host's file routes (section 9.6), which read
|
||||
`/home/dot` and nothing else, and dot-agentd's TCP listener refuses a path
|
||||
outside it (`403 outside_home`). The directory is made by the image's
|
||||
provisioner and by `install.sh` (dot owns it, the server runs as dot) and the
|
||||
`BrowserManager` makes and removes one subdirectory per identity.
|
||||
|
||||
Each socket sits in a directory its server owns and only the other side may
|
||||
enter, setgid so the socket takes that side's group (`install.sh` writes both
|
||||
to tmpfiles). `dot` cannot write `/run/invisible-dots-agent/`, so nothing of
|
||||
@@ -832,7 +842,13 @@ ignores anything else.
|
||||
the password replaced. The browser's own process is the one exception, by
|
||||
decision: it runs as `dot` with the proxy in its environment, so the `dot`
|
||||
user, and the model through `exec`, can read the proxy of an identity whose
|
||||
browser is open from that process's `/proc/<pid>/environ`. The proxy is never
|
||||
browser is open from that process's `/proc/<pid>/environ`. The server's
|
||||
session file is the second copy on disk, again by the server's own design
|
||||
(it saves who its browser is, the proxy included, in
|
||||
`/var/lib/invisible-dots/mcp/<identity_id>/sessions/<identity_id>.json`, as
|
||||
`dot`, after every `browser_open`): the model's `exec` can read it, but it is
|
||||
outside `/home/dot`, so the host's file routes (the API, the SDK, the Files
|
||||
tab) can never serve it; deleting the identity removes it. The proxy is never
|
||||
on a command line, which every user of the VM can read: the engine tells the
|
||||
relay the variable's name (`--env-from`) and the relay reads the value from
|
||||
its own environment, which only `dotengine` can read.
|
||||
@@ -842,7 +858,7 @@ ignores anything else.
|
||||
launch of the same profile gets the same one back.
|
||||
- Launching an identity starts one `invisible-playwright-mcp` process over
|
||||
stdio. Its environment is the allowlist of section 4.1 plus, through the MCP
|
||||
server's documented settings: its home (`INVISIBLE_MCP_HOME=<identity>/mcp`)
|
||||
server's documented settings: its home (`INVISIBLE_MCP_HOME=/var/lib/invisible-dots/mcp/<identity_id>`)
|
||||
and session id (`INVISIBLE_MCP_SESSION_ID=<identity_id>`), the identity's
|
||||
profile directory (`<identity>/profile`), headed mode, `DISPLAY=:0`, and the
|
||||
identity's proxy when it has one. The names of the browser layer's own
|
||||
@@ -1342,7 +1358,8 @@ state.
|
||||
registry no turn sees.
|
||||
- Browser identities. `BrowserManager` (`nanobot/dots/browser.py`) owns the
|
||||
identities (their rows in `dots_browser_identities`, their directories under
|
||||
`/home/dot/browsers`, made and removed as dot through the Computer) and one
|
||||
`/home/dot/browsers` and their servers' homes under `/var/lib/invisible-dots/mcp`,
|
||||
made and removed as dot through the Computer) and one
|
||||
`invisible-playwright-mcp` process per open identity, started as dot through
|
||||
`dot-agentd relay` with the environment of section 6 and nanobot's MCP client
|
||||
on a registry of its own. Launch, close and delete run one at a time; calls on
|
||||
@@ -1698,7 +1715,8 @@ only the early answer; the rule is dot-agentd's: its TCP listener (the host's
|
||||
door) resolves every path to its real location with the symbolic links followed
|
||||
and refuses one that is not under home, `403 outside_home`, which passes through
|
||||
(a link under home to `/proc/<pid>/environ`, where the browser server runs as `dot`
|
||||
with the proxy password in its environment, shows nothing). The same goes for the
|
||||
with the proxy password in its environment, shows nothing; the session file the
|
||||
server saves the proxy in is not under home at all, section 4.2). The same goes for the
|
||||
`PUT` and the listing of that listener; a link that stays in home works, one that
|
||||
leaves it lists as `other`. The engine's socket is not confined. A read is one buffered answer of at most 16 MiB; a larger
|
||||
file is a `413 file_too_large` and the host stops reading it as soon as it passes
|
||||
|
||||
@@ -91,6 +91,11 @@ install -d -o dot -g dot -m 2775 /home/dot/workspace
|
||||
install -d -o dot -g dot -m 0755 /home/dot/downloads /home/dot/documents /home/dot/memory
|
||||
install -d -o dotengine -g dotengine -m 0700 /home/dotengine
|
||||
install -d -o dot -g dot -m 0700 /home/dot/browsers
|
||||
# The home of each browser identity's MCP server (architecture 4.2). It is outside /home/dot on purpose: the
|
||||
# server saves the proxy of its browser, password included, in a session file under its home, and the
|
||||
# host's file routes read /home/dot and nothing else. dot owns it: the server runs as dot.
|
||||
install -d -o root -g root -m 0755 /var/lib/invisible-dots
|
||||
install -d -o dot -g dot -m 0700 /var/lib/invisible-dots/mcp
|
||||
|
||||
step "recording installed versions"
|
||||
# shellcheck source=/dev/null
|
||||
|
||||
@@ -39,6 +39,11 @@ systemd-tmpfiles --create "$tmpfiles"
|
||||
install -d -o dot -g dot -m 2775 /home/dot/workspace
|
||||
install -d -o dot -g dot -m 0755 /home/dot/downloads /home/dot/documents /home/dot/memory
|
||||
install -d -o dot -g dot -m 0700 /home/dot/browsers
|
||||
# The home of each browser identity's MCP server (architecture 4.2), outside /home/dot on purpose: the server
|
||||
# saves the proxy of its browser, password included, in a session file under its home, and the host's file
|
||||
# routes read /home/dot and nothing else. dot owns it: the server runs as dot.
|
||||
install -d -o root -g root -m 0755 /var/lib/invisible-dots
|
||||
install -d -o dot -g dot -m 0700 /var/lib/invisible-dots/mcp
|
||||
install -d -o dotengine -g dotengine -m 0700 /home/dotengine
|
||||
install -d -o dotengine -g dotengine -m 0700 /home/dotengine/state
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { GUEST_PATHS } from "@invisible-dots/shared";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { BUILDER_PYTHON_LOCK, defaultAssetRoot, GUEST_ASSETS, GUEST_UNITS, unitAsset } from "../src/assets.js";
|
||||
import { GUEST_PINS } from "../src/pins.js";
|
||||
@@ -213,6 +214,13 @@ describe("guest units", () => {
|
||||
expect(text("builder/user-data.yaml")).toMatch(/- name: dotengine\n[\s\S]*?groups: \[dot\]/);
|
||||
});
|
||||
|
||||
it("the provisioner and install.sh make the home of the MCP servers, dot's, outside the home the host reads", () => {
|
||||
expect(GUEST_PATHS.mcpHomes.startsWith(`${GUEST_PATHS.home}/`)).toBe(false);
|
||||
for (const file of ["builder/provision.sh", "runtime/install.sh"]) {
|
||||
expect(text(file), file).toContain(`install -d -o dot -g dot -m 0700 ${GUEST_PATHS.mcpHomes}\n`);
|
||||
}
|
||||
});
|
||||
|
||||
it("install.sh refuses a golden image without the engine's environment", () => {
|
||||
const install = text("runtime/install.sh");
|
||||
expect(install).toContain("[ -x /opt/invisible-dots-engine/bin/python ] || die ");
|
||||
|
||||
@@ -62,7 +62,10 @@ and the checks read that file. What they pin:
|
||||
none open, with no call made to find out;
|
||||
- a proxy password is in no approval, event, engine log or `dot-agentd` log, and on no process's command line (the
|
||||
relay is told the variable's name, `--env-from`, and reads the value from its own environment); `/health` counts the identities
|
||||
and the open ones.
|
||||
and the open ones;
|
||||
- a server's home is `/var/lib/invisible-dots/mcp/<identity_id>`, outside `/home/dot`, and a delete removes it; the stand-in
|
||||
saves the proxy with its password in `sessions/<identity_id>.json` there, as the real server does, and the TCP port refuses
|
||||
that file, its directory and a link to it with `403 outside_home`.
|
||||
|
||||
The real server and a real Firefox are not run by this smoke: they are the browser smoke's (below).
|
||||
|
||||
@@ -95,6 +98,9 @@ engine, the GeoIP file) and at a launch (the egress address, for the timezone).
|
||||
the server and Firefox and leaves the profile locked: the next launch works with that stale
|
||||
lock; what a page stored in the profile (localStorage) before the model's close, and before
|
||||
SIGTERM, which asks the browser to close, is still there after the next launch;
|
||||
- an identity with a proxy (a small authenticating forward proxy, `browser/proxy.py`): the real server saves the proxy with
|
||||
its password in its session file under its home, no file under `/home/dot` holds the password, the TCP port refuses the
|
||||
file (`403 outside_home`), the identity's directory has no MCP home, and a delete removes the home;
|
||||
- the key is in no file, process environment or log, the browser's included, and the identity's
|
||||
events are all in the stream.
|
||||
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
"""A forward proxy with Basic authentication, for the browser smoke's identity with a proxy.
|
||||
|
||||
python3 proxy.py <port> <user> <password>
|
||||
|
||||
It answers 407 to a request without the credentials, tunnels a CONNECT, and forwards a plain
|
||||
`GET http://host/...` request, which is what the browser's egress lookup and the library's probe of
|
||||
the exit send. It prints the method and target of each request it accepted (never the credentials) and listens
|
||||
on 127.0.0.1 only. Standard library only: the container has python3 and
|
||||
nothing else.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import sys
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
PORT = int(sys.argv[1])
|
||||
EXPECTED = "Basic " + base64.b64encode(f"{sys.argv[2]}:{sys.argv[3]}".encode()).decode()
|
||||
|
||||
|
||||
async def pipe(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
|
||||
try:
|
||||
while data := await reader.read(65536):
|
||||
writer.write(data)
|
||||
await writer.drain()
|
||||
except (ConnectionError, asyncio.CancelledError):
|
||||
pass
|
||||
finally:
|
||||
writer.close()
|
||||
|
||||
|
||||
async def refuse(writer: asyncio.StreamWriter, status: str) -> None:
|
||||
writer.write(f"HTTP/1.1 {status}\r\nProxy-Authenticate: Basic realm=smoke\r\nContent-Length: 0\r\nConnection: close\r\n\r\n".encode())
|
||||
await writer.drain()
|
||||
writer.close()
|
||||
|
||||
|
||||
async def serve(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
|
||||
try:
|
||||
head = await reader.readuntil(b"\r\n\r\n")
|
||||
lines = head.decode("latin-1").split("\r\n")
|
||||
method, target, version = lines[0].split(" ", 2)
|
||||
headers = {name.lower(): value for name, _, value in (line.partition(": ") for line in lines[1:] if line)}
|
||||
if headers.get("proxy-authorization") != EXPECTED:
|
||||
await refuse(writer, "407 Proxy Authentication Required")
|
||||
return
|
||||
print(method, target, flush=True) # the smoke reads what went through
|
||||
if method == "CONNECT":
|
||||
host, _, port = target.rpartition(":")
|
||||
upstream_reader, upstream_writer = await asyncio.open_connection(host, int(port))
|
||||
writer.write(f"{version} 200 Connection established\r\n\r\n".encode())
|
||||
await writer.drain()
|
||||
else:
|
||||
url = urlsplit(target)
|
||||
upstream_reader, upstream_writer = await asyncio.open_connection(url.hostname, url.port or 80)
|
||||
kept = [line for line in lines[1:] if line and not line.lower().startswith(("proxy-", "connection:"))]
|
||||
path = (url.path or "/") + (f"?{url.query}" if url.query else "")
|
||||
request = f"{method} {path} {version}\r\n" + "\r\n".join(kept) + "\r\nConnection: close\r\n\r\n"
|
||||
upstream_writer.write(request.encode("latin-1"))
|
||||
await upstream_writer.drain()
|
||||
await asyncio.gather(pipe(reader, upstream_writer), pipe(upstream_reader, writer))
|
||||
except (OSError, asyncio.IncompleteReadError, ValueError):
|
||||
writer.close()
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
server = await asyncio.start_server(serve, "127.0.0.1", PORT)
|
||||
async with server:
|
||||
await server.serve_forever()
|
||||
|
||||
|
||||
asyncio.run(main())
|
||||
@@ -92,6 +92,7 @@ sleep 3
|
||||
start_host_stream
|
||||
|
||||
BROWSERS=/home/dot/browsers
|
||||
MCP_HOMES=/var/lib/invisible-dots/mcp # the servers' homes, outside /home/dot (architecture 4.2)
|
||||
FIREFOX='\.cache/invisible-playwright/firefox-' # what the cached engine's processes are called
|
||||
firefox_running() { pgrep -u dot -f "$FIREFOX" | wc -l; }
|
||||
# A process's environment is read by its own user only (the container's root has no CAP_SYS_PTRACE), so what runs
|
||||
@@ -148,7 +149,7 @@ mcp_env_ok() {
|
||||
local env; env=$(su -s /bin/bash dot -c "tr '\\0' '\\n' < /proc/$MCP_PID/environ")
|
||||
[ "$(stat -c %U "/proc/$MCP_PID")" = dot ] \
|
||||
&& grep -qx "STEALTHFOX_PROFILE_DIR=$BROWSERS/$ID/profile" <<< "$env" \
|
||||
&& grep -qx "INVISIBLE_MCP_HOME=$BROWSERS/$ID/mcp" <<< "$env" \
|
||||
&& grep -qx "INVISIBLE_MCP_HOME=$MCP_HOMES/$ID" <<< "$env" \
|
||||
&& grep -qx 'STEALTHFOX_HEADLESS=0' <<< "$env" && grep -qx 'DISPLAY=:0' <<< "$env" && grep -qx 'HOME=/home/dot' <<< "$env" \
|
||||
&& ! grep -q '^INVISIBLE_DOTS_\|^TIKTOKEN_CACHE_DIR=\|^OPENROUTER_API_KEY=' <<< "$env" \
|
||||
&& ! grep -qF "$KEY" <<< "$env"
|
||||
@@ -226,13 +227,39 @@ check "the restarted engine answers /health" "wait_health && wait_key"
|
||||
check "the model launches the identity again after SIGTERM" "tool_turn 26 browser_identity_launch '{\"identity_id\":\"$ID\"}' && launched_times $ID 4"
|
||||
check "what the page stored before SIGTERM is in the profile after it (SIGTERM made the browser close, so Firefox flushed)" "tool_turn 27 browser_navigate '{\"identity_id\":\"$ID\",\"url\":\"$PAGES/store.html?k=term\"}' && tool_turn 28 browser_read_text '{\"identity_id\":\"$ID\"}' && sent_to_model 'stored before: kept-term'"
|
||||
|
||||
# --- an identity with a proxy: the real server saves the proxy, password included, and no file route of the host reaches it ---
|
||||
# The server writes who `main` is (seed, proxy, profile) to <home>/sessions/<session id>.json on every browser_open. That
|
||||
# file must not be under /home/dot, which is all the host's file routes read (architecture 4.2 and 6).
|
||||
PROXY_USER=smoke-user
|
||||
PROXY_PASSWORD=Pw7c1dSmokeReal
|
||||
install -m 0644 "$HERE/proxy.py" /tmp/proxy.py
|
||||
su -s /bin/bash nobody -c "python3 /tmp/proxy.py 8099 $PROXY_USER $PROXY_PASSWORD" > /tmp/proxy.log 2>&1 &
|
||||
proxy_up() { for _ in $(seq 1 20); do (exec 3<>/dev/tcp/127.0.0.1/8099) 2>/dev/null && return 0; sleep 0.5; done; return 1; }
|
||||
check "the authenticating proxy of the smoke is up" "proxy_up"
|
||||
CODE2=$(api -o /tmp/bid-2.json -w '%{http_code}' -X POST -H 'content-type: application/json' -d "{\"name\":\"proxied\",\"proxy\":\"http://$PROXY_USER:$PROXY_PASSWORD@127.0.0.1:8099\"}" "$A/browser-identities")
|
||||
ID2=$(jq -r .id /tmp/bid-2.json)
|
||||
check "POST /browser-identities with a proxy answers 201 and shows the proxy without its password" "[ '$CODE2' = 201 ] && jq -e '.proxy == \"http://$PROXY_USER:***@127.0.0.1:8099\"' /tmp/bid-2.json >/dev/null"
|
||||
check "the model launches it: the real server opens a browser behind the proxy" "tool_turn 40 browser_identity_launch '{\"identity_id\":\"$ID2\"}' && launched_times $ID2 1"
|
||||
SESSION_FILE=$MCP_HOMES/$ID2/sessions/$ID2.json
|
||||
check "the real server saved the proxy with its password in its session file, dot's, in its home outside /home/dot" "[ \"\$(stat -c %U $SESSION_FILE)\" = dot ] && grep -qF $PROXY_PASSWORD $SESSION_FILE && [ ! -e $BROWSERS/$ID2/mcp ]"
|
||||
check "no file under /home/dot holds the proxy password: the profile, the caches and the logs are clean too" "[ -z \"\$(grep -rlaF $PROXY_PASSWORD /home/dot 2>/dev/null)\" ]"
|
||||
session_file_refused() { # the TCP port, which the host's file routes are the client of, refuses every way to it
|
||||
su -s /bin/bash dot -c "ln -s $MCP_HOMES /home/dot/mcp-link"
|
||||
refuses_outside_home "$SESSION_FILE" "$MCP_HOMES/$ID2/sessions" "mcp-link/$ID2/sessions/$ID2.json" "../../var/lib/invisible-dots/mcp/$ID2/sessions/$ID2.json"
|
||||
}
|
||||
check "the TCP port refuses that file, its directory and a link to it (403 outside_home): no file route of the host answers with the proxy" "session_file_refused"
|
||||
check "the identity's directory, which the host can list, has its profile and no MCP home" "files_list browsers/$ID2 | head -n 1 | jq -e '([.entries[].name] | index(\"profile\") != null) and ([.entries[].name] | index(\"mcp\") == null)' >/dev/null"
|
||||
check "the model closes the proxied identity: Firefox and its server end" "tool_turn 42 browser_identity_close '{\"identity_id\":\"$ID2\"}' && closed $ID2 && { for _ in \$(seq 1 100); do [ -z \"\$(session_pids $ID2)\" ] && break; sleep 0.3; done; [ -z \"\$(session_pids $ID2)\" ]; }"
|
||||
check "the host deletes it: its directory and its server's home, with the session file, are gone" "[ \"\$(api -o /dev/null -w '%{http_code}' -X DELETE $A/browser-identities/$ID2)\" = 204 ] && [ ! -e $BROWSERS/$ID2 ] && [ ! -e $MCP_HOMES/$ID2 ]"
|
||||
|
||||
# --- what the real browser must not have leaked ---
|
||||
sleep 3
|
||||
stop_host_stream
|
||||
ALL=/tmp/stream-all.txt
|
||||
timeout 5 curl "${H[@]}" -N "$A/events/stream?after=0" > "$ALL" 2>/dev/null
|
||||
check "seqs of the full stream are 1..N without a gap" "[ \"\$(seqs $ALL | tr '\n' ' ')\" = \"\$(seq 1 \$(seqs $ALL | wc -l) | tr '\n' ' ')\" ]"
|
||||
check "the identity's events are all there: created once, launched four times, closed twice (the model's close and SIGTERM; kill -9 reports nothing)" "grep '^data: ' $ALL | sed 's/^data: //' | jq -s -e '([.[] | select(.type==\"browser.identity.created\")] | length) == 1 and ([.[] | select(.type==\"browser.identity.launched\")] | length) == 4 and ([.[] | select(.type==\"browser.identity.closed\")] | length) == 2' >/dev/null"
|
||||
check "the identities' events are all there: created twice, launched five times, closed three times (the model's close, SIGTERM and the proxied identity's close; kill -9 reports nothing), deleted once" "grep '^data: ' $ALL | sed 's/^data: //' | jq -s -e '([.[] | select(.type==\"browser.identity.created\")] | length) == 2 and ([.[] | select(.type==\"browser.identity.launched\")] | length) == 5 and ([.[] | select(.type==\"browser.identity.closed\")] | length) == 3 and ([.[] | select(.type==\"browser.identity.deleted\")] | length) == 1' >/dev/null"
|
||||
check "the proxy password is in no event of the whole stream, no engine log and no dot-agentd log" "! grep -qF $PROXY_PASSWORD $ALL /tmp/engine.log /tmp/agentd.log"
|
||||
check "the key is in no file of the engine, the config or the Dot (the browser's profile and cache included)" "! grep -rIl \"$KEY\" /home/dotengine /etc/invisible-dots /home/dot /run/invisible-dots /run/invisible-dots-agent 2>/dev/null | grep -q ."
|
||||
check "the key is not in the environment of any process, Firefox's included" "! environ_holds \"$KEY\""
|
||||
check "the key is in no engine log and no dot-agentd log" "! grep -q \"$KEY\" /tmp/engine.log /tmp/agentd.log"
|
||||
|
||||
@@ -55,6 +55,8 @@ lay_out_guest() {
|
||||
install -d -o dot -g dotengine -m 2750 /run/invisible-dots
|
||||
install -d -o dotengine -g dot -m 2750 /run/invisible-dots-agent
|
||||
install -d -o dot -g dot -m 2775 /home/dot/workspace
|
||||
install -d -o root -g root -m 0755 /var/lib/invisible-dots
|
||||
install -d -o dot -g dot -m 0700 /var/lib/invisible-dots/mcp
|
||||
install -d -o dotengine -g dotengine -m 0700 /home/dotengine /home/dotengine/state
|
||||
}
|
||||
|
||||
@@ -99,6 +101,18 @@ start_engine() { su -s /bin/bash dotengine -c "bash /tmp/engine.sh" >> /tmp/engi
|
||||
|
||||
# --- the host's side: dot-agentd's TCP port with the Dot's token ---
|
||||
api() { curl "${H[@]}" "$@"; }
|
||||
# The host's file routes, as the TCP port serves them (limited to /home/dot, every symbolic link followed): the answer
|
||||
# and, on a last line, its status.
|
||||
files_get() { api -w '\n%{http_code}' --get --data-urlencode "path=$1" http://127.0.0.1:1024/v1/files; }
|
||||
files_list() { api -w '\n%{http_code}' --get --data-urlencode "path=$1" http://127.0.0.1:1024/v1/files/list; }
|
||||
refuses_outside_home() { # paths: each is answered 403 outside_home
|
||||
local p out
|
||||
for p in "$@"; do
|
||||
out=$(files_get "$p")
|
||||
[ "$(printf '%s\n' "$out" | tail -n 1)" = 403 ] || { echo "no 403 for $p: $out"; return 1; }
|
||||
printf '%s\n' "$out" | grep -q '"error":"outside_home"' || { echo "not outside_home for $p: $out"; return 1; }
|
||||
done
|
||||
}
|
||||
init_host_side() {
|
||||
H=(-sS -H "Authorization: Bearer $TOKEN")
|
||||
A=http://127.0.0.1:1024/v1/agent
|
||||
|
||||
@@ -299,17 +299,7 @@ check "a PATCH of an automation whose body is not {enabled: bool} is a 400" "[ \
|
||||
# A link a page could make the Dot stage must not show the proxy password in /proc/<pid>/environ of the browser server
|
||||
# (it runs as dot), nor the Dot's token. The engine's socket takes any path dot can open.
|
||||
su -s /bin/bash dot -c 'printf hello > /home/dot/hello.txt; ln -s /home/dot/hello.txt /home/dot/hello-link; ln -s /proc/self/environ /home/dot/environ-link; ln -s /etc/invisible-dots/config.json /home/dot/config-link; ln -s /etc /home/dot/etc-link'
|
||||
files_get() { api -w '\n%{http_code}' --get --data-urlencode "path=$1" http://127.0.0.1:1024/v1/files; }
|
||||
files_list() { api -w '\n%{http_code}' --get --data-urlencode "path=$1" http://127.0.0.1:1024/v1/files/list; }
|
||||
reads_hello() { local out; out=$(files_get "$1"); [ "$(printf '%s\n' "$out" | tail -n 1)" = 200 ] && [ "$(printf '%s\n' "$out" | head -n 1)" = hello ]; }
|
||||
refuses_outside_home() {
|
||||
local p out
|
||||
for p in "$@"; do
|
||||
out=$(files_get "$p")
|
||||
[ "$(printf '%s\n' "$out" | tail -n 1)" = 403 ] || { echo "no 403 for $p: $out"; return 1; }
|
||||
printf '%s\n' "$out" | grep -q '"error":"outside_home"' || { echo "not outside_home for $p: $out"; return 1; }
|
||||
done
|
||||
}
|
||||
lists_links_as_other() {
|
||||
[ "$(files_list etc-link | tail -n 1)" = 403 ] || return 1
|
||||
files_list . | head -n 1 | jq -e '([.entries[]|select(.name=="environ-link" or .name=="etc-link" or .name=="config-link")|.type]|sort)==["other","other","other"] and ([.entries[]|select(.name=="hello-link")|.type]==["file"])' >/dev/null
|
||||
@@ -350,7 +340,8 @@ check "the host pushes the allow-everything config once more (204 204)" "[ \"\$(
|
||||
# One stand-in process per open identity, started by dot-agentd as dot through the relay. Its record
|
||||
# ($INVISIBLE_MCP_HOME/record.jsonl) holds its environment, its working directory and every call it received.
|
||||
BROWSERS=/home/dot/browsers
|
||||
rec() { echo "$BROWSERS/$1/mcp/record.jsonl"; }
|
||||
MCP_HOMES=/var/lib/invisible-dots/mcp # the stand-ins' homes, outside /home/dot (architecture 4.2)
|
||||
rec() { echo "$MCP_HOMES/$1/record.jsonl"; }
|
||||
fakes_running() { pgrep -u dot -f fake_mcp_server.py | wc -l; }
|
||||
wait_fakes() { # n: the stand-in's processes of dot number n within 10 s
|
||||
for _ in $(seq 1 50); do [ "$(fakes_running)" = "$1" ] && return 0; sleep 0.2; done
|
||||
@@ -362,7 +353,7 @@ mcp_env_ok() { # id, expected STEALTHFOX_PROXY ("" for none)
|
||||
jq -s -e --arg id "$1" --arg proxy "$2" --arg key "$KEY" '
|
||||
[.[] | select(.kind == "start")][0] as $s | ($s.env) as $e
|
||||
| $s.cwd == ("/home/dot/browsers/" + $id)
|
||||
and $e.INVISIBLE_MCP_HOME == ("/home/dot/browsers/" + $id + "/mcp")
|
||||
and $e.INVISIBLE_MCP_HOME == ("/var/lib/invisible-dots/mcp/" + $id)
|
||||
and $e.INVISIBLE_MCP_SESSION_ID == $id
|
||||
and $e.STEALTHFOX_PROFILE_DIR == ("/home/dot/browsers/" + $id + "/profile")
|
||||
and $e.STEALTHFOX_HEADLESS == "0" and $e.DISPLAY == ":0" and $e.HOME == "/home/dot"
|
||||
@@ -382,7 +373,7 @@ ID1=$(jq -r .id /tmp/bid-1.json)
|
||||
identity_one_ok() { jq -e --arg id "$1" '.id == $id and .name == "research" and .status == "available" and .profilePath == ("/home/dot/browsers/" + $id + "/profile") and (has("proxy") | not)' /tmp/bid-1.json >/dev/null; }
|
||||
check "POST /browser-identities answers 201 with a closed identity whose profile is under /home/dot/browsers" "[ '$CODE1' = 201 ] && identity_one_ok '$ID1'"
|
||||
check "browser.identity.created names it" "wait_event $STREAM '.type==\"browser.identity.created\" and .data.identity_id==\"$ID1\" and .data.name==\"research\"'"
|
||||
check "its profile and MCP directories exist, dot's" "[ \"\$(stat -c %U $BROWSERS/$ID1/profile $BROWSERS/$ID1/mcp | tr '\n' ' ')\" = 'dot dot ' ]"
|
||||
check "its profile and MCP home exist, dot's, the home outside /home/dot and not in the identity's directory" "[ \"\$(stat -c %U $BROWSERS/$ID1/profile $MCP_HOMES/$ID1 | tr '\n' ' ')\" = 'dot dot ' ] && [ ! -e $BROWSERS/$ID1/mcp ]"
|
||||
check "no browser runs for a closed identity, and /health counts one identity, none open" "[ \"\$(fakes_running)\" = 0 ] && health_is 1 0"
|
||||
|
||||
check "the model launches the identity" "tool_turn 1 browser_identity_launch '{\"identity_id\":\"$ID1\"}'"
|
||||
@@ -401,7 +392,7 @@ check "the screenshot reached the model's next request as an image part" "jq -s
|
||||
ID2=$(new_identity second); ID3=$(new_identity third); ID4=$(new_identity lossy)
|
||||
check "three more identities exist; /health counts four, one open" "[ -n '$ID2' ] && [ -n '$ID3' ] && [ -n '$ID4' ] && health_is 4 1"
|
||||
# The fourth identity's server reports its browser gone after the first page action, as after a Firefox crash.
|
||||
printf '{"lose_browser_once":true}' > "$BROWSERS/$ID4/mcp/control.json"
|
||||
printf '{"lose_browser_once":true}' > "$MCP_HOMES/$ID4/control.json"
|
||||
check "the model launches the second and the third identity: three are open (max_open 3), three servers run as dot" "tool_turn 4 browser_identity_launch '{\"identity_id\":\"$ID2\"}' && tool_turn 5 browser_identity_launch '{\"identity_id\":\"$ID3\"}' && wait_fakes 3 && health_is 4 3"
|
||||
check "the model launches a fourth identity" "tool_turn 6 browser_identity_launch '{\"identity_id\":\"$ID4\"}'"
|
||||
check "the least recently used identity (the first) was closed: browser.identity.closed" "closed $ID1"
|
||||
@@ -423,6 +414,15 @@ ID5=$(identity_named shopping)
|
||||
check "the identity is listed with its proxy redacted, and browser.identity.created names it" "[ \"\$(api $A/browser-identities/$ID5 | jq -r .proxy)\" = 'http://smoke-user:***@127.0.0.1:9' ] && wait_event $STREAM '.type==\"browser.identity.created\" and .data.identity_id==\"$ID5\" and .data.name==\"shopping\"'"
|
||||
check "the model launches it: the least recently used of the three open (the second) is closed" "tool_turn 9 browser_identity_launch '{\"identity_id\":\"$ID5\"}' && launched $ID5 && closed $ID2 && wait_fakes 3"
|
||||
check "its server got the proxy in STEALTHFOX_PROXY (dot's process, not the engine's), with the rest of the environment as before" "mcp_env_ok $ID5 '$PROXY'"
|
||||
# The real server writes who `main` is, the proxy with its password included, to <home>/sessions/<session id>.json
|
||||
# (the stand-in does the same). The host's file routes reach /home/dot only, and the home of the servers is outside it.
|
||||
SESSION_FILE=$MCP_HOMES/$ID5/sessions/$ID5.json
|
||||
session_file_refused() { # the TCP port, which the host's file routes are the client of, refuses every way to it
|
||||
su -s /bin/bash dot -c "ln -s $MCP_HOMES /home/dot/mcp-link"
|
||||
refuses_outside_home "$SESSION_FILE" "$MCP_HOMES/$ID5/sessions" "mcp-link/$ID5/sessions/$ID5.json" "../../var/lib/invisible-dots/mcp/$ID5/sessions/$ID5.json"
|
||||
}
|
||||
check "the session file the server saved holds the proxy with its password, and it is outside /home/dot" "[ \"\$(stat -c %U $SESSION_FILE)\" = dot ] && grep -q $PROXY_PASSWORD $SESSION_FILE && ! grep -rqs $PROXY_PASSWORD /home/dot"
|
||||
check "the TCP port refuses that file, its directory and a link to it (403 outside_home): no file route of the host can read the proxy" "session_file_refused"
|
||||
check "the proxy password is in no event, no engine log and no dot-agentd log after the launch either" "no_proxy_password_in $STREAM /tmp/engine.log /tmp/agentd.log"
|
||||
check "the proxy password is on no process's command line, which every user can read (the relay of the open browser names the variable and nothing else)" "cmdline_holds '--env-from STEALTHFOX_PROXY' && ! cmdline_holds \$PROXY_PASSWORD"
|
||||
|
||||
@@ -442,7 +442,7 @@ check "the delete is still pending after kill -9" "st | jq -e --arg id \"$APD\"
|
||||
check "after the restart every identity is available, none open, and /health counts five and none" "api $A/browser-identities | jq -e '(.identities|length)==5 and all(.identities[]; .status==\"available\")' >/dev/null && health_is 5 0"
|
||||
check "approval.received approve accepted (202)" "[ \"\$(ev ap-del approval.received '{\"approval_id\":\"'$APD'\",\"decision\":\"approve\"}')\" = 202 ]"
|
||||
check "tool.called browser_identity_delete: ok, decision ask" "wait_event $STREAM '.type==\"tool.called\" and .data.tool==\"browser_identity_delete\" and .data.ok==true and .data.decision==\"ask\" and .data.target==\"$ID3\" and .seq > $LASTD'"
|
||||
check "browser.identity.deleted names it; its directory is gone and it is no longer listed" "wait_event $STREAM '.type==\"browser.identity.deleted\" and .data.identity_id==\"$ID3\" and .data.name==\"third\"' && [ ! -e $BROWSERS/$ID3 ] && [ \"\$(api -o /dev/null -w '%{http_code}' $A/browser-identities/$ID3)\" = 404 ] && health_is 4 0"
|
||||
check "browser.identity.deleted names it; its directory is gone and it is no longer listed" "wait_event $STREAM '.type==\"browser.identity.deleted\" and .data.identity_id==\"$ID3\" and .data.name==\"third\"' && [ ! -e $BROWSERS/$ID3 ] && [ ! -e $MCP_HOMES/$ID3 ] && [ \"\$(api -o /dev/null -w '%{http_code}' $A/browser-identities/$ID3)\" = 404 ] && health_is 4 0"
|
||||
|
||||
# SIGTERM closes an open browser through browser_close before its server ends.
|
||||
check "the model launches an identity again after the restart" "tool_turn 10 browser_identity_launch '{\"identity_id\":\"$ID4\"}' && wait_fakes 1"
|
||||
@@ -477,7 +477,7 @@ check "a server killed while idle closes its identity at once, with no call: clo
|
||||
check "the model launches it once more, so the delete below meets an open identity" "tool_turn 13 browser_identity_launch '{\"identity_id\":\"$ID5\"}' && wait_fakes 1 && health_is 4 1"
|
||||
check "DELETE /browser-identities/:id of the open identity answers 204" "[ \"\$(api -o /dev/null -w '%{http_code}' -X DELETE $A/browser-identities/$ID5)\" = 204 ]"
|
||||
check "its browser was closed, then it was deleted: closed, then deleted" "wait_event $STREAM '.type==\"browser.identity.deleted\" and .data.identity_id==\"$ID5\"' && grep '^data: ' $STREAM | sed 's/^data: //' | jq -s -e '[.[] | select(.data.identity_id==\"$ID5\" and (.type==\"browser.identity.closed\" or .type==\"browser.identity.deleted\")) | .type] | .[-2:] == [\"browser.identity.closed\",\"browser.identity.deleted\"]' >/dev/null"
|
||||
check "its server ended, its directory is gone, /health counts three identities, none open" "wait_fakes 0 && [ ! -e $BROWSERS/$ID5 ] && health_is 3 0"
|
||||
check "its server ended, its directory and its server's home (the session file with the proxy) are gone, /health counts three identities, none open" "wait_fakes 0 && [ ! -e $BROWSERS/$ID5 ] && [ ! -e $MCP_HOMES/$ID5 ] && health_is 3 0"
|
||||
check "the proxy password is in no event, no engine log and no dot-agentd log at the end of the browser checks" "no_proxy_password_in $STREAM /tmp/engine.log /tmp/agentd.log"
|
||||
echo '{"computer.exec":"allow"}' > /tmp/perms.json
|
||||
check "the host pushes the allow-everything config once more (204 204)" "[ \"\$(push)\" = '204 204' ]"
|
||||
|
||||
@@ -53,7 +53,7 @@ from nanobot.dots.identity_rules import (
|
||||
new_identity_id,
|
||||
redact_proxy,
|
||||
)
|
||||
from nanobot.dots.protocol import BROWSER_ENV, BROWSERS_DIR, GUEST_DISPLAY
|
||||
from nanobot.dots.protocol import BROWSER_ENV, BROWSERS_DIR, GUEST_DISPLAY, MCP_HOMES_DIR
|
||||
from nanobot.dots.store import BrowserIdentityRow, DotStore
|
||||
|
||||
# The MCP server's own browser, the one carrying the identity. The server also has a `support` browser;
|
||||
@@ -184,6 +184,7 @@ class BrowserManager:
|
||||
max_open: int,
|
||||
max_identities: int,
|
||||
browsers_dir: str = BROWSERS_DIR,
|
||||
mcp_homes_dir: str = MCP_HOMES_DIR,
|
||||
display: str = GUEST_DISPLAY,
|
||||
open_deadline_s: float = 900.0,
|
||||
open_retry_initial_s: float = 2.0,
|
||||
@@ -201,6 +202,7 @@ class BrowserManager:
|
||||
self._max_open = max_open
|
||||
self._max_identities = max_identities
|
||||
self._browsers_dir = browsers_dir
|
||||
self._mcp_homes_dir = mcp_homes_dir
|
||||
self._display = display
|
||||
self._open_deadline_s = open_deadline_s
|
||||
self._open_retry_initial_s = open_retry_initial_s
|
||||
@@ -491,8 +493,14 @@ class BrowserManager:
|
||||
self._sessions[identity_id] = self._sessions.pop(identity_id)
|
||||
|
||||
def _paths(self, identity_id: str) -> tuple[str, str, str]:
|
||||
"""The identity's directory, its profile, and the home of its MCP server.
|
||||
|
||||
The home is not under the identity's directory, nor anywhere under /home/dot: the server saves the
|
||||
proxy of its browser, password included, in a session file under its home, and the host's file routes
|
||||
read /home/dot and nothing else.
|
||||
"""
|
||||
root = posixpath.join(self._browsers_dir, identity_id)
|
||||
return root, posixpath.join(root, "profile"), posixpath.join(root, "mcp")
|
||||
return root, posixpath.join(root, "profile"), posixpath.join(self._mcp_homes_dir, identity_id)
|
||||
|
||||
async def _make_directories(self, identity_id: str) -> None:
|
||||
_, profile, mcp_home = self._paths(identity_id)
|
||||
@@ -504,8 +512,8 @@ class BrowserManager:
|
||||
)
|
||||
|
||||
async def _remove_directory(self, identity_id: str) -> None:
|
||||
root, _, _ = self._paths(identity_id)
|
||||
result = await self._computer.run(["rm", "-rf", "--", root])
|
||||
root, _, mcp_home = self._paths(identity_id)
|
||||
result = await self._computer.run(["rm", "-rf", "--", root, mcp_home])
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(
|
||||
f"could not remove the directory of browser identity {identity_id}: "
|
||||
|
||||
@@ -77,9 +77,13 @@ MODEL_ROLES = ("summary",)
|
||||
# keeps the two equal, and packages/shared/test/events.test.ts pins the unit).
|
||||
TOOL_TARGET_MAX = 160
|
||||
|
||||
# Where the browser identities of a Dot live, and the display their browsers draw on (GUEST_PATHS.browsers and
|
||||
# GUEST_DISPLAY in packages/shared protocol.ts; tests/repo/vendored-nanobot.test.ts keeps them equal).
|
||||
# Where the browser identities of a Dot live, where the MCP server of each keeps its own files (outside /home/dot on
|
||||
# purpose: the server saves the proxy of the browser it opened, password included, under its home, and the host's
|
||||
# file routes read /home/dot and nothing else), and the display their browsers draw on (GUEST_PATHS.browsers,
|
||||
# GUEST_PATHS.mcpHomes and GUEST_DISPLAY in packages/shared protocol.ts; tests/repo/vendored-nanobot.test.ts keeps
|
||||
# them equal).
|
||||
BROWSERS_DIR = "/home/dot/browsers"
|
||||
MCP_HOMES_DIR = "/var/lib/invisible-dots/mcp"
|
||||
GUEST_DISPLAY = ":0"
|
||||
|
||||
# The environment of one identity's invisible-playwright-mcp process: each key is the name in ENV of
|
||||
|
||||
@@ -18,6 +18,7 @@ from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
from fakes.browser_manager import mcp_home
|
||||
from fakes.fake_mcp_server import install_fake_mcp, read_record, write_control
|
||||
from fakes.local_computer import LocalComputer
|
||||
|
||||
@@ -29,7 +30,7 @@ from nanobot.dots.browser import (
|
||||
result_is_error,
|
||||
result_text,
|
||||
)
|
||||
from nanobot.dots.protocol import BROWSER_ENV
|
||||
from nanobot.dots.protocol import BROWSER_ENV, BROWSERS_DIR, MCP_HOMES_DIR
|
||||
from nanobot.dots.store import DotStore
|
||||
|
||||
FAST = {"open_retry_initial_s": 0.01, "open_retry_max_s": 0.02}
|
||||
@@ -42,6 +43,7 @@ class Env:
|
||||
self.tmp_path = tmp_path
|
||||
self.store = store
|
||||
self.browsers = tmp_path / "browsers"
|
||||
self.mcp_homes = tmp_path / "mcp-homes"
|
||||
self.relay_log = tmp_path / "relay.jsonl"
|
||||
(tmp_path / "bin").mkdir()
|
||||
self.mcp_bin = install_fake_mcp(tmp_path / "bin")
|
||||
@@ -57,6 +59,7 @@ class Env:
|
||||
"max_open": 3,
|
||||
"max_identities": 20,
|
||||
"browsers_dir": str(self.browsers),
|
||||
"mcp_homes_dir": str(self.mcp_homes),
|
||||
**FAST,
|
||||
**options,
|
||||
}
|
||||
@@ -65,7 +68,7 @@ class Env:
|
||||
return manager
|
||||
|
||||
def mcp_home(self, identity_id: str) -> Path:
|
||||
return self.browsers / identity_id / "mcp"
|
||||
return mcp_home(self.tmp_path, identity_id)
|
||||
|
||||
def record(self, identity_id: str) -> list[dict[str, Any]]:
|
||||
return read_record(self.mcp_home(identity_id))
|
||||
@@ -107,7 +110,8 @@ async def test_creates_the_directories_and_the_row_and_follows_them_through_laun
|
||||
|
||||
assert identity.id.startswith("shopping-account-") and len(identity.id) == len("shopping-account-") + 6
|
||||
root = env.browsers / identity.id
|
||||
assert (root / "profile").is_dir() and (root / "mcp").is_dir()
|
||||
assert (root / "profile").is_dir() and env.mcp_home(identity.id).is_dir()
|
||||
assert not (root / "mcp").exists()
|
||||
assert not (root / "metadata.json").exists()
|
||||
assert (identity.name, identity.status, identity.last_used_at, identity.proxy) == (
|
||||
"Shopping Account",
|
||||
@@ -173,9 +177,12 @@ async def test_delete_closes_the_session_removes_the_directory_and_the_row(env:
|
||||
identity = await manager.create("temp")
|
||||
await manager.launch(identity.id)
|
||||
|
||||
assert env.mcp_home(identity.id).is_dir()
|
||||
|
||||
await manager.delete(identity.id)
|
||||
|
||||
assert not (env.browsers / identity.id).exists()
|
||||
assert not env.mcp_home(identity.id).exists()
|
||||
assert manager.get(identity.id) is None
|
||||
assert not manager.is_open(identity.id)
|
||||
assert env.event_types()[-2:] == ["browser.identity.closed", "browser.identity.deleted"]
|
||||
@@ -188,7 +195,7 @@ async def test_a_row_whose_directory_is_gone_can_still_be_deleted(env: Env) -> N
|
||||
manager = env.manager()
|
||||
identity = await manager.create("half")
|
||||
(env.browsers / identity.id / "profile").rmdir()
|
||||
(env.browsers / identity.id / "mcp").rmdir()
|
||||
env.mcp_home(identity.id).rmdir()
|
||||
(env.browsers / identity.id).rmdir()
|
||||
|
||||
await manager.delete(identity.id)
|
||||
@@ -249,7 +256,7 @@ async def test_starts_the_server_as_dot_through_the_relay_with_the_environment_o
|
||||
root = env.browsers / identity.id
|
||||
[start] = [entry for entry in env.record(identity.id) if entry["kind"] == "start"]
|
||||
environment = start["env"]
|
||||
assert environment[BROWSER_ENV["MCP_HOME"]] == str(root / "mcp")
|
||||
assert environment[BROWSER_ENV["MCP_HOME"]] == str(env.mcp_home(identity.id))
|
||||
assert environment[BROWSER_ENV["MCP_SESSION_ID"]] == identity.id
|
||||
assert environment[BROWSER_ENV["PROFILE_DIR"]] == str(root / "profile")
|
||||
assert environment[BROWSER_ENV["HEADLESS"]] == "0"
|
||||
@@ -274,6 +281,28 @@ async def test_starts_the_server_as_dot_through_the_relay_with_the_environment_o
|
||||
assert not mcp_runs[0]["tty"]
|
||||
|
||||
|
||||
def test_the_default_home_of_the_servers_is_outside_the_directory_the_host_reads() -> None:
|
||||
# The host's file routes read /home/dot and nothing else (checkHomePath); the server saves the proxy of its
|
||||
# browser, password included, under its home.
|
||||
assert BROWSERS_DIR.startswith("/home/dot/")
|
||||
assert not (MCP_HOMES_DIR + "/").startswith("/home/dot/")
|
||||
|
||||
|
||||
async def test_the_session_file_with_the_proxy_password_is_under_the_servers_home_and_nowhere_in_the_browsers_directory(
|
||||
env: Env,
|
||||
) -> None:
|
||||
manager = env.manager()
|
||||
identity = await manager.create("shop", "http://user:[email protected]:8080")
|
||||
|
||||
await manager.launch(identity.id)
|
||||
|
||||
session_file = env.mcp_home(identity.id) / "sessions" / f"{identity.id}.json"
|
||||
assert "pw-secret" in session_file.read_text(encoding="utf-8")
|
||||
assert env.mcp_homes not in env.browsers.parents and env.browsers not in env.mcp_homes.parents
|
||||
holders = [path for path in env.browsers.rglob("*") if path.is_file() and b"pw-secret" in path.read_bytes()]
|
||||
assert holders == []
|
||||
|
||||
|
||||
async def test_leaves_the_proxy_variable_unset_for_an_identity_without_a_proxy_even_if_the_engine_has_one(
|
||||
env: Env, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
|
||||
@@ -15,7 +15,7 @@ from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
from fakes.browser_manager import make_browser_manager
|
||||
from fakes.browser_manager import make_browser_manager, mcp_home
|
||||
from fakes.dot_config import ALLOW_ALL
|
||||
from fakes.fake_mcp_server import FIXTURE, PNG, read_record, write_control
|
||||
from fakes.local_computer import LocalComputer
|
||||
@@ -75,7 +75,7 @@ class Env:
|
||||
return identity.id
|
||||
|
||||
def calls(self, identity_id: str) -> list[tuple[str, dict[str, Any]]]:
|
||||
record = read_record(self.tmp_path / "browsers" / identity_id / "mcp")
|
||||
record = read_record(mcp_home(self.tmp_path, identity_id))
|
||||
return [(entry["name"], entry["args"]) for entry in record if entry["kind"] == "call"]
|
||||
|
||||
def page_calls(self, identity_id: str) -> list[tuple[str, dict[str, Any]]]:
|
||||
@@ -198,7 +198,7 @@ async def test_a_page_tool_on_an_identity_that_is_not_open_says_so_and_starts_no
|
||||
assert said(result) == NOT_OPEN.format(identity.id)
|
||||
assert not env.manager.is_open(identity.id)
|
||||
assert env.calls(identity.id) == []
|
||||
assert read_record(env.tmp_path / "browsers" / identity.id / "mcp") == []
|
||||
assert read_record(mcp_home(env.tmp_path, identity.id)) == []
|
||||
assert env.images.images == ()
|
||||
|
||||
|
||||
@@ -262,7 +262,7 @@ async def test_creating_with_a_proxy_works_and_the_launch_gives_it_to_the_browse
|
||||
assert "hunter2" not in created
|
||||
identity_id = created_id(created)
|
||||
await env.run("browser_identity_launch", identity_id=identity_id)
|
||||
starts = [entry for entry in read_record(env.tmp_path / "browsers" / identity_id / "mcp") if entry["kind"] == "start"]
|
||||
starts = [entry for entry in read_record(mcp_home(env.tmp_path, identity_id)) if entry["kind"] == "start"]
|
||||
assert starts[0]["env"]["STEALTHFOX_PROXY"] == "http://user:[email protected]:8080"
|
||||
|
||||
|
||||
@@ -295,12 +295,13 @@ async def test_delete_removes_the_identity_and_its_profile(env: Env) -> None:
|
||||
|
||||
assert deleted == f"Deleted the identity {identity_id} and its profile."
|
||||
assert not (env.tmp_path / "browsers" / identity_id).exists()
|
||||
assert not mcp_home(env.tmp_path, identity_id).exists()
|
||||
assert env.manager.get(identity_id) is None
|
||||
|
||||
|
||||
async def test_a_launch_the_server_refuses_is_an_error_result_without_the_proxy_password(env: Env) -> None:
|
||||
identity = await env.manager.create("failing", "http://user:[email protected]:8080")
|
||||
write_control(env.tmp_path / "browsers" / identity.id / "mcp", fail_open=True, echo_proxy=True)
|
||||
write_control(mcp_home(env.tmp_path, identity.id), fail_open=True, echo_proxy=True)
|
||||
|
||||
result = await env.run("browser_identity_launch", identity_id=identity.id)
|
||||
|
||||
@@ -336,7 +337,7 @@ async def test_a_selector_given_as_null_is_left_out_and_the_server_reads_the_pag
|
||||
|
||||
async def test_an_error_of_the_server_is_an_error_result(env: Env) -> None:
|
||||
identity_id = await env.open_identity()
|
||||
write_control(env.tmp_path / "browsers" / identity_id / "mcp", lose_browser_always=True)
|
||||
write_control(mcp_home(env.tmp_path, identity_id), lose_browser_always=True)
|
||||
await env.manager.close(identity_id)
|
||||
await env.manager.launch(identity_id)
|
||||
|
||||
@@ -589,7 +590,7 @@ async def test_a_policy_that_asks_before_navigating_shows_the_person_the_url_and
|
||||
assert asked["arguments"] == {"identity_id": identity_id, "url": "https://example.com/?q=1"}
|
||||
assert h.browser.is_open(identity_id)
|
||||
# Nothing was sent to the page.
|
||||
record = read_record(Path(h.browser._browsers_dir) / identity_id / "mcp")
|
||||
record = read_record(mcp_home(h.tmp_path, identity_id))
|
||||
assert [entry["name"] for entry in record if entry["kind"] == "call"] == ["browser_open"]
|
||||
|
||||
|
||||
|
||||
@@ -16,6 +16,7 @@ from typing import Any
|
||||
|
||||
import aiohttp
|
||||
import pytest
|
||||
from fakes.browser_manager import mcp_home
|
||||
from fakes.dot_config import ALLOW_ALL, runtime_config_body
|
||||
from fakes.engine_harness import EngineHarness, user_message
|
||||
from fakes.fake_mcp_server import write_control
|
||||
@@ -571,7 +572,7 @@ class TestBrowserIdentities:
|
||||
slow = (await api.call("POST", "/browser-identities", {"name": "slow"})).json
|
||||
other = (await api.call("POST", "/browser-identities", {"name": "other"})).json
|
||||
await api.h.browser.launch(other["id"])
|
||||
write_control(api.h.tmp_path / "browsers" / slow["id"] / "mcp", download_answers=100_000)
|
||||
write_control(mcp_home(api.h.tmp_path, slow["id"]), download_answers=100_000)
|
||||
launching = asyncio.create_task(api.h.browser.launch(slow["id"]))
|
||||
await asyncio.sleep(0.3)
|
||||
|
||||
@@ -654,7 +655,7 @@ class TestBrowserIdentityActions:
|
||||
async def test_a_frame_the_server_cannot_give_is_502_frame_failed(self, make_api: Callable[..., Any]) -> None:
|
||||
api: Api = await make_api()
|
||||
identity = (await api.call("POST", "/browser-identities", {"name": "blank"})).json
|
||||
write_control(api.h.tmp_path / "browsers" / identity["id"] / "mcp", fail_watch=True)
|
||||
write_control(mcp_home(api.h.tmp_path, identity["id"]), fail_watch=True)
|
||||
await api.h.browser.launch(identity["id"])
|
||||
|
||||
failed = await api.call("GET", f"/browser-identities/{identity['id']}/frame")
|
||||
|
||||
@@ -15,8 +15,13 @@ from nanobot.dots.store import DotStore
|
||||
FAST = {"open_retry_initial_s": 0.01, "open_retry_max_s": 0.02}
|
||||
|
||||
|
||||
def mcp_home(tmp_path: Path, identity_id: str) -> Path:
|
||||
"""Where the MCP server of an identity keeps its files: outside the browsers directory, as in a Dot."""
|
||||
return tmp_path / "mcp-homes" / identity_id
|
||||
|
||||
|
||||
def make_browser_manager(tmp_path: Path, store: DotStore, computer: Computer, **options: Any) -> BrowserManager:
|
||||
"""The browsers live under `tmp_path/browsers`, the MCP program is the fake one."""
|
||||
"""The browsers live under `tmp_path/browsers`, their MCP homes under `tmp_path/mcp-homes`, the MCP program is the fake one."""
|
||||
(tmp_path / "bin").mkdir(exist_ok=True)
|
||||
settings: dict[str, Any] = {
|
||||
"store": store,
|
||||
@@ -25,6 +30,7 @@ def make_browser_manager(tmp_path: Path, store: DotStore, computer: Computer, **
|
||||
"max_open": 3,
|
||||
"max_identities": 20,
|
||||
"browsers_dir": str(tmp_path / "browsers"),
|
||||
"mcp_homes_dir": str(tmp_path / "mcp-homes"),
|
||||
**FAST,
|
||||
**options,
|
||||
}
|
||||
|
||||
@@ -2,7 +2,9 @@
|
||||
|
||||
It speaks real MCP through the SDK, answers with the sentences the real server uses, and writes what
|
||||
it was started with and every call it receives to `$INVISIBLE_MCP_HOME/record.jsonl`, so a test can
|
||||
assert on the process's environment, working directory and calls. Its behavior is steered by
|
||||
assert on the process's environment, working directory and calls. Like the real server, a successful
|
||||
`browser_open` writes down who `main` is (seed, proxy with its password, profile directory) in
|
||||
`$INVISIBLE_MCP_HOME/sessions/<$INVISIBLE_MCP_SESSION_ID>.json`, so a test can assert where that file is. Its behavior is steered by
|
||||
`$INVISIBLE_MCP_HOME/control.json`, because the engine hands the process an environment of its own
|
||||
and a test cannot add a variable to it.
|
||||
|
||||
@@ -88,6 +90,18 @@ async def _serve() -> None:
|
||||
|
||||
record({"kind": "start", "pid": os.getpid(), "argv": sys.argv[1:], "env": dict(os.environ), "cwd": os.getcwd()})
|
||||
|
||||
def remember(role: str) -> None:
|
||||
"""What the real server's `Work.remember` saves: who the browser is, the proxy included."""
|
||||
session_id = os.environ.get("INVISIBLE_MCP_SESSION_ID") or "default"
|
||||
who = {
|
||||
"seed": 1,
|
||||
"proxy": os.environ.get("STEALTHFOX_PROXY"),
|
||||
"profile_dir": os.environ.get("STEALTHFOX_PROFILE_DIR"),
|
||||
}
|
||||
saved = home / "sessions" / f"{session_id}.json"
|
||||
saved.parent.mkdir(parents=True, exist_ok=True)
|
||||
saved.write_bytes(json.dumps({"browsers": {role: who}, "focus": role}).encode("utf-8"))
|
||||
|
||||
tools = {tool["name"]: tool for tool in _tools()}
|
||||
state: dict[str, Any] = {
|
||||
"download_left": int(control.get("download_answers", 0)),
|
||||
@@ -119,6 +133,7 @@ async def _serve() -> None:
|
||||
"Call browser_open again in a minute; nothing else needs doing."
|
||||
)
|
||||
state["open"] = True
|
||||
remember(role)
|
||||
return text(f"the {role} browser is open. seed: remembered by the profile")
|
||||
if name == "browser_close":
|
||||
if control.get("refuse_close"):
|
||||
|
||||
@@ -101,6 +101,12 @@ export const GUEST_PATHS = {
|
||||
documents: "/home/dot/documents",
|
||||
memory: "/home/dot/memory",
|
||||
browsers: "/home/dot/browsers",
|
||||
/**
|
||||
* The home of each browser identity's MCP server (`<mcpHomes>/<identity_id>`). Outside `/home/dot` on purpose: the
|
||||
* server saves the proxy of the browser it opened, password included, in a session file under its home, and the
|
||||
* file routes of the host read `/home/dot` and nothing else (architecture sections 4.2 and 6).
|
||||
*/
|
||||
mcpHomes: "/var/lib/invisible-dots/mcp",
|
||||
runDir: "/run/invisible-dots",
|
||||
agentdSocket: "/run/invisible-dots/agentd.sock",
|
||||
/** The engine's API, in a directory of the engine's user dot cannot write (architecture 4.2). */
|
||||
@@ -146,13 +152,20 @@ export function checkHomePath(raw: unknown): HomePathCheck {
|
||||
return { ok: true, path: `/${kept.join("/")}` };
|
||||
}
|
||||
|
||||
/** Paths of one browser identity under a browsers root (default `/home/dot/browsers`). */
|
||||
export function identityPaths(identityId: string, browsersDir: string = GUEST_PATHS.browsers) {
|
||||
/**
|
||||
* Paths of one browser identity: its directory and profile under a browsers root (default `/home/dot/browsers`), and
|
||||
* the home of its MCP server under an MCP homes root (default `/var/lib/invisible-dots/mcp`, outside the home).
|
||||
*/
|
||||
export function identityPaths(
|
||||
identityId: string,
|
||||
browsersDir: string = GUEST_PATHS.browsers,
|
||||
mcpHomesDir: string = GUEST_PATHS.mcpHomes,
|
||||
) {
|
||||
const root = join(browsersDir, identityId);
|
||||
return {
|
||||
root,
|
||||
profile: join(root, "profile"),
|
||||
mcp: join(root, "mcp"),
|
||||
mcp: join(mcpHomesDir, identityId),
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -31,9 +31,16 @@ describe("protocol constants", () => {
|
||||
expect(identityPaths("shop-ab12cd")).toEqual({
|
||||
root: "/home/dot/browsers/shop-ab12cd",
|
||||
profile: "/home/dot/browsers/shop-ab12cd/profile",
|
||||
mcp: "/home/dot/browsers/shop-ab12cd/mcp",
|
||||
mcp: "/var/lib/invisible-dots/mcp/shop-ab12cd",
|
||||
});
|
||||
expect(identityPaths("x", "/tmp/b/").profile).toBe("/tmp/b/x/profile");
|
||||
expect(identityPaths("x", "/tmp/b/", "/tmp/m/").profile).toBe("/tmp/b/x/profile");
|
||||
expect(identityPaths("x", "/tmp/b/", "/tmp/m/").mcp).toBe("/tmp/m/x");
|
||||
});
|
||||
|
||||
it("keeps the home of an MCP server, which saves its proxy with the password, out of what the file routes read", () => {
|
||||
const sessionFile = `${identityPaths("shop-ab12cd").mcp}/sessions/shop-ab12cd.json`;
|
||||
expect(checkHomePath(sessionFile)).toEqual({ ok: false, problem: "path must be inside /home/dot" });
|
||||
expect(checkHomePath(GUEST_PATHS.mcpHomes)).toEqual({ ok: false, problem: "path must be inside /home/dot" });
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -149,6 +149,7 @@ describe("the vendored nanobot fork", () => {
|
||||
|
||||
// Where the browser identities live, the display they draw on, and the environment of each one's MCP process.
|
||||
expect(/^BROWSERS_DIR = "([^"]+)"/m.exec(protocol)?.[1]).toBe(GUEST_PATHS.browsers);
|
||||
expect(/^MCP_HOMES_DIR = "([^"]+)"/m.exec(protocol)?.[1]).toBe(GUEST_PATHS.mcpHomes);
|
||||
expect(/^GUEST_DISPLAY = "([^"]+)"/m.exec(protocol)?.[1]).toBe(GUEST_DISPLAY);
|
||||
const browserEnv = /^BROWSER_ENV = \{([^}]*)\}/m.exec(protocol);
|
||||
expect(browserEnv, "BROWSER_ENV").not.toBeNull();
|
||||
|
||||
Reference in new issue
Block a user