QwenLM/qwen-code
1
0 Fork 0
mirror of https://github.com/QwenLM/qwen-code.git synced 2026-11-07 06:19:43 -10:00
Code Issues Packages Projects Releases 1 Wiki Activity

feat(browser-use): add Chrome Native Messaging relay (#11242)

Browse Source 
* feat(browser-use): add Playwright browser SDK

* docs(browser-use): document browser use architecture

* fix(browser-use): make browser input actions reliable

* feat(browser-use): preserve screenshot coordinate metadata

* fix(browser-use): recover stale sockets across VM realms

* fix(browser-use): bound locator action retries

* fix(browser-use): keep page input in the background

* fix(browser-use): reject ambiguous input arguments

* fix(browser-use): validate locator composition identity

* fix(browser-use): recover socket locks across VM realms

* fix(browser-use): preserve input results across navigation

* fix(browser-use): capture screenshots through bounded CDP frames

* fix(browser-use): separate input and navigation waits

* feat(browser-use): add Chrome native relay

* fix(chrome-extension): avoid foreground tabs and stale cursor markup

* fix(chrome-extension): require Chrome 125 for flat sessions

* feat(browser-use): add product integration

* test(browser-use): add managed Chrome smoke coverage

* feat(node-repl): expose structured image metadata

* test(browser-use): preserve complete smoke reports

* feat(browser-use): bundle skill and runtime with CLI

* fix(browser-use): install pinned Playwright in fresh workspaces

* feat(browser-use): require explicit skill opt-in

* feat(browser-use): make bundled skill available by default

Keep explicit skill disables authoritative, update documentation and tests, and use the latest Node REPL package in setup instructions.

* fix(ci): build browser-use before Live Host publish

Build the bundled browser runtime before core in the Live Host npm dependency chain, and cover that prerequisite order with a regression test.

* fix(browser-use): output screenshot metadata explicitly

* fix(node-repl): preserve image metadata outside prose budget

* fix(browser-use): address review round on workspace wiring and runtime correctness

- register the workspace in .pnpmfile.mjs, pnpm-lock.yaml, and the
  unit-vitest-configs registry, with the shared-pool timeout ternary and
  off-Linux unhandled-error exemption (fixes the red Install/Test checks)
- map Playwright's 'warning' console type to the contract's 'warn' level
- reject captures whose pixel dimensions contradict the css-pixels
  coordinate contract instead of certifying rescaled images
- keep one unattachable derived tab from failing tabs.list while still
  propagating a lost session
- clear the cached dialog when Chrome resolves it out of band or the page
  navigates, guarded by dialog identity so a newer dialog survives
- synthesize a parseable Browser.getVersion (numeric Chrome major, host
  platform token) so Playwright enables the right keyboard editing commands
- keep YAML-quoted snapshot keys and link props in interactive listings and
  stop claiming subtrees that filtering removed
- always release the mouse button when a drag gesture fails

* fix(dev): build browser-use runtime before startup

* fix(browser-use): isolate cleanup and correct smoke checks

* fix(browser-use): resolve managed Chrome through Playwright

* fix(browser-use): harden native relay and tab lifecycle

* fix(browser-use): drain host output and persist restored tab state

* fix(browser-use): scope dialog handles and clean navigation waiters

* fix(browser-use): preserve input outcomes and release modifiers

* fix(browser-use): drain session teardown and release debugger attachments

* fix(browser-use): correct locator validation and evaluation

* fix(dev): reuse prebuilt Browser Use runtime

* fix(browser-use): address review round on locator, bridge, and screenshot correctness

Co-authored-by: Qwen-Coder <[email protected]>

* fix(browser-use): require installed extension before native host setup

* fix(browser-use): address review round on schema probe bound and regex validation

* fix(browser-use): start transport before Chrome preflight

* fix(browser-use): exclude staged runtime from code checks

* fix(core): declare Browser Use build dependency

* fix(browser-use): preserve successful navigation and dialog input

* fix(browser-use): harden input drain and tab registration cleanup

* fix(browser-use): keep sessions bound to one Chrome profile

* fix(browser-use): bound commands and validate evaluation JSON

* fix(browser-use): reuse unchanged Native Host registration

* fix(browser-use): wait for fresh extension installation

* fix(browser-use): address review round on screenshot origin, dialog scope, and bridge privacy

- screenshot: tolerate settle-probe failure (measure the ratio from the
  captured pixels, re-capture on HiDPI), measure layout after the settle
  wait, and reject screencast frames whose scroll offsets disagree with
  the published origin
- session: clear the cached dialog only on main-frame navigation;
  abort tab attachment when the tab is removed mid-attach
- errors: classify crashes and timeouts by Playwright's structured error
  name instead of page-influenced message text
- locator.press: release chord modifiers when a later token is rejected;
  cap locator.type at the 60k characters its per-character budget can
  deliver inside the 120s ceiling
- snapshot: keep AI-mode [cursor=pointer] generic nodes in the
  interactive listing
- bridge: derive the fallback socket directory as an env-independent
  per-user 0700 subdirectory on all POSIX hosts and verify ownership
  before binding

* fix(browser-use): close review-critical gaps in error mapping, budgets, and downloads

- errors: classify by Playwright error name first; text phrases match only
  the start of the first message line so page-controlled text (log tails,
  quoted markup, page-thrown values) cannot pick the reported code, and a
  TimeoutError name wins over crash-shaped page text
- transport: cdp.send bridge requests default to 130s so operations up to
  the 120s schema ceiling report their own deadline; other requests keep
  the 30s default
- transport: the /tmp fallback socket directory is a per-user leaf directly
  under the sticky root (no shared intermediate a first mover could own);
  ensureSocketDirectory re-verifies the leaf after creation and walks
  ancestors up to the sticky root, requiring each to be owned by this user
  or root and not group/other-writable
- runtime: withModifiers no longer rewrites a completed action into a
  failure when a modifier release fails; chord failure cleanup releases the
  chord's own tokens (including non-modifiers) instead of four fixed names
- playwright: enable CDP download events on tab attach (Playwright's
  noDefaults connect leaves them off, so waitForEvent('download') could
  never resolve); locator.downloadMedia races the caller deadline, aborts
  the in-page fetch at it, resolves media contained in a located wrapper
  before an ancestor link, restricts URL schemes, and names CORS rejections
  distinctly; locator.allTextContents waits for the first match within the
  caller budget and still resolves [] when nothing attaches
- screenshot: full-page captures get a 16 Mi-pixel budget (1920x3000 pages
  now succeed) and the encoded byte ceiling drops to the 4 MiB model image
  limit so captures are not dropped at the model boundary
- snapshot: interactiveOnly promotes only a terminal [cursor=pointer]
  attribute (page text quoting the marker no longer promotes the line), and
  truncation skips over-long lines instead of discarding the rest
- schemas: reject empty string/regex matchers and exact:true combined with
  a regex matcher (Playwright silently drops exact for regexes)
- tests: the SDK evaluate fixture now mirrors production serialization;
  POSIX socket tests carry win32 gates; posix.join keeps the derived socket
  path a pure function of uid and platform on every host

* fix(browser-use): close review criticals in error mapping, chords, downloads, and snapshots

- errors: classify Playwright's genuine strict-mode violation, which
  crosses CDP as the raw exception description and reaches the client
  behind an "Error: " layer ("locator.click: Error: strict mode
  violation: ... resolved to N elements:"); requiring the element count
  keeps a page-thrown lookalike from forging LOCATOR_NOT_UNIQUE
- runtime-helpers: on a failed chord press, release the tokens
  Playwright actually derived ('+' separates only after a non-empty
  token, so a standalone '+' is a literal key) rather than the caller's
  array elements; an element embedding '+' previously released nothing
  and left modifiers held. locator.press uses the same tokenizer
- locator.downloadMedia: resolve an ordered candidate list (element,
  media descendants, anchor descendants, closest ancestor) and keep the
  first candidate that exposes a URL, so a wrapping anchor no longer
  beats the image it contains and a source srcset or an unloaded
  video's source child can supply the URL
- snapshot: unwrap Playwright's single-quoted YAML key before the
  [cursor=pointer] suffix test, so clickable nodes whose accessible
  name forces key quoting stay in the interactive listing
- docs: align the Input completion contract with the shipped modifier
  cleanup semantics (cleanup failures are discarded; only the action's
  own error propagates) in both languages

* fix(browser-use): close round-13 review criticals in errors, srcset, evaluateAll, snapshots, and screenshot origin

* fix(browser-use): point the pnpm importer at a vitest snapshot that exists

Merging main brought in #11625, which regenerated pnpm-lock.yaml and
dropped the [email protected] snapshot keyed with @vitest/browser and
[email protected]. The packages/browser-use importer still referenced
that key, so every Install leg failed with
ERR_PNPM_LOCKFILE_MISSING_DEPENDENCY. Re-key the importer to the snapshot
its sibling packages already use; nothing else in the lockfile changes.

* fix(browser-use): close round-14 criticals in error classification, dialog ordering and screenshot ratio

R13-2: the strict-mode branch of sanitizeOperationError was the only
text-phrase branch not gated on the evaluate channel, so a page-thrown
primitive "strict mode violation: ... resolved to 2 elements:" picked
LOCATOR_NOT_UNIQUE. On that channel only Playwright's "Error: "-wrapped
form classifies now; other channels keep the unwrapped form.

R14-1: Playwright hands a dialog over on a later turn than the bridge
reports its CDP events (a microtask in CRSession plus the in-process
dispatcher's setImmediate), so an open+close pair delivered in one chunk
left a phantom dialog that blocked the tab. Each tab now traces dialog
openings and closes in bridge order and drops a delivered dialog the
bridge already reported closed; a close with no traced opening (a dialog
open before the tab was attached) is never charged to a later dialog, and
the microtask clear stays for the delivered case.

R14-2: window.devicePixelRatio is page script, and a spoofed value became
an unbounded clip.scale while the pixel re-measure only ran when the
probe failed. The probed ratio is now trusted only within 0.25..8, and the
measure-and-recapture correction runs whenever the first fallback capture
disagrees with the requested CSS region, deriving the real ratio from
Chrome's own pixels (measured / (width * scale)).

Also resolve the temp root in the ensureSocketDirectory tests: the
ancestor sweep rejects symlinks, and macOS reaches os.tmpdir() through
/var -> /private/var, so the tests failed locally on macOS.

* fix(browser-use): tear down accepted peers on start failure and pin the transport's untested paths

The esbuild banner declared require/__filename/__dirname/process bindings
the bundle never uses, so any bundled module declaring one of them built
green but failed to load; drop it (and the unused .wasm loader) and import
dist/index.js after bundling so an unloadable bundle fails the build.

When start() fails after listen (chmod, handshake), a peer validated in
between kept server.close() waiting forever; destroy and clear the
accepted sockets first, mirroring stop(). net.Server has no
closeAllConnections(), so that is the whole fix.

Tests now cover the QWEN_BROWSER_USE_SOCKET_PATH override, a hello with a
mismatched protocol version or extension id, onConnectionChange on the
real transport, an in-flight request when the extension disconnects and
the configured request budget, the ok:false error-code mapping,
FrameDecoder split-frame reassembly and its length ceiling, the
start-failure teardown, and all five schema refine predicates.

* fix(browser-use): mirror the contract bounds in the SDK and state the documented limits

nth() now rejects indexes outside the contract's -10000..10000 before
dispatch, and all() fails closed when the plan already has 32 steps or
the match count exceeds what nth() can address, instead of handing back
locators that only fail on first use. Both evaluate script builders
normalise method-shorthand and class-method sources the way Playwright
does (compile-only vm.Script probe, then a function prefix), so those
well-typed callbacks no longer die as a page-side SyntaxError.

providerTab clamps title and url to the claimTab schema's 20000-char cap
so the documented openTabs() -> claimTab() round trip survives long URLs,
and orderOpenTabs() establishes the http(s)-only, newest-first listing
the model documentation promises. downloadMedia derives the file name
from the URL path only for http(s) sources and leaves blob:/data: names
to the browser.

The model-facing documentation now states the 1s read default, that
dom_cua.scroll's x/y are wheel deltas rather than coordinates, and that
a cross-origin resource without CORS headers cannot be downloaded from
the page origin; the design docs record that boundary as follow-up work.
Tests pin every one of these, plus buildLocator's frame, combinator,
filter and positional branches, the file-chooser SDK surface, and the
claimTab argument guards.

* fix(browser-use): keep background tab registration from moving the selection

registerTab selected every tab it registered, and two of its callers are
background paths (derivedTabTracked, syncDerivedTabs via tabs.list), so
a read-only tabs.list could move tabs.selected onto a popup. Selection is
now set only by newTab and claimTab from the info they return.

Also: tabs.get is allowed while a dialog is open, matching tabs.list and
tabs.selected, which already return the dialog-safe TabInfo; the
auxiliary-button click moves Playwright's tracked cursor before sending
the CDP button events so a later wheel lands where the click did; the
dead providerTabId lookup is removed; openTabs applies orderOpenTabs.

Tests: the runtime test file mocks node:timers so vi.useFakeTimers()
actually controls the input drain bound (the 250ms test drops to ~0ms),
and new cases pin the drain clearTimeout, the registration probe's
session detach, promptText pass-through, tabs.get under a dialog, the
openTabs/claimTab fail-closed guards and ordering, popup adoption from
derivedTabTracked, the abandoned navigation waiter's swallowed rejection,
the uploadFiles path guards, and the screencast fallback and byte budget
on the unconstrained capture path.

* test(browser-use): pin the element bindings the runtime gives evaluate scripts

The SDK emits scripts that read the free variables `element` and
`elements`, and the runtime binds them when it compiles the script inside
the page; no test ran either page-side callback, so swapping the binding
name on one side kept the suite green while every locator.evaluate would
throw a ReferenceError in production. Run the real callbacks with fake
elements so the contract is pinned from both ends.

* fix(browser-use): fail closed on page-controlled strict-mode text and prefer contained media over anchor hrefs

* fix(browser-use): bound page title reads and download a located anchor's file over its icon

* fix(browser-use): correct media downloads and frame composition

* fix(browser-use): bound locator.count and allTextContents by the caller deadline

* fix(browser-use): preserve native snapshots for DOM CUA

Co-authored-by: Qwen-Coder <[email protected]>

* fix(browser-use): classify a closed tab from the tab's own state

The pinned playwright-core client renders a closed target as a plain Error:
only TimeoutError sets `name`, so the `name === 'TargetClosedError'` branch
in sanitizeOperationError never fired, and because evaluate-channel text
fails closed by design, a tab that closed or crashed while a page command
ran was reported as OPERATION_FAILED instead of the documented STALE_TAB.

Decide tab-gone in the dispatcher from `tab.stale` and `page.isClosed()`,
which the page observers keep current, before the text classifier runs.
Drop the dead name branch and replace the test that fabricated the name
with one built the way the client builds the error, plus dispatch-level
cases for both signals that fail without the dispatcher check.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* fix(browser-use): restore the kernel realm process binding and land the round-5 review batch

Criticals from review round 5, each reproduced before fixing:

- The bundled skill imports the staged runtime inside the node_repl kernel,
  whose untrusted vm context has no `process` global and denies the
  `node:process` specifier. 3fc2dc3e10 removed the bundle banner that bound
  `process` through createRequire, so the skill's first cell threw
  `ReferenceError: process is not defined`; the build's own load check runs
  in the main realm and could not see it. Restore the binding on the index
  build and add a kernel-backed test that runs SKILL.md's first cell through
  the real kernel and fails on the ReferenceError (R5-4).
- Bump @qwen-code/node-repl-mcp to 0.1.6: this PR changes its published
  source (image metadata channel) and the package is excluded from the
  repo-wide bump, so the release lane's integrity check would fail. Pin the
  browser-use skill to that version, have the skill test assert the pin
  equals the package version, and extend the cua-driver release gate to
  cross-check the browser-use pin (R5-2, R3-5).
- Report an incompatible extension as EXTENSION_VERSION_MISMATCH so
  browser discovery surfaces the update guidance instead of an empty list
  after the connect budget (R5-5).
- Make tab grouping non-fatal in tabs.create, matching its siblings, so a
  cosmetic grouping failure no longer closes the tab just created (R5-6).

Review suggestions verified still applying, with tests proven red under
mutation: typecheck the scripts directory; pin the Native Messaging chunk
boundary and envelope; return installedPaths from the installer; treat a
blank install-home override as unset; warn on a skipped foreign manifest;
refuse a foreign launcher with the refusals' missing red arms; keep the
preflight download waiter handled; settle the parked-profile test; name
the private-directory cause in the bridge start error; declare the finalize
status union once; reject unknown or closed kept tabs; pin the extension's
protocol constants to the SDK source; resolve the extension test root from
the module; drop the duplicated permission assertion; make the group
barrier real; read the overlay style rules; map a vanished tab to STALE_TAB
in the relay; clamp listed tab text to the schema bound; remove the
unreachable onDetach child-session branch; assert the raced tab in the
disconnect test; cover the history scheme filter and session-name reset;
and fix the extension-source and prerequisite wording in the docs.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* test(browser-use): let the SauceDemo smoke recognize equivalent skill usage

The model-driven smoke completed the task correctly on every run but its
transcript checks asserted one exact spelling: the literal runtime entry
path in the model's code, and one parent-scoped price selector with an
exact number of dollar values in the cell's output. A model that builds
the entry from the skill root across cells, scopes the price locator
differently, or echoes its chosen items after the read failed the smoke.

Key the import check on what every spelling shares: the built-in skill
root the CLI substitutes into SKILL.md, the entry file name and the setup
call. Match price reads on the price class, prefer a read that printed
exactly the expected count, take the leading set otherwise, and take the
last such read for the cart since the flow reads it after the inventory.
The smoke-checks test covers all three import spellings, a foreign runtime
and a missing setup call, and the parser's scoped, echoed and non-list
reads. Verified against a real Chromium 151 run: all six checks true.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* fix(browser-use): close six fail-open paths in the playwright adapter

Round-22 review found six Critical defects in the new package; each is
fixed at its root and pinned by a test that goes red without the fix:

- locator.evaluateAll opened a second full timeout window after the
  attach wait had consumed the caller's budget; the wait and the read
  now share one deadline, and a wait that consumed the budget settles
  [] without evaluating.
- tab.goto passed any string to page.goto, so a file:// URL would have
  handed local file contents to the tab's evaluate and snapshot reads;
  goto is gated to http(s) before the navigation runs.
- Synthesized CDP error responses carried no code, so a reply orphaned
  by a target crash tripped Playwright's internal assert and tore down
  the whole transport; they now carry -32001, the orphan-swallow code.
- A dialog already open when a tab attaches is replayed by neither
  Chrome nor Playwright, so the DIALOG_OPEN gate stayed blind while the
  modal blocked the renderer; registration now probes the renderer and
  gates the tab on an unanswered probe, with getJsDialog reporting it
  under a stable handle whose accept/dismiss run over CDP.
- A dialog opening Playwright never delivered stayed in the trace
  forever and swallowed the charge of every later dialog on the tab; a
  closed, undelivered entry is retired once the dispatcher turn has
  passed.
- snapshotRefLocator validated only the ref's shape, so a ref from a
  pre-navigation snapshot could act on whichever element the new
  document happened to hand the same string; refs are bound to the
  snapshot that emitted them and cleared on main-frame navigation.

Co-Authored-By: Qwen-Coder <[email protected]>

* build: exempt the staged Browser Use skill runtime from the dependency-direction check

The browser-use package build stages the bundled skill's runtime, node_modules
included, under packages/core/src/skills/bundled/browser-use/runtime (git-ignored),
and the TUI dependency-direction gate fails on any node_modules inside a protected
root. Declare that staged, ignored root as a build artifact so the gate skips it
without a diagnostic; every other skipped-directory name still fails.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01MWe8PKNMpsKdQruTWkvFFc

* build: leave the staged Browser Use runtime out of the dependency-direction walk entirely

The staged root is a build artifact, not source: skipping only its
node_modules while still parsing the SDK bundle next to it was inconsistent.
Do not walk anything under a declared staged root; every other node_modules
inside the protected roots remains a diagnostic.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01MWe8PKNMpsKdQruTWkvFFc

* build(browser-use): stage the skill runtime only for the bundle and for dev mode

The package build used to copy the bundled skill runtime, node_modules
included, into packages/core/src, which every built tree then carried and
which the TUI dependency-direction gate rightly rejects. Follow the qc-helper
convention instead: the bundle step stages the runtime into dist/bundled for
the shipped CLI, and `npm run dev` stages the git-ignored development copy
next to the source SKILL.md, because only dev mode reads bundled skills from
the source tree. No build step writes into src any more, so the gate
exemption added earlier is dropped and the check script returns to main's
version. The kernel-realm test stages its own copy into a temporary skill
directory rather than depending on a build side effect.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01MWe8PKNMpsKdQruTWkvFFc

* fix(browser-use): deliver bridge messages in arrival order and keep dev info flags fast

Three fixes from the round-3/4 verification and review round 7–8:

- Transport: a response only settles a promise, so its consumer ran on the
  microtask queue while events from the same socket chunk were emitted
  synchronously and overtook it. Playwright installs a page's renderer and
  lifecycle listeners inside `Page.getFrameTree().then(...)`, so an
  overtaking `Runtime.executionContextCreated` found no listener and a
  claimed tab lost its main world for good (B1). Inbound messages now go
  through one per-socket queue whose drain yields a macrotask after
  settling a response, so every continuation runs before the next message;
  chunks arriving meanwhile join the same queue. Regression:
  `delivers a response ahead of the events that share its socket chunk`.
- Relay: `requireControllableTab` judged a popup by `tab.url`, which is
  empty until the navigation commits, while the claim keyed on
  `pendingUrl`; the runtime's immediate `tabs.get` therefore rejected the
  tab the relay had just reported (R8-2). It now judges by
  `pendingUrl || url`, like the claim. Regression:
  `a popup known only by its pending URL stays controllable`.
- dev launcher: the dev-mode runtime staging skips `--version`/`--help`
  (and `-v`/`-h`), restoring the fast path the existing tests pin (R7-1).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01MWe8PKNMpsKdQruTWkvFFc

* test(browser-use): make the smoke evidence and the persist-before-connect pin honest

Round 6–7 review findings on the verification tooling:

- The SauceDemo smoke decided "the staged runtime was loaded" from three
  substring tests over all cells joined, so a run whose staged import threw
  and fell back to a foreign entry still passed (R6-1). It now looks at the
  import expressions per cell: a literal path under the staged runtime, or a
  path assembled from the substituted skill root, in a cell that reaches the
  setup export; a literal entry loaded from anywhere else disqualifies the
  run.
- Price reads were attributed to the cart by count and read order, so an
  inventory sorted low-to-high passed as a cart read and the fallback to the
  first read never returned null (R6-2). Reads are now attributed by the
  tab.url() confirmation of cart.html the prompt requires: inventory before
  it, cart after it, and no cart read without it.
- The persist-before-connect invariant was asserted inside the mock's
  postMessage, which the bridge calls under a bare catch, so the reordered
  mutant stayed green (R7-2). The mock now records every hello with the
  identity persisted at that moment and the test asserts the order outside
  the bridge; the mutant is red.
- SKILL.md states the node-repl-mcp 0.1.6 requirement next to the
  emitImage step the agent always reads, with how to detect and re-register
  an older server (R6-3).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01MWe8PKNMpsKdQruTWkvFFc

---------

Co-authored-by: qwen-code-dev-bot <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Qwen-Coder <[email protected]>
Co-authored-by: qwen-code-dev-bot <[email protected]>
Co-authored-by: Shaojin Wen <[email protected]>
Co-authored-by: qwen-code-ci-bot <[email protected]>
Co-authored-by: Claude Fable 5.1 <[email protected]>
This commit is contained in:
tanzhenxinClaude Fable 5.1qwen-code-dev-botgithub-actions[bot]Qwen-Coderqwen-code-dev-botShaojin Wenqwen-code-ci-bot
 authored and GitHub committed 2026-09-18 05:50:24 +00:00
1 parent 83d597102a
commit b7543aeb1b
87 files changed
+9611 -410
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/workflows/cd-cua-driver.yml
+3 -1
View File
@@ -142,9 +142,11 @@ jobs:
fi
SKILL_NODE_REPL_VERSION=$(sed -nE 's/^qwen mcp add --scope user node-repl npx -y @qwen-code\/node-repl-mcp@([^ ]+)$/\1/p' packages/core/src/skills/bundled/computer-use/SKILL.md)
USER_GUIDE_NODE_REPL_VERSION=$(sed -nE 's/^qwen mcp add --scope user node-repl npx -y @qwen-code\/node-repl-mcp@([^ ]+)$/\1/p' docs/users/features/computer-use.md)
BROWSER_USE_SKILL_NODE_REPL_VERSION=$(sed -nE 's/^qwen mcp add --scope user node-repl npx -y @qwen-code\/node-repl-mcp@([^ ]+)$/\1/p' packages/core/src/skills/bundled/browser-use/SKILL.md)
for PAIR in \
"Computer Use skill Node REPL:$SKILL_NODE_REPL_VERSION" \
"Computer Use user guide Node REPL:$USER_GUIDE_NODE_REPL_VERSION"; do
"Computer Use user guide Node REPL:$USER_GUIDE_NODE_REPL_VERSION" \
"Browser Use skill Node REPL:$BROWSER_USE_SKILL_NODE_REPL_VERSION"; do
LABEL=${PAIR%%:*}
VALUE=${PAIR#*:}
if [[ "$VALUE" != "$NODE_REPL_VERSION" ]]; then
.github/workflows/live-host-release.yml
+2 -1
View File
@@ -359,8 +359,9 @@ jobs:
- name: 'Install dependencies'
run: 'npm install --ignore-scripts'
- name: 'Build dependency chain (core → acp-bridge → sdk → qwen-live)'
- name: 'Build dependency chain (browser-use → core → acp-bridge → sdk → qwen-live)'
run: |-
npm run build --workspace @qwen-code/browser-use
npm run build --workspace @qwen-code/qwen-code-core
npm run build --workspace @qwen-code/acp-bridge
npm run build --workspace @qwen-code/sdk
.gitignore
+2
View File
@@ -129,6 +129,8 @@ coverage/
# Dev symlink: qc-helper bundled skill docs (created by scripts/dev.js)
packages/core/src/skills/bundled/qc-helper/docs
# Generated browser-use runtime (staged by scripts/dev.js)
packages/core/src/skills/bundled/browser-use/runtime/
tmp/
.worktrees/
.prettierignore
+1
View File
@@ -34,6 +34,7 @@ packages/cua-driver/
scripts/codemod/fixtures/
packages/live-host/
packages/desktop-shell/runtime/
packages/core/src/skills/bundled/browser-use/runtime/
# Vendored from mobile-next/mobile-mcp and carries its own eslint config, so
# reformatting it to house style only widens the diff against upstream on the
# next sync. eslint.config.js already ignores packages/mobile-mcp/** for the
.qwen/e2e-tests/pr-11242-follow-up.md
+38
View File
@@ -0,0 +1,38 @@
# PR #11242 review follow-up verification
Baseline: `41d11d116d10bc92b4119418e7cc87a3350b3f95`. Updated stacked base: `2bce3892b40c4f480c08ada6a7f3a79302463833`. Tested on macOS arm64, Node 22, Chrome for Testing 151.0.7922.34 with isolated profiles. The global Qwen CLI predates this relay; its version was checked, then the actual PR sources and built artifacts were used for verification.
## Regression scenarios
| Scenario | Baseline observation | Required result |
| --- | --- | --- |
| Package verification | 3 failures / 420 tests; deleted ensureSocketDirectory import breaks typecheck | Current suite and typecheck pass with prepareSocketDirectory and protocol-v2 hello fixtures |
| Smoke completion verifier | Ping precedes start; debugger conflict does not retry | Start before ping, full discovery window, retry only TAB_DEBUGGER_CONFLICT |
| Crash and re-registration | Duplicate entries close handoff or close a tab twice | Replace stale entry after successful registration, retain created ownership, clean once |
| Chrome tab disappears after crash | Finalization repeatedly fails and retains stale entry | Forget confirmed STALE_TAB; retain other failures for retry |
| Managed Chrome interrupts | SIGINT/SIGTERM orphan the detached browser and profile | Normal exit, both signals and a signal during normal cleanup remove the process group and profile |
| Managed browser discovery | Branded Chrome can be selected despite extension loading restrictions | Prefer Chromium / Chrome for Testing; reject explicit Google Chrome 137+ |
| Page-owned overlay element | Page element is adopted, modified and removed | Independent overlay node; page element survives unchanged |
| Oversized UTF-8 response | Valid 17 MiB response kills the actual native-host process | Bounded operation error, same session and subsequent response work; multibyte length is counted in bytes |
| Tab registration in Chrome extension | Page.setDownloadBehavior is rejected before navigation | Register without modifying browser download policy; existing Page download events still reach Playwright |
| Windows socket fixtures | New generic tests bind filesystem paths | Unique Windows named pipes; Windows execution remains pending |
## Package checks
From the repository root, run `npm run build`, `npm run typecheck` and `npm run bundle`. From `packages/browser-use`, run `npm run test:ci`; from `packages/chrome-extension`, run `npm run test:ci` and `npm run typecheck`. Run ESLint on changed source and test files.
Accepted results: Browser Use 21 files / 510 tests, extension 10 files / 99 tests; full root build, typecheck and bundle; targeted ESLint.
## Actual browser check
Set `QWEN_BROWSER_USE_CHROME` to a supported cached Chrome for Testing executable and execute `node packages/browser-use/dist/scripts/managed-chrome-preflight.js` after the build. The script creates its own profile, Native Messaging registration and local HTTP fixture.
Verify extension identity and protocol; new tab and claim; AI snapshot and action using a returned ref; scrolled 100x50 JPEG clip; trusted pointer/keyboard/input events; local form navigation; same-origin download; History permission; deliverable finalization and reclaim; continued usability after 65 seconds idle; final cleanup. Verify both the temporary profile and browser processes disappear on exit.
The first full run failed at the unsupported Page.setDownloadBehavior call. An independent raw-CDP Chrome extension probe then confirmed that omitting this command and enabling Page still produces matching download-start/completed events and the expected 43-byte local fixture file. The rebuilt managed preflight passed in 72.103 seconds with exit 0 and all checks true, including 65-second idle keepalive. Its temporary profile was removed and all 20 observed descendant processes exited.
## Evidence boundaries
No personal browser profile or account was used. Lifecycle/signal and large-response failure probes execute the actual sources with controlled subprocesses or browser API fixtures; these are distinct from the real managed Chrome preflight. Windows and the full model-driven public SauceDemo flow were not executed.
Real Chrome also accepted exact 31-, 1,000- and 10,000-character tab-group titles and retrieved a seven-day-old history entry with explicit from. The omitted-from 24-hour default is now documented. Metadata-specific budgets remain a pending design decision; published Node REPL version synchronization remains explicitly deferred. PR #11241's separate cross-origin and timeout download issues were not changed.
AGENTS.md
+5 -2
View File
@@ -62,11 +62,14 @@ npm run bundle # Bundle dist/ into a single dist/cli.js via esbuild
### Development
```bash
npm run dev # Run CLI directly from TypeScript source (no build needed)
npm run dev # Run CLI from TypeScript source
```
Runs the CLI via `tsx` with `DEV=true`. Changes to `packages/core` or
`packages/cli` are reflected immediately without rebuilding.
`packages/cli` are reflected immediately without rebuilding. Browser Use reuses
its built runtime, like other compiled workspace dependencies. After editing
Browser Use, run `npm run build --workspace=@qwen-code/browser-use` to rebuild
and stage its runtime for dev. Normal `npm install` prepares it automatically.
### Unit Testing
docs/design/browser-use-profile-ownership.md
+88
View File
@@ -0,0 +1,88 @@
# Browser Use profile ownership
[English](browser-use-profile-ownership.md) | [简体中文](browser-use-profile-ownership.zh-CN.md)
## Problem and scope
Multiple Chrome profiles can load the Qwen extension and share one Native Host
registration. Each connects independently to the same Browser Use socket.
Previously, every handshake replaced the current connection, invalidating tabs
and routing later requests to another profile. A single Browser Use session
could therefore fail repeatedly as profiles reconnected.
This change retains the [single-session boundary](browser-use.md): one active
Browser Use runtime per OS user. It fixes profile contention without adding a
profile picker or concurrent Browser Use sessions.
## Identity and selection
Each extension profile persists a random UUID in `chrome.storage.local` under
`browserUseInstanceId`. It writes the value before connecting and reuses it
after Native Host, service worker, and browser restarts. A failed storage write
prevents that connection attempt; the existing reconnect alarm retries it.
Clearing extension data or reinstalling can change the identity.
Protocol version `2` requires `extensionInstanceId` in `hello`, alongside the
shared `extensionId`. The transport rejects missing or invalid instance IDs
before selection. Extension and runtime must be updated together; version `1`
handshakes cannot safely identify profiles and are rejected. Native Host remains
an opaque relay. These identifiers distinguish compatible instances; they do
not authenticate other processes running as the same OS user.
During discovery, an incompatible Qwen extension does not prevent a compatible
profile from connecting within the normal connection timeout. If none connects,
the request fails with `EXTENSION_VERSION_MISMATCH`, whose message identifies
whether the extension or Qwen Code needs updating. Browser discovery surfaces
this error instead of reporting an empty browser list, which it reports only
for a plain `BROWSER_DISCONNECTED`. After a profile is selected, only that
disconnected profile can supply this diagnostic; unrelated profiles cannot
override it. A compatible connection or stopping the transport clears the
diagnostic.
The first valid handshake selects a profile for the current transport lifetime.
Initial selection depends on connection order, not a prediction of the user's
preferred work or personal account. Other profiles remain connected but idle.
Their messages cannot resolve active requests, emit events to the SDK, replace
the selected socket, or reset its tabs and session name.
## Disconnect and stop
A disconnect fails the selected connection's pending requests and invalidates
its tab handles using the existing lifecycle. It retains the selected instance
identity. Only that instance can reconnect; another connected profile cannot
take over while it is absent. Requests time out with `BROWSER_DISCONNECTED` if
the selected profile does not return, or with `EXTENSION_VERSION_MISMATCH` if
it returned with an incompatible protocol version. Existing pages may be
rediscovered and claimed after reconnection; old handles are not transparently
restored.
Stopping the transport closes active and idle connections and clears selection.
A new transport lifetime can select a different profile. The extension's
persistent instance identity is not cleared by stopping a Browser Use session.
## Validation and acceptance
- Reproduce two profiles exposing the same tab ID; connecting the second must
not change where the first profile's request is sent.
- Pending requests and events remain isolated from idle profiles, and stopping
an idle connection does not notify the SDK that its browser disconnected.
- Two real isolated Chrome profiles share the same socket for at least 95
seconds without invalidating the original tab or resetting its group name.
- Closing and reopening the idle profile leaves the active profile usable.
Closing the active profile never fails over; restarting it permits recovery.
- An identity survives extension reconnects and worker restarts, differs across
profiles, and is persisted before use. A storage failure must not publish a
temporary identity.
- Stopping and restarting the transport permits a new selection and closes all
old connections. The existing second-runtime `BROWSER_USE_BUSY` behavior stays
in place.
## Concurrent-session follow-up
Tracked in [#11609](https://github.com/QwenLM/qwen-code/issues/11609).
Profile identity and Browser Use session identity are separate concepts. Future
sessions may share one profile while owning different tabs. That work requires
shared connection routing, session-scoped tab ownership, events, cleanup and
reconnection. It must not reuse CDP child-target `sessionId` as the Browser Use
session identity. This fix does not implement those mechanisms.
docs/design/browser-use-profile-ownership.zh-CN.md
+71
View File
@@ -0,0 +1,71 @@
# Browser Use 的 profile 归属
[English](browser-use-profile-ownership.md) | [简体中文](browser-use-profile-ownership.zh-CN.md)
## 问题与范围
多个 Chrome profile 可以安装 Qwen 扩展并共享同一份 Native Host 注册。
各 profile 独立连接同一个 Browser Use socket。此前，每次握手都会替换当前连接，
使标签页失效，并把后续请求发往另一个 profile。因此，即使只有一个 Browser Use
session，也会在多个 profile 重连时反复失败。
本次保持[单 session 边界](browser-use.md)：每个 OS 用户只能运行一个活跃的
Browser Use runtime。此次只解决 profile 抢占，不增加 profile 选择界面或并发
Browser Use session。
## 身份与选择
每个扩展 profile 在 `chrome.storage.local` 的 `browserUseInstanceId` 中持久保存
一个随机 UUID。连接之前先写入，Native Host、service worker 和浏览器重启后复用。
如果写入失败，本次不连接，由现有重连 alarm 重试。清除扩展数据或重新安装可能改变身份。
协议版本 `2` 要求 `hello` 除了共享的 `extensionId`，还必须包含
`extensionInstanceId`。传输层在选择 profile 前拒绝缺失或无效的实例 ID。
扩展与 runtime 必须一起更新；版本 `1` 的握手无法可靠区分 profile，因此会被拒绝。
Native Host 继续只转发消息。这些标识用于区分兼容的实例，不能认证以同一 OS 用户
身份运行的其他进程。
发现连接时，不兼容的 Qwen 扩展不会阻止兼容 profile 在正常连接超时窗口内接入。
如果没有兼容连接，请求以 `EXTENSION_VERSION_MISMATCH` 失败，其消息会明确指出
需要更新扩展还是 Qwen Code。浏览器发现会直接报告该错误，而不是返回空的浏览器
列表；只有普通的 `BROWSER_DISCONNECTED` 才会被报告为空列表。选定 profile 后，
只有该 profile 断线后的握手可以提供这一诊断，其他 profile 不能覆盖它。
兼容连接建立或停止 transport 时会清除诊断。
第一个有效握手选定当前 transport 生命周期使用的 profile。首次选择取决于连接顺序，
不会推测用户偏好的工作或个人账号。其他 profile 保持连接并待命。
它们的消息不能完成当前请求、向 SDK 发送事件、替换选中的 socket，或重置其标签页
和 session 名称。
## 断开与停止
连接断开时，按照现有生命周期让该连接的待完成请求失败，并使标签页句柄失效，
但保留选中的实例身份。只有同一实例可以重连；原实例不在线时，其他已连接 profile
也不能接管。如果选中的 profile 未返回，请求超时后报告 `BROWSER_DISCONNECTED`；
如果它以不兼容的协议版本返回，则报告 `EXTENSION_VERSION_MISMATCH`。
重连后可以重新发现并认领已有页面；旧句柄不会透明恢复。
停止 transport 会关闭活跃连接和待命连接，并清除选择。新的 transport 生命周期
可以选择另一个 profile。停止 Browser Use session 不会清除扩展持久保存的实例身份。
## 验证与验收
- 复现两个 profile 具有相同 tab ID 的场景；第二个连接不能改变原 profile 请求的去向。
- 待完成请求与事件必须和待命 profile 隔离，关闭待命连接不能通知 SDK 当前浏览器已断开。
- 两个真实的隔离 Chrome profile 共用同一 socket 至少 95 秒，原标签页不失效，
分组名称不重置。
- 关闭并重启待命 profile 不影响活跃 profile；关闭活跃 profile 不自动切换，
重启原 profile 后可以恢复使用。
- 扩展重连和 worker 重启后身份不变，不同 profile 的身份不同，且使用前已持久保存。
存储失败时不能发布临时身份。
- 停止并重启 transport 可以重新选择，且所有旧连接均被关闭。第二个 runtime 仍报告
`BROWSER_USE_BUSY`。
## 并发 session 后续工作
由 [#11609](https://github.com/QwenLM/qwen-code/issues/11609) 跟进。
Profile 身份与 Browser Use session 身份是两个概念。未来多个 session 可以共享
一个 profile，并分别拥有不同的标签页。这需要实现共享连接路由，以及按 session
隔离的标签页归属、事件、清理和重连。不能复用 CDP 子目标的 `sessionId` 作为
Browser Use session 身份。本次不实现这些机制。
docs/design/browser-use.md
+191 -25
View File
@@ -10,7 +10,7 @@ Chrome from Qwen Code.
- The Browser SDK runs as a library inside the persistent Node Kernel exposed
by the Node REPL MCP server.
- `playwright-core` provides standard browser automation semantics.
- The Qwen extension and `chrome.debugger` connect the runtime to Chrome.
- The Qwen Chrome extension and `chrome.debugger` connect the runtime to Chrome.
- The first release supports one active Browser Use session.
## Architecture
@@ -45,6 +45,66 @@ Playwright's browser-level CDP adapter.
`chromium.connectOverCDP(transport)`. Qwen therefore keeps Native Messaging and
does not add a local WebSocket server.
Browser Use ships with Qwen Code as a bundled skill and its runtime resources.
No separate Browser Use runtime package or second Qwen extension is required;
the existing Qwen Chrome extension is reused. The skill's `runtime/` directory
contains the Browser SDK, Native Host, and pinned Playwright dependency. The
skill registers `runtime/node_modules` with the existing Node
REPL and imports `runtime/index.js`; the CLI does not execute browser logic.
The kernel evaluates that import in an isolated realm without Node globals, so
the runtime bundle binds `process` itself at its top through `createRequire`;
the build's load check runs in the main realm and cannot detect a missing
binding, so a kernel-backed test pins it. Source development, transpiled builds, and the published CLI use this same
layout. The generic Node REPL MCP server must be configured, and the Qwen
Chrome extension must be installed in the browser. Bundling does not connect
to Chrome at CLI startup; the SDK connects when first used.
No build step writes this runtime into the source tree. The bundle step copies
it into `dist/bundled/browser-use/runtime` for the shipped CLI, and `npm run
dev` copies the same files next to the source `SKILL.md` (a git-ignored
location) because only development mode reads bundled skills from the source
tree; the normal installation `prepare` hook builds the package those copies
come from. After changing Browser Use sources or dependencies, run
`npm run build --workspace=@qwen-code/browser-use` and start `npm run dev`
again to refresh the development copy. CLI and Core continue to run directly
from TypeScript source. If the runtime is missing, the skill's existing setup
check reports the incomplete runtime when invoked.
Browser Use is available to the model by default and is selected according to
the user's task. Users can disable it through `/skills` or `skills.disabled`,
using the same controls as Computer Use. Disabled skills are excluded
from model discovery and skill invocation. This is the existing generic skill
mechanism used by Computer Use, not a browser permission boundary: disabling
the skill does not unload instructions already in a conversation or disconnect
an existing SDK session.
Native Host registration is native-side product setup, not a Chrome-extension
operation. On macOS and Linux, the first Browser runtime initialization
checks Google Chrome, Chrome for Testing, and Chromium's standard `Default`
and `Profile N` profiles for the Qwen extension. It reads the extension's
registration in `Secure Preferences` or `Preferences` and confirms that its
manifest exists, supporting both packaged and unpacked installations. Leftover
extension directories alone do not count as an installed extension. If the
extension is not found, initialization reports how to install it without
writing Native Host files. After detection, initialization idempotently
installs the launcher and manifests for existing browser roots. A configured
`QWEN_BROWSER_USE_SOCKET_PATH` keeps using its externally managed setup.
Installing the Qwen Chrome extension opts into this automatic local setup on
first use. The launcher and Native Messaging registrations persist after
Qwen exits. The installer refuses to overwrite
foreign files: a conflicting launcher aborts initialization, while a
conflicting browser manifest is skipped and named in a warning on stderr.
Running
`node <skill-base>/runtime/scripts/native-host-setup.js uninstall` removes
files owned by Browser Use; `status` checks them and `install` explicitly
registers them. To prevent automatic registration on a later Browser Use
initialization, also uninstall the Chrome extension. Only a missing file is
treated as absent; other
read failures abort the operation without overwriting the unreadable file. The
Chrome extension only opens the registered host
through `connectNative()`.
## Responsibilities
| Component | Responsibility |
@@ -94,6 +154,16 @@ styles:
`tab.playwright` is used when an element can be described semantically.
`tab.dom_cua` is used when the model identifies an element in a DOM snapshot.
`tab.cua` is used when the target is identified visually in a screenshot.
The extension renders a transient pointer overlay for coordinate mouse input,
but that decoration is best-effort and never delays the input command itself.
Its DOM node is created on mouse input and removed when the pointer expires;
read-only inspection does not create an overlay node. The extension creates
and retains its own node instead of adopting a page element with the same ID.
Browser operations run in the background. New tabs do not replace the user's
active tab, and input actions do not bring Chrome to the foreground. Page focus
emulation keeps background rendering and input active without changing desktop
focus.
Page and locator evaluation accept functions or strings. Functions are invoked
with their documented arguments; strings return their JavaScript `eval`
@@ -153,16 +223,23 @@ Snapshot truncation, screenshot encoding and budgets, stale-session detection,
and the JSON transport envelope remain runtime implementation details rather
than model-facing options.
Viewport screenshots return an image object accepted by `nodeRepl.emitImage()`.
Its metadata carries the original JPEG dimensions, viewport, device pixel ratio,
and CSS-pixel coordinate space so visual coordinates remain usable when a model
client resizes the preview. Viewport screenshots are limited by their encoded
byte size rather than rejected from viewport dimensions alone. Explicit clips
and full-page captures retain a pixel budget because their dimensions are
caller-controlled or potentially unbounded. The device pixel ratio probed
from page script is trusted only within the range a real Chrome window can
report; when a capture's pixels disagree with the requested CSS region, the
runtime derives the real ratio from Chrome's own output and re-captures once.
Viewport screenshots return JPEG bytes, a MIME type, and metadata carrying the
original image dimensions, viewport, device pixel ratio, and CSS-pixel coordinate
space so visual coordinates remain usable when a model client resizes the
preview. The skill passes the complete screenshot to `nodeRepl.emitImage()`.
Metadata travels on the image event and is returned immediately before each
retained image, independently of the ordinary text output budget. Rejected or
omitted images do not leave metadata behind. There is no metadata-specific size
cap; the existing protocol-frame and client output limits still apply.
Bundling the Node REPL MCP server with Qwen Code is deferred to a follow-up.
This protocol support first ships in `@qwen-code/node-repl-mcp` 0.1.6 and is
absent from the published 0.1.2 through 0.1.5 packages, so the Browser Use
skill pins that version instead of `latest`.
Viewport screenshots are limited by their encoded byte size rather than rejected
from viewport dimensions alone. Explicit clips and full-page captures retain a
pixel budget because their dimensions are caller-controlled or potentially
unbounded.
The device pixel ratio probed from page script is trusted only within the range a real Chrome window can report; when a capture's pixels disagree with the requested CSS region, the runtime derives the real ratio from Chrome's own output and re-captures once.
Screenshot acquisition follows the Codex Browser Use strategy independently of
Playwright's screenshot preparation. A short, bounded rendering synchronization
@@ -223,10 +300,22 @@ product caller requires it.
Chrome reports downloads from an extension debugger target as `Page` events,
while Playwright consumes the corresponding browser-level events. The
transport translates only those event names and preserves their payloads; it
does not maintain a separate download state machine.
does not maintain a separate download state machine. Tab registration leaves
Chrome's download policy unchanged: the extension cannot call
`Page.setDownloadBehavior`, and the page events arrive after Playwright enables
the Page domain without that command.
The Qwen control plane retains operations that are not CDP, including
`openTabs`, `claimTab`, `session.name`, and `history.query`.
History defaults to Chrome's last 24 hours when `from` is omitted; an explicit
`from` includes older visits without requiring `to`.
Native Host messages sent to Chrome are limited to 1 MiB. Larger
backend-to-extension messages are split into bounded protocol chunks and
reassembled by the extension before dispatch. Extension responses must fit the
16 MiB bridge frame limit, measured as serialized UTF-8 bytes. An oversized
operation result returns a bounded `OPERATION_FAILED` response so the Native
Host and the browser session remain available for subsequent requests.
## Session model
@@ -235,18 +324,69 @@ The Node Kernel directly owns the local Chrome extension transport:
- one Browser Use session may be active for the current OS user;
- one session may control multiple tabs;
- a second session fails with `BROWSER_USE_BUSY`;
- closing the session detaches its tabs and releases the local socket;
- closing the session closes still-controlled agent-created tabs, including
handoffs, releases claimed tabs, and then releases the local socket;
- a transport disconnect invalidates the current Playwright connection;
- tab-scoped objects from the disconnected connection fail with
`STALE_BROWSER_SESSION` and are never silently rebound.
- closing and reinitializing Browser Use creates a new SDK object generation;
handles retained from the closed generation remain stale.
When the backend session disappears, Native Host socket loss is reported to the
extension even if the Chrome Native Messaging port remains connected for
retry. The extension detaches the session's controlled tabs, removes Browser
Use overlays, clears ownership and derived-tab state, and ungroups managed tabs
without closing them.
On Unix, both endpoints prefer an existing private, user-owned
`/run/user/<uid>/bridge.sock`; otherwise they use
`/tmp/qwen-browser-use-<uid>/bridge.sock` (`/private/tmp` on macOS). The
backend creates a user-owned directory with mode `0700` and a socket with mode
`0600`. Both endpoints reject unsafe ownership, permissions, and replaceable
ancestors; the Native Host also rejects socket symlinks before forwarding any
traffic. An explicit socket override must use the same private-directory
boundary. Same-user processes remain inside the trust boundary.
When no backend is listening, the Native Host exits. The extension schedules
one retry using a 30-second Chrome alarm, which survives worker suspension;
it does not run a one-second retry loop or rewrite empty session state on
failed discovery. Initial backend discovery can wait up to 35 seconds, with
the normal request execution timeout starting after connection. Browser
listing and selection both allow this discovery window; explicit short
transport request timeouts still cap discovery.
The active `runtime.connectNative()` port keeps the worker alive on Chrome 105
and later, and an active `chrome.debugger` session provides an additional
keepalive on Chrome 118 and later. This differs from the separate `/cdp`
WebSocket bridge and follows Chrome's documented extension service-worker
lifecycle. A real-Chrome session must remain usable after more than 60 seconds
without Browser Use traffic.
When the backend socket disappears after connecting, the Native Host exits and
Chrome closes its Native Messaging port. The extension handles that port
disconnect by detaching the session's controlled tabs, removing Browser Use
overlays, clearing ownership and derived-tab state, ungrouping managed tabs
without closing them, and scheduling Native Host discovery for a future backend.
Debugger attach and detach operations are serialized per tab. A successful
release waits for Chrome to complete detach; a timeout in disconnect cleanup
does not discard an unfinished per-tab operation. If new-tab initialization
fails, the extension removes that newly created tab. Explicit user cancellation
of debugging releases ownership and the derived relationship, persists that
state, and ungroups the tab on a best-effort basis.
At the end of a browser turn, `tabs.finalize()` treats `keep` as the complete
set for that call: it closes unlisted agent-created tabs and releases unlisted
claimed tabs. Deliverable tabs remain open but are released; handoff tabs
remain open and controlled until the next finalization or runtime close. A
handoff that is still needed must be included again in the next turn. An
agent-created popup keeps that ownership if its opener closes before
finalization. The extension is the source of browser-side ownership, while the
runtime keeps the corresponding session projection; agent-created ownership
takes precedence if a derived tab is observed through both paths.
Re-registering a crashed Chrome tab replaces its stale runtime entry and
preserves that ownership. Cleanup forgets a tab already removed in Chrome,
while other close or release failures retain the entry for retry.
`tabs.finalize()` validates the complete `keep` set before closing anything. An
unknown, stale, or duplicate entry aborts finalization so a malformed keep list
cannot accidentally close a page the model intended to preserve.
Derived-tab synchronization attempts each attachment independently. If one
attachment or the discovery query fails, finalization still cleans up the other
known tabs according to their dispositions, then reports the failure. A failed
attachment does not authorize closing an unregistered tab.
The first release adds no separate Browser Use authorization or process
authentication layer.
@@ -287,11 +427,27 @@ that takes an AI snapshot and acts on one of its returned refs. Existing
workspace consumers remain on their current Playwright versions; this feature
does not require a repository-wide upgrade.
Managed smoke scripts use Chromium or Chrome for Testing. Automatic discovery
excludes branded Google Chrome, and an explicit Google Chrome 137+ executable
is rejected because it cannot load the unpacked extension. Normal completion,
SIGINT, and SIGTERM stop the managed browser process group and remove its
temporary profile before the script exits.
The managed preflight validates screenshot MIME type and decoded JPEG clip
dimensions. The SauceDemo smoke checks checkout state and prices; source-code
mentions of input or finalization methods are not evidence that those actions
ran, so its result does not claim to verify trusted input or tab finalization.
Its independent completion check starts the transport, allows the full
35-second discovery window, and retries only `TAB_DEBUGGER_CONFLICT` while the
previous session releases its debugger attachment.
## Product decisions
For the first release:
- installing the Qwen Chrome extension authorizes Browser Use;
- first use on macOS or Linux automatically registers the Native Host without
a separate prompt;
- Browser Use may enumerate and claim top-level HTTP(S) tabs by default, and
`tab.goto` navigates claimed tabs to http(s) URLs only;
- History is declared with the other required extension permissions; there is
@@ -299,18 +455,25 @@ For the first release:
- there is no Browser Use-specific origin allowlist, upload-root allowlist, or
snapshot redaction in this release;
- the existing Qwen toolbar action and side panel remain;
- there is no separate Browser Use enable/disable switch yet.
- the model can discover Browser Use by default; users can disable it through
the existing `/skills` controls.
## Current boundaries and future work
- **Sessions:** One Browser Use session may be active per OS user, and that
session may control multiple tabs. Future support for concurrent sessions
must isolate tab ownership, event routing, cleanup, and reconnect behavior.
- **Turn cleanup:** Finalization is an explicit final browser action. Closing
the runtime provides a fallback, but interrupting a model turn does not close
the persistent runtime; still-controlled tabs remain managed until a later
finalization or runtime close. A transport disconnect releases them without
closing their pages. A future Qwen turn-lifecycle hook should invoke
finalization independently of model behavior.
- **Browser backends:** The Qwen extension currently connects the SDK to
Chrome. Other browser families or an in-app browser should be added together
with capability discovery when products need them.
- **Product control:** Add Qwen-owned opt-in and authenticate the local
connection.
- **Product control:** Authenticate the local connection. Skill enablement
controls availability, not direct SDK access or active browser sessions.
- **History:** Make Chrome History optional through a Qwen-owned grant and
revoke flow outside the side panel.
- **Platform and optional APIs:** Native Host installation currently supports
@@ -369,10 +532,13 @@ Explicit CDP session detach emits the parent-scoped target-detached event that
Playwright uses to dispose its session listeners.
Stopping the runtime unsubscribes session listeners, drains tab registration,
and awaits transport cleanup before stopping the bridge. Page close and crash
release through the transport that registered the page. Reconnecting waits for
the previous transport cleanup so old releases cannot detach newly claimed
tabs. A request cannot implicitly restart a stopped bridge.
finalizes controlled tabs, and awaits transport cleanup before stopping the
bridge. Shutdown closes extension-reported derived tabs without admitting new
debugger attachments. Page close and crash release through the transport that
registered the page; a crashed tab retains its ownership until it is closed or
finalized. Reconnecting waits for the previous transport cleanup so old releases
cannot detach newly claimed tabs. A request cannot implicitly restart a stopped
bridge.
The adapter supplies a stable default-context id when CDP omits its optional
browserContextId, preserving supplied ids and rejecting malformed values
docs/design/browser-use.zh-CN.md
+39 -9
View File
@@ -8,7 +8,7 @@ Browser Use 为模型提供结构化 API，使其能够从 Qwen Code 控制用
- Browser SDK 作为库运行在 Node REPL MCP server 暴露的持久 Node Kernel 内。
- `playwright-core` 提供标准浏览器自动化语义。
- Qwen 扩展和 `chrome.debugger` 将运行时连接到 Chrome。
- Qwen Chrome 扩展和 `chrome.debugger` 将运行时连接到 Chrome。
- 首个版本支持一个活跃的 Browser Use 会话。
## 架构
@@ -39,6 +39,16 @@ flowchart TB
`[email protected]` 通过 `chromium.connectOverCDP(transport)` 接受公共自定义 CDP transport。因此 Qwen 保留 Native Messaging，不添加本地 WebSocket server。
Browser Use 以内置 skill 及其运行时资源随 Qwen Code 一起发布，无需单独安装 Browser Use 运行时包或第二个 Qwen 扩展，复用现有的 Qwen Chrome 扩展。Skill 的 `runtime/` 目录包含 Browser SDK、Native Host 和固定版本的 Playwright 依赖。Skill 向现有 Node REPL 注册 `runtime/node_modules` 并导入 `runtime/index.js`；CLI 本身不执行浏览器逻辑。kernel 在没有 Node 全局对象的隔离 realm 中执行该导入，因此运行时 bundle 在开头通过 `createRequire` 自行绑定 `process`；构建时的加载检查运行在主 realm，无法发现缺失的绑定，故由一个基于 kernel 的测试固定该行为。源码开发、转译构建和发布的 CLI 使用同一布局。仍需配置通用 Node REPL MCP server，并在浏览器安装 Qwen Chrome 扩展。内置资源不会在 CLI 启动时连接 Chrome；SDK 在首次使用时才连接。
没有任何构建步骤会把该运行时写进源码树：打包步骤把它复制到 `dist/bundled/browser-use/runtime` 供发布的 CLI 使用；`npm run dev` 把同样的文件复制到源码 `SKILL.md` 旁边（git 忽略的位置），因为只有开发模式会从源码树读取内置 skill；正常安装的 `prepare` hook 负责构建这些副本的来源包。修改 Browser Use 源码或依赖后，运行 `npm run build --workspace=@qwen-code/browser-use` 并重新启动 `npm run dev` 以刷新开发副本；CLI 和 Core 仍直接运行 TypeScript 源码。若运行时缺失，调用 skill 时已有的 setup 检查会报告资源不完整。
Browser Use 默认对模型可用，由模型根据用户任务选择。用户可通过 `/skills` 或 `skills.disabled` 禁用，使用与 Computer Use 相同的控制方式。禁用的 skill 不参与模型发现和 skill 调用。这是 Computer Use 也使用的通用 skill 机制，不是浏览器权限边界：禁用 skill 不会移除已有对话中的指令，也不会断开现有 SDK 会话。
Native Host 注册属于本机产品初始化，不由 Chrome 扩展执行。在 macOS 和 Linux 上，Browser runtime 首次初始化时检查 Google Chrome、Chrome for Testing 和 Chromium 的标准 `Default`、`Profile N` profile 中是否安装 Qwen 扩展。它读取 `Secure Preferences` 或 `Preferences` 中的扩展注册信息，并确认 manifest 存在，同时支持打包和解压安装。仅有残留扩展目录不视为已安装。若未找到扩展，初始化会提示安装方式，不写入 Native Host 文件。检测成功后，为已存在的浏览器根目录幂等安装 launcher 和 manifest。配置了 `QWEN_BROWSER_USE_SOCKET_PATH` 时继续使用外部管理的安装。
安装 Qwen Chrome 扩展即同意首次使用时自动完成上述本机配置。Qwen 退出后，launcher 和 Native Messaging 注册仍然保留。安装器拒绝覆盖其他程序的文件：launcher 冲突会终止初始化，浏览器 manifest 冲突则跳过该 manifest，并在 stderr 上给出指明该文件的警告。运行 `node <skill-base>/runtime/scripts/native-host-setup.js uninstall` 可删除 Browser Use 拥有的文件；`status` 检查这些文件，`install` 显式注册。若要阻止后续 Browser Use 初始化时自动注册，还需卸载 Chrome 扩展。只有文件不存在才视为缺失；其他读取失败会终止操作，不覆盖无法读取的文件。Chrome 扩展仅通过 `connectNative()` 打开已注册的 host。
## 职责
| 组件 | 职责 |
@@ -72,6 +82,10 @@ SDK 与浏览器运行时共用内部命令契约。SDK 对象将模型调用转
可以用语义描述元素时使用 `tab.playwright`；模型从 DOM 快照识别目标时使用 `tab.dom_cua`；从截图视觉定位目标时使用 `tab.cua`。
扩展为坐标鼠标输入显示短暂的指针 overlay，但该装饰仅尽力提供，绝不延迟输入命令本身。其 DOM 节点在鼠标输入时创建，指针到期后移除；只读检查不会创建 overlay 节点。扩展创建并持有自己的节点，不会接管页面中同 ID 的元素。
浏览器操作在后台运行。新标签页不会替换用户当前的活跃标签页，输入操作也不会将 Chrome 置于前台。页面焦点模拟使后台渲染和输入保持活跃，而不改变桌面焦点。
页面和 locator 的 evaluation 接受函数或字符串。函数按文档规定的参数调用；字符串返回 JavaScript `eval` 的完成值，支持尾部分号、注释和语句。字符串求值保留 SDK 的词法绑定 `arg`、`element` 和 `elements`。值为函数的字符串不会被调用。需要 `await` 时使用异步函数参数，在字符串中使用对象字面量时加括号。Evaluation 的超时覆盖整个调用，包括元素查找，并返回 `OPERATION_TIMEOUT`。显式操作超时必须为 1 到 120,000 ms 的整数；零值以 `INVALID_ARGUMENT` 拒绝。省略超时时保留各操作的默认值。仅用于延时的 `waitForTimeout` 接受 0 到 120,000 ms，其中零表示不等待。超时结束调用方的等待，不会终止已在页面运行的 JavaScript。
Evaluation 接受 JSON 数据：有限数、字符串、布尔值、null、数组和普通记录，包括 TypeScript readonly 数据。公开的 `JsonSerializable` 类型约束参数和结果。非 JSON 参数在 dispatch 前拒绝；结果在页面内、Playwright 序列化之前校验。运行时传输编码后的 JSON 文本，使 `__proto__` 等对象键保留数据含义。嵌套 undefined、非有限数、Date、RegExp、函数和循环引用会被拒绝，不会静默转换。省略的顶层参数保持 undefined；顶层 undefined 结果转换为 null，返回类型也反映这一行为。类型为 void 的回调可能只是丢弃了实际返回值，因此其返回类型使用更宽的 JsonSerializable，不承诺一定为 null。
@@ -84,7 +98,7 @@ Locator plan 的每个数组最多包含 32 步，最多嵌套 32 层，顶层
Playwright 公共 CDP session API 提供坐标 CUA 的按钮 4（后退）和 5（前进）；较高层的 Playwright mouse API 不暴露它们。快照截断、截图编码和预算、会话失效检测及 JSON 传输封装属于运行时实现细节，不作为面向模型的选项。
视口截图返回 `nodeRepl.emitImage()` 可接受的图像对象。元数据包含原始 JPEG 尺寸、视口、设备像素比和 CSS 像素坐标空间，使模型客户端缩放预览后视觉坐标仍可用。视口截图根据编码字节数限制，不会仅因视口尺寸而拒绝。显式 clip 和整页截图保留像素预算，因为其尺寸由调用方控制或可能无界。从页面脚本探测到的设备像素比只在真实 Chrome 窗口可能报告的范围内被信任；当截图像素与请求的 CSS 区域不一致时，运行时根据 Chrome 自身的输出推算真实像素比并重拍一次。
视口截图返回 JPEG 字节、MIME 类型及元数据。元数据包含原始图像尺寸、视口、设备像素比和 CSS 像素坐标空间，使模型客户端缩放预览后视觉坐标仍可用。Skill 将完整截图传给 `nodeRepl.emitImage()`。元数据随图像事件传递，紧邻每张保留图像之前返回，独立于普通文本输出预算。被拒绝或省略的图像不会遗留元数据。没有元数据专用大小上限；已有协议帧和客户端输出限制仍然适用。把 Node REPL MCP server 随 Qwen Code 一起打包留待后续工作。该协议支持自 `@qwen-code/node-repl-mcp` 0.1.6 起首次发布；已发布的 0.1.2 至 0.1.5 包不具备该支持，因此 Browser Use skill 固定使用该版本而非 `latest`。视口截图根据编码字节数限制，不会仅因视口尺寸而拒绝。显式 clip 和整页截图保留像素预算，因为其尺寸由调用方控制或可能无界。从页面脚本探测到的设备像素比只在真实 Chrome 窗口可能报告的范围内被信任；当截图像素与请求的 CSS 区域不一致时，运行时根据 Chrome 自身的输出推算真实像素比并重拍一次。
截图获取采用 Codex Browser Use 策略，独立于 Playwright 的截图准备。短暂且有上限的渲染同步让待处理绘制在截图前完成。普通视口截图请求新的 CDP screencast 帧，帧期限为两秒，然后回退到命令超时为五秒的 `Page.captureScreenshot`。Clip 和整页截图直接使用后者。请求之前的旧帧会被丢弃；每个标签页的截图串行执行，事件监听器和 screencast 均会清理。运行时拥有这些事件，Playwright 不会重复确认同一帧。图像采用 JPEG quality 80，保留 CSS 像素坐标，不需要激活标签页或将 Chrome 置于前台。单次截图超时不会断开浏览器会话。
@@ -107,9 +121,11 @@ JavaScript 对话框使用类型相关操作：alert 和 before-unload 可以 di
页面级 `Page`、`Runtime`、`DOM`、`Accessibility`、`Input`、`Network`、`Fetch`、`Storage` 和 `Emulation` 命令及事件直接转发，Qwen 不重新实现它们。浏览器诊断保留有上限的 Playwright console 事件内存视图。在产品调用方提出需求前，不包含 HAR 导出。
Chrome 将扩展 debugger target 的下载报告为 `Page` 事件，而 Playwright 消费对应的浏览器级事件。Transport 仅翻译这些事件名并保留载荷，不维护单独的下载状态机。
Chrome 将扩展 debugger target 的下载报告为 `Page` 事件，而 Playwright 消费对应的浏览器级事件。Transport 仅翻译这些事件名并保留载荷，不维护单独的下载状态机。注册标签页时保留 Chrome 的下载策略：扩展无法调用 `Page.setDownloadBehavior`，而 Playwright 启用 Page 域后，无需该命令即可收到页面下载事件。
Qwen 控制面保留非 CDP 操作，包括 `openTabs`、`claimTab`、`session.name` 和 `history.query`。
Qwen 控制面保留非 CDP 操作，包括 `openTabs`、`claimTab`、`session.name` 和 `history.query`。History 省略 `from` 时沿用 Chrome 最近 24 小时的默认范围；显式传入 `from` 可查询更早的访问记录，无需同时指定 `to`。
Native Host 发往 Chrome 的消息上限为 1 MiB。较大的后端到扩展消息拆分为有界协议分块，由扩展在 dispatch 前重新组装。扩展响应必须满足 16 MiB 的 bridge 帧限制，按序列化后的 UTF-8 字节数计算。超大的操作结果返回有界的 `OPERATION_FAILED` 响应，Native Host 和浏览器会话仍可处理后续请求。
## 会话模型
@@ -118,12 +134,20 @@ Node Kernel 直接拥有本地 Chrome 扩展 transport：
- 每个操作系统用户最多有一个活跃的 Browser Use 会话；
- 一个会话可以控制多个标签页；
- 第二个会话以 `BROWSER_USE_BUSY` 失败；
- 关闭会话会 detach 其标签页并释放本地 socket；
- 关闭会话会关闭仍受控的 agent 创建标签页（包括 handoff），释放认领的标签页，然后释放本地 socket；
- Transport 断开会使当前 Playwright 连接失效；
- 断开连接的标签页对象以 `STALE_BROWSER_SESSION` 失败，绝不静默重绑；
- 关闭并重新初始化 Browser Use 会创建新一代 SDK 对象，旧一代保留的句柄继续失效。
后端会话消失时，即使 Chrome Native Messaging port 为重试仍保持连接，也会向扩展报告 Native Host socket 丢失。扩展 detach 该会话控制的标签页，移除 Browser Use overlays，清除 ownership 和派生标签页状态，并取消托管标签页分组而不关闭它们。
在 Unix 上，两端优先使用已存在、私有且由当前用户拥有的 `/run/user/<uid>/bridge.sock`；否则使用 `/tmp/qwen-browser-use-<uid>/bridge.sock`（macOS 为 `/private/tmp`）。后端创建当前用户拥有、权限为 `0700` 的目录，以及权限为 `0600` 的 socket。两端都会拒绝不安全的 ownership、权限及可被替换的祖先目录；Native Host 还会在转发流量前拒绝 socket 符号链接。显式 socket 路径覆盖也必须遵守相同的私有目录边界。同一用户的进程仍处于信任边界内。
没有后端监听时，Native Host 退出。扩展使用 30 秒 Chrome alarm 安排一次重试，可跨 worker 挂起保留；发现失败不会启动每秒重试循环或重写空会话状态。后端首次发现最多等待 35 秒，连接建立后才开始常规请求执行超时。浏览器列表和选择都允许这个发现窗口；显式指定的短 transport 请求超时仍会限制发现等待。Chrome 105 及以上的活跃 `runtime.connectNative()` port 会保持 worker 存活，Chrome 118 及以上的活跃 `chrome.debugger` 会话还提供额外保活。这与独立的 `/cdp` WebSocket bridge 不同，遵循 Chrome 文档规定的扩展 service-worker 生命周期。真实 Chrome 会话必须在超过 60 秒无 Browser Use 流量后仍可使用。
连接建立后若后端 socket 消失，Native Host 退出，Chrome 关闭其 Native Messaging port。扩展处理 port 断开时，会 detach 会话控制的标签页、移除 Browser Use overlays、清除 ownership 和派生标签页状态、取消托管标签页分组而不关闭页面，并安排 Native Host 发现以连接未来的后端。Debugger attach 和 detach 按标签页串行执行。成功释放会等待 Chrome 完成 detach；断线清理超时不会丢弃未完成的单标签页操作。新标签页初始化失败时，扩展会删除该新标签页。用户显式取消调试时，扩展释放 ownership 和派生关系，持久化状态，并尽力取消分组。
每轮浏览器操作结束时，`tabs.finalize()` 将 `keep` 视为本次调用的完整保留集合：关闭未列出的 agent 创建标签页，释放未列出的认领标签页。Deliverable 标签页保持打开但释放控制；handoff 标签页保持打开并受控，直到下一次 finalization 或关闭 runtime。下一轮仍需使用的 handoff 必须再次列入。Agent 创建的 popup 在 opener 先于 finalization 关闭时仍保留该 ownership。扩展是浏览器侧 ownership 的权威来源，runtime 维护对应的会话视图；从两条路径观察到派生标签页时，agent 创建的 ownership 优先。重新注册崩溃的 Chrome 标签页会替换其过期 runtime 条目并保留该 ownership。清理时会移除 Chrome 中已不存在标签页的条目；其他关闭或释放失败则保留条目以便重试。
`tabs.finalize()` 在关闭任何页面前校验整个 `keep` 集合。未知、过期或重复条目会终止 finalization，避免格式错误的保留列表意外关闭模型想保留的页面。派生标签页同步会独立尝试每次 attachment。某次 attachment 或发现查询失败时，finalization 仍按 disposition 清理其他已知标签页，然后报告失败。Attachment 失败不代表有权关闭尚未注册的标签页。
首个版本不添加独立的 Browser Use 授权或进程认证层。
@@ -146,22 +170,28 @@ Native Messaging 协议和扩展 relay 还承载 Qwen 特有的标签页发现
Browser Use 包独立固定 `[email protected]`，因为自定义 CDP transport API，以及 `ariaSnapshot({ mode: "ai" })` 输出与 `aria-ref` locator 的配对均与版本有关。每次升级 Playwright 都必须通过真实 Chrome 冒烟测试：生成 AI 快照，并通过其中返回的 ref 执行操作。现有 workspace 消费方保留当前 Playwright 版本；该功能不需要全仓升级。
Managed 冒烟脚本使用 Chromium 或 Chrome for Testing。自动发现不选择普通 Google Chrome；显式指定的 Google Chrome 137+ 因无法加载未打包扩展而被拒绝。正常结束、SIGINT 和 SIGTERM 均在脚本退出前停止托管浏览器进程组并删除临时 profile。
Managed preflight 验证截图 MIME 类型及 JPEG 解码后的 clip 尺寸。SauceDemo 冒烟测试检查结账状态和价格；源码中出现输入或 finalization 方法名称不能证明执行过这些操作，因此该测试结果不声称验证了可信输入或标签页 finalization。独立的完成状态检查先启动 transport，允许完整的 35 秒发现窗口，并仅在前一会话释放 debugger attachment 期间重试 `TAB_DEBUGGER_CONFLICT`。
## 产品决策
首个版本：
- 安装 Qwen Chrome 扩展即授权 Browser Use；
- macOS 和 Linux 首次使用时自动注册 Native Host，无需单独提示；
- Browser Use 默认可以枚举和认领顶层 HTTP(S) 标签页，`tab.goto` 只允许把已认领标签页导航到 http(s) URL；
- History 与其他必需扩展权限一起声明，不提供 Browser Use 权限管理 UI；
- 不提供 Browser Use 专用 origin allowlist、上传根目录 allowlist 或快照脱敏；
- 保留现有 Qwen 工具栏操作和侧边栏；
- 暂不提供独立的 Browser Use 启用／禁用开关。
- 模型默认可发现 Browser Use；用户可通过现有 `/skills` 控制禁用。
## 当前边界与后续工作
- **会话：** 每个操作系统用户最多一个活跃 Browser Use 会话，该会话可控制多个标签页。未来支持并发会话时，必须隔离标签页 ownership、事件路由、清理和重连行为。
- **轮次清理：** Finalization 是显式的最后一个浏览器操作。关闭 runtime 提供兜底，但中断模型轮次不会关闭持久 runtime；仍受控的标签页会保持托管，直到后续 finalization 或 runtime 关闭。Transport 断开会释放控制但不关闭页面。未来应由 Qwen 轮次生命周期 hook 独立于模型行为触发 finalization。
- **浏览器后端：** 当前 Qwen 扩展把 SDK 连接到 Chrome。当产品需要其他浏览器家族或内置浏览器时，应同时添加能力发现。
- **产品控制：** 添加由 Qwen 管理的用户主动启用机制，并认证本地连接。
- **产品控制：** 认证本地连接。Skill 启用状态控制可用性，不控制直接 SDK 访问或活跃浏览器会话。
- **History：** 在侧边栏之外，通过 Qwen 管理的授予和撤销流程将 Chrome History 改为可选权限。
- **平台与可选 API：** Native Host 安装目前支持 macOS 和 Linux。Windows 支持，以及剪贴板、页面资源、HAR、只读 evaluate 等可选 API，应在产品工作流需要时独立引入。
@@ -183,7 +213,7 @@ Modifier 清理会按相反顺序尝试释放所有尝试按下的键，即使 k
BrowserModel 拥有 Chrome debugger attachment，包括仍在进行中的附加。每个 provider tab 的附加和释放串行化；关闭时拒绝新的附加，并等待已接纳的工作完成后释放拥有的标签页。显式 CDP session detach 发送以父会话为作用域的 target-detached 事件，Playwright 据此释放 session 监听器。
停止运行时会取消 session 监听、排空标签页注册，并在停止 bridge 前等待 transport 清理。页面关闭和崩溃通过注册该页面的 transport 释放。重连等待前一个 transport 清理完成，防止旧的释放操作 detach 新认领的标签页。请求不能隐式重启已停止的 bridge。
停止运行时会取消 session 监听、排空标签页注册、finalize 受控标签页，并在停止 bridge 前等待 transport 清理。关闭过程中会关闭扩展报告的派生标签页，不接纳新的 debugger attachment。页面关闭和崩溃通过注册该页面的 transport 释放；崩溃的标签页保留 ownership，直到关闭或 finalize。重连等待前一个 transport 清理完成，防止旧的释放操作 detach 新认领的标签页。请求不能隐式重启已停止的 bridge。
当 CDP 省略可选 browserContextId 时，适配器提供稳定的默认 context id；保留已有 id，并在 Playwright 接收 target 前拒绝格式错误的值。直接消息回调失败会关闭 transport 并释放其 attachment。
docs/users/features/browser-use.md
+29
View File
@@ -0,0 +1,29 @@
# Browser Use
Browser Use lets Qwen Code work with pages in your Chrome browser, using your
existing tabs and signed-in sessions.
## Use
Use macOS or Linux with Chrome 125 or later. **Install and enable the Qwen
Chrome extension in the Chrome profile you want to use.** The extension is
required and is not installed by the Qwen Code package. There is no Chrome Web
Store listing yet. Build it from the `packages/chrome-extension` directory of
the Qwen Code repository by following its
[README](https://github.com/QwenLM/qwen-code/tree/main/packages/chrome-extension#readme),
then open `chrome://extensions`, enable Developer mode, choose **Load
unpacked**, and pick the built `dist/extension` directory.
Describe your browser task directly, for example:
> Read my open dashboard and summarize today's orders.
Qwen selects the Browser Use skill when appropriate. If a runtime dependency
needs configuration on first use, Qwen will guide you and may ask you to restart.
No separate Browser Use Qwen extension or `qwen serve` process is needed.
## Disable
Use `/skills` to disable **browser-use**. This hides the skill from the model
but does not disconnect an existing browser session or remove instructions
already loaded in a conversation.
docs/users/features/computer-use.md
+1 -1
View File
@@ -25,7 +25,7 @@ Node.js 22 or later and npm are required.
When first used, the skill runs these commands itself:
```bash
qwen mcp add --scope user node-repl npx -y @qwen-code/[email protected].5
qwen mcp add --scope user node-repl npx -y @qwen-code/[email protected].6
npm install --no-save --package-lock=false @qwen-code/[email protected]
```
eslint.config.js
+1
View File
@@ -76,6 +76,7 @@ export default tseslint.config(
'.qwen/**',
'scripts/codemod/fixtures/**', // codemod test data; intentionally non-idiomatic ink input/output
'packages/desktop-shell/runtime/**',
'packages/core/src/skills/bundled/browser-use/runtime/**',
'packages/desktop-shell/src-tauri/target/**',
'packages/live-host/**', // standalone Electron app with its own Node test conventions
'packages/cua-driver/**', // vendored trycua/cua driver (Rust + scripts); not qwen-code TS
package-lock.json
Generated
+2 -1
View File
@@ -28133,6 +28133,7 @@
"zod": "^3.25.76"
},
"devDependencies": {
"@qwen-code/browser-use": "file:../browser-use",
"@types/diff": "^7.0.2",
"@types/dotenv": "^6.1.1",
"@types/fast-levenshtein": "^0.0.4",
@@ -29141,7 +29142,7 @@
},
"packages/node-repl": {
"name": "@qwen-code/node-repl-mcp",
"version": "0.1.5",
"version": "0.1.6",
"license": "Apache-2.0",
"dependencies": {
"@modelcontextprotocol/sdk": "^1.30.0",
packages/browser-use/build.mjs
+47 -1
View File
@@ -4,13 +4,27 @@
* SPDX-License-Identifier: Apache-2.0
*/
import { rmSync } from 'node:fs';
import { chmodSync, rmSync } from 'node:fs';
import path from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { build } from 'esbuild';
const root = path.dirname(fileURLToPath(import.meta.url));
const dist = path.join(root, 'dist');
const nodeBanner =
"import { createRequire as __qwenCreateRequire } from 'node:module'; import { fileURLToPath as __qwenFileURLToPath } from 'node:url'; import { dirname as __qwenDirname } from 'node:path'; const require = __qwenCreateRequire(import.meta.url); const __filename = __qwenFileURLToPath(import.meta.url); const __dirname = __qwenDirname(__filename);";
// The bundled skill imports dist/index.js (staged as runtime/index.js) from
// inside the node_repl kernel: packages/node-repl/src/runtime/module-loader.mjs
// compiles absolute-path imports as vm.SourceTextModule in the kernel's
// untrusted vm context, which is built from Object.create(null), never defines
// a `process` global, and denies the `node:process` import specifier. The SDK
// reads process.env/process.platform, so bind `process` through createRequire,
// the one route the kernel leaves open. The load check at the bottom of this
// file runs in Node's main realm, where `process` is always a global, so it
// cannot detect this binding going missing; src/skill-runtime-kernel.test.ts
// runs the skill's first cell through the real kernel for that.
const kernelBanner =
"import { createRequire as __qwenCreateRequire } from 'node:module'; const process = __qwenCreateRequire(import.meta.url)('node:process');";
rmSync(dist, { recursive: true, force: true });
@@ -24,10 +38,42 @@ await build({
platform: 'node',
target: 'node22',
format: 'esm',
banner: {
js: kernelBanner,
},
packages: 'bundle',
external: ['playwright-core', 'playwright-core/*'],
});
await build({
absWorkingDir: root,
entryPoints: {
'native-host': 'src/bridge/native-host/index.ts',
'scripts/native-host-setup': 'scripts/native-host-setup.ts',
'scripts/managed-chrome': 'scripts/managed-chrome.ts',
'scripts/managed-chrome-preflight': 'scripts/managed-chrome-preflight.ts',
'scripts/smoke-qwen-saucedemo': 'scripts/smoke-qwen-saucedemo.ts',
},
outdir: dist,
bundle: true,
platform: 'node',
target: 'node22',
format: 'esm',
banner: {
js: nodeBanner,
},
packages: 'bundle',
external: ['playwright-core', 'playwright-core/*'],
loader: { '.wasm': 'binary' },
});
for (const executable of [
path.join(dist, 'native-host.js'),
path.join(dist, 'scripts/native-host-setup.js'),
]) {
chmodSync(executable, 0o755);
}
// esbuild only proves the bundle parses; a load-time failure (a duplicate
// top-level binding, an unresolved external) would otherwise surface first in
// the Node kernel that imports the published artifact.
packages/browser-use/package.json
+7
View File
@@ -23,6 +23,8 @@
"dist/index.d.ts",
"dist/core/primitives.d.ts",
"dist/sdk/types.d.ts",
"dist/native-host.js",
"dist/scripts/native-host-setup.js",
"NOTICE"
],
"engines": {
@@ -30,6 +32,11 @@
},
"scripts": {
"build": "node build.mjs && tsc -p tsconfig.declarations.json",
"native-host:install": "node dist/scripts/native-host-setup.js install",
"native-host:status": "node dist/scripts/native-host-setup.js status",
"native-host:uninstall": "node dist/scripts/native-host-setup.js uninstall",
"smoke:managed-chrome-preflight": "npm run build --workspace=@qwen-code/chrome-bridge && npm run build && node dist/scripts/managed-chrome-preflight.js",
"smoke:qwen-saucedemo": "npm run build --workspace=@qwen-code/node-repl-mcp && npm run build --workspace=@qwen-code/chrome-bridge && npm run build && npm --prefix ../.. run bundle && node dist/scripts/smoke-qwen-saucedemo.js",
"test:ci": "vitest run",
"typecheck": "tsc --noEmit",
"test": "vitest run"
packages/browser-use/scripts/managed-chrome-preflight.ts
+319
View File
@@ -0,0 +1,319 @@
#!/usr/bin/env node
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { createServer } from 'node:http';
import { ChromeExtensionTransport } from '../src/bridge/index.js';
import { PlaywrightRuntime } from '../src/playwright/playwright-runtime.js';
import { jpegDimensions } from '../src/playwright/runtime-helpers.js';
import {
CHROME_BRIDGE_PROTOCOL_VERSION,
CHROME_EXTENSION_ID,
} from '../src/bridge/protocol.js';
import { withManagedChrome } from './managed-chrome.js';
const fixture = await startFixture();
try {
await withManagedChrome('preflight', async (chrome) => {
const transport = new ChromeExtensionTransport({
socketPath: chrome.socketPath,
});
const runtime = new PlaywrightRuntime({ bridge: transport });
try {
await transport.start();
const ping = asRecord(await transport.request('ping', {}, 20_000));
assert(
ping['extensionId'] === CHROME_EXTENSION_ID &&
ping['protocolVersion'] === CHROME_BRIDGE_PROTOCOL_VERSION,
'Chrome extension identity did not match',
);
await runtime.dispatch('browsers.get', { id: 'chrome' });
const created = asRecord(
await runtime.dispatch('tabs.new', { browserId: 'chrome' }),
);
await runtime.dispatch('tab.goto', {
tabId: created['id'],
url: fixture.origin + '/',
});
const openTabs = await runtime.dispatch('browser.user.openTabs', {
browserId: 'chrome',
});
assert(
Array.isArray(openTabs) && openTabs.length === 1,
'Tab discovery failed',
);
const claimed = asRecord(
await runtime.dispatch('browser.user.claimTab', {
browserId: 'chrome',
tab: asRecord(openTabs[0]),
}),
);
const snapshot = await runtime.dispatch('playwright.domSnapshot', {
tabId: claimed['id'],
});
assert(
typeof snapshot === 'string' && snapshot.includes('Username'),
'Fixture DOM snapshot was not observed: ' + String(snapshot),
);
const usernameRef =
typeof snapshot === 'string'
? /textbox "Username".*\[ref=([^\]]+)\]/.exec(snapshot)?.[1]
: undefined;
assert(
usernameRef !== undefined,
'Username ref was not present in the AI snapshot: ' + String(snapshot),
);
const scrollY = await runtime.dispatch('playwright.evaluate', {
tabId: claimed['id'],
script:
'document.body.style.minHeight = "2400px"; scrollTo(0, 900); return window.scrollY;',
});
assert(
typeof scrollY === 'number' && scrollY >= 800,
'Fixture did not reach the screenshot scroll position',
);
const screenshot = asRecord(
await runtime.dispatch('tab.screenshot', {
tabId: claimed['id'],
clip: { x: 0, y: 0, width: 100, height: 50 },
}),
);
const screenshotBytes = Buffer.from(
String(screenshot['base64']),
'base64',
);
const dimensions = jpegDimensions(screenshotBytes);
assert(
screenshot['mimeType'] === 'image/jpeg' &&
dimensions.width === 100 &&
dimensions.height === 50,
'Screenshot did not preserve the CSS-pixel clip contract',
);
await runtime.dispatch('playwright.evaluate', {
tabId: claimed['id'],
script: `
for (const type of ['pointerdown', 'click', 'keydown', 'input']) {
sessionStorage.removeItem('__qbu_' + type);
document.addEventListener(type, (event) => {
sessionStorage.setItem('__qbu_' + type, String(event.isTrusted));
}, { capture: true });
}
return true;`,
});
await runtime.dispatch('dom_cua.click', {
tabId: claimed['id'],
node_id: usernameRef,
});
const focusedId = await runtime.dispatch('playwright.evaluate', {
tabId: claimed['id'],
script: 'return document.activeElement?.id;',
});
assert(
focusedId === 'user-name',
'Snapshot ref did not focus the username input',
);
await runtime.dispatch('locator.type', {
tabId: claimed['id'],
steps: [{ kind: 'locator', selector: '#user-name' }],
value: 'qwen',
});
await runtime.dispatch('locator.type', {
tabId: claimed['id'],
steps: [{ kind: 'locator', selector: '#password' }],
value: 'browser-use',
});
const trusted = asRecord(
await runtime.dispatch('playwright.evaluate', {
tabId: claimed['id'],
script: `return ({
pointer: sessionStorage.getItem('__qbu_pointerdown'),
click: sessionStorage.getItem('__qbu_click'),
keydown: sessionStorage.getItem('__qbu_keydown'),
input: sessionStorage.getItem('__qbu_input'),
username: document.querySelector('#user-name')?.value,
password: document.querySelector('#password')?.value,
});`,
}),
);
assert(
trusted['pointer'] === 'true' &&
trusted['click'] === 'true' &&
trusted['keydown'] === 'true' &&
trusted['input'] === 'true',
'CDP input did not produce trusted page events: ' +
JSON.stringify(trusted),
);
assert(
trusted['username'] === 'qwen' && trusted['password'] === 'browser-use',
'CDP input did not set the requested values',
);
await runtime.dispatch('locator.click', {
tabId: claimed['id'],
steps: [{ kind: 'locator', selector: '#login-button' }],
});
await runtime.dispatch('playwright.waitForURL', {
tabId: claimed['id'],
url: fixture.origin + '/inventory.html',
});
const inventory = await runtime.dispatch('locator.innerText', {
tabId: claimed['id'],
steps: [{ kind: 'locator', selector: 'h1' }],
});
assert(inventory === 'Inventory', 'Fixture navigation was not observed');
const mediaDownload = runtime.dispatch('playwright.waitForEvent', {
tabId: claimed['id'],
event: 'download',
timeoutMs: 5_000,
});
// The waiter is only awaited after the click succeeds; if downloadMedia
// rejects first, this keeps the waiter's own rejection from surfacing
// as an unhandled rejection while the later await still reports it.
mediaDownload.catch(() => undefined);
await new Promise((resolvePromise) => setTimeout(resolvePromise, 100));
await runtime.dispatch('locator.downloadMedia', {
tabId: claimed['id'],
steps: [{ kind: 'locator', selector: '#download-link' }],
timeoutMs: 5_000,
});
await mediaDownload;
const history = await runtime.dispatch('browser.user.history', {
browserId: 'chrome',
options: { queries: ['inventory'], limit: 10 },
});
assert(
Array.isArray(history),
'Required History permission did not return a result list',
);
await runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [{ tabId: claimed['id'], status: 'deliverable' }],
});
const deliveredTabs = await runtime.dispatch('browser.user.openTabs', {
browserId: 'chrome',
});
assert(
Array.isArray(deliveredTabs),
'Deliverable tab discovery returned an invalid result',
);
const delivered = deliveredTabs.find(
(tab) =>
typeof tab === 'object' &&
tab !== null &&
'url' in tab &&
tab.url === fixture.origin + '/inventory.html',
);
assert(delivered, 'Finalization did not leave the deliverable tab open');
const reclaimed = asRecord(
await runtime.dispatch('browser.user.claimTab', {
browserId: 'chrome',
tab: delivered,
}),
);
assert(
(await runtime.dispatch('tab.url', { tabId: reclaimed['id'] })) ===
fixture.origin + '/inventory.html',
'Finalized deliverable tab could not be reclaimed',
);
await new Promise((resolvePromise) => setTimeout(resolvePromise, 65_000));
const titleAfterIdle = await runtime.dispatch('tab.title', {
tabId: reclaimed['id'],
});
assert(
titleAfterIdle === 'Inventory',
'Browser Use did not survive 65 seconds without traffic',
);
await runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [],
});
process.stdout.write(
JSON.stringify({
ok: true,
chromeVersion: chrome.chromeVersion,
extensionId: ping['extensionId'],
mv3IdleKeepaliveObserved: true,
snapshotObserved: true,
snapshotRefActionObserved: true,
screenshotObserved: true,
scrolledClipObserved: true,
trustedPointerObserved: true,
trustedKeyboardObserved: true,
trustedInputObserved: true,
historyPermissionAvailable: true,
fixtureNavigationObserved: true,
downloadObserved: true,
finalizationObserved: true,
}) + '\n',
);
} finally {
await runtime.stop();
}
});
} finally {
await fixture.close();
}
function asRecord(value: unknown): Record<string, unknown> {
if (typeof value !== 'object' || value === null || Array.isArray(value)) {
throw new Error('Expected an object');
}
return value as Record<string, unknown>;
}
async function startFixture(): Promise<{
origin: string;
close(): Promise<void>;
}> {
const server = createServer((request, response) => {
if (request.url === '/fixture.txt') {
response.setHeader('content-type', 'text/plain; charset=utf-8');
response.setHeader(
'content-disposition',
'attachment; filename="fixture.txt"',
);
response.end('browser-use download fixture');
return;
}
response.setHeader('content-type', 'text/html; charset=utf-8');
response.end(
request.url === '/inventory.html'
? [
'<!doctype html><title>Inventory</title><h1>Inventory</h1>',
'<a id="download-link" href="/fixture.txt" download>Download</a>',
].join('')
: [
'<!doctype html><title>Browser Use fixture</title>',
'<form id="login">',
'<label>Username <input id="user-name"></label>',
'<label>Password <input id="password" type="password"></label>',
'<button id="login-button" type="submit">Login</button>',
'</form>',
'<script>login.addEventListener("submit", event => {',
'event.preventDefault(); location.href = "/inventory.html";',
'});</script>',
].join(''),
);
});
await new Promise<void>((resolvePromise, rejectPromise) => {
server.once('error', rejectPromise);
server.listen(0, '127.0.0.1', resolvePromise);
});
const address = server.address();
assert(address && typeof address === 'object', 'Fixture did not start');
return {
origin: `http://127.0.0.1:${address.port}`,
close: async () =>
await new Promise<void>((resolvePromise, rejectPromise) => {
server.close((error) =>
error ? rejectPromise(error) : resolvePromise(),
);
}),
};
}
function assert(condition: unknown, message: string): asserts condition {
if (!condition) throw new Error(message);
}
packages/browser-use/scripts/managed-chrome.test.ts
+134
View File
@@ -0,0 +1,134 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { launchManagedChrome } from './managed-chrome.js';
const mocks = vi.hoisted(() => ({
access: vi.fn(),
executablePath: vi.fn(),
execFile: vi.fn(),
}));
vi.mock('node:fs/promises', async (importOriginal) => ({
...(await importOriginal<typeof import('node:fs/promises')>()),
access: mocks.access,
}));
vi.mock('node:child_process', async (importOriginal) => {
const { promisify } = await import('node:util');
Object.assign(mocks.execFile, {
[promisify.custom]: (...args: unknown[]) =>
new Promise((resolve, reject) => {
mocks.execFile(
...args,
(error: Error | null, stdout: string, stderr: string) => {
if (error) reject(error);
else resolve({ stdout, stderr });
},
);
}),
});
return {
...(await importOriginal<typeof import('node:child_process')>()),
execFile: mocks.execFile,
};
});
vi.mock('playwright-core', () => ({
chromium: { executablePath: mocks.executablePath },
}));
describe('managed Chrome discovery', () => {
beforeEach(() => {
vi.resetAllMocks();
vi.stubEnv('QWEN_BROWSER_USE_CHROME', '');
mocks.executablePath.mockReturnValue('/missing/playwright/chrome');
mocks.access.mockRejectedValue(new Error('ENOENT'));
mocks.execFile.mockImplementation(
(
_file: string,
_args: string[],
_options: object,
callback: (error: Error) => void,
) => callback(new Error('Stopped at browser version probe')),
);
});
afterEach(() => vi.unstubAllEnvs());
async function expectSelectedBrowser(path: string): Promise<void> {
await expect(launchManagedChrome('discovery-test')).rejects.toThrow(
'Stopped at browser version probe',
);
expect(mocks.execFile).toHaveBeenCalledWith(
path,
['--version'],
{ timeout: 5_000 },
expect.any(Function),
);
}
it('keeps the explicit browser override ahead of automatic discovery', async () => {
vi.stubEnv('QWEN_BROWSER_USE_CHROME', ' /custom/chrome ');
await expectSelectedBrowser('/custom/chrome');
expect(mocks.executablePath).not.toHaveBeenCalled();
expect(mocks.access).not.toHaveBeenCalled();
});
it.each([
'/cache/chromium-1234/chrome-mac-x64/Google Chrome for Testing.app/Contents/MacOS/Google Chrome for Testing',
'/cache/chromium-1234/chrome-mac-arm64/Google Chrome for Testing.app/Contents/MacOS/Google Chrome for Testing',
'/custom/playwright/chromium-1234/chrome-linux64/chrome',
])('uses the installed Playwright browser at %s', async (path) => {
mocks.executablePath.mockReturnValue(path);
mocks.access.mockImplementation(async (candidate: string) => {
if (candidate !== path && candidate !== '/usr/bin/google-chrome') {
throw new Error('ENOENT');
}
});
await expectSelectedBrowser(path);
});
it('falls back to system Chromium if the Playwright browser is absent', async () => {
mocks.access.mockImplementation(async (candidate: string) => {
if (candidate !== '/usr/bin/chromium') throw new Error('ENOENT');
});
await expectSelectedBrowser('/usr/bin/chromium');
});
it('ignores branded Chrome installations during automatic discovery', async () => {
mocks.access.mockImplementation(async (candidate: string) => {
if (candidate !== '/usr/bin/google-chrome') throw new Error('ENOENT');
});
await expect(launchManagedChrome('discovery-test')).rejects.toThrow(
'Chrome was not found',
);
expect(mocks.execFile).not.toHaveBeenCalled();
});
it('rejects an explicit Chrome build that cannot load the extension', async () => {
vi.stubEnv('QWEN_BROWSER_USE_CHROME', '/custom/google-chrome');
mocks.execFile.mockImplementation(
(
_file: string,
_args: string[],
_options: object,
callback: (error: null, stdout: string, stderr: string) => void,
) => callback(null, 'Google Chrome 137.0.0.0', ''),
);
await expect(launchManagedChrome('discovery-test')).rejects.toThrow(
'Chromium or Chrome for Testing',
);
});
it('reports a missing browser before launching a process', async () => {
await expect(launchManagedChrome('discovery-test')).rejects.toThrow(
'Chrome was not found; set QWEN_BROWSER_USE_CHROME',
);
expect(mocks.execFile).not.toHaveBeenCalled();
});
});
packages/browser-use/scripts/managed-chrome.ts
+215
View File
@@ -0,0 +1,215 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { execFile } from 'node:child_process';
import { once } from 'node:events';
import { access, mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
import { promisify } from 'node:util';
import { spawn, type ChildProcess } from 'node:child_process';
import { chromium } from 'playwright-core';
import {
CHROME_EXTENSION_ID,
CHROME_NATIVE_HOST_NAME,
} from '../src/bridge/protocol.js';
const execFileAsync = promisify(execFile);
export interface ManagedChrome {
root: string;
socketPath: string;
chromeVersion: string;
stop(): Promise<void>;
}
export async function launchManagedChrome(
label: string,
): Promise<ManagedChrome> {
const packageRoot = resolve(dirname(fileURLToPath(import.meta.url)), '../..');
const chromePath = await findChrome();
const { stdout } = await execFileAsync(chromePath, ['--version'], {
timeout: 5_000,
});
const chromeVersion = stdout.trim() || 'unknown';
const brandedVersion = /^Google Chrome (\d+)\./.exec(chromeVersion);
if (brandedVersion && Number(brandedVersion[1]) >= 137) {
throw new Error(
'Managed Chrome requires Chromium or Chrome for Testing; set QWEN_BROWSER_USE_CHROME to a supported build',
);
}
const parent = process.platform === 'darwin' ? '/tmp' : tmpdir();
const root = await mkdtemp(join(parent, 'qbu-' + label + '-'));
const socketPath = join(root, 'bridge.sock');
const profile = join(root, 'profile');
const extension = resolve(packageRoot, '../chrome-extension/dist/extension');
const host = join(packageRoot, 'dist/native-host.js');
try {
await access(extension);
await access(host);
await mkdir(join(profile, 'NativeMessagingHosts'), { recursive: true });
await mkdir(join(profile, 'Default'), { recursive: true });
await writeFile(
join(profile, 'Default', 'Preferences'),
JSON.stringify({
profile: { password_manager_leak_detection: false },
credentials_enable_service: false,
credentials_enable_autosignin: false,
}),
);
const launcher = join(root, 'native-host.sh');
await writeFile(
launcher,
'#!/bin/sh\nexport QWEN_BROWSER_USE_SOCKET_PATH=' +
shellQuote(socketPath) +
'\nexec ' +
shellQuote(process.execPath) +
' ' +
shellQuote(host) +
'\n',
{ mode: 0o700 },
);
await writeFile(
join(profile, 'NativeMessagingHosts', CHROME_NATIVE_HOST_NAME + '.json'),
JSON.stringify({
name: CHROME_NATIVE_HOST_NAME,
description: 'Qwen Browser Use managed Chrome',
path: launcher,
type: 'stdio',
allowed_origins: ['chrome-extension://' + CHROME_EXTENSION_ID + '/'],
}),
);
} catch (error) {
await rm(root, { recursive: true, force: true });
throw error;
}
const chrome = spawn(
chromePath,
[
'--headless=new',
'--user-data-dir=' + profile,
'--load-extension=' + extension,
'--disable-extensions-except=' + extension,
'--window-size=1280,900',
'--no-first-run',
'--no-default-browser-check',
'--disable-component-update',
'--disable-gpu',
'--no-sandbox',
'--disable-dev-shm-usage',
'about:blank',
],
{
detached: process.platform !== 'win32',
stdio: ['ignore', 'ignore', 'pipe'],
},
);
chrome.stderr?.on('data', (chunk: Buffer) => {
const text = chunk.toString('utf8');
if (!text.includes('DEPRECATED_ENDPOINT')) {
process.stderr.write('[managed-chrome] ' + text);
}
});
let stopping: Promise<void> | undefined;
return {
root,
socketPath,
chromeVersion,
stop() {
return (stopping ??= stopProcess(chrome).finally(() =>
rm(root, { recursive: true, force: true }),
));
},
};
}
export async function withManagedChrome(
label: string,
run: (chrome: ManagedChrome) => Promise<void>,
): Promise<void> {
const chrome = await launchManagedChrome(label);
const interrupt = () => {
void chrome.stop().finally(() => process.exit(130));
};
const terminate = () => {
void chrome.stop().finally(() => process.exit(143));
};
process.once('SIGINT', interrupt);
process.once('SIGTERM', terminate);
const previous = process.env['QWEN_BROWSER_USE_SOCKET_PATH'];
process.env['QWEN_BROWSER_USE_SOCKET_PATH'] = chrome.socketPath;
try {
await run(chrome);
} finally {
if (previous === undefined) {
delete process.env['QWEN_BROWSER_USE_SOCKET_PATH'];
} else {
process.env['QWEN_BROWSER_USE_SOCKET_PATH'] = previous;
}
try {
await chrome.stop();
} finally {
process.off('SIGINT', interrupt);
process.off('SIGTERM', terminate);
}
}
}
async function findChrome(): Promise<string> {
const configured = process.env['QWEN_BROWSER_USE_CHROME']?.trim();
if (configured) return configured;
const candidates = [
chromium.executablePath(),
'/Applications/Google Chrome for Testing.app/Contents/MacOS/Google Chrome for Testing',
'/usr/bin/chromium',
'/usr/bin/chromium-browser',
];
for (const candidate of candidates) {
try {
await access(candidate);
return candidate;
} catch {
// Try the next candidate.
}
}
throw new Error('Chrome was not found; set QWEN_BROWSER_USE_CHROME');
}
async function stopProcess(child: ChildProcess): Promise<void> {
if (child.exitCode !== null || child.signalCode !== null) return;
const exit = once(child, 'exit');
signal(child, 'SIGTERM');
await Promise.race([
exit,
new Promise((resolvePromise) => setTimeout(resolvePromise, 3_000)),
]);
if (child.exitCode === null && child.signalCode === null) {
signal(child, 'SIGKILL');
await Promise.race([
once(child, 'exit'),
new Promise((resolvePromise) => setTimeout(resolvePromise, 2_000)),
]);
}
}
function signal(child: ChildProcess, signalName: NodeJS.Signals): void {
try {
if (process.platform !== 'win32' && child.pid) {
process.kill(-child.pid, signalName);
} else {
child.kill(signalName);
}
} catch {
// Process is already gone.
}
}
function shellQuote(value: string): string {
return "'" + value.replaceAll("'", "'\\''") + "'";
}
packages/browser-use/scripts/native-host-setup.ts
+37
View File
@@ -0,0 +1,37 @@
#!/usr/bin/env node
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { fileURLToPath } from 'node:url';
import {
installChromeNativeHost,
nativeHostInstallHome,
statusChromeNativeHost,
uninstallChromeNativeHost,
} from '../src/native-host-installer.js';
const command = process.argv[2] ?? 'status';
const options = {
homeDir: nativeHostInstallHome(),
nativeHostPath: fileURLToPath(new URL('../native-host.js', import.meta.url)),
};
const result =
command === 'install'
? await installChromeNativeHost(options)
: command === 'uninstall'
? await uninstallChromeNativeHost(options)
: command === 'status'
? await statusChromeNativeHost(options)
: null;
if (!result) {
process.stderr.write(
'Usage: native-host-setup.js install|status|uninstall\n',
);
process.exit(2);
}
process.stdout.write(JSON.stringify({ command, ...result }) + '\n');
packages/browser-use/scripts/smoke-checks.test.ts
+371
View File
@@ -0,0 +1,371 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';
import { runInNewContext } from 'node:vm';
import { describe, expect, it, vi } from 'vitest';
import { transformSync } from 'esbuild';
import { BrowserRuntimeError } from '../src/bridge/errors.js';
import { jpegDimensions } from '../src/playwright/runtime-helpers.js';
const preflight = readFileSync(
new URL('./managed-chrome-preflight.ts', import.meta.url),
'utf8',
);
const screenshotCheck = preflight.match(
/const screenshotBytes =[\s\S]*?(?=\n\s*await runtime\.dispatch\('playwright\.evaluate')/,
)?.[0];
assert.ok(screenshotCheck, 'Preflight screenshot assertion must be exercised');
function validateScreenshot(bytes: Buffer, mimeType = 'image/jpeg'): void {
runInNewContext(screenshotCheck!, {
Buffer,
assert,
jpegDimensions,
screenshot: { base64: bytes.toString('base64'), mimeType },
});
}
describe('preflight screenshot validation', () => {
// JPEG SOF dimensions, independently encoded for the required 100x50 clip.
const jpeg = Buffer.from([
0xff, 0xd8, 0xff, 0xc0, 0, 11, 8, 0, 50, 0, 100, 1, 1, 0x11, 0, 0xff, 0xd9,
]);
it('accepts the JPEG clip returned by the screenshot contract', () => {
expect(() => validateScreenshot(jpeg)).not.toThrow();
});
it('rejects the wrong decoded dimensions', () => {
const oversized = Buffer.from(jpeg);
oversized.writeUInt16BE(200, 9);
expect(() => validateScreenshot(oversized)).toThrow();
});
it('rejects a mismatched MIME type', () => {
expect(() => validateScreenshot(jpeg, 'image/png')).toThrow();
});
it('rejects a PNG header even with the expected IHDR dimensions', () => {
const png = Buffer.alloc(24);
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]).copy(png);
png.writeUInt32BE(100, 16);
png.writeUInt32BE(50, 20);
expect(() => validateScreenshot(png, 'image/png')).toThrow();
});
});
describe('smoke observation claims', () => {
const smoke = readFileSync(
new URL('./smoke-qwen-saucedemo.ts', import.meta.url),
'utf8',
);
const checks = smoke.match(/const checks = \{[\s\S]*?\n {4}\};/)?.[0];
assert.ok(checks, 'Smoke checks must be exercised');
const helperStart = smoke.indexOf('function stagedRuntimeImported(');
const helperEnd = smoke.indexOf('\n}\n', helperStart) + 3;
assert.ok(helperStart >= 0, 'stagedRuntimeImported must be exercised');
const stagedRuntimeImported = runInNewContext(
transformSync(smoke.slice(helperStart, helperEnd), {
loader: 'ts',
target: 'node22',
}).code + '\nstagedRuntimeImported;',
{},
) as (
calls: Array<{ code: string; output: string }>,
browserRuntimeRoot: string,
builtinSkillRoot: string,
) => boolean;
it.each([
'// .type( .click( .finalize(',
'nodeRepl.write(".type( .click( .finalize(");',
])('does not report SDK setup from unrelated action text: %s', (code) => {
const result = runInNewContext(checks + '\nchecks;', {
calls: [{ code, output: '' }],
joinedCode: code,
joinedOutput: 'Total: $36.69',
browserRuntimeEntry: '/skill/runtime/index.js',
browserRuntimeRoot: '/skill/runtime',
builtinSkillRoot: '/skill',
stagedRuntimeImported,
events: [],
browserModuleRoot: '/runtime/node_modules',
successfulModuleRootRegistered: () => true,
completion: {
url: 'https://www.saucedemo.com/checkout-complete.html',
snapshot: 'Thank you for your order!',
},
cartMatchesLowestThree: true,
finalAnswer: '$36.69',
}) as Record<string, boolean>;
expect(result.browserSdkImported).toBe(false);
expect(result.checkoutCompletionObserved).toBe(true);
expect(result.reportedTotalIsExpected).toBe(true);
});
it.each([
[
'a literal entry path',
"await (await import('/skill/runtime/index.js')).setupBrowserRuntime();",
true,
],
[
'an entry built from the staged runtime directory',
"const SKILL_BASE = '/skill/runtime';\nawait (await import(SKILL_BASE + '/index.js')).setupBrowserRuntime();",
true,
],
[
'an entry built from the skill root',
"const base = '/skill';\nglobalThis.SKILL_BASE = base;\nawait (await import(SKILL_BASE + '/runtime/index.js')).setupBrowserRuntime();",
true,
],
[
'a runtime loaded from somewhere else',
"await (await import('/elsewhere/runtime/index.js')).setupBrowserRuntime();",
false,
],
[
'the skill root without setup',
"import('/skill/runtime/index.js');",
false,
],
[
'a staged import that threw and fell back to a foreign entry',
"let sdk; try { sdk = await import('/skill/runtime/index.js'); } catch { sdk = await import('/repo/packages/browser-use/dist/index.js'); } await sdk.setupBrowserRuntime();",
false,
],
[
'the setup export reached through an alias',
"const { setupBrowserRuntime: setup } = await import('/skill/runtime/index.js');\nawait setup();",
true,
],
[
'the skill root echoed in prose next to a foreign import',
"// staged at /skill/runtime/index.js\nawait (await import('/elsewhere/index.js')).setupBrowserRuntime();",
false,
],
])('reports SDK setup for %s', (_label, code, expected) => {
const result = runInNewContext(checks + '\nchecks;', {
calls: [{ code, output: '' }],
joinedCode: code,
joinedOutput: 'Total: $36.69',
browserRuntimeEntry: '/skill/runtime/index.js',
browserRuntimeRoot: '/skill/runtime',
builtinSkillRoot: '/skill',
stagedRuntimeImported,
events: [],
browserModuleRoot: '/runtime/node_modules',
successfulModuleRootRegistered: () => true,
completion: {
url: 'https://www.saucedemo.com/checkout-complete.html',
snapshot: 'Thank you for your order!',
},
cartMatchesLowestThree: true,
finalAnswer: '$36.69',
}) as Record<string, boolean>;
expect(result.browserSdkImported).toBe(expected);
});
});
describe('smoke price reads', () => {
const smoke = readFileSync(
new URL('./smoke-qwen-saucedemo.ts', import.meta.url),
'utf8',
);
const start = smoke.indexOf('function pricesFromCall(');
const end = smoke.indexOf('\n}\n', start) + 3;
assert.ok(start >= 0 && end > start, 'pricesFromCall must be exercised');
const pricesFromCall = runInNewContext(
transformSync(smoke.slice(start, end), { loader: 'ts', target: 'node22' })
.code + '\npricesFromCall;',
{},
) as (
calls: Array<{ code: string; output: string }>,
selector: string,
expectedCount: number,
scope: 'inventory' | 'cart',
) => number[] | null;
const inventory = {
code: "tab.playwright.locator('.inventory_item_price').allTextContents()",
output: "[ '$29.99', '$9.99', '$15.99', '$49.99', '$7.99', '$15.99' ]",
};
const cartConfirmed = {
code: 'tab.url()',
output: 'https://www.saucedemo.com/cart.html',
};
const cart = {
code: "tab.playwright.locator('.cart_item .inventory_item_price').allTextContents()",
output: "[ '$7.99', '$9.99', '$15.99' ]",
};
it('attributes reads to the inventory or the cart by the cart.html confirmation', () => {
const calls = [inventory, cartConfirmed, cart];
expect(
pricesFromCall(calls, '.inventory_item_price', 6, 'inventory'),
).toEqual([29.99, 9.99, 15.99, 49.99, 7.99, 15.99]);
expect(pricesFromCall(calls, '.inventory_item_price', 3, 'cart')).toEqual([
7.99, 9.99, 15.99,
]);
});
it('reports no cart read without a cart.html confirmation, even for a three-price read', () => {
// A run that only sorted the inventory "Price (low to high)" and read it
// again printed three matching prices without ever visiting the cart.
const sortedInventory = {
code: "tab.playwright.locator('.inventory_item_price').allTextContents()",
output: "[ '$7.99', '$9.99', '$15.99', '$15.99', '$29.99', '$49.99' ]",
};
expect(
pricesFromCall(
[inventory, sortedInventory],
'.inventory_item_price',
3,
'cart',
),
).toBeNull();
expect(
pricesFromCall([inventory, cart], '.inventory_item_price', 3, 'cart'),
).toBeNull();
});
it('takes the leading prices when the same cell echoes a selection afterwards', () => {
const inventoryWithEcho = {
code: "const prices = await tab.playwright.locator('.inventory_item_price').allTextContents(); nodeRepl.write(JSON.stringify(prices)); nodeRepl.write(JSON.stringify(prices.slice(0, 3)));",
output:
'["$29.99","$9.99","$15.99","$49.99","$7.99","$15.99"]["$7.99","$9.99","$15.99"]',
};
expect(
pricesFromCall(
[inventoryWithEcho],
'.inventory_item_price',
6,
'inventory',
),
).toEqual([29.99, 9.99, 15.99, 49.99, 7.99, 15.99]);
// An inventory echo is never a cart read.
expect(
pricesFromCall([inventoryWithEcho], '.inventory_item_price', 3, 'cart'),
).toBeNull();
const cartWithEcho = {
code: "const cartPrices = await tab.playwright.locator('.inventory_item_price').allTextContents(); nodeRepl.write(JSON.stringify(cartPrices)); nodeRepl.write('subtotal $33.97');",
output: '["$7.99","$9.99","$15.99"] subtotal $33.97',
};
expect(
pricesFromCall(
[inventoryWithEcho, cartConfirmed, cartWithEcho],
'.inventory_item_price',
3,
'cart',
),
).toEqual([7.99, 9.99, 15.99]);
});
it('ignores reads that did not use allTextContents on the price class', () => {
const innerText = {
code: "tab.playwright.locator('.inventory_item_price').first().innerText()",
output: '$29.99 $9.99 $15.99 $49.99 $7.99 $15.99',
};
expect(
pricesFromCall([innerText], '.inventory_item_price', 6, 'inventory'),
).toBeNull();
expect(
pricesFromCall(
[cartConfirmed, innerText],
'.inventory_item_price',
3,
'cart',
),
).toBeNull();
});
});
describe('smoke completion verification', () => {
const smoke = readFileSync(
new URL('./smoke-qwen-saucedemo.ts', import.meta.url),
'utf8',
);
const helperStart = smoke.indexOf('async function verifySauceCompletion(');
assert.ok(helperStart >= 0, 'Smoke verification helpers must be exercised');
const helpers = transformSync(smoke.slice(helperStart), {
loader: 'ts',
target: 'node22',
}).code;
it('starts the bridge before ping and allows the reconnect alarm', async () => {
let started = false;
const stop = vi.fn(async () => undefined);
const request = vi.fn(async (...args: unknown[]) => {
expect(started).toBe(true);
expect(args[0]).toBe('ping');
expect((args[2] as number | undefined) ?? 35_000).toBeGreaterThanOrEqual(
35_000,
);
});
class Bridge {
async start() {
await Promise.resolve();
started = true;
}
request = request;
}
class Runtime {
browserId = 'chrome';
stop = stop;
async dispatch(method: string) {
if (method === 'browser.user.openTabs') {
return [{ url: 'https://www.saucedemo.com/checkout-complete.html' }];
}
if (method === 'browser.user.claimTab') return { id: 'completion' };
return 'completed';
}
}
const result = runInNewContext(
helpers + '\nverifySauceCompletion("unused.socket");',
{
ChromeExtensionTransport: Bridge,
PlaywrightRuntime: Runtime,
isRecord: (value: unknown) =>
value !== null && typeof value === 'object',
Error,
Date,
setTimeout,
},
);
await expect(result).resolves.toEqual({
url: 'completed',
snapshot: 'completed',
});
expect(request).toHaveBeenCalledOnce();
expect(stop).toHaveBeenCalledOnce();
});
it.each(['TAB_DEBUGGER_CONFLICT', 'STALE_TAB'] as const)(
'only retries a previous debugger attachment: %s',
async (code) => {
const error = new BrowserRuntimeError(code, 'Controlled claim failure');
const dispatch = vi
.fn()
.mockRejectedValueOnce(error)
.mockRejectedValueOnce(error)
.mockResolvedValue({ id: 'completion' });
const result = runInNewContext(
helpers + '\nclaimAfterPreviousSessionExits(runtime, {});',
{ runtime: { browserId: 'chrome', dispatch }, Error, Date, setTimeout },
);
if (code === 'TAB_DEBUGGER_CONFLICT') {
await expect(result).resolves.toEqual({ id: 'completion' });
expect(dispatch).toHaveBeenCalledTimes(3);
} else {
await expect(result).rejects.toBe(error);
expect(dispatch).toHaveBeenCalledOnce();
}
},
);
});
packages/browser-use/scripts/smoke-qwen-saucedemo.ts
+603
View File
@@ -0,0 +1,603 @@
#!/usr/bin/env node
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { spawn } from 'node:child_process';
import {
access,
chmod,
mkdir,
mkdtemp,
readFile,
readdir,
rm,
writeFile,
} from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
import { ChromeExtensionTransport } from '../src/bridge/index.js';
import { PlaywrightRuntime } from '../src/playwright/playwright-runtime.js';
import { withManagedChrome } from './managed-chrome.js';
import { collectAssistantReport } from './smoke-transcript.js';
const PROMPT = [
'这是 Qwen Browser Use 的隔离 SauceDemo 冒烟测试。',
'使用 browser-use skill 和标准 node_repl，',
'用 standard_user / secret_sauce 登录 https://www.saucedemo.com/，',
'动态识别价格最低的三件商品并加入购物车，完成整个结账流程，',
'提交后验证完成页，最后告诉我含税总价。',
'所有表单输入必须使用 locator.type，不要使用 locator.fill。',
'在商品页和购物车分别用 locator.allTextContents 读取价格；导航到购物车后必须先调用 tab.url() 确认 cart.html，再读取购物车价格。',
'在结账概览用 locator.innerText 读取 Total；每次导航后调用 tab.url()，完成页使用 domSnapshot() 验证。',
'先观察再行动；browser.tabs.new() 已自动 grant，不要使用其他浏览器、shell 或网络工具。',
'完成验证后，把完成页作为 deliverable 调用 browser.tabs.finalize；finalize 必须是最后一个浏览器操作。',
].join('');
const MAX_OUTPUT_BYTES = 16 * 1024 * 1024;
interface CommandResult {
stdout: string;
stderr: string;
exitCode: number | null;
signal: NodeJS.Signals | null;
durationMs: number;
}
await withManagedChrome('sauce', async (chrome) => {
const apiKey = process.env['DASHSCOPE_API_KEY'];
assert(apiKey, 'DASHSCOPE_API_KEY is required');
const baseUrl = process.env['DASHSCOPE_BASE_URL'];
assert(baseUrl, 'DASHSCOPE_BASE_URL is required');
const model = process.env['QWEN_BROWSER_USE_MODEL'];
assert(model, 'QWEN_BROWSER_USE_MODEL is required');
const packageRoot = resolve(dirname(fileURLToPath(import.meta.url)), '../..');
const repositoryRoot = resolve(packageRoot, '../..');
const qwenEntry = join(repositoryRoot, 'dist/cli.js');
const nodeReplEntry = join(
repositoryRoot,
'packages/node-repl/dist/index.js',
);
const builtinSkillRoot = join(repositoryRoot, 'dist/bundled/browser-use');
const browserRuntimeRoot = join(builtinSkillRoot, 'runtime');
const browserRuntimeEntry = join(browserRuntimeRoot, 'index.js');
const browserModuleRoot = join(browserRuntimeRoot, 'node_modules');
await Promise.all(
[
qwenEntry,
nodeReplEntry,
join(builtinSkillRoot, 'SKILL.md'),
browserRuntimeEntry,
join(browserRuntimeRoot, 'native-host.js'),
join(browserModuleRoot, 'playwright-core/package.json'),
].map((path) => access(path)),
);
const runId = new Date()
.toISOString()
.replaceAll(':', '')
.replaceAll('.', '');
const runRoot = join(repositoryRoot, '.qwen/e2e-tests/browser-use', runId);
const providerLogs = join(runRoot, 'provider-logs');
const temporaryRoot = await mkdtemp(join(tmpdir(), 'qwen-browser-use-'));
try {
const qwenHome = join(temporaryRoot, 'qwen-home');
const workspace = join(temporaryRoot, 'workspace');
await mkdir(qwenHome, { recursive: true, mode: 0o700 });
await mkdir(workspace, { recursive: true, mode: 0o700 });
await mkdir(providerLogs, { recursive: true, mode: 0o700 });
const settings = {
general: { chatRecording: false },
telemetry: { enabled: false },
security: { auth: { selectedType: 'openai' } },
model: { name: model },
modelProviders: {
openai: [
{
id: model,
name: `[Dashscope] ${model}`,
baseUrl,
envKey: 'DASHSCOPE_API_KEY',
generationConfig: {
modalities: { image: true },
extra_body: { enable_thinking: true },
},
},
],
},
mcpServers: {
'node-repl': {
command: process.execPath,
args: [nodeReplEntry],
trust: true,
},
},
};
const settingsText = JSON.stringify(settings, null, 2) + '\n';
assert(!settingsText.includes(apiKey), 'API key entered settings');
await writeFile(join(qwenHome, 'settings.json'), settingsText, {
mode: 0o600,
});
await writeFile(
join(runRoot, 'manifest.json'),
JSON.stringify(
{
runId,
model,
prompt: PROMPT,
chromeVersion: chrome.chromeVersion,
managedChrome: true,
isolatedQwenHome: true,
},
null,
2,
) + '\n',
{ mode: 0o600 },
);
const environment: NodeJS.ProcessEnv = {
...process.env,
QWEN_HOME: qwenHome,
QWEN_BROWSER_USE_SOCKET_PATH: chrome.socketPath,
};
const qwen = await runCommand(
process.execPath,
[
qwenEntry,
'--auth-type',
'openai',
'--model',
model,
'--allowed-mcp-server-names',
'node-repl',
'--core-tools',
'skill,tool_search',
'--approval-mode',
'yolo',
'--max-wall-time',
'10m',
'--max-tool-calls',
'60',
'--max-session-turns',
'100',
'--chat-recording=false',
'--telemetry=false',
'--openai-logging',
'--openai-logging-dir',
providerLogs,
'--output-format',
'json',
'--prompt',
PROMPT,
],
workspace,
environment,
660_000,
);
assertNoSecret(apiKey, qwen.stdout, qwen.stderr);
await writeFile(join(runRoot, 'qwen-transcript.json'), qwen.stdout, {
mode: 0o600,
});
await writeFile(join(runRoot, 'qwen-stderr.txt'), qwen.stderr, {
mode: 0o600,
});
assertSuccess(qwen, 'Qwen SauceDemo smoke');
const events = JSON.parse(qwen.stdout.trim()) as unknown;
assert(Array.isArray(events), 'Qwen JSON output was not an event array');
const result = events.find(
(event) => isRecord(event) && event['type'] === 'result',
);
assert(
isRecord(result) &&
result['subtype'] === 'success' &&
result['is_error'] === false,
'Qwen did not finish successfully',
);
const finalAnswer = collectAssistantReport(events);
const calls = collectSuccessfulNodeReplCalls(events);
// Inventory and cart both render prices under `.inventory_item_price`;
// the model may or may not scope the selector to the row, so match the
// price class and tell the pages apart by the tab.url() confirmation of
// cart.html the prompt requires before the cart read.
const inventoryPrices = pricesFromCall(
calls,
'.inventory_item_price',
6,
'inventory',
);
const cartPrices = pricesFromCall(
calls,
'.inventory_item_price',
3,
'cart',
);
const lowestThreePrices = inventoryPrices
? [...inventoryPrices].sort((left, right) => left - right).slice(0, 3)
: [];
const cartMatchesLowestThree =
cartPrices !== null &&
JSON.stringify([...cartPrices].sort((left, right) => left - right)) ===
JSON.stringify(lowestThreePrices);
const completion = await verifySauceCompletion(chrome.socketPath);
const joinedOutput = calls.map((call) => call.output).join('\n');
const checks = {
existingNodeReplUsed: calls.length > 0,
browserSdkImported: stagedRuntimeImported(
calls,
browserRuntimeRoot,
builtinSkillRoot,
),
builtinModuleRootRegistered: successfulModuleRootRegistered(
events,
browserModuleRoot,
),
checkoutCompletionObserved:
completion.url === 'https://www.saucedemo.com/checkout-complete.html' &&
completion.snapshot.includes('Thank you for your order!'),
cartMatchesLowestThree,
reportedTotalIsExpected:
joinedOutput.includes('Total: $36.69') &&
finalAnswer.includes('$36.69'),
};
assert(
Object.values(checks).every(Boolean),
'SauceDemo checks failed: ' + JSON.stringify(checks),
);
const logFiles = await listFiles(providerLogs);
for (const file of logFiles) {
const bytes = await readFile(file);
assert(
!bytes.includes(Buffer.from(apiKey)),
'API key entered provider log',
);
await chmod(file, 0o600);
}
const summary = {
ok: true,
runId,
runDirectory: runRoot,
model,
durationMs: qwen.durationMs,
chromeVersion: chrome.chromeVersion,
checks,
reportedTotal: '$36.69',
lastObservedUrl: completion.url,
finalAnswer,
};
await writeFile(
join(runRoot, 'result.json'),
JSON.stringify(summary, null, 2) + '\n',
{ mode: 0o600 },
);
process.stdout.write(JSON.stringify(summary, null, 2) + '\n');
} finally {
await rm(temporaryRoot, { recursive: true, force: true });
}
});
async function runCommand(
program: string,
args: string[],
cwd: string,
env: NodeJS.ProcessEnv,
timeoutMs: number,
): Promise<CommandResult> {
return await new Promise((resolvePromise, rejectPromise) => {
const startedAt = Date.now();
const child = spawn(program, args, {
cwd,
env,
stdio: ['ignore', 'pipe', 'pipe'],
});
let stdout = '';
let stderr = '';
let bytes = 0;
let timedOut = false;
const append = (current: string, chunk: Buffer): string => {
bytes += chunk.byteLength;
if (bytes > MAX_OUTPUT_BYTES) child.kill('SIGTERM');
return current + chunk.toString('utf8');
};
child.stdout.on('data', (chunk: Buffer) => {
stdout = append(stdout, chunk);
});
child.stderr.on('data', (chunk: Buffer) => {
stderr = append(stderr, chunk);
});
const timer = setTimeout(() => {
timedOut = true;
child.kill('SIGTERM');
setTimeout(() => child.kill('SIGKILL'), 5_000).unref();
}, timeoutMs);
child.once('error', rejectPromise);
child.once('close', (exitCode, signal) => {
clearTimeout(timer);
if (bytes > MAX_OUTPUT_BYTES) {
rejectPromise(new Error('Qwen output exceeded the limit'));
} else if (timedOut) {
rejectPromise(new Error('Qwen command timed out'));
} else {
resolvePromise({
stdout,
stderr,
exitCode,
signal,
durationMs: Date.now() - startedAt,
});
}
});
});
}
function assertSuccess(result: CommandResult, label: string): void {
assert(
result.exitCode === 0,
label +
' failed with code ' +
String(result.exitCode) +
': ' +
result.stderr.slice(-4_000),
);
}
async function listFiles(root: string): Promise<string[]> {
const files: string[] = [];
for (const entry of await readdir(root, { withFileTypes: true })) {
const candidate = join(root, entry.name);
if (entry.isDirectory()) files.push(...(await listFiles(candidate)));
else if (entry.isFile()) files.push(candidate);
}
return files;
}
function assertNoSecret(secret: string, ...values: string[]): void {
assert(
values.every((value) => !value.includes(secret)),
'DASHSCOPE_API_KEY appeared in process output',
);
}
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === 'object' && value !== null && !Array.isArray(value);
}
interface NodeReplCall {
code: string;
output: string;
}
function collectSuccessfulNodeReplCalls(events: unknown[]): NodeReplCall[] {
const calls = new Map<string, { code: string; name: string }>();
for (const event of events) {
if (!isRecord(event) || event['type'] !== 'assistant') continue;
const message = event['message'];
if (!isRecord(message) || !Array.isArray(message['content'])) continue;
for (const block of message['content']) {
if (
!isRecord(block) ||
block['type'] !== 'tool_use' ||
typeof block['id'] !== 'string' ||
typeof block['name'] !== 'string' ||
!isRecord(block['input']) ||
typeof block['input']['code'] !== 'string'
) {
continue;
}
calls.set(block['id'], {
code: block['input']['code'],
name: block['name'],
});
}
}
const successful: NodeReplCall[] = [];
for (const event of events) {
if (!isRecord(event) || event['type'] !== 'user') continue;
const message = event['message'];
if (!isRecord(message) || !Array.isArray(message['content'])) continue;
for (const block of message['content']) {
if (
!isRecord(block) ||
block['type'] !== 'tool_result' ||
block['is_error'] !== false ||
typeof block['tool_use_id'] !== 'string' ||
typeof block['content'] !== 'string'
) {
continue;
}
const call = calls.get(block['tool_use_id']);
if (call?.name === 'mcp__node-repl__node_repl') {
successful.push({ code: call.code, output: block['content'] });
}
}
}
return successful;
}
function successfulModuleRootRegistered(
events: unknown[],
moduleRoot: string,
): boolean {
const calls = new Set<string>();
for (const event of events) {
if (!isRecord(event) || event['type'] !== 'assistant') continue;
const message = event['message'];
if (!isRecord(message) || !Array.isArray(message['content'])) continue;
for (const block of message['content']) {
if (
isRecord(block) &&
block['type'] === 'tool_use' &&
block['name'] === 'mcp__node-repl__node_repl_add_node_module_dir' &&
isRecord(block['input']) &&
block['input']['path'] === moduleRoot &&
typeof block['id'] === 'string'
) {
calls.add(block['id']);
}
}
}
return events.some((event) => {
if (!isRecord(event) || event['type'] !== 'user') return false;
const message = event['message'];
if (!isRecord(message) || !Array.isArray(message['content'])) return false;
return message['content'].some(
(block) =>
isRecord(block) &&
block['type'] === 'tool_result' &&
block['is_error'] === false &&
typeof block['tool_use_id'] === 'string' &&
calls.has(block['tool_use_id']),
);
});
}
// A run proves it loaded the staged built-in runtime only through an import
// expression that names that runtime: a literal path under the staged
// runtime directory, or a path the same or an earlier cell assembled from the
// skill root the CLI substituted into SKILL.md. The cell must also reach the
// setup export (a destructuring alias still names it). A literal entry loaded
// from anywhere else — the fallback a model reaches for when the staged
// import throws — disqualifies the run, because it is not the shipped
// runtime that was exercised.
function stagedRuntimeImported(
calls: NodeReplCall[],
browserRuntimeRoot: string,
builtinSkillRoot: string,
): boolean {
const entryFile = /\/(?:index|native-host)\.js/;
let rootBound = false;
let staged = false;
for (const call of calls) {
for (const root of [builtinSkillRoot, browserRuntimeRoot]) {
if (
call.code.includes(`'${root}'`) ||
call.code.includes(`"${root}"`) ||
call.code.includes(`\`${root}\``)
)
rootBound = true;
}
let importsStaged = false;
for (const match of call.code.matchAll(/import\(([^()]*)\)/g)) {
const expression = match[1] ?? '';
const literal = /^\s*(['"`])([^'"`$]+)\1\s*$/.exec(expression)?.[2];
if (literal !== undefined) {
if (literal.startsWith(`${browserRuntimeRoot}/`)) importsStaged = true;
else if (entryFile.test(literal)) return false;
continue;
}
if (/index\.js/.test(expression) && rootBound) importsStaged = true;
}
if (importsStaged && call.code.includes('setupBrowserRuntime'))
staged = true;
}
return staged;
}
// The prompt makes the model confirm cart.html with a separate tab.url() call
// before reading cart prices, so that confirmation is the boundary between
// the inventory and the cart: price reads before it belong to the inventory,
// reads after it to the cart. A read's own values come first in its output
// (a model may echo its selection afterwards in the same cell), so the
// leading set is the read. Without a cart confirmation there is no cart read.
function pricesFromCall(
calls: NodeReplCall[],
selector: string,
expectedCount: number,
scope: 'inventory' | 'cart',
): number[] | null {
const cartConfirmed = calls.findIndex(
(call) => call.code.includes('.url(') && call.output.includes('cart.html'),
);
if (scope === 'cart' && cartConfirmed === -1) return null;
const read = calls
.filter((_, index) =>
scope === 'cart'
? index > cartConfirmed
: cartConfirmed === -1 || index < cartConfirmed,
)
.filter(
(call) =>
call.code.includes(selector) && call.code.includes('allTextContents'),
)
.map((call) =>
[...call.output.matchAll(/\$(\d+(?:\.\d{2})?)/g)].map((match) =>
Number(match[1]),
),
)
.find(
(prices) =>
prices.length >= expectedCount && prices.every(Number.isFinite),
);
return read ? read.slice(0, expectedCount) : null;
}
async function verifySauceCompletion(socketPath: string): Promise<{
url: string;
snapshot: string;
}> {
const bridge = new ChromeExtensionTransport({ socketPath });
const runtime = new PlaywrightRuntime({ bridge });
try {
await bridge.start();
await bridge.request('ping');
const openTabs = await runtime.dispatch('browser.user.openTabs', {
browserId: runtime.browserId,
});
assert(Array.isArray(openTabs), 'Chrome returned an invalid tab list');
const completion = openTabs.find(
(tab) =>
isRecord(tab) &&
tab['url'] === 'https://www.saucedemo.com/checkout-complete.html',
);
assert(completion, 'SauceDemo completion tab was not open');
const claimed = await claimAfterPreviousSessionExits(runtime, completion);
assert(
isRecord(claimed) && typeof claimed['id'] === 'string',
'SauceDemo completion tab could not be claimed',
);
const url = await runtime.dispatch('tab.url', { tabId: claimed['id'] });
const snapshot = await runtime.dispatch('playwright.domSnapshot', {
tabId: claimed['id'],
});
assert(typeof url === 'string', 'SauceDemo completion URL was invalid');
assert(
typeof snapshot === 'string',
'SauceDemo completion snapshot was invalid',
);
return { url, snapshot };
} finally {
await runtime.stop();
}
}
async function claimAfterPreviousSessionExits(
runtime: PlaywrightRuntime,
tab: unknown,
): Promise<unknown> {
const deadline = Date.now() + 5_000;
while (true) {
try {
return await runtime.dispatch('browser.user.claimTab', {
browserId: runtime.browserId,
tab,
});
} catch (error) {
if (
Date.now() >= deadline ||
!(error instanceof Error) ||
!('code' in error) ||
error.code !== 'TAB_DEBUGGER_CONFLICT'
) {
throw error;
}
await new Promise((resolve) => setTimeout(resolve, 50));
}
}
}
function assert(condition: unknown, message: string): asserts condition {
if (!condition) throw new Error(message);
}
packages/browser-use/scripts/smoke-transcript.test.ts
+66
View File
@@ -0,0 +1,66 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { describe, expect, it } from 'vitest';
import { collectAssistantReport } from './smoke-transcript.js';
describe('smoke transcript report', () => {
it('retains the complete report before an extra tool call and short final response', () => {
const report = 'Order completed. Total: $36.69';
expect(
collectAssistantReport([
{
type: 'assistant',
message: { content: [{ type: 'text', text: report }] },
},
{
type: 'assistant',
message: { content: [{ type: 'tool_use', name: 'get_goal' }] },
},
{
type: 'user',
message: {
content: [{ type: 'tool_result', content: 'No active goal' }],
},
},
{
type: 'assistant',
message: { content: [{ type: 'text', text: 'Done' }] },
},
{ type: 'result', result: 'Done' },
]),
).toBe(`${report}\nDone`);
});
it('excludes reasoning, code, nested-agent reports, tool output, and malformed events', () => {
expect(
collectAssistantReport([
{
type: 'assistant',
message: {
content: [
{ type: 'thinking', thinking: 'private reasoning' },
{ type: 'tool_use', input: { code: 'printed text' } },
{ type: 'text', text: 'User report' },
],
},
},
{
type: 'assistant',
parent_tool_use_id: 'child',
message: { content: [{ type: 'text', text: 'Nested report' }] },
},
{
type: 'user',
message: { content: [{ type: 'text', text: 'User prompt' }] },
},
{ type: 'assistant' },
null,
'invalid event',
]),
).toBe('User report');
});
});
packages/browser-use/scripts/smoke-transcript.ts
+31
View File
@@ -0,0 +1,31 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
export function collectAssistantReport(events: unknown[]): string {
return events
.filter(
(event) =>
isRecord(event) &&
event.type === 'assistant' &&
event.parent_tool_use_id == null,
)
.flatMap((event) => {
if (
!isRecord(event) ||
!isRecord(event.message) ||
!Array.isArray(event.message.content)
)
return [];
return event.message.content.filter(isRecord);
})
.filter((block) => block.type === 'text')
.map((block) => (typeof block.text === 'string' ? block.text : ''))
.join('\n');
}
function isRecord(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === 'object' && !Array.isArray(value);
}
packages/browser-use/src/bridge/errors.ts
+2
View File
@@ -8,6 +8,7 @@ export type RuntimeErrorCode =
| 'BROWSER_DISCONNECTED'
| 'BROWSER_USE_BUSY'
| 'DIALOG_OPEN'
| 'EXTENSION_VERSION_MISMATCH'
| 'INVALID_ARGUMENT'
| 'INVALID_LOCATOR'
| 'INPUT_BLOCKED'
@@ -19,6 +20,7 @@ export type RuntimeErrorCode =
| 'PERMISSION_REQUIRED'
| 'STALE_TAB'
| 'STALE_BROWSER_SESSION'
| 'TAB_DEBUGGER_CONFLICT'
| 'TAB_NOT_GRANTED'
| 'TRANSPORT_UNAVAILABLE'
| 'UNKNOWN_METHOD'
packages/browser-use/src/bridge/native-host/index.test.ts
+235
View File
@@ -0,0 +1,235 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process';
import { once } from 'node:events';
import fs from 'node:fs';
import { createServer, type Server, type Socket } from 'node:net';
import os from 'node:os';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { build } from 'esbuild';
import { afterAll, afterEach, beforeAll, expect, test, vi } from 'vitest';
import { encodeFrame, FrameDecoder } from '../transport/framing.js';
import { encodeNativeMessagingOutput } from './native-messaging-output.js';
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'qbu-host-'));
const hostPath = path.join(root, 'host.cjs');
const children: ChildProcessWithoutNullStreams[] = [];
const servers: Server[] = [];
const sockets: Socket[] = [];
beforeAll(async () => {
await build({
entryPoints: [fileURLToPath(new URL('./index.ts', import.meta.url))],
outfile: hostPath,
bundle: true,
platform: 'node',
format: 'cjs',
});
});
afterEach(async () => {
for (const child of children.splice(0)) {
if (child.exitCode === null && child.signalCode === null) {
child.kill();
await once(child, 'exit');
}
}
for (const socket of sockets.splice(0)) socket.destroy();
for (const server of servers.splice(0))
await new Promise<void>((resolve) => server.close(() => resolve()));
});
afterAll(() => fs.rmSync(root, { recursive: true, force: true }));
function startHost(socketPath: string) {
const child = spawn(process.execPath, [hostPath], {
env: { ...process.env, QWEN_BROWSER_USE_SOCKET_PATH: socketPath },
stdio: 'pipe',
});
children.push(child);
return child;
}
async function backend(directory: string) {
fs.mkdirSync(directory, { recursive: true, mode: 0o700 });
const socketPath = path.join(directory, 'bridge.sock');
const received: unknown[] = [];
const server = createServer((socket) => {
sockets.push(socket);
const decoder = new FrameDecoder();
socket.on('data', (data) => received.push(...decoder.push(data)));
});
servers.push(server);
server.listen(socketPath);
await once(server, 'listening');
return { socketPath, received };
}
test.skipIf(process.platform === 'win32')(
'exits when no backend exists instead of retrying forever',
async () => {
const child = startHost(path.join(root, 'absent.sock'));
child.stdin.write(encodeFrame({ type: 'hello' }));
await vi.waitFor(() => expect(child.exitCode).toBe(0), { timeout: 10_000 });
},
15_000,
);
test.skipIf(process.platform === 'win32')(
'refuses a server in a shared writable directory before forwarding hello',
async () => {
const directory = path.join(root, 'shared');
const server = await backend(directory);
fs.chmodSync(directory, 0o777);
const child = startHost(server.socketPath);
child.stdin.write(encodeFrame({ type: 'hello' }));
await vi.waitFor(() => expect(child.exitCode).toBe(0), { timeout: 10_000 });
expect(server.received).toEqual([]);
},
15_000,
);
test.skipIf(process.platform === 'win32')(
'refuses a symlink socket before forwarding hello',
async () => {
const server = await backend(path.join(root, 'target'));
const link = path.join(root, 'link.sock');
fs.symlinkSync(server.socketPath, link);
const child = startHost(link);
child.stdin.write(encodeFrame({ type: 'hello' }));
await vi.waitFor(() => expect(child.exitCode).toBe(0), { timeout: 10_000 });
expect(server.received).toEqual([]);
},
15_000,
);
test.skipIf(process.platform === 'win32')(
'rejects a private socket reached through a replaceable ancestor',
async () => {
const server = await backend(path.join(root, 'safe', 'private'));
const shared = path.join(root, 'replaceable');
fs.mkdirSync(shared, { mode: 0o777 });
fs.chmodSync(shared, 0o777);
fs.symlinkSync(path.join(root, 'safe'), path.join(shared, 'link'));
const child = startHost(
path.join(shared, 'link', 'private', 'bridge.sock'),
);
child.stdin.write(encodeFrame({ type: 'hello' }));
await vi.waitFor(() => expect(child.exitCode).toBe(0), { timeout: 10_000 });
expect(server.received).toEqual([]);
},
15_000,
);
test.skipIf(process.platform === 'win32')(
'relays both ways after more than 60 seconds idle and exits on backend loss',
async () => {
const server = await backend(path.join(root, 'private'));
const child = startHost(server.socketPath);
const hello = { type: 'hello', protocolVersion: 1, extensionId: 'fixture' };
const output: unknown[] = [];
const decoder = new FrameDecoder();
child.stdout.on('data', (data: Buffer) =>
output.push(...decoder.push(data)),
);
child.stdin.write(encodeFrame(hello));
await vi.waitFor(() => expect(server.received).toEqual([hello]), {
timeout: 10_000,
});
await new Promise((resolve) => setTimeout(resolve, 61_000));
expect(child.exitCode).toBeNull();
sockets[0]!.write(
encodeFrame({ type: 'request', id: 'idle-ping', method: 'ping' }),
);
await vi.waitFor(() =>
expect(output).toContainEqual({
type: 'request',
id: 'idle-ping',
method: 'ping',
}),
);
child.stdin.write(
encodeFrame({ type: 'response', id: 'idle-ping', ok: true }),
);
await vi.waitFor(() => expect(server.received).toHaveLength(2));
sockets[0]!.destroy();
await vi.waitFor(() => expect(child.exitCode).toBe(0));
},
80_000,
);
test.skipIf(process.platform === 'win32').each(['eof', 'malformed'])(
'closes its socket when Chrome input ends with %s',
async (input) => {
const server = await backend(path.join(root, input));
const child = startHost(server.socketPath);
child.stdin.write(encodeFrame({ type: 'hello' }));
await vi.waitFor(() => expect(server.received).toHaveLength(1), {
timeout: 10_000,
});
const closed = once(sockets[0]!, 'close');
if (input === 'eof') child.stdin.end();
else child.stdin.write(Buffer.from([1, 0, 0, 0, 0]));
await vi.waitFor(
() => expect(child.exitCode).toBe(input === 'eof' ? 0 : 1),
{ timeout: 10_000 },
);
await closed;
},
15_000,
);
test.skipIf(process.platform === 'win32').each(['backend', 'stdin'])(
'drains a complete large response before exiting on %s close',
async (source) => {
const server = await backend(path.join(root, `drain-${source}`));
const child = startHost(server.socketPath);
const exited = once(child, 'close');
child.stdin.write(encodeFrame({ type: 'hello' }));
await vi.waitFor(() => expect(server.received).toHaveLength(1), {
timeout: 10_000,
});
const message = {
type: 'response',
id: 'large',
result: 'x'.repeat(2_000_000),
};
const readable = once(child.stdout, 'readable');
const backendClosed = once(sockets[0]!, 'close');
if (source === 'backend') sockets[0]!.end(encodeFrame(message));
else sockets[0]!.write(encodeFrame(message));
await readable;
if (source === 'stdin') child.stdin.end();
await backendClosed;
const output: Buffer[] = [];
child.stdout.on('data', (chunk: Buffer) => output.push(chunk));
child.stdout.resume();
expect((await exited)[0]).toBe(0);
const actual = Buffer.concat(output);
const expected = Buffer.concat(encodeNativeMessagingOutput(message, '1'));
expect(actual.length).toBe(expected.length);
expect(actual.equals(expected)).toBe(true);
},
15_000,
);
test.skipIf(process.platform === 'win32')(
'bounds shutdown when Chrome stops reading queued output',
async () => {
const server = await backend(path.join(root, 'blocked-reader'));
const child = startHost(server.socketPath);
child.stdin.write(encodeFrame({ type: 'hello' }));
await vi.waitFor(() => expect(server.received).toHaveLength(1), {
timeout: 10_000,
});
const readable = once(child.stdout, 'readable');
sockets[0]!.write(encodeFrame({ result: 'x'.repeat(2_000_000) }));
await readable;
child.stdin.end();
await vi.waitFor(() => expect(child.exitCode).toBe(0), { timeout: 5_000 });
},
15_000,
);
packages/browser-use/src/bridge/native-host/index.ts
+99
View File
@@ -0,0 +1,99 @@
#!/usr/bin/env node
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { connect, type Socket } from 'node:net';
import { defaultChromeBridgeSocketPath } from '../protocol.js';
import { verifySocketPeerPath } from '../socket-path.js';
import { encodeFrame, FrameDecoder } from '../transport/framing.js';
import { encodeNativeMessagingOutput } from './native-messaging-output.js';
const socketPath = defaultChromeBridgeSocketPath();
const nativeDecoder = new FrameDecoder();
const queued: unknown[] = [];
let latestHello: unknown;
let socket: Socket | undefined;
let outputSequence = 0;
let closing = false;
function shutdown(code = 0): void {
if (closing) return;
closing = true;
process.exitCode = code;
socket?.destroy();
process.stdin.destroy();
// Let queued stdout writes drain, but do not hang if Chrome stops reading.
setTimeout(() => process.exit(code), 2_000).unref();
}
async function connectBackend(): Promise<void> {
try {
await verifySocketPeerPath(socketPath);
} catch {
shutdown();
return;
}
if (closing) return;
const candidate = connect(socketPath);
socket = candidate;
candidate.once('connect', () => {
if (latestHello !== undefined) candidate.write(encodeFrame(latestHello));
for (const message of queued.splice(0))
candidate.write(encodeFrame(message));
});
candidate.once('error', () => shutdown());
candidate.once('close', () => shutdown());
const backendDecoder = new FrameDecoder();
candidate.on('data', (chunk) => {
try {
for (const message of backendDecoder.push(chunk)) {
outputSequence += 1;
for (const frame of encodeNativeMessagingOutput(
message,
String(outputSequence),
)) {
process.stdout.write(frame);
}
}
} catch {
shutdown(1);
}
});
}
process.stdin.on('data', (chunk: Buffer) => {
try {
for (const message of nativeDecoder.push(chunk)) {
if (
typeof message === 'object' &&
message !== null &&
'type' in message &&
message.type === 'hello'
) {
latestHello = message;
if (socket !== undefined && !socket.connecting && !socket.destroyed)
socket.write(encodeFrame(message));
continue;
}
if (socket !== undefined && !socket.connecting && !socket.destroyed)
socket.write(encodeFrame(message));
else {
queued.push(message);
if (queued.length > 100) queued.shift();
}
}
} catch {
shutdown(1);
}
});
process.stdin.on('end', () => shutdown());
process.stdin.on('error', () => shutdown(1));
process.stdout.on('error', () => shutdown());
void connectBackend();
packages/browser-use/src/bridge/native-host/native-messaging-output.test.ts
+108
View File
@@ -0,0 +1,108 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import assert from 'node:assert/strict';
import { test } from 'vitest';
import { encodeFrame, FrameDecoder } from '../transport/framing.js';
import {
encodeNativeMessagingOutput,
NATIVE_MESSAGE_CHUNK_TYPE,
} from './native-messaging-output.js';
// The wire contract this module exists to honour (docs/design/browser-use.md:
// "Native Host messages sent to Chrome are limited to 1 MiB"). Pinned as a
// literal on purpose: a boundary derived from the module's own threshold
// would follow the threshold if it were ever raised past what Chrome accepts.
const CHROME_NATIVE_MESSAGE_LIMIT_BYTES = 1_048_576;
interface ChunkEnvelope {
type: string;
id: string;
index: number;
total: number;
data: string;
}
function decodeAll(frames: Buffer[]): ChunkEnvelope[] {
return frames.flatMap(
(frame) => new FrameDecoder().push(frame) as ChunkEnvelope[],
);
}
/** A request whose encoded frame is exactly `frameBytes` long. */
function messageWithFrameLength(frameBytes: number): {
type: string;
id: string;
text: string;
} {
const base = { type: 'request', id: 'boundary', text: '' };
const overhead = encodeFrame(base).length;
return { ...base, text: 'a'.repeat(frameBytes - overhead) };
}
test('chunks host output below the Chrome Native Messaging limit', () => {
const message = {
type: 'request',
id: 'large-request',
text: '\u0000'.repeat(1_000_000),
};
const frames = encodeNativeMessagingOutput(message, 'chunk-1');
assert.ok(frames.length > 1);
assert.ok(
frames.every((frame) => frame.length <= CHROME_NATIVE_MESSAGE_LIMIT_BYTES),
);
const decoded = decodeAll(frames);
assert.equal(decoded.length, frames.length);
// The envelope the extension validates before reassembly: the caller's id,
// a 0-based contiguous index in emission order, and the group total.
decoded.forEach((part, index) => {
const { data, ...envelope } = part;
assert.deepEqual(envelope, {
type: NATIVE_MESSAGE_CHUNK_TYPE,
id: 'chunk-1',
index,
total: frames.length,
});
assert.equal(typeof data, 'string');
});
const payload = Buffer.concat(
decoded.map((part) => Buffer.from(part.data, 'base64')),
);
assert.deepEqual(JSON.parse(payload.toString()), message);
});
test('sends a frame exactly at the Chrome limit unchunked', () => {
const message = messageWithFrameLength(CHROME_NATIVE_MESSAGE_LIMIT_BYTES);
const frames = encodeNativeMessagingOutput(message, 'chunk-2');
assert.equal(frames.length, 1);
assert.equal(frames[0]!.length, CHROME_NATIVE_MESSAGE_LIMIT_BYTES);
assert.deepEqual(new FrameDecoder().push(frames[0]!), [message]);
});
test('chunks a frame one byte over the Chrome limit', () => {
const message = messageWithFrameLength(CHROME_NATIVE_MESSAGE_LIMIT_BYTES + 1);
const frames = encodeNativeMessagingOutput(message, 'chunk-3');
assert.ok(frames.length > 1);
assert.ok(
frames.every((frame) => frame.length <= CHROME_NATIVE_MESSAGE_LIMIT_BYTES),
);
const decoded = decodeAll(frames);
assert.deepEqual(
decoded.map(({ type, id, index, total }) => ({ type, id, index, total })),
decoded.map((_, index) => ({
type: NATIVE_MESSAGE_CHUNK_TYPE,
id: 'chunk-3',
index,
total: frames.length,
})),
);
const payload = Buffer.concat(
decoded.map((part) => Buffer.from(part.data, 'base64')),
);
assert.deepEqual(JSON.parse(payload.toString()), message);
});
packages/browser-use/src/bridge/native-host/native-messaging-output.ts
+33
View File
@@ -0,0 +1,33 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { encodeFrame } from '../transport/framing.js';
export const NATIVE_MESSAGE_CHUNK_TYPE = 'qwen.browser.chunk';
const MAX_NATIVE_MESSAGE_BYTES = 1_048_576;
const CHUNK_BYTES = 700_000;
export function encodeNativeMessagingOutput(
message: unknown,
chunkId: string,
): Buffer[] {
const direct = encodeFrame(message);
if (direct.length <= MAX_NATIVE_MESSAGE_BYTES) return [direct];
const payload = Buffer.from(JSON.stringify(message));
const total = Math.ceil(payload.length / CHUNK_BYTES);
return Array.from({ length: total }, (_, index) =>
encodeFrame({
type: NATIVE_MESSAGE_CHUNK_TYPE,
id: chunkId,
index,
total,
data: payload
.subarray(index * CHUNK_BYTES, (index + 1) * CHUNK_BYTES)
.toString('base64'),
}),
);
}
packages/browser-use/src/bridge/protocol.ts
+4 -6
View File
@@ -7,7 +7,7 @@
import { statSync } from 'node:fs';
import { posix } from 'node:path';
export const CHROME_BRIDGE_PROTOCOL_VERSION = 1;
export const CHROME_BRIDGE_PROTOCOL_VERSION = 2;
export const CHROME_NATIVE_HOST_NAME = 'com.qwen.browser';
export const CHROME_EXTENSION_ID = 'idkijaaipeeinemigojbjkmfmabokbdk';
export const MAX_BRIDGE_FRAME_BYTES = 16 * 1024 * 1024;
@@ -29,12 +29,9 @@ export function defaultChromeBridgeSocketPath(
}
const uid =
typeof process.getuid === 'function' ? process.getuid() : 'default';
// The win32 branch returned above; keep the remaining joins POSIX so the
// The win32 branch returned above; keep the remaining join POSIX so the
// derived path is a pure function of uid and platform on every host.
return posix.join(
defaultChromeBridgeSocketDirectory(uid),
`qwen-browser-use-${uid}.sock`,
);
return posix.join(defaultChromeBridgeSocketDirectory(uid), 'bridge.sock');
}
interface DirectoryStat {
@@ -85,6 +82,7 @@ export interface BridgeHello {
type: 'hello';
protocolVersion: number;
extensionId: string;
extensionInstanceId: string;
}
export interface BridgeRequest {
packages/browser-use/src/bridge/transport/ensure-socket-directory.test.ts → packages/browser-use/src/bridge/socket-path.test.ts
+15 -24
View File
@@ -10,9 +10,8 @@ import os from 'node:os';
import path from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
// A scripted lstat simulates the lstat→mkdir race: the directory appears
// between the existence check and the create, owned by someone else or with
// permissive bits.
// A scripted lstat returns the directory seen after mkdir, including a
// concurrent creator that supplied unsafe ownership or permissions.
const lstatMock = vi.hoisted(() => vi.fn());
// A scripted chmod fails the post-listen step of start() while the socket is
// already bound and accepting.
@@ -74,12 +73,10 @@ import {
CHROME_BRIDGE_PROTOCOL_VERSION,
CHROME_EXTENSION_ID,
type BridgeRequest,
} from '../protocol.js';
import {
ChromeExtensionTransport,
ensureSocketDirectory,
} from './chrome-extension-transport.js';
import { encodeFrame, FrameDecoder } from './framing.js';
} from './protocol.js';
import { ChromeExtensionTransport } from './transport/chrome-extension-transport.js';
import { prepareSocketDirectory } from './socket-path.js';
import { encodeFrame, FrameDecoder } from './transport/framing.js';
const roots: string[] = [];
const transports: ChromeExtensionTransport[] = [];
@@ -115,7 +112,7 @@ function directoryInfo(uid: number, mode: number) {
};
}
describe('ensureSocketDirectory create race', () => {
describe('prepareSocketDirectory creation checks', () => {
it.skipIf(process.platform === 'win32' || owner === undefined)(
'rejects a permissive directory that appears during creation',
async () => {
@@ -124,18 +121,13 @@ describe('ensureSocketDirectory create race', () => {
const leaf = path.join(root, 'leaf');
lstatMock.mockImplementation(async (target: string) => {
if (target === leaf) {
// First look: missing. After mkdir: a permissive entry the
// mkdir(0o700) could not have produced itself.
const seen = lstatMock.mock.calls.filter(
([candidate]) => candidate === leaf,
).length;
if (seen === 1)
throw Object.assign(new Error('missing'), { code: 'ENOENT' });
return directoryInfo(owner!, 0o755);
}
return await fs.promises.lstat(target);
});
await expect(ensureSocketDirectory(leaf)).rejects.toThrow('not usable');
await expect(
prepareSocketDirectory(path.join(leaf, 'bridge.sock')),
).rejects.toThrow('private user-owned directory');
},
);
@@ -147,16 +139,13 @@ describe('ensureSocketDirectory create race', () => {
const leaf = path.join(root, 'leaf');
lstatMock.mockImplementation(async (target: string) => {
if (target === leaf) {
const seen = lstatMock.mock.calls.filter(
([candidate]) => candidate === leaf,
).length;
if (seen === 1)
throw Object.assign(new Error('missing'), { code: 'ENOENT' });
return directoryInfo(owner! + 1, 0o700);
}
return await fs.promises.lstat(target);
});
await expect(ensureSocketDirectory(leaf)).rejects.toThrow('not usable');
await expect(
prepareSocketDirectory(path.join(leaf, 'bridge.sock')),
).rejects.toThrow('private user-owned directory');
},
);
});
@@ -180,6 +169,7 @@ describe('start failure after listen', () => {
type: 'hello',
protocolVersion: CHROME_BRIDGE_PROTOCOL_VERSION,
extensionId: CHROME_EXTENSION_ID,
extensionInstanceId: 'deadline-test',
}),
);
await vi.waitFor(() => expect(transport.isConnected()).toBe(true));
@@ -220,6 +210,7 @@ describe('request deadlines', () => {
type: 'hello',
protocolVersion: CHROME_BRIDGE_PROTOCOL_VERSION,
extensionId: CHROME_EXTENSION_ID,
extensionInstanceId: 'deadline-test',
}),
);
await vi.waitFor(() => expect(transport.isConnected()).toBe(true));
packages/browser-use/src/bridge/socket-path.ts
+66
View File
@@ -0,0 +1,66 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { lstat, mkdir, realpath, stat } from 'node:fs/promises';
import { dirname, resolve } from 'node:path';
export async function prepareSocketDirectory(
socketPath: string,
): Promise<void> {
if (process.platform === 'win32') return;
await mkdir(dirname(socketPath), { mode: 0o700 }).catch((error: unknown) => {
if ((error as NodeJS.ErrnoException).code !== 'EEXIST') throw error;
});
await verifySocketDirectory(socketPath);
}
async function verifySocketDirectory(socketPath: string): Promise<void> {
if (process.platform === 'win32') return;
const directory = dirname(socketPath);
const info = await lstat(directory);
if (
!info.isDirectory() ||
info.uid !== process.getuid!() ||
(info.mode & 0o077) !== 0
) {
throw new Error(
`Chrome bridge requires a private user-owned directory: ${directory}`,
);
}
// Ancestors must not let another user replace the private directory. Root's
// sticky /tmp is safe; canonicalization also handles macOS /var and /tmp.
for (const start of [resolve(directory), await realpath(directory)]) {
let ancestor = dirname(start);
for (;;) {
const entry = await lstat(ancestor);
const parent = await stat(ancestor);
if (
(entry.uid !== 0 && entry.uid !== process.getuid!()) ||
(parent.uid !== 0 && parent.uid !== process.getuid!()) ||
((parent.mode & 0o022) !== 0 &&
!(parent.uid === 0 && (parent.mode & 0o1000) !== 0))
) {
throw new Error(
`Chrome bridge socket has an unsafe ancestor: ${ancestor}`,
);
}
const next = dirname(ancestor);
if (next === ancestor) break;
ancestor = next;
}
}
}
export async function verifySocketPeerPath(socketPath: string): Promise<void> {
if (process.platform === 'win32') return;
await verifySocketDirectory(socketPath);
const info = await lstat(socketPath);
if (!info.isSocket() || info.uid !== process.getuid!()) {
throw new Error(
`Chrome bridge requires a user-owned socket: ${socketPath}`,
);
}
}
packages/browser-use/src/bridge/transport/chrome-extension-transport.test.ts
+510 -76
View File
@@ -4,7 +4,9 @@
* SPDX-License-Identifier: Apache-2.0
*/
import { connect } from 'node:net';
import { connect, type Socket } from 'node:net';
import { once } from 'node:events';
import { randomUUID } from 'node:crypto';
import { spawnSync } from 'node:child_process';
import fs from 'node:fs';
import { createRequire } from 'node:module';
@@ -18,17 +20,17 @@ import {
CHROME_BRIDGE_PROTOCOL_VERSION,
CHROME_EXTENSION_ID,
MAX_BRIDGE_FRAME_BYTES,
defaultChromeBridgeSocketDirectory,
defaultChromeBridgeSocketPath,
defaultChromeBridgeSocketDirectory,
type BridgeRequest,
} from '../protocol.js';
import {
ChromeExtensionTransport,
ensureSocketDirectory,
isAddressInUse,
type ChromeExtensionTransportOptions,
} from './chrome-extension-transport.js';
import { encodeFrame, FrameDecoder } from './framing.js';
import { PlaywrightRuntime } from '../../playwright/playwright-runtime.js';
const roots: string[] = [];
const transports: ChromeExtensionTransport[] = [];
@@ -40,65 +42,96 @@ afterEach(async () => {
}
});
describe('ensureSocketDirectory', () => {
it('creates a missing directory with owner-only permissions', async () => {
if (process.platform === 'win32') return;
// The ancestor sweep rejects symlinks, and macOS reaches its temp root
// through one (/var -> /private/var), so resolve the real path first.
const root = fs.mkdtempSync(
path.join(fs.realpathSync(os.tmpdir()), 'qbu-dir-'),
);
roots.push(root);
const dir = path.join(root, 'qwen-browser-use', '1000');
await ensureSocketDirectory(dir);
expect(fs.statSync(dir).mode & 0o777).toBe(0o700);
});
it('tightens a permissive owned directory and rejects a non-directory', async () => {
if (process.platform === 'win32') return;
const root = fs.mkdtempSync(
path.join(fs.realpathSync(os.tmpdir()), 'qbu-dir-'),
);
roots.push(root);
const dir = path.join(root, 'loose');
fs.mkdirSync(dir, { mode: 0o755 });
await ensureSocketDirectory(dir);
expect(fs.statSync(dir).mode & 0o077).toBe(0);
const squat = path.join(root, 'squat');
fs.writeFileSync(squat, '');
await expect(ensureSocketDirectory(squat)).rejects.toThrow('not usable');
});
it('rejects a socket directory behind a symlinked parent', async () => {
if (process.platform === 'win32') return;
const root = fs.mkdtempSync(
path.join(fs.realpathSync(os.tmpdir()), 'qbu-dir-'),
);
roots.push(root);
const target = path.join(root, 'target');
fs.mkdirSync(target);
fs.symlinkSync(target, path.join(root, 'link'));
await expect(
ensureSocketDirectory(path.join(root, 'link', 'leaf')),
).rejects.toThrow('not usable');
});
it('rejects a socket directory under a group/other-writable ancestor', async () => {
if (process.platform === 'win32') return;
const root = fs.mkdtempSync(
path.join(fs.realpathSync(os.tmpdir()), 'qbu-dir-'),
);
roots.push(root);
const loose = path.join(root, 'loose');
fs.mkdirSync(loose);
fs.chmodSync(loose, 0o777);
await expect(
ensureSocketDirectory(path.join(loose, 'leaf')),
).rejects.toThrow('not usable');
});
});
describe('ChromeExtensionTransport', () => {
it.each(['before', 'during'])(
'reports an outdated extension seen %s the connection wait',
async (when) => {
const transport = await startProfileTransport();
const wait = () =>
transport.request('ping', {}, 150).catch((error: unknown) => error);
const pending = when === 'during' ? wait() : undefined;
await rejectedHello(transport, { protocolVersion: 1 });
expect(await (pending ?? wait())).toMatchObject({
code: 'EXTENSION_VERSION_MISMATCH',
message: expect.stringContaining('extension is out of date'),
});
expect(transport.isConnected()).toBe(false);
await transport.stop();
await transport.start();
await expect(transport.request('ping', {}, 20)).rejects.toMatchObject({
message: expect.stringContaining('extension is not connected'),
});
},
);
it('allows a compatible profile to connect after an incompatible hello', async () => {
const transport = await startProfileTransport();
const pending = transport.request('ping');
await rejectedHello(transport, { protocolVersion: 1 });
const owner = await connectProfile(transport, 'profile-a');
await expect(pending).resolves.toBe('profile-a');
owner.destroy();
await vi.waitFor(() => expect(transport.isConnected()).toBe(false));
await expect(transport.request('ping', {}, 20)).rejects.toMatchObject({
message: expect.stringContaining('extension is not connected'),
});
});
it.each([
{ extensionId: 'other-extension', protocolVersion: 1 },
{ protocolVersion: '1' },
{ protocolVersion: -1 },
])('does not diagnose an upgrade from an invalid peer: %j', async (hello) => {
const transport = await startProfileTransport();
await rejectedHello(transport, hello);
await expect(transport.request('ping', {}, 20)).rejects.toMatchObject({
message: expect.stringContaining('extension is not connected'),
});
});
it('scopes mismatch guidance to discovery or the disconnected selected profile', async () => {
const transport = await startProfileTransport();
const owner = await connectProfile(transport, 'profile-a');
await expect(transport.request('ping')).resolves.toBe('profile-a');
await rejectedHello(transport, { protocolVersion: 1 });
await expect(transport.request('ping')).resolves.toBe('profile-a');
owner.destroy();
await vi.waitFor(() => expect(transport.isConnected()).toBe(false));
await rejectedHello(transport, {
protocolVersion: 1,
extensionInstanceId: 'profile-b',
});
await expect(transport.request('ping', {}, 20)).rejects.toMatchObject({
message: expect.stringContaining('extension is not connected'),
});
await rejectedHello(transport, {
protocolVersion: CHROME_BRIDGE_PROTOCOL_VERSION + 1,
extensionInstanceId: 'profile-a',
});
await expect(transport.request('ping', {}, 20)).rejects.toMatchObject({
code: 'EXTENSION_VERSION_MISMATCH',
message: expect.stringContaining('Update Qwen Code'),
});
await connectProfile(transport, 'profile-a');
await expect(transport.request('ping')).resolves.toBe('profile-a');
});
it('surfaces an outdated extension from browser discovery instead of an empty list', async () => {
const transport = await startProfileTransport({ connectTimeoutMs: 200 });
await rejectedHello(transport, { protocolVersion: 1 });
const runtime = new PlaywrightRuntime({ bridge: transport });
await expect(runtime.dispatch('browsers.list', {})).rejects.toMatchObject({
code: 'EXTENSION_VERSION_MISMATCH',
message: expect.stringContaining('out of date'),
});
});
it('reports no browsers when no extension connects at all', async () => {
const transport = await startProfileTransport({ connectTimeoutMs: 200 });
const runtime = new PlaywrightRuntime({ bridge: transport });
await expect(runtime.dispatch('browsers.list', {})).resolves.toEqual([]);
});
it.skipIf(process.platform === 'win32').each([
['Input.dispatchMouseEvent', undefined],
['Input.dispatchKeyEvent', undefined],
@@ -202,13 +235,185 @@ describe('ChromeExtensionTransport', () => {
},
);
it('parks other profiles without interrupting requests or accepting their events', async () => {
const transport = await startProfileTransport();
const changes: boolean[] = [];
const events: unknown[] = [];
transport.onConnectionChange((connected) => changes.push(connected));
transport.onEvent((event) => events.push(event));
let requestId: string | undefined;
const a = await connectProfile(transport, 'profile-a', (request) => {
requestId = request.id;
});
const pending = transport.request('cdp.send', {
tabId: 7,
method: 'Input.dispatchMouseEvent',
});
await vi.waitFor(() => expect(requestId).toBeDefined());
const b = await connectProfile(transport, 'profile-b');
b.write(
encodeFrame({
type: 'response',
id: requestId,
ok: true,
result: 'wrong profile',
}),
);
b.write(
encodeFrame({
type: 'event',
tabId: 7,
method: 'Page.javascriptDialogOpening',
params: { type: 'alert', message: 'other profile' },
}),
);
a.write(encodeFrame({ type: 'event', tabId: 7, method: 'Page.fromA' }));
await vi.waitFor(() => expect(events).toHaveLength(1));
a.write(
encodeFrame({
type: 'response',
id: requestId,
ok: true,
result: 'profile-a:7',
}),
);
await expect(pending).resolves.toBe('profile-a:7');
expect(events).toEqual([
{ type: 'event', tabId: 7, method: 'Page.fromA', params: undefined },
]);
expect(changes).toEqual([true]);
expect(a.destroyed).toBe(false);
expect(b.destroyed).toBe(false);
b.destroy();
await vi.waitFor(() => expect(b.closed).toBe(true));
// `b.closed` flips synchronously on destroy(); the server observes the
// close only on a later event-loop turn, so settle before asserting.
await new Promise((resolve) => setTimeout(resolve, 20));
expect(changes).toEqual([true]);
expect(transport.isConnected()).toBe(true);
const firstRequestId = requestId;
const followUp = transport.request('ping', {}, 500);
await vi.waitFor(() => expect(requestId).not.toBe(firstRequestId));
a.write(
encodeFrame({
type: 'response',
id: requestId,
ok: true,
result: 'profile-a:still-selected',
}),
);
await expect(followUp).resolves.toBe('profile-a:still-selected');
expect(changes).toEqual([true]);
expect(a.destroyed).toBe(false);
});
it('waits for the original profile after disconnect instead of failing over', async () => {
const transport = await startProfileTransport();
const a = await connectProfile(transport, 'profile-a');
await expect(transport.request('ping')).resolves.toBe('profile-a');
await connectProfile(transport, 'profile-b');
a.destroy();
await vi.waitFor(() => expect(transport.isConnected()).toBe(false));
await connectProfile(transport, 'profile-b');
await expect(transport.request('ping', {}, 20)).rejects.toMatchObject({
code: 'BROWSER_DISCONNECTED',
});
const waiting = transport.request('ping');
await connectProfile(transport, 'profile-a');
await expect(waiting).resolves.toBe('profile-a');
});
it('can select another profile after the transport is stopped and restarted', async () => {
const transport = await startProfileTransport();
const a = await connectProfile(transport, 'profile-a');
await expect(transport.request('ping')).resolves.toBe('profile-a');
const b = await connectProfile(transport, 'profile-b');
await transport.stop();
await vi.waitFor(() => expect(a.destroyed && b.destroyed).toBe(true));
await transport.start();
await connectProfile(transport, 'profile-b');
await expect(transport.request('ping')).resolves.toBe('profile-b');
});
it.each([undefined, '', ' ', 42, 'x'.repeat(129)])(
'rejects an invalid instance identity (%s) before selecting a profile',
async (extensionInstanceId) => {
const transport = await startProfileTransport();
const candidate = connect(transport.socketPath);
candidate.on('error', () => undefined);
await once(candidate, 'connect');
candidate.write(
encodeFrame({
type: 'hello',
protocolVersion: CHROME_BRIDGE_PROTOCOL_VERSION,
extensionId: CHROME_EXTENSION_ID,
extensionInstanceId,
}),
);
await once(candidate, 'close');
expect(transport.isConnected()).toBe(false);
await connectProfile(transport, 'profile-b');
await expect(transport.request('ping')).resolves.toBe('profile-b');
},
);
it.skipIf(process.platform === 'win32')(
'delivers a response ahead of the events that share its socket chunk',
async () => {
// Playwright installs a page's renderer and lifecycle listeners inside
// `Page.getFrameTree().then(...)`. A response only settles a promise, so
// its consumer runs on the microtask queue; events emitted synchronously
// from the same chunk overtook it, found no listener, and a claimed tab
// never regained its main world. Delivery must follow arrival order.
const { transport, socket, requests } = await connectedTransport();
const order: string[] = [];
transport.onEvent((event) => order.push(`event:${event.method}`));
const result = transport
.request('cdp.send', { tabId: 7, method: 'Page.getFrameTree' }, 1_000)
.then((value) => {
order.push('response:Page.getFrameTree');
return value;
});
await vi.waitFor(() => expect(requests).toHaveLength(1));
socket.write(
Buffer.concat([
encodeFrame({
type: 'response',
id: requests[0]!.id,
ok: true,
result: { frameTree: {} },
}),
encodeFrame({
type: 'event',
tabId: 7,
method: 'Runtime.executionContextCreated',
params: {},
}),
encodeFrame({
type: 'event',
tabId: 7,
method: 'Page.lifecycleEvent',
params: { name: 'load' },
}),
]),
);
await expect(result).resolves.toEqual({ frameTree: {} });
await vi.waitFor(() => expect(order).toHaveLength(3));
expect(order).toEqual([
'response:Page.getFrameTree',
'event:Runtime.executionContextCreated',
'event:Page.lifecycleEvent',
]);
},
);
it.skipIf(process.platform === 'win32')(
'does not reopen a stopped socket for a late request',
async () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'qbu-transport-'));
roots.push(root);
const transport = new ChromeExtensionTransport({
socketPath: path.join(root, 'bridge.sock'),
socketPath: testSocketPath(root),
connectTimeoutMs: 20,
});
transports.push(transport);
@@ -310,14 +515,54 @@ describe('ChromeExtensionTransport', () => {
expect(defaultChromeBridgeSocketPath({ TMPDIR: '/tmp/one' })).toBe(
defaultChromeBridgeSocketPath({ TMPDIR: '/tmp/two' }),
);
expect(path.dirname(defaultChromeBridgeSocketPath({}))).not.toBe('/tmp');
expect(
defaultChromeBridgeSocketPath({
AGENT_BROWSER_SOCKET_PATH: '/tmp/legacy.sock',
}),
).not.toBe('/tmp/legacy.sock');
expect(CHROME_BRIDGE_PROTOCOL_VERSION).toBe(1);
expect(CHROME_BRIDGE_PROTOCOL_VERSION).toBe(2);
});
it.skipIf(process.platform === 'win32')(
'refuses a shared socket directory without changing its permissions',
async () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'qbu-transport-'));
roots.push(root);
fs.chmodSync(root, 0o777);
const transport = new ChromeExtensionTransport({
socketPath: testSocketPath(root),
});
transports.push(transport);
await expect(transport.start()).rejects.toMatchObject({
code: 'TRANSPORT_UNAVAILABLE',
});
expect(fs.statSync(root).mode & 0o777).toBe(0o777);
},
);
it.skipIf(process.platform === 'win32')(
'names the private-directory requirement for a world-readable socket directory',
async () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'qbu-transport-'));
roots.push(root);
const shared = path.join(root, 'shared');
fs.mkdirSync(shared, { mode: 0o755 });
fs.chmodSync(shared, 0o755);
const transport = new ChromeExtensionTransport({
socketPath: path.join(shared, 'bridge.sock'),
});
transports.push(transport);
await expect(transport.start()).rejects.toMatchObject({
code: 'TRANSPORT_UNAVAILABLE',
message: expect.stringContaining(
`requires a private user-owned directory: ${shared}`,
),
});
expect(fs.existsSync(transport.socketPath)).toBe(false);
},
);
it('honours an explicit QWEN_BROWSER_USE_SOCKET_PATH and ignores a blank one', () => {
expect(
defaultChromeBridgeSocketPath({
@@ -382,6 +627,56 @@ describe('ChromeExtensionTransport', () => {
).toBe('/tmp');
});
it.skipIf(process.platform === 'win32')(
'creates a private socket directory and rejects directory symlinks',
async () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'qbu-transport-'));
roots.push(root);
const directory = path.join(root, 'private');
const transport = new ChromeExtensionTransport({
socketPath: path.join(directory, 'bridge.sock'),
});
transports.push(transport);
await transport.start();
expect(fs.statSync(directory).mode & 0o777).toBe(0o700);
expect(fs.statSync(transport.socketPath).mode & 0o777).toBe(0o600);
const link = path.join(root, 'link');
fs.symlinkSync(directory, link);
const linked = new ChromeExtensionTransport({
socketPath: path.join(link, 'bridge.sock'),
});
transports.push(linked);
await expect(linked.start()).rejects.toMatchObject({
code: 'TRANSPORT_UNAVAILABLE',
});
expect(transport.isConnected()).toBe(false);
expect(fs.existsSync(transport.socketPath)).toBe(true);
},
);
it.skipIf(process.platform === 'win32')(
'rejects a private directory belonging to another UID',
async () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'qbu-transport-'));
roots.push(root);
const transport = new ChromeExtensionTransport({
socketPath: testSocketPath(root),
});
transports.push(transport);
const uid = vi
.spyOn(process as { getuid: () => number }, 'getuid')
.mockReturnValue(fs.statSync(root).uid + 1);
try {
await expect(transport.start()).rejects.toMatchObject({
code: 'TRANSPORT_UNAVAILABLE',
});
expect(fs.existsSync(transport.socketPath)).toBe(false);
} finally {
uid.mockRestore();
}
},
);
it('derives the macOS socket directory without reading ambient TMPDIR', () => {
const original = process.env.TMPDIR;
try {
@@ -477,7 +772,7 @@ describe('ChromeExtensionTransport', () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'qbu-transport-'));
roots.push(root);
const transport = new ChromeExtensionTransport({
socketPath: path.join(root, 'bridge.sock'),
socketPath: testSocketPath(root),
});
transports.push(transport);
await transport.start();
@@ -488,6 +783,7 @@ describe('ChromeExtensionTransport', () => {
type: 'hello',
protocolVersion: CHROME_BRIDGE_PROTOCOL_VERSION,
extensionId: CHROME_EXTENSION_ID,
extensionInstanceId: 'profile-a',
}),
);
const decoder = new FrameDecoder();
@@ -495,18 +791,33 @@ describe('ChromeExtensionTransport', () => {
for (const message of decoder.push(chunk)) {
const request = message as { id: string; method: string };
socket.write(
encodeFrame({
type: 'response',
id: request.id,
ok: true,
result: { method: request.method },
}),
encodeFrame(
request.method === 'conflict'
? {
type: 'response',
id: request.id,
ok: false,
error: {
code: 'TAB_DEBUGGER_CONFLICT',
message: 'Another debugger is already attached',
},
}
: {
type: 'response',
id: request.id,
ok: true,
result: { method: request.method },
},
),
);
}
});
await expect(transport.request('ping')).resolves.toEqual({
method: 'ping',
});
await expect(transport.request('conflict')).rejects.toMatchObject({
code: 'TAB_DEBUGGER_CONFLICT',
});
const events: unknown[] = [];
transport.onEvent((event) => events.push(event));
socket.write(
@@ -539,13 +850,59 @@ describe('ChromeExtensionTransport', () => {
},
);
it('allows browser listing to wait for discovery while keeping explicit probes short', async () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'qbu-transport-'));
roots.push(root);
const transport = new ChromeExtensionTransport({
socketPath: testSocketPath(root),
connectTimeoutMs: 5_000,
requestTimeoutMs: 500,
});
transports.push(transport);
await transport.start();
await expect(transport.request('ping', {}, 5)).rejects.toMatchObject({
code: 'BROWSER_DISCONNECTED',
});
const runtime = new PlaywrightRuntime({ bridge: transport });
const request = runtime.dispatch('browsers.list', {});
// Discovery also needs to outlast the old 1.5-second browser-list probe.
await new Promise((resolve) => setTimeout(resolve, 1_800));
const socket = connect(transport.socketPath);
const decoder = new FrameDecoder();
socket.on('data', (chunk: Buffer) => {
for (const message of decoder.push(chunk)) {
socket.write(
encodeFrame({
type: 'response',
id: (message as { id: string }).id,
ok: true,
result: 'ready',
}),
);
}
});
await new Promise<void>((resolve) => socket.once('connect', resolve));
socket.write(
encodeFrame({
type: 'hello',
protocolVersion: CHROME_BRIDGE_PROTOCOL_VERSION,
extensionId: CHROME_EXTENSION_ID,
extensionInstanceId: 'profile-a',
}),
);
await expect(request).resolves.toEqual([
expect.objectContaining({ id: 'chrome' }),
]);
socket.destroy();
});
it.skipIf(process.platform === 'win32')(
'rejects an oversized request before registering its timeout',
async () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'qbu-transport-'));
roots.push(root);
const transport = new ChromeExtensionTransport({
socketPath: path.join(root, 'bridge.sock'),
socketPath: testSocketPath(root),
});
transports.push(transport);
await transport.start();
@@ -556,6 +913,7 @@ describe('ChromeExtensionTransport', () => {
type: 'hello',
protocolVersion: CHROME_BRIDGE_PROTOCOL_VERSION,
extensionId: CHROME_EXTENSION_ID,
extensionInstanceId: 'profile-a',
}),
);
await vi.waitFor(() => expect(transport.isConnected()).toBe(true));
@@ -605,6 +963,7 @@ describe('ChromeExtensionTransport', () => {
type: 'hello',
protocolVersion: CHROME_BRIDGE_PROTOCOL_VERSION,
extensionId: CHROME_EXTENSION_ID,
extensionInstanceId: 'profile-a',
}),
);
const decoder = new FrameDecoder();
@@ -638,7 +997,7 @@ describe('ChromeExtensionTransport', () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'qbu-transport-'));
roots.push(root);
const transport = new ChromeExtensionTransport({
socketPath: path.join(root, 'bridge.sock'),
socketPath: testSocketPath(root),
});
transports.push(transport);
await transport.start();
@@ -655,7 +1014,7 @@ describe('ChromeExtensionTransport', () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'qbu-transport-'));
roots.push(root);
const transport = new ChromeExtensionTransport({
socketPath: path.join(root, 'bridge.sock'),
socketPath: testSocketPath(root),
});
transports.push(transport);
await transport.start();
@@ -672,6 +1031,7 @@ describe('ChromeExtensionTransport', () => {
{
protocolVersion: CHROME_BRIDGE_PROTOCOL_VERSION + 1,
extensionId: CHROME_EXTENSION_ID,
extensionInstanceId: 'profile-a',
},
],
[
@@ -679,6 +1039,7 @@ describe('ChromeExtensionTransport', () => {
{
protocolVersion: CHROME_BRIDGE_PROTOCOL_VERSION,
extensionId: 'wrong-extension-id',
extensionInstanceId: 'profile-a',
},
],
] as const)(
@@ -687,7 +1048,7 @@ describe('ChromeExtensionTransport', () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'qbu-transport-'));
roots.push(root);
const transport = new ChromeExtensionTransport({
socketPath: path.join(root, 'bridge.sock'),
socketPath: testSocketPath(root),
});
transports.push(transport);
await transport.start();
@@ -706,6 +1067,7 @@ describe('ChromeExtensionTransport', () => {
type: 'hello',
protocolVersion: CHROME_BRIDGE_PROTOCOL_VERSION,
extensionId: CHROME_EXTENSION_ID,
extensionInstanceId: 'profile-a',
}),
);
await vi.waitFor(() => expect(transport.isConnected()).toBe(true));
@@ -719,7 +1081,7 @@ describe('ChromeExtensionTransport', () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'qbu-transport-'));
roots.push(root);
const transport = new ChromeExtensionTransport({
socketPath: path.join(root, 'bridge.sock'),
socketPath: testSocketPath(root),
});
transports.push(transport);
await transport.start();
@@ -734,6 +1096,7 @@ describe('ChromeExtensionTransport', () => {
type: 'hello',
protocolVersion: CHROME_BRIDGE_PROTOCOL_VERSION,
extensionId: CHROME_EXTENSION_ID,
extensionInstanceId: 'profile-a',
}),
);
await vi.waitFor(() => expect(states).toEqual([true]));
@@ -748,6 +1111,7 @@ describe('ChromeExtensionTransport', () => {
type: 'hello',
protocolVersion: CHROME_BRIDGE_PROTOCOL_VERSION,
extensionId: CHROME_EXTENSION_ID,
extensionInstanceId: 'profile-a',
}),
);
await vi.waitFor(() => expect(transport.isConnected()).toBe(true));
@@ -838,13 +1202,82 @@ describe('ChromeExtensionTransport', () => {
);
});
function testSocketPath(root: string): string {
return process.platform === 'win32'
? String.raw`\\.\pipe\qbu-test-${randomUUID()}`
: path.join(root, 'bridge.sock');
}
async function startProfileTransport(
options: Omit<ChromeExtensionTransportOptions, 'socketPath'> = {},
): Promise<ChromeExtensionTransport> {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'qbu-profiles-'));
roots.push(root);
const transport = new ChromeExtensionTransport({
socketPath: testSocketPath(root),
...options,
});
transports.push(transport);
await transport.start();
return transport;
}
async function connectProfile(
transport: ChromeExtensionTransport,
extensionInstanceId: string,
onRequest?: (request: { id: string }) => void,
): Promise<Socket> {
const socket = connect(transport.socketPath);
socket.on('error', () => undefined);
const decoder = new FrameDecoder();
socket.on('data', (chunk: Buffer) => {
for (const message of decoder.push(chunk)) {
const request = message as { id: string };
if (onRequest) onRequest(request);
else
socket.write(
encodeFrame({
type: 'response',
id: request.id,
ok: true,
result: extensionInstanceId,
}),
);
}
});
await once(socket, 'connect');
socket.write(
encodeFrame({
type: 'hello',
protocolVersion: CHROME_BRIDGE_PROTOCOL_VERSION,
extensionId: CHROME_EXTENSION_ID,
extensionInstanceId,
}),
);
return socket;
}
async function rejectedHello(
transport: ChromeExtensionTransport,
hello: Record<string, unknown>,
): Promise<void> {
const socket = connect(transport.socketPath);
socket.on('error', () => undefined);
await once(socket, 'connect');
const closed = once(socket, 'close');
socket.write(
encodeFrame({ type: 'hello', extensionId: CHROME_EXTENSION_ID, ...hello }),
);
await closed;
}
async function connectedTransport(
options: Omit<ChromeExtensionTransportOptions, 'socketPath'> = {},
) {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'qbu-transport-'));
roots.push(root);
const transport = new ChromeExtensionTransport({
socketPath: path.join(root, 'bridge.sock'),
socketPath: testSocketPath(root),
...options,
});
transports.push(transport);
@@ -861,6 +1294,7 @@ async function connectedTransport(
type: 'hello',
protocolVersion: CHROME_BRIDGE_PROTOCOL_VERSION,
extensionId: CHROME_EXTENSION_ID,
extensionInstanceId: 'profile-a',
}),
);
await vi.waitFor(() => expect(transport.isConnected()).toBe(true));
packages/browser-use/src/bridge/transport/chrome-extension-transport.ts
+121 -131
View File
@@ -7,14 +7,12 @@
import {
chmod,
lstat,
mkdir,
open,
readFile,
unlink,
type FileHandle,
} from 'node:fs/promises';
import { connect, createServer, type Server, type Socket } from 'node:net';
import { dirname } from 'node:path';
import { randomUUID } from 'node:crypto';
import { clearTimeout, setTimeout } from 'node:timers';
@@ -29,6 +27,7 @@ import {
} from '../protocol.js';
import { BrowserRuntimeError, type RuntimeErrorCode } from '../errors.js';
import { encodeFrame, FrameDecoder } from './framing.js';
import { prepareSocketDirectory } from '../socket-path.js';
export type BridgeEventListener = (event: BridgeEvent) => void;
export type BridgeConnectionListener = (connected: boolean) => void;
@@ -79,13 +78,13 @@ const RECOVERY_LOCK_STALE_MS = 60_000;
export class ChromeExtensionTransport implements ChromeBridge {
readonly socketPath: string;
private readonly derivedSocketDirectory: string | undefined;
private readonly connectTimeoutMs: number;
private readonly requestTimeoutMs: number;
private server: Server | undefined;
private socket: Socket | undefined;
private hello: BridgeHello | undefined;
private selectedExtensionInstanceId: string | undefined;
private incompatibleExtensionError: BrowserRuntimeError | undefined;
private socketIdentity: SocketIdentity | undefined;
private startPromise: Promise<void> | undefined;
private stopPromise: Promise<void> | undefined;
@@ -101,15 +100,7 @@ export class ChromeExtensionTransport implements ChromeBridge {
constructor(options: ChromeExtensionTransportOptions = {}) {
this.socketPath = options.socketPath ?? defaultChromeBridgeSocketPath();
// Only the derived default path gets its private parent directory
// created and verified; a configured path's parent stays the caller's.
this.derivedSocketDirectory =
options.socketPath === undefined &&
!process.env.QWEN_BROWSER_USE_SOCKET_PATH?.trim() &&
process.platform !== 'win32'
? dirname(this.socketPath)
: undefined;
this.connectTimeoutMs = options.connectTimeoutMs ?? 5_000;
this.connectTimeoutMs = options.connectTimeoutMs ?? 35_000;
this.requestTimeoutMs = options.requestTimeoutMs ?? 30_000;
}
@@ -149,13 +140,21 @@ export class ChromeExtensionTransport implements ChromeBridge {
async request(
method: string,
params: Record<string, unknown> = {},
timeoutMs = method === 'cdp.send'
? CDP_REQUEST_TIMEOUT_MS
: this.requestTimeoutMs,
timeoutMs?: number,
): Promise<unknown> {
if (this.stopPromise !== undefined || !this.server?.listening)
throw disconnectedError();
await this.waitForConnection(Math.min(this.connectTimeoutMs, timeoutMs));
// A CDP command carries an operation deadline up to the 120s schema
// ceiling, so its default response budget must outlive that and let the
// caller's own deadline report first. The connection wait stays capped
// only by an explicit caller budget: with none, discovery may legitimately
// take the whole connect timeout.
const budget =
timeoutMs ??
(method === 'cdp.send' ? CDP_REQUEST_TIMEOUT_MS : this.requestTimeoutMs);
await this.waitForConnection(
Math.min(this.connectTimeoutMs, timeoutMs ?? this.connectTimeoutMs),
);
const socket = this.socket;
if (socket === undefined || socket.destroyed) {
throw disconnectedError();
@@ -177,7 +176,7 @@ export class ChromeExtensionTransport implements ChromeBridge {
`Chrome bridge request timed out: ${method}`,
),
);
}, timeoutMs);
}, budget);
this.pending.set(id, {
resolve,
reject,
@@ -224,8 +223,7 @@ export class ChromeExtensionTransport implements ChromeBridge {
const server = createServer((socket) => this.accept(socket));
this.server = server;
try {
if (this.derivedSocketDirectory !== undefined)
await ensureSocketDirectory(this.derivedSocketDirectory);
await prepareSocketDirectory(this.socketPath);
try {
await listen(server, this.socketPath);
} catch (error) {
@@ -257,7 +255,7 @@ export class ChromeExtensionTransport implements ChromeBridge {
(await socketAcceptsConnections(this.socketPath));
const message = addressInUse
? `Chrome bridge socket is already in use: ${this.socketPath}`
: 'Could not start the local Chrome bridge';
: `Could not start the local Chrome bridge: ${errorMessage(error)}`;
const runtimeError = new BrowserRuntimeError(
busy ? 'BROWSER_USE_BUSY' : 'TRANSPORT_UNAVAILABLE',
message,
@@ -276,36 +274,95 @@ export class ChromeExtensionTransport implements ChromeBridge {
socket.destroy(new Error('Chrome bridge hello timed out'));
}, this.connectTimeoutMs);
handshakeTimer.unref();
// Messages are delivered in arrival order, one queue per socket. A
// response settles a promise, so the consumer's continuation runs on the
// microtask queue after this handler returns; an event emitted
// synchronously from the same chunk would overtake it. Playwright installs
// its page listeners inside `Page.getFrameTree().then(...)`, so an
// overtaking `Runtime.executionContextCreated` had no listener yet and the
// claimed tab lost its main world for good. After settling a response the
// drain yields a macrotask, which lets every pending continuation run
// before the next message; chunks arriving meanwhile join the same queue.
const inbound: unknown[] = [];
let draining = false;
const deliver = (message: unknown): 'continue' | 'yield' | 'stop' => {
if (!validated) {
if (!isObject(message) || message.type !== 'hello') return 'continue';
if (
message.extensionId === CHROME_EXTENSION_ID &&
typeof message.protocolVersion === 'number' &&
Number.isInteger(message.protocolVersion) &&
message.protocolVersion > 0 &&
message.protocolVersion !== CHROME_BRIDGE_PROTOCOL_VERSION &&
!this.isConnected() &&
(this.selectedExtensionInstanceId === undefined ||
message.extensionInstanceId === this.selectedExtensionInstanceId)
) {
// Not BROWSER_DISCONNECTED: discovery maps that code to an
// empty browser list, which would hide the update guidance.
this.incompatibleExtensionError = new BrowserRuntimeError(
'EXTENSION_VERSION_MISMATCH',
message.protocolVersion < CHROME_BRIDGE_PROTOCOL_VERSION
? 'The Qwen Code Chrome extension is out of date. Update or reload it at chrome://extensions to match this Qwen Code version, then retry Browser Use.'
: 'This Qwen Code version is older than the Chrome extension. Update Qwen Code to match the installed extension, then retry Browser Use.',
);
}
if (
message.protocolVersion !== CHROME_BRIDGE_PROTOCOL_VERSION ||
message.extensionId !== CHROME_EXTENSION_ID ||
typeof message.extensionInstanceId !== 'string' ||
message.extensionInstanceId.trim() === '' ||
message.extensionInstanceId.length > 128
) {
socket.destroy(
new Error(
'Chrome extension identity or protocol version did not match',
),
);
return 'stop';
}
validated = true;
clearTimeout(handshakeTimer);
this.promote(socket, message as unknown as BridgeHello);
return 'continue';
}
if (this.socket !== socket) return 'stop';
const settlesResponse =
isObject(message) &&
message.type === 'response' &&
typeof message.id === 'string' &&
this.pending.has(message.id);
this.handleMessage(message);
return settlesResponse ? 'yield' : 'continue';
};
const drain = (): void => {
while (inbound.length > 0) {
const outcome = deliver(inbound.shift());
if (outcome === 'stop') {
inbound.length = 0;
break;
}
if (outcome === 'yield' && inbound.length > 0) {
setImmediate(drain);
return;
}
}
draining = false;
};
socket.on('data', (chunk) => {
try {
for (const message of decoder.push(chunk)) {
if (!validated) {
if (!isObject(message) || message.type !== 'hello') continue;
if (
message.protocolVersion !== CHROME_BRIDGE_PROTOCOL_VERSION ||
message.extensionId !== CHROME_EXTENSION_ID
) {
socket.destroy(
new Error(
'Chrome extension identity or protocol version did not match',
),
);
return;
}
validated = true;
clearTimeout(handshakeTimer);
this.promote(socket, message as unknown as BridgeHello);
continue;
}
if (this.socket !== socket) return;
this.handleMessage(message);
}
inbound.push(...decoder.push(chunk));
} catch {
socket.destroy(new Error('Invalid Chrome bridge frame'));
return;
}
if (draining) return;
draining = true;
drain();
});
socket.on('error', () => undefined);
socket.on('close', () => {
inbound.length = 0;
clearTimeout(handshakeTimer);
this.acceptedSockets.delete(socket);
if (this.socket === socket) this.disconnect(disconnectedError());
@@ -313,7 +370,12 @@ export class ChromeExtensionTransport implements ChromeBridge {
}
private promote(socket: Socket, hello: BridgeHello): void {
this.selectedExtensionInstanceId ??= hello.extensionInstanceId;
// Keep other profiles connected but idle so they cannot evict this session
// or enter a disconnect/reconnect loop. Ownership survives a disconnect.
if (hello.extensionInstanceId !== this.selectedExtensionInstanceId) return;
this.disconnect(disconnectedError('Chrome extension reconnected'));
this.incompatibleExtensionError = undefined;
this.socket = socket;
this.hello = hello;
this.notifyConnectionChange(true);
@@ -400,9 +462,10 @@ export class ChromeExtensionTransport implements ChromeBridge {
timer: setTimeout(() => {
this.connectionWaiters.delete(waiter);
reject(
disconnectedError(
'Chrome extension is not connected. Load the extension and verify the Native Messaging host installation.',
),
this.incompatibleExtensionError ??
disconnectedError(
'Chrome extension is not connected. Load the extension and verify the Native Messaging host installation.',
),
);
}, timeoutMs),
};
@@ -451,91 +514,8 @@ export class ChromeExtensionTransport implements ChromeBridge {
if (server !== undefined) await closeServer(server);
await unlinkOwnedSocket(this.socketPath, this.socketIdentity);
this.socketIdentity = undefined;
}
}
// The derived socket directory lives under a world-writable temp root, so
// bind only into a directory this user owns alone: a foreign-owned or
// symlinked entry means a co-tenant is squatting the rendezvous.
export async function ensureSocketDirectory(directory: string): Promise<void> {
const owner =
typeof process.getuid === 'function' ? process.getuid() : undefined;
const info = await lstat(directory).catch(() => undefined);
if (info !== undefined) {
await assertOwnedSocketDirectory(directory, info, owner, true);
await assertUsableAncestors(directory, owner);
return;
}
await mkdir(directory, { recursive: true, mode: 0o700 });
// mkdir(recursive) neither fails on an entry that already exists nor
// tightens its mode, so a co-tenant who won the lstat/mkdir window must
// not silently inherit the socket directory: re-verify what the create
// actually landed on. A strict check (no tightening) keeps the create
// branch fail-closed — a fresh mkdir(0o700) never needs a chmod.
const created = await lstat(directory).catch(() => undefined);
if (created === undefined)
throw new Error(
`Chrome bridge socket directory is not usable: ${directory}`,
);
await assertOwnedSocketDirectory(directory, created, owner, false);
await assertUsableAncestors(directory, owner);
}
async function assertOwnedSocketDirectory(
directory: string,
info: {
isDirectory(): boolean;
isSymbolicLink(): boolean;
uid: number;
mode: number;
},
owner: number | undefined,
tighten: boolean,
): Promise<void> {
if (
!info.isDirectory() ||
info.isSymbolicLink() ||
(owner !== undefined && info.uid !== owner)
)
throw new Error(
`Chrome bridge socket directory is not usable: ${directory}`,
);
if ((info.mode & 0o077) !== 0) {
if (!tighten)
throw new Error(
`Chrome bridge socket directory is not usable: ${directory}`,
);
await chmod(directory, 0o700);
}
}
// Every ancestor up to the sticky world-writable temp root must be owned by
// this user or root and not writable by anyone else; a writable or symlinked
// ancestor lets a co-tenant swap the socket directory out from under us.
async function assertUsableAncestors(
directory: string,
owner: number | undefined,
): Promise<void> {
if (owner === undefined) return;
let current = dirname(directory);
for (;;) {
const info = await lstat(current).catch(() => undefined);
if (info === undefined || !info.isDirectory() || info.isSymbolicLink())
throw new Error(
`Chrome bridge socket directory is not usable: ${directory}`,
);
const mode = info.mode & 0o1777;
// A sticky world-writable root (/tmp, /private/tmp) is the trust
// boundary: every tenant may create entries there, but the sticky bit
// keeps anyone from renaming another tenant's entries.
if ((mode & 0o002) !== 0 && (mode & 0o1000) !== 0) return;
if ((info.uid !== owner && info.uid !== 0) || (mode & 0o022) !== 0)
throw new Error(
`Chrome bridge socket directory is not usable: ${directory}`,
);
const parent = dirname(current);
if (parent === current) return;
current = parent;
this.selectedExtensionInstanceId = undefined;
this.incompatibleExtensionError = undefined;
}
}
@@ -744,6 +724,8 @@ function bridgeRuntimeErrorCode(code: string | undefined): RuntimeErrorCode {
return 'UNSUPPORTED_TAB';
case 'PERMISSION_REQUIRED':
return 'PERMISSION_REQUIRED';
case 'TAB_DEBUGGER_CONFLICT':
return 'TAB_DEBUGGER_CONFLICT';
default:
return 'OPERATION_FAILED';
}
@@ -759,6 +741,14 @@ function isObject(value: unknown): value is Record<string, unknown> {
return typeof value === 'object' && value !== null && !Array.isArray(value);
}
function errorMessage(error: unknown): string {
return isObject(error) &&
typeof error.message === 'string' &&
error.message !== ''
? error.message
: String(error);
}
async function closeServer(server: Server): Promise<void> {
if (!server.listening) return;
await new Promise<void>((resolve) => server.close(() => resolve()));
packages/browser-use/src/core/chrome-runtime-documentation.ts
+2 -2
View File
@@ -16,7 +16,7 @@ This runtime controls the user's existing Chrome through an explicitly installed
- A claim fails closed if the tab title or URL changed after discovery; list open tabs again instead of claiming a different tab.
- Do not guess tab ids. Only claim ids that came from the current openTabs() result.
- tabs.new() creates a new agent tab in the user's Chrome; it shares the profile's cookies and signed-in state.
- browser.user.history({ queries?, from?, to?, limit? }) lists recent browsing history ordered by dateVisited descending.
- browser.user.history({ queries?, from?, to?, limit? }) lists recent browsing history ordered by dateVisited descending. Without from, Chrome searches the last 24 hours; pass from explicitly to include older visits.
- Kernel reset loses JavaScript handles but does not close Chrome or erase its profile. Re-run setup and claim the tab again.
## API
@@ -38,7 +38,7 @@ This runtime controls the user's existing Chrome through an explicitly installed
- tab.dev.logs({ filter?, levels?, limit? }) returns console messages and uncaught exceptions captured since the tab was claimed.
- JavaScript dialogs: opening a dialog does not by itself make an already-dispatched click or keypress fail. Call tab.getJsDialog() to read the dialog type and message. While a dialog is open, page operations fail with DIALOG_OPEN instead of waiting for a timeout. Alerts can be dismissed, confirms and before-unload dialogs can be accepted or dismissed, and prompts require text when accepted. A dialog handle applies only to the dialog that was read; get a new handle if it has closed or been replaced. A dialog that was already open when the tab was claimed is reported as an alert with an empty message; accept or dismiss it once to unblock the tab.
- A claimed tab that is closed, crashed, or whose debugger the user revoked reports STALE_TAB on the next command; claim a tab again to continue.
- Screenshots return JPEG image objects for nodeRepl.emitImage(). Their metadata reports the original image dimensions, viewport, device pixel ratio and CSS-pixel coordinate space. Only a viewport screenshot is directly usable as cua coordinate space; for clip and fullPage captures the metadata origin names the document point of the image's top-left pixel.
- Screenshots return JPEG bytes, mimeType and metadata with the original image dimensions, viewport, device pixel ratio and CSS-pixel coordinate space. Pass the complete object to nodeRepl.emitImage(await tab.screenshot()) so metadata stays attached to the image. Only a viewport screenshot is directly usable as cua coordinate space; for clip and fullPage captures the metadata origin names the document point of the image's top-left pixel.
- Clipboard and raw CDP are not exposed.
- Browser transport, serialization, snapshot truncation and screenshot budgets are runtime details rather than model-facing controls.`;
packages/browser-use/src/core/primitives.ts
+9
View File
@@ -17,6 +17,15 @@ export interface TabInfo {
url: string | null;
}
export const FINALIZE_TAB_STATUSES = ['handoff', 'deliverable'] as const;
export type FinalizeTabStatus = (typeof FINALIZE_TAB_STATUSES)[number];
export interface FinalizeTabDisposition {
tabId: string;
status: FinalizeTabStatus;
}
export interface BrowserUserTabInfo {
id: string;
title: string | null;
packages/browser-use/src/core/schemas.ts
+21 -1
View File
@@ -6,7 +6,11 @@
import { z } from 'zod';
import { SNAPSHOT_REF_PATTERN, type LocatorStep } from './primitives.js';
import {
FINALIZE_TAB_STATUSES,
SNAPSHOT_REF_PATTERN,
type LocatorStep,
} from './primitives.js';
import {
MAX_SCREENSHOT_EDGE,
MAX_SCREENSHOT_PIXELS,
@@ -265,6 +269,22 @@ export const commandSchemas = {
'tabs.list': browserArgs,
'tabs.get': z.object({ browserId: id, tabId: id }).strict(),
'tabs.selected': browserArgs,
'tabs.finalize': z
.object({
browserId: id,
keep: z
.array(
z
.object({
tabId: id,
status: z.enum(FINALIZE_TAB_STATUSES),
})
.strict(),
)
.max(100)
.optional(),
})
.strict(),
'tab.goto': z
.object({
packages/browser-use/src/native-host-installer.test.ts
+674
View File
@@ -0,0 +1,674 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { execFileSync } from 'node:child_process';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterEach, describe, expect, it } from 'vitest';
import {
installChromeNativeHost,
isChromeExtensionInstalled,
nativeHostInstallHome,
statusChromeNativeHost,
uninstallChromeNativeHost,
} from './native-host-installer.js';
import {
CHROME_EXTENSION_ID,
CHROME_NATIVE_HOST_NAME,
} from './bridge/protocol.js';
const roots: string[] = [];
afterEach(() => {
for (const root of roots.splice(0)) {
fs.rmSync(root, { recursive: true, force: true });
}
});
describe('Chrome extension installation detection', () => {
it.each(['darwin', 'linux'] as const)(
'finds packaged extensions in a secondary profile on %s',
async (platform) => {
const fixture = createFixture();
const browserRoot = createBrowserProfile(
fixture.homeDir,
platform,
'chrome',
);
const profile = path.join(browserRoot, 'Profile 2');
const extensionPath = path.join(CHROME_EXTENSION_ID, '1.0.0_0');
writeExtensionPreferences(profile, 'Secure Preferences', extensionPath);
const installedPath = path.join(profile, 'Extensions', extensionPath);
fs.mkdirSync(installedPath, { recursive: true });
fs.writeFileSync(path.join(installedPath, 'manifest.json'), '{}');
await expect(
isChromeExtensionInstalled({ ...fixture, platform }),
).resolves.toBe(true);
expect(fs.existsSync(path.join(fixture.homeDir, '.qwen'))).toBe(false);
expect(
fs.existsSync(path.join(browserRoot, 'NativeMessagingHosts')),
).toBe(false);
},
);
it.each(['chrome', 'chrome-for-testing', 'chromium'] as const)(
'finds an unpacked extension through %s preferences',
async (browser) => {
const fixture = createFixture();
const browserRoot = createBrowserProfile(
fixture.homeDir,
'darwin',
browser,
);
const extensionPath = path.join(fixture.homeDir, 'unpacked extension');
fs.mkdirSync(extensionPath, { recursive: true });
fs.writeFileSync(path.join(extensionPath, 'manifest.json'), '{}');
writeExtensionPreferences(
path.join(browserRoot, 'Default'),
'Preferences',
extensionPath,
);
await expect(
isChromeExtensionInstalled({ ...fixture, platform: 'darwin' }),
).resolves.toBe(true);
},
);
it.each([
'no profile',
'no extension',
'other extension',
'leftover files',
'missing files',
'invalid preferences',
])('does not confirm installation with %s', async (scenario) => {
const fixture = createFixture();
if (scenario !== 'no profile') {
const browserRoot = createBrowserProfile(
fixture.homeDir,
'darwin',
'chrome',
);
const profile = path.join(browserRoot, 'Default');
const extensionPath = path.join(fixture.homeDir, 'unpacked extension');
fs.mkdirSync(extensionPath, { recursive: true });
if (scenario !== 'missing files') {
fs.writeFileSync(path.join(extensionPath, 'manifest.json'), '{}');
}
writeExtensionPreferences(profile, 'Preferences', extensionPath);
if (scenario === 'no extension' || scenario === 'leftover files') {
fs.writeFileSync(path.join(profile, 'Preferences'), '{}');
}
if (scenario === 'leftover files') {
const leftover = path.join(
profile,
'Extensions',
CHROME_EXTENSION_ID,
'1.0.0_0',
);
fs.mkdirSync(leftover, { recursive: true });
fs.writeFileSync(path.join(leftover, 'manifest.json'), '{}');
}
if (scenario === 'other extension') {
fs.writeFileSync(
path.join(profile, 'Preferences'),
JSON.stringify({
extensions: { settings: { other: { path: extensionPath } } },
}),
);
}
if (scenario === 'invalid preferences') {
fs.writeFileSync(path.join(profile, 'Preferences'), '{');
}
}
await expect(
isChromeExtensionInstalled({ ...fixture, platform: 'darwin' }),
).resolves.toBe(false);
});
it('uses Secure Preferences ahead of an older Preferences entry', async () => {
const fixture = createFixture();
const profile = path.join(
createBrowserProfile(fixture.homeDir, 'darwin', 'chrome'),
'Default',
);
const extensionPath = path.join(fixture.homeDir, 'old extension');
fs.mkdirSync(extensionPath, { recursive: true });
fs.writeFileSync(path.join(extensionPath, 'manifest.json'), '{}');
writeExtensionPreferences(profile, 'Preferences', extensionPath);
writeExtensionPreferences(
profile,
'Secure Preferences',
'/missing/extension',
);
await expect(
isChromeExtensionInstalled({ ...fixture, platform: 'darwin' }),
).resolves.toBe(false);
});
});
function writeExtensionPreferences(
profile: string,
file: string,
extensionPath: string,
): void {
fs.mkdirSync(profile, { recursive: true });
fs.writeFileSync(
path.join(profile, file),
JSON.stringify({
extensions: {
settings: { [CHROME_EXTENSION_ID]: { path: extensionPath } },
},
}),
);
}
describe('Chrome Native Host installer', () => {
it('installs the manifest for an existing macOS Chrome profile', async () => {
const fixture = createFixture();
createBrowserProfile(fixture.homeDir, 'darwin', 'chrome');
const result = await installChromeNativeHost({
...fixture,
platform: 'darwin',
});
expect(result.manifestPaths).toHaveLength(3);
expect(result.manifestPaths).toContain(
path.join(
fixture.homeDir,
'Library/Application Support/Google/Chrome/NativeMessagingHosts/com.qwen.browser.json',
),
);
const manifest = JSON.parse(
fs.readFileSync(result.manifestPaths[0]!, 'utf8'),
) as Record<string, unknown>;
expect(manifest).toEqual({
name: CHROME_NATIVE_HOST_NAME,
description: 'Qwen Browser Use',
path: result.launcherPath,
type: 'stdio',
allowed_origins: ['chrome-extension://' + CHROME_EXTENSION_ID + '/'],
});
if (process.platform !== 'win32') {
expect(fs.statSync(result.launcherPath).mode & 0o777).toBe(0o700);
}
const launcher = fs.readFileSync(result.launcherPath, 'utf8');
expect(launcher.startsWith('#!/bin/sh\n')).toBe(true);
expect(launcher).toContain("'" + fixture.nativeHostPath + "'");
});
it('registers the documented macOS Chrome for Testing profile', async () => {
const fixture = createFixture();
createBrowserProfile(fixture.homeDir, 'darwin', 'chrome-for-testing');
const options = { ...fixture, platform: 'darwin' as const };
await installChromeNativeHost(options);
expect((await statusChromeNativeHost(options)).installedPaths).toContain(
path.join(
fixture.homeDir,
'Library/Application Support/Google/Chrome for Testing/NativeMessagingHosts/com.qwen.browser.json',
),
);
});
it
.skipIf(process.platform === 'win32' || process.getuid?.() === 0)
.each(['launcher', 'manifest'])(
'preserves an unreadable %s during install, status and uninstall',
async (kind) => {
const fixture = createFixture();
createBrowserProfile(fixture.homeDir, 'darwin', 'chrome');
const options = { ...fixture, platform: 'darwin' as const };
const installed = await installChromeNativeHost(options);
const target =
kind === 'launcher'
? installed.launcherPath
: installed.manifestPaths[0]!;
fs.writeFileSync(target, 'foreign file');
fs.chmodSync(target, 0);
try {
for (const operation of [
installChromeNativeHost,
statusChromeNativeHost,
uninstallChromeNativeHost,
]) {
await expect(operation(options)).rejects.toMatchObject({
code: 'EACCES',
});
expect(fs.statSync(target).mode & 0o777).toBe(0);
}
} finally {
fs.chmodSync(target, 0o600);
}
expect(fs.readFileSync(target, 'utf8')).toBe('foreign file');
},
);
it('uses the documented Linux user paths and updates idempotently', async () => {
const fixture = createFixture();
createBrowserProfile(fixture.homeDir, 'linux', 'chrome');
createBrowserProfile(fixture.homeDir, 'linux', 'chromium');
const first = await installChromeNativeHost({
...fixture,
platform: 'linux',
});
const files = [
first.launcherPath,
...first.manifestPaths.filter((file) => fs.existsSync(file)),
];
const before = files.map((file) => fs.statSync(file));
const second = await installChromeNativeHost({
...fixture,
platform: 'linux',
});
expect(second).toEqual(first);
expect(files.map((file) => fs.statSync(file).ino)).toEqual(
before.map((file) => file.ino),
);
expect(files.map((file) => fs.statSync(file).mtimeMs)).toEqual(
before.map((file) => file.mtimeMs),
);
expect(second.manifestPaths).toEqual(
expect.arrayContaining([
path.join(
fixture.homeDir,
'.config/google-chrome/NativeMessagingHosts/com.qwen.browser.json',
),
path.join(
fixture.homeDir,
'.config/google-chrome-for-testing/NativeMessagingHosts/com.qwen.browser.json',
),
path.join(
fixture.homeDir,
'.config/chromium/NativeMessagingHosts/com.qwen.browser.json',
),
]),
);
expect(second.installedPaths).toEqual([
second.manifestPaths[0],
second.manifestPaths[2],
second.launcherPath,
]);
expect(fs.existsSync(second.manifestPaths[1]!)).toBe(false);
const status = await statusChromeNativeHost({
...fixture,
platform: 'linux',
});
expect(status.installedPaths).toEqual(second.installedPaths);
});
it
.skipIf(process.platform === 'win32' || process.getuid?.() === 0)
.each(['launcher', 'manifest'] as const)(
'keeps a correct %s usable in a read-only directory',
async (kind) => {
const fixture = createFixture();
createBrowserProfile(fixture.homeDir, 'darwin', 'chrome');
const options = { ...fixture, platform: 'darwin' as const };
const installed = await installChromeNativeHost(options);
const target =
kind === 'launcher'
? installed.launcherPath
: installed.manifestPaths[0]!;
const before = fs.statSync(target);
const directory = path.dirname(target);
fs.chmodSync(directory, 0o500);
try {
await expect(installChromeNativeHost(options)).resolves.toEqual(
installed,
);
expect(fs.statSync(target)).toMatchObject({
ino: before.ino,
mtimeMs: before.mtimeMs,
mode: before.mode,
});
} finally {
fs.chmodSync(directory, 0o700);
}
},
);
it.skipIf(process.platform === 'win32')(
'repairs permissions on unchanged owned files',
async () => {
const fixture = createFixture();
createBrowserProfile(fixture.homeDir, 'darwin', 'chrome');
const options = { ...fixture, platform: 'darwin' as const };
const installed = await installChromeNativeHost(options);
const manifest = installed.manifestPaths[0]!;
fs.chmodSync(installed.launcherPath, 0o600);
fs.chmodSync(manifest, 0o644);
await installChromeNativeHost(options);
expect(fs.statSync(installed.launcherPath).mode & 0o777).toBe(0o700);
expect(fs.statSync(manifest).mode & 0o777).toBe(0o600);
},
);
it.skipIf(process.platform === 'win32' || process.getuid?.() === 0)(
'propagates a required launcher update failure and updates when writable',
async () => {
const fixture = createFixture();
createBrowserProfile(fixture.homeDir, 'darwin', 'chrome');
const options = { ...fixture, platform: 'darwin' as const };
const installed = await installChromeNativeHost(options);
const before = fs.readFileSync(installed.launcherPath, 'utf8');
const updated = { ...options, nodePath: '/updated/node' };
const directory = path.dirname(installed.launcherPath);
fs.chmodSync(directory, 0o500);
try {
await expect(installChromeNativeHost(updated)).rejects.toMatchObject({
code: 'EACCES',
});
expect(fs.readFileSync(installed.launcherPath, 'utf8')).toBe(before);
} finally {
fs.chmodSync(directory, 0o700);
}
await installChromeNativeHost(updated);
expect(fs.readFileSync(installed.launcherPath, 'utf8')).toContain(
"exec '/updated/node'",
);
},
);
it('uninstalls owned files without deleting a foreign manifest', async () => {
const fixture = createFixture();
createBrowserProfile(fixture.homeDir, 'darwin', 'chrome');
const installed = await installChromeNativeHost({
...fixture,
platform: 'darwin',
});
const foreign = installed.manifestPaths[0]!;
fs.writeFileSync(foreign, JSON.stringify({ name: 'foreign.host' }));
const result = await uninstallChromeNativeHost({
...fixture,
platform: 'darwin',
});
expect(result.skippedForeignPaths).toEqual([foreign]);
expect(fs.existsSync(foreign)).toBe(true);
expect(fs.existsSync(result.launcherPath)).toBe(false);
for (const manifestPath of installed.manifestPaths.slice(1)) {
expect(fs.existsSync(manifestPath)).toBe(false);
}
});
it('does not overwrite a foreign manifest during install', async () => {
const fixture = createFixture();
const manifestPath = path.join(
fixture.homeDir,
'Library/Application Support/Google/Chrome/NativeMessagingHosts/com.qwen.browser.json',
);
fs.mkdirSync(path.dirname(manifestPath), { recursive: true });
fs.writeFileSync(manifestPath, JSON.stringify({ name: 'foreign.host' }));
const result = await installChromeNativeHost({
...fixture,
platform: 'darwin',
});
expect(result.skippedForeignPaths).toEqual([manifestPath]);
expect(fs.readFileSync(manifestPath, 'utf8')).toContain('foreign.host');
});
it('does not claim a matching host manifest for another launcher', async () => {
const fixture = createFixture();
const manifestPath = path.join(
fixture.homeDir,
'Library/Application Support/Google/Chrome/NativeMessagingHosts/com.qwen.browser.json',
);
fs.mkdirSync(path.dirname(manifestPath), { recursive: true });
fs.writeFileSync(
manifestPath,
JSON.stringify({
name: CHROME_NATIVE_HOST_NAME,
path: path.join(fixture.homeDir, 'other-launcher.sh'),
type: 'stdio',
allowed_origins: ['chrome-extension://' + CHROME_EXTENSION_ID + '/'],
}),
);
const result = await installChromeNativeHost({
...fixture,
platform: 'darwin',
});
expect(result.skippedForeignPaths).toContain(manifestPath);
});
it('requires absolute executable paths', async () => {
const fixture = createFixture();
await expect(
installChromeNativeHost({
...fixture,
nativeHostPath: 'native-host.js',
platform: 'linux',
}),
).rejects.toThrow('must be absolute');
});
it('requires an absolute Node path', async () => {
const fixture = createFixture();
await expect(
installChromeNativeHost({
...fixture,
nodePath: 'node',
platform: 'linux',
}),
).rejects.toThrow('must be absolute');
expect(fs.existsSync(path.join(fixture.homeDir, '.qwen'))).toBe(false);
});
it('rejects a missing Native Host before writing anything', async () => {
const fixture = createFixture();
createBrowserProfile(fixture.homeDir, 'linux', 'chrome');
await expect(
installChromeNativeHost({
...fixture,
nativeHostPath: path.join(fixture.homeDir, 'missing/native-host.js'),
platform: 'linux',
}),
).rejects.toMatchObject({ code: 'ENOENT' });
expect(fs.existsSync(path.join(fixture.homeDir, '.qwen'))).toBe(false);
expect(
fs.existsSync(
path.join(
fixture.homeDir,
'.config/google-chrome/NativeMessagingHosts',
),
),
).toBe(false);
});
it('refuses unsupported platforms without writing anything', async () => {
const fixture = createFixture();
await expect(
installChromeNativeHost({ ...fixture, platform: 'win32' }),
).rejects.toThrow(
'Automatic Native Messaging installation supports macOS and Linux',
);
expect(fs.existsSync(fixture.homeDir)).toBe(false);
});
it('refuses to replace a foreign launcher and skips it afterwards', async () => {
const fixture = createFixture();
createBrowserProfile(fixture.homeDir, 'darwin', 'chrome');
const options = { ...fixture, platform: 'darwin' as const };
const launcherPath = path.join(
fixture.homeDir,
'.qwen/browser-use/native-host.sh',
);
// A different program's launcher: a shell script without the owned marker.
const foreign = '#!/bin/sh\nexec /other/tool "$@"\n';
fs.mkdirSync(path.dirname(launcherPath), { recursive: true });
fs.writeFileSync(launcherPath, foreign, { mode: 0o755 });
const manifestDirectory = path.join(
fixture.homeDir,
'Library/Application Support/Google/Chrome/NativeMessagingHosts',
);
await expect(installChromeNativeHost(options)).rejects.toThrow(
'Refusing to replace a foreign Native Host launcher: ' + launcherPath,
);
expect(fs.readFileSync(launcherPath, 'utf8')).toBe(foreign);
expect(fs.existsSync(manifestDirectory)).toBe(false);
const status = await statusChromeNativeHost(options);
expect(status.installedPaths).toEqual([]);
expect(status.skippedForeignPaths).toEqual([launcherPath]);
const uninstalled = await uninstallChromeNativeHost(options);
expect(uninstalled.skippedForeignPaths).toEqual([launcherPath]);
expect(fs.readFileSync(launcherPath, 'utf8')).toBe(foreign);
});
it('removes every owned manifest on uninstall', async () => {
const fixture = createFixture();
createBrowserProfile(fixture.homeDir, 'linux', 'chrome');
createBrowserProfile(fixture.homeDir, 'linux', 'chromium');
const options = { ...fixture, platform: 'linux' as const };
const installed = await installChromeNativeHost(options);
expect(installed.installedPaths).toHaveLength(3);
for (const file of installed.installedPaths) {
expect(fs.existsSync(file)).toBe(true);
}
const result = await uninstallChromeNativeHost(options);
expect(result.skippedForeignPaths).toEqual([]);
for (const file of installed.installedPaths) {
expect(fs.existsSync(file)).toBe(false);
}
expect((await statusChromeNativeHost(options)).installedPaths).toEqual([]);
});
it('keeps a same-name manifest for another launcher on uninstall', async () => {
const fixture = createFixture();
createBrowserProfile(fixture.homeDir, 'darwin', 'chrome');
const options = { ...fixture, platform: 'darwin' as const };
const installed = await installChromeNativeHost(options);
const manifestPath = installed.manifestPaths[0]!;
const other = JSON.stringify({
name: CHROME_NATIVE_HOST_NAME,
description: 'Qwen Browser Use',
path: path.join(fixture.homeDir, 'other-launcher.sh'),
type: 'stdio',
allowed_origins: ['chrome-extension://' + CHROME_EXTENSION_ID + '/'],
});
fs.writeFileSync(manifestPath, other);
const result = await uninstallChromeNativeHost(options);
expect(result.skippedForeignPaths).toEqual([manifestPath]);
expect(fs.readFileSync(manifestPath, 'utf8')).toBe(other);
expect(fs.existsSync(installed.launcherPath)).toBe(false);
});
it.skipIf(process.platform === 'win32')(
'quotes the interpreter and an apostrophe-bearing host path in the launcher',
async () => {
const fixture = createFixture();
createBrowserProfile(fixture.homeDir, 'linux', 'chrome');
const nativeHostPath = path.join(
path.dirname(fixture.nativeHostPath),
"it's here/native-host.js",
);
fs.mkdirSync(path.dirname(nativeHostPath), { recursive: true });
fs.writeFileSync(nativeHostPath, '# host');
// A stand-in interpreter that echoes its operands one per line.
fs.writeFileSync(fixture.nodePath, '#!/bin/sh\nprintf "%s\\n" "$@"\n', {
mode: 0o755,
});
const installed = await installChromeNativeHost({
...fixture,
nativeHostPath,
platform: 'linux',
});
const launcher = fs.readFileSync(installed.launcherPath, 'utf8');
expect(launcher).toContain(
"exec '" +
fixture.nodePath +
"' '" +
nativeHostPath.replace("'", "'\\''") +
'\' "$@"',
);
const output = execFileSync(installed.launcherPath, ['--stdio'], {
encoding: 'utf8',
});
expect(output.split('\n')).toEqual([nativeHostPath, '--stdio', '']);
},
);
});
describe('nativeHostInstallHome', () => {
it('resolves a configured install home', () => {
expect(
nativeHostInstallHome({ QWEN_BROWSER_USE_INSTALL_HOME: '/custom/home' }),
).toBe(path.resolve('/custom/home'));
});
it.each(['', ' '])(
'treats a blank QWEN_BROWSER_USE_INSTALL_HOME (%j) as unset',
(value) => {
expect(
nativeHostInstallHome({ QWEN_BROWSER_USE_INSTALL_HOME: value }),
).toBe(path.resolve(os.homedir()));
},
);
it('falls back to the home directory when unset', () => {
expect(nativeHostInstallHome({})).toBe(path.resolve(os.homedir()));
});
});
function createFixture(): {
homeDir: string;
nativeHostPath: string;
nodePath: string;
} {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'qbu-install-'));
roots.push(root);
const homeDir = path.join(root, 'home');
const nativeHostPath = path.join(
root,
'extension with spaces/native-host.js',
);
fs.mkdirSync(path.dirname(nativeHostPath), { recursive: true });
fs.writeFileSync(nativeHostPath, '# host');
return {
homeDir,
nativeHostPath,
nodePath: path.join(root, 'node with spaces'),
};
}
function createBrowserProfile(
homeDir: string,
platform: 'darwin' | 'linux',
browser: 'chrome' | 'chrome-for-testing' | 'chromium',
): string {
const roots =
platform === 'darwin'
? {
chrome: 'Library/Application Support/Google/Chrome',
'chrome-for-testing':
'Library/Application Support/Google/Chrome for Testing',
chromium: 'Library/Application Support/Chromium',
}
: {
chrome: '.config/google-chrome',
'chrome-for-testing': '.config/google-chrome-for-testing',
chromium: '.config/chromium',
};
const root = path.join(homeDir, roots[browser]);
fs.mkdirSync(root, { recursive: true });
return root;
}
packages/browser-use/src/native-host-installer.ts
+341
View File
@@ -0,0 +1,341 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { randomUUID } from 'node:crypto';
import type { Dirent } from 'node:fs';
import {
access,
chmod,
mkdir,
readFile,
readdir,
rename,
rm,
stat,
writeFile,
} from 'node:fs/promises';
import { homedir } from 'node:os';
import { dirname, isAbsolute, join, resolve } from 'node:path';
import {
CHROME_EXTENSION_ID,
CHROME_NATIVE_HOST_NAME,
} from './bridge/protocol.js';
const LAUNCHER_MARKER = '# qwen-browser-use native host';
const MANIFEST_FILE = CHROME_NATIVE_HOST_NAME + '.json';
const ALLOWED_ORIGIN = 'chrome-extension://' + CHROME_EXTENSION_ID + '/';
export interface NativeHostInstallOptions {
nativeHostPath: string;
homeDir?: string;
nodePath?: string;
platform?: NodeJS.Platform;
}
export interface NativeHostInstallResult {
launcherPath: string;
manifestPaths: string[];
skippedForeignPaths: string[];
}
export async function isChromeExtensionInstalled(
options: NativeHostInstallOptions,
): Promise<boolean> {
for (const manifestPath of resolveOptions(options).manifestPaths) {
const browserRoot = dirname(dirname(manifestPath));
let profiles: Dirent[];
try {
profiles = await readdir(browserRoot, { withFileTypes: true });
} catch (error) {
if ((error as NodeJS.ErrnoException).code === 'ENOENT') continue;
throw error;
}
for (const profile of profiles) {
if (
!profile.isDirectory() ||
!/^(Default|Profile \d+)$/.test(profile.name)
) {
continue;
}
const profilePath = join(browserRoot, profile.name);
for (const file of ['Secure Preferences', 'Preferences']) {
const contents = await readExistingFile(join(profilePath, file));
if (contents === null) continue;
let preferences: {
extensions?: {
settings?: Record<string, { path?: unknown }>;
};
} | null;
try {
preferences = JSON.parse(contents) as typeof preferences;
} catch {
continue;
}
const extension =
preferences?.extensions?.settings?.[CHROME_EXTENSION_ID];
if (extension === undefined) continue;
if (
typeof extension?.path === 'string' &&
extension.path.length > 0 &&
(await pathExists(
join(
resolve(profilePath, 'Extensions', extension.path),
'manifest.json',
),
))
) {
return true;
}
break;
}
}
}
return false;
}
export async function installChromeNativeHost(
options: NativeHostInstallOptions,
): Promise<NativeHostInstallResult & { installedPaths: string[] }> {
const resolved = resolveOptions(options);
await access(resolved.nativeHostPath);
const existingLauncher = await readExistingFile(resolved.launcherPath);
if (existingLauncher !== null && !isOwnedLauncher(existingLauncher)) {
throw new Error(
'Refusing to replace a foreign Native Host launcher: ' +
resolved.launcherPath,
);
}
await mkdir(dirname(resolved.launcherPath), {
recursive: true,
mode: 0o700,
});
await atomicWrite(
resolved.launcherPath,
[
'#!/bin/sh',
LAUNCHER_MARKER,
'exec ' +
shellQuote(resolved.nodePath) +
' ' +
shellQuote(resolved.nativeHostPath) +
' "$@"',
'',
].join('\n'),
0o700,
existingLauncher,
);
const manifest =
JSON.stringify(
{
name: CHROME_NATIVE_HOST_NAME,
description: 'Qwen Browser Use',
path: resolved.launcherPath,
type: 'stdio',
allowed_origins: [ALLOWED_ORIGIN],
},
null,
2,
) + '\n';
const installedPaths: string[] = [];
const skippedForeignPaths: string[] = [];
for (const manifestPath of resolved.manifestPaths) {
const existing = await readExistingFile(manifestPath);
if (
existing === null &&
!(await pathExists(dirname(dirname(manifestPath))))
) {
continue;
}
if (
existing !== null &&
!isOwnedManifest(existing, resolved.launcherPath)
) {
skippedForeignPaths.push(manifestPath);
continue;
}
await mkdir(dirname(manifestPath), { recursive: true });
await atomicWrite(manifestPath, manifest, 0o600, existing);
installedPaths.push(manifestPath);
}
// Same order as statusChromeNativeHost: manifests first, then the launcher.
installedPaths.push(resolved.launcherPath);
return {
launcherPath: resolved.launcherPath,
manifestPaths: resolved.manifestPaths,
installedPaths,
skippedForeignPaths,
};
}
export async function uninstallChromeNativeHost(
options: NativeHostInstallOptions,
): Promise<NativeHostInstallResult> {
const resolved = resolveOptions(options);
const skippedForeignPaths: string[] = [];
for (const manifestPath of resolved.manifestPaths) {
const contents = await readExistingFile(manifestPath);
if (contents === null) continue;
if (isOwnedManifest(contents, resolved.launcherPath)) {
await rm(manifestPath, { force: true });
} else {
skippedForeignPaths.push(manifestPath);
}
}
const launcher = await readExistingFile(resolved.launcherPath);
if (launcher !== null) {
if (isOwnedLauncher(launcher)) {
await rm(resolved.launcherPath, { force: true });
} else {
skippedForeignPaths.push(resolved.launcherPath);
}
}
return {
launcherPath: resolved.launcherPath,
manifestPaths: resolved.manifestPaths,
skippedForeignPaths,
};
}
export async function statusChromeNativeHost(
options: NativeHostInstallOptions,
): Promise<NativeHostInstallResult & { installedPaths: string[] }> {
const resolved = resolveOptions(options);
const installedPaths: string[] = [];
const skippedForeignPaths: string[] = [];
for (const manifestPath of resolved.manifestPaths) {
const contents = await readExistingFile(manifestPath);
if (contents === null) continue;
if (isOwnedManifest(contents, resolved.launcherPath)) {
installedPaths.push(manifestPath);
} else {
skippedForeignPaths.push(manifestPath);
}
}
const launcher = await readExistingFile(resolved.launcherPath);
if (launcher !== null && isOwnedLauncher(launcher)) {
installedPaths.push(resolved.launcherPath);
} else if (launcher !== null) {
skippedForeignPaths.push(resolved.launcherPath);
}
return {
launcherPath: resolved.launcherPath,
manifestPaths: resolved.manifestPaths,
installedPaths,
skippedForeignPaths,
};
}
export function nativeHostInstallHome(
environment: NodeJS.ProcessEnv = process.env,
): string {
const configured = environment['QWEN_BROWSER_USE_INSTALL_HOME']?.trim();
return resolve(configured ? configured : homedir());
}
function resolveOptions(options: NativeHostInstallOptions): {
nativeHostPath: string;
nodePath: string;
launcherPath: string;
manifestPaths: string[];
} {
const platform = options.platform ?? process.platform;
if (platform !== 'darwin' && platform !== 'linux') {
throw new Error(
'Automatic Native Messaging installation supports macOS and Linux',
);
}
const homeDir = resolve(options.homeDir ?? nativeHostInstallHome());
const nativeHostPath = options.nativeHostPath;
const nodePath = options.nodePath ?? process.execPath;
if (!isAbsolute(nativeHostPath) || !isAbsolute(nodePath)) {
throw new Error('Native Host and Node paths must be absolute');
}
const manifestRoots =
platform === 'darwin'
? [
'Library/Application Support/Google/Chrome/NativeMessagingHosts',
'Library/Application Support/Google/Chrome for Testing/NativeMessagingHosts',
'Library/Application Support/Chromium/NativeMessagingHosts',
]
: [
'.config/google-chrome/NativeMessagingHosts',
'.config/google-chrome-for-testing/NativeMessagingHosts',
'.config/chromium/NativeMessagingHosts',
];
return {
nativeHostPath,
nodePath,
launcherPath: join(homeDir, '.qwen/browser-use/native-host.sh'),
manifestPaths: manifestRoots.map((root) =>
join(homeDir, root, MANIFEST_FILE),
),
};
}
function isOwnedManifest(contents: string, launcherPath: string): boolean {
try {
const value = JSON.parse(contents) as Record<string, unknown>;
return (
value['name'] === CHROME_NATIVE_HOST_NAME &&
value['type'] === 'stdio' &&
value['path'] === launcherPath &&
Array.isArray(value['allowed_origins']) &&
value['allowed_origins'].length === 1 &&
value['allowed_origins'][0] === ALLOWED_ORIGIN
);
} catch {
return false;
}
}
function isOwnedLauncher(contents: string): boolean {
return contents.startsWith('#!/bin/sh\n' + LAUNCHER_MARKER + '\n');
}
async function readExistingFile(path: string): Promise<string | null> {
try {
return await readFile(path, 'utf8');
} catch (error) {
if ((error as NodeJS.ErrnoException).code === 'ENOENT') return null;
throw error;
}
}
async function atomicWrite(
target: string,
contents: string,
mode: number,
existingContents: string | null,
): Promise<void> {
if (existingContents === contents) {
if (((await stat(target)).mode & 0o777) !== mode) {
await chmod(target, mode);
}
return;
}
const temporary = target + '.' + randomUUID() + '.tmp';
try {
await writeFile(temporary, contents, { mode });
await chmod(temporary, mode);
await rename(temporary, target);
} finally {
await rm(temporary, { force: true });
}
}
async function pathExists(path: string): Promise<boolean> {
return await access(path).then(
() => true,
() => false,
);
}
function shellQuote(value: string): string {
return "'" + value.replaceAll("'", "'\\''") + "'";
}
packages/browser-use/src/playwright/browser-model.ts
-10
View File
@@ -315,16 +315,6 @@ export class BrowserModel {
this.assertOpen();
this.assertOwned(tabId);
const info = targetInfo(response.targetInfo);
// Playwright never enables CDP download events here (noDefaults), so the
// bridge must: without this, Page.downloadWillBegin never fires and
// waitForEvent('download') cannot resolve.
await this.bridge.request('cdp.send', {
tabId,
method: 'Page.setDownloadBehavior',
params: { behavior: 'default', eventsEnabled: true },
});
this.assertOpen();
this.assertOwned(tabId);
const session: TabSession = {
tabId,
sessionId: `pw-tab-${this.nextSessionId++}`,
packages/browser-use/src/playwright/playwright-runtime.test.ts
+613 -17
View File
@@ -1011,8 +1011,8 @@ describe('PlaywrightRuntime command contracts', () => {
await expect(
fixture.runtime.dispatch('tab.title', { tabId: tab.id }),
).resolves.toBe('Fixture');
// The unrelated popup and the matching page were each probed once.
expect(fixture.probeDetach).toHaveBeenCalledTimes(2);
// All three other pages and the matching page were each probed once.
expect(fixture.probeDetach).toHaveBeenCalledTimes(4);
expect(fixture.cdp.detach).not.toHaveBeenCalled();
});
@@ -1555,7 +1555,7 @@ describe('PlaywrightRuntime command contracts', () => {
).resolves.toMatchObject({ message: 'Second' });
fixture.emitEvent({
type: 'event',
tabId: 18,
tabId: 22,
method: 'Page.javascriptDialogClosed',
params: {},
});
@@ -2010,13 +2010,13 @@ describe('PlaywrightRuntime command contracts', () => {
if (method === 'tabs.queryDerived')
return [
{
providerTabId: 18,
providerTabId: 22,
title: 'Popup',
url: 'https://example.com/popup',
derivedFromProviderTabId: 17,
},
];
if (method === 'tabs.attach' && params.tabId === 18)
if (method === 'tabs.attach' && params.tabId === 22)
throw new Error('Cannot attach to target');
return null;
},
@@ -2027,6 +2027,50 @@ describe('PlaywrightRuntime command contracts', () => {
).resolves.toEqual([expect.objectContaining({ id: tab.id })]);
});
it('lists healthy derived tabs after an earlier popup fails to attach', async () => {
const fixture = await runtimeFixture();
const tab = await createTab(fixture.runtime);
fixture.derivedTabs.push({ providerTabId: 22 }, { providerTabId: 23 });
const request = fixture.request.getMockImplementation()!;
fixture.request.mockImplementation(async (method, params = {}) => {
if (method === 'tabs.attach' && params.tabId === 22)
throw new Error('Cannot attach to target');
return await request(method, params);
});
await expect(
fixture.runtime.dispatch('tabs.list', { browserId: 'chrome' }),
).resolves.toEqual([
expect.objectContaining({ id: tab.id }),
expect.objectContaining({ title: 'Nested popup' }),
]);
expect(fixture.request).toHaveBeenCalledWith('tabs.attach', { tabId: 23 });
});
it.each(['BROWSER_DISCONNECTED', 'STALE_BROWSER_SESSION'] as const)(
'propagates %s after an earlier derived-tab attachment failure',
async (code) => {
const fixture = await runtimeFixture();
await createTab(fixture.runtime);
fixture.derivedTabs.push({ providerTabId: 22 }, { providerTabId: 23 });
const request = fixture.request.getMockImplementation()!;
fixture.request.mockImplementation(async (method, params = {}) => {
if (method === 'tabs.attach' && params.tabId === 22)
throw new Error('Cannot attach to target');
if (method === 'tabs.attach' && params.tabId === 23)
throw new BrowserRuntimeError(code, 'Chrome session lost');
return await request(method, params);
});
await expect(
fixture.runtime.dispatch('tabs.list', { browserId: 'chrome' }),
).rejects.toMatchObject({ code });
expect(fixture.request).toHaveBeenCalledWith('tabs.attach', {
tabId: 23,
});
},
);
it('propagates a lost connection while syncing derived tabs', async () => {
const fixture = await runtimeFixture();
await createTab(fixture.runtime);
@@ -2035,13 +2079,13 @@ describe('PlaywrightRuntime command contracts', () => {
if (method === 'tabs.queryDerived')
return [
{
providerTabId: 18,
providerTabId: 22,
title: 'Popup',
url: 'https://example.com/popup',
derivedFromProviderTabId: 17,
},
];
if (method === 'tabs.attach' && params.tabId === 18)
if (method === 'tabs.attach' && params.tabId === 22)
throw new BrowserRuntimeError(
'BROWSER_DISCONNECTED',
'Chrome extension disconnected',
@@ -2169,13 +2213,13 @@ describe('PlaywrightRuntime command contracts', () => {
fixture.request.mockImplementation(
async (method: string, params: Record<string, unknown> = {}) => {
if (method === 'tabs.get')
return { providerTabId: 18, title: 'Popup', url: 'about:blank' };
return { providerTabId: 22, title: 'Popup', url: 'about:blank' };
return await original(method, params);
},
);
const popupTracked = (openerTabId: number): BridgeEvent => ({
type: 'event',
tabId: 18,
tabId: 22,
method: 'qwenBrowser.derivedTabTracked',
params: { openerTabId },
});
@@ -2192,7 +2236,7 @@ describe('PlaywrightRuntime command contracts', () => {
fixture.runtime.dispatch('tabs.list', { browserId: 'chrome' }),
).resolves.toHaveLength(2);
});
expect(lookups()).toEqual([['tabs.get', { tabId: 18 }]]);
expect(lookups()).toEqual([['tabs.get', { tabId: 22 }]]);
await expect(
fixture.runtime.dispatch('tabs.list', { browserId: 'chrome' }),
).resolves.toEqual([
@@ -2216,7 +2260,7 @@ describe('PlaywrightRuntime command contracts', () => {
if (method === 'tabs.queryDerived')
return [
{
providerTabId: 18,
providerTabId: 22,
title: 'Popup',
url: 'about:blank',
derivedFromProviderTabId: 17,
@@ -2296,13 +2340,13 @@ describe('PlaywrightRuntime command contracts', () => {
);
fixture.emitEvent({
type: 'event',
tabId: 18,
tabId: 22,
method: 'qwenBrowser.derivedTabTracked',
params: { openerTabId: 17 },
});
await fixture.runtime.stop();
expect(fixture.listenerCount()).toBe(0);
finish({ providerTabId: 18, title: 'Popup', url: 'about:blank' });
finish({ providerTabId: 22, title: 'Popup', url: 'about:blank' });
await new Promise<void>((resolve) => setImmediate(resolve));
expect(
fixture.request.mock.calls.filter(([method]) => method === 'tabs.attach'),
@@ -2424,6 +2468,73 @@ describe('PlaywrightRuntime command contracts', () => {
).toHaveLength(1);
});
it.each(['handoff', 'cleanup'] as const)(
'finalizes a re-registered crashed provider once for %s',
async (disposition) => {
const fixture = await runtimeFixture();
const oldTab = await createTab(fixture.runtime);
const listeners = new Map(
fixture.page.on.mock.calls as Array<[string, () => void]>,
);
listeners.get('crash')!();
const newTab = await createTab(fixture.runtime);
expect(newTab.id).not.toBe(oldTab.id);
listeners.get('close')!();
await fixture.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep:
disposition === 'handoff'
? [{ tabId: newTab.id, status: 'handoff' }]
: [],
});
expect(
fixture.request.mock.calls.filter(
([method]) => method === 'tabs.close',
),
).toEqual(
disposition === 'handoff' ? [] : [['tabs.close', { tabId: 17 }]],
);
},
);
it.each(['STALE_TAB', 'OPERATION_FAILED'] as const)(
'only forgets a crash-retained tab when cleanup reports it gone: %s',
async (code) => {
const fixture = await runtimeFixture();
await createTab(fixture.runtime);
const listeners = new Map(
fixture.page.on.mock.calls as Array<[string, () => void]>,
);
listeners.get('crash')!();
listeners.get('close')!();
const request = fixture.request.getMockImplementation()!;
const error = new BrowserRuntimeError(code, 'Controlled cleanup failure');
fixture.request.mockImplementation(async (method, params = {}) => {
if (method === 'tabs.close') throw error;
return await request(method, params);
});
try {
for (let attempt = 0; attempt < 2; attempt++) {
const result = fixture.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [],
});
if (code === 'STALE_TAB') await expect(result).resolves.toBeNull();
else await expect(result).rejects.toBe(error);
}
expect(
fixture.request.mock.calls.filter(
([method]) => method === 'tabs.close',
),
).toHaveLength(code === 'STALE_TAB' ? 1 : 2);
} finally {
fixture.request.mockImplementation(request);
}
},
);
it('releases tab and transport state when a page closes', async () => {
const fixture = await runtimeFixture();
const oldTab = await createTab(fixture.runtime);
@@ -2447,6 +2558,464 @@ describe('PlaywrightRuntime command contracts', () => {
fixture.request.mock.calls.filter(([method]) => method === 'tabs.attach'),
).toHaveLength(2);
});
it('finalizes created, deliverable, and handoff tabs by disposition', async () => {
const disposable = await runtimeFixture();
await createTab(disposable.runtime);
await disposable.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [],
});
expect(disposable.request).toHaveBeenCalledWith('tabs.close', {
tabId: 17,
});
const claimed = await runtimeFixture();
const candidates = (await claimed.runtime.dispatch(
'browser.user.openTabs',
{ browserId: 'chrome' },
)) as Array<{ id: string; title: string | null; url: string | null }>;
await claimed.runtime.dispatch('browser.user.claimTab', {
browserId: 'chrome',
tab: candidates[0],
});
await claimed.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
});
expect(claimed.request).toHaveBeenCalledWith('tabs.release', { tabId: 17 });
expect(claimed.request).not.toHaveBeenCalledWith(
'tabs.close',
expect.anything(),
);
const deliverable = await runtimeFixture();
const deliverableTab = await createTab(deliverable.runtime);
await deliverable.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [{ tabId: deliverableTab.id, status: 'deliverable' }],
});
expect(deliverable.request).toHaveBeenCalledWith('tabs.release', {
tabId: 17,
});
expect(deliverable.request).not.toHaveBeenCalledWith(
'tabs.close',
expect.anything(),
);
const handoff = await runtimeFixture();
const handoffTab = await createTab(handoff.runtime);
await handoff.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [{ tabId: handoffTab.id, status: 'handoff' }],
});
await expect(
handoff.runtime.dispatch('tab.url', { tabId: handoffTab.id }),
).resolves.toBe('about:blank');
expect(handoff.request).not.toHaveBeenCalledWith(
'tabs.close',
expect.anything(),
);
expect(handoff.request).not.toHaveBeenCalledWith(
'tabs.release',
expect.anything(),
);
await handoff.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [],
});
expect(handoff.request).toHaveBeenCalledWith('tabs.close', { tabId: 17 });
const popup = await runtimeFixture();
await createTab(popup.runtime);
popup.derivedTabs.push(
{
providerTabId: 23,
derivedFromProviderTabId: 22,
title: 'Nested popup',
url: 'https://example.com/nested-popup',
},
{
providerTabId: 22,
derivedFromProviderTabId: 17,
title: 'Popup',
url: 'https://example.com/popup',
},
);
await popup.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [],
});
expect(popup.request).toHaveBeenCalledWith('tabs.close', { tabId: 22 });
expect(popup.request).toHaveBeenCalledWith('tabs.close', { tabId: 23 });
const latePopup = await runtimeFixture();
await createTab(latePopup.runtime);
latePopup.openTabs.splice(0, 1, {
providerTabId: 22,
title: 'Late popup',
url: 'https://example.com/late-popup',
});
const lateCandidates = (await latePopup.runtime.dispatch(
'browser.user.openTabs',
{ browserId: 'chrome' },
)) as Array<{ id: string; title: string | null; url: string | null }>;
await latePopup.runtime.dispatch('browser.user.claimTab', {
browserId: 'chrome',
tab: lateCandidates[0],
});
latePopup.derivedTabs.push({
providerTabId: 22,
derivedFromProviderTabId: 17,
title: 'Late popup',
url: 'https://example.com/late-popup',
});
await latePopup.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [],
});
expect(latePopup.request).toHaveBeenCalledWith('tabs.close', { tabId: 22 });
expect(latePopup.request).not.toHaveBeenCalledWith('tabs.release', {
tabId: 22,
});
const crashed = await runtimeFixture();
await createTab(crashed.runtime);
const crashHandler = crashed.page.on.mock.calls.find(
([event]) => event === 'crash',
)?.[1] as (() => void) | undefined;
expect(crashHandler).toBeDefined();
crashHandler?.();
await crashed.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [],
});
expect(crashed.request).toHaveBeenCalledWith('tabs.close', { tabId: 17 });
});
it.each(['created', 'claimed', 'deliverable', 'handoff'] as const)(
'preserves %s disposition and cleans healthy popups after an attachment failure',
async (disposition) => {
const fixture = await runtimeFixture();
let tab: TabInfo;
if (disposition === 'claimed') {
const candidates = (await fixture.runtime.dispatch(
'browser.user.openTabs',
{ browserId: 'chrome' },
)) as Array<{ id: string }>;
tab = (await fixture.runtime.dispatch('browser.user.claimTab', {
browserId: 'chrome',
tab: candidates[0],
})) as TabInfo;
} else {
tab = await createTab(fixture.runtime);
}
fixture.derivedTabs.push({ providerTabId: 22 }, { providerTabId: 23 });
const request = fixture.request.getMockImplementation()!;
fixture.request.mockImplementation(async (method, params = {}) => {
if (method === 'tabs.attach' && params.tabId === 22) {
throw new Error('Popup attachment failed');
}
return await request(method, params);
});
await expect(
fixture.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep:
disposition === 'handoff' || disposition === 'deliverable'
? [{ tabId: tab.id, status: disposition }]
: [],
}),
).rejects.toMatchObject({
message: expect.stringContaining('Popup attachment failed'),
});
expect(fixture.request).toHaveBeenCalledWith('tabs.close', { tabId: 23 });
expect(fixture.request).not.toHaveBeenCalledWith('tabs.close', {
tabId: 22,
});
if (disposition === 'handoff') {
expect(fixture.request).not.toHaveBeenCalledWith('tabs.close', {
tabId: 17,
});
expect(fixture.request).not.toHaveBeenCalledWith('tabs.release', {
tabId: 17,
});
await expect(
fixture.runtime.dispatch('tab.url', { tabId: tab.id }),
).resolves.toBe('about:blank');
} else {
expect(fixture.request).toHaveBeenCalledWith(
disposition === 'created' ? 'tabs.close' : 'tabs.release',
{ tabId: 17 },
);
if (disposition !== 'created') {
expect(fixture.request).not.toHaveBeenCalledWith('tabs.close', {
tabId: 17,
});
}
await expect(
fixture.runtime.dispatch('tab.url', { tabId: tab.id }),
).rejects.toMatchObject({ code: 'STALE_TAB' });
}
},
);
it('cleans known tabs and reports a failed derived-tab query', async () => {
const fixture = await runtimeFixture();
await createTab(fixture.runtime);
const request = fixture.request.getMockImplementation()!;
fixture.request.mockImplementation(async (method, params = {}) => {
if (method === 'tabs.queryDerived')
throw new Error('Derived query failed');
return await request(method, params);
});
await expect(
fixture.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [],
}),
).rejects.toMatchObject({
message: expect.stringContaining('Derived query failed'),
});
expect(fixture.request).toHaveBeenCalledWith('tabs.close', { tabId: 17 });
});
it('finalizes an agent-created popup after its opener closes', async () => {
const fixture = await runtimeFixture();
await createTab(fixture.runtime);
const closeHandler = fixture.page.on.mock.calls.find(
([event]) => event === 'close',
)?.[1] as (() => void) | undefined;
expect(closeHandler).toBeDefined();
closeHandler?.();
fixture.derivedTabs.push({
providerTabId: 22,
derivedFromProviderTabId: 17,
title: 'Orphaned popup',
url: 'https://example.com/orphaned-popup',
});
await fixture.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [],
});
expect(fixture.request).toHaveBeenCalledWith('tabs.close', { tabId: 22 });
});
it('syncs authoritative derived tabs without changing the selected tab', async () => {
const fixture = await runtimeFixture();
const selected = await createTab(fixture.runtime);
fixture.derivedTabs.push(
{
providerTabId: 23,
derivedFromProviderTabId: 22,
title: 'Nested popup',
url: 'https://example.com/nested-popup',
},
{
providerTabId: 22,
derivedFromProviderTabId: 17,
title: 'Popup',
url: 'https://example.com/popup',
},
);
await expect(
fixture.runtime.dispatch('tabs.list', { browserId: 'chrome' }),
).resolves.toHaveLength(3);
await expect(
fixture.runtime.dispatch('tabs.selected', { browserId: 'chrome' }),
).resolves.toMatchObject({ id: selected.id });
});
it('cleans up controlled tabs when the runtime closes', async () => {
const created = await runtimeFixture();
await createTab(created.runtime);
await created.runtime.stop();
expect(created.request).toHaveBeenCalledWith('tabs.close', { tabId: 17 });
const claimed = await runtimeFixture();
const candidates = (await claimed.runtime.dispatch(
'browser.user.openTabs',
{ browserId: 'chrome' },
)) as Array<{ id: string }>;
await claimed.runtime.dispatch('browser.user.claimTab', {
browserId: 'chrome',
tab: candidates[0],
});
await claimed.runtime.stop();
expect(claimed.request).toHaveBeenCalledWith('tabs.release', { tabId: 17 });
const handoff = await runtimeFixture();
const handoffTab = await createTab(handoff.runtime);
await handoff.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [{ tabId: handoffTab.id, status: 'handoff' }],
});
await handoff.runtime.stop();
expect(handoff.request).toHaveBeenCalledWith('tabs.close', { tabId: 17 });
});
it('preserves crashed tab ownership when detachment emits close', async () => {
const fixture = await runtimeFixture();
await createTab(fixture.runtime);
for (const event of ['crash', 'close']) {
const handler = fixture.page.on.mock.calls.find(
([name]) => name === event,
)?.[1];
expect(handler).toBeDefined();
handler?.();
}
await fixture.runtime.stop();
expect(fixture.request).toHaveBeenCalledWith('tabs.close', { tabId: 17 });
expect(
fixture.request.mock.calls.filter(([method]) => method === 'tabs.detach'),
).toEqual([['tabs.detach', { tabId: 17 }, 2_000]]);
});
it('closes derived tabs on stop without admitting new attachments', async () => {
const fixture = await runtimeFixture();
await createTab(fixture.runtime);
fixture.derivedTabs.push(
{ providerTabId: 22, derivedFromProviderTabId: 17 },
{ providerTabId: 23, derivedFromProviderTabId: 22 },
);
await fixture.runtime.stop();
for (const tabId of [17, 22, 23]) {
expect(fixture.request).toHaveBeenCalledWith('tabs.close', { tabId });
}
expect(
fixture.request.mock.calls.filter(([method]) => method === 'tabs.attach'),
).toEqual([['tabs.attach', { tabId: 17 }]]);
expect(fixture.listenerCount()).toBe(0);
expect(fixture.stopBridge).toHaveBeenCalledOnce();
});
it('retains authoritative popup ownership during shutdown', async () => {
const fixture = await runtimeFixture();
const candidates = (await fixture.runtime.dispatch(
'browser.user.openTabs',
{
browserId: 'chrome',
},
)) as Array<{ id: string }>;
await fixture.runtime.dispatch('browser.user.claimTab', {
browserId: 'chrome',
tab: candidates[0],
});
fixture.derivedTabs.push({ providerTabId: 17 });
await fixture.runtime.stop();
expect(fixture.request).toHaveBeenCalledWith('tabs.close', { tabId: 17 });
expect(fixture.request).not.toHaveBeenCalledWith('tabs.release', {
tabId: 17,
});
});
it('validates every kept tab before finalization mutates Chrome', async () => {
const fixture = await runtimeFixture();
const tab = await createTab(fixture.runtime);
await expect(
fixture.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [
{ tabId: tab.id, status: 'deliverable' },
{ tabId: tab.id, status: 'handoff' },
],
}),
).rejects.toMatchObject({ code: 'INVALID_ARGUMENT' });
expect(fixture.request).not.toHaveBeenCalledWith(
'tabs.close',
expect.anything(),
);
expect(fixture.request).not.toHaveBeenCalledWith(
'tabs.release',
expect.anything(),
);
});
it('rejects an unknown kept tab id before finalization mutates Chrome', async () => {
const fixture = await runtimeFixture();
await createTab(fixture.runtime);
await expect(
fixture.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [{ tabId: 'tab-not-registered', status: 'deliverable' }],
}),
).rejects.toMatchObject({ code: 'STALE_TAB' });
expect(fixture.request).not.toHaveBeenCalledWith(
'tabs.close',
expect.anything(),
);
expect(fixture.request).not.toHaveBeenCalledWith(
'tabs.release',
expect.anything(),
);
});
it('rejects a kept tab whose page already closed before finalization mutates Chrome', async () => {
const fixture = await runtimeFixture();
const tab = await createTab(fixture.runtime);
const closeHandler = fixture.page.on.mock.calls.find(
([name]) => name === 'close',
)?.[1];
expect(closeHandler).toBeDefined();
closeHandler?.();
await expect(
fixture.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [{ tabId: tab.id, status: 'deliverable' }],
}),
).rejects.toMatchObject({ code: 'STALE_TAB' });
expect(fixture.request).not.toHaveBeenCalledWith(
'tabs.close',
expect.anything(),
);
expect(fixture.request).not.toHaveBeenCalledWith(
'tabs.release',
expect.anything(),
);
});
it('rejects an invalid keep set without cleanup even when synchronization fails', async () => {
const fixture = await runtimeFixture();
const tab = await createTab(fixture.runtime);
const request = fixture.request.getMockImplementation()!;
fixture.request.mockImplementation(async (method, params = {}) => {
if (method === 'tabs.queryDerived')
throw new Error('Derived query failed');
return await request(method, params);
});
await expect(
fixture.runtime.dispatch('tabs.finalize', {
browserId: 'chrome',
keep: [
{ tabId: tab.id, status: 'deliverable' },
{ tabId: tab.id, status: 'handoff' },
],
}),
).rejects.toMatchObject({ code: 'INVALID_ARGUMENT' });
expect(fixture.request).not.toHaveBeenCalledWith(
'tabs.close',
expect.anything(),
);
expect(fixture.request).not.toHaveBeenCalledWith(
'tabs.release',
expect.anything(),
);
});
});
interface RuntimeFixture {
@@ -2462,6 +3031,8 @@ interface RuntimeFixture {
detach: ReturnType<typeof vi.fn>;
};
request: ReturnType<typeof vi.fn>;
derivedTabs: Array<Record<string, unknown>>;
openTabs: Array<Record<string, unknown>>;
probeDetach: ReturnType<typeof vi.fn>;
disconnect(): void;
browserDisconnect(): void;
@@ -2537,9 +3108,11 @@ async function runtimeFixture(
const unrelatedPage = options.unrelatedPage
? fakePage(locator.value, 'Unrelated popup')
: undefined;
const nestedPopupPage = fakePage(locator.value, 'Nested popup');
const pageTargetIds = new Map<Page, string>([
[page.value, 'target-17'],
[popupPage.value, 'target-18'],
[popupPage.value, 'target-22'],
[nestedPopupPage.value, 'target-23'],
]);
if (unrelatedPage !== undefined)
pageTargetIds.set(unrelatedPage.value, 'target-popup');
@@ -2562,8 +3135,9 @@ async function runtimeFixture(
)?.predicate;
for (const candidate of [
unrelatedPage?.value,
page.value,
nestedPopupPage.value,
popupPage.value,
page.value,
]) {
if (
candidate !== undefined &&
@@ -2595,6 +3169,7 @@ async function runtimeFixture(
} as unknown as BrowserContext;
page.methods.context.mockReturnValue(context);
popupPage.methods.context.mockReturnValue(context);
nestedPopupPage.methods.context.mockReturnValue(context);
unrelatedPage?.methods.context.mockReturnValue(context);
const browserOn = vi.fn();
const browser = {
@@ -2606,6 +3181,14 @@ async function runtimeFixture(
const connectionListeners = new Set<(connected: boolean) => void>();
const eventListeners = new Set<(event: BridgeEvent) => void>();
const derivedTabs: Array<Record<string, unknown>> = [];
const openTabs: Array<Record<string, unknown>> = [
{
providerTabId: 17,
title: 'Fixture',
url: 'https://example.com/',
},
];
const request = vi.fn(
async (method: string, params: Record<string, unknown> = {}) => {
if (method === 'tabs.create') {
@@ -2615,7 +3198,8 @@ async function runtimeFixture(
url: 'about:blank',
};
}
if (method === 'tabs.queryDerived') return [];
if (method === 'tabs.queryOpen') return openTabs;
if (method === 'tabs.queryDerived') return derivedTabs;
if (method === 'cdp.send' && params.method === 'Page.getLayoutMetrics')
return {
cssVisualViewport: {
@@ -2630,10 +3214,20 @@ async function runtimeFixture(
return { result: { value: 2 } };
if (method === 'cdp.send' && params.method === 'Page.captureScreenshot')
return { data: jpeg(2, 3).toString('base64') };
if (method === 'tabs.get') {
return (
openTabs.find((tab) => tab['providerTabId'] === params.tabId) ?? {
providerTabId: params.tabId,
title: 'Fixture',
url: 'https://example.com/',
}
);
}
if (method === 'cdp.send' && params.method === 'Target.getTargetInfo') {
const tabId = params.tabId as number;
return {
targetInfo: {
targetId: `target-${params.tabId}`,
targetId: `target-${tabId}`,
type: 'page',
title: 'Fixture',
url: 'about:blank',
@@ -2672,6 +3266,8 @@ async function runtimeFixture(
typingState,
cdp,
request,
derivedTabs,
openTabs,
probeDetach,
stopBridge: bridge.stop as ReturnType<typeof vi.fn>,
browserDisconnect() {
packages/browser-use/src/playwright/playwright-runtime.ts
+10 -1
View File
@@ -28,6 +28,7 @@ import type {
BrowserHistoryEntry,
BrowserInfo,
DispatchResult,
FinalizeTabDisposition,
LogEntry,
} from '../core/primitives.js';
import { commandSchemas, type SupportedCommand } from '../core/schemas.js';
@@ -162,7 +163,7 @@ export class PlaywrightRuntime {
switch (method) {
case 'browsers.list':
try {
await this.bridge.request('ping', {}, 1_500);
await this.bridge.request('ping');
return [this.browserInfo()];
} catch (error) {
if (
@@ -207,6 +208,14 @@ export class PlaywrightRuntime {
case 'tabs.selected':
this.assertBrowser(stringArg(args, 'browserId'));
return await this.session.selectedTabInfo();
case 'tabs.finalize':
this.assertBrowser(stringArg(args, 'browserId'));
await this.session.finalizeTabs(
Array.isArray(args.keep)
? (args.keep as FinalizeTabDisposition[])
: [],
);
return null;
case 'tab.goto': {
const tab = this.tab(args);
const url = stringArg(args, 'url');
packages/browser-use/src/playwright/playwright-session.ts
+121 -40
View File
@@ -16,7 +16,12 @@ import {
import type { ChromeBridge } from '../bridge/index.js';
import { BrowserRuntimeError, staleSessionError } from '../core/errors.js';
import type { BrowserUserTabInfo, TabInfo } from '../core/primitives.js';
import type {
BrowserUserTabInfo,
FinalizeTabDisposition,
FinalizeTabStatus,
TabInfo,
} from '../core/primitives.js';
import {
playwrightTransportAdapter,
QwenPlaywrightTransport,
@@ -93,7 +98,7 @@ export class PlaywrightSession {
.then(async (value) => {
if (this.stopped || tabIdPrefix !== this.tabIdPrefix) return;
const provider = providerTab(value);
await this.registerTab(provider);
await this.registerTab(provider, 'created', false);
})
.catch(() => undefined);
}
@@ -174,6 +179,9 @@ export class PlaywrightSession {
this.removeConnectionListener();
await this.starting?.catch(() => undefined);
await this.registration;
if (this.bridge.isConnected()) {
await this.finalizeTabs([]).catch(() => undefined);
}
await this.transport?.close();
this.invalidateSession();
await this.bridge.stop();
@@ -204,13 +212,11 @@ export class PlaywrightSession {
async newTab(): Promise<TabInfo> {
const provider = providerTab(await this.bridge.request('tabs.create'));
const info = await this.registerTab(provider);
this.selectedTabId = info.id;
return info;
return await this.registerTab(provider, 'created', true);
}
async listTabs(): Promise<TabInfo[]> {
await this.syncDerivedTabs();
await this.syncDerivedTabs('list');
return await Promise.all(
[...this.tabs.values()]
.filter((tab) => !tab.stale)
@@ -291,20 +297,22 @@ export class PlaywrightSession {
'STALE_TAB',
'The Chrome tab changed after discovery; list open tabs again',
);
const info = await this.registerTab(current);
this.selectedTabId = info.id;
return info;
return await this.registerTab(current, 'claimed', true);
}
// Background callers (popup adoption) register tabs too, so registration
// must not move the selection; newTab and claimTab select what they return.
private async registerTab(provider: ProviderTab): Promise<TabInfo> {
private async registerTab(
provider: ProviderTab,
ownership: 'created' | 'claimed',
select: boolean,
): Promise<TabInfo> {
this.assertRunning();
const tabIdPrefix = this.tabIdPrefix;
const existing = [...this.tabs.values()].find(
(tab) => tab.providerTabId === provider.providerTabId && !tab.stale,
);
if (existing !== undefined) {
if (ownership === 'created') existing.ownership = 'created';
if (select) this.selectedTabId = existing.id;
const info = await this.tabInfo(existing);
this.assertRunning();
if (tabIdPrefix !== this.tabIdPrefix) throw staleSessionError();
@@ -317,6 +325,8 @@ export class PlaywrightSession {
(tab) => tab.providerTabId === provider.providerTabId && !tab.stale,
);
if (registered !== undefined) {
if (ownership === 'created') registered.ownership = 'created';
if (select) this.selectedTabId = registered.id;
const info = await this.tabInfo(registered);
this.assertRunning();
if (tabIdPrefix !== this.tabIdPrefix) throw staleSessionError();
@@ -357,9 +367,16 @@ export class PlaywrightSession {
dialogTrace: [],
fileChoosers: new Map(),
navigationWaiters: new Map(),
ownership,
};
for (const previous of this.tabs.values()) {
if (previous.providerTabId !== tab.providerTabId) continue;
if (previous.ownership === 'created') tab.ownership = 'created';
this.tabs.delete(previous.id);
}
this.installPageObservers(tab, transport);
this.tabs.set(tab.id, tab);
if (select) this.selectedTabId = tab.id;
// Chrome does not replay Page.javascriptDialogOpening for a dialog
// that was already open when the tab attached, and Playwright never
// delivers one either — but the modal still blocks the renderer.
@@ -416,11 +433,14 @@ export class PlaywrightSession {
tab.dialogTrace.length = 0;
tab.fileChoosers.clear();
tab.navigationWaiters.clear();
this.tabs.delete(tab.id);
if (this.selectedTabId === tab.id) this.selectedTabId = undefined;
void transport.unregisterTab(tab.providerTabId).catch(() => undefined);
};
page.on('close', release);
page.on('close', () => {
// Detaching a crashed page emits close but must retain cleanup ownership.
if (!released) this.tabs.delete(tab.id);
release();
});
page.on('crash', release);
page.on('dialog', (dialog) => {
// The bridge saw this dialog's CDP events before Playwright delivered
@@ -458,34 +478,36 @@ export class PlaywrightSession {
});
}
private async syncDerivedTabs(): Promise<void> {
const controlledProviders = new Set(
[...this.tabs.values()].map((tab) => tab.providerTabId),
);
const derived = providerTabs(
private async syncDerivedTabs(mode: 'list' | 'finalize'): Promise<void> {
const providers = providerTabs(
await this.bridge.request('tabs.queryDerived'),
);
for (const provider of derived) {
if (
!controlledProviders.has(provider.providerTabId) &&
provider.derivedFromProviderTabId !== undefined &&
controlledProviders.has(provider.derivedFromProviderTabId)
) {
// One unattachable derived tab must not fail the whole listing,
// but a lost session still propagates.
try {
await this.registerTab(provider);
} catch (error) {
if (
error instanceof BrowserRuntimeError &&
(error.code === 'STALE_BROWSER_SESSION' ||
error.code === 'BROWSER_DISCONNECTED')
)
throw error;
const results = await Promise.allSettled(
providers.map((provider) => {
if (this.stopped && mode === 'finalize') {
const tab = [...this.tabs.values()].find(
(tab) => tab.providerTabId === provider.providerTabId,
);
if (tab !== undefined) {
tab.ownership = 'created';
return;
}
return this.bridge.request('tabs.close', {
tabId: provider.providerTabId,
});
}
controlledProviders.add(provider.providerTabId);
}
}
return this.registerTab(provider, 'created', false);
}),
);
const failed = results.find(
(result): result is PromiseRejectedResult =>
result.status === 'rejected' &&
(mode === 'finalize' ||
(result.reason instanceof BrowserRuntimeError &&
(result.reason.code === 'STALE_BROWSER_SESSION' ||
result.reason.code === 'BROWSER_DISCONNECTED'))),
);
if (failed !== undefined) throw failed.reason;
}
private async tabInfo(tab: TabState): Promise<TabInfo> {
@@ -503,13 +525,72 @@ export class PlaywrightSession {
async closeTab(tab: TabState): Promise<void> {
const transport = this.requireTransport();
await this.bridge.request('tabs.close', { tabId: tab.providerTabId });
await this.bridge
.request('tabs.close', { tabId: tab.providerTabId })
.catch((error: unknown) => {
if (
!(error instanceof BrowserRuntimeError) ||
error.code !== 'STALE_TAB'
)
throw error;
});
tab.stale = 'tab';
this.tabs.delete(tab.id);
await transport.unregisterTab(tab.providerTabId);
if (this.selectedTabId === tab.id) this.selectedTabId = undefined;
}
async finalizeTabs(keep: FinalizeTabDisposition[]): Promise<void> {
await this.registration;
const results = await Promise.allSettled([
this.syncDerivedTabs('finalize'),
]);
await this.registration;
const dispositions = new Map<string, FinalizeTabStatus>();
for (const { tabId, status } of keep) {
if (dispositions.has(tabId)) {
throw new BrowserRuntimeError(
'INVALID_ARGUMENT',
`Tab appears more than once in finalize(): ${tabId}`,
);
}
this.claimed(tabId);
dispositions.set(tabId, status);
}
const operations = [...this.tabs.values()]
.filter((tab) => tab.stale !== 'session')
.map(async (tab) => {
const status = dispositions.get(tab.id);
if (status === 'handoff') return;
if (status === 'deliverable' || tab.ownership === 'claimed') {
await this.releaseTab(tab);
return;
}
await this.closeTab(tab);
});
results.push(...(await Promise.allSettled(operations)));
const failed = results.find(
(result): result is PromiseRejectedResult => result.status === 'rejected',
);
if (failed !== undefined) throw failed.reason;
}
private async releaseTab(tab: TabState): Promise<void> {
await this.bridge
.request('tabs.release', { tabId: tab.providerTabId })
.catch((error: unknown) => {
if (
!(error instanceof BrowserRuntimeError) ||
error.code !== 'STALE_TAB'
)
throw error;
});
tab.stale = 'tab';
this.tabs.delete(tab.id);
await this.transport?.unregisterTab(tab.providerTabId);
if (this.selectedTabId === tab.id) this.selectedTabId = undefined;
}
claimed(id: string): TabState {
const tab = this.tabs.get(id);
if (tab?.stale === 'session') throw staleSessionError();
packages/browser-use/src/playwright/qwen-playwright-transport.test.ts
+9 -10
View File
@@ -84,18 +84,17 @@ class FakeBridge implements ChromeBridge {
}
describe('QwenPlaywrightTransport', () => {
it('enables CDP download events when attaching a tab', async () => {
it('registers tabs without changing the browser download policy', async () => {
const bridge = new FakeBridge();
const transport = new QwenPlaywrightTransport(bridge);
await transport.registerTab(7);
expect(bridge.calls).toContainEqual({
method: 'cdp.send',
params: {
tabId: 7,
method: 'Page.setDownloadBehavior',
params: { behavior: 'default', eventsEnabled: true },
},
const request = bridge.request.bind(bridge);
vi.spyOn(bridge, 'request').mockImplementation(async (method, params) => {
if (params?.['method'] === 'Page.setDownloadBehavior') {
throw new Error('Cannot not access browser-level commands');
}
return await request(method, params);
});
const transport = new QwenPlaywrightTransport(bridge);
await expect(transport.registerTab(7)).resolves.toBe('target-7');
await transport.close();
});
packages/browser-use/src/playwright/runtime-state.ts
+1
View File
@@ -49,6 +49,7 @@ export interface TabState {
dialogTrace: DialogTraceEntry[];
fileChoosers: Map<string, FileChooser>;
navigationWaiters: Map<string, Promise<unknown>>;
ownership: 'created' | 'claimed';
/**
* Refs the latest snapshot on this tab actually emitted. Playwright
* restarts ref numbering on every new document, so a ref is valid only
packages/browser-use/src/runtime.test.ts
+181
View File
@@ -0,0 +1,181 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
const installer = vi.hoisted(() => ({
extensionInstalled: vi.fn(async () => true),
install: vi.fn(async () => ({
launcherPath: '/tmp/qwen-home/.qwen/browser-use/native-host.sh',
manifestPaths: [] as string[],
installedPaths: [] as string[],
skippedForeignPaths: [] as string[],
})),
home: vi.fn(() => '/tmp/qwen-home'),
}));
vi.mock('./native-host-installer.js', () => ({
isChromeExtensionInstalled: installer.extensionInstalled,
installChromeNativeHost: installer.install,
nativeHostInstallHome: installer.home,
}));
import { createBrowserBackend } from './runtime.js';
beforeEach(() => {
vi.useFakeTimers();
});
afterEach(() => {
vi.resetAllMocks();
vi.restoreAllMocks();
vi.unstubAllEnvs();
vi.useRealTimers();
});
describe('createBrowserBackend', () => {
it('checks extension installation before registering the Native Host', async () => {
vi.spyOn(process, 'platform', 'get').mockReturnValue('darwin');
vi.stubEnv('QWEN_BROWSER_USE_SOCKET_PATH', '');
const stderr = vi
.spyOn(process.stderr, 'write')
.mockImplementation(() => true);
await createBrowserBackend();
expect(stderr).not.toHaveBeenCalled();
expect(installer.extensionInstalled).toHaveBeenCalledWith({
homeDir: '/tmp/qwen-home',
nativeHostPath: expect.stringMatching(/native-host\.js$/),
});
expect(
installer.extensionInstalled.mock.invocationCallOrder[0],
).toBeLessThan(installer.install.mock.invocationCallOrder[0]!);
expect(installer.install).toHaveBeenCalledWith({
homeDir: '/tmp/qwen-home',
nativeHostPath: expect.stringMatching(/native-host\.js$/),
});
expect(installer.extensionInstalled).toHaveBeenCalledTimes(1);
expect(vi.getTimerCount()).toBe(0);
});
it('waits for Chrome to persist a freshly installed extension', async () => {
vi.spyOn(process, 'platform', 'get').mockReturnValue('darwin');
vi.stubEnv('QWEN_BROWSER_USE_SOCKET_PATH', '');
installer.extensionInstalled.mockResolvedValue(false);
const pending = createBrowserBackend();
await vi.advanceTimersByTimeAsync(10_999);
expect(installer.install).not.toHaveBeenCalled();
expect(installer.extensionInstalled).toHaveBeenCalledTimes(11);
installer.extensionInstalled.mockResolvedValue(true);
await vi.advanceTimersByTimeAsync(1);
await pending;
expect(installer.install).toHaveBeenCalledTimes(1);
expect(vi.getTimerCount()).toBe(0);
});
it('reports an undetected extension after 30 seconds without registering', async () => {
vi.spyOn(process, 'platform', 'get').mockReturnValue('darwin');
vi.stubEnv('QWEN_BROWSER_USE_SOCKET_PATH', '');
installer.extensionInstalled.mockResolvedValue(false);
const pending = createBrowserBackend().catch((error: unknown) => error);
await vi.advanceTimersByTimeAsync(29_999);
expect(installer.extensionInstalled).toHaveBeenCalledTimes(30);
expect(installer.install).not.toHaveBeenCalled();
expect(vi.getTimerCount()).toBe(1);
await vi.advanceTimersByTimeAsync(1);
expect(await pending).toMatchObject({
message: expect.stringContaining(
'Could not detect the Qwen Code Chrome extension after waiting 30 seconds. ' +
'If you just installed it, wait a few seconds and retry Browser Use. ' +
'If it is not installed, install it at chrome://extensions',
),
});
expect(installer.extensionInstalled).toHaveBeenCalledTimes(31);
expect(installer.install).not.toHaveBeenCalled();
expect(vi.getTimerCount()).toBe(0);
});
it('accepts an installation detected on the final check', async () => {
vi.spyOn(process, 'platform', 'get').mockReturnValue('linux');
vi.stubEnv('QWEN_BROWSER_USE_SOCKET_PATH', '');
installer.extensionInstalled.mockResolvedValue(false);
const pending = createBrowserBackend();
await vi.advanceTimersByTimeAsync(29_999);
installer.extensionInstalled.mockResolvedValue(true);
await vi.advanceTimersByTimeAsync(1);
await pending;
expect(installer.install).toHaveBeenCalledTimes(1);
expect(vi.getTimerCount()).toBe(0);
});
it('propagates a read failure while retrying without registering', async () => {
vi.spyOn(process, 'platform', 'get').mockReturnValue('linux');
vi.stubEnv('QWEN_BROWSER_USE_SOCKET_PATH', '');
installer.extensionInstalled
.mockResolvedValueOnce(false)
.mockRejectedValueOnce(new Error('read failed'));
const pending = createBrowserBackend().catch((error: unknown) => error);
await vi.advanceTimersByTimeAsync(1_000);
expect(await pending).toEqual(new Error('read failed'));
expect(installer.install).not.toHaveBeenCalled();
expect(vi.getTimerCount()).toBe(0);
});
it('does not register when extension detection fails', async () => {
vi.spyOn(process, 'platform', 'get').mockReturnValue('linux');
vi.stubEnv('QWEN_BROWSER_USE_SOCKET_PATH', '');
installer.extensionInstalled.mockRejectedValueOnce(
new Error('read failed'),
);
await expect(createBrowserBackend()).rejects.toThrow('read failed');
expect(installer.install).not.toHaveBeenCalled();
});
it('warns about a skipped foreign manifest without aborting', async () => {
vi.spyOn(process, 'platform', 'get').mockReturnValue('linux');
vi.stubEnv('QWEN_BROWSER_USE_SOCKET_PATH', '');
const foreign =
'/tmp/qwen-home/.config/google-chrome/NativeMessagingHosts/com.qwen.browser.json';
installer.install.mockResolvedValue({
launcherPath: '/tmp/qwen-home/.qwen/browser-use/native-host.sh',
manifestPaths: [foreign],
installedPaths: ['/tmp/qwen-home/.qwen/browser-use/native-host.sh'],
skippedForeignPaths: [foreign],
});
const stderr = vi
.spyOn(process.stderr, 'write')
.mockImplementation(() => true);
await expect(createBrowserBackend()).resolves.toBeDefined();
expect(stderr).toHaveBeenCalledTimes(1);
expect(stderr.mock.calls[0]![0]).toContain(foreign);
expect(stderr.mock.calls[0]![0]).toContain(
'another program owns the Chrome Native Messaging manifest',
);
});
it('does not install when a managed socket is configured', async () => {
vi.spyOn(process, 'platform', 'get').mockReturnValue('darwin');
vi.stubEnv('QWEN_BROWSER_USE_SOCKET_PATH', '/tmp/managed.sock');
await createBrowserBackend();
expect(installer.extensionInstalled).not.toHaveBeenCalled();
expect(installer.install).not.toHaveBeenCalled();
});
});
packages/browser-use/src/runtime.ts
+48
View File
@@ -4,13 +4,61 @@
* SPDX-License-Identifier: Apache-2.0
*/
import { fileURLToPath } from 'node:url';
import { ChromeExtensionTransport } from './bridge/index.js';
import { DEFAULT_CHROME_DOCUMENTATION } from './core/chrome-runtime-documentation.js';
import {
installChromeNativeHost,
isChromeExtensionInstalled,
nativeHostInstallHome,
} from './native-host-installer.js';
import { PlaywrightRuntime } from './playwright/playwright-runtime.js';
export type BrowserBackend = Pick<PlaywrightRuntime, 'dispatch' | 'stop'>;
export async function createBrowserBackend(): Promise<BrowserBackend> {
if (
!process.env['QWEN_BROWSER_USE_SOCKET_PATH'] &&
(process.platform === 'darwin' || process.platform === 'linux')
) {
const options = {
homeDir: nativeHostInstallHome(),
nativeHostPath: fileURLToPath(
new URL('./native-host.js', import.meta.url),
),
};
const deadline = Date.now() + 30_000;
while (!(await isChromeExtensionInstalled(options))) {
const remainingMs = deadline - Date.now();
if (remainingMs <= 0) {
throw new Error(
'Could not detect the Qwen Code Chrome extension after waiting 30 seconds. ' +
'If you just installed it, wait a few seconds and retry Browser Use. ' +
'If it is not installed, install it at chrome://extensions ' +
'(Developer mode > Load unpacked), then retry Browser Use.',
);
}
await new Promise((resolve) =>
setTimeout(resolve, Math.min(1_000, remainingMs)),
);
}
const installed = await installChromeNativeHost(options);
if (installed.skippedForeignPaths.length > 0) {
// A foreign manifest under a browser root the user does not run is
// harmless, so this is a warning rather than a failure; but when it is
// the browser in use, Chrome keeps launching the other program's host
// and the bridge only ever reports a generic connection timeout.
process.stderr.write(
'Browser Use: another program owns the Chrome Native Messaging ' +
'manifest at ' +
installed.skippedForeignPaths.join(', ') +
'. It was left unchanged; if that browser is the one you use, ' +
'Chrome will launch that host instead of Qwen Code. ' +
'Remove or move the file, then retry Browser Use.\n',
);
}
}
return new PlaywrightRuntime({
bridge: new ChromeExtensionTransport(),
documentation: DEFAULT_CHROME_DOCUMENTATION,
packages/browser-use/src/sdk.test.ts
+19
View File
@@ -31,6 +31,7 @@ const backend = vi.hoisted(() => ({
return { chooserId: 'chooser-1', multiple: true };
if (method === 'fileChooser.setFiles') return null;
if (method === 'tab.goto') return null;
if (method === 'tabs.finalize') return null;
if (method === 'tab.dialog.accept' || method === 'tab.dialog.dismiss')
return null;
if (method === 'locator.click' || method === 'locator.downloadMedia')
@@ -537,6 +538,24 @@ describe('Browser SDK in the existing Node REPL', () => {
});
});
it('maps tab dispositions to the shared command contract', async () => {
const agent = await setupBrowserRuntime();
const browser = await agent.browsers.get('chrome');
const tab = await browser.tabs.new();
await browser.tabs.finalize({
keep: [{ tab, status: 'handoff' }],
});
expect(backend.calls.at(-1)).toEqual({
method: 'tabs.finalize',
args: {
browserId: 'chrome',
keep: [{ tabId: 'tab-1', status: 'handoff' }],
},
});
});
it('exposes only the actions supported by each dialog type', async () => {
const agent = await setupBrowserRuntime();
const browser = await agent.browsers.get('chrome');
packages/browser-use/src/sdk/browser.ts
+17
View File
@@ -20,6 +20,7 @@ import type {
Browsers,
BrowserTabs,
BrowserUser,
FinalizeTabsOptions,
} from './types.js';
type Args = Record<string, unknown>;
@@ -64,6 +65,22 @@ class TabsProxy implements BrowserTabs {
? undefined
: new TabProxy(this.context, this.browserId, info);
}
finalize(options: FinalizeTabsOptions): Promise<void> {
if (!options || typeof options !== 'object') {
throw new TypeError('tabs.finalize expects an options object');
}
return this.context.call<void>('tabs.finalize', {
browserId: this.browserId,
...(options.keep === undefined
? {}
: {
keep: options.keep.map(({ tab, status }) => ({
tabId: typeof tab === 'string' ? tab : tab.id,
status,
})),
}),
});
}
}
class BrowserUserProxy implements BrowserUser {
packages/browser-use/src/sdk/types.ts
+10
View File
@@ -10,6 +10,7 @@ import type {
BrowserInfo,
BrowserSelectOption,
BrowserUserTabInfo,
FinalizeTabStatus,
LogEntry,
TabInfo,
} from '../core/primitives.js';
@@ -20,6 +21,7 @@ export type {
BrowserInfo,
BrowserSelectOption,
BrowserUserTabInfo,
FinalizeTabStatus,
LogEntry,
TabInfo,
} from '../core/primitives.js';
@@ -366,11 +368,19 @@ export interface BrowserTab {
getJsDialog(): Promise<BrowserDialog | undefined>;
}
export interface FinalizeTabsOptions {
keep?: ReadonlyArray<{
tab: string | BrowserTab | TabInfo;
status: FinalizeTabStatus;
}>;
}
export interface BrowserTabs {
new: () => Promise<BrowserTab>;
list(): Promise<TabInfo[]>;
get(tabId: string): Promise<BrowserTab>;
selected(): Promise<BrowserTab | undefined>;
finalize(options: FinalizeTabsOptions): Promise<void>;
}
export interface BrowserHistoryOptions {
packages/browser-use/src/skill-runtime-kernel.test.ts
+143
View File
@@ -0,0 +1,143 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { afterEach, describe, expect, it, vi } from 'vitest';
// The bundled skill has the model import the staged runtime/index.js from
// inside the node_repl kernel's untrusted vm context, which has no `process`
// global. build.mjs's own load check imports dist/index.js in Node's main
// realm and therefore cannot see that difference, so this test runs the
// skill's first cell, verbatim from SKILL.md, through the real kernel built in
// packages/node-repl/dist against a runtime it stages itself, the way the
// bundle step and `npm run dev` do, into a temporary skill directory.
const repoRoot = fileURLToPath(new URL('../../../', import.meta.url));
const skillFile = path.join(
repoRoot,
'packages/core/src/skills/bundled/browser-use/SKILL.md',
);
const browserUseDist = path.join(repoRoot, 'packages/browser-use/dist');
const nodeReplDist = path.join(repoRoot, 'packages/node-repl/dist');
const requiredArtifacts = [
path.join(nodeReplDist, 'kernel-manager.js'),
path.join(nodeReplDist, 'security-policy.js'),
path.join(nodeReplDist, 'runtime/kernel.mjs'),
path.join(browserUseDist, 'index.js'),
path.join(browserUseDist, 'native-host.js'),
];
interface KernelOutcome {
status: string;
error?: { name: string; message: string; code?: string };
}
interface Kernel {
exec(request: { code: string; timeoutMs: number }): Promise<KernelOutcome>;
addModuleRoot(rawPath: string): Promise<{ path: string; added: boolean }>;
dispose(): void;
}
interface KernelManagerModule {
NodeReplKernelManager: new (options: {
cwd: string;
homeDir: string;
tmpRootDir: string;
policy: unknown;
readableRoots: string[];
}) => Kernel;
}
interface SecurityPolicyModule {
NodeReplSecurityPolicy: { default(): unknown };
}
interface CopyAssetsModule {
copyBrowserUseAssets(root: string, skillDir: string): void;
}
function firstSkillCell(skillBase: string): string {
const skill = fs.readFileSync(skillFile, 'utf8');
const cell = /```js\n([\s\S]*?)```/.exec(skill)?.[1];
if (cell === undefined) throw new Error('SKILL.md has no ```js cell');
expect(cell).toContain("import('/absolute/skill/base/runtime/index.js')");
return cell.replaceAll('/absolute/skill/base', skillBase);
}
describe('bundled skill runtime inside the node_repl kernel', () => {
afterEach(() => {
vi.unstubAllEnvs();
});
it("runs SKILL.md's first cell up to the Chrome bridge without a ReferenceError", async (context) => {
const missing = requiredArtifacts.filter((file) => !fs.existsSync(file));
context.skip(
missing.length > 0,
'needs `npm run build` in packages/node-repl and packages/browser-use; ' +
`missing ${missing.join(', ')}`,
);
context.skip(
process.platform === 'win32',
'asserts the Unix-socket bridge failure; the realm check is host-neutral',
);
const { NodeReplKernelManager } = (await import(
pathToFileURL(path.join(nodeReplDist, 'kernel-manager.js')).href
)) as KernelManagerModule;
const { NodeReplSecurityPolicy } = (await import(
pathToFileURL(path.join(nodeReplDist, 'security-policy.js')).href
)) as SecurityPolicyModule;
const { copyBrowserUseAssets } = (await import(
pathToFileURL(path.join(repoRoot, 'scripts/copy-browser-use-assets.js'))
.href
)) as CopyAssetsModule;
const tmpRootDir = fs.mkdtempSync(
path.join(os.tmpdir(), 'qwen-browser-use-kernel-'),
);
const skillBase = path.join(tmpRootDir, 'skill');
copyBrowserUseAssets(repoRoot, skillBase);
const stagedModules = path.join(skillBase, 'runtime/node_modules');
// A socket whose grandparent directory does not exist makes the bridge
// fail deterministically, after the runtime has already loaded and read
// process.env/process.platform, without touching the user's Chrome.
vi.stubEnv(
'QWEN_BROWSER_USE_SOCKET_PATH',
path.join(tmpRootDir, 'missing', 'nested', 'bridge.sock'),
);
const manager = new NodeReplKernelManager({
cwd: process.cwd(),
homeDir: os.homedir(),
tmpRootDir,
policy: NodeReplSecurityPolicy.default(),
readableRoots: [process.cwd()],
});
try {
await manager.addModuleRoot(stagedModules);
const outcome = await manager.exec({
code: firstSkillCell(skillBase),
timeoutMs: 60_000,
});
expect(outcome.status).toBe('error');
// A `ReferenceError: process is not defined` here means the staged
// runtime lost the kernel-realm `process` binding that build.mjs adds
// through createRequire.
expect(outcome.error).toMatchObject({
name: 'BrowserRuntimeError',
code: 'TRANSPORT_UNAVAILABLE',
message: expect.stringContaining(
'Could not start the local Chrome bridge',
),
});
} finally {
manager.dispose();
fs.rmSync(tmpRootDir, { recursive: true, force: true });
}
}, 90_000);
});
packages/browser-use/tsconfig.json
+1 -1
View File
@@ -7,6 +7,6 @@
"types": ["node", "vitest/globals"],
"lib": ["ES2022", "DOM", "DOM.Iterable"]
},
"include": ["src/**/*.ts"],
"include": ["src/**/*.ts", "scripts/**/*.ts"],
"exclude": ["node_modules", "dist"]
}
packages/browser-use/vitest.config.ts
+1 -1
View File
@@ -8,7 +8,7 @@ import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
include: ['src/**/*.test.ts'],
include: ['src/**/*.test.ts', 'scripts/**/*.test.ts'],
environment: 'node',
// RPC-timeout exemption; see scripts/tests/unit-vitest-configs.test.ts.
dangerouslyIgnoreUnhandledErrors: process.platform !== 'linux',
packages/chrome-extension/PRIVACY.md
+20 -16
View File
@@ -1,38 +1,42 @@
# Privacy Policy — Qwen Code Chrome Extension
Last updated: 2026-07-30
Last updated: 2026-09-03
## Summary
The Qwen Code Chrome Extension ("the Extension") bridges the Chrome browser with a locally running Qwen Code daemon process on your machine. The Extension itself does **not** collect, transmit, or share your personal data with any external server — all of its own communication stays on `localhost` / `127.0.0.1`. However, the local daemon it talks to may send page content to the AI model provider you configure; see [What the daemon does with page content](#what-the-daemon-does-with-page-content).
The Qwen Code Chrome Extension ("the Extension") bridges Chrome with locally running Qwen Code processes on your machine. The Extension itself does **not** collect, transmit, or share your personal data with an external server: its own communication uses `localhost` / `127.0.0.1` or Chrome Native Messaging. Qwen Code may send browser data to the AI model provider you configure; see [What Qwen Code does with browser data](#what-qwen-code-does-with-browser-data).
## What data the Extension accesses
| Permission | Purpose | Data handled |
| ----------- | ----------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| `tabs` | Select the active tab and read its URL and title for AI-assisted browser automation | Active tab metadata — sent only to your local daemon |
| `debugger` | Attach Chrome DevTools Protocol (CDP) to the active tab so the local daemon can drive browser actions | CDP commands/responses — the attached tab's DOM/text, console output, network activity, and cookies, sent to your local daemon |
| `storage` | Persist extension configuration (daemon address, optional bearer token) | Key-value settings only, no browsing data |
| `alarms` | Schedule periodic connection health checks to the local daemon | No user data |
| `sidePanel` | Display the extension's side panel UI | No data access beyond rendering UI |
| Permission | Purpose | Data handled |
| ----------------- | --------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `tabs` | List open HTTP(S) tabs and read their URL and title for AI-assisted browser automation | Open-tab metadata — sent only to the local Browser Use process |
| `debugger` | Attach Chrome DevTools Protocol (CDP) to a claimed tab so Qwen Code can drive browser actions | CDP commands/responses — the attached tab's DOM/text, console output, network activity, and cookies, sent to a local process |
| `history` | Search browsing history when a Browser Use task explicitly needs it | Matching URLs, titles, and visit times — sent only to the local Browser Use process |
| `nativeMessaging` | Connect the extension to the local Browser Use process | Browser commands and results exchanged with a native host on this machine |
| `storage` | Persist extension configuration and transient Browser Use tab state | Configuration, owned-tab identifiers, and tab-group identifiers |
| `tabGroups` | Group tabs created by Browser Use and label the active session | Group identifiers and the Browser Use session name |
| `alarms` | Schedule periodic connection health checks to the local daemon | No user data |
| `sidePanel` | Display the extension's side panel UI | No data access beyond rendering UI |
## Where data goes
- **Nowhere external _from the Extension_.** The Extension's `host_permissions` are restricted to `http://127.0.0.1/*` and `http://localhost/*` — it can only talk to a daemon process running on your own machine.
- Tab content and CDP traffic are forwarded **exclusively** to `127.0.0.1` / `localhost` on a port you configure.
- Tab content and CDP traffic are forwarded **exclusively** to a local Qwen process through `127.0.0.1` / `localhost` or Chrome Native Messaging.
- No analytics, no telemetry, and no third-party endpoints _in the Extension itself_.
### What the daemon does with page content
### What Qwen Code does with browser data
The Extension only ever talks to localhost, but understand the wider system before you enable browser automation:
The Extension only communicates with local processes, but understand the wider system before you use browser automation:
- Through the `debugger` permission the local daemon can read the attached tab's **page DOM/text, console output, network activity, and cookies**, and can drive actions on the page.
- Through the `history` permission Browser Use can search matching URLs, titles, and visit times. Those results can be included in model requests when a task uses them.
- The daemon forwards that page content to the **AI model provider you have configured** in Qwen Code (for example an OpenAI-, Anthropic-, or Alibaba-hosted endpoint) so the model can act on it. Page content therefore **leaves your machine** as part of ordinary model requests.
- "Stays on localhost" describes the Extension's own network boundary; it does **not** describe the daemon-to-model-provider path. Only enable browser automation with a model provider you trust to receive the content of the pages you automate.
- "Stays local" describes the Extension's own communication boundary; it does **not** describe the Qwen Code-to-model-provider path. Only use browser automation with a model provider you trust to receive the browser data needed by the task.
## Data storage
- The Extension stores only configuration keys (daemon address and an optional bearer token) in `chrome.storage`.
- The Extension stores configuration plus transient Browser Use session state such as owned-tab and tab-group identifiers in `chrome.storage`.
- No browsing history, page content, or personal information is persisted by the Extension.
## Data sharing
@@ -42,8 +46,8 @@ The Extension only ever talks to localhost, but understand the wider system befo
## User control
- You can uninstall the Extension at any time via `chrome://extensions`.
- You can review and revoke all permissions via Chrome's site settings.
- Chrome shows the Extension's declared permissions during installation or an update that adds new warning permissions.
- You can disable or uninstall the Extension at any time via `chrome://extensions` to remove Browser Use access.
- Clearing extension storage removes all persisted configuration.
## Contact
packages/chrome-extension/README.md
+35 -9
View File
@@ -1,16 +1,19 @@
# @qwen-code/chrome-bridge
A Chrome extension that brings Qwen Code into the browser as a thin client of a
local [`qwen serve`](../../docs/users/qwen-serve.md) daemon — no Native
Messaging host to install.
local [`qwen serve`](../../docs/users/qwen-serve.md) daemon and provides the
Chrome side of Browser Use.
It does two things:
It provides:
- **Side panel** — frames the daemon's Web Shell (chat + tools), the same UI the
daemon serves to the browser. The panel has no UI of its own.
- **Service worker** — a CDP-tunnel pipe. It connects to the daemon's `/acp`
WebSocket and bridges `cdp_*` frames into `chrome.debugger`, so the agent can
drive the real browser when an external CDP MCP adapter is configured.
- **Browser Use bridge** — connects to the local `com.qwen.browser` Native
Messaging host, lists open HTTP(S) tabs, and forwards Playwright CDP traffic
for tabs claimed by Browser Use.
- **Readiness warning** — the framed Web Shell stays usable for chat while a
small status message distinguishes a disabled CDP tunnel from a missing
browser automation adapter.
@@ -45,19 +48,42 @@ Do not replace this command with `--open-with-auth`. That mode delivers its gene
Once the daemon is reachable and permits framing, the side panel swaps the
welcome screen for the chat UI automatically.
## Browser Automation Tools
## Browser Use
The command above only makes the side panel and Web Shell available. Browser
automation tools such as console/network inspection, screenshots, and page
clicking require an explicit external MCP adapter command:
Browser Use ships with Qwen Code as a built-in skill and SDK runtime, using
Qwen's standard Node REPL. No separate Qwen extension installation is needed.
On macOS and Linux, its first initialization checks that this Chrome extension
is installed before automatically registering the local Native Messaging host.
Installing the extension opts into that local setup. If the extension is not
found, Browser Use prompts you to install it without writing Native Host files.
It does not require `qwen serve`, does not expose
a Chrome debugging port, and supports one active Browser Use session in this
release.
The Native Host launcher and browser registrations remain after Qwen exits.
Run `node <skill-base>/runtime/scripts/native-host-setup.js status` to inspect
them, or use `uninstall` to remove files owned by Browser Use. `<skill-base>` is
the Browser Use skill's base directory shown when loading the skill. Uninstall
the Chrome extension as well to prevent automatic registration on a later use.
Installing the Chrome extension authorizes Browser Use to list and claim open
top-level HTTP(S) tabs. Its declared `history` permission supports explicit,
bounded history queries. The existing toolbar action and side panel continue to
open the Qwen UI.
## Web Shell CDP tunnel
The `qwen serve` command above only makes the side panel and Web Shell
available. Browser automation through the Web Shell's older CDP tunnel still
requires an explicit external MCP adapter command:
```bash
QWEN_CDP_MCP_COMMAND=/path/to/cdp-mcp-adapter \
qwen serve --allow-origin chrome-extension://<this-extension-id>
```
No browser automation adapter is bundled with the main `@qwen-code/qwen-code`
package. When `QWEN_CDP_MCP_COMMAND` is unset, the extension can still open the
No adapter for this older CDP tunnel is bundled with `@qwen-code/qwen-code`.
When `QWEN_CDP_MCP_COMMAND` is unset, the extension can still open the
Web Shell, but the daemon will not register browser automation MCP tools.
Install the adapter separately and point the daemon at its executable:
packages/chrome-extension/package.json
+1 -1
View File
@@ -38,7 +38,7 @@
"dev:chrome": "open -a 'Google Chrome' --args --load-extension=$PWD/dist/extension --auto-open-devtools-for-tabs",
"package": "npm run build && node scripts/package-extension.js",
"clean": "./scripts/clean.sh",
"typecheck": "tsc --noEmit"
"typecheck": "tsc --noEmit && tsc --noEmit -p tsconfig.browser-use-bridge.json"
},
"engines": {
"node": ">=22.0.0"
packages/chrome-extension/public/manifest.json
+11 -1
View File
@@ -3,9 +3,19 @@
"name": "Qwen Code",
"version": "0.0.0",
"description": "Bridge between Chrome browser and Qwen CLI for enhanced AI interactions",
"minimum_chrome_version": "125",
"key": "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAjG5E28TwkSdpl08Bqat3ZslK+HFrhsU5bFo8bvkbhkJaFrvw+NTUOtAzF+/G7oSuzhwk6YijlCzCh5/LrMkgF78gfs31Iz1P2dKAffcGGlAjvP4wAT7qWoKxnOkUy1t4NvvU0oF5WjO48Q697gn3VhmFXYcg9oYbGZBg5DHDUTXBJ6cx5WpTn3GwHQzpPXJZ64BCTQTSL5kp6bcDS74LmTJTREEfE/pHOlqBmzj3VcRN5nF2z/gz/ftZucEQJIm2qjcDqrC9aFDxCQHS7WMcii3FXj6SvjbTccXjiIAIZSAUNUpeIG9z+Zzgm9ycPoqvoBSyTBuPwTplXIneM4XyBQIDAQAB",
"permissions": ["tabs", "storage", "debugger", "alarms", "sidePanel"],
"permissions": [
"tabs",
"tabGroups",
"storage",
"debugger",
"history",
"alarms",
"sidePanel",
"nativeMessaging"
],
"host_permissions": ["http://127.0.0.1/*", "http://localhost/*"],
packages/chrome-extension/src/background/browser-use-bridge.integration.test.ts
+1151
View File
File diff suppressed because it is too large. Load diff
packages/chrome-extension/src/background/browser-use-bridge.js
+1180
View File
File diff suppressed because it is too large. Load diff
packages/chrome-extension/src/background/browser-use-bridge.lifecycle.test.ts
+508
View File
@@ -0,0 +1,508 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
// @vitest-environment node
import { readFile } from 'node:fs/promises';
import { webcrypto } from 'node:crypto';
import vm from 'node:vm';
import { afterEach, expect, test, vi } from 'vitest';
const source = await readFile(
new URL('./browser-use-bridge.js', import.meta.url),
'utf8',
);
afterEach(() => vi.useRealTimers());
async function fixture(
alarms = new Map<string, chrome.alarms.AlarmCreateInfo>(),
localState: Record<string, unknown> = {},
) {
vi.useFakeTimers();
const listeners = new Map<string, (...args: unknown[]) => void>();
const event = (name: string) => ({
addListener: (listener: (...args: unknown[]) => void) =>
listeners.set(name, listener),
});
const emit = (name: string, ...args: unknown[]) =>
listeners.get(name)?.(...args);
const tabs = new Map<number, Record<string, unknown>>([
[1, { id: 1, url: 'https://example.test', windowId: 1, groupId: -1 }],
]);
const attached = new Set<number>();
const saved: Record<string, unknown> = {};
const port = {
onMessage: event('message'),
onDisconnect: event('disconnect'),
postMessage: vi.fn(),
disconnect: vi.fn(),
};
const chromeApi = {
runtime: { id: 'extension-id', connectNative: vi.fn(() => port) },
alarms: {
onAlarm: event('alarm'),
create: vi.fn(
async (name: string, info: chrome.alarms.AlarmCreateInfo) => {
alarms.set(name, info);
},
),
get: vi.fn(async (name: string) => alarms.get(name)),
clear: vi.fn(async (name: string) => alarms.delete(name)),
},
storage: {
local: {
get: vi.fn(async () => ({ ...localState })),
set: vi.fn(async (value: Record<string, unknown>) => {
Object.assign(localState, structuredClone(value));
}),
},
session: {
get: vi.fn(async () => saved),
set: vi.fn(async (value: Record<string, unknown>) => {
Object.assign(saved, structuredClone(value));
}),
},
},
tabs: {
query: vi.fn(async () => [...tabs.values()]),
get: vi.fn(async (id: number) => {
if (!tabs.has(id)) throw new Error('No tab');
return tabs.get(id);
}),
create: vi.fn(async () => {
const tab = { id: 2, url: 'about:blank', windowId: 1, groupId: -1 };
tabs.set(2, tab);
return tab;
}),
remove: vi.fn(async (id: number) => {
tabs.delete(id);
attached.delete(id);
emit('removed', id);
}),
group: vi.fn(async ({ tabIds }: { tabIds: number[] }) => {
for (const id of tabIds) tabs.get(id)!.groupId = 10;
return 10;
}),
ungroup: vi.fn(async (id: number) => {
if (tabs.has(id)) tabs.get(id)!.groupId = -1;
}),
onRemoved: event('removed'),
onCreated: event('created'),
},
tabGroups: {
get: vi.fn(async () => ({ title: 'Qwen Browser' })),
update: vi.fn(async () => ({})),
},
debugger: {
attach: vi.fn(async ({ tabId }: { tabId: number }) => {
if (attached.has(tabId))
throw new Error('Another debugger is already attached');
attached.add(tabId);
}),
detach: vi.fn(async ({ tabId }: { tabId: number }) => {
attached.delete(tabId);
}),
sendCommand: vi.fn(async () => ({})),
onDetach: event('detached'),
onEvent: event('debuggerEvent'),
},
};
const context = vm.createContext({
chrome: chromeApi,
crypto: webcrypto,
setTimeout,
clearTimeout,
Date,
Error,
TextDecoder,
TextEncoder,
Uint8Array,
atob,
});
vm.runInContext(source, context);
const api = vm.runInContext(
'({dispatch, restoreState, attachedTabs, agentOwnedTabs, derivedTabParents})',
context,
) as {
restoreState(): Promise<void>;
dispatch(
method: string,
params?: Record<string, unknown>,
): Promise<unknown>;
attachedTabs: Set<number>;
agentOwnedTabs: Set<number>;
derivedTabParents: Map<number, number>;
};
await vi.advanceTimersByTimeAsync(0);
return {
...api,
chromeApi,
tabs,
attached,
saved,
localState,
alarms,
emit,
port,
};
}
test.each(['group', 'title'])(
'persists pruned restored state when %s setup fails',
async (stage) => {
const f = await fixture();
Object.assign(f.saved, {
agentOwnedTabs: [1, 99],
derivedTabParents: [[99, 1]],
});
if (stage === 'group')
f.chromeApi.tabs.group.mockRejectedValueOnce(new Error('group failed'));
else
f.chromeApi.tabGroups.update.mockRejectedValueOnce(
new Error('title failed'),
);
await expect(f.restoreState()).resolves.toBeUndefined();
expect(f.saved.agentOwnedTabs).toEqual([1]);
expect(f.saved.derivedTabParents).toEqual([]);
expect(f.tabs.has(1)).toBe(true);
},
);
test('restores the remaining owned tabs after one grouping failure', async () => {
const f = await fixture();
f.tabs.set(3, {
id: 3,
url: 'https://example.test/other',
windowId: 2,
groupId: -1,
});
Object.assign(f.saved, { agentOwnedTabs: [1, 3, 99] });
f.chromeApi.tabs.group.mockRejectedValueOnce(new Error('group failed'));
await expect(f.restoreState()).resolves.toBeUndefined();
expect(f.chromeApi.tabs.group).toHaveBeenCalledWith({ tabIds: [3] });
expect(f.tabs.get(3)?.groupId).toBe(10);
expect(f.saved.agentOwnedTabs).toEqual([1, 3]);
});
test('detach and reattach wait for Chrome to actually release its debugger', async () => {
const f = await fixture();
await f.dispatch('tabs.attach', { tabId: 1 });
let finish!: () => void;
f.chromeApi.debugger.detach.mockImplementationOnce(async ({ tabId }) => {
await new Promise<void>((resolve) => {
finish = resolve;
});
f.attached.delete(tabId);
});
let released = false;
const release = f.dispatch('tabs.detach', { tabId: 1 }).then(() => {
released = true;
});
await vi.advanceTimersByTimeAsync(0);
let reattached = false;
const attach = f.dispatch('tabs.attach', { tabId: 1 }).then(() => {
reattached = true;
});
// Observe rejections immediately too, so the red baseline has no unhandled rejection.
const result = Promise.allSettled([release, attach]);
await vi.advanceTimersByTimeAsync(500);
expect(released).toBe(false);
expect(reattached).toBe(false);
expect(f.chromeApi.debugger.attach).toHaveBeenCalledTimes(1);
finish();
expect(await result).toEqual([
{ status: 'fulfilled', value: undefined },
{ status: 'fulfilled', value: undefined },
]);
expect(f.attached.has(1)).toBe(true);
});
test('failed detach retains attachment and does not report successful release', async () => {
const f = await fixture();
await f.dispatch('tabs.attach', { tabId: 1 });
f.chromeApi.debugger.detach.mockRejectedValueOnce(new Error('detach failed'));
await expect(f.dispatch('tabs.detach', { tabId: 1 })).rejects.toThrow(
'detach failed',
);
expect(f.attachedTabs.has(1)).toBe(true);
await f.dispatch('tabs.detach', { tabId: 1 });
expect(f.attachedTabs.has(1)).toBe(false);
});
test('a pending advisory overlay does not block debugger release', async () => {
const f = await fixture();
let finish!: () => void;
f.chromeApi.debugger.sendCommand.mockImplementationOnce(async () => {
await new Promise<void>((resolve) => {
finish = resolve;
});
return {};
});
const attaching = f.dispatch('tabs.attach', { tabId: 1 });
await vi.advanceTimersByTimeAsync(0);
expect(f.attached.has(1)).toBe(true);
let released = false;
const release = f.dispatch('tabs.detach', { tabId: 1 }).then(() => {
released = true;
});
await vi.advanceTimersByTimeAsync(500);
expect(released).toBe(true);
expect(f.attached.has(1)).toBe(false);
finish();
await Promise.all([attaching, release]);
});
test('a foreign debugger still causes a conflict and is never detached', async () => {
const f = await fixture();
f.attached.add(1);
await expect(f.dispatch('tabs.attach', { tabId: 1 })).rejects.toMatchObject({
code: 'TAB_DEBUGGER_CONFLICT',
});
expect(f.chromeApi.debugger.detach).not.toHaveBeenCalled();
});
test.each([' ', 'Enter'])(
'only Enter input can claim a later popup (key=%j)',
async (key) => {
const f = await fixture();
await f.dispatch('cdp.send', {
tabId: 1,
method: 'Input.dispatchKeyEvent',
params: { type: 'keyDown', key, text: key },
});
const popup = {
id: 3,
openerTabId: 1,
url: 'https://popup.test',
windowId: 1,
};
f.tabs.set(3, popup);
f.emit('created', popup);
await vi.advanceTimersByTimeAsync(0);
expect(f.agentOwnedTabs.has(3)).toBe(key === 'Enter');
expect(f.derivedTabParents.has(3)).toBe(key === 'Enter');
},
);
test('a popup known only by its pending URL stays controllable', async () => {
// `chrome.tabs.onCreated` delivers a cross-origin popup with an empty `url`
// and the destination in `pendingUrl`. The claim already keys on
// `pendingUrl`; the runtime's immediate `tabs.get` must not then reject the
// same tab as unsupported because it only looked at `url`.
const f = await fixture();
await f.dispatch('cdp.send', {
tabId: 1,
method: 'Input.dispatchKeyEvent',
params: { type: 'keyDown', key: 'Enter', text: 'Enter' },
});
const popup = {
id: 3,
openerTabId: 1,
url: '',
pendingUrl: 'https://popup.test/cart',
windowId: 1,
};
f.tabs.set(3, popup);
f.emit('created', popup);
await vi.advanceTimersByTimeAsync(0);
expect(f.agentOwnedTabs.has(3)).toBe(true);
await expect(f.dispatch('tabs.get', { tabId: 3 })).resolves.toMatchObject({
providerTabId: 3,
});
await expect(f.dispatch('tabs.attach', { tabId: 3 })).resolves.toBeDefined();
});
test.each(['persist', 'attach'])(
'create failure during %s removes only the new tab',
async (stage) => {
const f = await fixture();
if (stage === 'persist')
f.chromeApi.storage.session.set.mockRejectedValueOnce(
new Error('create failed'),
);
if (stage === 'attach')
f.chromeApi.debugger.attach.mockRejectedValueOnce(
new Error('create failed'),
);
f.chromeApi.tabs.remove.mockImplementationOnce(async (id) => {
f.tabs.delete(id);
f.attached.delete(id);
});
await expect(f.dispatch('tabs.create', {})).rejects.toThrow(
'create failed',
);
expect([...f.tabs.keys()]).toEqual([1]);
expect(f.agentOwnedTabs.size).toBe(0);
expect(f.derivedTabParents.size).toBe(0);
expect(f.attached.size).toBe(0);
expect(f.saved.agentOwnedTabs).toEqual([]);
},
);
test('user cancellation releases derived ownership, grouping and persisted state', async () => {
const f = await fixture();
await f.dispatch('cdp.send', {
tabId: 1,
method: 'Input.dispatchKeyEvent',
params: { type: 'keyDown', key: 'Enter' },
});
const popup = {
id: 3,
openerTabId: 1,
url: 'https://popup.test',
windowId: 1,
};
f.tabs.set(3, popup);
f.emit('created', popup);
await vi.advanceTimersByTimeAsync(0);
await f.dispatch('tabs.attach', { tabId: 3 });
f.attached.delete(3);
f.emit('detached', { tabId: 3 }, 'canceled_by_user');
await vi.advanceTimersByTimeAsync(0);
expect(f.tabs.get(3)?.groupId).toBe(-1);
expect(f.agentOwnedTabs.has(3)).toBe(false);
expect(f.derivedTabParents.has(3)).toBe(false);
expect(f.saved.agentOwnedTabs).toEqual([]);
expect(f.saved.derivedTabParents).toEqual([]);
expect(await f.dispatch('tabs.queryDerived', {})).toEqual([]);
expect(f.port.postMessage).toHaveBeenCalledWith(
expect.objectContaining({ method: 'qwenBrowser.detached', tabId: 3 }),
);
});
test('an absent host waits for an alarm without rewriting session state or retrying every second', async () => {
const f = await fixture();
const writes = f.chromeApi.storage.session.set.mock.calls.length;
f.emit('disconnect');
await vi.advanceTimersByTimeAsync(29_000);
expect(f.chromeApi.runtime.connectNative).toHaveBeenCalledTimes(1);
expect(f.chromeApi.storage.session.set).toHaveBeenCalledTimes(writes);
expect(f.alarms.size).toBe(1);
const [name, info] = [...f.alarms][0]!;
expect(info.delayInMinutes).toBe(0.5);
f.alarms.delete(name);
f.emit('alarm', { name });
await vi.advanceTimersByTimeAsync(0);
expect(f.chromeApi.runtime.connectNative).toHaveBeenCalledTimes(2);
f.emit('message', {
type: 'request',
id: 'ping',
method: 'ping',
params: {},
});
await vi.advanceTimersByTimeAsync(61_000);
expect(f.port.postMessage).toHaveBeenCalledWith(
expect.objectContaining({ id: 'ping', ok: true }),
);
expect(f.chromeApi.runtime.connectNative).toHaveBeenCalledTimes(2);
expect(f.port.disconnect).not.toHaveBeenCalled();
});
test('worker restart preserves the scheduled retry instead of starting another host', async () => {
const first = await fixture();
first.emit('disconnect');
await vi.advanceTimersByTimeAsync(0);
const restarted = await fixture(first.alarms, first.localState);
expect(restarted.chromeApi.runtime.connectNative).not.toHaveBeenCalled();
const name = [...restarted.alarms.keys()][0]!;
restarted.alarms.delete(name);
restarted.emit('alarm', { name });
await vi.advanceTimersByTimeAsync(0);
expect(restarted.chromeApi.runtime.connectNative).toHaveBeenCalledTimes(1);
});
test('user cancellation persists even while grouping another tab is pending', async () => {
const f = await fixture();
await f.dispatch('tabs.attach', { tabId: 1 });
f.agentOwnedTabs.add(1);
let finish!: () => void;
f.chromeApi.tabs.group.mockImplementationOnce(async () => {
await new Promise<void>((resolve) => {
finish = resolve;
});
return 10;
});
const creating = f.dispatch('tabs.create', {});
await vi.advanceTimersByTimeAsync(0);
f.emit('detached', { tabId: 1 }, 'canceled_by_user');
await vi.advanceTimersByTimeAsync(0);
expect(f.saved.agentOwnedTabs).not.toContain(1);
finish();
await creating;
await vi.advanceTimersByTimeAsync(0);
expect(f.chromeApi.tabs.ungroup).toHaveBeenCalledWith(1);
});
test('a grouping failure during create keeps the new tab owned and attached', async () => {
const f = await fixture();
f.chromeApi.tabs.group.mockRejectedValueOnce(
new Error('Tabs cannot be edited right now (user may be dragging a tab).'),
);
await expect(f.dispatch('tabs.create', {})).resolves.toMatchObject({
providerTabId: 2,
});
expect(f.chromeApi.tabs.remove).not.toHaveBeenCalled();
expect(f.tabs.has(2)).toBe(true);
expect(f.attached.has(2)).toBe(true);
expect(f.agentOwnedTabs.has(2)).toBe(true);
expect(f.saved.agentOwnedTabs).toEqual([2]);
await f.dispatch('tabs.release', { tabId: 2 });
expect(f.chromeApi.tabs.ungroup).toHaveBeenCalledWith(2);
expect(f.saved.agentOwnedTabs).toEqual([]);
});
test('closing an owned tab that already vanished is not a failure', async () => {
const f = await fixture();
await f.dispatch('tabs.attach', { tabId: 1 });
f.chromeApi.tabs.remove.mockRejectedValueOnce(
new Error('No tab with id: 1.'),
);
await expect(f.dispatch('tabs.close', { tabId: 1 })).resolves.toBeNull();
f.chromeApi.tabs.remove.mockRejectedValueOnce(
new Error('Tabs cannot be edited right now.'),
);
await expect(f.dispatch('tabs.close', { tabId: 1 })).rejects.toThrow(
'cannot be edited',
);
});
test('a tab vanishing before the debugger attaches is reported as stale', async () => {
const f = await fixture();
f.chromeApi.debugger.attach.mockRejectedValueOnce(
new Error('No tab with id: 1.'),
);
await expect(f.dispatch('tabs.attach', { tabId: 1 })).rejects.toMatchObject({
code: 'STALE_TAB',
});
expect(f.attachedTabs.has(1)).toBe(false);
});
test('listed tab titles and urls stay within the SDK schema bound', async () => {
const f = await fixture();
f.tabs.set(4, {
id: 4,
title: 't'.repeat(30_000),
url: `https://example.test/?q=${'u'.repeat(30_000)}`,
windowId: 1,
groupId: -1,
});
const listed = (await f.dispatch('tabs.queryOpen', {})) as Array<{
providerTabId: number;
title: string | null;
url: string | null;
}>;
const long = listed.find((tab) => tab.providerTabId === 4);
expect(long?.url?.length).toBe(20_000);
expect(long?.url?.startsWith('https://example.test/?q=u')).toBe(true);
expect(long?.title?.length).toBe(20_000);
const short = listed.find((tab) => tab.providerTabId === 1);
expect(short?.url).toBe('https://example.test');
expect(short?.title).toBeNull();
});
packages/chrome-extension/src/background/service-worker.ts
+1
View File
@@ -21,6 +21,7 @@ import {
} from './cdp-bridge';
import { getDaemonConfig } from '../daemon/config.js';
import { checkDaemonHealth } from '../daemon/discovery.js';
import './browser-use-bridge.js';
/* global WebSocket, console, setTimeout, chrome, TextEncoder, btoa */
packages/chrome-extension/src/sidepanel/sidepanel-assets.test.ts
+3
View File
@@ -55,10 +55,13 @@ describe('side panel capability status assets', () => {
expect(manifest.permissions).toEqual([
'tabs',
'tabGroups',
'storage',
'debugger',
'history',
'alarms',
'sidePanel',
'nativeMessaging',
]);
});
packages/chrome-extension/tsconfig.browser-use-bridge.json
+8
View File
@@ -0,0 +1,8 @@
{
"extends": "./tsconfig.json",
"compilerOptions": {
"allowJs": true,
"checkJs": true
},
"include": ["src/background/browser-use-bridge.js"]
}
packages/cli/src/config/config.integration.test.ts
+14
View File
@@ -487,6 +487,20 @@ describe('skill settings providers', async () => {
expect(provider()).toEqual(new Set(['hard']));
});
it('reflects Browser Use disable and re-enable changes without restarting', () => {
const settings = {
merged: { skills: { enabled: [] as string[], disabled: [] as string[] } },
forScope: () => ({ settings: { skills: {} } }),
};
const provider = buildDisabledSkillNamesProvider(settings as never);
expect(provider().has('browser-use')).toBe(false);
settings.merged.skills.disabled = ['browser-use'];
expect(provider().has('browser-use')).toBe(true);
settings.merged.skills.disabled = [];
expect(provider().has('browser-use')).toBe(false);
expect(provider().has('computer-use')).toBe(false);
});
it('reads explicit enables from the current merged settings', () => {
const settings = {
merged: { skills: { enabled: [' REVIEW '] } },
packages/cli/src/config/skill-settings.test.ts
+30
View File
@@ -47,6 +47,36 @@ function fakeSettings({
}
describe('resolveSkillSettings', () => {
it('makes Browser Use and Computer Use available by default', () => {
const result = resolveSkillSettings(fakeSettings({ merged: {} }));
expect(result.disabledNames).toEqual(new Set());
expect(result.disablements.has('browser-use')).toBe(false);
expect(result.disabledNames.has('computer-use')).toBe(false);
});
it('lets the skills picker enable, disable, and re-enable Browser Use', () => {
let lists = { disabled: [] as string[], enabled: [] as string[] };
for (const enabled of [true, false, true]) {
lists = updateWorkspaceSkillSettingLists(lists, 'browser-use', enabled);
const result = resolveSkillSettings(fakeSettings({ merged: lists }));
expect(result.disabledNames.has('browser-use')).toBe(!enabled);
}
});
it('keeps a hard Browser Use disable authoritative over an explicit opt-in', () => {
const result = resolveSkillSettings(
fakeSettings({
merged: { disabled: ['browser-use'], enabled: [' BROWSER-USE '] },
user: { disabled: ['browser-use'] },
}),
);
expect(result.disablements.get('browser-use')).toEqual({
reason: 'hard',
lockedScope: 'user',
});
});
it('lets a workspace opt-in override a user default case-insensitively', () => {
const result = resolveSkillSettings(
fakeSettings({
packages/core/package.json
+1
View File
@@ -171,6 +171,7 @@
"@lydell/node-pty-win32-x64": "1.2.0-beta.10"
},
"devDependencies": {
"@qwen-code/browser-use": "file:../browser-use",
"@types/diff": "^7.0.2",
"@types/dotenv": "^6.1.1",
"@types/fast-levenshtein": "^0.0.4",
packages/core/src/skills/bundled/browser-use/SKILL.md
+136
View File
@@ -0,0 +1,136 @@
---
name: browser-use
description: Control the user's Chrome through the existing persistent Node REPL and the Qwen Browser SDK.
---
# Browser Use
Browser Use is bundled with Qwen Code and uses the generic Node REPL MCP tools
and the Qwen Chrome extension. Do not install a separate Qwen extension or
start a separate Browser Use MCP server.
## Setup
The Qwen Code Chrome extension must already be installed. On macOS and Linux,
the SDK checks for it before automatically registering the local Native
Messaging host. If it is not detected immediately, the SDK retries for up to
30 seconds so Chrome can finish saving a new installation. If setup still
cannot detect it, tell a user who just installed it to wait a few seconds and
retry; if it is not installed, tell them there is no store listing yet: build
the extension from `packages/chrome-extension` in the Qwen Code repository (its
README) and load the built `dist/extension` directory through
`chrome://extensions` (Developer mode → Load unpacked). Then stop. Do not run
the Native Host installer to bypass this check.
Installing the Chrome extension opts into this automatic local setup. Its
Native Host files persist after Qwen exits. The user can inspect or remove
them with `node <skill-base>/runtime/scripts/native-host-setup.js status` or
`uninstall`. Removing the Chrome extension also prevents automatic registration
on a later Browser Use initialization.
If `node_repl` is unavailable, configure it with:
```bash
qwen mcp add --scope user node-repl npx -y @qwen-code/[email protected]
```
Then tell the user to restart Qwen Code and stop. Do not start a separate
Browser Use MCP server. Screenshot metadata requires `@qwen-code/node-repl-mcp`
0.1.6 or later, so keep this exact pin.
Qwen reports the absolute `Base directory for this skill` when loading this
file. Use that directory as `<skill-base>`. Confirm that
`<skill-base>/runtime/index.js` and
`<skill-base>/runtime/node_modules/playwright-core/package.json` exist. If
either is missing, stop and report an incomplete Browser Use runtime instead
of installing dependencies into the workspace. Before the first Node REPL
cell, call `node_repl_add_node_module_dir` once with the absolute
`<skill-base>/runtime/node_modules` path. Import the bundled SDK, replacing the
example skill base below with that absolute path:
```js
globalThis.browserAgent ??= await (
await import('/absolute/skill/base/runtime/index.js')
).setupBrowserRuntime();
globalThis.browser ??= await browserAgent.browsers.get('chrome');
nodeRepl.write(await browser.documentation());
```
Create a tab or claim an exact result returned by `openTabs()`:
```js
globalThis.tab = await browser.tabs.new();
await tab.goto('https://example.com/');
nodeRepl.write(await tab.playwright.domSnapshot());
```
```js
const candidates = await browser.user.openTabs();
nodeRepl.write(candidates);
globalThis.tab = await browser.user.claimTab(
candidates.find((candidate) => candidate.url === 'https://example.com/'),
);
```
## Interaction
Observe before acting and observe again after acting. Prefer semantic
Playwright locators. Use `tab.dom_cua.get_visible_dom()` and its `node_id`
values when snapshot refs are clearer, and use `tab.cua` for visual coordinate
targets.
```js
await tab.playwright.getByRole('button', { name: 'Continue' }).click();
await tab.playwright.getByLabel('Email').type('[email protected]');
await tab.dom_cua.click({ node_id: 'e4' });
await tab.dom_cua.type({ text: 'hello' });
```
When an action should navigate, arm the Playwright watcher around it:
```js
await tab.playwright.expectNavigation(
() => tab.playwright.getByRole('link', { name: 'Next' }).click(),
{ url: '**/next.html' },
);
```
Render screenshots with their metadata:
```js
await nodeRepl.emitImage(await tab.screenshot());
```
The metadata reaches the model only through `@qwen-code/node-repl-mcp` 0.1.6
or later; an older server accepts the call and silently drops it. If
`node_repl` was registered before this skill existed, check its command with
`qwen mcp list`, and when it pins an older version, re-register it with the
`qwen mcp add` command from Setup above and ask the user to restart Qwen Code.
Use `tab.dev.logs()` for bounded console diagnostics. Arm
`tab.playwright.waitForEvent('download' | 'filechooser')` before the action that
triggers it. Handle JavaScript dialogs through `tab.getJsDialog()`. Use a
focused `tab.playwright.evaluate()` only when locators cannot obtain the needed
data.
Use `browser.user.history()` only when the task needs browser history, with a
focused query and bounded result count.
## Finish
Treat `browser.tabs.finalize({ keep })` as the final browser action of the turn.
Omitted agent-created tabs close; omitted claimed tabs are released without
closing. `keep` is the complete set for this call. Keep a user-facing result as
`deliverable`, and keep a live page under control as `handoff`. Repeat a
handoff in each later turn that still needs it; omitting it from the next
finalize call or closing the runtime closes an agent-created tab.
```js
await browser.tabs.finalize({
keep: [{ tab, status: 'deliverable' }],
});
```
The Node kernel preserves `globalThis` and top-level bindings across cells.
Keep cells short, await every Browser SDK promise, and use `nodeRepl.write()` to
return text or structured data.
packages/core/src/skills/bundled/browser-use/skill.test.ts
+116
View File
@@ -0,0 +1,116 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import fs from 'node:fs';
import { fileURLToPath } from 'node:url';
import { runInNewContext } from 'node:vm';
import { describe, expect, it, vi } from 'vitest';
import { makeFakeConfig } from '../../../test-utils/config.js';
import { SkillManager } from '../../skill-manager.js';
const skillUrl = new URL('./SKILL.md', import.meta.url);
const skill = fs.readFileSync(skillUrl, 'utf8');
const nodeReplPackage = JSON.parse(
fs.readFileSync(
new URL('../../../../../node-repl/package.json', import.meta.url),
'utf8',
),
) as { version: string };
describe('bundled browser-use skill', () => {
it('is discovered without a Qwen extension', async () => {
const manager = new SkillManager(makeFakeConfig());
await expect(
manager.loadSkill('browser-use', 'bundled'),
).resolves.toMatchObject({
name: 'browser-use',
level: 'bundled',
filePath: fileURLToPath(skillUrl),
});
});
it('loads its bundled runtime through the generic Node REPL', () => {
expect(skill).toContain('If `node_repl` is unavailable');
expect(skill).toContain('qwen mcp add --scope user node-repl');
expect(skill).toContain(
`@qwen-code/node-repl-mcp@${nodeReplPackage.version}`,
);
expect(skill).not.toContain('@qwen-code/node-repl-mcp@latest');
expect(skill).toContain('node_repl_add_node_module_dir');
expect(skill).toContain('<skill-base>/runtime/node_modules');
expect(skill).toContain('node_modules/playwright-core/package.json');
expect(skill).toContain("import('/absolute/skill/base/runtime/index.js')");
expect(skill).not.toContain('<extension-root>');
expect(skill).not.toContain('qwen extensions install');
expect(skill).not.toContain('npm install --no-save');
});
it('pins the Node REPL version that carries screenshot metadata', () => {
const pin = skill.match(
/^qwen mcp add --scope user node-repl npx -y @qwen-code\/node-repl-mcp@(\S+)$/m,
)?.[1];
expect(pin).toBe(nodeReplPackage.version);
expect(skill.replace(/\s+/g, ' ')).toContain(
`Screenshot metadata requires \`@qwen-code/node-repl-mcp\` ${nodeReplPackage.version} or later`,
);
});
it('names where the Qwen Chrome extension comes from', () => {
expect(skill).toContain('no store listing yet');
expect(skill).toContain('`packages/chrome-extension`');
expect(skill).toContain('`dist/extension`');
expect(skill).toContain('Load unpacked');
});
it('uses the current Browser SDK contract', () => {
expect(skill).toContain('setupBrowserRuntime()');
expect(skill).toContain('nodeRepl.write');
expect(skill).toContain('browser.tabs.finalize');
expect(skill).toContain('complete set');
expect(skill).toContain('handoff in each later turn');
expect(skill).toContain('node_id');
expect(skill).not.toContain('setupBrowserRuntime(nodeRepl)');
expect(skill).not.toContain('dev.network');
expect(skill).not.toContain('markDeliverable');
expect(skill).not.toContain('markHandoff');
expect(skill).not.toContain('domSnapshot({ filter:');
});
it('passes the complete screenshot and metadata to emitImage', async () => {
const example = [...skill.matchAll(/```js\n([\s\S]*?)```/g)].find(
([, code]) => code.includes('tab.screenshot()'),
)?.[1];
expect(example).toBeDefined();
const shot = {
bytes: new Uint8Array([1, 2, 3]),
mimeType: 'image/jpeg',
metadata: {
width: 1280,
height: 720,
viewport: { width: 1280, height: 720 },
devicePixelRatio: 2,
coordinateSpace: 'css-pixels',
},
};
const screenshot = vi.fn().mockResolvedValue(shot);
const output: unknown[] = [];
await runInNewContext(`(async () => {\n${example}\n})()`, {
tab: { screenshot },
nodeRepl: {
write: (text: string) => output.push({ text }),
emitImage: async (image: unknown) => {
output.push({ image });
},
},
});
expect(screenshot).toHaveBeenCalledTimes(1);
expect(output).toEqual([{ image: shot }]);
});
});
packages/core/src/skills/bundled/computer-use/SKILL.md
+1 -1
View File
@@ -46,7 +46,7 @@ whole result, so do not use `image(result)` either.
If `node_repl` is unavailable, run:
```bash
qwen mcp add --scope user node-repl npx -y @qwen-code/[email protected].5
qwen mcp add --scope user node-repl npx -y @qwen-code/[email protected].6
npm install --no-save --package-lock=false @qwen-code/[email protected]
```
packages/core/tsconfig.json
+1 -1
View File
@@ -10,5 +10,5 @@
}
},
"include": ["index.ts", "src/**/*.ts", "src/**/*.json"],
"exclude": ["node_modules", "dist"]
"exclude": ["node_modules", "dist", "src/skills/bundled/browser-use/runtime"]
}
packages/node-repl/README.md
+4 -1
View File
@@ -21,7 +21,10 @@ client (Qwen Code via `mcpServers`, Claude, Codex, etc.) can run it.
### Cell semantics
- Explicit output only: `nodeRepl.write(value)` for text, `nodeRepl.emitImage(png|jpeg|webp)`
for images; `console.*` is captured. Plain expression results are not returned.
for images; byte payloads may include JSON-serializable `metadata`, which is
returned immediately before each retained image, independently of the prose
output budget. Metadata for rejected or omitted images is also omitted.
`console.*` is captured. Plain expression results are not returned.
- `nodeRepl.cwd` / `homeDir` / `tmpDir` and `nodeRepl.getHeapStatus()` are available.
- Top-level static `import` is not allowed — use dynamic `await import()`.
- Declared bindings stay live across cells: after replacing an observation,
packages/node-repl/package.json
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@qwen-code/node-repl-mcp",
"version": "0.1.5",
"version": "0.1.6",
"description": "Standalone MCP server exposing a session-persistent Node.js REPL (node_repl) — independent of qwen-code core.",
"repository": {
"type": "git",
packages/node-repl/src/kernel-manager.ts
+24 -4
View File
@@ -54,6 +54,7 @@ export interface NodeReplImageEvent {
type: 'image';
data: string;
mimeType: string;
metadata?: string;
}
export type NodeReplOutputEvent = NodeReplTextEvent | NodeReplImageEvent;
@@ -705,12 +706,21 @@ export class NodeReplKernelManager {
typeof message.execId !== 'string' ||
typeof message.data !== 'string' ||
typeof message.mimeType !== 'string' ||
message.mimeType.length > MAX_IMAGE_MIME_CHARS
message.mimeType.length > MAX_IMAGE_MIME_CHARS ||
!(
message.metadata === undefined ||
typeof message.metadata === 'string'
)
) {
this.handleProtocolError(handle, new Error('invalid image frame'));
return;
}
this.collectImage(message.execId, message.data, message.mimeType);
this.collectImage(
message.execId,
message.data,
message.mimeType,
message.metadata,
);
return;
case 'execResult':
this.handleExecResult(handle, message as ExecResultMessage);
@@ -843,7 +853,12 @@ export class NodeReplKernelManager {
});
}
private collectImage(execId: string, data: string, mimeType: string): void {
private collectImage(
execId: string,
data: string,
mimeType: string,
metadata?: string,
): void {
const inflight = this.inflight;
if (!inflight || inflight.execId !== execId) {
if (inflight) inflight.droppedStaleFrames++;
@@ -858,7 +873,12 @@ export class NodeReplKernelManager {
}
inflight.imageCount++;
inflight.imageChars += data.length;
inflight.events.push({ type: 'image', data, mimeType });
inflight.events.push({
type: 'image',
data,
mimeType,
...(metadata === undefined ? {} : { metadata }),
});
}
/**
packages/node-repl/src/node-repl.semantics.test.ts
+63
View File
@@ -314,6 +314,69 @@ describe('binding semantics through the real kernel', () => {
expect(viaBytes.events.some((e) => e.type === 'image')).toBe(true);
});
it('carries structured image metadata across the kernel boundary', async () => {
const manager = makeManager();
const png =
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==';
const result = await manager.exec({
code: `await nodeRepl.emitImage({ bytes: Uint8Array.from(atob("${png}"), (c) => c.charCodeAt(0)), mimeType: "image/png", metadata: { width: 1, height: 1, coordinateSpace: "css-pixels" } });`,
timeoutMs: 30_000,
});
expect(result.status).toBe('ok');
expect(result.events.find((event) => event.type === 'image')).toMatchObject(
{
metadata: '{"width":1,"height":1,"coordinateSpace":"css-pixels"}',
},
);
});
it('preserves large metadata without a separate size cap', async () => {
const manager = makeManager();
const png =
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==';
const result = await manager.exec({
code: `await nodeRepl.emitImage({ bytes: Uint8Array.from(atob("${png}"), c => c.charCodeAt(0)), mimeType: "image/png", metadata: "界".repeat(20000) });`,
timeoutMs: 30_000,
});
expect(result.status).toBe('ok');
expect(result.events.find((event) => event.type === 'image')).toMatchObject(
{ metadata: JSON.stringify('界'.repeat(20000)) },
);
});
it.each([
['function', '() => {}', /metadata must be JSON-serializable/],
['bigint', '1n', /BigInt/],
[
'circular',
'(() => { const value = {}; value.self = value; return value; })()',
/circular/i,
],
])(
'rejects %s metadata without emitting its image',
async (_, value, error) => {
const manager = makeManager();
const png =
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==';
const result = await manager.exec({
code: `await nodeRepl.emitImage({ bytes: Uint8Array.from(atob("${png}"), c => c.charCodeAt(0)), mimeType: "image/png", metadata: ${value} });`,
timeoutMs: 30_000,
});
expect(result.status).toBe('error');
expect(result.error?.message).toMatch(error);
expect(result.events.some((event) => event.type === 'image')).toBe(false);
const next = await manager.exec({
code: 'nodeRepl.write("still usable");',
timeoutMs: 30_000,
});
expect(next.status).toBe('ok');
expect(textOf(next.events)).toBe('still usable');
},
);
it('caps live sandbox timers instead of letting one cell saturate the loop', async () => {
const manager = makeManager();
const r = await manager.exec({
packages/node-repl/src/output-adapter.test.ts
+114 -2
View File
@@ -7,6 +7,7 @@
import { describe, expect, it } from 'vitest';
import {
convertOutcomeToMcpResult,
MAX_MODEL_IMAGES,
MAX_MODEL_TEXT_TOKENS,
} from './output-adapter.js';
import {
@@ -98,15 +99,124 @@ describe('convertOutcomeToMcpResult', () => {
expect(image?.data).toBe(PNG_BASE64);
});
it('places image metadata immediately before the image', () => {
const result = convertOutcomeToMcpResult(
outcome({
status: 'ok',
events: [
{
type: 'image',
data: PNG_BASE64,
mimeType: 'image/png',
metadata: '{"width":1,"height":1,"coordinateSpace":"css-pixels"}',
},
],
}),
);
expect(result.content).toEqual([
{
type: 'text',
text: '[image metadata] {"width":1,"height":1,"coordinateSpace":"css-pixels"}\n',
},
{ type: 'image', data: PNG_BASE64, mimeType: 'image/png' },
]);
});
it.each(['before', 'after'])(
'preserves image metadata with oversized prose %s the image',
(position) => {
const metadata = JSON.stringify({ label: '界'.repeat(20000) });
const text = {
type: 'text' as const,
kind: 'write' as const,
text: 'x'.repeat(500_000),
};
const image = {
type: 'image' as const,
data: PNG_BASE64,
mimeType: 'image/png',
metadata,
};
const result = convertOutcomeToMcpResult(
outcome({
status: 'ok',
events: position === 'before' ? [text, image] : [image, text],
}),
);
const imageIndex = result.content.findIndex((b) => b.type === 'image');
expect(imageIndex).toBeGreaterThan(0);
expect(result.content[imageIndex - 1]).toEqual({
type: 'text',
text: `[image metadata] ${metadata}\n`,
});
expect(textOf(result)).toContain('truncated');
const imageWithoutMetadata = {
type: 'image' as const,
data: PNG_BASE64,
mimeType: 'image/png',
};
const withoutMetadata = convertOutcomeToMcpResult(
outcome({
status: 'ok',
events:
position === 'before'
? [text, imageWithoutMetadata]
: [imageWithoutMetadata, text],
}),
);
expect(textOf(result).replace(`[image metadata] ${metadata}\n`, '')).toBe(
textOf(withoutMetadata),
);
},
);
it('keeps metadata paired only with images inside the output count limit', () => {
const result = convertOutcomeToMcpResult(
outcome({
status: 'ok',
events: Array.from({ length: MAX_MODEL_IMAGES + 1 }, (_, index) => ({
type: 'image' as const,
data: PNG_BASE64,
mimeType: 'image/png',
metadata: JSON.stringify({ index }),
})),
}),
);
for (let index = 0; index < MAX_MODEL_IMAGES; index++) {
expect(result.content[index * 2]).toEqual({
type: 'text',
text: `[image metadata] ${JSON.stringify({ index })}\n`,
});
expect(result.content[index * 2 + 1]).toEqual({
type: 'image',
data: PNG_BASE64,
mimeType: 'image/png',
});
}
expect(textOf(result)).not.toContain(
JSON.stringify({ index: MAX_MODEL_IMAGES }),
);
expect(textOf(result)).toContain('1 image(s) omitted');
});
it('rejects an image whose bytes do not match the declared MIME', () => {
const result = convertOutcomeToMcpResult(
outcome({
status: 'ok',
events: [{ type: 'image', data: PNG_BASE64, mimeType: 'image/jpeg' }],
events: [
{
type: 'image',
data: PNG_BASE64,
mimeType: 'image/jpeg',
metadata: '{"rejected":true}',
},
],
}),
);
expect(result.content.some((b) => b.type === 'image')).toBe(false);
expect(textOf(result)).toContain('image rejected');
expect(textOf(result)).not.toContain('rejected":true');
});
it('folds non-ok status into isError with a leading status note', () => {
@@ -187,15 +297,17 @@ describe('convertOutcomeToMcpResult', () => {
type: 'image',
data: big.toString('base64'),
mimeType: 'image/png',
metadata: '{"tooLarge":true}',
},
],
}),
);
expect(result.content.some((b) => b.type === 'image')).toBe(false);
expect(textOf(result)).toMatch(/exceeds/);
expect(textOf(result)).not.toContain('tooLarge');
});
it('keeps the whole result within the token budget when every notice fires', () => {
it('keeps prose within the token budget when every notice fires', () => {
const huge = 'x'.repeat(400_000);
const events = [
{ type: 'text' as const, kind: 'write' as const, text: huge },
packages/node-repl/src/output-adapter.ts
+9 -2
View File
@@ -189,8 +189,9 @@ function pushText(blocks: OutputBlock[], text: string): void {
/**
* Converts a kernel execution outcome into an MCP CallToolResult:
* text + image content blocks, with a ~10k-token text budget, base64/MIME image
* validation, and status folded into `isError` plus a leading status text block
* text + image content blocks, with a ~10k-token prose budget, image metadata
* outside that budget, and base64/MIME image validation. Status is folded
* into `isError` plus a leading status text block
* (MCP has no first-class error-type or display/model split).
*/
export function convertOutcomeToMcpResult(
@@ -301,6 +302,12 @@ export function convertOutcomeToMcpResult(
}
validImages++;
imageBytesEmitted += verdict.byteLength;
if (event.metadata !== undefined) {
blocks.push({
type: 'text',
text: `[image metadata] ${event.metadata}\n`,
});
}
blocks.push({ type: 'image', data: event.data, mimeType: event.mimeType });
}
packages/node-repl/src/protocol.ts
+1
View File
@@ -85,6 +85,7 @@ export interface ImageMessage {
execId: string | null;
data: string;
mimeType: string;
metadata?: string;
}
export interface ExecResultMessage {
packages/node-repl/src/runtime/kernel.mjs
+16 -1
View File
@@ -392,7 +392,13 @@ function resolveImagePayload(payload) {
);
}
}
return { data: buffer.toString('base64'), mimeType: sniffed };
return {
data: buffer.toString('base64'),
mimeType: sniffed,
...(typeof payload.metadata === 'string'
? { metadata: payload.metadata }
: {}),
};
}
function scheduleTimer(callback, delay, repeat) {
@@ -567,6 +573,7 @@ const intrinsicObjectGetOwnPropertyDescriptor = Object.getOwnPropertyDescriptor;
const intrinsicObjectDefineProperties = Object.defineProperties;
const intrinsicArrayBufferIsView = ArrayBuffer.isView;
const intrinsicJSONParse = JSON.parse;
const intrinsicJSONStringify = JSON.stringify;
const intrinsicReflectApply = Reflect.apply;
const intrinsicWeakSetHas = WeakSet.prototype.has;
const intrinsicWeakSetAdd = WeakSet.prototype.add;
@@ -620,12 +627,20 @@ const runtime = {
};
} else if (image && typeof image === 'object' && 'bytes' in image) {
const bytes = image.bytes;
let metadata = null;
if ('metadata' in image && image.metadata !== undefined) {
metadata = intrinsicJSONStringify(image.metadata);
if (typeof metadata !== 'string') {
throw new TypeError('emitImage metadata must be JSON-serializable');
}
}
payload = {
kind: 'bytes',
bytes: intrinsicArrayBufferIsView(bytes)
? bytes
: new intrinsicUint8Array(bytes),
mimeType: typeof image.mimeType === 'string' ? image.mimeType : null,
metadata,
};
} else {
throw new TypeError('unsupported emitImage input');
pnpm-lock.yaml
Generated
+3
View File
@@ -986,6 +986,9 @@ importers:
specifier: ^3.25.76
version: 3.25.76
devDependencies:
'@qwen-code/browser-use':
specifier: workspace:*
version: link:../browser-use
'@types/diff':
specifier: ^7.0.2
version: 7.0.2
scripts/build.js
+2
View File
@@ -38,6 +38,7 @@ execSync('npm run generate', { stdio: 'inherit', cwd: root });
const cliOnly = process.argv.includes('--cli-only');
// Build in dependency order:
// 0. browser-use (built first so core can stage the builtin skill's runtime)
// 1. core (foundation package, includes test-utils)
// 2. channel-base (base channel infrastructure - used by channel adapters and cli)
// 3. channel adapters (depend on channel-base)
@@ -50,6 +51,7 @@ const cliOnly = process.argv.includes('--cli-only');
// 10. vscode-ide-companion
// 11. external-context integrations (private Qwen extensions)
const buildOrder = [
'packages/browser-use',
'packages/core',
'packages/channels/base',
'packages/channels/telegram',
scripts/copy-browser-use-assets.js
+83
View File
@@ -0,0 +1,83 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import fs from 'node:fs';
import { createRequire } from 'node:module';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
/**
* Copy the built Browser Use runtime (SDK bundle, Native Host, its setup
* script, NOTICE and the pinned playwright-core) into `skillDir/runtime`.
*
* Two callers: `copy_bundle_assets.js` stages it into `dist/bundled` for the
* shipped CLI, and `scripts/dev.js` (or running this file directly) stages it
* next to the source SKILL.md for `npm run dev`. No build step writes into the
* source tree; the dev copy is git-ignored.
*/
export function copyBrowserUseAssets(root, skillDir) {
const browserUseDir = path.join(root, 'packages', 'browser-use');
const runtimeFiles = [
'index.js',
'native-host.js',
'scripts/native-host-setup.js',
];
for (const file of runtimeFiles) {
const source = path.join(browserUseDir, 'dist', file);
if (!fs.existsSync(source)) {
throw new Error(
`Browser-use runtime not found: ${source}. ` +
'Run "npm run build --workspace=@qwen-code/browser-use" first.',
);
}
}
const manifestPath = path.join(browserUseDir, 'package.json');
const manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf8'));
const require = createRequire(manifestPath);
const playwrightManifestPath = require.resolve(
'playwright-core/package.json',
);
const playwrightManifest = JSON.parse(
fs.readFileSync(playwrightManifestPath, 'utf8'),
);
const pinnedVersion = manifest.dependencies['playwright-core'];
if (playwrightManifest.version !== pinnedVersion) {
throw new Error(
`Browser-use requires playwright-core ${pinnedVersion}, ` +
`but resolved ${playwrightManifest.version}. Run "npm install" first.`,
);
}
const runtimeDir = path.join(skillDir, 'runtime');
fs.rmSync(runtimeDir, { recursive: true, force: true });
for (const file of runtimeFiles) {
const target = path.join(runtimeDir, file);
fs.mkdirSync(path.dirname(target), { recursive: true });
fs.copyFileSync(path.join(browserUseDir, 'dist', file), target);
}
fs.copyFileSync(
path.join(browserUseDir, 'NOTICE'),
path.join(runtimeDir, 'NOTICE'),
);
fs.cpSync(
path.dirname(playwrightManifestPath),
path.join(runtimeDir, 'node_modules', 'playwright-core'),
{ recursive: true },
);
}
if (
process.argv[1] &&
fs.existsSync(process.argv[1]) &&
fs.realpathSync(process.argv[1]) === fileURLToPath(import.meta.url)
) {
const root = fileURLToPath(new URL('..', import.meta.url));
copyBrowserUseAssets(
root,
path.join(root, 'packages/core/src/skills/bundled/browser-use'),
);
}
scripts/copy_bundle_assets.js
+10 -4
View File
@@ -39,6 +39,7 @@ import { createRequire } from 'node:module';
import { fileURLToPath } from 'node:url';
import { glob } from 'glob';
import fs from 'node:fs';
import { copyBrowserUseAssets } from './copy-browser-use-assets.js';
const __dirname = dirname(fileURLToPath(import.meta.url));
const defaultRoot = join(__dirname, '..');
@@ -516,9 +517,14 @@ export function copyBundleAssets({ root = defaultRoot } = {}) {
// DESIGN.md files are maintainer design narratives, not runtime inputs;
// shipping one would hand a review a ~125 KB read_file target that
// outweighs the context the slimmed skill saves.
skipEntry: (entry) =>
isBundledSkillTestFile(entry) || entry === 'DESIGN.md',
skipEntry: (entry, sourcePath) =>
isBundledSkillTestFile(entry) ||
entry === 'DESIGN.md' ||
sourcePath === join(bundledSkillsDir, 'browser-use', 'runtime'),
});
if (existsSync(join(bundledSkillsDir, 'browser-use', 'SKILL.md'))) {
copyBrowserUseAssets(root, join(destBundledDir, 'browser-use'));
}
console.log('Copied bundled skills to dist/bundled/');
} else {
console.warn(
@@ -712,11 +718,11 @@ function copyRecursiveSync(src, dest, options = {}) {
const entries = fs.readdirSync(src);
for (const entry of entries) {
if (entry === '.DS_Store' || options.skipEntry?.(entry)) {
const srcPath = join(src, entry);
if (entry === '.DS_Store' || options.skipEntry?.(entry, srcPath)) {
continue;
}
const srcPath = join(src, entry);
const destPath = join(dest, entry);
copyRecursiveSync(srcPath, destPath, options);
}
scripts/copy_files.js
+14 -3
View File
@@ -23,6 +23,7 @@
import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { copyBrowserUseAssets } from './copy-browser-use-assets.js';
const extensionsToCopy = ['.md', '.json', '.sb'];
@@ -46,15 +47,16 @@ function copyFilesRecursive(source, target, rootSourceDir) {
for (const item of items) {
const sourcePath = path.join(source, item.name);
const targetPath = path.join(target, item.name);
const relativePath = path.relative(rootSourceDir, sourcePath);
const normalizedPath = relativePath.replace(/\\/g, '/');
if (item.isDirectory()) {
if (normalizedPath === 'skills/bundled/browser-use/runtime') continue;
copyFilesRecursive(sourcePath, targetPath, rootSourceDir);
} else {
const ext = path.extname(item.name);
// Copy standard extensions, or .js files in i18n/locales directory
// Use path.relative for precise matching to avoid false positives
const relativePath = path.relative(rootSourceDir, sourcePath);
const normalizedPath = relativePath.replace(/\\/g, '/');
const isLocaleJs =
ext === '.js' && normalizedPath.startsWith('i18n/locales/');
if (
@@ -78,8 +80,17 @@ export function copyFiles({ root = process.cwd() } = {}) {
copyFilesRecursive(sourceDir, targetDir, sourceDir);
// Copy example extensions into the bundle.
const packageName = path.basename(root);
if (
packageName === 'core' &&
fs.existsSync(path.join(sourceDir, 'skills/bundled/browser-use/SKILL.md'))
) {
copyBrowserUseAssets(
path.resolve(root, '../..'),
path.join(targetDir, 'skills/bundled/browser-use'),
);
}
// Copy example extensions into the bundle.
if (packageName === 'cli') {
const examplesSource = path.join(
sourceDir,
scripts/dev.js
+27 -1
View File
@@ -8,7 +8,7 @@
/**
* Development entry point for Qwen Code CLI.
*
* Runs the CLI directly from TypeScript source files without requiring a build step.
* Runs the CLI from TypeScript source with prebuilt workspace dependencies.
* Changes to packages/core or packages/cli are reflected immediately.
*
* Usage: npm run dev -- [args]
@@ -28,6 +28,7 @@ import {
readFileSync,
} from 'node:fs';
import { tmpdir, platform } from 'node:os';
import { copyBrowserUseAssets } from './copy-browser-use-assets.js';
const __dirname = dirname(fileURLToPath(import.meta.url));
const root = join(__dirname, '..');
@@ -58,6 +59,31 @@ if (existsSync(userDocsTarget) && !existsSync(qcHelperDocsLink)) {
}
}
// Stage the Browser Use skill runtime next to its SKILL.md for dev mode. The
// bundle step does the same into dist/bundled; only dev mode reads the skill
// from the source tree, so this copy exists nowhere else (it is git-ignored,
// and no build step writes it). It needs the built browser-use package. The
// informational passthrough flags never reach a skill, so they skip the copy
// and stay a fast path.
const passthroughArgs = process.argv.slice(2);
const informationalOnly = passthroughArgs.some((argument) =>
['--version', '-v', '--help', '-h'].includes(argument),
);
if (!informationalOnly) {
try {
copyBrowserUseAssets(
root,
join(root, 'packages', 'core', 'src', 'skills', 'bundled', 'browser-use'),
);
} catch (error) {
console.warn(
`Browser Use skill runtime not staged for dev mode: ${
error instanceof Error ? error.message : String(error)
}`,
);
}
}
// Entry point for the CLI
const cliEntry = join(cliPackageDir, 'index.ts');
scripts/tests/browser-use-assets.test.js
+282
View File
@@ -0,0 +1,282 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import fs from 'node:fs';
import { spawnSync } from 'node:child_process';
import { tmpdir } from 'node:os';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { copyBrowserUseAssets } from '../copy-browser-use-assets.js';
import { copyBundleAssets } from '../copy_bundle_assets.js';
import { copyFiles } from '../copy_files.js';
import { getWorkflowJob } from './workflow-helpers.js';
const skillPath = 'packages/core/src/skills/bundled/browser-use';
const runtimeFiles = [
'index.js',
'native-host.js',
'scripts/native-host-setup.js',
];
const playwrightPath = 'packages/browser-use/node_modules/playwright-core';
describe('browser-use builtin resources', () => {
let root;
beforeEach(() => {
root = fs.mkdtempSync(path.join(tmpdir(), 'qwen-browser-use-assets-'));
vi.spyOn(console, 'log').mockImplementation(() => {});
vi.spyOn(console, 'warn').mockImplementation(() => {});
write(`${skillPath}/SKILL.md`, '# Browser use\n');
write(
'packages/browser-use/package.json',
JSON.stringify({
name: '@qwen-code/browser-use',
dependencies: { 'playwright-core': '1.2.3' },
}),
);
write('packages/browser-use/NOTICE', 'Browser use notice\n');
for (const file of runtimeFiles) {
write(`packages/browser-use/dist/${file}`, `// ${file}\n`);
}
write('packages/browser-use/dist/index.d.ts', 'export {};\n');
write(
'packages/browser-use/dist/scripts/smoke.js',
'// development only\n',
);
write(
`${playwrightPath}/package.json`,
JSON.stringify({
name: 'playwright-core',
version: '1.2.3',
}),
);
write(`${playwrightPath}/lib/server/injected.js`, '// runtime asset\n');
write(`${playwrightPath}/LICENSE`, 'Apache-2.0\n');
write(`${playwrightPath}/NOTICE`, 'Playwright notice\n');
write(`${playwrightPath}/types/types.d.ts`, 'export {};\n');
write(`${playwrightPath}/bin/install.sh`, '#!/bin/sh\n');
});
afterEach(() => {
vi.restoreAllMocks();
fs.rmSync(root, { force: true, recursive: true });
});
it.each([
['dev', skillPath],
['core', 'packages/core/dist/src/skills/bundled/browser-use'],
['bundle', 'dist/bundled/browser-use'],
])(
'stages the complete runtime for %s and removes stale generated files',
(mode, target) => {
write(`${target}/runtime/stale.json`, '{}\n');
if (mode === 'dev') {
copyBrowserUseAssets(root, path.join(root, target));
} else {
write(`${skillPath}/runtime/source-only.json`, '{}\n');
const readdirSync = fs.readdirSync;
vi.spyOn(fs, 'readdirSync').mockImplementation((directory, ...args) => {
if (directory === path.join(root, skillPath, 'runtime')) {
throw new Error(
'General asset copying must not walk generated runtime',
);
}
return readdirSync(directory, ...args);
});
if (mode === 'core')
copyFiles({ root: path.join(root, 'packages/core') });
else copyBundleAssets({ root });
}
const runtimeDir = path.join(root, target, 'runtime');
expect(fs.readdirSync(runtimeDir).sort()).toEqual([
'NOTICE',
'index.js',
'native-host.js',
'node_modules',
'scripts',
]);
expect(fs.readdirSync(path.join(runtimeDir, 'scripts'))).toEqual([
'native-host-setup.js',
]);
for (const file of runtimeFiles) {
expect(fs.readFileSync(path.join(runtimeDir, file), 'utf8')).toBe(
`// ${file}\n`,
);
}
for (const file of [
'package.json',
'lib/server/injected.js',
'LICENSE',
'NOTICE',
'types/types.d.ts',
'bin/install.sh',
]) {
expect(
fs.readFileSync(
path.join(runtimeDir, 'node_modules/playwright-core', file),
),
).toEqual(fs.readFileSync(path.join(root, playwrightPath, file)));
}
expect(fs.readFileSync(path.join(runtimeDir, 'NOTICE'), 'utf8')).toBe(
'Browser use notice\n',
);
},
);
it.each(runtimeFiles)(
'fails with build advice when %s is missing',
(file) => {
fs.rmSync(path.join(root, 'packages/browser-use/dist', file));
expect(() =>
copyBrowserUseAssets(root, path.join(root, skillPath)),
).toThrow('npm run build --workspace=@qwen-code/browser-use');
},
);
it('stages the dev runtime when the copy script runs from another directory', () => {
const scriptPath = path.join(root, 'scripts/copy-browser-use-assets.js');
fs.mkdirSync(path.dirname(scriptPath), { recursive: true });
fs.copyFileSync(
new URL('../copy-browser-use-assets.js', import.meta.url),
scriptPath,
);
write('package.json', '{"type":"module"}\n');
const result = spawnSync(process.execPath, [scriptPath], {
cwd: tmpdir(),
encoding: 'utf8',
});
expect(result.status, result.stderr).toBe(0);
for (const file of runtimeFiles) {
expect(
fs.readFileSync(path.join(root, skillPath, 'runtime', file), 'utf8'),
).toBe(`// ${file}\n`);
}
expect(
fs.existsSync(
path.join(
root,
skillPath,
'runtime/node_modules/playwright-core/LICENSE',
),
),
).toBe(true);
});
it('uses the SDK-local pinned Playwright instead of the hoisted version', () => {
write(
'node_modules/playwright-core/package.json',
JSON.stringify({
name: 'playwright-core',
version: '9.9.9',
}),
);
copyBrowserUseAssets(root, path.join(root, skillPath));
const copied = JSON.parse(
fs.readFileSync(
path.join(
root,
skillPath,
'runtime/node_modules/playwright-core/package.json',
),
'utf8',
),
);
expect(copied.version).toBe('1.2.3');
});
it('rejects a resolved Playwright version different from the SDK pin', () => {
write(
`${playwrightPath}/package.json`,
JSON.stringify({
name: 'playwright-core',
version: '9.9.9',
}),
);
expect(() =>
copyBrowserUseAssets(root, path.join(root, skillPath)),
).toThrow('Browser-use requires playwright-core 1.2.3, but resolved 9.9.9');
});
it('does not silently omit resources when the builtin skill exists', () => {
fs.rmSync(path.join(root, 'packages/browser-use/dist/index.js'));
expect(() => copyBundleAssets({ root })).toThrow(
'Browser-use runtime not found',
);
expect(() => copyFiles({ root: path.join(root, 'packages/core') })).toThrow(
'Browser-use runtime not found',
);
});
it('builds browser-use before core in the root build', () => {
const script = fs.readFileSync(
fileURLToPath(new URL('../build.js', import.meta.url)),
'utf8',
);
const browserUseIndex = script.indexOf("'packages/browser-use'");
expect(browserUseIndex).toBeGreaterThan(0);
expect(browserUseIndex).toBeLessThan(script.indexOf("'packages/core'"));
});
it('builds browser-use before core when publishing Live Host', () => {
const publish = getWorkflowJob(
fs.readFileSync(
new URL(
'../../.github/workflows/live-host-release.yml',
import.meta.url,
),
'utf8',
),
'publish',
);
const browserUseIndex = publish.indexOf(
'npm run build --workspace @qwen-code/browser-use',
);
expect(browserUseIndex).toBeGreaterThanOrEqual(0);
expect(browserUseIndex).toBeLessThan(
publish.indexOf('npm run build --workspace @qwen-code/qwen-code-core'),
);
});
it('keeps the pinned Playwright installable as a workspace dependency', () => {
const manifest = JSON.parse(
fs.readFileSync(
new URL('../../packages/browser-use/package.json', import.meta.url),
'utf8',
),
);
const lock = JSON.parse(
fs.readFileSync(
new URL('../../package-lock.json', import.meta.url),
'utf8',
),
);
const workspace = lock.packages['packages/browser-use'];
const playwright =
lock.packages['packages/browser-use/node_modules/playwright-core'];
expect(manifest.dependencies['playwright-core']).toMatch(/^\d+\.\d+\.\d+$/);
expect(manifest.bundledDependencies).toBeUndefined();
expect(manifest.bundleDependencies).toBeUndefined();
expect(workspace.bundleDependencies).toBeUndefined();
expect(playwright.inBundle).not.toBe(true);
expect(playwright.version).toBe(manifest.dependencies['playwright-core']);
});
function write(relativePath, contents) {
const file = path.join(root, relativePath);
fs.mkdirSync(path.dirname(file), { recursive: true });
fs.writeFileSync(file, contents);
}
});
scripts/tests/dev.test.js
+23
View File
@@ -11,20 +11,29 @@ import { fileURLToPath } from 'node:url';
const {
spawnMock,
execSyncMock,
platformMock,
existsSyncMock,
readFileSyncMock,
writeFileSyncMock,
copyBrowserUseAssetsMock,
} = vi.hoisted(() => ({
spawnMock: vi.fn(() => ({ on: vi.fn() })),
execSyncMock: vi.fn(),
platformMock: vi.fn(() => 'darwin'),
existsSyncMock: vi.fn(() => false),
readFileSyncMock: vi.fn(() => JSON.stringify({ version: '0.0.0-test' })),
writeFileSyncMock: vi.fn(),
copyBrowserUseAssetsMock: vi.fn(),
}));
vi.mock('../copy-browser-use-assets.js', () => ({
copyBrowserUseAssets: copyBrowserUseAssetsMock,
}));
vi.mock('node:child_process', () => ({
spawn: spawnMock,
execSync: execSyncMock,
}));
vi.mock('node:os', async (importOriginal) => {
@@ -93,6 +102,20 @@ describe('scripts/dev.js launcher', () => {
expect(options).toEqual(expect.objectContaining({ shell: false }));
});
it.each(['--version', '--help'])(
'launches %s without building or staging Browser Use',
async (flag) => {
process.argv = ['node', 'scripts/dev.js', flag];
await import('../dev.js?browser-use');
expect(execSyncMock).not.toHaveBeenCalled();
expect(copyBrowserUseAssetsMock).not.toHaveBeenCalled();
expect(spawnMock).toHaveBeenCalledOnce();
expect(spawnMock.mock.calls[0][1]).toContain(flag);
},
);
it('keeps shell fallback for Windows tsx.cmd resolution', async () => {
platformMock.mockReturnValue('win32');
existsSyncMock.mockImplementation((filePath) =>
scripts/tests/release-workflow.test.js
+1 -1
View File
@@ -94,7 +94,7 @@ const liveHostOssWorkflow = readFileSync(
describe('CUA release workflow', () => {
it('keeps the Node REPL package independently versioned', () => {
expect(nodeReplPackage.name).toBe('@qwen-code/node-repl-mcp');
expect(nodeReplPackage.version).toBe('0.1.5');
expect(nodeReplPackage.version).toBe('0.1.6');
expect(cuaReleaseWorkflow).toContain(
"node_repl_version: '${{ steps.release.outputs.node_repl_version }}'",
);