QwenLM/qwen-code
1
0 Fork 0
mirror of https://github.com/QwenLM/qwen-code.git synced 2026-11-07 06:19:43 -10:00
Code Issues Packages Projects Releases 1 Wiki Activity
Files
main
qwen-code/packages/cua-driver/README.md
T
顾盼 4d050906e5 feat(cua): enable default cursor and live macOS PiP (#15669) 
* feat(cua): enable native cursor and live macOS previews

* fix(cua): preserve focus in standalone UI hosts

* feat(cua): polish floating preview cards

* fix(cua): stabilize PiP hover and modal interactions

* test(node-repl): use a lint-compatible status assertion

* fix(cua): retain the preview while a newer frame is pending

* fix(cua): use an auxiliary fullscreen role for the preview

* fix(cua): preserve worker lifetime and preview controls
2026-10-29 17:37:27 +08:00

302 lines
13 KiB
Markdown
Raw Permalink Blame History

# Qwen CUA Driver
Qwen Code's vendored distribution of the cross-platform Cua Driver runtime.
It provides native desktop and browser automation over MCP, a one-shot CLI,
and in-process Python and TypeScript SDKs.
This tree is based on upstream `cua-driver-rs-v0.20.0`. The upstream snapshot
is recorded in [`.vendored-from`](.vendored-from); Qwen-owned differences are
documented in [`.vendored-patches.md`](.vendored-patches.md) and
[`docs/relative-coordinates-design.md`](docs/relative-coordinates-design.md).
## Install
The commands below target release **0.27.0** and require its GitHub Release
(`cua-driver-rs-v0.27.0`) to be published. The native driver does not require
Node.js or npm. Install the MCP client you want to use separately.
macOS and Linux:
```bash
CUA_DRIVER_RS_VERSION=0.27.0 \
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/QwenLM/qwen-code/main/packages/cua-driver/scripts/install.sh)"
```
Windows PowerShell:
```powershell
$env:CUA_DRIVER_RS_VERSION = "0.27.0"
irm https://raw.githubusercontent.com/QwenLM/qwen-code/main/packages/cua-driver/scripts/install.ps1 | iex
```
The default executable is `~/.local/bin/qwen-cua-driver` on macOS/Linux and
`%LOCALAPPDATA%\Programs\Qwen\qwen-cua-driver\bin\qwen-cua-driver.exe` on
Windows. Check the installation:
```bash
# macOS / Linux
"$HOME/.local/bin/qwen-cua-driver" --version
```
```powershell
# Windows PowerShell
$driver = Join-Path $env:LOCALAPPDATA "Programs\Qwen\qwen-cua-driver\bin\qwen-cua-driver.exe"
& $driver --version
```
Expected: qwen-cua-driver 0.27.0.
If you chose a custom install directory, use its absolute executable path in
the client commands below.
The released product uses Qwen-owned identities throughout:
- executable: `qwen-cua-driver` / `qwen-cua-driver.exe`
- macOS app: `/Applications/QwenCuaDriver.app`
- bundle identifier: `com.qwencode.cua-driver`
- state home: `~/.cua-driver`
- Windows task: `qwen-cua-driver-serve`
Source builds install as the separate `qwen-cua-driver-local` product. The
Qwen executable, app, service, and local-build identities are distinct from
upstream CuaDriver. The release state home remains `~/.cua-driver` for
compatibility with existing Qwen releases, so installing both release
distributions for the same user is not supported: they can share or overwrite
that state directory.
## Agent integration
Choose an interface for your agent:
| Interface | Install and configure |
| ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Direct native MCP tools | Install this driver, then follow the client instructions below. |
| Persistent JavaScript with preloaded `cua` (Node REPL + SDK) | Follow the [Node REPL README](../node-repl/README.md#usage). It installs the matching SDK dependencies automatically and does not require this standalone driver installation. |
Both interfaces work with Qwen Code, Claude Code, and Codex over stdio MCP.
Claude Code and Codex can use them independently of Qwen Code. The version
`0.27.0` here belongs to the CUA components; it does not require a Qwen Code
CLI or Web Shell release with that version.
### Qwen Code
On macOS/Linux, register the installed executable at user scope:
```bash
qwen mcp add --scope user cua-driver "$HOME/.local/bin/qwen-cua-driver" mcp
```
This writes `mcpServers.cua-driver` in `~/.qwen/settings.json`. Use
`--scope project` for the current project's `.qwen/settings.json` instead.
### Claude Code
On macOS/Linux:
```bash
claude mcp add --scope user --transport stdio cua-driver -- "$HOME/.local/bin/qwen-cua-driver" mcp
```
User scope is stored in `~/.claude.json`. Use `--scope project` to write the
current project's `.mcp.json` instead. See the
[Claude Code MCP documentation](https://code.claude.com/docs/en/mcp).
### Codex
On macOS/Linux:
```bash
codex mcp add cua-driver -- "$HOME/.local/bin/qwen-cua-driver" mcp
```
Alternatively, add this table to `~/.codex/config.toml`, replacing the path
with your actual installed executable. TOML command strings are literal paths;
do not use `$HOME` or `~` in place of the absolute path.
```toml
[mcp_servers.cua-driver]
command = "/absolute/path/to/.local/bin/qwen-cua-driver"
args = ["mcp"]
```
See the [official OpenAI MCP documentation](https://learn.chatgpt.com/docs/extend/mcp).
### Windows client commands
In PowerShell, use the installed executable's absolute path. Run the command
for your chosen client:
```powershell
$driver = Join-Path $env:LOCALAPPDATA "Programs\Qwen\qwen-cua-driver\bin\qwen-cua-driver.exe"
# Qwen Code
qwen mcp add --scope user cua-driver $driver mcp
# Claude Code
claude mcp add --scope user --transport stdio cua-driver -- $driver mcp
# Codex
codex mcp add cua-driver -- $driver mcp
```
### Permissions and connection check
Restart the client after registration. Check its server list with
`qwen mcp list`, `claude mcp list`, or `codex mcp list`, and use `/mcp` inside
the session to inspect the connection and available tools. The default native
MCP interface exposes 19 Computer Use tools; advanced tools require the
explicit `CUA_DRIVER_EXTRA_TOOLS=1` opt-in.
On macOS, grant Accessibility and Screen Recording to
`/Applications/QwenCuaDriver.app`; normal `mcp` uses its app-owned daemon.
On Windows, run the installer in the logged-in desktop user's session and
allow its UAC prompt. On Linux, use an X11 desktop (including X11 apps under
XWayland); native Wayland apps are unsupported. Launch the client with that
desktop session's environment. Codex needs the matching environment allowlist
shown in [Linux desktop sessions](../node-repl/README.md#linux-desktop-sessions),
applied to its `mcp_servers.cua-driver` table.
Clients start `qwen-cua-driver mcp` themselves; a separately started MCP
process is unnecessary. Shell-oriented automation can instead call tools
through `qwen-cua-driver call`.
The native agent cursor is enabled by default. On macOS, a local PiP also
updates the two most recently observed, authorized windows in each session,
including while the agent is idle. Drag a card or its Move preview handle to move the borderless window.
Hover to reveal its floating controls, or close it to stop that preview; a later authorized observation can recreate it. It does not forward input to the target app. Use
`--no-overlay` and `--no-pip` when starting the owning host to disable them;
`experimental_pip: false` in `~/.cua-driver/config.json` also disables PiP.
Restart the owning host after changing flags or editing the file, preserving its other keys.
For the installed app's default MCP proxy, restart the app-owned daemon.
Restarting only the MCP client or proxy does not reconfigure an existing daemon.
CLI flags take precedence: remove an explicit `--pip` or `--experimental-pip`
if the file should control the setting. Disable PiP and restart its owning
host before sharing or recording the desktop if its mirrored windows should
not be visible. Closing the panel alone is temporary; screen sharing is not
detected automatically.
Windows and X11 retain cursor support; PiP is macOS-only.
## 0.20 runtime and SDK surface
The 0.20 base includes the SDK-owned runtime and versioned C ABI, generated
Python and TypeScript UniFFI bindings, typed browser automation, permission
modes, runtime-owned consent adapters, transport-owned implicit lifecycle
sessions, per-action target selection, capability manifests across permission
profiles, snapshot-bound element tokens, closed action results, `verify_state`,
foreground-focus verification, native menu and clipboard operations, window
framing, and semantic cursor themes. Browser approval tokens are retired;
existing-profile access now requires a trusted launch grant, bounded manifest,
or embedding-host authorization.
Python applications import `cua_driver`. TypeScript applications import the
Qwen-owned `@qwen-code/cua-sdk` package. Both use the same
in-process native runtime; MCP remains the agent-facing boundary implemented by
`qwen-cua-driver`.
The stable C ABI is declared in
[`rust/include/cua_driver_abi.h`](rust/include/cua_driver_abi.h). The contract
and generated bindings are documented in [`contract/README.md`](contract/README.md).
## Permission modes
`standard` is the promptless default for ordinary automation. `bounded`
admits only reviewed tools and resources. `unrestricted` requires
`--dangerously-bypass-approvals`.
The mode belongs to the process that owns the runtime and is fixed at launch:
`qwen-cua-driver serve` takes the flags, while `qwen-cua-driver mcp` and embedding hosts
use the matching `CUA_DRIVER_PERMISSION_MODE`,
`CUA_DRIVER_CAPABILITY_MANIFEST_FILE`, and
`CUA_DRIVER_CAPABILITY_MANIFEST_APPROVED` variables. Choose it before starting
the daemon; a running daemon must be restarted to change it.
Attaching to an existing logged-in Chromium profile remains explicit:
```bash
qwen-cua-driver mcp --grant existing-profile
```
An embedding application can instead provide `DriverAuthorizationHost` and
own the permission prompt and grant lifecycle.
## Qwen normalized coordinates
Pixel coordinates remain the default. Set:
```bash
export CUA_DRIVER_RS_COORDINATE_SPACE=1
export CUA_DRIVER_RS_COORDINATE_SCALE=1000 # optional; 1000 is the default
```
to expose and accept a 0-1000 coordinate grid. Translation happens at the
canonical tool-registry boundary shared by MCP, CLI, daemon, private worker,
replay, and direct SDK execution. Window-local actions use the most recent
snapshot dimensions; screen-space actions use the logical screen dimensions.
Missing or stale coordinate bases fail closed instead of being guessed.
Browser CSS coordinates are deliberately not normalized. In normalized mode,
use a fresh browser element reference for `browser_click` and
`browser_pointer`; raw CSS coordinates are rejected.
## Model-visible payload filtering
Set `MCP_MODEL_PAYLOAD_FILTER=1` to filter affected Qwen-facing textual MCP
payloads. The filter covers both text and structured content and leaves binary
image/media payloads opaque. It is off by default and does not change the
direct SDK contract.
## Telemetry boundary
Telemetry is disabled by default in the Qwen distribution. It can be enabled
explicitly with `qwen-cua-driver telemetry enable` or the documented
`CUA_DRIVER_RS_TELEMETRY_ENABLED=1` environment override. Explicit opt-in sends
the upstream content-free event schema to Cua's PostHog endpoint; no Qwen
telemetry service or proxy is involved.
Use `qwen-cua-driver telemetry status` to inspect the effective decision and
`qwen-cua-driver telemetry disable` to turn it off. A normal uninstall
preserves the preference and pseudonymous installation identifier; use
`--purge` on Unix or `CUA_DRIVER_RS_UNINSTALL_PURGE=1` on Windows to remove
them.
## macOS identity and permissions
macOS attributes Accessibility and Screen Recording grants to the responsible
app identity. Install and grant permissions to `QwenCuaDriver.app`. The
installed CLI can proxy through that app-owned daemon. `qwen-cua-driver mcp
--direct` deliberately uses the spawning host's TCC attribution and is not a
substitute for a certified embedding host.
Do not grant permissions to an arbitrary loose binary path in production.
Signed and notarized release artifacts are produced only by the Qwen-owned
release workflow.
## Repository layout
| Path | Purpose |
| ----------------- | -------------------------------------------------------------------- |
| `rust/` | Cargo workspace for the CLI, daemon, SDK, platform crates, and tests |
| `python/` | Python SDK and bundled-runtime wrapper |
| `typescript/` | TypeScript SDK and native runtime loader |
| `contract/` | Generated portable contract and fixtures |
| `tests/fixtures/` | Cross-platform GUI harness applications |
| `scripts/` | Release/local install, uninstall, generation, and sync helpers |
| `docs/` | Package-local architecture and behavior notes |
Start with [`rust/README.md`](rust/README.md),
[`docs/test-matrix.md`](docs/test-matrix.md), and
[`tests/fixtures/README.md`](tests/fixtures/README.md) when changing runtime
behavior or test coverage.
## Development
```bash
cd packages/cua-driver/rust
cargo fmt --all -- --check
cargo check -p cua-driver -p cua-driver-core -p cua-driver-sdk
cargo test -p cua-driver-core
```
Generated contract and language bindings must be checked with the package-local
scripts before release. Signed/notarized macOS, Windows UIAccess, Linux X11, and real MCP/model verification remain platform release gates.
Reference in new issue View Git Blame Copy Permalink