codewhale-hq/Codewhale
1
0 Fork 0
mirror of https://github.com/codewhale-hq/Codewhale.git synced 2026-11-11 10:01:30 -10:00
Code Issues Packages Projects Releases 1 Wiki Activity

fix(plugins): drop Cua and ship Computer Use 0.4.0 as the only desktop-control plugin

Browse Source 
Extensions and /mcp recommendations no longer offer Cua as a parallel
computer-use MCP. The first-party computer-use plugin is the only
desktop-control product, vendored at 0.4.0 (codewhale-cu-plugin@249ae77):
Return/Enter from type, filtered and paginated get_app_state,
focus/get_value, and strategy:"app" window-scoped clicks. Shared-desktop
pointer gestures stay gated.

Gates: localization parity 4 passed; codewhale-tui commands:: 1049 passed;
tui:: 3582 passed; builtin computer_use embed 2 passed; vendored plugin
npm test 223 passed, 0 failed, 15 skipped; git diff --check clean.
This commit is contained in:
CodeWhale Bot committed 2026-09-13 22:15:15 -07:00
1 parent 3b00c1a156
commit deb1dcdc39
36 files changed
+456 -184
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

CHANGELOG.md
+10
View File
@@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Changed
- Computer Use is the only computer-use product in Extensions and
`/mcp recommendations`. Cua is no longer suggested as a parallel
desktop-control MCP; enable the first-party `computer-use` plugin
instead. The bundled plugin is 0.4.0: Return/Enter from `type`,
filtered and paginated `get_app_state`, `focus`/`get_value`, and
`strategy:"app"` window-scoped clicks. Shared-desktop pointer
gestures stay gated.
### Fixed
- MCP OAuth login no longer fails with "Authorization server response missing
crates/localization/locales/ca.json
-2
View File
@@ -75,7 +75,6 @@
"ExtensionsPluginDetail": "{inventory} · confiança: {trust} · compatibilitat: {compatibility} · diagnòstics: {diagnostics}",
"ExtensionsProductBrowserUseDescription": "Flux de control del navegador empaquetat com una Habilitat d'agent.",
"ExtensionsProductChromeDescription": "Inspecciona i controla una sessió local de Chrome mitjançant l'MCP oficial.",
"ExtensionsProductCuaDescription": "Entorn local d'ús de l'ordinador amb controlador natiu de macOS.",
"ExtensionsProductDetail": "editor: {publisher} · components: {components} · origen: {source}",
"ExtensionsProductPlaywrightDescription": "Automatització del navegador basada en el Playwright MCP de Microsoft.",
"ExtensionsProductCodewhaleComputerUseDescription": "Veu la pantalla i la controla: control amb accessibilitat primer, captures, zoom i gravació de pantalla, més ordinadors remots per ssh.",
@@ -457,7 +456,6 @@
"McpRecommendationGithub": "• github — punt final MCP remot oficial de GitHub\n punt final: {endpoint}\n l’autenticació va a part: usa {login_command} només si el servidor anuncia OAuth;\n si no, configura fora de l’historial un PAT amb privilegis mínims. Els permisos\n concedits poden escriure o suprimir dades del repositori; comença en només lectura si és possible.\n afegeix explícitament: {add_command}",
"McpRecommendationChrome": "• chrome-devtools — MCP oficial de Chrome DevTools mitjançant un paquet npm fixat\n paquet: {package} ({launcher})\n pot inspeccionar/controlar Chrome i llegir pàgines autenticades. Tanca les pestanyes\n sensibles i verifica el paquet abans d’afegir-lo; {restart_command} pot baixar-lo i executar-lo.\n afegeix explícitament: {add_command}",
"McpRecommendationPlaywright": "• playwright — MCP oficial de Playwright de Microsoft mitjançant un paquet npm fixat\n paquet: {package} ({launcher})\n font: {source}\n --isolated inicia un perfil de navegador nou. Pot navegar, controlar pàgines i enviar\n formularis; revisa les accions primer. {restart_command} pot baixar i executar el paquet.\n afegeix explícitament: {add_command}",
"McpRecommendationCua": "• cua — plugin d'ús de l'ordinador Cua Driver (component de servidor MCP)\n ordre: cua-driver mcp\n font: {source}\n vista prèvia: instal·la i verifica Cua Driver per separat; Codewhale no el baixa mai. Pot\n controlar l'escriptori i necessita permisos del sistema operatiu. Revisa-ho abans de {restart_command}.\n afegeix explícitament: {add_command}",
"McpRecommendationContainerUse": "• container-use — MCP experimental container-use de Dagger\n ordre: container-use stdio\n font: {source}\n requereix el binari container-use instal·lat per separat; Codewhale mai no baixa ni\n instal·la aquest binari. Pot crear contenidors i canviar fitxers muntats de l’espai de treball.\n Revisa la instal·lació i l’ordre abans de {restart_command}.\n afegeix explícitament: {add_command}",
"PluginKimiUsage": "Ús:\n {list_command}\n {approve_command}\nLa llista és de només lectura. L’aprovació copia un connector canònic gestionat per Kimi mitjançant l’instal·lador revisat; roman desactivat i no fiable.",
"PluginKimiManagedRootHeading": "Connectors gestionats per Kimi a {root}:",
crates/localization/locales/de.json
-2
View File
@@ -75,7 +75,6 @@
"ExtensionsPluginDetail": "{inventory} · Vertrauen: {trust} · Kompatibilität: {compatibility} · Diagnosen: {diagnostics}",
"ExtensionsProductBrowserUseDescription": "Als Agent-Skill verpackter Browser-Steuerungsablauf.",
"ExtensionsProductChromeDescription": "Eine lokale Chrome-Sitzung über das offizielle MCP untersuchen und steuern.",
"ExtensionsProductCuaDescription": "Lokale Computersteuerungs-Laufzeit mit nativem macOS-Treiber.",
"ExtensionsProductDetail": "Herausgeber: {publisher} · Komponenten: {components} · Quelle: {source}",
"ExtensionsProductPlaywrightDescription": "Browser-Automatisierung auf Basis von Microsofts Playwright MCP.",
"ExtensionsProductCodewhaleComputerUseDescription": "Den Bildschirm sehen und bedienen: Steuerung über Bedienungshilfen, Screenshots, Zoom und Bildschirmaufnahme, dazu entfernte Rechner über ssh.",
@@ -457,7 +456,6 @@
"McpRecommendationGithub": "• github — offizieller Remote-MCP-Endpunkt von GitHub\n Endpunkt: {endpoint}\n Authentifizierung erfolgt getrennt: {login_command} nur verwenden, wenn der Server OAuth anbietet;\n andernfalls außerhalb des Befehlsverlaufs ein PAT mit minimalen Rechten konfigurieren. Erteilte\n Rechte können Repository-Daten schreiben oder löschen; möglichst schreibgeschützt beginnen.\n explizit hinzufügen: {add_command}",
"McpRecommendationChrome": "• chrome-devtools — offizielles Chrome-DevTools-MCP über ein fest versioniertes npm-Paket\n Paket: {package} ({launcher})\n es kann Chrome untersuchen/steuern und authentifizierte Seiten lesen. Vertrauliche Tabs\n schließen und das Paket vor dem Hinzufügen prüfen; {restart_command} kann es laden und ausführen.\n explizit hinzufügen: {add_command}",
"McpRecommendationPlaywright": "• playwright — Microsofts offizielles Playwright-MCP über ein fest versioniertes npm-Paket\n Paket: {package} ({launcher})\n Quelle: {source}\n --isolated startet ein neues Browserprofil. Es kann Seiten aufrufen/steuern und Formulare\n absenden; Aktionen vorher prüfen. {restart_command} kann das Paket laden und ausführen.\n explizit hinzufügen: {add_command}",
"McpRecommendationCua": "• cua — Cua-Driver-Plugin zur Computersteuerung (MCP-Serverkomponente)\n Befehl: cua-driver mcp\n Quelle: {source}\n Vorschau: Cua Driver separat installieren und prüfen; Codewhale lädt es nie herunter. Es kann\n den Desktop steuern und benötigt Betriebssystemrechte. Beides vor {restart_command} prüfen.\n explizit hinzufügen: {add_command}",
"McpRecommendationContainerUse": "• container-use — Dagger's experimentelles container-use-MCP\n Befehl: container-use stdio\n Quelle: {source}\n erfordert das separat installierte container-use-Programm; Codewhale lädt oder installiert\n dieses Programm nie. Es kann Container erstellen und eingebundene Arbeitsdateien ändern.\n Installation und Befehl vor {restart_command} prüfen.\n explizit hinzufügen: {add_command}",
"PluginKimiUsage": "Verwendung:\n {list_command}\n {approve_command}\nDie Liste ist schreibgeschützt. Die Freigabe kopiert ein kanonisches, von Kimi verwaltetes Plugin über den geprüften Installer; es bleibt deaktiviert und nicht vertrauenswürdig.",
"PluginKimiManagedRootHeading": "Von Kimi verwaltete Plugins unter {root}:",
crates/localization/locales/en.json
-2
View File
@@ -75,7 +75,6 @@
"ExtensionsPluginDetail": "{inventory} · trust: {trust} · compatibility: {compatibility} · diagnostics: {diagnostics}",
"ExtensionsProductBrowserUseDescription": "Browser-control workflow packaged as an agent Skill.",
"ExtensionsProductChromeDescription": "Inspect and control a local Chrome session through the official MCP.",
"ExtensionsProductCuaDescription": "Local computer-use runtime with a native macOS driver.",
"ExtensionsProductDetail": "publisher: {publisher} · components: {components} · source: {source}",
"ExtensionsProductPlaywrightDescription": "Browser automation built around Microsoft's Playwright MCP.",
"ExtensionsProductCodewhaleComputerUseDescription": "See the screen and operate it: accessibility-first control, screenshots, zoom, and screen recording, plus remote computers over ssh.",
@@ -460,7 +459,6 @@
"McpRecommendationGithub": "• github — GitHub's official remote MCP endpoint\n endpoint: {endpoint}\n auth is separate: {login_command} only for advertised OAuth;\n otherwise set a least-privilege PAT outside history. Scopes\n can write or delete repo data, so start read-only.\n add explicitly: {add_command}",
"McpRecommendationChrome": "• chrome-devtools — official Chrome DevTools MCP (pinned npm package)\n package: {package} ({launcher})\n it can drive Chrome and read signed-in pages. Close sensitive\n tabs and verify the package first; {restart_command} may download and run it.\n add explicitly: {add_command}",
"McpRecommendationPlaywright": "• playwright — Microsoft's official Playwright MCP (pinned npm package)\n package: {package} ({launcher})\n source: {source}\n --isolated starts a fresh browser profile. It can drive pages and submit\n forms, so review actions first; {restart_command} may download and run the package.\n add explicitly: {add_command}",
"McpRecommendationCua": "• cua — Cua Driver computer-use plugin (MCP server component)\n command: cua-driver mcp\n source: {source}\n preview: install and verify Cua Driver separately; Codewhale never downloads it. It can\n control the desktop and needs operating-system permissions. Review both before {restart_command}.\n add explicitly: {add_command}",
"McpRecommendationContainerUse": "• container-use — Dagger's experimental container-use MCP\n command: container-use stdio\n source: {source}\n requires the separately installed container-use binary; Codewhale never downloads or\n installs this binary. It can create containers and change mounted workspace files.\n Review the install and command before {restart_command}.\n add explicitly: {add_command}",
"CmdPluginDescription": "Manage trusted plugin bundles (legacy tools stay separate)",
"PluginKimiUsage": "Usage:\n {list_command}\n {approve_command}\nListing changes nothing. Approval copies one plugin through the reviewed installer; it stays disabled and untrusted.",
crates/localization/locales/es-419.json
-2
View File
@@ -75,7 +75,6 @@
"ExtensionsPluginDetail": "{inventory} · confianza: {trust} · compatibilidad: {compatibility} · diagnósticos: {diagnostics}",
"ExtensionsProductBrowserUseDescription": "Flujo de control del navegador empaquetado como Habilidad de agente.",
"ExtensionsProductChromeDescription": "Inspecciona y controla una sesión local de Chrome mediante el MCP oficial.",
"ExtensionsProductCuaDescription": "Entorno local de uso de computadora con controlador nativo de macOS.",
"ExtensionsProductDetail": "publicador: {publisher} · componentes: {components} · fuente: {source}",
"ExtensionsProductPlaywrightDescription": "Automatización del navegador basada en Playwright MCP de Microsoft.",
"ExtensionsProductCodewhaleComputerUseDescription": "Ve la pantalla y la opera: control basado en accesibilidad, capturas, zoom y grabación de pantalla, además de equipos remotos por ssh.",
@@ -460,7 +459,6 @@
"McpRecommendationGithub": "• github — endpoint MCP remoto oficial de GitHub\n endpoint: {endpoint}\n la autenticación es aparte: usa {login_command} solo si el servidor anuncia OAuth;\n de lo contrario, configura un PAT con privilegios mínimos fuera del historial de comandos. Los\n permisos concedidos pueden escribir o borrar datos del repositorio; empieza en modo de solo lectura cuando sea posible.\n agregar explícitamente: {add_command}",
"McpRecommendationChrome": "• chrome-devtools — MCP oficial de Chrome DevTools mediante un paquete npm fijado\n paquete: {package} ({launcher})\n puede inspeccionar o controlar Chrome y leer páginas autenticadas. Cierra las pestañas\n sensibles y verifica el paquete antes de agregarlo; {restart_command} puede descargarlo y ejecutarlo.\n agregar explícitamente: {add_command}",
"McpRecommendationPlaywright": "• playwright — MCP oficial de Playwright de Microsoft mediante un paquete npm fijado\n paquete: {package} ({launcher})\n fuente: {source}\n --isolated inicia un perfil de navegador nuevo. Puede navegar, controlar páginas y enviar\n formularios; revisa las acciones primero. {restart_command} puede descargar y ejecutar el paquete.\n agregar explícitamente: {add_command}",
"McpRecommendationCua": "• cua — plugin de uso de computadora Cua Driver (componente de servidor MCP)\n comando: cua-driver mcp\n fuente: {source}\n vista previa: instala y verifica Cua Driver por separado; Codewhale nunca lo descarga. Puede\n controlar el escritorio y necesita permisos del sistema operativo. Revisa ambos antes de {restart_command}.\n agregar explícitamente: {add_command}",
"McpRecommendationContainerUse": "• container-use — MCP experimental container-use de Dagger\n comando: container-use stdio\n fuente: {source}\n requiere el binario container-use instalado por separado; Codewhale nunca descarga ni\n instala este binario. Puede crear contenedores y cambiar archivos montados del espacio de trabajo.\n Revisa la instalación y el comando antes de {restart_command}.\n agregar explícitamente: {add_command}",
"PluginKimiUsage": "Uso:\n {list_command}\n {approve_command}\nLa lista es de solo lectura. La aprobación copia un plugin canónico gestionado por Kimi mediante el instalador revisado; permanece deshabilitado y no confiable.",
"PluginKimiManagedRootHeading": "Plugins gestionados por Kimi en {root}:",
crates/localization/locales/fr.json
-2
View File
@@ -75,7 +75,6 @@
"ExtensionsPluginDetail": "{inventory} · confiance : {trust} · compatibilité : {compatibility} · diagnostics : {diagnostics}",
"ExtensionsProductBrowserUseDescription": "Flux de contrôle du navigateur fourni comme Compétence d'agent.",
"ExtensionsProductChromeDescription": "Inspectez et contrôlez une session Chrome locale via le MCP officiel.",
"ExtensionsProductCuaDescription": "Environnement local d'utilisation de l'ordinateur avec pilote macOS natif.",
"ExtensionsProductDetail": "éditeur : {publisher} · composants : {components} · source : {source}",
"ExtensionsProductPlaywrightDescription": "Automatisation du navigateur fondée sur le Playwright MCP de Microsoft.",
"ExtensionsProductCodewhaleComputerUseDescription": "Voit l'écran et le pilote : contrôle par accessibilité, captures, zoom et enregistrement d'écran, plus des machines distantes en ssh.",
@@ -457,7 +456,6 @@
"McpRecommendationGithub": "• github — point de terminaison MCP distant officiel de GitHub\n point de terminaison : {endpoint}\n l’authentification est séparée : utilisez {login_command} uniquement si le serveur annonce OAuth ;\n sinon, configurez hors de l’historique un PAT aux privilèges minimaux. Les autorisations\n accordées peuvent écrire ou supprimer des données du dépôt ; commencez en lecture seule si possible.\n ajouter explicitement : {add_command}",
"McpRecommendationChrome": "• chrome-devtools — MCP Chrome DevTools officiel via un paquet npm à version fixe\n paquet : {package} ({launcher})\n il peut inspecter/contrôler Chrome et lire des pages authentifiées. Fermez les onglets\n sensibles et vérifiez le paquet avant l’ajout ; {restart_command} peut le télécharger et l’exécuter.\n ajouter explicitement : {add_command}",
"McpRecommendationPlaywright": "• playwright — MCP Playwright officiel de Microsoft via un paquet npm à version fixe\n paquet : {package} ({launcher})\n source : {source}\n --isolated démarre un nouveau profil de navigateur. Il peut parcourir/contrôler des pages et\n envoyer des formulaires ; vérifiez d’abord les actions. {restart_command} peut télécharger et exécuter le paquet.\n ajouter explicitement : {add_command}",
"McpRecommendationCua": "• cua — plugin de contrôle d'ordinateur Cua Driver (composant serveur MCP)\n commande : cua-driver mcp\n source : {source}\n aperçu : installez et vérifiez Cua Driver séparément ; Codewhale ne le télécharge jamais. Il peut\n contrôler le bureau et exige des autorisations système. Vérifiez les deux avant {restart_command}.\n ajouter explicitement : {add_command}",
"McpRecommendationContainerUse": "• container-use — MCP container-use expérimental de Dagger\n commande : container-use stdio\n source : {source}\n exige le binaire container-use installé séparément ; Codewhale ne télécharge ni n’installe\n jamais ce binaire. Il peut créer des conteneurs et modifier les fichiers de travail montés.\n Vérifiez l’installation et la commande avant {restart_command}.\n ajouter explicitement : {add_command}",
"PluginKimiUsage": "Utilisation :\n {list_command}\n {approve_command}\nLa liste est en lecture seule. L’approbation copie un plugin canonique géré par Kimi via l’installateur vérifié ; il reste désactivé et non approuvé.",
"PluginKimiManagedRootHeading": "Plugins gérés par Kimi dans {root} :",
crates/localization/locales/hi.json
-2
View File
@@ -75,7 +75,6 @@
"ExtensionsPluginDetail": "{inventory} · भरोसा: {trust} · अनुकूलता: {compatibility} · निदान: {diagnostics}",
"ExtensionsProductBrowserUseDescription": "एजेंट स्किल के रूप में पैक किया गया ब्राउज़र-नियंत्रण वर्कफ़्लो।",
"ExtensionsProductChromeDescription": "आधिकारिक MCP के माध्यम से स्थानीय Chrome सत्र की जाँच और नियंत्रण करें।",
"ExtensionsProductCuaDescription": "मूल macOS ड्राइवर वाला स्थानीय कंप्यूटर-उपयोग रनटाइम।",
"ExtensionsProductDetail": "प्रकाशक: {publisher} · घटक: {components} · स्रोत: {source}",
"ExtensionsProductPlaywrightDescription": "Microsoft के Playwright MCP पर आधारित ब्राउज़र स्वचालन।",
"ExtensionsProductCodewhaleComputerUseDescription": "स्क्रीन देखें और चलाएँ: एक्सेसिबिलिटी-आधारित नियंत्रण, स्क्रीनशॉट, ज़ूम और स्क्रीन रिकॉर्डिंग, साथ ही ssh से दूरस्थ कंप्यूटर।",
@@ -457,7 +456,6 @@
"McpRecommendationGithub": "• github — GitHub का आधिकारिक रिमोट MCP एंडपॉइंट\n एंडपॉइंट: {endpoint}\n प्रमाणीकरण अलग है: {login_command} केवल तभी चलाएँ जब सर्वर OAuth उपलब्ध बताए;\n अन्यथा कम-से-कम अधिकार वाला PAT कमांड इतिहास से बाहर कॉन्फ़िगर करें। दिए गए\n स्कोप रिपॉज़िटरी डेटा लिख या मिटा सकते हैं; जहाँ संभव हो केवल-पढ़ने से शुरू करें।\n स्पष्ट रूप से जोड़ें: {add_command}",
"McpRecommendationChrome": "• chrome-devtools — निश्चित संस्करण वाले npm पैकेज से आधिकारिक Chrome DevTools MCP\n पैकेज: {package} ({launcher})\n यह Chrome की जाँच/नियंत्रण और प्रमाणित पेज पढ़ सकता है। संवेदनशील टैब बंद करें\n और जोड़ने से पहले पैकेज जाँचें; {restart_command} इसे डाउनलोड करके चला सकता है।\n स्पष्ट रूप से जोड़ें: {add_command}",
"McpRecommendationPlaywright": "• playwright — निश्चित संस्करण वाले npm पैकेज से Microsoft का आधिकारिक Playwright MCP\n पैकेज: {package} ({launcher})\n स्रोत: {source}\n --isolated नया ब्राउज़र प्रोफ़ाइल शुरू करता है। यह पेज देख/नियंत्रित और फ़ॉर्म जमा कर\n सकता है; पहले कार्रवाइयाँ जाँचें। {restart_command} पैकेज डाउनलोड करके चला सकता है।\n स्पष्ट रूप से जोड़ें: {add_command}",
"McpRecommendationCua": "• cua — Cua Driver कंप्यूटर-उपयोग प्लगइन (MCP सर्वर घटक)\n कमांड: cua-driver mcp\n स्रोत: {source}\n पूर्वावलोकन: Cua Driver अलग से इंस्टॉल और सत्यापित करें; Codewhale इसे कभी डाउनलोड नहीं करता।\n यह डेस्कटॉप नियंत्रित कर सकता है और OS अनुमतियाँ चाहिए। {restart_command} से पहले दोनों जाँचें।\n स्पष्ट रूप से जोड़ें: {add_command}",
"McpRecommendationContainerUse": "• container-use — Dagger का प्रायोगिक container-use MCP\n कमांड: container-use stdio\n स्रोत: {source}\n अलग से इंस्टॉल किया गया container-use बाइनरी चाहिए; Codewhale इस बाइनरी को कभी\n डाउनलोड या इंस्टॉल नहीं करता। यह कंटेनर बना और माउंट की गई कार्यक्षेत्र फ़ाइलें बदल सकता है।\n {restart_command} से पहले इंस्टॉल और कमांड की जाँच करें।\n स्पष्ट रूप से जोड़ें: {add_command}",
"PluginKimiUsage": "उपयोग:\n {list_command}\n {approve_command}\nसूची केवल-पढ़ने योग्य है। मंज़ूरी पर Kimi-प्रबंधित एक मानक प्लगइन समीक्षित इंस्टॉलर से कॉपी होता है; वह बंद और अविश्वसनीय रहता है।",
"PluginKimiManagedRootHeading": "{root} पर Kimi-प्रबंधित प्लगइन:",
crates/localization/locales/id.json
-2
View File
@@ -75,7 +75,6 @@
"ExtensionsPluginDetail": "{inventory} · kepercayaan: {trust} · kompatibilitas: {compatibility} · diagnostik: {diagnostics}",
"ExtensionsProductBrowserUseDescription": "Alur kontrol browser yang dikemas sebagai Keahlian agen.",
"ExtensionsProductChromeDescription": "Periksa dan kendalikan sesi Chrome lokal melalui MCP resmi.",
"ExtensionsProductCuaDescription": "Runtime penggunaan komputer lokal dengan driver macOS native.",
"ExtensionsProductDetail": "penerbit: {publisher} · komponen: {components} · sumber: {source}",
"ExtensionsProductPlaywrightDescription": "Otomatisasi browser yang dibangun dengan Playwright MCP Microsoft.",
"ExtensionsProductCodewhaleComputerUseDescription": "Melihat layar dan mengoperasikannya: kendali berbasis aksesibilitas, tangkapan layar, zoom, dan perekaman layar, plus komputer jarak jauh lewat ssh.",
@@ -457,7 +456,6 @@
"McpRecommendationGithub": "• github — endpoint MCP jarak jauh resmi GitHub\n endpoint: {endpoint}\n autentikasi terpisah: gunakan {login_command} hanya jika server menawarkan OAuth;\n jika tidak, atur PAT dengan hak minimum di luar riwayat perintah. Cakupan yang\n diberikan dapat menulis atau menghapus data repositori; mulai dengan akses hanya-baca jika memungkinkan.\n tambahkan secara eksplisit: {add_command}",
"McpRecommendationChrome": "• chrome-devtools — MCP Chrome DevTools resmi lewat paket npm dengan versi terkunci\n paket: {package} ({launcher})\n ini dapat memeriksa/mengontrol Chrome dan membaca halaman terautentikasi. Tutup tab\n sensitif dan verifikasi paket sebelum menambahkannya; {restart_command} dapat mengunduh dan menjalankannya.\n tambahkan secara eksplisit: {add_command}",
"McpRecommendationPlaywright": "• playwright — MCP Playwright resmi Microsoft lewat paket npm dengan versi terkunci\n paket: {package} ({launcher})\n sumber: {source}\n --isolated memulai profil browser baru. Ini dapat menjelajah/mengontrol halaman dan mengirim\n formulir; tinjau tindakan dahulu. {restart_command} dapat mengunduh dan menjalankan paket.\n tambahkan secara eksplisit: {add_command}",
"McpRecommendationCua": "• cua — plugin penggunaan komputer Cua Driver (komponen server MCP)\n perintah: cua-driver mcp\n sumber: {source}\n pratinjau: pasang dan verifikasi Cua Driver secara terpisah; Codewhale tidak pernah mengunduhnya.\n Ini dapat mengontrol desktop dan memerlukan izin sistem operasi. Tinjau keduanya sebelum {restart_command}.\n tambahkan secara eksplisit: {add_command}",
"McpRecommendationContainerUse": "• container-use — MCP container-use eksperimental dari Dagger\n perintah: container-use stdio\n sumber: {source}\n memerlukan biner container-use yang dipasang terpisah; Codewhale tidak pernah mengunduh atau\n memasang biner ini. Ini dapat membuat kontainer dan mengubah berkas ruang kerja yang dipasang.\n Tinjau pemasangan dan perintah sebelum {restart_command}.\n tambahkan secara eksplisit: {add_command}",
"PluginKimiUsage": "Penggunaan:\n {list_command}\n {approve_command}\nDaftar hanya-baca. Persetujuan menyalin satu plugin kanonis kelolaan Kimi melalui penginstal yang ditinjau; plugin tetap nonaktif dan tidak dipercaya.",
"PluginKimiManagedRootHeading": "Plugin kelolaan Kimi di {root}:",
crates/localization/locales/ja.json
-2
View File
@@ -75,7 +75,6 @@
"ExtensionsPluginDetail": "{inventory} · 信頼: {trust} · 互換性: {compatibility} · 診断: {diagnostics}",
"ExtensionsProductBrowserUseDescription": "エージェントスキルとして提供されるブラウザ操作ワークフロー。",
"ExtensionsProductChromeDescription": "公式 MCP を通じてローカルの Chrome セッションを検査・操作します。",
"ExtensionsProductCuaDescription": "macOS ネイティブドライバーを備えたローカルのコンピューター操作ランタイム。",
"ExtensionsProductDetail": "発行元: {publisher} · コンポーネント: {components} · ソース: {source}",
"ExtensionsProductPlaywrightDescription": "Microsoft の Playwright MCP を基盤とするブラウザ自動化。",
"ExtensionsProductCodewhaleComputerUseDescription": "画面を見て操作する。アクセシビリティ優先の操作、スクリーンショット、ズーム、画面録画に加え、ssh 経由のリモートコンピューター。",
@@ -460,7 +459,6 @@
"McpRecommendationGithub": "• github — GitHub 公式のリモート MCP エンドポイント\n エンドポイント: {endpoint}\n 認証は別です。サーバーが OAuth を提供する場合だけ {login_command} を使用してください。\n それ以外は、コマンド履歴の外で最小権限の PAT を設定してください。付与した\n スコープはリポジトリデータを書き込み・削除できるため、可能なら読み取り専用で始めてください。\n 明示的に追加: {add_command}",
"McpRecommendationChrome": "• chrome-devtools — バージョン固定 npm パッケージによる公式 Chrome DevTools MCP\n パッケージ: {package} ({launcher})\n Chrome の調査・操作や認証済みページの読み取りが可能です。機密タブを\n 閉じ、追加前にパッケージを確認してください。{restart_command} はダウンロードして実行する場合があります。\n 明示的に追加: {add_command}",
"McpRecommendationPlaywright": "• playwright — バージョン固定 npm パッケージによる Microsoft 公式 Playwright MCP\n パッケージ: {package} ({launcher})\n ソース: {source}\n --isolated は新しいブラウザープロファイルを起動します。ページの閲覧・操作やフォーム送信が\n 可能なため、先に操作を確認してください。{restart_command} はパッケージをダウンロードして実行する場合があります。\n 明示的に追加: {add_command}",
"McpRecommendationCua": "• cua — Cua Driver コンピューター操作プラグイン（MCP サーバーコンポーネント）\n コマンド: cua-driver mcp\n ソース: {source}\n プレビュー: Cua Driver は別途インストールして検証してください。Codewhale はダウンロードしません。\n デスクトップを操作し OS 権限を必要とします。{restart_command} の前に両方を確認してください。\n 明示的に追加: {add_command}",
"McpRecommendationContainerUse": "• container-use — Dagger の実験的な container-use MCP\n コマンド: container-use stdio\n ソース: {source}\n 別途インストールした container-use バイナリが必要です。Codewhale がこのバイナリを\n ダウンロードまたはインストールすることはありません。コンテナを作成し、マウントした作業ファイルを変更できます。\n {restart_command} の前にインストール元とコマンドを確認してください。\n 明示的に追加: {add_command}",
"PluginKimiUsage": "使い方:\n {list_command}\n {approve_command}\n一覧表示は読み取り専用です。承認すると、Kimi 管理の正規プラグイン 1 件を確認済みインストーラーでコピーします。コピー後も無効・未信頼のままです。",
"PluginKimiManagedRootHeading": "{root} の Kimi 管理プラグイン:",
crates/localization/locales/ko.json
-2
View File
@@ -75,7 +75,6 @@
"ExtensionsPluginDetail": "{inventory} · 신뢰: {trust} · 호환성: {compatibility} · 진단: {diagnostics}",
"ExtensionsProductBrowserUseDescription": "에이전트 스킬로 패키지된 브라우저 제어 워크플로입니다.",
"ExtensionsProductChromeDescription": "공식 MCP를 통해 로컬 Chrome 세션을 검사하고 제어합니다.",
"ExtensionsProductCuaDescription": "네이티브 macOS 드라이버를 갖춘 로컬 컴퓨터 사용 런타임입니다.",
"ExtensionsProductDetail": "게시자: {publisher} · 구성 요소: {components} · 소스: {source}",
"ExtensionsProductPlaywrightDescription": "Microsoft의 Playwright MCP를 기반으로 한 브라우저 자동화입니다.",
"ExtensionsProductCodewhaleComputerUseDescription": "화면을 보고 조작합니다. 접근성 우선 제어, 스크린샷, 확대, 화면 녹화에 ssh 원격 컴퓨터까지.",
@@ -460,7 +459,6 @@
"McpRecommendationGithub": "• github — GitHub 공식 원격 MCP 엔드포인트\n 엔드포인트: {endpoint}\n 인증은 별도입니다. 서버가 OAuth를 제공할 때만 {login_command} 명령을 사용하세요.\n 그 외에는 명령 기록 밖에서 최소 권한 PAT를 설정하세요. 부여한 범위는\n 저장소 데이터를 쓰거나 삭제할 수 있으므로 가능하면 읽기 전용으로 시작하세요.\n 명시적으로 추가: {add_command}",
"McpRecommendationChrome": "• chrome-devtools — 버전을 고정한 npm 패키지 기반 공식 Chrome DevTools MCP\n 패키지: {package} ({launcher})\n Chrome을 검사/제어하고 인증된 페이지를 읽을 수 있습니다. 민감한 탭을\n 닫고 추가 전에 패키지를 확인하세요. {restart_command} 명령이 다운로드하여 실행할 수 있습니다.\n 명시적으로 추가: {add_command}",
"McpRecommendationPlaywright": "• playwright — 버전을 고정한 npm 패키지 기반 Microsoft 공식 Playwright MCP\n 패키지: {package} ({launcher})\n 소스: {source}\n --isolated는 새 브라우저 프로필을 시작합니다. 페이지 탐색/제어와 양식 제출이\n 가능하므로 작업을 먼저 검토하세요. {restart_command} 명령이 패키지를 다운로드하여 실행할 수 있습니다.\n 명시적으로 추가: {add_command}",
"McpRecommendationCua": "• cua — Cua Driver 컴퓨터 사용 플러그인(MCP 서버 구성 요소)\n 명령: cua-driver mcp\n 소스: {source}\n 미리 보기: Cua Driver를 별도로 설치하고 확인하세요. Codewhale은 다운로드하지 않습니다.\n 데스크톱을 제어하며 운영 체제 권한이 필요합니다. {restart_command} 전에 둘 다 검토하세요.\n 명시적으로 추가: {add_command}",
"McpRecommendationContainerUse": "• container-use — Dagger의 실험적 container-use MCP\n 명령: container-use stdio\n 소스: {source}\n 별도로 설치한 container-use 바이너리가 필요합니다. Codewhale은 이 바이너리를\n 다운로드하거나 설치하지 않습니다. 컨테이너를 만들고 마운트된 작업공간 파일을 변경할 수 있습니다.\n {restart_command} 전에 설치와 명령을 검토하세요.\n 명시적으로 추가: {add_command}",
"PluginKimiUsage": "사용법:\n {list_command}\n {approve_command}\n목록은 읽기 전용입니다. 승인하면 Kimi 관리 정식 플러그인 하나를 검토된 설치 프로그램으로 복사하며, 비활성 및 신뢰되지 않은 상태를 유지합니다.",
"PluginKimiManagedRootHeading": "{root}의 Kimi 관리 플러그인:",
crates/localization/locales/pt-BR.json
-2
View File
@@ -75,7 +75,6 @@
"ExtensionsPluginDetail": "{inventory} · confiança: {trust} · compatibilidade: {compatibility} · diagnósticos: {diagnostics}",
"ExtensionsProductBrowserUseDescription": "Fluxo de controle do navegador empacotado como uma Habilidade de agente.",
"ExtensionsProductChromeDescription": "Inspecione e controle uma sessão local do Chrome pelo MCP oficial.",
"ExtensionsProductCuaDescription": "Runtime local de uso do computador com driver nativo do macOS.",
"ExtensionsProductDetail": "publicador: {publisher} · componentes: {components} · origem: {source}",
"ExtensionsProductPlaywrightDescription": "Automação de navegador criada em torno do Playwright MCP da Microsoft.",
"ExtensionsProductCodewhaleComputerUseDescription": "Vê a tela e a opera: controle por acessibilidade, capturas, zoom e gravação de tela, além de computadores remotos por ssh.",
@@ -460,7 +459,6 @@
"McpRecommendationGithub": "• github — endpoint MCP remoto oficial do GitHub\n endpoint: {endpoint}\n a autenticação é separada: use {login_command} somente se o servidor anunciar OAuth;\n caso contrário, configure um PAT com privilégios mínimos fora do histórico de comandos. Os\n escopos concedidos podem gravar ou excluir dados do repositório; comece em modo somente leitura quando possível.\n adicionar explicitamente: {add_command}",
"McpRecommendationChrome": "• chrome-devtools — MCP oficial do Chrome DevTools via pacote npm com versão fixada\n pacote: {package} ({launcher})\n ele pode inspecionar/controlar o Chrome e ler páginas autenticadas. Feche abas\n confidenciais e verifique o pacote antes de adicioná-lo; {restart_command} pode baixá-lo e executá-lo.\n adicionar explicitamente: {add_command}",
"McpRecommendationPlaywright": "• playwright — MCP oficial do Playwright da Microsoft via pacote npm com versão fixada\n pacote: {package} ({launcher})\n fonte: {source}\n --isolated inicia um novo perfil do navegador. Ele pode navegar/controlar páginas e enviar\n formulários; revise as ações primeiro. {restart_command} pode baixar e executar o pacote.\n adicionar explicitamente: {add_command}",
"McpRecommendationCua": "• cua — plugin de uso do computador Cua Driver (componente de servidor MCP)\n comando: cua-driver mcp\n fonte: {source}\n prévia: instale e verifique o Cua Driver separadamente; o Codewhale nunca o baixa. Ele pode\n controlar a área de trabalho e requer permissões do sistema. Revise ambos antes de {restart_command}.\n adicionar explicitamente: {add_command}",
"McpRecommendationContainerUse": "• container-use — MCP container-use experimental da Dagger\n comando: container-use stdio\n fonte: {source}\n requer o binário container-use instalado separadamente; o Codewhale nunca baixa nem\n instala esse binário. Ele pode criar contêineres e alterar arquivos montados do espaço de trabalho.\n Revise a instalação e o comando antes de {restart_command}.\n adicionar explicitamente: {add_command}",
"PluginKimiUsage": "Uso:\n {list_command}\n {approve_command}\nA listagem é somente leitura. A aprovação copia um plugin canônico gerenciado pelo Kimi pelo instalador revisado; ele permanece desativado e não confiável.",
"PluginKimiManagedRootHeading": "Plugins gerenciados pelo Kimi em {root}:",
crates/localization/locales/ru.json
-2
View File
@@ -75,7 +75,6 @@
"ExtensionsPluginDetail": "{inventory} · доверие: {trust} · совместимость: {compatibility} · диагностика: {diagnostics}",
"ExtensionsProductBrowserUseDescription": "Сценарий управления браузером, упакованный как навык агента.",
"ExtensionsProductChromeDescription": "Просмотр и управление локальным сеансом Chrome через официальный MCP.",
"ExtensionsProductCuaDescription": "Локальная среда управления компьютером с нативным драйвером macOS.",
"ExtensionsProductDetail": "издатель: {publisher} · компоненты: {components} · источник: {source}",
"ExtensionsProductPlaywrightDescription": "Автоматизация браузера на основе Playwright MCP от Microsoft.",
"ExtensionsProductCodewhaleComputerUseDescription": "Видит экран и управляет им: управление через специальные возможности, снимки, масштаб и запись экрана, а также удалённые компьютеры по ssh.",
@@ -457,7 +456,6 @@
"McpRecommendationGithub": "• github — официальный удалённый MCP-адрес GitHub\n адрес: {endpoint}\n аутентификация выполняется отдельно: используйте {login_command}, только если сервер заявляет OAuth;\n иначе настройте PAT с минимальными правами вне истории команд. Выданные\n области доступа могут изменять или удалять данные репозитория; по возможности начните с чтения.\n добавить явно: {add_command}",
"McpRecommendationChrome": "• chrome-devtools — официальный Chrome DevTools MCP через npm-пакет закреплённой версии\n пакет: {package} ({launcher})\n он может исследовать/управлять Chrome и читать страницы с авторизацией. Закройте\n конфиденциальные вкладки и проверьте пакет до добавления; {restart_command} может скачать и запустить его.\n добавить явно: {add_command}",
"McpRecommendationPlaywright": "• playwright — официальный Playwright MCP от Microsoft через npm-пакет закреплённой версии\n пакет: {package} ({launcher})\n источник: {source}\n --isolated запускает новый профиль браузера. Сервер может открывать страницы, управлять ими\n и отправлять формы; сначала проверьте действия. {restart_command} может скачать и запустить пакет.\n добавить явно: {add_command}",
"McpRecommendationCua": "• cua — плагин компьютерного управления Cua Driver (компонент MCP-сервера)\n команда: cua-driver mcp\n источник: {source}\n предварительная версия: установите и проверьте Cua Driver отдельно; Codewhale его не скачивает.\n Он может управлять рабочим столом и требует разрешений ОС. Проверьте всё перед {restart_command}.\n добавить явно: {add_command}",
"McpRecommendationContainerUse": "• container-use — экспериментальный container-use MCP от Dagger\n команда: container-use stdio\n источник: {source}\n требуется отдельно установленный исполняемый файл container-use; Codewhale никогда не\n скачивает и не устанавливает его. Сервер может создавать контейнеры и менять подключённые файлы проекта.\n Проверьте установку и команду перед {restart_command}.\n добавить явно: {add_command}",
"PluginKimiUsage": "Использование:\n {list_command}\n {approve_command}\nСписок доступен только для чтения. После подтверждения один канонический плагин под управлением Kimi копируется проверенным установщиком; он остаётся отключённым и недоверенным.",
"PluginKimiManagedRootHeading": "Плагины под управлением Kimi в {root}:",
crates/localization/locales/uk.json
-2
View File
@@ -75,7 +75,6 @@
"ExtensionsPluginDetail": "{inventory} · довіра: {trust} · сумісність: {compatibility} · діагностика: {diagnostics}",
"ExtensionsProductBrowserUseDescription": "Сценарій керування браузером, запакований як навичка агента.",
"ExtensionsProductChromeDescription": "Перегляд і керування локальним сеансом Chrome через офіційний MCP.",
"ExtensionsProductCuaDescription": "Локальне середовище керування комп'ютером із нативним драйвером macOS.",
"ExtensionsProductDetail": "видавець: {publisher} · компоненти: {components} · джерело: {source}",
"ExtensionsProductPlaywrightDescription": "Автоматизація браузера на основі Playwright MCP від Microsoft.",
"ExtensionsProductCodewhaleComputerUseDescription": "Бачить екран і керує ним: керування через спеціальні можливості, знімки, масштаб і запис екрана, а також віддалені комп'ютери через ssh.",
@@ -457,7 +456,6 @@
"McpRecommendationGithub": "• github — офіційна віддалена MCP-адреса GitHub\n адреса: {endpoint}\n автентифікація виконується окремо: використовуйте {login_command}, лише якщо сервер заявляє OAuth;\n інакше налаштуйте PAT із мінімальними правами поза історією команд. Надані\n області доступу можуть змінювати або видаляти дані репозиторію; за можливості почніть із читання.\n додати явно: {add_command}",
"McpRecommendationChrome": "• chrome-devtools — офіційний Chrome DevTools MCP через npm-пакет закріпленої версії\n пакет: {package} ({launcher})\n він може досліджувати/керувати Chrome і читати сторінки з авторизацією. Закрийте\n конфіденційні вкладки й перевірте пакет до додавання; {restart_command} може завантажити та запустити його.\n додати явно: {add_command}",
"McpRecommendationPlaywright": "• playwright — офіційний Playwright MCP від Microsoft через npm-пакет закріпленої версії\n пакет: {package} ({launcher})\n джерело: {source}\n --isolated запускає новий профіль браузера. Сервер може відкривати сторінки, керувати ними\n та надсилати форми; спершу перевірте дії. {restart_command} може завантажити й запустити пакет.\n додати явно: {add_command}",
"McpRecommendationCua": "• cua — плагін комп'ютерного керування Cua Driver (компонент MCP-сервера)\n команда: cua-driver mcp\n джерело: {source}\n попередня версія: встановіть і перевірте Cua Driver окремо; Codewhale його не завантажує.\n Він може керувати робочим столом і потребує дозволів ОС. Перевірте все перед {restart_command}.\n додати явно: {add_command}",
"McpRecommendationContainerUse": "• container-use — експериментальний container-use MCP від Dagger\n команда: container-use stdio\n джерело: {source}\n потрібен окремо встановлений виконуваний файл container-use; Codewhale ніколи не\n завантажує й не встановлює його. Сервер може створювати контейнери та змінювати підключені файли проєкту.\n Перевірте встановлення й команду перед {restart_command}.\n додати явно: {add_command}",
"PluginKimiUsage": "Використання:\n {list_command}\n {approve_command}\nСписок доступний лише для читання. Після підтвердження один канонічний плагін під керуванням Kimi копіюється перевіреним інсталятором; він залишається вимкненим і недовіреним.",
"PluginKimiManagedRootHeading": "Плагіни під керуванням Kimi у {root}:",
crates/localization/locales/vi.json
-2
View File
@@ -75,7 +75,6 @@
"ExtensionsPluginDetail": "{inventory} · độ tin cậy: {trust} · tính tương thích: {compatibility} · chẩn đoán: {diagnostics}",
"ExtensionsProductBrowserUseDescription": "Quy trình điều khiển trình duyệt được đóng gói dưới dạng Kỹ năng tác nhân.",
"ExtensionsProductChromeDescription": "Kiểm tra và điều khiển phiên Chrome cục bộ qua MCP chính thức.",
"ExtensionsProductCuaDescription": "Môi trường chạy điều khiển máy tính cục bộ với trình điều khiển macOS gốc.",
"ExtensionsProductDetail": "nhà phát hành: {publisher} · thành phần: {components} · nguồn: {source}",
"ExtensionsProductPlaywrightDescription": "Tự động hóa trình duyệt dựa trên Playwright MCP của Microsoft.",
"ExtensionsProductCodewhaleComputerUseDescription": "Nhìn màn hình và điều khiển: điều khiển ưu tiên trợ năng, ảnh chụp, thu phóng và ghi màn hình, cùng máy tính từ xa qua ssh.",
@@ -460,7 +459,6 @@
"McpRecommendationGithub": "• github — điểm cuối MCP từ xa chính thức của GitHub\n điểm cuối: {endpoint}\n xác thực là bước riêng: chỉ dùng {login_command} khi máy chủ công bố OAuth;\n nếu không, hãy cấu hình PAT có quyền tối thiểu ngoài lịch sử lệnh. Phạm vi được\n cấp có thể ghi hoặc xóa dữ liệu kho mã; nên bắt đầu ở chế độ chỉ đọc khi có thể.\n thêm rõ ràng: {add_command}",
"McpRecommendationChrome": "• chrome-devtools — MCP Chrome DevTools chính thức qua gói npm đã ghim phiên bản\n gói: {package} ({launcher})\n có thể kiểm tra/điều khiển Chrome và đọc trang đã xác thực. Đóng các thẻ\n nhạy cảm và xác minh gói trước khi thêm; {restart_command} có thể tải xuống và chạy gói.\n thêm rõ ràng: {add_command}",
"McpRecommendationPlaywright": "• playwright — MCP Playwright chính thức của Microsoft qua gói npm đã ghim phiên bản\n gói: {package} ({launcher})\n nguồn: {source}\n --isolated khởi động hồ sơ trình duyệt mới. Nó có thể duyệt/điều khiển trang và gửi\n biểu mẫu; hãy xem lại thao tác trước. {restart_command} có thể tải xuống và chạy gói.\n thêm rõ ràng: {add_command}",
"McpRecommendationCua": "• cua — plugin sử dụng máy tính Cua Driver (thành phần máy chủ MCP)\n lệnh: cua-driver mcp\n nguồn: {source}\n bản xem trước: cài và xác minh Cua Driver riêng; Codewhale không bao giờ tải nó xuống. Nó có thể\n điều khiển màn hình và cần quyền hệ điều hành. Hãy xem lại cả hai trước {restart_command}.\n thêm rõ ràng: {add_command}",
"McpRecommendationContainerUse": "• container-use — MCP container-use thử nghiệm của Dagger\n lệnh: container-use stdio\n nguồn: {source}\n cần cài riêng tệp nhị phân container-use; Codewhale không bao giờ tải xuống hoặc\n cài tệp này. Nó có thể tạo container và thay đổi tệp không gian làm việc được gắn.\n Hãy xem lại bản cài và lệnh trước {restart_command}.\n thêm rõ ràng: {add_command}",
"PluginKimiUsage": "Cách dùng:\n {list_command}\n {approve_command}\nDanh sách chỉ để đọc. Khi phê duyệt, một plugin chuẩn do Kimi quản lý được sao chép qua trình cài đặt đã duyệt; plugin vẫn bị tắt và chưa được tin cậy.",
"PluginKimiManagedRootHeading": "Plugin do Kimi quản lý tại {root}:",
crates/localization/locales/zh-Hans.json
-2
View File
@@ -75,7 +75,6 @@
"ExtensionsPluginDetail": "{inventory} · 信任：{trust} · 兼容性：{compatibility} · 诊断：{diagnostics}",
"ExtensionsProductBrowserUseDescription": "以代理技能形式打包的浏览器控制工作流。",
"ExtensionsProductChromeDescription": "通过官方 MCP 检查和控制本地 Chrome 会话。",
"ExtensionsProductCuaDescription": "带有原生 macOS 驱动的本地计算机操作运行时。",
"ExtensionsProductDetail": "发布者：{publisher} · 组件：{components} · 来源：{source}",
"ExtensionsProductPlaywrightDescription": "基于 Microsoft Playwright MCP 的浏览器自动化。",
"ExtensionsProductCodewhaleComputerUseDescription": "看见屏幕并操作它：以无障碍接口为先的控制、截图、缩放与录屏，还可通过 ssh 操作远程计算机。",
@@ -460,7 +459,6 @@
"McpRecommendationGithub": "• github — GitHub 官方远程 MCP 端点\n 端点：{endpoint}\n 身份验证独立进行：仅在服务器声明支持 OAuth 时使用 {login_command}；\n 否则请在命令历史之外配置最小权限 PAT。授予的范围可能写入或删除\n 仓库数据，因此请尽可能从只读权限开始。\n 显式添加：{add_command}",
"McpRecommendationChrome": "• chrome-devtools — 通过锁定版本的 npm 包提供的官方 Chrome DevTools MCP\n 包：{package}（{launcher}）\n 它可以检查或控制 Chrome，并读取已认证页面。请关闭敏感标签页并在\n 添加前验证软件包；{restart_command} 可能会下载并运行它。\n 显式添加：{add_command}",
"McpRecommendationPlaywright": "• playwright — 通过锁定版本的 npm 包提供的 Microsoft 官方 Playwright MCP\n 包：{package}（{launcher}）\n 源：{source}\n --isolated 会启动新的浏览器配置文件。它可以浏览或控制页面并提交表单，\n 因此请先审查操作；{restart_command} 可能会下载并运行该软件包。\n 显式添加：{add_command}",
"McpRecommendationCua": "• cua — Cua Driver 计算机操作插件（MCP 服务器组件）\n 命令：cua-driver mcp\n 源：{source}\n 预览：请另行安装并验证 Cua Driver；Codewhale 绝不会下载它。它可以控制桌面，\n 并需要操作系统权限。请在 {restart_command} 前审查安装和权限。\n 显式添加：{add_command}",
"McpRecommendationContainerUse": "• container-use — Dagger 的实验性 container-use MCP\n 命令：container-use stdio\n 源：{source}\n 需要另行安装 container-use 二进制文件；Codewhale 绝不会下载或安装此文件。\n 它可以创建容器并修改已挂载的工作区文件。请在 {restart_command} 前审查安装来源和命令。\n 显式添加：{add_command}",
"PluginKimiUsage": "用法：\n {list_command}\n {approve_command}\n列表操作为只读。批准后会通过已审查的安装器复制一个规范的 Kimi 托管插件；它仍保持禁用且不受信任。",
"PluginKimiManagedRootHeading": "{root} 中的 Kimi 托管插件：",
crates/localization/locales/zh-Hant.json
-2
View File
@@ -75,7 +75,6 @@
"ExtensionsPluginDetail": "{inventory} · 信任：{trust} · 相容性：{compatibility} · 診斷：{diagnostics}",
"ExtensionsProductBrowserUseDescription": "以代理技能形式封裝的瀏覽器控制工作流程。",
"ExtensionsProductChromeDescription": "透過官方 MCP 檢查並控制本機 Chrome 工作階段。",
"ExtensionsProductCuaDescription": "具備原生 macOS 驅動程式的本機電腦操作執行環境。",
"ExtensionsProductDetail": "發行者：{publisher} · 元件：{components} · 來源：{source}",
"ExtensionsProductPlaywrightDescription": "以 Microsoft Playwright MCP 為基礎的瀏覽器自動化。",
"ExtensionsProductCodewhaleComputerUseDescription": "看見螢幕並操作它：以無障礙介面為先的控制、螢幕擷取、縮放與錄影，還可透過 ssh 操作遠端電腦。",
@@ -352,7 +351,6 @@
"McpRecommendationGithub": "• github — GitHub 官方遠端 MCP 端點\n 端點：{endpoint}\n 驗證會另外進行：只有伺服器宣告支援 OAuth 時才使用 {login_command}；\n 否則請在指令歷程之外設定最小權限 PAT。授予的範圍可能寫入或刪除\n 儲存庫資料，因此請盡可能從唯讀權限開始。\n 明確新增：{add_command}",
"McpRecommendationChrome": "• chrome-devtools — 透過鎖定版本 npm 套件提供的官方 Chrome DevTools MCP\n 套件：{package}（{launcher}）\n 它可檢查或控制 Chrome，並讀取已驗證頁面。請關閉敏感分頁並在\n 新增前驗證套件；{restart_command} 可能會下載並執行它。\n 明確新增：{add_command}",
"McpRecommendationPlaywright": "• playwright — 透過鎖定版本 npm 套件提供的 Microsoft 官方 Playwright MCP\n 套件：{package}（{launcher}）\n 來源：{source}\n --isolated 會啟動新的瀏覽器設定檔。它可瀏覽或控制頁面並提交表單，\n 因此請先審查操作；{restart_command} 可能會下載並執行該套件。\n 明確新增：{add_command}",
"McpRecommendationCua": "• cua — Cua Driver 電腦操作外掛（MCP 伺服器元件）\n 指令：cua-driver mcp\n 來源：{source}\n 預覽：請另行安裝並驗證 Cua Driver；Codewhale 絕不會下載它。它可以控制桌面，\n 並需要作業系統權限。請在 {restart_command} 前審查安裝與權限。\n 明確新增：{add_command}",
"McpRecommendationContainerUse": "• container-use — Dagger 的實驗性 container-use MCP\n 指令：container-use stdio\n 來源：{source}\n 需要另行安裝 container-use 二進位檔；Codewhale 絕不會下載或安裝此檔案。\n 它可以建立容器並修改已掛載的工作區檔案。請在 {restart_command} 前審查安裝來源和指令。\n 明確新增：{add_command}",
"PluginKimiUsage": "用法：\n {list_command}\n {approve_command}\n列出操作為唯讀。核准後會透過已審查的安裝程式複製一個標準的 Kimi 受管外掛程式；它仍保持停用且不受信任。",
"PluginKimiManagedRootHeading": "{root} 中的 Kimi 受管外掛程式：",
crates/localization/src/lib.rs
+1 -7
View File
@@ -535,7 +535,6 @@ pub enum MessageId {
McpRecommendationGithub,
McpRecommendationChrome,
McpRecommendationPlaywright,
McpRecommendationCua,
McpRecommendationContainerUse,
McpCapabilitiesAdvertised,
McpCapabilitiesLegacyFallback,
@@ -591,7 +590,6 @@ pub enum MessageId {
ExtensionsPluginDetail,
ExtensionsProductBrowserUseDescription,
ExtensionsProductChromeDescription,
ExtensionsProductCuaDescription,
ExtensionsProductDetail,
ExtensionsProductPlaywrightDescription,
ExtensionsProductCodewhaleComputerUseDescription,
@@ -2876,7 +2874,6 @@ pub const ALL_MESSAGE_IDS: &[MessageId] = &[
MessageId::McpRecommendationGithub,
MessageId::McpRecommendationChrome,
MessageId::McpRecommendationPlaywright,
MessageId::McpRecommendationCua,
MessageId::McpRecommendationContainerUse,
MessageId::McpCapabilitiesAdvertised,
MessageId::McpCapabilitiesLegacyFallback,
@@ -2931,7 +2928,6 @@ pub const ALL_MESSAGE_IDS: &[MessageId] = &[
MessageId::ExtensionsPluginDetail,
MessageId::ExtensionsProductBrowserUseDescription,
MessageId::ExtensionsProductChromeDescription,
MessageId::ExtensionsProductCuaDescription,
MessageId::ExtensionsProductDetail,
MessageId::ExtensionsProductPlaywrightDescription,
MessageId::ExtensionsProductCodewhaleComputerUseDescription,
@@ -6080,7 +6076,6 @@ mod tests {
MessageId::McpRecommendationGithub,
MessageId::McpRecommendationChrome,
MessageId::McpRecommendationPlaywright,
MessageId::McpRecommendationCua,
MessageId::McpRecommendationContainerUse,
MessageId::PluginKimiUsage,
MessageId::PluginKimiManagedRootHeading,
@@ -6157,7 +6152,7 @@ mod tests {
.filter(|key| key.starts_with("Extensions"))
.cloned()
.collect::<Vec<_>>();
assert_eq!(keys.len(), 94, "the complete extensions locale set changed");
assert_eq!(keys.len(), 93, "the complete extensions locale set changed");
let prose_keys = [
"ExtensionsMarketplaceUnavailable",
@@ -6167,7 +6162,6 @@ mod tests {
"ExtensionsNoMatches",
"ExtensionsProductBrowserUseDescription",
"ExtensionsProductChromeDescription",
"ExtensionsProductCuaDescription",
"ExtensionsProductPlaywrightDescription",
"ExtensionsProductSandboxDescription",
];
crates/tui/CHANGELOG.md
+10
View File
@@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Changed
- Computer Use is the only computer-use product in Extensions and
`/mcp recommendations`. Cua is no longer suggested as a parallel
desktop-control MCP; enable the first-party `computer-use` plugin
instead. The bundled plugin is 0.4.0: Return/Enter from `type`,
filtered and paginated `get_app_state`, `focus`/`get_value`, and
`strategy:"app"` window-scoped clicks. Shared-desktop pointer
gestures stay gated.
### Fixed
- MCP OAuth login no longer fails with "Authorization server response missing
crates/tui/plugins/computer-use.upstream-sha
+1 -1
View File
@@ -1 +1 @@
9f6c39f738c0d8e8dcc93af11af5e00d19081b60
249ae77fad9162c2af11d5460d91b2b5b909c06c
crates/tui/plugins/computer-use/mcp/server.mjs
+164 -24
View File
@@ -32,7 +32,11 @@ const backendCache = new Map();
const ROUTE_INSPECTION_TOOLS = new Set([
"request_access", "list_displays", "list_apps", "list_windows", "get_app_state", "screenshot",
"cursor_position", "read_clipboard", "recording_list", "recording_status",
"find_elements", "get_value",
]);
const STATE_CHAR_BUDGET = Number(process.env.CODEWHALE_CU_MAX_STATE_CHARS) > 0
? Number(process.env.CODEWHALE_CU_MAX_STATE_CHARS)
: 16_000;
/**
* Largest base64 image payload we will put in one JSON-RPC message. Hosts cap
* how much a stdio server may write between message boundaries (Claude Code
@@ -154,8 +158,15 @@ function rasterToPoints(computerId, x, y) {
*/
async function normalizeTarget(computer, target, kind, resolve, sink) {
if (target?.type === "coordinate") {
if (target.space === "screen") {
if (!Number.isFinite(target.x) || !Number.isFinite(target.y)) {
throw new ServerError("bad_target", "screen coordinates must be finite numbers");
}
return { x: Math.round(target.x), y: Math.round(target.y), strategy: "event", coordinate_space: "screen" };
}
if (target.x < 0 || target.y < 0) throw new ServerError("bad_target", "raster coordinates must be non-negative");
const pt = rasterToPoints(computer.id, target.x, target.y);
return { x: Math.round(pt.x), y: Math.round(pt.y), strategy: "event" };
return { x: Math.round(pt.x), y: Math.round(pt.y), strategy: "event", coordinate_space: "raster" };
}
if (target?.type === "element") {
const { state, element } = resolveElement(target);
@@ -240,21 +251,105 @@ function rememberState(computer, app_ref, result) {
return id;
}
function observeState(computer, app_ref, result, detail) {
function filterElements(elements, { detail, query, role, limit, offset, compact }) {
const full = detail === "full";
let rows = (elements ?? []).map((el, i) => ({ ...el, index: el.index ?? i }));
if (!full) {
rows = rows.filter((el) => el.windowIndex !== -1 || !Array.isArray(el.path) || el.path.length <= 1);
}
if (role) rows = rows.filter((el) => el.role === role);
if (query) {
const q = String(query).toLowerCase();
rows = rows.filter((el) => [el.label, el.value, el.role, el.subrole].some((v) => String(v ?? "").toLowerCase().includes(q)));
}
const matched = rows.length;
const start = Math.max(0, Number(offset) || 0);
const cap = limit != null ? Math.max(1, Math.min(200, Number(limit))) : null;
const sliced = cap != null ? rows.slice(start, start + cap) : rows.slice(start);
const view = sliced.map((el) => {
if (full) return el;
const { path, windowIndex, ...rest } = el;
if (!compact) return rest;
const label = rest.label != null ? String(rest.label).slice(0, 80) : rest.label;
const value = rest.value != null && String(rest.value).length > 200 ? String(rest.value).slice(0, 200) : rest.value;
return { index: rest.index, role: rest.role, label, value, focused: rest.focused, enabled: rest.enabled, actions: rest.actions };
});
return { elements: view, matched, offset: start, returned: view.length, truncated: start + view.length < matched };
}
function fitStatePayload(data, budget) {
let payload = data;
let json = JSON.stringify(payload);
if (json.length <= budget) return payload;
if (payload.ocr) {
payload = { ...payload, ocr: { status: payload.ocr.status ?? "omitted", omitted: true, reason: "ocr_too_large", note: "OCR omitted so this observation stays readable. Retry include_ocr with ocr_region, query, or a smaller window." } };
json = JSON.stringify(payload);
if (json.length <= budget) return { ...payload, truncated: true };
}
let elements = payload.elements ?? [];
const matched = payload.matched ?? elements.length;
while (elements.length > 4 && json.length > budget) {
elements = elements.slice(0, Math.max(4, Math.floor(elements.length / 2)));
payload = {
...payload,
elements,
truncated: true,
matched,
returned: elements.length,
next_offset: (payload.offset ?? 0) + elements.length,
note: "Observation truncated to keep the transport intact. Pass query, role, limit and offset; do not retry an unfiltered dump.",
};
json = JSON.stringify(payload);
}
return payload;
}
async function invokeType(invoke, prepared) {
const text = String(prepared.text ?? "");
const pressEnter = prepared.press_enter === true;
const parts = text.split(/\r\n|\n|\r/);
const rest = { ...prepared };
delete rest.press_enter;
if (parts.length === 1 && !pressEnter) return invoke("type", rest);
const steps = [];
for (let i = 0; i < parts.length; i++) {
if (parts[i]) steps.push(await invoke("type", { ...rest, text: parts[i] }));
if (i < parts.length - 1 || (pressEnter && i === parts.length - 1)) {
steps.push(await invoke("key", { text: "return" }));
}
}
const last = steps.at(-1) ?? { action_sent: true };
return { ...last, newlines_as_return: true, typed_parts: steps.length };
}
function observeState(computer, app_ref, result, args = {}) {
// Cache the complete backend records before making the model-facing view.
// Public indices still address those records, including their private AX
// paths; a compact response must never weaken live target revalidation.
const state_id = rememberState(computer, app_ref, result);
const full = detail === "full";
const elements = full ? result.elements : (result.elements ?? [])
.filter((el) => el.windowIndex !== -1 || !Array.isArray(el.path) || el.path.length <= 1)
.map(({ path, windowIndex, ...el }) => el);
return {
...result, state_id, elements, detail: full ? "full" : "summary",
note: "Target observed elements with {type:'element', state_id, index}; observe again after UI changes. " +
(full ? "" : "Summary keeps app content and top-level menus; use detail:'full' for nested menus and tree structure. ") +
"Missing labels or values are unknown; do not guess their contents.",
const compact = args.detail === "compact" || args.compact === true;
const detail = args.detail === "full" ? "full" : compact ? "compact" : "summary";
const filtered = filterElements(result.elements, {
detail: args.detail === "full" ? "full" : "summary",
query: args.query,
role: args.role,
limit: args.limit,
offset: args.offset,
compact,
});
const data = {
...result,
state_id,
elements: filtered.elements,
detail,
matched: filtered.matched,
offset: filtered.offset,
returned: filtered.returned,
truncated: filtered.truncated,
note: "Target observed elements with {type:'element', state_id, index}. Indices address the cached tree, including rows omitted from this page. Filter with query/role/limit/offset instead of requesting a larger dump. Missing labels or values are unknown; do not guess.",
};
if (compact && data.ocr && args.include_ocr !== true) delete data.ocr;
return fitStatePayload(data, STATE_CHAR_BUDGET);
}
// ---------- tool dispatch ----------
@@ -361,6 +456,38 @@ async function callTool(params) {
if (binding.needsObservation && !ROUTE_INSPECTION_TOOLS.has(name)) {
throw new ServerError("computer_observation_required", "Computer route changed — call screenshot or get_app_state on the registered target before acting");
}
if (name === "run_actions") {
const steps = args.steps;
if (!Array.isArray(steps) || steps.length < 1 || steps.length > 8) throw new ServerError("bad_args", "run_actions needs 1..8 steps");
const results = [];
for (const [i, step] of steps.entries()) {
if (!step || typeof step.tool !== "string") throw new ServerError("bad_args", `step ${i} needs a tool name`);
if (step.tool === "run_actions") throw new ServerError("bad_args", "run_actions cannot nest");
if (!TOOL_NAMES.has(step.tool)) throw new ServerError("unknown_tool", `unknown tool "${step.tool}"`);
const result = await callTool({ name: step.tool, arguments: { ...(step.arguments ?? {}), computer: computer.id } });
const body = JSON.parse(result.content[0].text);
results.push({ tool: step.tool, ok: body.ok !== false, receipt: body });
if (body.ok === false || result.isError) {
return { content: [{ type: "text", text: JSON.stringify(fail(computer, body.error?.code ?? "step_failed", body.error?.message ?? "step failed", { tool: "run_actions", switched, stopped_at: i, steps: results })) }], isError: true };
}
}
return { content: [{ type: "text", text: JSON.stringify(receipt(computer, { ok: true, tool: "run_actions", switched, steps: results })) }] };
}
if (name === "find_elements") {
const st = args.state_id ? appStates.get(args.state_id) : null;
if (args.state_id && !st) throw new ServerError("unknown_state", `state_id "${args.state_id}" is unknown or expired — call get_app_state again`);
if (st) {
if (st.computerId && st.computerId !== computer.id) {
throw new ServerError("state_wrong_computer", `state_id "${args.state_id}" belongs to computer "${st.computerId}", not "${computer.id}"`);
}
const filtered = filterElements(st.elements, {
detail: "summary", query: args.query, role: args.role,
limit: args.limit ?? 20, offset: args.offset ?? 0, compact: true,
});
return { content: [{ type: "text", text: JSON.stringify(receipt(computer, { ok: true, tool: "find_elements", switched, state_id: args.state_id, ...filtered, note: "Indices address the cached tree from this state_id." })) }] };
}
return callTool({ name: "get_app_state", arguments: { ...args, detail: "compact", limit: args.limit ?? 20, computer: computer.id } });
}
// Out-of-process runners (the desktop app for the local computer, the
// remote agent for ssh computers) get the request over the wire.
const backendMethod = BACKEND_METHOD[name] === "request_access" ? "probe" : BACKEND_METHOD[name];
@@ -390,16 +517,19 @@ async function callTool(params) {
// being resolved still blocks this dispatch.
if (controlStopped && !READ_ONLY_TOOLS.has(name)) throw new ServerError("control_stopped", "stop_computer_control is active; no further actions are permitted this session");
inFlight++;
let reply;
try {
dispatched = true;
reply = await ex.remote({ tool: backendMethod, args: wireArgs }, { timeoutMs: backendMethod.startsWith("recording") || backendMethod === "get_app_state" ? 60_000 : 30_000 });
const timeoutMs = backendMethod.startsWith("recording") || backendMethod === "get_app_state" ? 60_000 : 30_000;
const invoke = async (tool, a) => {
const r = await ex.remote({ tool, args: a }, { timeoutMs });
if (!r.ok) throw new ServerError(r.error?.code ?? "remote_error", r.error?.message ?? "remote agent failed");
return r.data;
};
data = name === "type" ? await invokeType(invoke, wireArgs) : await invoke(backendMethod, wireArgs);
} finally {
inFlight--;
}
if (!reply.ok) throw new ServerError(reply.error?.code ?? "remote_error", reply.error?.message ?? "remote agent failed");
await assertCurrentRoute(computer, binding, true);
data = reply.data;
if (Array.isArray(data)) data = { items: data };
if ((backendMethod === "screenshot" || backendMethod === "zoom") && data?.file) {
if (ex.filesLocal) bindRaster(computer, data);
@@ -411,7 +541,7 @@ async function callTool(params) {
}
if (backendMethod === "zoom") bindZoomRaster(computer, zoomParent, args.region, ex.filesLocal ? data?.file ?? data?.path : null);
if (name === "get_app_state") {
data = observeState(computer, wireArgs.app_ref, data, args.detail);
data = observeState(computer, wireArgs.app_ref, data, args);
}
if (backendMethod === "probe") Object.assign(data, { via: ex.kind, app: ex.app ?? null });
} else {
@@ -427,7 +557,9 @@ async function callTool(params) {
inFlight++;
try {
dispatched = true;
data = await backend[backendMethod](prepared);
data = name === "type"
? await invokeType((tool, a) => backend[BACKEND_METHOD[tool] ?? tool](a), prepared)
: await backend[backendMethod](prepared);
} finally {
inFlight--;
}
@@ -436,7 +568,7 @@ async function callTool(params) {
if (name === "screenshot") bindRaster(computer, data);
if (backendMethod === "zoom") bindZoomRaster(computer, zoomParent, args.region, data?.file ?? data?.path);
if (name === "get_app_state") {
data = observeState(computer, prepared.app_ref, data, args.detail);
data = observeState(computer, prepared.app_ref, data, args);
}
if (backendMethod === "probe" && computer.transport === "local") {
// Direct mode: permissions belong to whatever hosts this server. Say so.
@@ -450,7 +582,7 @@ async function callTool(params) {
const localFile = typeof ex?.remote !== "function" || ex.filesLocal;
bindRaster(computer, localFile ? data.ocr.raster : { ...data.ocr.raster, file: null, path: null });
}
data.ocr.note = "Recognized text may be imperfect. These coordinate targets belong to this captured image, not to accessibility elements; observe again after the UI changes.";
data.ocr.note = "Recognized text may be imperfect. These coordinate targets belong to this captured image, not to accessibility elements; observe again after the UI changes. Prefer ocr_region or query over a second full-window OCR.";
}
// Inline the raster only when it fits the budget. One oversized JSON-RPC
@@ -468,7 +600,7 @@ async function callTool(params) {
bytes: size,
encoded_bytes: encodedSize(size),
limit_bytes: INLINE_IMAGE_MAX_BYTES,
note: "The capture is on disk at the returned path, but inlining it would exceed this host's single-message budget and drop the connection. Capture one display, a region, or an app window, or zoom into part of this raster to get a viewable image.",
note: "The capture is on disk at the returned path, but inlining it would exceed this host's single-message budget and drop the connection. Capture one display, a region, or an app window, or call zoom on this raster to get a viewable image.",
};
} else {
const bytes = fs.readFileSync(file);
@@ -512,18 +644,26 @@ async function callTool(params) {
async function prepareArgs(computer, name, args, resolve, sink) {
const out = { ...args };
delete out.computer;
const semantic = new Set(["set_value", "select_text", "perform_action"]);
const semantic = new Set(["set_value", "select_text", "perform_action", "focus", "get_value"]);
for (const key of ["target", "from_target", "to"]) {
const given = out[key];
if (!given?.type) continue;
const kind = key === "target" && semantic.has(name) ? "semantic" : "pointer";
out[key] = { ...given, ...(await normalizeTarget(computer, given, kind, resolve, sink)) };
}
if (name === "get_app_state") {
if (out.detail != null && !["summary", "compact", "full"].includes(out.detail)) throw new ServerError("bad_args", "detail must be summary or full (compact is an alias for summary)");
out.detail = out.detail === "full" ? "full" : "summary";
if (name === "get_app_state" || name === "find_elements") {
if (out.detail != null && !["summary", "compact", "full"].includes(out.detail)) throw new ServerError("bad_args", "detail must be summary, compact or full");
if (name === "get_app_state") {
out.compact = out.detail === "compact";
out.detail = out.detail === "full" ? "full" : "summary";
}
if (out.include_ocr != null && typeof out.include_ocr !== "boolean") throw new ServerError("bad_args", "include_ocr must be true or false");
if (out.window_id != null && (!Number.isSafeInteger(out.window_id) || out.window_id < 0)) throw new ServerError("bad_args", "window_id must be a non-negative window index from list_windows");
if (out.limit != null && (!Number.isSafeInteger(out.limit) || out.limit < 1 || out.limit > 200)) throw new ServerError("bad_args", "limit must be an integer 1..200");
if (out.offset != null && (!Number.isSafeInteger(out.offset) || out.offset < 0)) throw new ServerError("bad_args", "offset must be a non-negative integer");
if (out.query != null && typeof out.query !== "string") throw new ServerError("bad_args", "query must be a string");
if (out.role != null && typeof out.role !== "string") throw new ServerError("bad_args", "role must be a string");
if (out.ocr_region != null && (!Array.isArray(out.ocr_region) || out.ocr_region.length !== 4)) throw new ServerError("bad_args", "ocr_region must be [x, y, w, h] in screen points");
}
return out;
}
crates/tui/plugins/computer-use/package.json
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "codewhale-cu",
"version": "0.3.1",
"version": "0.4.0",
"description": "Codewhale's included Computer Use plugin: accessibility, screenshots, keyboard and pointer control, and recording through the Engine's reviewed plugin authority.",
"license": "MIT",
"type": "module",
crates/tui/plugins/computer-use/plugin.json
+1 -1
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://agent-plugins.org/schemas/plugin.json",
"name": "computer-use",
"version": "0.3.1",
"version": "0.4.0",
"description": "Control apps with Codewhale. macOS beta; Windows and Linux backends are experimental and source-only.",
"author": {
"name": "Codewhale"
crates/tui/plugins/computer-use/skills/computer-use/SKILL.md
+54 -33
View File
@@ -49,28 +49,38 @@ Observe once, act once, then verify.
3. `get_app_state` defaults to a text-first summary (macOS AX / Windows
UIA / Linux AT-SPI / HarmonyOS uitest) with controls, values, actions,
layout, element indices and a `state_id`. Start here without a screenshot,
whether or not the model supports vision. Use `detail:"full"` for nested
menus and tree structure; `compact` remains a summary alias. Missing labels
or values mean unknown content, not something to guess.
4. If the tree contains the target, act on the element: `perform_action`
(AXPress/Invoke/click…), `set_value` for editable fields, element click.
whether or not the model supports vision. Pass `query`, `role`, `limit`
and `offset` instead of dumping the whole tree — truncated dumps hide the
title and search field. `detail:"compact"` is smaller (same indices,
shorter labels). `detail:"full"` adds nested menus and tree paths.
`find_elements` searches a cached `state_id` or observes now. Missing
labels or values mean unknown content, not something to guess. `get_value`
reads one field live.
4. If the tree contains the target, act on the element: `focus` then `type`
or `key` for composers, `set_value` for ordinary fields, `perform_action`
(AXPress/Invoke/click…) for advertised actions, element click. Newlines in
`type` are Return/Enter; `press_enter:true` sends after the text. Never
expect `\\n` to send a chat message. `run_actions` batches up to 8 steps
(click → type → key return → get_value).
macOS provides background element actions; Linux AT-SPI support depends
on the control. Windows currently refuses scoped semantic mutations.
Windows and Linux are development backends: do not assume their raw
input is background-safe or that native Pause/Stop controls are available.
5. When accessibility cannot read visible text, macOS supports
`get_app_state({app_ref, include_ocr:true})`. This explicitly captures the
selected app window and recognizes text locally, without a vision model or
remote service. Check `ocr.status`; recognized blocks include confidence,
pixel bounds and ready-to-use coordinate targets. OCR text is not a control
role or an advertised action. Verify uncertain text and observe again after
changes. Other platforms return an explicit unavailable status while keeping
their accessibility state usable. A text-only model must not infer unlabeled
icons, charts or other graphical meaning from OCR or a screenshot file path.
With vision, when accessibility cannot express the target: `screenshot` (optionally
`zoom` for small targets) and act with a coordinate target. Coordinates are
pixels **in the latest returned raster** for that computer; the server maps
them to screen points. After a new screenshot, old pixels are stale.
`get_app_state({app_ref, include_ocr:true})`. Pass `ocr_region:[x,y,w,h]`
in screen points to recognize one rect instead of the whole window. This
captures locally, without a vision model. Check `ocr.status`; recognized
blocks include confidence, pixel bounds and ready-to-use coordinate
targets. OCR text is not a control role or an advertised action. Verify
uncertain text and observe again after changes. Other platforms return an
explicit unavailable status while keeping their accessibility state usable.
A text-only model must not infer unlabeled icons, charts or other graphical
meaning from OCR or a screenshot file path.
With vision, when accessibility cannot express the target: `screenshot`
(optionally `zoom` for small targets) and act with a coordinate target.
Default coordinates are pixels **in the latest returned raster**. Pass
`space:"screen"` to send absolute screen points from the AX tree and skip
conversion. After a new screenshot, old raster pixels are stale.
If the host reports an omitted or oversized image, capture a smaller app
window/region or zoom, then use that returned raster. Do not guess from a
file path or reuse coordinates from an image the model never received.
@@ -92,9 +102,11 @@ Observe once, act once, then verify.
(`state_wrong_computer`).
- Coordinate: `{"type":"coordinate","x":496,"y":331}` — pixels from the latest
raster only; submit `x`/`y` unchanged, never transform them yourself.
`zoom` returns a bindable raster of its own: after zooming, coordinates are
pixels in the zoomed image. Points outside the bound raster fail
`target_outside_raster` instead of landing somewhere unintended.
`{"type":"coordinate","x":100,"y":200,"space":"screen"}` is an absolute
screen point (what AX `position` uses). `zoom` returns a bindable raster of
its own: after zooming, raster coordinates are pixels in the zoomed image.
Points outside the bound raster fail `target_outside_raster` instead of
landing somewhere unintended.
- Never translate pixels into an element target; never invent `state_id`s.
## Raw input reality (read before clicking)
@@ -102,20 +114,25 @@ Observe once, act once, then verify.
- macOS: call `open_application` with `activate:false` to bind input to the
intended process, even when the app is already running; pass `pid` when two
processes share a bundle id. Then the two halves behave differently:
- **Keyboard and element actions are quiet.** `type`, `key`, `set_value`,
`select_text` and `perform_action` reach the bound process without moving
the pointer or changing the foreground. Prefer them. Text entry uses writable
accessibility selection when available; verify the resulting value.
`get_app_state`, `list_windows` and `screenshot` default to the selected app.
- **Keyboard and element actions are quiet.** `type`, `key`, `focus`,
`set_value`, `get_value`, `select_text` and `perform_action` reach the
bound process without moving the pointer or changing the foreground.
Prefer them. Text entry uses writable accessibility selection when
available; verify the resulting value. `get_app_state`, `list_windows`
and `screenshot` default to the selected app.
- **Background mode never takes the shared pointer.** A coordinate
`left_click` first tries the bound application's accessibility action,
including field focus and row selection. `right_click` uses advertised
context-menu actions. `scroll` uses the target's accessibility scrollbar;
prefer a scroll-area element and read the receipt's unit and value change.
Raw double/triple/middle click, drag and hover fail with
including focusing a field that is not AXPressable. `right_click` uses
advertised context-menu actions. `scroll` uses the target's accessibility
scrollbar; prefer a scroll-area element and read the receipt's unit and
value change. If accessibility cannot act, `strategy:"app"` posts a
pointer event only when the point is inside the bound app's window, then
restores the cursor — not a global desktop click. Raw double/triple/middle
click, drag, hover, and `strategy:"event"` fail with
`shared_pointer_required` before moving the cursor. Missing semantic
scrolling or context-menu support is a refusal, never permission to activate.
Use another advertised accessibility action or a separate computer.
scrolling or context-menu support is a refusal, never permission to
activate. Use `strategy:"app"`, another advertised accessibility action,
or a separate computer.
- Shared-desktop gestures and foreground keyboard delivery require explicit
user authorization for exclusive desktop use, followed by
`open_application(activate:true)`. Do not select it merely to work around a
@@ -145,8 +162,12 @@ Observe once, act once, then verify.
## Keyboard
- macOS uses `cmd` (`cmd+c`), Linux/Windows use `ctrl` (`ctrl+c`).
- `key` for chords and repeats, `hold_key` for a duration, `type` for text.
- Prefer `set_value` on editable elements over typing.
- `key` is the key-press tool: `return`, `enter`, `backspace`, `tab`,
`escape`, chords and repeats. `hold_key` holds for a duration.
- `type` sends unicode. Newlines and `press_enter` become Return; they do
not insert a literal line break or U+FFFC.
- Prefer `set_value` on ordinary fields; prefer `focus` then `type`/`key`
on chat composers.
## Recording
crates/tui/plugins/computer-use/src/app-handler.mjs
+2 -2
View File
@@ -11,7 +11,7 @@ export const ALLOWED = new Set([
"open_application", "get_app_state", "resolve_element", "screenshot", "zoom",
"left_click", "double_click", "triple_click", "right_click", "middle_click",
"mouse_move", "left_click_drag", "left_mouse_down", "left_mouse_up", "scroll",
"type", "key", "hold_key", "set_value", "select_text", "perform_action",
"type", "key", "hold_key", "set_value", "focus", "get_value", "select_text", "perform_action",
"read_clipboard", "write_clipboard", "cursor_position",
"recordingStart", "recordingStop", "recordingStatus", "recordingList",
]);
@@ -20,7 +20,7 @@ const backends = new Map();
const heldPointers = new Map();
const INPUT_MUTATIONS = new Set([
"open_application", "left_click", "double_click", "triple_click", "right_click", "middle_click", "mouse_move",
"left_click_drag", "left_mouse_down", "left_mouse_up", "scroll", "type", "key", "hold_key", "set_value", "select_text", "perform_action",
"left_click_drag", "left_mouse_down", "left_mouse_up", "scroll", "type", "key", "hold_key", "set_value", "focus", "select_text", "perform_action",
]);
let queue = Promise.resolve();
crates/tui/plugins/computer-use/src/backends/darwin-accessibility.m
+24 -6
View File
@@ -232,6 +232,10 @@ static NSString *cuClickAction(AXUIElementRef el, BOOL context) {
if([actions containsObject:@"AXPress"]) return @"AXPress";
if([role isEqual:@"AXMenuItem"] && [actions containsObject:@"AXPick"]) return @"AXPick";
if([@[@"AXRow",@"AXCell"] containsObject:role] && cuSettable(el,@"AXSelected")) return @"AXSelected";
// Qt and other custom composers often expose AXFocused without AXTextField
// or AXPress. Focusing is the missing middle between "not pressable" and
// a global pointer click.
if(cuSettable(el,@"AXFocused")) return @"AXFocused";
return nil;
}
static NSDictionary *cuClick(AXUIElementRef el, BOOL context) {
@@ -457,10 +461,10 @@ static NSDictionary *windowAtPoint(NSArray *windows, CGPoint p) {
static id execute(NSDictionary *p) {
NSString *tool=p[@"tool"]; NSDictionary *args=p[@"args"]?:@{};
if([tool isEqual:@"pointer_sequence"] && ![args[@"foreground_input"] boolValue])
@throw [NSException exceptionWithName:@"shared_pointer_required" reason:@"shared macOS pointer input is unavailable in background mode; use an accessibility action or a separate computer" userInfo:nil];
if([tool isEqual:@"pointer_sequence"] && ![args[@"foreground_input"] boolValue] && ![args[@"app_scoped"] boolValue])
@throw [NSException exceptionWithName:@"shared_pointer_required" reason:@"shared macOS pointer input is unavailable in background mode; use an accessibility action, strategy 'app' for a click inside the bound window, or a separate computer" userInfo:nil];
cuOwnerPipe=[args[@"owner_pipe"] boolValue];
BOOL mutates=[@[@"type",@"key_event",@"mouse_event",@"scroll",@"pointer_sequence",@"release_input",@"set_value",@"select_text",@"perform_action",@"click_element",@"scroll_element"] containsObject:tool]
BOOL mutates=[@[@"type",@"key_event",@"mouse_event",@"scroll",@"pointer_sequence",@"release_input",@"set_value",@"focus_element",@"select_text",@"perform_action",@"click_element",@"scroll_element"] containsObject:tool]
|| ([tool isEqual:@"hit_test"] && [args[@"perform"] boolValue])
|| ([tool isEqual:@"app_info"] && [args[@"activate"] boolValue]);
if([tool isEqual:@"release_input"]) {
@@ -707,7 +711,9 @@ static id execute(NSDictionary *p) {
cuCheckCancelled();
// Activation is a separate, explicit operation. A stale foreground mode
// must never reclaim focus after the user has switched applications.
cuRequireForeground(inputApp);
// App-scoped clicks stay inside the bound window and do not steal the
// foreground; they still move the real cursor and restore it.
if([args[@"foreground_input"] boolValue]) cuRequireForeground(inputApp);
// AppKit only assembles a drag out of events that look like they came from
// the input hardware; a NULL-source stream delivers down and up but drops
// every mouseDragged in between.
@@ -715,7 +721,7 @@ static id execute(NSDictionary *p) {
BOOL held[3]={NO,NO,NO};
CGPoint last=home;
for(NSDictionary *step in args[@"steps"]) {
@try { cuCheckCancelled(); cuRequireForeground(inputApp); } @catch(NSException *e) { cuCancelled=1; break; }
@try { cuCheckCancelled(); if([args[@"foreground_input"] boolValue]) cuRequireForeground(inputApp); } @catch(NSException *e) { cuCancelled=1; break; }
CGEventRef event;
if(step[@"scroll"]) {
NSArray *d=step[@"scroll"];
@@ -808,11 +814,23 @@ static id execute(NSDictionary *p) {
if([tool isEqual:@"click_element"]) return cuClick((__bridge AXUIElementRef)el,[args[@"context"] boolValue]);
if([tool isEqual:@"scroll_element"]) return cuScroll((__bridge AXUIElementRef)el,args);
AXError e=kAXErrorFailure;
if([tool isEqual:@"get_value"]) {
id v=attr((__bridge AXUIElementRef)el,@"AXValue");
return @{@"ok":@YES,@"strategy":@"a11y",@"value":v?:[NSNull null],@"role":attr((__bridge AXUIElementRef)el,@"AXRole")?:[NSNull null]};
}
if([tool isEqual:@"set_value"]) e=AXUIElementSetAttributeValue((__bridge AXUIElementRef)el,kAXValueAttribute,(__bridge CFTypeRef)args[@"value"]);
else if([tool isEqual:@"focus_element"]) e=AXUIElementSetAttributeValue((__bridge AXUIElementRef)el,kAXFocusedAttribute,kCFBooleanTrue);
else if([tool isEqual:@"select_text"]){ NSArray *r=args[@"text_range"]?:@[@0,@0]; if(r.count!=2 || [r[0] longValue]<0 || [r[1] longValue]<0) @throw [NSException exceptionWithName:@"range" reason:@"text_range must be [start, length], both nonnegative" userInfo:nil]; CFRange range=CFRangeMake([r[0] longValue],[r[1] longValue]); AXValueRef v=AXValueCreate(kAXValueCFRangeType,&range); e=AXUIElementSetAttributeValue((__bridge AXUIElementRef)el,kAXSelectedTextRangeAttribute,v); CFRelease(v); }
else if([tool isEqual:@"perform_action"]){ CFArrayRef actions=NULL; AXUIElementCopyActionNames((__bridge AXUIElementRef)el,&actions); NSArray *names=CFBridgingRelease(actions); if(![names containsObject:args[@"action"]]) @throw [NSException exceptionWithName:@"action" reason:@"action is not advertised by this element" userInfo:nil]; cuCheckCancelled(); e=AXUIElementPerformAction((__bridge AXUIElementRef)el,(__bridge CFStringRef)args[@"action"]); }
if(e!=kAXErrorSuccess) @throw [NSException exceptionWithName:@"action" reason:[NSString stringWithFormat:@"accessibility action failed: %d",e] userInfo:nil];
return @{@"action_sent":@YES,@"strategy":@"a11y"};
NSMutableDictionary *done=[@{@"action_sent":@YES,@"strategy":@"a11y"} mutableCopy];
if([tool isEqual:@"focus_element"]) done[@"focused"]=@YES;
if([tool isEqual:@"set_value"]) {
id after=attr((__bridge AXUIElementRef)el,@"AXValue");
done[@"after"]=after?:[NSNull null];
done[@"verified"]=@([after isKindOfClass:NSString.class] && [after isEqual:args[@"value"]]);
}
return done;
} @finally { CFRelease(app); }
}
int main(int argc, const char **argv){ @autoreleasepool {
crates/tui/plugins/computer-use/src/backends/darwin.mjs
+26 -9
View File
@@ -142,7 +142,7 @@ export function create({ exec }) {
}
async function native(tool, args = {}) {
if (tool === "pointer_sequence") requireSharedPointer();
if (tool === "pointer_sequence" && !args.app_scoped) requireSharedPointer();
const helper = await nativeHelper();
const r = await runL(helper, [JSON.stringify({ tool, args: { ...args, input_app_ref: state.inputApp, foreground_input: state.foregroundInput, owner_pipe: true } })], { timeoutMs: 20_000, ownerPipe: true });
if (r.aborted || r.timedOut || r.code !== 0) {
@@ -150,7 +150,7 @@ export function create({ exec }) {
if (r.aborted) error.code = "cancelled";
// A deterministic native refusal sent no input. A killed/timed-out
// helper may have posted the press before losing its response.
const postsPress = (tool === "key_event" && args.down) || ["type", "perform_action", "click_element", "scroll_element", "set_value", "select_text"].includes(tool) || (tool === "hit_test" && args.perform) || (tool === "pointer_sequence" && args.steps?.some((step) => [1, 3, 25].includes(step.type)));
const postsPress = (tool === "key_event" && args.down) || ["type", "perform_action", "click_element", "scroll_element", "set_value", "focus_element", "select_text"].includes(tool) || (tool === "hit_test" && args.perform) || (tool === "pointer_sequence" && args.steps?.some((step) => [1, 3, 25].includes(step.type)));
error.inputMayHaveBeenSent = postsPress && r.spawned === true && (r.aborted || r.timedOut);
if (error.inputMayHaveBeenSent) error.message += "; input may already have been sent — observe the target before doing anything else";
throw error;
@@ -266,7 +266,7 @@ export function create({ exec }) {
*/
async function pointerClick(button, x, y, clicks, strategy = "auto") {
assertInScreen(x, y);
if (!["auto", "a11y", "event"].includes(strategy)) throw new ExecError(`strategy must be auto, a11y or event (got ${JSON.stringify(strategy)})`);
if (!["auto", "a11y", "event", "app"].includes(strategy)) throw new ExecError(`strategy must be auto, a11y, app or event (got ${JSON.stringify(strategy)})`);
let a11yReason = null;
if (strategy !== "event" && ["left", "right"].includes(button) && clicks === 1) {
if (button === "right") await requireBackgroundActions();
@@ -277,11 +277,25 @@ export function create({ exec }) {
}
a11yReason = hit?.reason ?? "not_found";
if (strategy === "a11y") {
throw new ExecError(`no supported accessibility click at (${x}, ${y}) in the bound application (${a11yReason}) — observe the available actions or use a separate computer`);
throw new ExecError(`no supported accessibility click at (${x}, ${y}) in the bound application (${a11yReason}) — observe the available actions, use strategy "app" for a window-scoped pointer click, or a separate computer`);
}
} else if (strategy === "a11y") {
throw new ExecError(`strategy "a11y" is only available for a left single click on this backend; ${mouseName(button)} x${clicks} has no accessibility equivalent`);
}
if (strategy === "app" || (strategy === "auto" && !state.foregroundInput)) {
if (strategy !== "app") {
// auto in background still fails closed for raw pointer; app is the
// explicit missing middle.
requireSharedPointer();
}
const owner = await assertOwnsPoint(x, y);
const r = await native("pointer_sequence", { steps: clickSteps(button, x, y, clicks), restore: true, app_scoped: true });
const last = { x, y };
state.pointer = last;
return { action_sent: true, strategy: "app-pointer", input_scope: "application-window",
at: last, button, clicks, window: { id: owner.window_id, owner_pid: owner.owner_pid },
...pointerCost(r), ...(a11yReason ? { a11y_reason: a11yReason } : {}) };
}
const r = await gesture(clickSteps(button, x, y, clicks), { restore: true, guard: { x, y } });
return { action_sent: true, strategy: "event", at: { x, y }, button, clicks, ...pointerCost(r),
...(a11yReason ? { a11y_reason: a11yReason } : {}) };
@@ -613,7 +627,7 @@ export function create({ exec }) {
list_apps: listApps,
list_windows: listWindows,
open_application: openApplication,
get_app_state: async ({ app_ref, detail, depth, window_id, include_ocr = false }) => {
get_app_state: async ({ app_ref, detail, depth, window_id, include_ocr = false, ocr_region } = {}) => {
const t = await native("get_app_state", { app_ref: app_ref === undefined ? state.inputApp ?? undefined : app_ref, detail, window_id });
if (!t.found) throw new ExecError("application not found — call list_apps for exact names/pids");
if (include_ocr) {
@@ -630,8 +644,9 @@ export function create({ exec }) {
const ocrDir = path.join(recordingsDir(), "captures");
fs.mkdirSync(ocrDir, { recursive: true });
raster = await screenshot({
app_ref: { pid: t.pid, ...(t.bundle_id ? { bundle_id: t.bundle_id } : {}) },
window_id,
...(ocr_region
? { region: ocr_region }
: { app_ref: { pid: t.pid, ...(t.bundle_id ? { bundle_id: t.bundle_id } : {}) }, window_id }),
path: path.join(ocrDir, `ocr-${crypto.randomBytes(4).toString("hex")}.png`),
});
const ocr = await native("recognize_text", { file: raster.file });
@@ -663,8 +678,8 @@ export function create({ exec }) {
screenshot,
zoom,
left_click: async ({ target, strategy = "auto" }) => {
if (target.type !== "element" || strategy === "event") return pointerClick("left", target.x, target.y, 1, strategy);
if (!["auto", "a11y"].includes(strategy)) throw new ExecError(`strategy must be auto, a11y or event (got ${JSON.stringify(strategy)})`);
if (target.type !== "element" || strategy === "event" || strategy === "app") return pointerClick("left", target.x, target.y, 1, strategy);
if (!["auto", "a11y"].includes(strategy)) throw new ExecError(`strategy must be auto, a11y, app or event (got ${JSON.stringify(strategy)})`);
try {
assertBoundElement(target);
if ((await native("input_capabilities"))?.element_identity !== 1) throw new ExecError("native helper needs an update for element identity validation");
@@ -784,6 +799,8 @@ export function create({ exec }) {
return { action_sent: true, key, keyboard_delivery: state.foregroundInput ? "foreground-guarded" : "process", heldSec: d };
},
set_value: (args) => native("set_value", args),
focus: (args) => native("focus_element", args),
get_value: (args) => native("get_value", args),
select_text: (args) => native("select_text", args),
perform_action: (args) => native("perform_action", args),
read_clipboard: readClipboard,
crates/tui/plugins/computer-use/src/tools.mjs
+66 -11
View File
@@ -7,8 +7,8 @@ const computerParam = {
};
const strategyParam = {
enum: ["auto", "a11y", "event"],
description: "macOS auto (default): element targets press that exact revalidated element and fail closed, with no coordinate fallback; coordinate targets hit-test the point for an accessibility press, falling back to a guarded raw event. a11y: require an accessibility press and fail closed otherwise. event: force the guarded raw pointer event. Other platforms use raw events. action_sent confirms dispatch, not the effect; observe again before deciding another action.",
enum: ["auto", "a11y", "event", "app"],
description: "macOS auto (default): element targets press that exact revalidated element and fail closed, with no coordinate fallback; coordinate targets hit-test the point for an accessibility press, including focus of a field that is not AXPressable. a11y: require an accessibility press or focus and fail closed otherwise. app: if accessibility cannot act, post a pointer event only when the point is inside the bound app's window, then restore the cursor — never a global desktop click. event: force the guarded raw pointer event (shared-desktop / activate:true). Other platforms use raw events. action_sent confirms dispatch, not the effect; observe again before deciding another action.",
};
const targetSchema = {
@@ -30,8 +30,9 @@ const targetSchema = {
required: ["type", "x", "y"],
properties: {
type: { const: "coordinate" },
x: { type: "integer", minimum: 0 },
y: { type: "integer", minimum: 0 },
x: { type: "integer" },
y: { type: "integer" },
space: { enum: ["raster", "screen"], description: "raster (default): pixels in the latest screenshot/OCR/zoom. screen: absolute screen points; do not convert them yourself." },
},
additionalProperties: false,
},
@@ -122,8 +123,13 @@ export const TOOLS = [
properties: {
app_ref: { type: "object", properties: { pid: { type: "integer" }, name: { type: "string" }, bundle_id: { type: "string" } }, additionalProperties: false, description: "macOS accepts PID, name and bundle identity. Linux accepts only a unique exact AT-SPI app name. Windows accepts only a unique exact window title in name (from list_windows.title). HarmonyOS rejects explicit app selectors." },
window_id: { type: "integer", description: "macOS only: zero-based window index within the app. Other platforms reject this selector." },
detail: { enum: ["summary", "compact", "full"], default: "summary", description: "Summary is the concise default; full includes nested menus and internal tree structure. Compact is a compatibility alias for summary." },
include_ocr: { type: "boolean", default: false, description: "On macOS, also recognize visible text locally from the selected app window. Requires Screen Recording permission. Returns text, confidence and raster coordinate targets for UI that accessibility cannot read; no vision model is required." },
detail: { enum: ["summary", "compact", "full"], default: "summary", description: "Summary is the concise default (controls, values, actions, layout). compact is smaller: same indices, shorter labels, no nested menus. full includes nested menus and tree paths." },
query: { type: "string", description: "Case-insensitive substring over label, value and role. Use this instead of downloading the whole tree." },
role: { type: "string", description: "Exact accessibility role filter, e.g. AXButton, AXTextField." },
limit: { type: "integer", minimum: 1, maximum: 200, description: "Max elements to return after filtering. Prefer this over a second unfiltered dump." },
offset: { type: "integer", minimum: 0, description: "Skip this many matching elements (pagination)." },
include_ocr: { type: "boolean", default: false, description: "On macOS, also recognize visible text locally from the selected app window. Requires Screen Recording permission. Returns text, confidence and raster coordinate targets for UI that accessibility cannot read; no vision model is required. Do not combine with compact unless you need the blocks." },
ocr_region: { type: "array", items: { type: "number" }, minItems: 4, maxItems: 4, description: "[x, y, w, h] in screen points. When include_ocr is true, recognize only this rect instead of the whole window." },
computer: computerParam,
},
additionalProperties: false,
@@ -220,11 +226,11 @@ export const TOOLS = [
},
// ---- text & keyboard ----
{
name: "type", description: "Type text into the focused control (unicode). Focus the field first (click/element action). On macOS the receipt carries `verified:true` only when the focused control's value actually reflects the typed text; on `verified:false` the text may have gone nowhere — confirm with a screenshot before relying on it.",
inputSchema: { type: "object", required: ["text"], properties: { text: { type: "string" }, computer: computerParam }, additionalProperties: false },
name: "type", description: "Type unicode text into the focused control. Newlines in `text` are Return/Enter key presses, not literal characters — never put \\n in a composer by hoping it will send. Focus the field first (click, focus, or set_value). On macOS the receipt carries `verified:true` only when the focused control's value actually reflects the typed text; on `verified:false` the text may have gone nowhere — observe again before relying on it.",
inputSchema: { type: "object", required: ["text"], properties: { text: { type: "string" }, press_enter: { type: "boolean", description: "After typing, press Return/Enter once. Prefer this to putting a newline in `text` when you want to send." }, computer: computerParam }, additionalProperties: false },
},
{
name: "key", description: "Press a key or chord, e.g. 'return', 'cmd+c' (macOS), 'ctrl+c' (Linux/Windows). Repeat with `repeat`.",
name: "key", description: "Press a named key or chord. Examples: return, enter, backspace, tab, escape, cmd+c (macOS), ctrl+c (Linux/Windows). This is the key-press tool; type() cannot send modifiers or Return by itself except via newlines/press_enter. Repeat with `repeat`.",
inputSchema: { type: "object", required: ["text"], properties: { text: { type: "string" }, repeat: { type: "integer", minimum: 1, maximum: 100 }, computer: computerParam }, additionalProperties: false },
},
{
@@ -235,6 +241,54 @@ export const TOOLS = [
name: "set_value", description: "Set an editable element's value through the accessibility layer (background-safe, no keystrokes). Element targets only.",
inputSchema: { type: "object", required: ["target", "value"], properties: { target: targetSchema, value: { type: "string" }, computer: computerParam }, additionalProperties: false },
},
{
name: "focus", description: "Focus an observed element through the accessibility layer (background-safe). Prefer this before type() on composers that ignore AXPress.",
inputSchema: { type: "object", required: ["target"], properties: { target: targetSchema, computer: computerParam }, additionalProperties: false },
},
{
name: "get_value", description: "Read the live accessibility value of an observed element (text fields, sliders). Prefer this over dumping the whole tree.",
inputSchema: { type: "object", required: ["target"], properties: { target: targetSchema, computer: computerParam }, additionalProperties: false },
},
{
name: "find_elements", description: "Search the latest get_app_state (or take a fresh one) for elements matching query/role without returning the full dump.",
inputSchema: {
type: "object",
properties: {
query: { type: "string" },
role: { type: "string" },
state_id: { type: "string", description: "Reuse a previous observation; omit to observe now." },
limit: { type: "integer", minimum: 1, maximum: 100 },
app_ref: { type: "object", properties: { pid: { type: "integer" }, name: { type: "string" }, bundle_id: { type: "string" } }, additionalProperties: false },
computer: computerParam,
},
additionalProperties: false,
},
},
{
name: "run_actions", description: "Run up to 8 computer-use tools in order on this computer. Stops on the first failure. Each step is {tool, arguments}. Use for click→type→key(return)→get_value without extra round trips.",
inputSchema: {
type: "object",
required: ["steps"],
properties: {
steps: {
type: "array",
minItems: 1,
maxItems: 8,
items: {
type: "object",
required: ["tool"],
properties: {
tool: { type: "string" },
arguments: { type: "object" },
},
additionalProperties: false,
},
},
computer: computerParam,
},
additionalProperties: false,
},
},
{
name: "select_text", description: "Select a text range [start, length] in an element, or place the caret when omitted.",
inputSchema: { type: "object", properties: { target: targetSchema, text_range: { type: "array", items: { type: "integer" }, minItems: 2, maxItems: 2 }, computer: computerParam }, additionalProperties: false },
@@ -302,6 +356,7 @@ export const TOOL_NAMES = new Set(TOOLS.map((t) => t.name));
/** Tools that never touch a computer (available even after kill switch). */
export const READ_ONLY_TOOLS = new Set([
"computer_list", "stop_computer_control", "wait", "request_access", "recording_list", "recording_status",
"find_elements", "get_value",
]);
/** Tools dispatchable to a remote agent over ssh (allow-list must match agent.mjs). */
@@ -310,14 +365,14 @@ export const REMOTE_TOOLS = new Set([
"open_application", "get_app_state", "resolve_element", "screenshot", "zoom",
"left_click", "double_click", "triple_click", "right_click", "middle_click",
"mouse_move", "left_click_drag", "left_mouse_down", "left_mouse_up", "scroll",
"type", "key", "hold_key", "set_value", "select_text", "perform_action",
"type", "key", "hold_key", "set_value", "focus", "get_value", "select_text", "perform_action",
"read_clipboard", "write_clipboard", "cursor_position",
"recordingStart", "recordingStop", "recordingStatus", "recordingList",
]);
/** Map public tool name -> backend method name. */
export const BACKEND_METHOD = Object.fromEntries(
TOOLS.filter((t) => !["computer_list", "computer_switch", "computer_register", "computer_remove", "stop_computer_control", "wait"].includes(t.name))
TOOLS.filter((t) => !["computer_list", "computer_switch", "computer_register", "computer_remove", "stop_computer_control", "wait", "find_elements", "run_actions"].includes(t.name))
.map((t) => [t.name, {
recording_start: "recordingStart",
recording_stop: "recordingStop",
crates/tui/plugins/computer-use/tests/darwin.test.mjs
+15 -2
View File
@@ -49,7 +49,7 @@ test('native summary keeps text and top-level menus without spending the UI budg
[{AXRole:'AXMenuItem',actions:['AXPick']},false,'AXPick'],
[{AXRole:'AXButton',actions:['AXShowMenu']},true,'AXShowMenu'],
[{AXRole:'AXButton',AXEnabled:false,actions:['AXPress']},false,null],
[{AXRole:'AXGroup',settable:['AXFocused']},false,null],
[{AXRole:'AXGroup',settable:['AXFocused']},false,'AXFocused'],
]) {
const r=spawnSync(binary,[JSON.stringify({tool:'inspect_click_action',args:{element,context}})],{encoding:'utf8'});
assert.equal(r.status,0,r.stderr);
@@ -284,6 +284,19 @@ test('macOS failed activation cannot leave a previous shared-desktop binding arm
assert.ok(!calls.some(r=>r.tool==='pointer_sequence'));
});
test('macOS strategy app posts a window-scoped pointer after accessibility misses', async t => {
const {backend,calls}=stubBackend(t,r=>r.tool==='input_capabilities'?{input_lease:1,element_identity:1,background_actions:1}:r.tool==='hit_test'?NOT_PRESSABLE:null);
await backend.open_application({name:'Fixture'});
const receipt=await backend.left_click({target:{x:70,y:80},strategy:'app'});
assert.equal(receipt.strategy,'app-pointer');
assert.equal(receipt.input_scope,'application-window');
assert.equal(receipt.action_sent,true);
assert.ok(calls.some(r=>r.tool==='window_at_point'));
const gesture=calls.find(r=>r.tool==='pointer_sequence');
assert.equal(gesture.args.app_scoped,true);
assert.equal(gesture.args.foreground_input,false);
});
test('macOS background control never escalates an unavailable semantic action to shared pointer input', async t => {
const {backend,calls}=stubBackend(t,r=>r.tool==='input_capabilities'?{input_lease:1,element_identity:1,background_actions:1}:r.tool==='hit_test'?NOT_PRESSABLE:null);
const binding=await backend.open_application({name:'Fixture'});
@@ -533,7 +546,7 @@ test('macOS click strategies: event skips the tree and a11y fails closed', async
assert.ok(!calls.some((c) => c.tool === 'hit_test'), 'strategy=event never hit-tests');
await assert.rejects(backend.left_click({ target: { x: 10, y: 20 }, strategy: 'a11y' }), /no supported accessibility click/);
await assert.rejects(backend.left_click({ target: { x: 10, y: 20 }, strategy: 'sideways' }), /strategy must be auto, a11y or event/);
await assert.rejects(backend.left_click({ target: { x: 10, y: 20 }, strategy: 'sideways' }), /strategy must be auto, a11y, app or event/);
calls.length = 0;
const dbl = await backend.double_click({ target: { x: 10, y: 20 } });
crates/tui/plugins/computer-use/tests/fixtures/fake-backend.mjs
Vendored
+5 -1
View File
@@ -73,7 +73,11 @@ export function create() {
async double_click({ target } = {}) { record("double_click", { target }); return { action_sent: true, at: { x: target?.x, y: target?.y } }; },
async mouse_move({ target } = {}) { record("mouse_move", { target }); return { action_sent: true, at: { x: target?.x, y: target?.y } }; },
async perform_action(args) { record("perform_action", args); return { action_sent: true, strategy: "a11y" }; },
async set_value(args) { record("set_value", args); return { action_sent: true, strategy: "a11y" }; },
async set_value(args) { record("set_value", args); return { action_sent: true, strategy: "a11y", verified: true, after: args.value }; },
async type(args) { record("type", args); return { action_sent: true, text: args.text, verified: true }; },
async key(args) { record("key", args); return { action_sent: true, key: args.text ?? "return" }; },
async focus(args) { record("focus", args); return { action_sent: true, focused: true, strategy: "a11y" }; },
async get_value(args) { record("get_value", args); return { value: "Fixture text", strategy: "a11y" }; },
};
}
crates/tui/plugins/computer-use/tests/server-protocol.test.mjs
+1 -1
View File
@@ -114,7 +114,7 @@ test("tools/list exposes the full frontier surface with valid schemas", async ()
const names = new Set(tools.map((t) => t.name));
for (const required of ["screenshot", "zoom", "left_click", "double_click", "triple_click", "right_click", "middle_click",
"mouse_move", "left_click_drag", "left_mouse_down", "left_mouse_up", "scroll", "type", "key", "hold_key",
"set_value", "select_text", "perform_action", "get_app_state", "list_apps", "list_windows", "list_displays",
"set_value", "focus", "get_value", "find_elements", "run_actions", "select_text", "perform_action", "get_app_state", "list_apps", "list_windows", "list_displays",
"switch_display", "open_application", "read_clipboard", "write_clipboard", "cursor_position", "wait",
"recording_start", "recording_stop", "recording_status", "recording_list",
"computer_list", "computer_switch", "computer_register", "computer_remove", "request_access", "stop_computer_control"]) {
crates/tui/plugins/computer-use/tests/server-targets.test.mjs
+55 -2
View File
@@ -103,7 +103,7 @@ test("app-state arguments distinguish an omitted reference from an explicit null
});
test("summary preserves readable UI and original target indices while full retains tree structure", async () => {
for (const detail of [undefined, "summary", "compact"]) {
for (const detail of [undefined, "summary"]) {
const state = await tool("get_app_state", { detail });
assert.equal(state.detail, "summary");
assert.deepEqual(state.elements.map(e => e.index), [0, 1, 2, 3, 6, 7, 8]);
@@ -132,6 +132,59 @@ test("summary preserves readable UI and original target indices while full retai
assert.equal((await tool("get_app_state", { include_ocr: "yes" })).error.code, "bad_args");
});
test("compact and query return a filterable page instead of the whole tree", async () => {
const compact = await tool("get_app_state", { detail: "compact" });
assert.equal(compact.detail, "compact");
assert.equal(compact.ocr, undefined);
const field = compact.elements.find(e => e.index === 8);
assert.equal(field.role, "AXTextField");
assert.equal(field.value, "Fixture text");
assert.equal(field.position, undefined);
const found = await tool("find_elements", { query: "whale", state_id: compact.state_id });
assert.equal(found.ok, true);
assert.equal(found.matched, 0);
const buttons = await tool("find_elements", { role: "AXButton", state_id: compact.state_id });
assert.equal(buttons.matched, 1);
assert.equal(buttons.elements[0].label, "OK");
const paged = await tool("get_app_state", { limit: 2, offset: 0 });
assert.equal(paged.returned, 2);
assert.equal(paged.matched, 7);
assert.equal(paged.truncated, true);
});
test("type treats newlines and press_enter as Return rather than unicode", async () => {
const typed = await tool("type", { text: "hello\nworld", press_enter: true });
assert.equal(typed.ok, true, JSON.stringify(typed.error));
assert.equal(typed.newlines_as_return, true);
const typeCalls = calls("type");
const keyCalls = calls("key");
assert.deepEqual(typeCalls.slice(-2).map(c => c.args.text), ["hello", "world"]);
assert.ok(keyCalls.filter(c => c.args.text === "return").length >= 2);
});
test("screen-space coordinates skip raster conversion", async () => {
await tool("screenshot");
const r = await tool("left_click", { target: { type: "coordinate", x: 400, y: 300, space: "screen" } });
assert.equal(r.ok, true, JSON.stringify(r.error));
const last = calls("left_click").at(-1);
assert.deepEqual({ x: last.args.target.x, y: last.args.target.y }, { x: 400, y: 300 });
assert.equal(last.args.target.coordinate_space, "screen");
});
test("run_actions sequences click, type and key then stops on failure", async () => {
const shot = await tool("screenshot");
assert.equal(shot.ok, true);
const batch = await tool("run_actions", { steps: [
{ tool: "type", arguments: { text: "hi" } },
{ tool: "key", arguments: { text: "return" } },
{ tool: "wait", arguments: { seconds: 0 } },
] });
assert.equal(batch.ok, true, JSON.stringify(batch.error));
assert.equal(batch.steps.length, 3);
const nested = await tool("run_actions", { steps: [{ tool: "run_actions", arguments: { steps: [] } }] });
assert.equal(nested.ok, false);
});
test("optional OCR binds its exact raster for coordinate actions while preserving AX state", async () => {
const state = await tool("get_app_state", { include_ocr: true });
assert.equal(state.ok, true);
@@ -140,7 +193,7 @@ test("optional OCR binds its exact raster for coordinate actions while preservin
assert.equal(state.ocr.blocks[0].role, undefined, "recognized text is not a fabricated semantic element");
const clicked = await tool("left_click", { target: state.ocr.blocks[0].target });
assert.equal(clicked.ok, true);
assert.deepEqual(calls("left_click").at(-1).args.target, { type: "coordinate", x: 140, y: 70, strategy: "event" });
assert.deepEqual(calls("left_click").at(-1).args.target, { type: "coordinate", x: 140, y: 70, strategy: "event", coordinate_space: "raster" });
assert.equal((await tool("left_click", { target: { type: "coordinate", x: 400, y: 0 } })).error.code, "target_outside_raster");
const unavailable = await tool("get_app_state", { include_ocr: true, app_ref: { name: "OCR unavailable" } });
assert.equal(unavailable.ok, true);
crates/tui/src/commands/contract.rs
-1
View File
@@ -1835,7 +1835,6 @@ fn key_to_utility_message_id(key: &str) -> Option<MessageId> {
"mcp_recommendation_github" => MessageId::McpRecommendationGithub,
"mcp_recommendation_chrome" => MessageId::McpRecommendationChrome,
"mcp_recommendation_playwright" => MessageId::McpRecommendationPlaywright,
"mcp_recommendation_cua" => MessageId::McpRecommendationCua,
"mcp_recommendation_container_use" => MessageId::McpRecommendationContainerUse,
_ => return None,
})
crates/tui/src/commands/groups/utility/mcp.rs
+16 -32
View File
@@ -11,7 +11,6 @@ const GITHUB_MCP_URL: &str = "https://api.githubcopilot.com/mcp/";
const CHROME_DEVTOOLS_MCP_PACKAGE: &str = "[email protected]";
const PLAYWRIGHT_MCP_PACKAGE: &str = "@playwright/[email protected]";
const PLAYWRIGHT_MCP_SOURCE: &str = "https://github.com/microsoft/playwright-mcp";
const CUA_DRIVER_SOURCE: &str = "https://github.com/trycua/cua";
const CONTAINER_USE_SOURCE: &str = "https://github.com/dagger/container-use";
pub(in crate::commands) const COMMAND_INFO: CommandInfo = CommandInfo {
@@ -189,13 +188,6 @@ fn parse_add_for_platform(
],
}))
}
[_, id] if id.eq_ignore_ascii_case("cua") || id.eq_ignore_ascii_case("cua-driver") => {
CommandResult::action(AppAction::Mcp(McpUiAction::AddStdio {
name: "cua-driver".to_string(),
command: "cua-driver".to_string(),
args: vec!["mcp".to_string()],
}))
}
[_, id]
if id.eq_ignore_ascii_case("container-use")
|| id.eq_ignore_ascii_case("container") =>
@@ -303,16 +295,6 @@ fn recommended_mcp_text(presentation: &mut dyn CommandPresentationContext) -> St
.unwrap_or_else(|_| {
format!("• playwright — Microsoft's official Playwright MCP via pinned npm package\n package: {PLAYWRIGHT_MCP_PACKAGE}")
});
let cua = presentation
.translate(
"mcp_recommendation_cua",
&[
("source", CUA_DRIVER_SOURCE),
("restart_command", "/mcp restart"),
("add_command", "/mcp add recommended cua"),
],
)
.unwrap_or_else(|_| format!("• cua — Cua Driver computer-use plugin (MCP server component)\n source: {CUA_DRIVER_SOURCE}"));
let container_use = presentation
.translate(
"mcp_recommendation_container_use",
@@ -338,8 +320,6 @@ fn recommended_mcp_text(presentation: &mut dyn CommandPresentationContext) -> St
\n\
{playwright}\n\
\n\
{cua}\n\
\n\
{container_use}\n\
\n\
External sources (~/.claude.json, .mcp.json, marketplace manifests):\n\
@@ -418,9 +398,6 @@ mod tests {
"mcp_recommendation_playwright" => {
"• playwright — Microsoft's official Playwright MCP via pinned npm package\n package: {package} ({launcher})\n source: {source}\n --isolated starts a fresh browser profile. It can browse/control pages and read authenticated pages.\n add: {add_command}".to_string()
}
"mcp_recommendation_cua" => {
"• cua — Cua Driver computer-use plugin (MCP server component)\n command: cua-driver mcp\n source: {source}\n preview: install and verify Cua Driver separately; Codewhale never downloads it. It can control the desktop and requires operating-system permissions.\n add: {add_command}".to_string()
}
"mcp_recommendation_container_use" => {
"• container-use — Dagger's experimental container-use MCP\n command: container-use stdio\n source: {source}\n requires the separately installed container-use binary; Codewhale never downloads or installs this binary.\n add: {add_command}".to_string()
}
@@ -478,7 +455,8 @@ mod tests {
assert!(recommended.contains("@playwright/[email protected]"));
assert!(recommended.contains("https://github.com/microsoft/playwright-mcp"));
assert!(recommended.contains("https://github.com/dagger/container-use"));
assert!(recommended.contains("https://github.com/trycua/cua"));
assert!(!recommended.contains("https://github.com/trycua/cua"));
assert!(!recommended.to_ascii_lowercase().contains("cua-driver"));
assert!(recommended.contains("least-privilege PAT outside command history"));
assert!(recommended.contains("read authenticated pages"));
@@ -536,13 +514,18 @@ mod tests {
));
let add_cua = mcp(&mut FakePresentation, Some("add recommended cua"));
assert!(matches!(
add_cua.action,
Some(AppAction::Mcp(McpUiAction::AddStdio { name, command, args }))
if name == "cua-driver"
&& command == "cua-driver"
&& args == vec!["mcp".to_string()]
));
assert!(
add_cua.is_error,
"Cua is not a Codewhale computer-use recommendation"
);
assert!(
add_cua
.message
.as_deref()
.is_some_and(|message| message.contains("Unknown MCP suggestion")),
"{:?}",
add_cua.message
);
let import_list = mcp(&mut FakePresentation, Some("import"));
assert!(matches!(
@@ -617,7 +600,8 @@ mod tests {
assert!(text.contains("installs this binary"));
assert!(text.contains("experimental"));
assert!(text.contains("--isolated"));
assert!(text.contains("operating-system permissions"));
assert!(!text.contains("operating-system permissions"));
assert!(!text.to_ascii_lowercase().contains("cua"));
}
#[test]
crates/tui/src/tui/views/extensions.rs
+2 -18
View File
@@ -97,7 +97,6 @@ pub enum PluginProductComponentKind {
Skills,
BrowserDriver,
SandboxRuntime,
NativeRuntime,
}
impl PluginProductComponentKind {
@@ -107,7 +106,6 @@ impl PluginProductComponentKind {
Self::Skills => tr(locale, MessageId::HelpSkills),
Self::BrowserDriver => tr(locale, MessageId::ExtensionsComponentBrowserDriver),
Self::SandboxRuntime => tr(locale, MessageId::ExtensionsComponentSandboxRuntime),
Self::NativeRuntime => tr(locale, MessageId::ExtensionsComponentNativeRuntime),
}
.into_owned()
}
@@ -382,7 +380,6 @@ impl ExtensionsSnapshot {
let recommendation = match item.id.as_str() {
"playwright-browser" => Some(("playwright", "playwright")),
"chrome-devtools" => Some(("chrome-devtools", "chrome-devtools")),
"cua-computer-use" => Some(("cua-driver", "cua")),
_ => None,
};
if let Some((server_name, recommendation_id)) = recommendation {
@@ -437,9 +434,8 @@ impl ExtensionsSnapshot {
/// manifests produced by the packaging lane remain the installation authority.
fn reviewed_product_catalog(locale: Locale) -> Vec<PluginProduct> {
vec![
// Codewhale's own, and the reason this row exists: someone browsing
// the marketplace for computer use saw Cua and Browser Use and not
// the plugin that already ships inside the binary.
// First-party computer use. This is the only computer-use product
// row: third-party desktop-control MCPs are not recommended here.
PluginProduct {
id: "codewhale-computer-use".into(),
name: "Computer Use".into(),
@@ -498,18 +494,6 @@ fn reviewed_product_catalog(locale: Locale) -> Vec<PluginProduct> {
],
maturity: tr(locale, MessageId::ExtensionsStateReviewedCandidate).into_owned(),
},
PluginProduct {
id: "cua-computer-use".into(),
name: "Cua Computer Use".into(),
description: tr(locale, MessageId::ExtensionsProductCuaDescription).into_owned(),
publisher: "Cua".into(),
source_reference: "trycua/cua".into(),
components: vec![PluginProductComponent {
kind: PluginProductComponentKind::NativeRuntime,
name: "Cua Driver".into(),
}],
maturity: tr(locale, MessageId::ExtensionsStateUnderEvaluation).into_owned(),
},
PluginProduct {
id: "browser-use".into(),
name: "Browser Use".into(),
docs/MCP.md
+1 -1
View File
@@ -138,7 +138,7 @@ diagnostic surfaces:
| --- | --- | --- | --- | --- |
| Chrome DevTools | MCP server (stdio) | `npx -y [email protected]` (`npx.cmd` on Windows) | [Official ChromeDevTools project](https://github.com/ChromeDevTools/chrome-devtools-mcp) | npm may download the pinned package when the user restarts MCP. |
| Playwright | MCP server (stdio) | `npx -y @playwright/[email protected] --isolated` (`npx.cmd` on Windows) | [Official Microsoft project](https://github.com/microsoft/playwright-mcp) | `--isolated` starts a fresh browser profile; npm may download the pinned package only after an explicit restart. |
| Cua Computer Use | MCP server (stdio) | `cua-driver mcp`; Driver `0.20.0` reviewed for this release | [Official Cua project](https://github.com/trycua/cua); preview integration | The signed driver and OS permissions are separate, explicit installs. `/mcp add recommended cua` only writes config and never installs or grants either. |
| Computer Use | First-party plugin (MCP + skill) | Ships in the binary as the `computer-use` plugin | Codewhale; enable through `/plugin` or the Extensions marketplace | This is the only computer-use integration Codewhale recommends. Third-party desktop-control MCPs are not listed here. |
| Browser Use | Skill plus separately installed Python runtime | Skill/runtime release `0.13.8` | [Official browser-use project](https://github.com/browser-use/browser-use) | Optional companion: not an MCP server. Codewhale does not auto-run the upstream Skill installer or install its browser/runtime dependencies. |
| Anthropic Sandbox Runtime | Sandbox adapter companion | `@anthropic-ai/[email protected]` | [Official anthropic-experimental project](https://github.com/anthropic-experimental/sandbox-runtime); beta | Documentation-only adapter candidate in v0.9.10: not an MCP server and not an active Codewhale plugin adapter. It does not replace Codewhale's sandbox policy. |
docs/zh_hans/MCP.md
+1 -1
View File
@@ -97,7 +97,7 @@ v0.9.10 的产品建议使用这些经过审查的固定版本定义。Plugins
| --- | --- | --- | --- | --- |
| Chrome DevTools | MCP 服务器（stdio） | `npx -y [email protected]`（Windows 上为 `npx.cmd`） | [ChromeDevTools 官方项目](https://github.com/ChromeDevTools/chrome-devtools-mcp) | 用户重启 MCP 时，npm 可能会下载固定版本的包。 |
| Playwright | MCP 服务器（stdio） | `npx -y @playwright/[email protected] --isolated`（Windows 上为 `npx.cmd`） | [微软官方项目](https://github.com/microsoft/playwright-mcp) | `--isolated` 会启动全新的浏览器配置文件；npm 只会在显式重启后才下载固定版本的包。 |
| Cua Computer Use | MCP 服务器（stdio） | `cua-driver mcp`；Driver `0.20.0` 已为本版本审查 | [Cua 官方项目](https://github.com/trycua/cua)；预览集成 | 签名驱动和操作系统权限是分开的、显式的安装。`/mcp add recommended cua` 只写入配置，绝不会安装或授予其中任何一项。 |
| Computer Use | 第一方插件（MCP + skill） | 随二进制内置为 `computer-use` 插件 | Codewhale；通过 `/plugin` 或扩展市场启用 | 这是 Codewhale 唯一推荐的计算机操作集成。此处不再列出第三方桌面控制 MCP。 |
| Browser Use | Skill 加单独安装的 Python 运行时 | Skill/运行时版本 `0.13.8` | [browser-use 官方项目](https://github.com/browser-use/browser-use) | 可选配套：不是 MCP 服务器。Codewhale 不会自动运行上游的 Skill 安装器，也不会安装其浏览器/运行时依赖。 |
| Anthropic Sandbox Runtime | 沙箱适配器配套 | `@anthropic-ai/[email protected]` | [anthropic-experimental 官方项目](https://github.com/anthropic-experimental/sandbox-runtime)；测试版 | v0.9.10 中仅文档说明的适配器候选：不是 MCP 服务器，也不是活跃的 Codewhale 插件适配器。它不会取代 Codewhale 的沙箱策略。 |