Compare commits

...
Author SHA1 Message Date
l0ng-ai 6f01d74860 test(plugins): let the first call in the deadline test start its instance
The first highlights call of edit_path_calls_have_soft_and_hard_deadlines
starts the plugin instance, which can take longer than the 50 ms soft
deadline on a slow runner (Windows CI). Wait for it as the later calls do.
2026-11-04 19:58:07 +08:00
l0ng-ai 82fe52f595 fix(agents): wait for the agent to exit before deleting its worktree (#1240)
Closes #969

## What and why

On Windows, deleting an agent task and its worktree could fail with `os error 32` while the agent's process tree still held the directory or its files open. Deletion now waits up to 5 seconds for the agent process to be killed and reaped, then retries folder removal for transient locks. This also covers persistent ACP agents such as CodeBuddy that cannot reload their session and therefore stay alive while idle.

## How it was verified

- Windows desktop acceptance through the Tauri MCP bridge, using a deterministic ACP agent with a real PowerShell descendant holding a file open. Closed the conversation tab, then deleted the task with worktree removal enabled. A separate helper retained another file lock for 1.2 seconds: the pre-fix implementation showed `os error 32`; the fixed implementation deleted the task and directory, with both agent processes gone.
- Also verified direct deletion, archive then delete, and deletion with worktree removal unchecked. Processes exited in all cases; branches were preserved, and unchecked removal preserved the folder.
- `cargo test -p ide-jsonrpc -p ide-agent -- --test-threads=2`: 155 passed, 5 real-agent tests ignored. The test runner used Git Bash on PATH and a process-local `core.autocrlf=false`, matching CI's LF fixture setup; no global Git settings were changed.
- `cargo clippy -p ide-jsonrpc -p ide-agent --all-targets -- -D warnings`, `cargo fmt --all --check`, and `git diff --check`: passed.
- Full workspace (Windows/macOS/Linux), frontend and E2E CI: [final PR run](https://github.com/l0ng-ai/openidea/actions/runs/41819945536).
- Windows regression coverage now checks deletion while waiting for permission and while idle without `session/load`, actual sharing violations, delayed file-lock release, and `Peer::exited()` waiting for a descendant's file handle to close. Existing Unix process tests remain enabled.
- Evidence (local, not committed):
  - Before: `C:/Users/thoma/repo/pr1240-evidence-20261104/before.png`
  - After: `C:/Users/thoma/repo/pr1240-evidence-20261104/after.png`
  - Recording: `C:/Users/thoma/repo/pr1240-evidence-20261104/after.gif`
  - Runtime results: `C:/Users/thoma/repo/pr1240-evidence-20261104/runtime-cases.json`
- Real signed-in CodeBuddy and native Claude/Codex were not exercised. Native Claude/Codex use an in-process connection, so their `exited()` resolves immediately and removal relies on the bounded retry. Shared-worktree fork protection remains unchanged.
2026-11-04 19:39:40 +08:00
l0ng-ai d94a117d69 test(plugins): don't time the plugin host tests against the runner's speed (#1250)
Closes #1248
2026-11-04 18:46:52 +08:00
Marius Ciepluch b6b0ce46c3 fix(fonts): enumerate all faces and per-user fonts on Windows (#908)
Enumerate every locally installed Windows font face, including per-user font files. Correct glyph-width probing and collection-file enumeration, and exclude cloud fonts that are not installed.

Closes #1241
2026-11-04 18:46:23 +08:00
l0ng-ai 953e844541 feat(plugins): community plugins, version compatibility and categories in the marketplace (#1226)
* feat(plugins): list community plugins from a registry in the marketplace

The marketplace index can now carry plugins of other publishers, published
from their authors' repositories through the l0ng-ai/openidea-plugins
registry.

- `openidea-plugin index` with a key replaces only that publisher's plugins
  (it used to drop every other entry); without a key it indexes each package
  against the publishers the index already lists, checking the key and the
  id prefix instead of only the signature itself.
- `openidea-plugin merge` takes another index's publishers, plugins and
  revocations in place of every publisher but `--keep`; the other index can't
  list or revoke the kept publishers' plugins.
- `openidea-plugin check` accepts a `.oiplugin` file.
- Releases merge the registry's index into plugins-index.json and attach the
  Linux x64 tool; plugins-index.yml refreshes the community plugins in the
  latest release hourly or when the registry signals a change.
- plugins/README.md and CONTRIBUTING.md describe official and community
  plugins.

* feat(plugins): offer the newest plugin version this IDE can load

Index entries record the plugin API each version needs (`engine`: the
manifest's `engines.openidea` narrowed to the API its component was built
for), and an index may list several versions of a plugin. Settings › Plugins
offers the newest version this IDE can load; with none, the plugin is shown
as incompatible with the API it needs, and downloading it is refused.

`openidea-plugin index` keeps, of each plugin, the newest version per API
requirement, so users of an older IDE keep the last version built for them;
`--download-base` makes downloads absolute, and the release workflow points
them at the release that published them. Scratch directories are now unique
per call.

* feat(plugins): file marketplace plugins under categories

`plugin.toml` takes `categories` (up to 3 of languages, themes, icon-themes,
linters, vcs, ai, frameworks, tools, other). A plugin that declares none is
filed under what its contributions imply (themes, icon themes, languages),
else under other; categories an IDE doesn't know are ignored, and `check`
warns about them. Index entries carry the categories, and Settings › Plugins
› Available filters by them and shows them in a plugin's details. The
official plugins declare theirs.

* feat(plugins): let a marketplace index point to where it moved

An index with `moved` sends the IDE to the index at that URL (relative or
absolute, up to three hops; an https index can only move to https). Downloads
resolve against the index actually read, so a marketplace can change hosts
while released IDEs keep the address they were built with.

* fix(plugins): full-width category filter and aligned incompatibility note

* fix(plugins): translate the AI category in Chinese

* feat(plugins): keep a plugin's signer across updates and announce updates

- Index format: fields are only added; an IDE reading a newer `format`
  still applies its revocations and `moved`, lists no plugins and asks to
  be updated, instead of failing to read the index.
- Key continuity: an installed plugin updates silently only to a package
  signed with the same key, or a key it endorsed (`KeyEntry.endorsement`,
  `index --previous-key`, `openidea-plugin endorse`). Another signer, or an
  unsigned package replacing a signed one, is shown on the install page and
  needs the user's confirmation (`Consent.signerChange`).
- Official key: release builds carry the official publisher's public key
  (`OPENIDEA_OFFICIAL_PLUGIN_KEYS`, derived from PLUGIN_SIGNING_KEY);
  `openidea.*` plugins must be signed with it or a key it endorsed,
  whatever the index lists.
- Updates: the daily marketplace check announces new versions once, with
  Update All (also in the gear menu and as an action). An update that asks
  for nothing new installs with the grants the installed version has.
- Index entries need `engine`; `categories` beyond the first three are
  ignored instead of rejecting the manifest.
2026-11-04 18:24:43 +08:00
l0ng-ai 48e23c012f test(plugins): time only the restart wait in the crash backoff test
The waits included the restart that ended them, and instantiating the plugin can take a few hundred milliseconds on CI, so a slow second restart made the third wait look no longer. Measure until the call that is let through and require the third wait to exceed the second by half the base backoff.
2026-11-04 17:26:25 +08:00
l0ng-ai 0bc0396f80 test(sdk): only use a subst drive that shows the test's own directory
The two mapped-drive tests run in parallel processes and could race for the same free drive letter, then see the other test's directory. Mark the target and accept a drive only when the marker shows through it.
2026-11-04 16:57:30 +08:00
l0ng-ai d20be9c5cc fix(window): declare the detach fullscreen flag only on macOS
It is read only in macOS code, so Linux and Windows builds failed on an unused variable.
2026-11-04 16:57:28 +08:00
l0ng-ai 2d3b7ebf54 fix(window): keep a fullscreen project window group in one Space on macOS (#1218)
Native fullscreen gives every window its own Space, so switching projects in a fullscreen window group left one Space and entered another: the desktop showed in between, and quick switching could drop out of fullscreen.

A grouped window now uses simple fullscreen for every member at once; switching projects only shows one window and hides the others. The green button, the system Enter Full Screen item and Toggle Full Screen take this path for grouped windows, which keep their window buttons; ungrouped windows keep native fullscreen. Menu bar and Dock auto-hide while any window is in simple fullscreen, and detaching or closing a tab keeps the rest of the group fullscreen.
2026-11-04 16:35:43 +08:00
Lee Dong Geon 6dec1252db feat(plugins): import IntelliJ theme plugins (#1184)
Settings › Plugins (gear menu) gets Import Theme from IntelliJ IDE… (theme plugins of the JetBrains IDEs on this machine, the one each IDE uses first) and Import IntelliJ Theme from File… (a Marketplace .jar / .zip or a .theme.json). The host reads the plugin with size limits; the UI converts it with the IntelliJ mapping (moved from scripts/theme to src/features/plugins/intellijTheme.ts, still used by pnpm theme:import-intellij) into one OpenIDEA theme plugin (engines.openidea ^1.0) with all of its themes, installed through the usual install page, and applies the chosen theme.

Theme pickers separate each plugin's themes, and Quick Switch Scheme › Theme previews the theme under the pointer or keyboard after 500 ms and restores the selected one when closed without a choice.

Co-authored-by: ldg030201 <[email protected]>
2026-11-04 15:28:00 +08:00
l0ng-ai a610982655 fix(debugger): keep multi-line values on their row in the variables tree (#1196)
Multi-line value renderings (a pandas Series) were drawn with white-space: pre in a one-line row and spilled over the rows below. VariablesView rows now join them onto one line like inline values; this covers Variables, Watches, value hints and Evaluate results. Copy/Set Value keep the full text.
2026-11-04 14:59:03 +08:00
Park Jong Hyun ed145e3046 feat(refactor): Java Pull Members Up / Push Members Down (#1183)
Refactor | Pull Members Up… and Push Members Down… for Java, from the Refactor menu, the editor context menu and Refactor This. A dialog picks the destination and the members (Make abstract / Keep abstract); problems are confirmed before anything changes; the edit is applied as one change or previewed first.

JDT LS offers neither as a code action, so the app-owned JDT LS bundle (plugins/java-usages 1.3.0) adds two delegate commands that run JDT's PullUpRefactoringProcessor / PushDownRefactoringProcessor and return the problems and the edit without applying it, exposed as lsp_hierarchy_members / lsp_hierarchy_refactor (also over the remote host dispatcher). The bundled jar was rebuilt from source with JDT LS 1.63.0 and matches the recorded SHA-256.

Co-authored-by: fhrmrk <[email protected]>
2026-11-04 14:55:42 +08:00
l0ng-ai 5b46844f88 fix(process): give child processes the login shell's variables the IDE lacks (#1195)
Finder/Dock-launched apps only got PATH from the login shell; children outside run configs, LSP and DAP missed rc exports such as GOMODCACHE, so go list showed only some modules. host_env now adds the shell's variables the process lacks (call-site values win); the login-shell probe leaves them out so Reload Environment stays accurate. Covers every crate that spawns through ide_core::process::prepare/host_env.
2026-11-04 14:38:34 +08:00
Sergei Melnikov c87d2a6b55 feat(plugins): let icon themes match by pattern, replace built-in icons and describe their settings page (#1177)
Icon themes could only map files by exact name or extension and folders by name. Themes can now match by regular expression or path (patterns that could backtrack badly, lookaround, inline flags and named groups are refused at install), ship dark variants, replace built-in UI and symbol (PSI) icons, and offer packs (groups) and variants (choices). A theme describes its own options page in Settings › Plugins as sections of items naming options the IDE implements (hide file/folder/UI/PSI icons, hollow folders, accent, monochrome, saturation, opacity, icon size, row height); values are stored per theme in appearance.fileIconThemeOptions. Existing themes are unchanged.

Closes #1096

Co-authored-by: pdasilem <[email protected]>
2026-11-04 14:22:38 +08:00
l0ng-ai 4fbe41194f feat(gradle): analyze a Gradle project's dependencies per configuration (#1181)
The Dependency Analyzer lists Gradle projects next to Maven modules. For a Gradle project it runs `<project>:dependencies --configuration <name>` (compile/runtime/test classpaths) in the background (on the project host for remote projects), cancellable and cached per session, and reuses the list/tree views, search, conflicts-only filter and usage paths: an exact request Gradle replaced (1.0 -> 1.1) is a conflict showing the requested version, constraints are left out, unresolved dependencies are marked and build projects (quoted or not) are recognised. Exclude / Jump to Source stay Maven-only. Opened from the Gradle tool window toolbar, a Gradle project's context menu and a build script's menus.

Closes #1005
2026-11-04 13:51:40 +08:00
l0ng-ai 16df2ae472 fix(run): remove temporary configurations from Edit Configurations for good (#1180)
Removing a temporary configuration in Edit Configurations only changed the dialog's draft; the reload after OK read it back from the saved temporary list. OK / Apply now drops removed temporaries from that list, keeps unedited ones temporary instead of saving every non-detected temporary configuration as a user configuration, and saves edited ones as before. Removing the last temporary configuration also cancels a pending rewrite of the list that could bring the removed entry back (the chooser's Delete had the same race).

Fixes #1162
2026-11-04 13:20:41 +08:00
l0ng-ai cbc77941d4 feat(vcs): bind Git › Add to ⌥⌘A / Ctrl+Alt+A (#1159)
Git › Add (vcs.addToVcs) had no shortcut in the IntelliJ keymaps. It now uses ⌥⌘A on macOS and Ctrl+Alt+A on Windows/Linux, so it shows next to Add in the Git menus and works from the project view, editor and Commit view. On Windows the diff viewer's Apply Right Side keeps Ctrl+Alt+A while a diff is focused. The VS Code preset lists the action without a shortcut.

Closes #1158
2026-11-04 13:16:19 +08:00
Nguyen Thinh 3d7768f47f feat(window): graphql window (#1166)
Adds a GraphQL tool window: schema discovery from graphql-config files (or every GraphQL file without one), schema errors and endpoints, running the operation at the caret against a configured endpoint with ${NAME} environment variables, Query Result, and introspection into the schema file. The editor gets a GraphQL toolbar with endpoint selection, variables and run.

The tool window is listed on the stripe once used, so projects without GraphQL get no extra button.

Closes #1168

Co-authored-by: kidp2h <[email protected]>
2026-11-04 13:02:01 +08:00
ppxbandppxb 8687f6b1e3 perf(settings): open the Settings window shell before its startup chain (#1175)
The Settings window ran its whole startup chain (importing every settings page, then the project and settings IPC, one after another) before opening the dialog, so it showed an empty title bar for a while on every open. The dialog now opens at once with a loading state while the pages and the settings load in parallel, and re-bases the open dialog when the settings arrive; the search field is focused once the pages are in.

Co-authored-by: ppxb <[email protected]>
2026-11-04 12:54:53 +08:00
l0ng-ai 85c03d33af fix(editor): release a decision to leave whose project change never came (#1178)
When the project change after an authorized leave never came, the editors' close decision stayed authorized: files failed with "The project is closing" and later leaves with "An editor close decision is already pending" until restart. An authorized decision with no project opening is now released when the window opens a file or is left again.

Fixes #1139
2026-11-04 12:51:27 +08:00
l0ng-ai 65ca0390de feat(run): choose the JRE and the module of an Application configuration (#1172)
Application / Spring Boot configurations found their module only from the main class, so the same class in two modules failed with "Cannot determine the module for main class", and the JVM was always the module's. The form now has JRE (the module's JDK or an installed JDK) and Use classpath of module (found by main class, or a project module, stored as RunConfig.module). Run configuration selects can load their options when shown.

Fixes #1153
2026-11-04 12:41:34 +08:00
Mariano Dupontandmarianomd b31938db00 feat(deployment): add standalone comparison and selected-file sync (#1070)
Sync with Deployed opens in its own window, one per project window. It lists local files next to the server's copies, with filters, content and binary comparison, and an embedded side-by-side diff whose local side can be edited. The selected file can be synchronized (Enter) without closing the window.

The window is the `deployment` kind of the auxiliary windows that Settings and comparisons use. A cancelled close in it stops its project window from closing. The preview uses the vcs diff viewer through its `diff` editor kind. Also covers #811 (compare with the deployed versions).

Co-authored-by: marianomd <[email protected]>
2026-11-04 12:40:45 +08:00
l0ng-ai 60969eccb6 fix(java): preserve mapped toolchain paths and refresh Maven run settings (#1147)
* fix(sdk): keep the drive letter of a Maven home found on a mapped drive

Maven discovery canonicalized mvn on PATH and each Maven home to resolve symlinks (Homebrew's
bin/mvn). On Windows, a home on a mapped network drive or a subst drive then became
\\?\UNC\server\share\… (or another drive), the Maven home setting picked from the list stored
that path, and cmd.exe can't start mvn.cmd from it: every Maven run failed with "The system
cannot find the path specified". Discovery now keeps the path as found when canonicalizing would
change its drive letter.

Refs #822

* fix(sdk): keep the drive letter of a JDK home on a mapped drive

A JDK on a mapped network drive was canonicalized to \\?\UNC\server\share\…
too, so Maven ran with that JAVA_HOME and mvn.cmd failed with "The system
cannot find the path specified" even with a local Maven home. JDK homes
(configured, JAVA_HOME, the java on PATH) now keep their drive letter like
Maven homes do.

* fix(maven): resolve current toolchain settings for each run

* fix(sdk): distinguish mapped roots from external launcher links

* test(run): preserve explicit Maven choices across save and reload

* fix(maven): keep edited nested test launchers when refreshing defaults
2026-11-04 12:35:01 +08:00
xdpan1234andxdpan1234 0a70b0af01 fix(vcs): open comparisons in independently resizable native windows (#818)
Compare with Local, Compare Versions and the other revision comparisons open in a separate window that belongs to their project window. The window can be moved outside the main window, resized from its edges and corners, and shows the full diff. Edits made there and in the project window are mirrored both ways.

Root cause of #798: the comparison was an in-window dialog with a fixed size.

Settings and comparison windows now share one auxiliary-window mechanism (`AuxKind`, `open_auxiliary_window`). It handles labels, the title bar, placement over the parent, event forwarding, document-edit mirroring, menu routing, closing together with the parent, and remembered geometry per kind.

Co-authored-by: xdpan1234 <[email protected]>
2026-11-04 12:27:45 +08:00
l0ng-ai 268cf01a86 fix(markdown): preview headings in the preview's text colour (#1171)
Preview headings used the editor scheme's Markdown › Heading attribute (meant for the source) since #992; they now use the editor foreground like the rest of the preview.

Fixes #1151
2026-11-04 12:22:32 +08:00
l0ng-ai fd4921c58a fix(agents): answer a permission request from its line in the conversation (#1170)
A pending agent question / permission request could only be answered from the banner at the top of the session; the conversation line said "Waiting for your answer" with nothing to click. The line now offers the same answers, and a request without options gets Cancel in both places so the agent isn't left waiting.

Fixes #1148
2026-11-04 12:22:27 +08:00
kkmihaiandkkMihai e04e5dd57d feat(database): add ClickHouse data sources (#865)
Adds ClickHouse to Data Sources over its HTTP interface (HTTP/HTTPS, SSH tunnels, session settings, server-side cancellation): browse databases, tables and columns, inspect DDL, run queries and EXPLAIN plans, with ClickHouse dialect completion. Table data stays read-only and manual transactions are rejected, since ClickHouse keys are not unique and HTTP has no transactions.

Results are streamed as JSONCompactEachRowWithNamesAndTypes and parsing stops at the row limit, because result_overflow_mode=break still returns a whole block; errors show only the server's "Code: …" message, including failures raised mid-stream.

Fixes #862

Co-authored-by: kkMihai <[email protected]>
2026-11-04 12:21:03 +08:00
l0ng-ai c5e188d1e3 fix(i18n): translate the GitHub pull request operations' progress and failure titles (#1169)
The GitHub tool window's long operations passed an untagged banner title and appended " failed" for the error, so the banner stayed English and failures read half translated. withBusy now takes a tx-tagged progress title and its own failure title per operation (checkout, fetch for diff, worktree, apply/commit suggestion, resolve thread).
2026-11-04 12:02:06 +08:00
l0ng-ai d3b9a39831 fix(run): keep temporary run configurations following the toolchain settings (#1167)
Temporary run configurations were kept with what settings derived when they were made (absolute mvn, -s / -Dmaven.repo.local, JAVA_HOME), and never went through the backend again, so after a Maven home / settings file / JDK change they and gutter test runs narrowed from them still ran the old ones. They are now kept without derived values (run_without_derived) and get the current settings applied when the run list loads (run_apply_settings). Maven's settings file and local repository apply to every Maven configuration unless it sets them itself. Earlier stored entries are dropped once.

Refs #822
2026-11-04 11:56:03 +08:00
Julien Dudek 59567f1af0 feat(i18n): add French (#730)
Adds French as a UI language, with French catalogs for message keys and source text,
and the French space before a colon in settings labels.

UI languages are now registered in one place, `LOCALES` in `src/platform/i18n.ts` (name
in its own language and `<html lang>` tag); the Language and Region options, settings
validation and `<html lang>` all come from it. The catalog tests read every catalog in
`locales/` and `locales/text/`, so a new file is always checked, and literal message keys
given to `t` must exist in `en.json`. Only Chinese must cover every UI string; French and
Korean show English for text they lack.

Also makes two date tests independent of DST changes.

Co-authored-by: J-Dudek <[email protected]>
2026-11-04 11:48:45 +08:00
Park Jong Hyun 7f0c78893e feat(i18n): add Korean (#1126)
Adds Korean as a UI language: Settings › Appearance & Behavior › System Settings ›
Language and Region lists 한국어, with Korean catalogs for message keys and source text.

Only Chinese must translate every UI text. Other locales may lack entries and show
the English text for them; their catalogs are still checked for stale entries, keys
English lacks and placeholders that differ from the English text.

Co-authored-by: fhrmrk <[email protected]>
2026-11-04 11:43:30 +08:00
l0ng-ai fc6a27d4ba fix(java): give jdtls memory back when idle and stop it showing as not responding (#1137)
jdtls ran on the parallel collector, which never shrinks the heap while idle, so each server kept its import/build peak committed for as long as it lived. It now runs on G1 with a periodic collection every minute and a 10–30% free-heap band, so memory goes back to the OS once the work is done (a SkyWalking checkout: 1.9 GB → 1.25–1.3 GB idle). A collector set in the JVM options still replaces ours with its tuning; setting either heap-free ratio drops both of ours.

The Equinox launcher gets -nosplash: without it its native library loads AppKit on macOS and every jdtls shows as Not Responding in Activity Monitor.

Part of #1008.
2026-11-04 11:40:02 +08:00
l0ng-ai 5e73b1634f feat(agents): environment variables for built-in agents (#1157)
Settings › Tools › Agents › Built-in has an Environment variables field under the executable. They are saved in the agent's file like a command override, and the agent starts with them on top of the shell environment, replacing variables of the same name. Changing the command or the environment keeps the other; a file left with only the built-in's values is removed.

Closes #1133
2026-11-04 11:07:52 +08:00
l0ng-ai e88e7e6459 fix(i18n): translate Git operation titles and their notifications (#1154)
Git operations passed their title, success and failure text to runOp / runGitOperation as plain strings the text catalog check doesn't see, so notifications after a commit, merge, rebase, stash, shelve, tag, patch or rollback stayed in English, several splicing plurals or English words. Call sites now tag the text with tx, each count and variant has its own sentence, and a failure is titled "{title} failed" from the translated title.
2026-11-04 10:57:10 +08:00
l0ng-ai c86928eee7 fix(deployment): Windows absolute mapping paths, and a flaky plugin syntax test (#1130)
An absolute local mapping path outside the project falls back to the same path under the project. On Windows, joining it onto the project root kept its drive and replaced the root, so an existing outside folder became a mapping. The drive and root are now dropped first, and the mapping tests use the platform's absolute paths.

The Solidity syntax test asks again when the first call of a new instance runs past the edit-path budget and its answer is dropped.
2026-11-04 10:27:50 +08:00
l0ng-ai fa6d7276b8 fix(i18n): name each Docker prune and Compose operation in a full sentence (#1150)
Prune and Compose task titles spliced English into a sentence ("Pruning ${target}…", "Compose ${op} ${project}…"), so they stayed partly or wholly English. Each prune target and Compose operation now has its own title and failure text.
2026-11-04 10:16:15 +08:00
l0ng-ai 1bc7fd06c1 feat(gradle): navigate to version catalogs imported from published artifacts (#1149)
A catalog declared in settings as create("name") { from("group:artifact:version") } is a published .toml Gradle downloads into its cache. Build-script accessors of such a catalog now go to and describe its entries like a local catalog's: gradle_published_catalog finds the file in Gradle's cache or the local Maven repository on the project's host. A catalog not downloaded yet is looked up again the next time the script opens.

Closes #1006
2026-11-04 10:14:05 +08:00
Park Jong Hyun 280c2da48f fix(editor): keep an editor color scheme chosen together with the theme (#1138)
Since #1100 a theme that turns light or dark switches a chosen scheme of the other kind back to the theme's own. It did so also when the scheme was chosen in the same Apply as the theme, so One Light + Solarized Dark left the default light editor. A scheme set by the settings change that also changes the theme now goes with it; a later theme change on its own still switches it back.

Closes #1136

Co-authored-by: fhrmrk <[email protected]>
2026-11-04 10:07:04 +08:00
l0ng-ai 2ec2b76d7a fix(remote): keep the host in a remote window's title (#1146)
Connecting a remote window titles it `<project> [<user@host>]`, but the frontend rewrote the title from the project and active file whenever either changed (and after a reload), dropping the host. The title now includes the remote session's host.
2026-11-04 09:58:51 +08:00
l0ng-ai 425a7da003 feat(run): import the Node interpreter of JetBrains npm configurations (#1144)
An npm configuration's own Node interpreter was listed as ignored on import. The import now fills the npm form's Node interpreter field, together with the form's other fields so it shows the configuration as it runs. Jest, Vitest and Node.js configurations still list it as ignored.
2026-11-04 09:57:44 +08:00
l0ng-ai 87e9bec4f9 fix(todo): colour TODO comments in the TODO tool window's preview (#1143)
The preview is an editor view over the real document but doesn't get the editor contributions, so TODO comments showed in plain comment colour there. It now adds the same TODO colouring; the live item scan stays with the editor contribution.
2026-11-04 09:55:32 +08:00
l0ng-ai f088a4afb0 fix(i18n): translate notification titles passed through helpers (#1142)
Debugger action failures, project tree clipboard / create-file failures and Docker operation failures pass their title through a helper before notify, so the text catalog check never saw them and they showed in English. The call sites now tag the text with tx, and Docker failures name each operation in a full sentence instead of splicing an English verb or object into one.
2026-11-04 09:53:56 +08:00
l0ng-ai 2e6cda1d86 feat(run): update a Tomcat deployment from compiled classes instead of rebuilding the WAR (#1140)
"Update classes and resources" / "Update resources" on a Tomcat configuration deploying the project's WAR artifact ran a full package build and re-exploded the archive. They now compile the WAR module incrementally through jdtls and copy only the changed classes and resources into WEB-INF/classes and changed web files (src/main/webapp) into the exploded deployment; Tomcat reloads the context, or HotSwap applies the classes when debugging. Falls back to rebuilding the WAR when a dependency module was compiled since the last update (its WEB-INF/lib jar would be stale), the deployment doesn't exist yet, or no Java language server is available. Redeploy still rebuilds the WAR.

Closes #970
2026-11-04 09:43:54 +08:00
l0ng-ai 42968a39b3 style(ui): remove hover backgrounds from icon actions 2026-11-04 09:39:32 +08:00
l0ng-ai 4d086c44c9 fix(workbench): show splash-style progress when opening projects 2026-11-04 09:39:32 +08:00
l0ng-ai 770178966f fix(lsp): tell language servers about files changed outside the editor (#1135)
Language servers never got workspace/didChangeWatchedFiles, so files changed on disk outside the editor (checkout, pull, another tool) stayed unknown to them; jdtls's incremental build skipped them and Java / Spring Boot launches ran their old classes (e.g. another module of a multi-module project). The project's file watcher now forwards each batch to the running servers: created, changed and removed files of a server's languages under its root.

Refs #1015
2026-11-04 09:16:32 +08:00
Qingandqtvbwfn 2f2d3a25ae feat(todo): edit the file right in the TODO preview (#1129)
The TODO preview is an editor on the file, as in IntelliJ: a CodeMirror view attached to the real document, so edits share the buffer with editor tabs and Save All saves them; no editing switch. The caret lands on the selected item, also when another item of the same file is selected. Read-only files fall back to the read-only preview. Dirty buffers are held rather than released (releasing would auto-save or drop the edits), and previewText reads the held text.

Co-authored-by: qtvbwfn <[email protected]>
2026-11-04 08:58:53 +08:00
Qingandqtvbwfn 893cbede9c fix(todo): only count real comment markers around TODO keywords (#1128)
The comment filter accepted `#`, `--`, `;` and `%` anywhere before the keyword, so `foo(); TODO`, `if (a % 2) TODO`, `i--; TODO`, URLs and strings produced items. Inline openers (// /* <!-- # -- {- (*) now count only at the line start or after whitespace; `*` `;` `%` and docstring quotes only at the line start. A TODO anywhere inside a comment still counts, as in IntelliJ. Lines the backend truncated keep the lenient rule.

Co-authored-by: qtvbwfn <[email protected]>
2026-11-04 08:56:00 +08:00
Qingandqtvbwfn e5718d41d8 feat(ui): make preview panes selectable and copyable (#1127)
The workbench disables selection globally, so the read-only preview panes beside result trees (TODO, Usages, Problems…) could not be selected or copied. The pane opts back in (user-select: text) and uses useSelectionBoundary so drags stay inside it; the line-number gutter stays unselectable so copied text is code only.

Co-authored-by: qtvbwfn <[email protected]>
2026-11-04 08:54:25 +08:00
l0ng-ai 53b8cdeb51 feat(remote-dev): forward ports of the remote host (#1131)
Remote windows get a Ports tool window: forward localhost:<port> of the remote host to this machine (same port when free here), with Open in Browser / Copy / Stop. Forwarded ports are remembered per host and folder and forwarded again on reconnect. Tunnels reuse the window's SSH session (ide_ssh::Tunnel can prefer a local port) and close with the connection.

Closes #819
2026-11-04 08:50:25 +08:00
l0ng-ai a3bbbf90a4 feat(vcs): clone from the repositories of your GitHub account (#1125)
The Clone Repository dialog lists GitHub next to Repository URL: the repositories of the account gh is signed in to, searchable; picking one fills URL and directory, double-click or Enter clones. Clone URL follows gh's git_protocol. Clone sources are a workbench contribution (registerCloneSource); gh_account_repos needs no project.

Closes #750
2026-11-04 08:40:06 +08:00
l0ng-ai 9988ce4113 Revert "feat(i18n): add Korean" (#1124)
Reverts #1122, merged by mistake (wrong PR number) before review and under another change's commit message. Adding UI locales is still an open decision (#914), and the Korean catalog fails the catalog tests on current main.
2026-11-04 08:31:26 +08:00
l0ng-ai 35c6af2367 feat(app): open a file at a line and column from the command line (#1123)
`openidea --line 120 --column 8 <file>` opens the file at that position (1-based; without --column the line start; clamped). The position is kept by file until the window that opens it takes its launch: new instance, second instance, a project open elsewhere, an already open file. The path argument skips these flags' values.

Closes #829
2026-11-04 08:26:41 +08:00
Park Jong Hyun 3eabaf0d39 feat(app): open a file at a line and column from the command line (#1122)
`openidea --line 120 --column 8 <file>` opens the file at that position (1-based; without --column the line start; clamped). The position is kept by file until the window that opens it takes its launch: new instance, second instance, a project open elsewhere, an already open file. The path argument skips these flags' values.

Closes #829
2026-11-04 08:25:49 +08:00
l0ng-ai 192e613830 feat(run): list recently started configurations at the top of the chooser (#1121)
The Run/Debug configuration popup lists the five configurations started last under "Recent", newest first, after the pinned ones and not repeated in their group. Starts are recorded from run and debug sessions (any executor, reruns included). Kept per project.

Refs #997
2026-11-04 08:15:13 +08:00
l0ng-ai 4c5b6de9ad feat(run): choose the Node interpreter of an npm configuration (#1120)
npm configurations get a "Node interpreter" field; the script and the package manager run on that node (its directory first on the PATH). Empty keeps the project's interpreter from Settings › Languages & Frameworks › Node.js. Saving drops the PATH settings derived before the choice, so a stale project PATH isn't kept.

Closes #997
2026-11-04 08:11:43 +08:00
l0ng-ai 6bf1d25711 feat(run): pin configurations to the top of the chooser (#1119)
Hovering a configuration in the Run/Debug configuration popup shows a Pin button. Pinned configurations are listed first under "Pinned", in pin order, and not repeated in their group; a search keeps pinned matches on top. Pins are kept per project.

Refs #997
2026-11-04 08:06:23 +08:00
l0ng-ai aeb3abca34 feat(run): mark running configurations in the chooser, with Rerun and Stop (#1118)
A configuration with a running or debugged process gets a green dot on its icon in the Run/Debug configuration popup; its row offers Rerun and Stop in place of Run and Debug. Rerun closes the popup, Stop keeps it open.

Refs #997
2026-11-04 08:02:54 +08:00
l0ng-ai b9f3c07f9b fix(debugger): identifier boundary check for astral letters under V8 (#1117)
Smart Step Into and Safe Delete tested the text before a match with /[\p{L}\p{N}_$]$/u, which V8 in Node 26 fails for a letter outside the BMP (a lone property class before $), so smartStep.test.ts failed. Both checks now use an alternation with the same meaning.
2026-11-04 07:57:12 +08:00
l0ng-ai a1dd16f26d fix(i18n): translate the Delete question, prompt labels and the shortcut recorder placeholder (#1116)
The project tree's Delete question was concatenated from English fragments, and the VCS prompt / multi-choice labels and the shortcut recorder placeholder were rendered as given, so they stayed English in the Chinese UI. The question is now tx templates with zh entries; the labels and placeholder go through the catalog.
2026-11-04 07:55:44 +08:00
Mariano Dupontandmarianomd 94665528b9 fix(vcs): refresh packed refs after file replacement (#1115)
Git Log could keep old commits and report 'unknown revision HEAD' after Git replaced packed-refs without a newer modification time: gitoxide's packed-ref snapshot only reloads for a newer mtime. The cached repository handle now also watches the common packed-refs file and compares modification time plus length before reuse; covered for the main repository and a linked worktree.

Co-authored-by: marianomd <[email protected]>
2026-11-04 07:50:13 +08:00
l0ng-ai 809c6adcf1 fix(search): keep Search Everywhere's keys from the editor's actions behind it (#1114)
The capture-phase keymap ran editor actions on keys the Search Everywhere field handles, since editor actions apply while a popup is open over an editor: ⌥↵ ran Show Context Actions instead of Assign Shortcut, and ⌥↑/⌥↓ and the other navigation keys could reach the editor too. The field now claims the keys its onKeyDown handles while it has focus.
2026-11-04 07:26:56 +08:00
l0ng-ai 2bb8f3b091 feat(run): Stop lists the running processes with Stop All when several run (#1113)
Stop from the shortcut, menu or Find Action stopped the selected configuration's process whenever it was running, so with several processes there was no way to pick another or stop them all. Like IDEA's Stop it now stops the only stoppable process or lists them all with Stop All; the run widget's stop button keeps stopping the selected configuration and the Run tool window its active tab.
2026-11-04 07:23:09 +08:00
arookieofcanda-rookie-of-C-language cd77c9c6fb fix(vcs): prefill the New Branch dialog with the current branch name (#1073)
A plain New Branch (single and multi-repository) now opens with the current branch name, like JetBrains IDEs, so a related branch is an edit away; detached HEAD stays empty. New Branch from a local ref now applies its <ref>-copy suggestion, which was computed but never used.

Co-authored-by: a-rookie-of-C-language <[email protected]>
2026-11-04 07:11:19 +08:00
l0ng-ai d72f8a1506 fix(agents): show a pending question in full in the permission banner (#1111)
The permission banner kept its text on one line, so an agent's question (Claude Code's AskUserQuestion arrives as a permission request) was cut off and its answers crowded into the rest of the line. The question now wraps and scrolls past a third of the editor's height, and the answers wrap after it; permission lines in the conversation show up to three lines with the full text as tooltip.
2026-11-04 06:55:08 +08:00
l0ng-ai c5749b0c17 fix(settings): open the Settings window's content even when a startup step fails; log frontend errors (#1110)
The native Settings window showed only its title bar until its startup chain finished; a page that failed to register ended the chain silently. The steps are now guarded and openSettings runs in finally. Uncaught errors and unhandled rejections of every window (and caught errors via logError) are written to the log file through log_frontend, deduplicated and capped per window.
2026-11-04 06:51:41 +08:00
Andrea Basileandandreabasile 4d224f80c9 feat(lsp): per-server features and code-action-kinds filter (#1099)
A [[language-servers]] entry (or languageServers.<name> in settings) can list features and code-action-kinds: capabilities outside features are dropped after initialize and from later registrations, publishDiagnostics and Inspect Code only use servers with diagnostics, and code actions are narrowed to the kept kinds (advertised kinds and answers both). No built-in definition uses them yet. First stage of #901: lets a refactoring-only server sit next to the language's own one.

Co-authored-by: andreabasile <[email protected]>
2026-11-04 06:43:31 +08:00
l0ng-ai 63dd83c129 feat(build-tools): navigate Gradle version catalog accessors to their TOML entries (#1109)
In build scripts, Go to Declaration on a catalog accessor (libs.<alias>, versions./bundles./plugins.) opens the entry in the catalog file and hover shows its declaration with the referenced version. Catalogs are the build's gradle/libs.versions.toml and those declared in settings with from(files(...)), re-read when they change. In a catalog, version.ref values and bundle members go to their entries.
2026-11-04 06:14:07 +08:00
l0ng-ai afea58c1c6 feat(run): browse buttons on path fields and a resizable Run/Debug Configurations dialog (#1108)
Run configuration types mark path fields (browse: file or directory, or chosen from the other params) and the form puts a folder button next to them; Working directory and Redirect input get one too, storing project paths relative to the root. Covers Tomcat, Shell Script, Executable, Godot, npm, Gradle, Maven, Go, Ruby and Docker path fields. The dialog is resizable and its content grows with it.
2026-11-04 06:06:38 +08:00
l0ng-ai 19615e00b9 feat(vcs): select several repositories in the Git log Repository filter (#1107)
Clicking a repository in the Repository filter menu checks or unchecks it instead of replacing the selection, so the log shows any subset of the project's repositories; unchecking the last one shows all.
2026-11-04 06:00:20 +08:00
l0ng-ai c347e331ab feat(run): browse for .env files from the Environment variables field (#1106)
A folder button next to the Environment variables input opens a file dialog at the project root and appends the chosen files to the configuration's .env files (relative to the root inside the project, duplicates skipped). Hidden in remote windows, where the dialog would pick local paths.
2026-11-04 05:55:20 +08:00
Andrea Basileandandreabasile 9a6b84e287 feat(run): import JetBrains run configurations (.run/*.run.xml, .idea/workspace.xml) (#1039)
Run/Debug Configurations offers Import from JetBrains run configurations when the project has .run/*.run.xml, .idea/runConfigurations/*.xml or run configurations in .idea/workspace.xml; Find Action has Import Run Configurations from JetBrains (.run, .idea). Python, Django server/tests, pytest, npm, Jest, Vitest, Node.js, Shell Script and compound configurations map to the matching kinds; other types, unknown path variables and multi-module $MODULE_DIR$ are skipped with the reason. Shared .run files take precedence over .idea/runConfigurations and the workspace's RunManager (templates and temporary configurations left out). Path and argument helpers shared with the launch.json importer moved to import/mod.rs.

Co-authored-by: andreabasile <[email protected]>
2026-11-04 05:46:50 +08:00
Andrea Basileandandreabasile df6fbb6d21 fix(tools): offer to install only the first missing server of interchangeable ones (#1082)
Opening a Python file with none of basedpyright, Pyright, ty and Pyrefly installed raised four install notices at once: each fallback server reports missing after the one before it, and every missing server was prompted. Among interchangeable servers only the first missing one is offered now (events handled one at a time so a fallback sees whether the server before it was offered); a fallback is still offered when the servers before it can't be installed or are turned off.

Co-authored-by: andreabasile <[email protected]>
2026-11-04 05:24:23 +08:00
l0ng-ai d94499b7ac feat(vcs): the Push dialog shows the outgoing commits' changed files (#1105)
Below the outgoing commits, the Push dialog lists the files they change: all outgoing commits' combined changes by default, or the selected commit's (Show All Commits goes back). Selecting a file shows its diff beside the tree inside the modal dialog. The multi-repository Push dialog does the same for the selected repository or commit.
2026-11-04 05:20:58 +08:00
l0ng-ai f148c1ac6b fix(debugger): muted breakpoints show as grey dots, not as disabled ones (#1104)
With Mute Breakpoints on (remembered per project), every breakpoint was drawn as a hollow red circle, the disabled look. Muted breakpoints now use the grey muted icon; disabled ones keep the hollow circle. Method breakpoints and field watchpoints follow the same rule.
2026-11-04 04:56:21 +08:00
l0ng-ai 5ad5dc2a28 feat(project-tree): Undo restores files deleted in the Project view (#1103)
Ctrl+Z / Cmd+Z in the Project view (and Edit › Undo while it has focus) reverts the latest file operation there: a delete as well as a move or rename. Deleted files come back from their last Local History snapshot, folders are re-created and the restored items selected; items without a snapshot are named in a warning. Edit › Undo reaches the view through a new registerUndoHandler contribution; the Project view is found by a data attribute instead of its translated aria-label (also fixes the New popup's fallback anchor).
2026-11-04 04:53:48 +08:00
l0ng-ai fe0ee338f4 feat(debugger): completion in Evaluate, the Variables evaluate field and the Debug console (#1102)
While suspended, the debugger's expression fields complete from the debug adapter (DAP completions) in the selected frame: Evaluate, the Variables evaluate field and the Debug console share one hook. The list opens while typing an identifier or after '.', or on Ctrl+Space; Up/Down, Enter/Tab, Escape. The Evaluate dialog is resizable and its result tree grows with it. Backend: ide-dap completions (skipped without supportsCompletionsRequest), debug_completions on the host and the desktop app.
2026-11-04 04:47:12 +08:00
l0ng-ai e575bf8c24 fix(editor): the editor color scheme follows a switch between light and dark themes (#1100)
Switching the IDE theme kept an explicitly chosen editor color scheme, so a light theme could keep a dark editor. When the applied theme turns light or dark, a chosen scheme of the other kind now switches back to the theme's own; a scheme picked to go with the current theme is kept. Reacts to the applied theme, so Quick Switch, Settings, Welcome › Customize, other windows and Sync with OS are all covered.
2026-11-04 04:31:59 +08:00
l0ng-ai 1ed9275da0 fix(remote): Docker run configurations take their environment from the remote host (#1098)
The remote server didn't serve docker_run_env, so a remote window asked the local app, whose PATH then replaced the remote host's PATH in a run that starts there: docker compose failed with "docker: not found". The host now serves docker_run_env from its own PATH and docker executable; covers the Dockerfile, image and Compose run types.
2026-11-04 04:24:53 +08:00
l0ng-ai 3a98d5c5be fix(remote-dev): list this machine's SSH configurations in a remote project's window (#1097)
A remote project's window gets the remote host's settings, which don't include this machine's SSH configurations, so the Remote Development dialog showed "No SSH configurations" there. The connection is made from this machine, so the dialog reads the local global SSH configurations and reloads them when settings change. Other SSH-configuration consumers run on the host and keep the host's settings.
2026-11-04 04:16:18 +08:00
l0ng-ai 6b9ab3b323 fix(settings): Export Settings includes IDE view options (#1095)
Project view options such as "Open Files with Single Click" live in localStorage under openidea.projectTree.options, which was not on the export allowlist, so Export/Import Settings dropped them.

Other IDE-level preferences stored the same way are exported and restored too: TODO patterns, IdeaVim settings, diff/merge options, branches popup options, markdown and image viewer defaults, reopen last project. Per-project state stays out.
2026-11-04 04:01:30 +08:00
l0ng-ai 23e468252f fix(vcs): link filtered Git Log commits past the commits a filter hides (#1094)
With a User or text filter, the Git Log graph opened a lane for every commit whose parent the filter hid; the lanes never closed and filled the graph with arrows, since git keeps the real parents for these filters.

The log now links each shown commit to its nearest shown ancestors, dashed where the link passes hidden commits (the date range and No Merges count as hidden there too); a page's last commits are linked by walking on past the page. Collapse Linear Branches keeps those dashed links.

Fixes #872
2026-11-04 03:57:08 +08:00
l0ng-ai a6c5574194 feat(build): Build Project for CMake projects (#1093)
Build › Build Project and Rebuild Project reported "no supported build system" in a CMake project. They now configure the build directory when needed and run cmake --build (--clean-first for Rebuild), with the directory and options of the first CMake profile, else a configured directory or build. GCC / Clang errors and warnings are reported at their source positions.

Fixes #873
2026-11-04 03:33:59 +08:00
l0ng-ai 5d63f618ef fix(maven): include the modules of the profiles chosen in the Maven tool window (#1092)
Checking a profile in Maven › Profiles only passed -P to Maven runs and syncs: the project model was read from the POMs with the activeByDefault profiles alone, so a module declared in a chosen profile stayed a plain folder and its dependencies and properties were missing.

The POMs are now read with the chosen profiles (saved with the last sync), with Maven's rule that choosing one of a POM's profiles turns its activeByDefault ones off. A profile change refreshes the project roots at once, so the project tree and Maven tool window show the new modules even when Maven itself cannot run.

Fixes #850
2026-11-04 03:27:10 +08:00
l0ng-ai 2ba07341c2 feat(editor): mark syntax errors in files no language server checks (#1087)
Without a running language server (none configured for the language, or not installed, stopped or crashed), the editor showed no syntax errors at all, though the built-in parser already finds them. The editor now reports the parser's errors for such files, checked again after edits and when a language server starts or stops; while a server runs for the language it reports syntax errors itself. SQL is left to the database support. The check is the built-in Syntax errors inspection.

Fixes #824
2026-11-04 03:02:32 +08:00
l0ng-ai 0dffad814a fix(editor): show wavy and dotted underlines on macOS (#1086)
Error, warning, weak-warning and typo underlines never showed on macOS, nor did underline effects of syntax tokens from a colour scheme: WebKit rejects a text-decoration shorthand with a style or colour (underline wavy <colour>), so the declaration fell back to none.

The editor's diagnostic and token styles also set -webkit-text-decoration, which WebKit takes the full value through; the red wavy underlines of the Problems, Build, Project and Database views use the longhand properties.
2026-11-04 03:01:03 +08:00
1c5a92f952 feat(plugins): plugin system with extension points, consent, Wasm runtime and plugin API 1.0 (#971)
A plugin system built on WebAssembly components (wasmtime, WASI p2), with a
typed WIT API (`openidea:[email protected]`) and a Rust SDK, shipped as a preview.

- Extension points declared in `plugin.toml` (commands, languages, themes,
  icon themes, menus, key bindings, status items, settings, views, custom
  editors, file icons and templates, run detectors, editor providers),
  validated before anything takes effect and projected onto the workbench's
  own registries: plugin themes and file icon themes are chosen in
  Settings › Appearance and Quick Switch, plugin shortcuts are edited in
  Settings › Keymap, plugin problems are grouped in Settings › Inspections.
- Capabilities with risk levels and consent at install, pinned `process:`
  programs, secrets in the OS keychain.
- Runtime: one thread per active plugin, fuel/time/memory budgets, soft and
  hard deadlines on the editing path, cancellation, crash backoff, idle
  reclaim, a precompiled-component cache; remote projects run code plugins on
  `openidea-server`.
- Web views and custom editors with a sandboxed page, a document bridge
  joined to the text editor's undo history, and split or preview layouts.
- Distribution: signed `.oiplugin` packages, a static marketplace index with
  revocation, versioned installs with rollback, install from a `.zip` or a
  folder. The release workflow signs and publishes the official plugins and
  their index, which released builds use as the default marketplace.
- Official plugins: String Tools, Key Promoter, Git Toolbox, AI Completion,
  Checkstyle, MyBatis Navigator, CSV Editor, Solidity, Material Icons, Harbor
  Night, Excalidraw, Graphviz, Regex Tester, License Header.

Closes #732

Co-authored-by: Deng Liu <[email protected]>
Co-authored-by: florentvdry <[email protected]>
2026-11-04 02:54:31 +08:00
l0ng-ai cfdbd3872a feat(find): browse for the Directory scope's folder in Find in Files (#1085)
The Directory scope of Find in Files only took a typed path. A folder button next to the field now opens a folder chooser at the current directory and fills the field with the chosen folder.

Fixes #847
2026-11-04 02:53:03 +08:00
852 changed files with 88232 additions and 4700 deletions

No files matched your search

+51
View File
@@ -71,6 +71,57 @@ jobs:
- run: cargo build --locked -p ide-host --bin openidea-server --target x86_64-unknown-linux-musl
- run: target/x86_64-unknown-linux-musl/debug/openidea-server --version
# The plugin SDK, the official plugins and the test fixtures build in their
# own workspaces (the root excludes them), so the jobs above don't see them.
plugins:
name: Plugins (SDK, official, fixtures)
runs-on: ubuntu-22.04
env:
CARGO_TARGET_DIR: ${{ github.workspace }}/target-plugins
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with: { targets: wasm32-wasip2, components: "clippy, rustfmt" }
- uses: Swatinem/rust-cache@v2
with:
key: plugins
cache-directories: target-plugins
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Format, lint (host and wasm) and test each crate
shell: bash
run: |
set -euo pipefail
for dir in sdk/rust/openidea-plugin plugins/official/*/ crates/ide-plugin/tests/fixtures/*/; do
[ -f "$dir/Cargo.toml" ] || continue
echo "::group::$dir"
cargo fmt --check --manifest-path "$dir/Cargo.toml"
cargo clippy --locked --all-targets --manifest-path "$dir/Cargo.toml" -- -D warnings
cargo clippy --locked --target wasm32-wasip2 --manifest-path "$dir/Cargo.toml" -- -D warnings
cargo test --locked --manifest-path "$dir/Cargo.toml"
echo "::endgroup::"
done
- uses: pnpm/action-setup@v4
with: { version: 11 }
- uses: actions/setup-node@v4
with: { node-version: 22 }
# Plugin web pages (`web-src/`, their own pnpm projects): typecheck and test.
- name: Web pages typecheck and test
shell: bash
run: |
set -euo pipefail
for dir in plugins/official/*/web-src/; do
[ -f "$dir/package.json" ] || continue
echo "::group::$dir"
pnpm --dir "$dir" install --frozen-lockfile --ignore-workspace
pnpm --dir "$dir" exec tsc --noEmit
pnpm --dir "$dir" test
echo "::endgroup::"
done
# The host's tests run the checked-in components: they must be built from these sources.
# Web pages aren't checked in: this builds them and checks the result.
- name: Checked-in components are up to date, web pages build
run: plugins/build.sh --check
deny:
name: Cargo deny
runs-on: ubuntu-latest
+46
View File
@@ -0,0 +1,46 @@
name: Plugin index
# Refreshes the community plugins in the latest release's plugins-index.json
# (what released apps read) from the registry, l0ng-ai/openidea-plugins. The
# official plugins, their key and revocations stay as the release wrote them.
on:
schedule:
- cron: "17 * * * *"
workflow_dispatch:
# Sent by the registry when its index changes.
repository_dispatch:
types: [community-plugins]
permissions:
contents: write
concurrency:
group: plugins-index
env:
COMMUNITY_PLUGINS_INDEX: https://github.com/l0ng-ai/openidea-plugins/releases/download/index/index.json
jobs:
merge:
runs-on: ubuntu-latest
steps:
- name: Merge the community index
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
cd "$RUNNER_TEMP"
tag="$(gh release view --repo "$GITHUB_REPOSITORY" --json tagName -q .tagName)"
tool=openidea-plugin-linux-x86_64
if ! gh release download "$tag" --repo "$GITHUB_REPOSITORY" -p plugins-index.json -p "$tool.gz"; then
echo "::notice::$tag has no plugin index or openidea-plugin tool yet"
exit 0
fi
gunzip "$tool.gz" && chmod +x "$tool"
cp plugins-index.json before.json
"./$tool" merge plugins-index.json "$COMMUNITY_PLUGINS_INDEX" --keep openidea
if cmp -s before.json plugins-index.json; then
echo "unchanged"
else
gh release upload "$tag" plugins-index.json --repo "$GITHUB_REPOSITORY" --clobber
fi
+92 -1
View File
@@ -18,6 +18,7 @@ env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
TAG: ${{ github.event.inputs.tag || github.ref_name }}
COMMUNITY_PLUGINS_INDEX: https://github.com/l0ng-ai/openidea-plugins/releases/download/index/index.json
jobs:
# Creates (or reuses) one draft release that every platform build uploads into.
@@ -25,6 +26,7 @@ jobs:
runs-on: ubuntu-latest
outputs:
release_id: ${{ steps.release.outputs.id }}
official_plugin_keys: ${{ steps.plugin-keys.outputs.keys }}
steps:
- uses: actions/checkout@v4
with: { ref: "${{ env.TAG }}" }
@@ -35,6 +37,20 @@ jobs:
echo "::error::tag $TAG does not match tauri.conf.json version $app"
exit 1
fi
# The official plugins' public key (from PLUGIN_SIGNING_KEY, the seed
# `openidea-plugin keygen` writes), built into the app: their packages
# must carry its signature whatever the marketplace index lists. Without
# the secret the app takes the index's word, as development builds do.
- id: plugin-keys
env:
SIGNING_KEY: ${{ secrets.PLUGIN_SIGNING_KEY }}
run: |
if [ -n "$SIGNING_KEY" ]; then
seed="$(printf %s "$SIGNING_KEY" | base64 -d | od -An -tx1 | tr -d ' \n')"
key="$(printf '302e020100300506032b657004220420%s' "$seed" | xxd -r -p |
openssl pkey -inform DER -pubout -outform DER | tail -c 32 | base64)"
echo "keys=$key" >> "$GITHUB_OUTPUT"
fi
- id: release
uses: actions/github-script@v7
with:
@@ -57,6 +73,8 @@ jobs:
build:
needs: create-release
name: Build (${{ matrix.name }})
env:
OPENIDEA_OFFICIAL_PLUGIN_KEYS: ${{ needs.create-release.outputs.official_plugin_keys }}
strategy:
fail-fast: false
matrix:
@@ -180,6 +198,79 @@ jobs:
if command -v sha256sum >/dev/null; then sha256sum "$ASSET"; else shasum -a 256 "$ASSET"; fi > "$ASSET.sha256"
gh release upload "$TAG" "$ASSET" "$ASSET.sha256" --clobber
# The official plugins, signed, and the marketplace index released apps read
# (releases/latest/download/plugins-index.json). They are not in the
# installers: users install them from Settings › Plugins. Without the
# signing key (secret PLUGIN_SIGNING_KEY: the file `openidea-plugin keygen`
# writes) nothing is published and the release goes on. The index also lists
# the community plugins of the registry (COMMUNITY_PLUGINS_INDEX; refreshed
# between releases by plugins-index.yml), and the release carries the
# openidea-plugin tool for Linux x64, which that workflow and the registry's
# checks run.
plugins:
needs: create-release
name: Official plugins
runs-on: ubuntu-22.04
env:
SIGNING_KEY: ${{ secrets.PLUGIN_SIGNING_KEY }}
steps:
- id: key
run: |
if [ -z "$SIGNING_KEY" ]; then
echo "::warning::PLUGIN_SIGNING_KEY is not set: the official plugins are not published"
echo "skip=true" >> "$GITHUB_OUTPUT"
fi
- uses: actions/checkout@v4
if: steps.key.outputs.skip != 'true'
with: { ref: "${{ env.TAG }}" }
- uses: dtolnay/rust-toolchain@stable
if: steps.key.outputs.skip != 'true'
with: { targets: wasm32-wasip2 }
- uses: Swatinem/rust-cache@v2
if: steps.key.outputs.skip != 'true'
with: { key: release-plugins }
- uses: pnpm/action-setup@v4
if: steps.key.outputs.skip != 'true'
with: { version: 11 }
- uses: actions/setup-node@v4
if: steps.key.outputs.skip != 'true'
with: { node-version: 22 }
# Builds the web pages; fails when a checked-in component isn't built from these sources.
- name: Build
if: steps.key.outputs.skip != 'true'
run: |
plugins/build.sh --check
cargo build --release --locked -p ide-host --bin openidea-plugin
- name: Pack, index and upload
if: steps.key.outputs.skip != 'true'
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
tool="$PWD/target/release/openidea-plugin"
key="$RUNNER_TEMP/openidea.key"
out="$RUNNER_TEMP/market"
(umask 077 && printf '%s' "$SIGNING_KEY" > "$key")
trap 'rm -f "$key"' EXIT
mkdir -p "$out"
for dir in plugins/official/*/; do
id="$(sed -n 's/^id *= *"\(.*\)".*/\1/p' "$dir/plugin.toml" | head -1)"
version="$(sed -n 's/^version *= *"\(.*\)".*/\1/p' "$dir/plugin.toml" | head -1)"
"$tool" pack "$dir" --key "$key" --publisher openidea --out "$out/$id-$version.oiplugin"
done
# The published index's revocations, earlier keys and community plugins carry over.
curl -fsSL "https://github.com/$GITHUB_REPOSITORY/releases/latest/download/plugins-index.json" \
-o "$out/plugins-index.json" || rm -f "$out/plugins-index.json"
# Downloads point at this release: older versions kept for older IDEs stay where they were published.
"$tool" index "$out" --publisher openidea --name OpenIDEA --key "$key" --out "$out/plugins-index.json" \
--download-base "https://github.com/$GITHUB_REPOSITORY/releases/download/$TAG"
"$tool" merge "$out/plugins-index.json" "$COMMUNITY_PLUGINS_INDEX" --keep openidea ||
echo "::warning::the community plugins were not refreshed"
gzip -9 -c "$tool" > "$out/openidea-plugin-linux-x86_64.gz"
gh release upload "$TAG" "$out"/*.oiplugin "$out/plugins-index.json" \
"$out/openidea-plugin-linux-x86_64.gz" --clobber
# The updater manifest (latest.json), built once from the signed bundles every build uploaded.
# Fails when a platform is missing, so a release never ships a manifest without it.
updater-manifest:
@@ -243,7 +334,7 @@ jobs:
core.info(`latest.json: ${Object.keys(platforms).sort().join(", ")}`);
publish:
needs: [create-release, build, server, updater-manifest]
needs: [create-release, build, server, plugins, updater-manifest]
runs-on: ubuntu-latest
steps:
- uses: actions/github-script@v7
+3
View File
@@ -10,6 +10,9 @@ lerna-debug.log*
node_modules
dist
dist-ssr
# Plugin components are checked in (plugins/build.sh --check keeps them current).
!plugins/official/*/dist
!crates/ide-plugin/tests/fixtures/*/dist
*.local
# Editor directories and files
+2
View File
@@ -15,6 +15,8 @@
"pnpm-lock.yaml",
"**/*.toml",
"fixtures",
"plugins/*/web",
"plugins/official/*/web",
"crates/**/tests/fixtures"
],
"overrides": [
+1
View File
@@ -39,6 +39,7 @@ Visual style: dark, low-contrast greys, one cool accent
icons from `src/ui/icons.tsx`, never emoji. Real `<button>`/`<input>`; `aria-label` on icon buttons.
- User-visible text goes through the text catalog: add every new string to both
`src/platform/locales/text/en.json` and `zh.json` (`textCatalog.test.ts` enforces it).
Other locales may lack entries; their UI shows the English text until a translator adds it.
- Keep comments sparse and purposeful; match surrounding style.
- For GUI verification, prefer Tauri MCP; use Computer Use only when Tauri MCP is
unavailable or cannot cover the required checks.
+5
View File
@@ -31,5 +31,10 @@ Project layout, coding rules, commands and checks are in [AGENTS.md](AGENTS.md).
- If a change needs a product decision you can't make, ask in the issue and stop
rather than guessing.
Plugins: pull requests here add or change official plugins (`plugins/official/`)
and the plugin API. Community plugins live in their authors' repositories and
are listed through [l0ng-ai/openidea-plugins](https://github.com/l0ng-ai/openidea-plugins);
see [`plugins/README.md`](plugins/README.md) § 8.1.
A check on each PR adds `needs-info` when the issue link or the verification
section is missing, and removes it once they're filled in.
Generated
+67 -3
View File
@@ -4154,8 +4154,11 @@ dependencies = [
"dirs 6.0.0",
"futures",
"ide-core",
"ide-http",
"ide-ssh",
"parking_lot",
"reqwest",
"rustls",
"serde",
"serde_json",
"specta",
@@ -4166,6 +4169,7 @@ dependencies = [
"tokio-util",
"toml 0.9.12+spec-1.1.0",
"tracing",
"uuid",
]
[[package]]
@@ -4181,6 +4185,26 @@ dependencies = [
"which",
]
[[package]]
name = "ide-extension"
version = "0.1.35"
dependencies = [
"base64 0.22.1",
"blake3",
"ed25519-dalek",
"ide-core",
"parking_lot",
"semver",
"serde",
"serde_json",
"specta",
"specta-typescript",
"tempfile",
"toml 0.9.12+spec-1.1.0",
"tracing",
"zip",
]
[[package]]
name = "ide-fonts"
version = "0.1.35"
@@ -4210,9 +4234,13 @@ dependencies = [
name = "ide-host"
version = "0.1.35"
dependencies = [
"base64 0.22.1",
"criterion",
"dirs 6.0.0",
"dunce",
"encoding_rs",
"getrandom 0.3.4",
"globset",
"ide-agent",
"ide-analysis",
"ide-build",
@@ -4220,12 +4248,15 @@ dependencies = [
"ide-coverage",
"ide-dap",
"ide-db",
"ide-docker",
"ide-extension",
"ide-http",
"ide-index",
"ide-jsonrpc",
"ide-language",
"ide-lsp",
"ide-notebook",
"ide-plugin",
"ide-remote",
"ide-run",
"ide-sdk",
@@ -4239,18 +4270,25 @@ dependencies = [
"ide-vcs",
"ide-vfs",
"ide-workspace",
"ignore",
"parking_lot",
"quick-xml",
"regex-syntax",
"reqwest",
"rustix",
"semver",
"serde",
"serde_json",
"specta",
"specta-typescript",
"tempfile",
"tokio",
"toml 0.9.12+spec-1.1.0",
"tracing",
"tracing-subscriber",
"uuid",
"windows-sys 0.61.2",
"zip",
]
[[package]]
@@ -4303,6 +4341,7 @@ dependencies = [
"parking_lot",
"serde",
"serde_json",
"tempfile",
"thiserror 2.0.21",
"tokio",
"tokio-util",
@@ -4316,6 +4355,7 @@ dependencies = [
"blake3",
"criterion",
"ide-core",
"ide-extension",
"ide-text",
"parking_lot",
"regex-lite",
@@ -4417,19 +4457,20 @@ dependencies = [
name = "ide-plugin"
version = "0.1.35"
dependencies = [
"blake3",
"ide-core",
"ide-language",
"ide-extension",
"parking_lot",
"semver",
"serde",
"serde_json",
"specta",
"tempfile",
"thiserror 2.0.21",
"toml 0.9.12+spec-1.1.0",
"tokio",
"tracing",
"wasmtime",
"wasmtime-wasi",
"zip",
]
[[package]]
@@ -6150,6 +6191,7 @@ dependencies = [
name = "openidea"
version = "0.1.35"
dependencies = [
"base64 0.22.1",
"blake3",
"block2 0.6.2",
"dirs 6.0.0",
@@ -6164,6 +6206,7 @@ dependencies = [
"ide-dap",
"ide-db",
"ide-docker",
"ide-extension",
"ide-fonts",
"ide-github",
"ide-host",
@@ -6195,6 +6238,8 @@ dependencies = [
"objc2-foundation 0.3.2",
"objc2-web-kit 0.3.2",
"parking_lot",
"percent-encoding",
"security-framework",
"serde",
"serde_json",
"specta",
@@ -7096,6 +7141,7 @@ dependencies = [
"rustls-platform-verifier",
"serde",
"serde_json",
"serde_urlencoded",
"sync_wrapper",
"tokio",
"tokio-rustls",
@@ -7454,6 +7500,12 @@ dependencies = [
"wait-timeout",
]
[[package]]
name = "ryu"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f"
[[package]]
name = "saa"
version = "5.6.2"
@@ -7758,6 +7810,18 @@ dependencies = [
"serde_core",
]
[[package]]
name = "serde_urlencoded"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3491c14715ca2294c4d6a88f15e84739788c1d030eed8c110436aafdaa2f3fd"
dependencies = [
"form_urlencoded",
"itoa",
"ryu",
"serde",
]
[[package]]
name = "serde_with"
version = "3.24.0"
+6
View File
@@ -1,6 +1,8 @@
[workspace]
resolver = "3"
members = ["crates/*", "src-tauri"]
# Plugins and the guest SDK build for wasm32-wasip2 in their own workspaces.
exclude = ["sdk", "plugins"]
[workspace.package]
version = "0.1.35"
@@ -41,6 +43,8 @@ ide-ssh = { path = "crates/ide-ssh" }
ide-remote = { path = "crates/ide-remote" }
ide-host = { path = "crates/ide-host" }
ide-notebook = { path = "crates/ide-notebook" }
ide-extension = { path = "crates/ide-extension" }
ide-plugin = { path = "crates/ide-plugin" }
# serialization & IPC typing
serde = { version = "1", features = ["derive"] }
@@ -69,6 +73,7 @@ ropey = "1.6"
encoding_rs = "0.8"
chardetng = "0.1"
ignore = "0.4"
base64 = "0.22"
globset = "0.4"
notify = "8"
notify-debouncer-full = "0.6"
@@ -81,6 +86,7 @@ zip = { version = "9", default-features = false, features = ["deflate"] }
# search, index, language
grep-searcher = "0.1"
grep-regex = "0.1"
regex-syntax = "0.8"
grep-matcher = "0.1"
nucleo-matcher = "0.3"
redb = "4"
+21 -5
View File
@@ -58,6 +58,8 @@ pub(crate) const LIMITS_UPDATE: &str = "openidea_limits";
const MAX_EMBED: u64 = 256 * 1024;
/// How long a probe waits for the agent to answer `initialize`.
const PROBE_TIMEOUT: Duration = Duration::from_secs(30);
/// How long deleting a task waits for its agent to exit before removing the worktree.
const EXIT_WAIT: Duration = Duration::from_secs(5);
/// The agent tasks of one project.
pub struct AgentManager {
@@ -345,6 +347,14 @@ impl AgentManager {
provider::set_builtin_command(&dir, id, command.as_deref())
}
/// Sets the environment variables built-in `id` starts with; empty clears them.
pub fn set_builtin_env(&self, id: &str, env: std::collections::BTreeMap<String, String>) -> IdeResult<()> {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::set_builtin_env(&dir, id, env)
}
/// Re-reads configured/installed agents, drops cached probes, and notifies subscribers.
pub fn reload_providers(&self) {
let specs = provider::load(self.shared.paths.custom_agents_dir.as_deref());
@@ -831,7 +841,7 @@ impl AgentManager {
/// Forgets the task; with `remove_worktree` also deletes its worktree (the branch stays).
pub async fn delete(&self, id: &str, remove_worktree: bool) -> IdeResult<()> {
let task = self.task(id)?;
task.kill();
let live = task.kill();
let (isolation, cwd, root) = {
let s = task.state.lock();
(
@@ -843,6 +853,11 @@ impl AgentManager {
// A copy of the task (a fork) may work in the same worktree.
let shared = self.tasks.read().values().any(|t| t.id != id && t.state.lock().task.cwd.0 == cwd);
if remove_worktree && !shared && isolation == Isolation::Worktree && cwd.exists() {
// On Windows the agent's processes lock the folder (their working directory,
// open files) until they have exited.
if let Some(peer) = live {
let _ = tokio::time::timeout(EXIT_WAIT, peer.exited()).await;
}
let worktrees = self.shared.paths.worktrees_dir.clone();
blocking(move || worktree::remove(&root, &worktrees, &cwd)).await?;
}
@@ -1597,11 +1612,12 @@ impl Task {
}
}
fn kill(&self) {
/// Stops the agent; returns its connection, to wait for its process to exit.
fn kill(&self) -> Option<acp::Conn> {
self.cancel_permissions();
if let Some(live) = self.live.lock().take() {
live.peer.kill();
}
let live = self.live.lock().take()?;
live.peer.kill();
Some(live.peer)
}
}
+3
View File
@@ -3,6 +3,7 @@
use ide_core::IpcPath;
use serde::{Deserialize, Serialize};
use specta::Type;
use std::collections::BTreeMap;
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
@@ -357,6 +358,8 @@ pub struct AgentProvider {
pub command_read_only_reason: Option<AgentCommandReadOnlyReason>,
/// Whether the command was found (on `PATH`).
pub installed: bool,
/// Environment variables it starts with (from its agent file).
pub env: BTreeMap<String, String>,
/// Settings (models, modes…) the agent offered last time it ran; empty before its first session.
pub config: Vec<AgentConfigOption>,
/// Slash commands it offered last time.
+30 -2
View File
@@ -124,6 +124,7 @@ pub(crate) fn describe(spec: &ProviderSpec) -> AgentProvider {
command_editable: reason.is_none(),
command_read_only_reason: reason,
installed: installed(&spec.command),
env: spec.env.iter().cloned().collect(),
config: vec![],
commands: vec![],
source: spec.source.clone(),
@@ -133,6 +134,16 @@ pub(crate) fn describe(spec: &ProviderSpec) -> AgentProvider {
/// Overrides built-in `id`'s command through its agent file, preserving other
/// customizations. `None` restores the built-in command and removes a redundant file.
pub(crate) fn set_builtin_command(custom_dir: &Path, id: &str, command: Option<&str>) -> IdeResult<()> {
update_builtin(custom_dir, id, |agent, builtin| agent.command = command.unwrap_or(&builtin.command).to_owned())
}
/// The environment built-in `id` starts with, through its agent file (empty clears it).
pub(crate) fn set_builtin_env(custom_dir: &Path, id: &str, env: BTreeMap<String, String>) -> IdeResult<()> {
update_builtin(custom_dir, id, |agent, _| agent.env = env)
}
/// Changes built-in `id`'s agent file; a file left with only the built-in's values is removed.
fn update_builtin(custom_dir: &Path, id: &str, change: impl FnOnce(&mut CustomAgent, &ProviderSpec)) -> IdeResult<()> {
let Some(builtin) = builtin_spec(id) else {
return Err(IdeError::invalid(format!("{id} is not a built-in agent")));
};
@@ -152,7 +163,8 @@ pub(crate) fn set_builtin_command(custom_dir: &Path, id: &str, command: Option<&
if agent.registry.is_some() {
return Err(IdeError::invalid(format!("{id} is installed from the ACP Registry")));
}
if command.is_none()
change(&mut agent, &builtin);
if agent.command == builtin.command
&& agent.name.as_deref() == Some(builtin.name.as_str())
&& agent.driver == builtin.driver
&& agent.args == builtin.args
@@ -164,7 +176,6 @@ pub(crate) fn set_builtin_command(custom_dir: &Path, id: &str, command: Option<&
}
return Ok(());
}
agent.command = command.unwrap_or(&builtin.command).to_owned();
agent.write(&file)?;
Ok(())
}
@@ -368,6 +379,23 @@ mod tests {
assert!(installed("sh"));
}
#[test]
fn set_builtin_env_writes_and_clears_the_environment() {
let dir = tempfile::tempdir().unwrap();
let env = BTreeMap::from([("CLAUDE_CODE_ENTRYPOINT".to_owned(), "openidea".to_owned())]);
set_builtin_env(dir.path(), "claude", env.clone()).unwrap();
let claude = load(Some(dir.path())).into_iter().find(|s| s.id == "claude").unwrap();
assert_eq!(claude.env, vec![("CLAUDE_CODE_ENTRYPOINT".to_owned(), "openidea".to_owned())]);
assert_eq!(describe(&claude).env, env);
// The command override and the environment are kept independently.
set_builtin_command(dir.path(), "claude", Some("/opt/claude")).unwrap();
set_builtin_command(dir.path(), "claude", None).unwrap();
assert!(dir.path().join("claude.toml").is_file());
set_builtin_env(dir.path(), "claude", BTreeMap::new()).unwrap();
assert!(!dir.path().join("claude.toml").exists());
set_builtin_env(dir.path(), "not-a-builtin", env).unwrap_err();
}
#[test]
fn set_builtin_command_overrides_and_clears_a_builtin() {
let dir = tempfile::tempdir().unwrap();
+111
View File
@@ -1109,3 +1109,114 @@ async fn idle_agents_stop_and_resume_their_session_on_the_next_message() {
assert!(live(&mgr, &task.id));
mgr.shutdown();
}
/// Deleting a task whose agent still runs in the worktree removes both and leaves no
/// process behind, whether it is waiting for permission or idle without session/load.
#[tokio::test(flavor = "multi_thread")]
async fn deleting_a_task_waits_for_its_agent_to_exit_before_removing_the_worktree() {
for no_load in [false, true] {
delete_running_agent(no_load).await;
}
}
async fn delete_running_agent(no_load: bool) {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
let git = |args: &[&str]| {
let out = std::process::Command::new("git")
.args(["-c", "user.name=t", "-c", "user.email=t@t", "-c", "commit.gpgsign=false"])
.args(args)
.current_dir(&root)
.output()
.unwrap();
assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr));
};
git(&["init", "-q", "-b", "main"]);
std::fs::write(root.join("file.txt"), "original\n").unwrap();
git(&["add", "."]);
git(&["commit", "-qm", "initial"]);
let agent = Arc::new(FakeAgent { no_load, ..FakeAgent::default() });
let pids: Arc<Mutex<Vec<u32>>> = Arc::default();
let (fake, started) = (agent.clone(), pids.clone());
// The fake agent, served on behalf of a real process that exits only when killed.
let connector: Connector = Arc::new(move |_spec, cwd, client, _stderr| {
#[cfg(unix)]
let mut cmd = {
let mut cmd = tokio::process::Command::new("sleep");
cmd.arg("30");
cmd
};
#[cfg(windows)]
let mut cmd = {
let mut cmd = tokio::process::Command::new("cmd");
cmd.args(["/D", "/C", "echo ready>agent-ready & ping -n 30 127.0.0.1 >nul"]);
cmd
};
ide_core::process::prepare(cmd.as_std_mut());
ide_core::process::own_group(cmd.as_std_mut());
cmd.current_dir(cwd).kill_on_drop(true);
let child = cmd.spawn()?;
started.lock().push(child.id().unwrap());
let (a, b) = duplex(1 << 20);
let (ar, aw) = tokio::io::split(a);
let (br, bw) = tokio::io::split(b);
// `connect_with_child` frames with Content-Length.
fake.servers.lock().push(Peer::connect(br, bw, fake.clone()));
Ok(Peer::connect_with_child(ar, aw, client, child))
});
let mgr = AgentManager::with_connector(
AgentPaths {
project_root: root.clone(),
store_dir: root.join("store"),
worktrees_dir: root.join("worktrees"),
models_file: root.join("models.json"),
custom_agents_dir: None,
home_dir: None,
},
tokio::runtime::Handle::current(),
connector,
);
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text(if no_load { "again" } else { "Fix the lexer" }),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Worktree,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, if no_load { AgentStatus::Ready } else { AgentStatus::NeedsInput }).await;
let pid = *pids.lock().last().expect("the agent was started");
#[cfg(windows)]
{
tokio::time::timeout(Duration::from_secs(5), async {
while !task.cwd.0.join("agent-ready").exists() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await
.unwrap();
assert_eq!(
std::fs::rename(&task.cwd.0, task.cwd.0.with_extension("moved")).unwrap_err().raw_os_error(),
Some(32),
"the running agent really locks its worktree"
);
}
mgr.delete(&task.id, true).await.unwrap();
assert!(!task.cwd.0.exists());
// Killed and reaped: not even a zombie is left.
#[cfg(unix)]
let alive = std::process::Command::new("kill").args(["-0", &pid.to_string()]).status().unwrap().success();
#[cfg(windows)]
let alive = {
let mut cmd = std::process::Command::new("tasklist");
ide_core::process::prepare(&mut cmd).args(["/FI", &format!("PID eq {pid}"), "/FO", "CSV", "/NH"]);
String::from_utf8_lossy(&cmd.output().unwrap().stdout).contains(&format!("\"{pid}\""))
};
assert!(!alive, "the agent (pid {pid}) is still running");
}
+70 -2
View File
@@ -5,7 +5,7 @@ use ide_core::{IdeError, IdeResult, IpcPath};
use ide_vcs::{ChangeKind, Repo};
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use std::time::UNIX_EPOCH;
use std::time::{Duration, UNIX_EPOCH};
pub(crate) struct Created {
pub path: PathBuf,
@@ -65,7 +65,27 @@ pub(crate) fn remove(project_root: &Path, worktrees_dir: &Path, path: &Path) ->
if !path.starts_with(worktrees_dir) || path == worktrees_dir {
return Err(err);
}
std::fs::remove_dir_all(path).map_err(|e| IdeError::internal(e.to_string()))
retrying(REMOVE_ATTEMPTS, REMOVE_PAUSE, || std::fs::remove_dir_all(path))
.map_err(|e| IdeError::internal(format!("{}: {e}", path.display())))
}
/// On Windows a folder can't be deleted while a process still has a file in it open
/// (os error 32): an agent's helper that outlived it, an antivirus scan, the search indexer.
/// Those let go within moments, so removing is retried for a few seconds.
const REMOVE_ATTEMPTS: u32 = 10;
const REMOVE_PAUSE: Duration = Duration::from_millis(300);
fn retrying(attempts: u32, pause: Duration, mut op: impl FnMut() -> std::io::Result<()>) -> std::io::Result<()> {
for _ in 1..attempts {
match op() {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => std::thread::sleep(pause),
_ => return Ok(()),
}
}
match op() {
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()),
r => r,
}
}
/// Files under `cwd` that differ from `base` (untracked files included), with line counts.
@@ -149,6 +169,54 @@ fn unique_slug(base: &str, taken: impl Fn(&str) -> bool) -> String {
mod tests {
use super::*;
#[test]
fn removing_is_retried_while_a_file_is_still_in_use() {
use std::io::{Error, ErrorKind};
// A sharing violation as Windows reports it (os error 32), a vanished folder, other errors.
let in_use = || Error::from_raw_os_error(32);
for (fails, outcome) in [(0, true), (3, true), (REMOVE_ATTEMPTS - 1, true), (REMOVE_ATTEMPTS, false)] {
let mut calls = 0;
let r = retrying(REMOVE_ATTEMPTS, Duration::ZERO, || {
calls += 1;
if calls <= fails { Err(in_use()) } else { Ok(()) }
});
assert_eq!(r.is_ok(), outcome, "{fails} failures");
assert_eq!(calls, (fails + 1).min(REMOVE_ATTEMPTS));
}
let mut calls = 0;
let gone = retrying(REMOVE_ATTEMPTS, Duration::ZERO, || {
calls += 1;
if calls == 1 { Err(in_use()) } else { Err(Error::from(ErrorKind::NotFound)) }
});
assert!(gone.is_ok(), "deleted meanwhile: done");
assert_eq!(calls, 2);
}
#[cfg(windows)]
#[test]
fn removing_waits_for_a_windows_file_lock_to_be_released() {
use std::os::windows::fs::OpenOptionsExt;
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
std::fs::write(repo.join("a.txt"), "one\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
let worktrees = tmp.path().join("wt");
let created = create(&repo, &worktrees, "Locked file", None).unwrap();
let file = std::fs::OpenOptions::new().read(true).share_mode(0).open(created.path.join("a.txt")).unwrap();
assert_eq!(std::fs::remove_file(created.path.join("a.txt")).unwrap_err().raw_os_error(), Some(32));
let release = std::thread::spawn(move || {
std::thread::sleep(Duration::from_millis(800));
drop(file);
});
let result = remove(&repo, &worktrees, &created.path);
release.join().unwrap();
result.unwrap();
assert!(!created.path.exists());
}
#[test]
fn slugs_are_branch_safe() {
assert_eq!(slugify("Fix lexer UTF-16 columns!"), "fix-lexer-utf-16-columns");
+104 -13
View File
@@ -10,6 +10,7 @@ pub use fingerprint::{LaunchFingerprint, launch_fingerprint};
use crate::process::{self, Stream};
use crate::{BuildModel, maven};
use ide_core::settings::CMakeProfile;
use ide_core::{CancellationToken, Diagnostic, IdeError, IdeResult, Pos, Range, Severity};
use parse::{ColUnit, Format, OutputParser, RawMessage};
use serde::{Deserialize, Serialize};
@@ -41,6 +42,8 @@ pub enum BuildSystem {
Go,
#[serde(rename = "typescript")]
TypeScript,
#[serde(rename = "cmake")]
CMake,
}
/// One command of a build.
@@ -152,6 +155,8 @@ fn detect_dir(dir: &Path, model: &BuildModel, root: &Path) -> Option<BuildSystem
Some(BuildSystem::Go)
} else if has(dir, &["tsconfig.json", "package.json"]) {
Some(BuildSystem::TypeScript)
} else if has(dir, &["CMakeLists.txt"]) {
Some(BuildSystem::CMake)
} else {
None
}
@@ -176,7 +181,14 @@ fn ancestors_within<'a>(root: &'a Path, file: &'a Path) -> impl Iterator<Item =
}
/// The build of `root`, or of the module containing `file` (Build Module / Recompile).
pub fn plan(root: &Path, model: &BuildModel, kind: BuildKind, file: Option<&Path>) -> IdeResult<BuildPlan> {
/// `cmake`: the CMake profiles of the settings (the first one is built), if any.
pub fn plan(
root: &Path,
model: &BuildModel,
kind: BuildKind,
file: Option<&Path>,
cmake: Option<&[CMakeProfile]>,
) -> IdeResult<BuildPlan> {
let project_name = root.file_name().map_or_else(|| "project".to_owned(), |n| n.to_string_lossy().into_owned());
let module_dir = match file {
Some(f) => {
@@ -192,7 +204,9 @@ pub fn plan(root: &Path, model: &BuildModel, kind: BuildKind, file: Option<&Path
let system = detect_dir(&dir, model, root)
.or_else(|| if model.gradle.is_some() { Some(BuildSystem::Gradle) } else { None })
.or_else(|| model.maven.as_ref().filter(|m| !m.projects.is_empty()).map(|_| BuildSystem::Maven))
.ok_or_else(|| IdeError::not_found("no supported build system (Maven, Gradle, Cargo, Go, TypeScript) found"))?;
.ok_or_else(|| {
IdeError::not_found("no supported build system (Maven, Gradle, Cargo, Go, TypeScript, CMake) found")
})?;
let module = module_dir.as_deref().filter(|d| *d != root);
let title = match module {
Some(d) => format!(
@@ -209,6 +223,7 @@ pub fn plan(root: &Path, model: &BuildModel, kind: BuildKind, file: Option<&Path
// Go builds the file's package (a module holds many).
BuildSystem::Go => go_steps(file.and_then(Path::parent).unwrap_or(&dir), kind),
BuildSystem::TypeScript => ts_steps(&dir, kind),
BuildSystem::CMake => cmake_steps(&dir, kind, cmake.and_then(<[_]>::first)),
};
Ok(BuildPlan { system, title, steps })
}
@@ -302,6 +317,50 @@ fn cargo_steps(dir: &Path, kind: BuildKind) -> Vec<BuildStep> {
}
}
/// CLion's Build Project for a CMake project: configures the build directory when it has no
/// `CMakeCache.txt` yet, then `cmake --build` (`--clean-first` to rebuild). The directory and
/// options come from the CMake profile; without one, a configured directory of the project
/// (the first by name), else `build`, as the generated run configurations use.
fn cmake_steps(dir: &Path, kind: BuildKind, profile: Option<&CMakeProfile>) -> Vec<BuildStep> {
let build_dir = match profile {
Some(p) if !p.build_dir.trim().is_empty() => p.build_dir.trim().to_owned(),
Some(p) => format!("cmake-build-{}", p.build_type.to_lowercase()),
None => {
let mut configured: Vec<String> = std::fs::read_dir(dir)
.into_iter()
.flatten()
.flatten()
.filter(|e| e.path().join("CMakeCache.txt").is_file())
.map(|e| e.file_name().to_string_lossy().into_owned())
.collect();
configured.sort();
configured.into_iter().next().unwrap_or_else(|| "build".into())
}
};
let build_type = profile.map_or("Debug", |p| p.build_type.as_str());
let env = profile.map(|p| p.env.clone()).unwrap_or_default();
let mut steps = Vec::new();
if !dir.join(&build_dir).join("CMakeCache.txt").is_file() {
let mut configure = BuildStep::new("cmake", &["-S", ".", "-B", &build_dir], dir, Format::Gcc);
if let Some(p) = profile.filter(|p| !p.generator.trim().is_empty()) {
configure.args.extend(["-G".into(), p.generator.clone()]);
}
configure.args.push(format!("-DCMAKE_BUILD_TYPE={build_type}"));
configure.args.push("-DCMAKE_EXPORT_COMPILE_COMMANDS=ON".into());
configure.args.extend(profile.map(|p| p.cmake_args.clone()).unwrap_or_default());
configure.env = env.clone();
steps.push(configure);
}
let mut build = BuildStep::new("cmake", &["--build", &build_dir, "--config", build_type], dir, Format::Gcc);
if kind == BuildKind::Rebuild {
build.args.push("--clean-first".into());
}
build.args.extend(profile.map(|p| p.build_args.clone()).unwrap_or_default());
build.env = env;
steps.push(build);
steps
}
fn go_steps(dir: &Path, kind: BuildKind) -> Vec<BuildStep> {
let module_root = dir.ancestors().find(|d| d.join("go.mod").is_file()).unwrap_or(dir);
let target = if module_root == dir { "./...".to_owned() } else { format!("./{}/...", rel_str(module_root, dir)) };
@@ -687,7 +746,7 @@ fn launch_plan(root: &Path, model: &BuildModel, module: &Path, test: bool) -> Id
} else {
BuildKind::Build
};
let mut plan = plan(root, model, kind, Some(module))?;
let mut plan = plan(root, model, kind, Some(module), None)?;
if plan.system == BuildSystem::Maven && !test {
for step in &mut plan.steps {
if let Some(goal) = step.args.last_mut() {
@@ -796,12 +855,12 @@ mod tests {
write(&root, "core/pom.xml", "<project><artifactId>core</artifactId></project>");
write(&root, "core/src/main/java/A.java", "class A {}");
let model = crate::load(&root, &tmp.path().join("cache"), None);
let p = plan(&root, &model, BuildKind::Build, None).unwrap();
let p = plan(&root, &model, BuildKind::Build, None, None).unwrap();
assert_eq!(p.system, BuildSystem::Maven);
assert_eq!(p.title, "demo");
assert_eq!(p.steps.len(), 1);
assert_eq!(p.steps[0].args, ["-B", "-Dstyle.color=never", "test-compile"]);
let p = plan(&root, &model, BuildKind::Rebuild, Some(&root.join("core/src/main/java/A.java"))).unwrap();
let p = plan(&root, &model, BuildKind::Rebuild, Some(&root.join("core/src/main/java/A.java")), None).unwrap();
assert_eq!(p.steps[0].args, ["-B", "-Dstyle.color=never", "-pl", "core", "-am", "clean", "test-compile"]);
assert_eq!(p.steps[0].cwd, root);
assert_eq!(p.title, "demo: build module 'core'");
@@ -810,24 +869,24 @@ mod tests {
write(&g, "settings.gradle", "include 'app'");
write(&g, "app/build.gradle", "apply plugin: 'java'");
let model = crate::load(&g, &tmp.path().join("cache2"), None);
let p = plan(&g, &model, BuildKind::Build, Some(&g.join("app/src/A.java"))).unwrap();
let p = plan(&g, &model, BuildKind::Build, Some(&g.join("app/src/A.java")), None).unwrap();
assert_eq!(p.system, BuildSystem::Gradle);
assert_eq!(p.steps[0].args, ["--console=plain", ":app:classes", ":app:testClasses"]);
assert_eq!(
plan(&g, &model, BuildKind::Artifacts, None).unwrap().steps[0].args,
plan(&g, &model, BuildKind::Artifacts, None, None).unwrap().steps[0].args,
["--console=plain", "assemble"]
);
let c = tmp.path().join("c");
write(&c, "Cargo.toml", "[package]\nname='c'");
let p = plan(&c, &BuildModel::default(), BuildKind::Rebuild, None).unwrap();
let p = plan(&c, &BuildModel::default(), BuildKind::Rebuild, None, None).unwrap();
assert_eq!(p.system, BuildSystem::Cargo);
assert_eq!(p.steps.len(), 2);
assert_eq!(p.steps[0].args[0], "clean");
let go = tmp.path().join("go");
write(&go, "go.mod", "module x");
let p = plan(&go, &BuildModel::default(), BuildKind::Build, Some(&go.join("pkg/a/a.go"))).unwrap();
let p = plan(&go, &BuildModel::default(), BuildKind::Build, Some(&go.join("pkg/a/a.go")), None).unwrap();
assert_eq!(p.steps[0].args, ["build", "./pkg/a/..."]);
assert_eq!(p.steps[0].cwd, go);
@@ -835,19 +894,51 @@ mod tests {
write(&ts, "package.json", r#"{"scripts":{"build":"vite build"}}"#);
write(&ts, "tsconfig.json", "{}");
write(&ts, "pnpm-lock.yaml", "");
let p = plan(&ts, &BuildModel::default(), BuildKind::Build, None).unwrap();
let p = plan(&ts, &BuildModel::default(), BuildKind::Build, None, None).unwrap();
assert_eq!(p.steps[0].args, ["--noEmit", "--pretty", "false", "-p", "."]);
let p = plan(&ts, &BuildModel::default(), BuildKind::Artifacts, None).unwrap();
let p = plan(&ts, &BuildModel::default(), BuildKind::Artifacts, None, None).unwrap();
assert_eq!(
(p.steps[0].program.as_str(), p.steps[0].args.as_slice()),
("pnpm", &["run".to_owned(), "build".to_owned()][..])
);
let cm = tmp.path().join("cm");
write(&cm, "CMakeLists.txt", "project(app)");
let args = |p: &BuildPlan| p.steps.iter().map(|s| s.args.join(" ")).collect::<Vec<_>>();
let p = plan(&cm, &BuildModel::default(), BuildKind::Build, None, None).unwrap();
assert_eq!(p.system, BuildSystem::CMake);
assert_eq!(
args(&p),
[
"-S . -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_EXPORT_COMPILE_COMMANDS=ON",
"--build build --config Debug"
],
"an unconfigured project is configured first"
);
write(&cm, "cmake-build-release/CMakeCache.txt", "");
let p = plan(&cm, &BuildModel::default(), BuildKind::Rebuild, None, None).unwrap();
assert_eq!(
args(&p),
["--build cmake-build-release --config Debug --clean-first"],
"a configured directory is reused"
);
let profile = CMakeProfile {
build_type: "Release".into(),
generator: "Ninja".into(),
build_args: vec!["-j".into(), "8".into()],
..CMakeProfile::default()
};
let p =
plan(&cm, &BuildModel::default(), BuildKind::Build, None, Some(std::slice::from_ref(&profile))).unwrap();
assert_eq!(args(&p), ["--build cmake-build-release --config Release -j 8"]);
let p = plan(&cm, &BuildModel::default(), BuildKind::Build, None, Some(&[CMakeProfile::default()])).unwrap();
assert_eq!(p.steps[0].args[..4], ["-S", ".", "-B", "cmake-build-debug"]);
let siblings = tmp.path().join("siblings");
write(&siblings, "core/pom.xml", "<project><artifactId>core</artifactId></project>");
write(&siblings, "system/pom.xml", "<project><artifactId>system</artifactId></project>");
let model = crate::load(&siblings, &tmp.path().join("cache3"), None);
let p = plan(&siblings, &model, BuildKind::Build, None).unwrap();
let p = plan(&siblings, &model, BuildKind::Build, None, None).unwrap();
assert_eq!(p.system, BuildSystem::Maven);
assert_eq!(
p.steps.iter().map(|s| s.cwd.clone()).collect::<Vec<_>>(),
@@ -856,7 +947,7 @@ mod tests {
let none = tmp.path().join("none");
std::fs::create_dir_all(&none).unwrap();
assert!(plan(&none, &BuildModel::default(), BuildKind::Build, None).is_err());
assert!(plan(&none, &BuildModel::default(), BuildKind::Build, None, None).is_err());
}
#[test]
+32
View File
@@ -22,6 +22,8 @@ pub(crate) enum Format {
Tsc,
/// `go build`: `./a.go:12:5: msg`.
Go,
/// GCC / Clang (through `cmake --build`): `src/a.cpp:12:5: error: msg`.
Gcc,
}
/// What a printed column counts.
@@ -296,6 +298,17 @@ fn go_line(line: &str) -> Option<RawMessage> {
(!msg.is_empty()).then(|| RawMessage::new(Severity::Error, msg, ColUnit::Byte).at(path, ln, col))
}
/// A GCC / Clang diagnostic line; notes and the context lines after it are left as output.
fn gcc_line(line: &str) -> Option<RawMessage> {
let (path, ln, col, rest) = path_line_col(line)?;
let rest = rest.strip_prefix(':')?.trim_start();
let (severity, msg) =
[("fatal error:", Severity::Error), ("error:", Severity::Error), ("warning:", Severity::Warning)]
.into_iter()
.find_map(|(prefix, severity)| rest.strip_prefix(prefix).map(|m| (severity, m.trim())))?;
(!msg.is_empty()).then(|| RawMessage::new(severity, msg, ColUnit::Byte).at(path, ln, col))
}
/// rustc's JSON diagnostic (`reason: compiler-message`) → message and its rendered text.
fn cargo_json(v: &Value) -> Option<(Option<RawMessage>, String)> {
if v.get("reason")?.as_str()? != "compiler-message" {
@@ -410,6 +423,7 @@ impl OutputParser {
Format::Gradle => javac_header(&line).or_else(|| kotlin_line(&line)),
Format::Tsc => tsc_line(&line),
Format::Go => go_line(&line),
Format::Gcc => gcc_line(&line),
Format::Cargo => None,
};
if let Some(msg) = parsed {
@@ -563,6 +577,24 @@ mod tests {
assert_eq!(out[2].text, "Cannot find name 'foo'.");
}
#[test]
fn gcc_errors_and_warnings() {
let out = parse(
Format::Gcc,
"[1/2] Building CXX object CMakeFiles/app.dir/main.cpp.o\n\
/p/src/main.cpp:12:5: error: use of undeclared identifier 'foo'\n\
\x20 12 | foo();\n\
/p/src/util.h:3:10: warning: unused variable 'x' [-Wunused-variable]\n\
/p/src/main.cpp:4:10: fatal error: 'missing.h' file not found\n\
/p/src/main.cpp:2:1: note: in file included from here\n",
);
assert_eq!(out.len(), 3, "{out:?}");
assert_eq!(out[0].severity, Severity::Error);
assert_eq!(out[0].text, "use of undeclared identifier 'foo'");
assert_eq!(out[1].severity, Severity::Warning);
assert_eq!(out[2].text, "'missing.h' file not found");
}
#[test]
fn go_errors() {
let out = parse(
+257
View File
@@ -0,0 +1,257 @@
//! A Gradle project's resolved dependencies for the Dependency Analyzer:
//! `gradle <project>:dependencies --configuration <name>`, parsed from its tree report.
use crate::gradle;
use crate::process;
use crate::sync::{SyncOptions, error_summary};
use ide_core::{CancellationToken, IdeError, IdeResult};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::path::Path;
/// One line of the report: an occurrence of a dependency where it is requested.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct GradleDependency {
/// `group`, empty for a project dependency.
pub group: String,
/// Artifact name, or the project path (`:core`) of a project dependency.
pub name: String,
/// The version asked for here; `None` when a platform or constraint supplies it.
pub requested: Option<String>,
/// The version Gradle selected (`-> 1.2`), when it differs from the request.
pub selected: Option<String>,
/// A project of the build (`project :core`, or a module substituted by one).
pub project: bool,
/// `(*)`: listed with its dependencies earlier in the tree.
pub repeated: bool,
/// `(c)`: a dependency constraint rather than a dependency.
pub constraint: bool,
/// `(n)` / `FAILED`: not resolved.
pub unresolved: bool,
pub children: Vec<GradleDependency>,
}
/// Parses the tree of a `dependencies --configuration` report (lines `+--- g:a:1.0 -> 1.1 (*)`).
pub fn parse_report(output: &str) -> Vec<GradleDependency> {
let mut roots: Vec<GradleDependency> = Vec::new();
// Path of child indexes from the roots to the last node of each depth.
let mut path: Vec<usize> = Vec::new();
for line in output.lines() {
let Some((depth, text)) = tree_line(line) else { continue };
let node = parse_entry(text);
path.truncate(depth);
if depth > path.len() {
continue; // A line deeper than its parent allows: not part of a well-formed tree.
}
let siblings = if depth == 0 {
&mut roots
} else {
let mut list = &mut roots;
for &i in &path[..depth - 1] {
list = &mut list[i].children;
}
let Some(parent) = list.get_mut(path[depth - 1]) else { continue };
&mut parent.children
};
siblings.push(node);
path.push(siblings.len() - 1);
}
roots
}
/// `| \--- g:a:1.0` → (1, "g:a:1.0").
fn tree_line(line: &str) -> Option<(usize, &str)> {
let mut rest = line;
let mut depth = 0;
loop {
if let Some(r) = rest.strip_prefix("+--- ").or_else(|| rest.strip_prefix("\\--- ")) {
return Some((depth, r.trim()));
}
rest = rest.strip_prefix("| ").or_else(|| rest.strip_prefix(" "))?;
depth += 1;
}
}
fn parse_entry(text: &str) -> GradleDependency {
let mut d = GradleDependency::default();
let mut text = text.trim();
loop {
if let Some(t) = text.strip_suffix(" (*)") {
d.repeated = true;
text = t;
} else if let Some(t) = text.strip_suffix(" (c)") {
d.constraint = true;
text = t;
} else if let Some(t) = text.strip_suffix(" (n)").or_else(|| text.strip_suffix(" FAILED")) {
d.unresolved = true;
text = t;
} else {
break;
}
text = text.trim_end();
}
let (left, right) = match text.split_once(" -> ") {
Some((l, r)) => (l.trim(), Some(r.trim())),
None => (text, None),
};
if let Some(project) = left.strip_prefix("project ") {
d.project = true;
d.name = project_path(project);
} else {
let mut parts = left.splitn(3, ':');
d.group = parts.next().unwrap_or_default().to_owned();
d.name = parts.next().unwrap_or_default().to_owned();
d.requested = parts.next().filter(|v| !v.is_empty()).map(str::to_owned);
}
if let Some(right) = right {
if let Some(project) = right.strip_prefix("project ") {
// A module substituted by a project of the build.
d.project = true;
d.selected = Some(project_path(project));
} else {
d.selected = Some(right.to_owned());
}
}
d
}
/// `:core`, or `':core'` as Gradle 9 quotes it.
fn project_path(text: &str) -> String {
let t = text.trim();
t.strip_prefix('\'').and_then(|t| t.strip_suffix('\'')).unwrap_or(t).to_owned()
}
/// Runs the `dependencies` report of `project` (a Gradle path, `:` for the root) for
/// `configuration` and returns its tree.
pub fn gradle_dependencies(
root: &Path,
options: &SyncOptions,
project: &str,
configuration: &str,
cancel: &CancellationToken,
) -> IdeResult<Vec<GradleDependency>> {
if !configuration.chars().all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '-') || configuration.is_empty() {
return Err(IdeError::invalid(format!("not a configuration name: {configuration}")));
}
if !project.starts_with(':') || project.chars().any(char::is_whitespace) {
return Err(IdeError::invalid(format!("not a Gradle project path: {project}")));
}
let model = gradle::load(root).ok_or_else(|| IdeError::not_found("no Gradle build in this project"))?;
let task = if project == ":" { ":dependencies".to_owned() } else { format!("{project}:dependencies") };
let mut args: Vec<String> =
vec!["--console=plain".into(), "-q".into(), task, "--configuration".into(), configuration.into()];
if options.offline {
args.push("--offline".into());
}
let out = process::run(
options.gradle_command.as_deref().unwrap_or(&model.command),
&args,
&root.join(&model.dir),
&options.env,
cancel,
|_, _| {},
)?;
if !out.success {
return Err(IdeError::unavailable(error_summary(&out.text)));
}
Ok(parse_report(&out.text))
}
#[cfg(test)]
mod tests {
use super::*;
fn dep(group: &str, name: &str, requested: Option<&str>) -> GradleDependency {
GradleDependency {
group: group.into(),
name: name.into(),
requested: requested.map(Into::into),
..Default::default()
}
}
#[test]
fn parses_the_tree_with_selected_versions_and_markers() {
let report = "\
------------------------------------------------------------
Project ':app'
------------------------------------------------------------
runtimeClasspath - Runtime classpath of source set 'main'.
+--- org.springframework.boot:spring-boot-starter-web -> 3.2.0
| +--- org.springframework.boot:spring-boot-starter:3.2.0
| | \\--- org.yaml:snakeyaml:2.2
| \\--- com.fasterxml.jackson.core:jackson-databind:2.15.0 -> 2.15.3 (*)
+--- project ':core'
| \\--- com.google.guava:guava:{strictly 32.1.3-jre} -> 32.1.3-jre (c)
+--- org.example:legacy:1.0 -> project :legacy
\\--- org.missing:gone:9 FAILED
(c) - A dependency constraint, not a dependency. The dependency affected by the constraint occurs elsewhere in the tree.
(*) - Indicates repeated occurrences of a transitive dependency subtree. Gradle expands transitive dependency subtrees only once per project; repeat occurrences only display the root of the subtree, followed by this annotation.
";
let tree = parse_report(report);
assert_eq!(tree.len(), 4);
let web = &tree[0];
assert_eq!((web.group.as_str(), web.name.as_str()), ("org.springframework.boot", "spring-boot-starter-web"));
assert_eq!((web.requested.as_deref(), web.selected.as_deref()), (None, Some("3.2.0")));
assert_eq!(web.children.len(), 2);
assert_eq!(web.children[0].children, vec![dep("org.yaml", "snakeyaml", Some("2.2"))]);
let databind = &web.children[1];
assert_eq!((databind.requested.as_deref(), databind.selected.as_deref()), (Some("2.15.0"), Some("2.15.3")));
assert!(databind.repeated);
let core = &tree[1];
assert!(core.project && core.group.is_empty());
assert_eq!(core.name, ":core");
let guava = &core.children[0];
assert!(guava.constraint);
assert_eq!(guava.requested.as_deref(), Some("{strictly 32.1.3-jre}"));
let legacy = &tree[2];
assert!(legacy.project);
assert_eq!((legacy.name.as_str(), legacy.selected.as_deref()), ("legacy", Some(":legacy")));
assert!(tree[3].unresolved);
assert_eq!(tree[3].requested.as_deref(), Some("9"));
}
#[test]
fn an_empty_configuration_has_no_dependencies() {
assert!(
parse_report("compileClasspath - Compile classpath for source set 'main'.\nNo dependencies\n").is_empty()
);
}
#[test]
fn rejects_names_that_are_not_configurations_or_projects() {
let tmp = tempfile::tempdir().unwrap();
let options = SyncOptions::default();
let cancel = CancellationToken::new();
assert!(gradle_dependencies(tmp.path(), &options, ":", "x --init-script y", &cancel).is_err());
assert!(gradle_dependencies(tmp.path(), &options, "app", "runtimeClasspath", &cancel).is_err());
}
#[cfg(unix)]
#[test]
fn runs_the_report_of_the_project_and_configuration() {
use std::os::unix::fs::PermissionsExt;
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
std::fs::write(root.join("settings.gradle"), "include 'app'").unwrap();
std::fs::write(root.join("build.gradle"), "").unwrap();
let launcher = root.join("fake-gradle");
std::fs::write(&launcher, "#!/bin/sh\necho \"$@\" > args\necho '\\--- org.a:b:1.0'\n").unwrap();
std::fs::set_permissions(&launcher, std::fs::Permissions::from_mode(0o755)).unwrap();
let options = SyncOptions {
gradle_command: Some(launcher.to_string_lossy().into_owned()),
offline: true,
..Default::default()
};
let tree =
gradle_dependencies(root, &options, ":app", "testRuntimeClasspath", &CancellationToken::new()).unwrap();
assert_eq!(tree, vec![dep("org.a", "b", Some("1.0"))]);
assert_eq!(
std::fs::read_to_string(root.join("args")).unwrap().trim(),
"--console=plain -q :app:dependencies --configuration testRuntimeClasspath --offline"
);
}
}
+1 -1
View File
@@ -81,7 +81,7 @@ pub fn build(
cancel: &CancellationToken,
) -> IdeResult<Outcome> {
std::fs::create_dir_all(generation)?;
let mut plan = compile::plan(root, model, BuildKind::Build, None)?;
let mut plan = compile::plan(root, model, BuildKind::Build, None, None)?;
if plan.system != compile::BuildSystem::Maven {
return Err(IdeError::invalid("Kotlin bridge requires a Maven build root"));
}
+4 -1
View File
@@ -8,6 +8,7 @@
pub mod application_update;
pub mod compile;
pub mod gradle;
mod gradle_deps;
pub mod kotlin_bridge;
pub mod maven;
pub mod pom_edit;
@@ -17,6 +18,7 @@ pub mod web;
mod xml;
pub use gradle::{GradleModel, GradleProject, GradleTask};
pub use gradle_deps::{GradleDependency, gradle_dependencies};
pub use maven::{MavenDependency, MavenModel, MavenPlugin, MavenProfile, MavenProject, Omission};
pub use sync::{SyncOptions, SyncProgress, maven_profiles, save_maven_profiles, sync_gradle, sync_maven};
@@ -53,7 +55,8 @@ pub fn load_linked(
settings: &ide_core::settings::MavenSettings,
) -> BuildModel {
let maven = maven::is_maven_project(root).then(|| {
let mut m = maven::load_with_repo(root, maven_repo.or_else(maven::default_local_repo));
let profiles = sync::maven_profiles(cache_dir);
let mut m = maven::load_with(root, maven_repo.or_else(maven::default_local_repo), &profiles);
maven::UnlinkedProjects::from_model(&m, settings).filter(&mut m);
sync::apply_maven_cache(root, cache_dir, &mut m);
m
+86 -6
View File
@@ -460,6 +460,8 @@ const MAX_PARENT_DEPTH: usize = 10;
struct Resolver {
local_repo: Option<PathBuf>,
/// The chosen profiles (`-P` entries; `!id` / `-id` deactivates).
profiles: Vec<String>,
poms: HashMap<PathBuf, Option<RawPom>>,
contexts: HashMap<PathBuf, Inherited>,
}
@@ -484,8 +486,16 @@ pub fn local_repository_of(settings: &Path) -> Option<PathBuf> {
}
impl Resolver {
fn new(local_repo: Option<PathBuf>) -> Self {
Self { local_repo, poms: HashMap::new(), contexts: HashMap::new() }
fn new(local_repo: Option<PathBuf>, profiles: &[String]) -> Self {
Self { local_repo, profiles: profiles.to_vec(), poms: HashMap::new(), contexts: HashMap::new() }
}
/// The active profiles of a POM: the chosen ones, and the `activeByDefault` ones not turned
/// off while none of the POM's other profiles is chosen (Maven's rule).
fn active<'a>(&self, profiles: &'a [RawProfile]) -> impl Iterator<Item = &'a RawProfile> {
let chosen = |id: &str| profile_choice(&self.profiles, id);
let any_chosen = profiles.iter().any(|p| chosen(&p.id) == Some(true));
profiles.iter().filter(move |p| chosen(&p.id).unwrap_or(p.active_by_default && !any_chosen))
}
fn pom(&mut self, path: &Path) -> Option<RawPom> {
@@ -550,7 +560,7 @@ impl Resolver {
if pom.compiler_release.is_some() {
ctx.compiler_release = pom.compiler_release.clone();
}
for p in pom.profiles.iter().filter(|p| p.active_by_default) {
for p in self.active(&pom.profiles) {
merge_by(&mut ctx.properties, &p.properties, |(k, _)| k.clone());
merge_by(&mut ctx.deps, &p.deps, RawDep::key);
merge_plugins(&mut ctx.plugins, &p.plugins);
@@ -666,7 +676,7 @@ pub(crate) fn launch_inputs(
}
Some(())
}
let mut resolver = Resolver::new(local_repo.or_else(default_local_repo));
let mut resolver = Resolver::new(local_repo.or_else(default_local_repo), &[]);
let mut out = Vec::new();
for project in projects {
let dir = root.join(&project.dir);
@@ -1055,8 +1065,32 @@ pub fn load(root: &Path) -> MavenModel {
load_with_repo(root, default_local_repo())
}
/// [`load`] with the chosen `profiles` (`-P` entries; `!id` deactivates): their modules,
/// dependencies and properties are part of the model, those turned off are not.
pub fn load_with_profiles(root: &Path, profiles: &[String]) -> MavenModel {
load_with(root, default_local_repo(), profiles)
}
pub(crate) fn load_with_repo(root: &Path, local_repo: Option<PathBuf>) -> MavenModel {
let mut resolver = Resolver::new(local_repo);
load_with(root, local_repo, &[])
}
/// The `-P` entries' choice for profile `id`: on, off (`!id` / `-id`), or none (the last wins).
fn profile_choice(chosen: &[String], id: &str) -> Option<bool> {
chosen.iter().rev().find_map(|c| {
let c = c.trim();
if c == id {
Some(true)
} else if c.strip_prefix('!').or_else(|| c.strip_prefix('-')) == Some(id) {
Some(false)
} else {
None
}
})
}
pub(crate) fn load_with(root: &Path, local_repo: Option<PathBuf>, profiles: &[String]) -> MavenModel {
let mut resolver = Resolver::new(local_repo, profiles);
let mut model = MavenModel::default();
let mut seen: HashSet<PathBuf> = HashSet::new();
// (pom, aggregated)
@@ -1076,7 +1110,7 @@ pub(crate) fn load_with_repo(root: &Path, local_repo: Option<PathBuf>) -> MavenM
let i = |s: &str| interpolate(s, &props);
let mut module_names: Vec<String> = pom.modules.clone();
for p in pom.profiles.iter().filter(|p| p.active_by_default) {
for p in resolver.active(&pom.profiles) {
module_names.extend(p.modules.iter().cloned());
}
let mut modules: Vec<PathBuf> = Vec::new();
@@ -1699,6 +1733,52 @@ mod tests {
assert!(model.projects.iter().all(|p| p.command == if cfg!(windows) { "mvn.cmd" } else { "mvn" }));
}
#[test]
fn chosen_profiles_bring_their_modules_and_dependencies() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
write(
root,
"pom.xml",
r#"<project><groupId>demo</groupId><artifactId>parent</artifactId><version>1.0</version><packaging>pom</packaging>
<modules><module>core</module></modules>
<profiles>
<profile><id>flink-1.17</id><activation><activeByDefault>true</activeByDefault></activation>
<modules><module>flink-1.17</module></modules></profile>
<profile><id>flink-1.20</id><modules><module>flink-1.20</module></modules>
<dependencies><dependency><groupId>org.apache.flink</groupId><artifactId>flink-core</artifactId><version>1.20.0</version></dependency></dependencies>
</profile>
</profiles>
</project>"#,
);
for m in ["core", "flink-1.17", "flink-1.20"] {
write(
root,
&format!("{m}/pom.xml"),
&format!(
"<project><parent><groupId>demo</groupId><artifactId>parent</artifactId><version>1.0</version></parent><artifactId>{m}</artifactId></project>"
),
);
}
let dirs = |m: &MavenModel| m.projects.iter().map(|p| p.dir.clone()).collect::<Vec<_>>();
let flink = |m: &MavenModel, dir: &str| {
m.projects.iter().find(|p| p.dir == dir).unwrap().dependencies.iter().any(|d| d.artifact_id == "flink-core")
};
let default = load_with(root, None, &[]);
assert_eq!(dirs(&default), ["", "core", "flink-1.17"]);
assert!(!flink(&default, "core"));
// Choosing another profile of the POM turns its activeByDefault one off (as `-P` does).
let chosen = load_with(root, None, &["flink-1.20".into()]);
assert_eq!(dirs(&chosen), ["", "core", "flink-1.20"]);
assert!(flink(&chosen, "core"), "a module inherits the chosen profile's dependencies");
let both = load_with(root, None, &["flink-1.20".into(), "flink-1.17".into()]);
assert_eq!(dirs(&both), ["", "core", "flink-1.17", "flink-1.20"]);
assert_eq!(dirs(&load_with(root, None, &["!flink-1.17".into()])), ["", "core"]);
}
#[test]
fn profiles_of_a_parent_pom_are_inherited() {
let tmp = tempfile::tempdir().unwrap();
+1 -1
View File
@@ -283,7 +283,7 @@ pub fn sync_maven(
cancel: &CancellationToken,
progress: &dyn Fn(SyncProgress),
) -> IdeResult<()> {
let mut model = maven::load(root);
let mut model = maven::load_with_profiles(root, &options.profiles);
maven::UnlinkedProjects::from_model(&model, &options.maven_settings).filter(&mut model);
std::fs::create_dir_all(cache_dir)?;
let mut cache = MavenCache {
+21 -6
View File
@@ -123,12 +123,14 @@ fn taskkill_tree(child: &Child) {
/// inherits them loads the image's (older) libraries and plugins next to its own, e.g.
/// `curl: symbol lookup error ... nghttp2_...`. The IDE process keeps its environment
/// (WebKit and GTK still need it). Then applies the [`set_env_overlay`] variables and
/// those of a manual HTTP proxy ([`crate::proxy_settings::ProxySettings::child_env`]).
/// Variables the call site has already set are kept. A no-op outside an AppImage
/// without an overlay.
/// those of a manual HTTP proxy ([`crate::proxy_settings::ProxySettings::child_env`]),
/// and the login shell's variables this process lacks ([`shell_env`]: an app started from
/// Finder/Dock doesn't have what rc files export, such as `GOPATH`, `JAVA_HOME` or
/// `MAVEN_OPTS`). Variables the call site has already set are kept.
pub fn host_env(cmd: &mut Command) -> &mut Command {
let fixes = child_env_changes();
if fixes.is_empty() {
let shell: Vec<_> = shell_env().collect();
if fixes.is_empty() && shell.is_empty() {
return cmd;
}
let explicit: Vec<OsString> = cmd.get_envs().map(|(k, _)| k.to_owned()).collect();
@@ -138,6 +140,9 @@ pub fn host_env(cmd: &mut Command) -> &mut Command {
None => cmd.env_remove(key),
};
}
for (key, value) in shell.into_iter().filter(|(k, _)| !explicit.contains(k)) {
cmd.env(key, value);
}
cmd
}
@@ -154,12 +159,13 @@ pub fn host_vars() -> impl Iterator<Item = (OsString, OsString)> {
apply_changes(std::env::vars_os().collect(), &child_env_changes()).into_iter()
}
/// [`std::env::var_os`] as a child should see it (see [`host_env`]).
/// [`std::env::var_os`] as a child should see it (see [`host_env`]), the login shell's value
/// included when this process lacks the variable.
pub fn host_var_os(key: impl AsRef<OsStr>) -> Option<OsString> {
let key = key.as_ref();
match child_env_changes().into_iter().find(|(k, _)| k == key) {
Some((_, fixed)) => fixed,
None => std::env::var_os(key),
None => std::env::var_os(key).or_else(|| shell_env().find(|(k, _)| k == key).map(|(_, v)| v)),
}
}
@@ -572,6 +578,15 @@ mod tests {
assert_eq!(get("HOME"), Some(None));
let shell: Vec<_> = shell_env().map(|(k, _)| k).collect();
assert_eq!(shell, [OsString::from("OPENIDEA_OVERLAY_B")]);
// Every child gets what the login shell exports and the IDE process lacks (a Finder
// launch: `GOPATH`, `JAVA_HOME`...), unless the call site sets it.
assert_eq!(get("OPENIDEA_OVERLAY_B"), Some(Some("b".into())));
let mut own = Command::new("true");
own.env("OPENIDEA_OVERLAY_B", "call site");
host_env(&mut own);
let value = own.get_envs().find(|(n, _)| *n == "OPENIDEA_OVERLAY_B").and_then(|(_, v)| v);
assert_eq!(value, Some(OsStr::new("call site")));
assert_eq!(host_var_os("OPENIDEA_OVERLAY_B"), Some("b".into()));
assert_eq!(host_var_os("PATH"), Some("/reloaded".into()));
assert!(host_vars().all(|(k, _)| k != "HOME"));
assert!(host_vars().any(|(k, v)| k == "OPENIDEA_OVERLAY_A" && v == "project"));
+71 -1
View File
@@ -61,6 +61,9 @@ pub struct Settings {
pub cmake: CMakeSettings,
/// Tools › Agents › Chat.
pub agents: AgentSettings,
/// Plugin settings (`[contributes.settings]`): plugin id → its values by key.
#[specta(type = BTreeMap<String, BTreeMap<String, specta_typescript::Unknown>>)]
pub plugins: BTreeMap<String, serde_json::Map<String, serde_json::Value>>,
}
/// What the meters in an agent task's header show.
@@ -154,6 +157,16 @@ pub struct LanguageServerSettings {
pub languages: Vec<String>,
/// Files marking a workspace root for a server defined here (`pyproject.toml`).
pub root_markers: Vec<String>,
/// Replaces the definition's `features`: what the server is used for
/// (`code-action`, `hover`, …); an empty list means everything.
#[serde(skip_serializing_if = "Option::is_none")]
#[specta(optional)]
pub features: Option<Vec<String>>,
/// Replaces the definition's `code-action-kinds` (`refactor`); an empty
/// list keeps every kind.
#[serde(skip_serializing_if = "Option::is_none")]
#[specta(optional)]
pub code_action_kinds: Option<Vec<String>>,
}
impl Default for LanguageServerSettings {
@@ -167,6 +180,8 @@ impl Default for LanguageServerSettings {
settings: None,
languages: Vec::new(),
root_markers: Vec::new(),
features: None,
code_action_kinds: None,
}
}
}
@@ -215,6 +230,7 @@ impl Default for Settings {
run_targets: Vec::new(),
cmake: CMakeSettings::default(),
agents: AgentSettings::default(),
plugins: BTreeMap::new(),
}
}
}
@@ -323,7 +339,8 @@ impl Default for NotificationGroupSettings {
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct AppearanceSettings {
/// `dark` | `light` | `system` | `dark-hc` (high-contrast dark)
/// A built-in theme id (`light`, `dark`, `dark-hc`, …), `system` (legacy
/// "Sync with OS"), or a plugin's theme: `plugin:<plugin-id>/<theme-id>`.
pub theme: String,
pub accent: String,
pub ui_font_size: f32,
@@ -332,6 +349,9 @@ pub struct AppearanceSettings {
pub sync_with_os: bool,
pub preferred_light_theme: String,
pub preferred_dark_theme: String,
/// A plugin's file icon theme (`<plugin-id>/<theme-id>`); empty = the built-in icons.
pub file_icon_theme: String,
pub file_icon_theme_options: BTreeMap<String, FileIconThemeOptions>,
/// IDE zoom in percent (50–300).
pub zoom: u32,
/// Apply `ui_font_family` / `ui_font_size` (otherwise the theme's).
@@ -367,6 +387,24 @@ pub struct AppearanceSettings {
/// Main toolbar entries per side, in order: title widget ids (`project`,
/// `vcs.branch`…) or `action:<id>`. Widgets in neither list nor `hidden`
/// (registered later) keep their default place.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct FileIconThemeOptions {
pub groups: BTreeMap<String, bool>,
pub choices: BTreeMap<String, String>,
pub hide_files: bool,
pub hide_folders: bool,
pub hollow_folders: bool,
pub hide_ui_icons: bool,
pub hide_psi_icons: bool,
pub monochrome: Option<String>,
pub saturation: Option<u8>,
pub opacity: Option<u8>,
pub icon_size: Option<u8>,
pub row_height: Option<u8>,
pub accent: Option<String>,
}
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct MainToolbarSettings {
@@ -386,6 +424,8 @@ impl Default for AppearanceSettings {
sync_with_os: false,
preferred_light_theme: "light".into(),
preferred_dark_theme: "dark".into(),
file_icon_theme: String::new(),
file_icon_theme_options: BTreeMap::new(),
zoom: 100,
use_custom_font: false,
ui_font_family: "Inter".into(),
@@ -911,6 +951,36 @@ mod tests {
assert_eq!(serde_json::from_value::<Settings>(value).unwrap(), settings);
}
#[test]
fn file_icon_theme_options_default_empty_and_round_trip() {
let old: Settings = serde_json::from_str(r#"{"appearance":{"fileIconTheme":"p/t"}}"#).unwrap();
assert!(old.appearance.file_icon_theme_options.is_empty());
let json = serde_json::json!({"appearance":{"fileIconThemeOptions":{"p/t":{
"groups":{"tests":true},"choices":{"arrows":"plusminus"},"hideFiles":true,"hideFolders":false,"hollowFolders":true,"hideUiIcons":true,"hidePsiIcons":false,
"monochrome":"#546e7a","saturation":5,"opacity":80,"iconSize":20,"rowHeight":30,"accent":"#009688"}}}});
let settings: Settings = serde_json::from_value(json.clone()).unwrap();
let options = &settings.appearance.file_icon_theme_options["p/t"];
assert!(options.hide_files && options.hollow_folders && !options.hide_folders);
assert!(options.hide_ui_icons && !options.hide_psi_icons);
assert!(options.groups["tests"]);
assert_eq!(options.choices["arrows"], "plusminus");
assert_eq!(
(options.saturation, options.opacity, options.icon_size, options.row_height),
(Some(5), Some(80), Some(20), Some(30))
);
assert_eq!(options.monochrome.as_deref(), Some("#546e7a"));
assert_eq!(options.accent.as_deref(), Some("#009688"));
let value = serde_json::to_value(&settings).unwrap();
let restored: Settings = serde_json::from_value(value).unwrap();
assert_eq!(restored, settings);
let plain = serde_json::to_value(FileIconThemeOptions::default()).unwrap();
assert_eq!(plain["hideFiles"], false);
assert_eq!(plain["hideUiIcons"], false);
assert_eq!(plain["hidePsiIcons"], false);
assert_eq!(plain["monochrome"], serde_json::Value::Null);
assert_eq!(plain["groups"], serde_json::json!({}));
}
#[test]
fn ui_font_fallback_defaults_empty_and_round_trips() {
let old: Settings = serde_json::from_str(r#"{"appearance":{"useCustomFont":true}}"#).unwrap();
+19 -7
View File
@@ -8,7 +8,7 @@
use ide_core::process::executable_names;
use ide_core::{IdeError, IdeResult};
use ide_language::{DebuggerConfig, LanguageRegistry};
use ide_language::{DebuggerConfig, LanguageRegistry, ProgramPin};
use std::future::Future;
use std::path::{Path, PathBuf};
use std::pin::Pin;
@@ -44,6 +44,8 @@ pub struct AdapterSpec {
/// `host:port` of an adapter inside another program (the Godot editor): connected to
/// when it accepts, else `command` is started.
pub connect: Option<String>,
/// A plugin language's adapter: start exactly this approved program.
pub pinned: Option<ProgramPin>,
}
/// Built-in adapters used when no language definition names `adapter`.
@@ -55,6 +57,7 @@ fn builtin(adapter: &str) -> Option<DebuggerConfig> {
transport: transport.to_owned(),
connect: None,
install_hint: Some(hint.to_owned()),
pinned: None,
};
Some(match adapter {
"codelldb" => cfg(
@@ -133,6 +136,7 @@ pub fn resolve(config: &DebuggerConfig, cwd: PathBuf) -> AdapterSpec {
env: Vec::new(),
server_host: None,
connect: config.connect.clone(),
pinned: config.pinned.clone(),
}
}
@@ -426,12 +430,20 @@ impl AdapterLauncher for ProcessLauncher {
if is_plugin_bundle(&spec.command) {
return Err(not_standalone(&spec));
}
let (command, cwd) = (spec.command.clone(), spec.cwd.clone());
let binary = tokio::task::spawn_blocking(move || resolve_binary(&command, &cwd))
.await
.ok()
.flatten()
.ok_or_else(|| missing(&spec))?;
let binary = match spec.pinned.clone() {
Some(pin) => tokio::task::spawn_blocking(move || pin.verify().map(Path::to_path_buf))
.await
.map_err(|e| IdeError::internal(e.to_string()))?
.map_err(|e| IdeError::new(ide_core::error::ErrorKind::PermissionDenied, e))?,
None => {
let (command, cwd) = (spec.command.clone(), spec.cwd.clone());
tokio::task::spawn_blocking(move || resolve_binary(&command, &cwd))
.await
.ok()
.flatten()
.ok_or_else(|| missing(&spec))?
}
};
let port = match spec.transport {
// An embedded adapter started without `${port}` listens on its usual address.
Transport::Tcp => match &embedded {
+54
View File
@@ -212,6 +212,19 @@ pub struct Variable {
pub evaluate_name: Option<String>,
}
/// A completion the adapter offers for an expression (DAP `completions`).
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct DebugCompletion {
pub label: String,
/// What to insert, when it differs from `label`.
pub text: Option<String>,
/// DAP item type: `method`, `field`, `variable`, `class`, `keyword`, …
pub kind: Option<String>,
pub detail: Option<String>,
pub sort_text: Option<String>,
}
#[derive(Clone, Copy, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum StepKind {
@@ -1157,6 +1170,47 @@ impl DebugManager {
})
}
/// Completions for `text` at `line` / `column` (0-based, UTF-16) in the frame's scope; none
/// when the adapter doesn't offer them.
pub async fn completions(
&self,
s: DebugSessionId,
frame_id: Option<i64>,
text: String,
line: u32,
column: u32,
) -> IdeResult<Vec<DebugCompletion>> {
let session = self.session(s)?;
if !session.supports("supportsCompletionsRequest") {
return Ok(Vec::new());
}
let mut args = json!({ "text": text, "line": line + 1, "column": column + 1 });
if let Some(f) = frame_id {
args["frameId"] = f.into();
}
let body = session.conn.request_timeout("completions", args, REQUEST_TIMEOUT).await?;
let text =
|v: &Value, key: &str| v.get(key).and_then(Value::as_str).filter(|t| !t.is_empty()).map(str::to_owned);
Ok(body
.get("targets")
.and_then(Value::as_array)
.map(|targets| {
targets
.iter()
.filter_map(|t| {
Some(DebugCompletion {
label: text(t, "label")?,
text: text(t, "text"),
kind: text(t, "type"),
detail: text(t, "detail"),
sort_text: text(t, "sortText"),
})
})
.collect()
})
.unwrap_or_default())
}
pub async fn resume(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
let body = session.conn.request_timeout("continue", json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await?;
+73
View File
@@ -73,6 +73,7 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
msg,
json!({
"supportsConfigurationDoneRequest": true,
"supportsCompletionsRequest": true,
"supportsTerminateRequest": true,
"supportsSetVariable": true,
"supportsFunctionBreakpoints": true,
@@ -172,6 +173,14 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
]})
),
"evaluate" => ok!(msg, json!({"result": "42", "type": "int", "variablesReference": 0})),
"completions" => ok!(
msg,
json!({"targets": [
{"label": "getName()", "text": "getName", "type": "method", "detail": "String"},
{"label": "name", "type": "field"},
{"label": ""},
]})
),
"continue" => {
ok!(msg, json!({"allThreadsContinued": true}));
send!(json!({"type": "event", "event": "exited", "body": {"exitCode": 3}}));
@@ -368,6 +377,14 @@ async fn full_protocol_flow() {
let eval = h.request("evaluate");
assert_eq!(eval["arguments"]["context"], "watch");
assert_eq!(eval["arguments"]["frameId"], 10);
let items = h.manager.completions(id, Some(10), "req.na".into(), 0, 6).await.unwrap();
assert_eq!(
items.iter().map(|c| (c.label.as_str(), c.text.as_deref(), c.kind.as_deref())).collect::<Vec<_>>(),
[("getName()", Some("getName"), Some("method")), ("name", None, Some("field"))]
);
let asked = h.request("completions");
assert_eq!((asked["arguments"]["line"].clone(), asked["arguments"]["column"].clone()), (json!(1), json!(7)));
assert_eq!(asked["arguments"]["frameId"], 10);
// Live breakpoint updates reach the session.
let states = h.manager.set_breakpoints(path.clone(), vec![bp(0)]).await.unwrap();
@@ -915,6 +932,7 @@ fn godot_registry(port: u16, command: &str, args: &[&str]) -> Arc<LanguageRegist
transport: "tcp".into(),
connect: Some(format!("127.0.0.1:{port}")),
install_hint: Some("Open the project in the Godot editor".into()),
pinned: None,
}),
};
Arc::new(LanguageRegistry::from_configs(vec![lang]))
@@ -1147,3 +1165,58 @@ async fn java_stack_frames_without_breakpoints_normalize_windows_paths() {
h.manager.stop(id).await.unwrap();
}
}
/// A built-in language definition (GDScript) shipped by a plugin
/// instead: its debugger starts the program approved for the plugin's
/// `process:godot` grant, and refuses once that file changed.
#[cfg(unix)]
#[tokio::test]
async fn a_plugin_language_runs_its_debug_adapter_from_the_approved_program() {
use ide_language::ProgramPin;
use ide_language::plugin_point::{PluginLanguages, parse_definition};
use std::os::unix::fs::PermissionsExt;
let dir = std::env::temp_dir().join(format!("openidea-plugin-dap-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
// Stands in for `godot --headless --editor …`.
let program = dir.join("godot");
std::fs::write(&program, "#!/bin/sh\nexec sleep 30\n").unwrap();
std::fs::set_permissions(&program, std::fs::Permissions::from_mode(0o755)).unwrap();
let port = std::net::TcpListener::bind(("127.0.0.1", 0)).unwrap().local_addr().unwrap().port();
let mut gdscript = parse_definition(include_str!("../../../languages/gdscript.toml")).unwrap();
let debugger = gdscript.debugger.as_mut().unwrap();
debugger.args.clear();
debugger.connect = Some(format!("127.0.0.1:{port}"));
// No built-in GDScript: it comes from the plugin.
let registry = Arc::new(LanguageRegistry::from_configs(Vec::new()));
let store = PluginLanguages::new();
registry.attach_plugins(store.clone());
let pin = ProgramPin::approve(&program).unwrap();
store.set_process_policy(Arc::new(move |plugin, cap| {
(plugin == "acme.godot" && cap.to_string() == "process:godot").then(|| pin.clone())
}));
let _language = store.add("acme.godot".into(), gdscript);
let log: Log = Arc::default();
let server_log = log.clone();
tokio::spawn(async move {
tokio::time::sleep(std::time::Duration::from_millis(300)).await;
let listener = tokio::net::TcpListener::bind(("127.0.0.1", port)).await.unwrap();
let (stream, _) = listener.accept().await.unwrap();
fake_adapter(stream, server_log, Script::default()).await;
});
let (owner, _events) = recording_owner("main");
let manager = DebugManager::new(registry.clone());
let id = manager.start(godot_launch(&dir), owner).await.unwrap();
assert!(log.lock().iter().any(|m| m["command"] == "launch"));
manager.stop(id).await.unwrap();
// Replaced after approval: refused before anything starts.
std::fs::write(&program, "#!/bin/sh\nexec sleep 31\n").unwrap();
let (owner, _events) = recording_owner("main");
let err = manager.start(godot_launch(&dir), owner).await.unwrap_err();
assert_eq!(err.kind, ide_core::error::ErrorKind::PermissionDenied, "{}", err.message);
let _ = std::fs::remove_dir_all(&dir);
}
+5 -1
View File
@@ -8,7 +8,7 @@ license.workspace = true
[dependencies]
ide-core.workspace = true
serde.workspace = true
serde_json.workspace = true
serde_json = { workspace = true, features = ["raw_value"] }
specta.workspace = true
toml.workspace = true
tokio.workspace = true
@@ -21,6 +21,10 @@ bb8.workspace = true
tokio-util = { workspace = true, features = ["compat"] }
dirs.workspace = true
ide-ssh.workspace = true
ide-http.workspace = true
reqwest = { workspace = true, features = ["query"] }
rustls.workspace = true
uuid.workspace = true
[dev-dependencies]
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
+844
View File
@@ -0,0 +1,844 @@
use crate::conn::{QueryResult, ResultColumn, Setup};
use crate::source::{DataSource, DbSslMode, DbTxMode};
use ide_core::{CancellationToken, IdeError, IdeResult, error::ErrorKind};
use parking_lot::RwLock;
use reqwest::{Client, RequestBuilder, StatusCode, Url};
use std::sync::Arc;
use std::time::{Duration, Instant};
use tokio::sync::Mutex;
#[derive(Clone)]
pub struct ClickhousePool(Arc<State>);
struct State {
client: RwLock<Option<Client>>,
endpoint: Url,
user: String,
password: String,
database: String,
settings: Vec<(String, String)>,
read_only: bool,
session_id: String,
session: Mutex<bool>,
closed: CancellationToken,
}
/// The response format: a names line, a types line, then one JSON array per row, so rows past
/// the grid's limit are skipped without being parsed.
const FORMAT: &str = "JSONCompactEachRowWithNamesAndTypes";
pub(crate) fn transaction_error() -> IdeError {
IdeError::invalid("ClickHouse HTTP does not support SQL transactions; use auto-commit and run writes directly")
}
impl ClickhousePool {
pub(crate) async fn connect(source: &DataSource, password: Option<&str>, setup: Setup) -> IdeResult<Self> {
if source.options.transaction == DbTxMode::Manual {
return Err(transaction_error());
}
for (name, _) in &setup.params {
if reserved_setting(name) {
return Err(IdeError::invalid(format!("ClickHouse HTTP manages the {name} setting")));
}
}
let secure = matches!(source.ssl, DbSslMode::Require | DbSslMode::VerifyCa | DbSslMode::VerifyFull);
let tls_files = [&source.ssl_ca_file, &source.ssl_cert_file, &source.ssl_key_file];
if !secure && tls_files.iter().any(|p| !p.trim().is_empty()) {
return Err(IdeError::invalid(
"ClickHouse TLS certificate files require an SSL mode of Require, Verify CA, or Verify Full",
));
}
let host = source.host.trim();
if host.is_empty() || host.contains(['/', '@', '?', '#']) {
return Err(IdeError::invalid("ClickHouse host must contain only a hostname or IP address"));
}
let host = if host.contains(':') && !host.starts_with('[') { format!("[{host}]") } else { host.to_owned() };
let scheme = if secure { "https" } else { "http" };
let port = source.port.unwrap_or(if secure { 8443 } else { 8123 });
let endpoint = Url::parse(&format!("{scheme}://{host}:{port}/"))
.map_err(|_| IdeError::invalid("invalid ClickHouse HTTP host or port"))?;
let timeout = setup.connect_timeout.unwrap_or(Duration::from_secs(30));
let _ = rustls::crypto::ring::default_provider().install_default();
let mut builder = Client::builder()
.connect_timeout(timeout)
.pool_max_idle_per_host(4)
.redirect(reqwest::redirect::Policy::none())
.user_agent(concat!("OpenIDEA/", env!("CARGO_PKG_VERSION")));
let ca = source.ssl_ca_file.trim();
if !ca.is_empty() {
let pem = read_pem(ca, "CA certificate")?;
let certificates = reqwest::Certificate::from_pem_bundle(&pem)
.map_err(|_| IdeError::invalid("invalid ClickHouse CA certificate PEM"))?;
if certificates.is_empty() {
return Err(IdeError::invalid("ClickHouse CA certificate PEM contains no certificates"));
}
builder = builder.tls_certs_only(certificates);
}
if source.ssl == DbSslMode::VerifyCa {
if ca.is_empty() {
return Err(IdeError::invalid("ClickHouse Verify CA requires a CA certificate file"));
}
builder = builder.tls_danger_accept_invalid_hostnames(true);
}
let cert = source.ssl_cert_file.trim();
let key = source.ssl_key_file.trim();
if cert.is_empty() != key.is_empty() {
return Err(IdeError::invalid("ClickHouse client TLS requires both certificate and key files"));
}
if !cert.is_empty() {
let mut pem = read_pem(cert, "client certificate")?;
pem.push(b'\n');
pem.extend(read_pem(key, "client key")?);
let identity = reqwest::Identity::from_pem(&pem)
.map_err(|_| IdeError::invalid("invalid ClickHouse client certificate or key PEM"))?;
builder = builder.identity(identity);
}
let proxy = ide_core::proxy_settings::proxy();
let client = ide_http::client::with_proxy(builder, &proxy).build().map_err(http_error)?;
let database = if source.options.default_schema.trim().is_empty() {
source.database.trim()
} else {
source.options.default_schema.trim()
};
let pool = Self(Arc::new(State {
client: RwLock::new(Some(client)),
endpoint,
user: if source.user.is_empty() { "default".into() } else { source.user.clone() },
password: password.unwrap_or_default().to_owned(),
database: if database.is_empty() { "default".into() } else { database.into() },
settings: setup.params,
read_only: source.read_only,
session_id: uuid::Uuid::new_v4().to_string(),
session: Mutex::new(false),
closed: CancellationToken::new(),
}));
let cancel = CancellationToken::new();
let initialize = async {
pool.execute("SELECT 1", 1, &cancel).await?;
for sql in setup.statements {
pool.execute(&sql, 0, &cancel).await?;
}
Ok(pool.clone())
};
tokio::pin!(initialize);
tokio::select! {
result = &mut initialize => {
if result.is_err() {
pool.close();
}
result
},
() = tokio::time::sleep(timeout) => {
cancel.cancel();
let _ = initialize.await;
pool.close();
Err(IdeError::new(ErrorKind::TimedOut, "timed out connecting to ClickHouse HTTP"))
}
}
}
pub fn close(&self) {
self.0.closed.cancel();
self.0.client.write().take();
}
pub fn is_closed(&self) -> bool {
self.0.closed.is_cancelled()
}
fn client(&self) -> IdeResult<Client> {
self.0.client.read().clone().ok_or_else(|| IdeError::new(ErrorKind::Io, "ClickHouse connection is closed"))
}
fn request(&self, client: &Client, query_id: &str, max_rows: usize, initialized: bool) -> RequestBuilder {
let mut request = client
.post(self.0.endpoint.clone())
.basic_auth(&self.0.user, Some(&self.0.password))
.header(reqwest::header::CONTENT_TYPE, "text/plain; charset=utf-8")
.query(&self.0.settings)
.query(&[
("session_id", self.0.session_id.clone()),
("session_timeout", "3600".into()),
("query_id", query_id.to_owned()),
("default_format", FORMAT.into()),
("output_format_json_quote_denormals", "1".into()),
("max_result_rows", max_rows.saturating_add(1).to_string()),
("result_overflow_mode", "break".into()),
]);
if initialized {
request = request.query(&[("session_check", "1")]);
} else {
request = request.query(&[("database", &self.0.database)]);
}
if self.0.read_only {
request = request.query(&[("readonly", "1")]);
}
request
}
pub async fn execute(&self, sql: &str, max_rows: usize, cancel: &CancellationToken) -> IdeResult<QueryResult> {
if transaction_statement(sql) {
return Err(transaction_error());
}
let mut session = tokio::select! {
biased;
() = cancel.cancelled() => return Err(IdeError::cancelled()),
() = self.0.closed.cancelled() => return Err(IdeError::new(ErrorKind::Io, "ClickHouse connection is closed")),
session = self.0.session.lock() => session,
};
let client = self.client()?;
let query_id = uuid::Uuid::new_v4().to_string();
let request = self.request(&client, &query_id, max_rows, *session).body(sql.to_owned());
let started = Instant::now();
let fetch = async {
let mut response = request.send().await.map_err(http_error)?;
let execution_ms = started.elapsed().as_secs_f64() * 1000.0;
let status = response.status();
let header = |name: &str| response.headers().get(name).and_then(|v| v.to_str().ok()).map(str::to_owned);
let exception_code = header("X-ClickHouse-Exception-Code");
let format = header("X-ClickHouse-Format");
let affected = header("X-ClickHouse-Summary")
.and_then(|v| serde_json::from_str::<serde_json::Value>(&v).ok())
.and_then(|v| {
v.get("written_rows")
.and_then(|n| n.as_str().and_then(|n| n.parse::<u64>().ok()).or_else(|| n.as_u64()))
});
if !status.is_success() || exception_code.as_deref().is_some_and(|code| code != "0") {
let body = response.bytes().await.map_err(http_error)?;
return Err(self.server_error(status, exception_code.as_deref(), &body));
}
let (columns, rows, has_more) = if format.as_deref().is_some_and(|f| f != FORMAT) {
let body = response.bytes().await.map_err(http_error)?;
if !body.iter().all(u8::is_ascii_whitespace) {
return Err(IdeError::invalid(
"ClickHouse result grids need the default output format; remove the query's FORMAT clause",
));
}
(Vec::new(), Vec::new(), false)
} else {
let mut reader = RowReader::new(max_rows);
while let Some(chunk) = response.chunk().await.map_err(http_error)? {
reader.push(&chunk)?;
}
reader.finish().map_err(|error| self.redact_error(error))?
};
let affected = if columns.is_empty() { affected.map(|n| n as f64) } else { None };
Ok(QueryResult {
statement: sql.to_owned(),
columns,
rows,
affected,
has_more,
elapsed_ms: started.elapsed().as_secs_f64() * 1000.0,
execution_ms,
})
};
let result = tokio::select! {
biased;
() = cancel.cancelled() => None,
() = self.0.closed.cancelled() => None,
result = fetch => Some(result),
};
match result {
Some(result) => {
if result.is_ok() {
*session = true;
}
result
}
None => {
let killed = self.kill(&client, &query_id).await;
match killed {
Ok(()) => Err(IdeError::cancelled()),
Err(error) => Err(IdeError::new(
ErrorKind::Cancelled,
format!("cancelled locally; ClickHouse server cancellation failed: {}", error.message),
)),
}
}
}
}
async fn kill(&self, client: &Client, query_id: &str) -> IdeResult<()> {
let response = client
.post(self.0.endpoint.clone())
.basic_auth(&self.0.user, Some(&self.0.password))
.timeout(Duration::from_secs(3))
.body(format!("KILL QUERY WHERE query_id = '{query_id}' SYNC"))
.send()
.await
.map_err(http_error)?;
let status = response.status();
let code = response
.headers()
.get("X-ClickHouse-Exception-Code")
.and_then(|value| value.to_str().ok())
.map(str::to_owned);
let body = response.bytes().await.map_err(http_error)?;
if status.is_success() && !code.as_deref().is_some_and(|code| code != "0") && !body.starts_with(b"Code:") {
Ok(())
} else {
Err(self.server_error(status, code.as_deref(), &body))
}
}
fn redact_error(&self, mut error: IdeError) -> IdeError {
if !self.0.password.is_empty() {
error.message = error.message.replace(&self.0.password, "[redacted]");
}
error
}
fn server_error(&self, status: StatusCode, code: Option<&str>, body: &[u8]) -> IdeError {
let message = String::from_utf8_lossy(body);
let message: String = exception_text(&message).chars().take(16_384).collect();
let auth = matches!(status, StatusCode::UNAUTHORIZED | StatusCode::FORBIDDEN)
|| matches!(code, Some("193" | "194" | "195" | "516" | "497"));
let detail = if message.is_empty() { status.canonical_reason().unwrap_or("request failed") } else { &message };
let detail = if detail.contains("SESSION_NOT_FOUND") || detail.contains("Session not found") {
"ClickHouse HTTP session expired; reconnect to restore session settings"
} else {
detail
};
self.redact_error(IdeError::new(
if auth { ErrorKind::PermissionDenied } else { ErrorKind::InvalidInput },
format!("ClickHouse HTTP {}: {detail}", status.as_u16()),
))
}
}
type DecodedResult = (Vec<ResultColumn>, Vec<Vec<Option<String>>>, bool);
/// Decodes a streamed [`FORMAT`] body. Keeps the first `max_rows` rows; later rows only set
/// `has_more` and are read past unparsed, so a grid limit bounds memory however large the
/// block `result_overflow_mode=break` lets the server send. A line that is not a JSON array
/// starts the exception ClickHouse writes when a query fails mid-stream.
struct RowReader {
max_rows: usize,
pending: Vec<u8>,
names: Option<Vec<String>>,
types: Option<Vec<String>>,
rows: Vec<Vec<Option<String>>>,
more: bool,
exception: Option<String>,
}
impl RowReader {
fn new(max_rows: usize) -> Self {
Self { max_rows, pending: Vec::new(), names: None, types: None, rows: Vec::new(), more: false, exception: None }
}
fn push(&mut self, chunk: &[u8]) -> IdeResult<()> {
self.pending.extend_from_slice(chunk);
let buffer = std::mem::take(&mut self.pending);
let mut start = 0;
while let Some(end) = buffer[start..].iter().position(|&b| b == b'\n') {
self.line(&buffer[start..start + end])?;
start += end + 1;
}
self.pending = buffer[start..].to_vec();
Ok(())
}
fn line(&mut self, line: &[u8]) -> IdeResult<()> {
if let Some(exception) = &mut self.exception {
if exception.len() < 16_384 {
exception.push_str(&String::from_utf8_lossy(line));
exception.push('\n');
}
return Ok(());
}
let text = line.trim_ascii();
if text.is_empty() {
return Ok(());
}
if text[0] != b'[' {
self.exception = Some(format!("{}\n", String::from_utf8_lossy(line)));
return Ok(());
}
if self.names.is_some() && self.types.is_some() && self.rows.len() >= self.max_rows {
self.more = true;
return Ok(());
}
let Ok(values) = serde_json::from_slice::<Vec<Box<serde_json::value::RawValue>>>(text) else {
self.exception = Some(format!("{}\n", String::from_utf8_lossy(line)));
return Ok(());
};
let protocol = |message: String| IdeError::new(ErrorKind::Protocol, message);
let header = |values: Vec<Box<serde_json::value::RawValue>>| {
values
.iter()
.map(|v| serde_json::from_str::<String>(v.get()))
.collect::<Result<Vec<_>, _>>()
.map_err(|e| protocol(format!("invalid ClickHouse result header: {e}")))
};
let Some(names) = &self.names else {
self.names = Some(header(values)?);
return Ok(());
};
if self.types.is_none() {
self.types = Some(header(values)?);
return Ok(());
}
if values.len() != names.len() {
return Err(protocol("ClickHouse result row width does not match its metadata".into()));
}
let row = values
.into_iter()
.map(|value| {
let text = value.get();
if text == "null" {
Ok(None)
} else if text.starts_with('"') {
serde_json::from_str::<String>(text).map(Some).map_err(|e| protocol(e.to_string()))
} else {
Ok(Some(text.to_owned()))
}
})
.collect::<IdeResult<_>>()?;
self.rows.push(row);
Ok(())
}
fn finish(mut self) -> IdeResult<DecodedResult> {
let rest = std::mem::take(&mut self.pending);
self.line(&rest)?;
if let Some(exception) = self.exception {
return Err(IdeError::invalid(format!("ClickHouse: {}", exception_text(&exception))));
}
let Some(names) = self.names else { return Ok((Vec::new(), Vec::new(), false)) };
let types = self.types.unwrap_or_default();
if types.len() != names.len() {
return Err(IdeError::new(ErrorKind::Protocol, "ClickHouse result types do not match its columns"));
}
let columns = names.into_iter().zip(types).map(|(name, type_name)| ResultColumn { name, type_name }).collect();
Ok((columns, self.rows, self.more))
}
}
/// The server's message in a failed response: rows already streamed and the
/// `__exception__` block markers ClickHouse wraps a mid-stream error in are dropped.
fn exception_text(body: &str) -> &str {
let Some(at) = body.find("Code:") else { return body.trim() };
let text = &body[at..];
match text.find("\n__exception__") {
// The line before the closing marker is `<length> <tag>`.
Some(end) => text[..end].trim_end().rsplit_once('\n').map_or(text[..end].trim(), |(message, _)| message.trim()),
None => text.trim(),
}
}
fn http_error(error: reqwest::Error) -> IdeError {
use std::error::Error;
let kind = if error.is_timeout() { ErrorKind::TimedOut } else { ErrorKind::Io };
let error = error.without_url();
let mut message = error.to_string();
let mut source = error.source();
while let Some(error) = source {
let detail = error.to_string();
if !message.contains(&detail) {
message.push_str(": ");
message.push_str(&detail);
}
source = error.source();
}
IdeError::new(kind, format!("ClickHouse HTTP: {message}"))
}
fn read_pem(path: &str, label: &str) -> IdeResult<Vec<u8>> {
std::fs::read(ide_ssh::expand(path))
.map_err(|error| IdeError::new(ErrorKind::Io, format!("cannot read ClickHouse {label}: {error}")))
}
fn reserved_setting(name: &str) -> bool {
let name = name.to_ascii_lowercase();
name.starts_with("output_format_json_")
|| matches!(
name.as_str(),
"user"
| "password"
| "query"
| "database"
| "session_id"
| "session_check"
| "session_timeout"
| "query_id"
| "default_format"
| "max_result_rows"
| "result_overflow_mode"
| "readonly"
)
}
fn transaction_statement(sql: &str) -> bool {
let mut sql = sql.trim_start();
loop {
if let Some(rest) = sql.strip_prefix("--").or_else(|| sql.strip_prefix('#')) {
sql = rest.split_once('\n').map_or("", |(_, rest)| rest).trim_start();
} else if let Some(rest) = sql.strip_prefix("/*") {
sql = rest.split_once("*/").map_or("", |(_, rest)| rest).trim_start();
} else {
break;
}
}
let mut words = sql.split(|c: char| !c.is_ascii_alphanumeric() && c != '_').filter(|s| !s.is_empty());
let first = words.next().unwrap_or_default();
["BEGIN", "COMMIT", "ROLLBACK", "SAVEPOINT", "RELEASE"].iter().any(|s| first.eq_ignore_ascii_case(s))
|| (first.eq_ignore_ascii_case("START") && words.next().is_some_and(|s| s.eq_ignore_ascii_case("TRANSACTION")))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::conn::Pool;
use crate::source::{DbDriver, DbOptions, DbProperty};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
const ONE: &str = "[\"1\"]\n[\"UInt8\"]\n[1]\n";
async fn read_request(stream: &mut tokio::net::TcpStream) -> String {
let mut bytes = Vec::new();
let mut buffer = [0; 4096];
loop {
let count = tokio::time::timeout(Duration::from_secs(5), stream.read(&mut buffer)).await.unwrap().unwrap();
assert!(count > 0, "request ended early");
bytes.extend_from_slice(&buffer[..count]);
if let Some(end) = bytes.windows(4).position(|s| s == b"\r\n\r\n") {
let header = String::from_utf8_lossy(&bytes[..end]);
let length: usize = header
.lines()
.find_map(|line| {
line.to_ascii_lowercase().strip_prefix("content-length:").map(|v| v.trim().parse().unwrap())
})
.unwrap_or(0);
if bytes.len() >= end + 4 + length {
return String::from_utf8(bytes).unwrap();
}
}
}
}
async fn reply(stream: &mut tokio::net::TcpStream, status: &str, headers: &str, body: &str) {
let response =
format!("HTTP/1.1 {status}\r\nContent-Length: {}\r\nConnection: close\r\n{headers}\r\n{body}", body.len());
stream.write_all(response.as_bytes()).await.unwrap();
}
async fn server(
responses: Vec<(&'static str, &'static str, &'static str)>,
) -> (u16, tokio::task::JoinHandle<Vec<String>>) {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let port = listener.local_addr().unwrap().port();
let task = tokio::spawn(async move {
let mut requests = Vec::new();
for (status, headers, body) in responses {
let (mut stream, _) =
tokio::time::timeout(Duration::from_secs(5), listener.accept()).await.unwrap().unwrap();
requests.push(read_request(&mut stream).await);
reply(&mut stream, status, headers, body).await;
}
requests
});
(port, task)
}
fn source(port: u16) -> DataSource {
DataSource { driver: DbDriver::Clickhouse, host: "127.0.0.1".into(), port: Some(port), ..Default::default() }
}
fn decode(body: &[u8], max_rows: usize, chunk: usize) -> IdeResult<DecodedResult> {
let mut reader = RowReader::new(max_rows);
for part in body.chunks(chunk) {
reader.push(part)?;
}
reader.finish()
}
#[test]
fn rows_preserve_precision_nulls_and_strings_across_chunks() {
let body = concat!(
r#"["n", "d", "v", "v", "v"]"#,
"\n",
r#"["UInt64", "Decimal(38, 9)", "Nullable(String)", "Nullable(String)", "Nullable(String)"]"#,
"\n",
r#"[18446744073709551615, 12345678901234567890.123456789, null, "null", "ᴺᵁᴸᴸ"]"#,
);
for chunk in [1, 7, body.len()] {
let (columns, rows, more) = decode(body.as_bytes(), 1, chunk).unwrap();
assert_eq!(columns.len(), 5);
assert_eq!(columns[1].type_name, "Decimal(38, 9)");
assert_eq!(
rows[0],
vec![
Some("18446744073709551615".into()),
Some("12345678901234567890.123456789".into()),
None,
Some("null".into()),
Some("ᴺᵁᴸᴸ".into()),
]
);
assert!(!more);
}
let (columns, rows, more) = decode(b"[\"x\"]\n[\"String\"]\n", 0, 64).unwrap();
assert_eq!(columns[0].name, "x");
assert!(rows.is_empty() && !more);
assert_eq!(decode(b"", 5, 64).unwrap().0.len(), 0);
assert!(decode(b"[\"x\"]\n[\"UInt8\"]\n[1, 2]\n", 1, 64).is_err());
}
#[test]
fn rows_past_the_limit_are_skipped_but_a_later_exception_still_fails() {
let mut body = String::from("[\"n\"]\n[\"UInt64\"]\n");
for n in 0..1000 {
body.push_str(&format!("[{n}]\n"));
}
let (_, rows, more) = decode(body.as_bytes(), 2, 4096).unwrap();
assert_eq!(rows, vec![vec![Some("0".into())], vec![Some("1".into())]]);
assert!(more);
body.push_str("\n__exception__\nabcdefgh\nCode: 395. DB::Exception: Value passed to 'throwIf' is non-zero\n");
body.push_str("62 abcdefgh\n__exception__\n");
let error = decode(body.as_bytes(), 2, 4096).unwrap_err();
assert_eq!(error.message, "ClickHouse: Code: 395. DB::Exception: Value passed to 'throwIf' is non-zero");
let multiline = "Code: 62. DB::Exception: Syntax error\nat line 2";
assert_eq!(exception_text(multiline), multiline);
}
#[tokio::test]
async fn http_requests_use_auth_settings_limits_and_a_shared_session() {
let (port, requests) = server(vec![
("200 OK", "X-ClickHouse-Format: JSONCompactEachRowWithNamesAndTypes\r\n", ONE),
("200 OK", "", ""),
(
"200 OK",
"X-ClickHouse-Format: JSONCompactEachRowWithNamesAndTypes\r\n",
"[\"n\"]\n[\"UInt64\"]\n[\"1\"]\n[\"2\"]\n[\"3\"]\n",
),
])
.await;
let source = DataSource {
user: "reader".into(),
read_only: true,
options: DbOptions { default_schema: "analytics".into(), time_zone: "UTC".into(), ..Default::default() },
properties: vec![DbProperty { name: "max_threads".into(), value: "2".into() }],
..source(port)
};
let pool = Pool::connect(&source, Some("mock-password"), None).await.unwrap();
pool.execute("USE other", 0, &CancellationToken::new()).await.unwrap();
let result = pool.execute("SELECT number FROM numbers(3)", 1, &CancellationToken::new()).await.unwrap();
assert_eq!(result.rows, vec![vec![Some("1".into())]]);
assert!(result.has_more && result.affected.is_none());
let requests = requests.await.unwrap();
let target = |request: &String| {
Url::parse(&format!(
"http://localhost{}",
request.lines().next().unwrap().split_whitespace().nth(1).unwrap()
))
.unwrap()
};
let urls: Vec<_> = requests.iter().map(target).collect();
let value = |url: &Url, key: &str| url.query_pairs().find(|(k, _)| k == key).map(|(_, v)| v.into_owned());
assert_eq!(value(&urls[0], "database").as_deref(), Some("analytics"));
assert_eq!(value(&urls[1], "database"), None);
assert_eq!(value(&urls[1], "session_check").as_deref(), Some("1"));
assert_eq!(value(&urls[0], "session_id"), value(&urls[2], "session_id"));
assert_ne!(value(&urls[0], "query_id"), value(&urls[2], "query_id"));
assert_eq!(value(&urls[2], "max_result_rows").as_deref(), Some("2"));
for (request, url) in requests.iter().zip(&urls) {
assert!(request.starts_with("POST /?"));
assert!(request.contains("Basic cmVhZGVyOm1vY2stcGFzc3dvcmQ="));
assert!(!url.as_str().contains("reader") && !url.as_str().contains("mock-password"));
assert_eq!(value(url, "readonly").as_deref(), Some("1"));
assert_eq!(value(url, "session_timezone").as_deref(), Some("UTC"));
assert_eq!(value(url, "max_threads").as_deref(), Some("2"));
}
assert!(requests[0].ends_with("SELECT 1"));
assert!(pool.transaction(&[]).await.is_err());
let cancel = CancellationToken::new();
cancel.cancel();
assert_eq!(pool.execute("SELECT 1", 1, &cancel).await.unwrap_err().kind, ErrorKind::Cancelled);
pool.close().await;
assert!(pool.is_closed());
assert!(pool.execute("SELECT 1", 1, &CancellationToken::new()).await.is_err());
}
#[tokio::test]
async fn sequential_queries_reuse_the_http_connection() {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let port = listener.local_addr().unwrap().port();
let task = tokio::spawn(async move {
let (mut stream, _) = listener.accept().await.unwrap();
for _ in 0..2 {
read_request(&mut stream).await;
let response = format!("HTTP/1.1 200 OK\r\nContent-Length: {}\r\n\r\n{ONE}", ONE.len());
stream.write_all(response.as_bytes()).await.unwrap();
}
});
let pool = Pool::connect(&source(port), None, None).await.unwrap();
assert_eq!(
pool.execute("SELECT 1", 1, &CancellationToken::new()).await.unwrap().rows,
vec![vec![Some("1".into())]]
);
task.await.unwrap();
}
#[tokio::test]
async fn server_errors_keep_details_and_hide_passwords() {
let (port, requests) = server(vec![(
"403 Forbidden",
"X-ClickHouse-Exception-Code: 516\r\n",
"Code: 516. DB::Exception: bad mock-password",
)])
.await;
let error = match Pool::connect(&source(port), Some("mock-password"), None).await {
Ok(_) => panic!("expected login failure"),
Err(error) => error,
};
assert_eq!(error.kind, ErrorKind::PermissionDenied);
assert!(error.message.contains("516") && error.message.contains("[redacted]"));
assert!(!error.message.contains("mock-password"));
requests.await.unwrap();
let (port, requests) = server(vec![("200 OK", "", "Code: 241. DB::Exception: memory limit")]).await;
let error = match Pool::connect(&source(port), None, None).await {
Ok(_) => panic!("expected late query failure"),
Err(error) => error,
};
assert!(error.message.contains("memory limit"));
requests.await.unwrap();
let (port, requests) = server(vec![(
"500 Internal Server Error",
"X-ClickHouse-Exception-Code: 395\r\n",
"[\"1\"]\n[\"UInt8\"]\n[1]\n\n__exception__\ntag\nCode: 395. DB::Exception: thrown\n32 tag\n__exception__\n",
)])
.await;
let error = match Pool::connect(&source(port), None, None).await {
Ok(_) => panic!("expected mid-stream failure"),
Err(error) => error,
};
assert_eq!(error.message, "ClickHouse HTTP 500: Code: 395. DB::Exception: thrown");
requests.await.unwrap();
}
#[tokio::test]
async fn cancellation_sends_kill_outside_the_busy_session() {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let port = listener.local_addr().unwrap().port();
let (started, received) = tokio::sync::oneshot::channel();
let server = tokio::spawn(async move {
let (mut health, _) = listener.accept().await.unwrap();
read_request(&mut health).await;
reply(&mut health, "200 OK", "", ONE).await;
drop(health);
let (mut query, _) = listener.accept().await.unwrap();
let request = read_request(&mut query).await;
started.send(()).unwrap();
let (mut kill, _) = tokio::time::timeout(Duration::from_secs(5), listener.accept()).await.unwrap().unwrap();
let kill_request = read_request(&mut kill).await;
reply(&mut kill, "200 OK", "", "").await;
(request, kill_request)
});
let pool = Pool::connect(&source(port), None, None).await.unwrap();
let cancel = CancellationToken::new();
let query = pool.execute("SELECT sleep(30)", 1, &cancel);
let cancelled = async {
received.await.unwrap();
cancel.cancel();
};
let (result, ()) = tokio::join!(query, cancelled);
assert_eq!(result.unwrap_err().kind, ErrorKind::Cancelled);
let (query, kill) = server.await.unwrap();
let target = query.lines().next().unwrap().split_whitespace().nth(1).unwrap();
let url = Url::parse(&format!("http://localhost{target}")).unwrap();
let query_id = url.query_pairs().find(|(k, _)| k == "query_id").unwrap().1.into_owned();
assert!(kill.ends_with(&format!("KILL QUERY WHERE query_id = '{query_id}' SYNC")));
assert!(!kill.contains("session_id="));
}
#[tokio::test]
async fn unsupported_settings_and_transactions_fail_before_connecting() {
for statement in ["BEGIN", "-- comment\n COMMIT", "/* comment */ ROLLBACK", "START TRANSACTION", "SAVEPOINT s"]
{
assert!(transaction_statement(statement));
}
assert!(!transaction_statement("SELECT 'BEGIN'"));
let sources = [
DataSource { options: DbOptions { transaction: DbTxMode::Manual, ..Default::default() }, ..source(1) },
DataSource {
properties: vec![DbProperty { name: "statement_cache_capacity".into(), value: "4".into() }],
..source(1)
},
DataSource { properties: vec![DbProperty { name: "password".into(), value: "x".into() }], ..source(1) },
DataSource { ssl_ca_file: "unused.pem".into(), ..source(1) },
DataSource { ssl: DbSslMode::VerifyCa, ..source(1) },
DataSource { ssl: DbSslMode::Require, ssl_cert_file: "unused.pem".into(), ..source(1) },
];
for source in sources {
let error = match Pool::connect(&source, None, None).await {
Ok(_) => panic!("unsupported settings were accepted"),
Err(error) => error,
};
assert_eq!(error.kind, ErrorKind::InvalidInput);
}
}
#[tokio::test]
#[ignore = "requires CLICKHOUSE_TEST_HOST, CLICKHOUSE_TEST_USER, and CLICKHOUSE_TEST_PASSWORD"]
async fn clickhouse_cloud_transport() {
let source = DataSource {
driver: DbDriver::Clickhouse,
host: std::env::var("CLICKHOUSE_TEST_HOST").expect("CLICKHOUSE_TEST_HOST"),
port: Some(std::env::var("CLICKHOUSE_TEST_PORT").ok().and_then(|v| v.parse().ok()).unwrap_or(8443)),
user: std::env::var("CLICKHOUSE_TEST_USER").expect("CLICKHOUSE_TEST_USER"),
ssl: DbSslMode::Require,
options: DbOptions {
time_zone: "UTC".into(),
startup_script: "SET max_threads = 2".into(),
..Default::default()
},
properties: vec![DbProperty { name: "max_block_size".into(), value: "50".into() }],
..Default::default()
};
let password = std::env::var("CLICKHOUSE_TEST_PASSWORD").expect("CLICKHOUSE_TEST_PASSWORD");
let pool = Pool::connect(&source, Some(&password), None).await.unwrap();
let cancel = CancellationToken::new();
let settings = pool
.execute("SELECT getSetting('max_threads'), getSetting('max_block_size'), timezone()", 1, &cancel)
.await
.unwrap();
assert_eq!(settings.rows, vec![vec![Some("2".into()), Some("50".into()), Some("UTC".into())]]);
let result = pool.execute("SELECT toUInt64('18446744073709551615') AS n, CAST(NULL AS Nullable(String)) AS missing, CAST('ᴺᵁᴸᴸ' AS Nullable(String)) AS text, 'a\\nb' AS lines", 1, &cancel).await.unwrap();
assert_eq!(
result.rows,
vec![vec![Some("18446744073709551615".into()), None, Some("ᴺᵁᴸᴸ".into()), Some("a\nb".into())]]
);
assert_eq!(result.columns[1].type_name, "Nullable(String)");
let limited = pool.execute("SELECT number FROM numbers(100)", 2, &cancel).await.unwrap();
assert_eq!(limited.rows.len(), 2);
assert!(limited.has_more);
let marker = format!("transport_cancel_{}", uuid::Uuid::new_v4().simple());
let sql = format!("SELECT sleepEachRow(0.05) FROM numbers(50) /* {marker} */");
let stop = CancellationToken::new();
let query = pool.execute(&sql, 1, &stop);
let cancel_query = async {
tokio::time::sleep(Duration::from_millis(500)).await;
stop.cancel();
};
let (cancelled, ()) = tokio::join!(query, cancel_query);
let cancelled = cancelled.unwrap_err();
assert_eq!(cancelled.kind, ErrorKind::Cancelled);
assert_eq!(cancelled.message, "cancelled");
let active = pool.execute(&format!("SELECT count() FROM system.processes WHERE position(query, '{marker}') > 0 AND query_id != currentQueryID()"), 1, &cancel).await.unwrap();
assert_eq!(active.rows, vec![vec![Some("0".into())]]);
let table = format!("transport_test_{}", uuid::Uuid::new_v4().simple());
pool.execute(&format!("CREATE TEMPORARY TABLE {table} (n UInt8)"), 0, &cancel).await.unwrap();
let continuity = async {
pool.execute(&format!("INSERT INTO {table} VALUES (7)"), 0, &cancel).await?;
pool.execute(&format!("SELECT n FROM {table}"), 1, &cancel).await
}
.await;
let cleanup = pool.execute(&format!("DROP TEMPORARY TABLE {table}"), 0, &cancel).await;
assert_eq!(continuity.unwrap().rows, vec![vec![Some("7".into())]]);
cleanup.unwrap();
pool.close().await;
assert!(pool.is_closed());
}
}
+37 -3
View File
@@ -106,6 +106,7 @@ pub enum Pool {
Postgres(PgPool),
Mysql(MySqlPool),
SqlServer(Bb8Pool<MssqlConnectionManager>),
Clickhouse(crate::clickhouse::ClickhousePool),
}
/// Runs `$body` with `$p` bound to the concrete pool of each driver.
@@ -115,7 +116,7 @@ macro_rules! with_pool {
Pool::Sqlite($p) => $body,
Pool::Postgres($p) => $body,
Pool::Mysql($p) => $body,
Pool::SqlServer(_) => unreachable!(),
Pool::SqlServer(_) | Pool::Clickhouse(_) => unreachable!(),
}
};
}
@@ -503,6 +504,9 @@ impl Setup {
value.parse::<u64>().map_err(|_| IdeError::invalid(format!("{what} must be a whole number: {value}")))
};
match (source.driver, name.to_ascii_lowercase().as_str()) {
(DbDriver::Clickhouse, "statement_cache_capacity") => {
return Err(IdeError::invalid("ClickHouse HTTP does not support statement caching"));
}
(_, "statement_cache_capacity") => s.statement_cache = Some(number(&name)? as usize),
(_, "connect_timeout") => s.connect_timeout = Some(Duration::from_secs(number(&name)?)),
(DbDriver::Postgres, "application_name") => s.application_name = Some(value.to_owned()),
@@ -512,6 +516,9 @@ impl Setup {
_ => s.params.push((name, value.to_owned())),
}
}
if source.driver == DbDriver::Clickhouse && !o.time_zone.trim().is_empty() {
s.params.push(("session_timezone".into(), o.time_zone.trim().into()));
}
let script = o.startup_script.trim();
if !script.is_empty() {
s.statements.extend(crate::split::split(script, source.driver).into_iter().map(|st| st.text));
@@ -527,6 +534,7 @@ impl Pool {
Pool::Postgres(_) => DbDriver::Postgres,
Pool::Mysql(_) => DbDriver::Mysql,
Pool::SqlServer(_) => DbDriver::SqlServer,
Pool::Clickhouse(_) => DbDriver::Clickhouse,
}
}
@@ -534,6 +542,9 @@ impl Pool {
pub async fn connect(source: &DataSource, password: Option<&str>, root: Option<&Path>) -> IdeResult<Self> {
const MAX: u32 = 4;
let setup = Setup::of(source)?;
if source.driver == DbDriver::Clickhouse {
return crate::clickhouse::ClickhousePool::connect(source, password, setup).await.map(Pool::Clickhouse);
}
let timeout = setup.connect_timeout.unwrap_or(Duration::from_secs(30));
let init = Arc::new(setup.statements);
// Runs the session statements (startup script…) on every new connection.
@@ -695,12 +706,14 @@ impl Pool {
conn.query("SELECT 1", &[]).await.map_err(mssql_err)?.into_results().await.map_err(mssql_err)?;
Ok(Pool::SqlServer(pool))
}
DbDriver::Clickhouse => unreachable!(),
}
}
pub async fn close(&self) {
match self {
Pool::SqlServer(_) => {}
Pool::Clickhouse(p) => p.close(),
_ => with_pool!(self, p => p.close().await),
}
}
@@ -708,12 +721,16 @@ impl Pool {
pub fn is_closed(&self) -> bool {
match self {
Pool::SqlServer(_) => false,
Pool::Clickhouse(p) => p.is_closed(),
_ => with_pool!(self, p => p.is_closed()),
}
}
/// Runs `sql` (no row limit beyond `max`), for introspection: cells are text (see [`Cells::Text`]).
pub async fn rows(&self, sql: &str, max: usize) -> IdeResult<Vec<Vec<Option<String>>>> {
if let Pool::Clickhouse(p) = self {
return Ok(p.execute(sql, max, &CancellationToken::new()).await?.rows);
}
let fetched: IdeResult<Fetched> = match self {
Pool::SqlServer(p) => {
let mut conn = p.get_owned().await.map_err(mssql_pool_err)?;
@@ -727,6 +744,9 @@ impl Pool {
/// Runs one statement and keeps at most `max_rows` rows. Cancelling stops
/// the statement on the server (PostgreSQL, MySQL) and drops the connection.
pub async fn execute(&self, sql: &str, max_rows: usize, cancel: &CancellationToken) -> IdeResult<QueryResult> {
if let Pool::Clickhouse(p) = self {
return p.execute(sql, max_rows, cancel).await;
}
let mut conn = Held::acquire(self).await?;
let r = conn.execute(self, sql, max_rows, cancel).await;
if cancel.is_cancelled() {
@@ -740,6 +760,7 @@ impl Pool {
/// row when flagged (table editor submits).
pub async fn transaction(&self, statements: &[(String, bool)]) -> IdeResult<u64> {
match self {
Pool::Clickhouse(_) => Err(crate::clickhouse::transaction_error()),
Pool::SqlServer(p) => {
let mut conn = p.get_owned().await.map_err(mssql_pool_err)?;
let result = async {
@@ -790,6 +811,7 @@ pub enum Held {
Postgres(PoolConnection<Postgres>),
Mysql(PoolConnection<MySql>),
SqlServer(Box<bb8::PooledConnection<'static, MssqlConnectionManager>>),
Clickhouse(crate::clickhouse::ClickhousePool),
}
impl Held {
@@ -799,6 +821,12 @@ impl Held {
Pool::Postgres(p) => Held::Postgres(p.acquire().await.map_err(db_err)?),
Pool::Mysql(p) => Held::Mysql(p.acquire().await.map_err(db_err)?),
Pool::SqlServer(p) => Held::SqlServer(Box::new(p.get_owned().await.map_err(mssql_pool_err)?)),
Pool::Clickhouse(p) => {
if p.is_closed() {
return Err(IdeError::new(ErrorKind::Io, "ClickHouse connection is closed"));
}
Held::Clickhouse(p.clone())
}
})
}
@@ -806,13 +834,14 @@ impl Held {
pub async fn command(&mut self, sql: &str) -> IdeResult<()> {
match self {
Held::SqlServer(c) => mssql_command(c, sql).await,
Held::Clickhouse(_) => Err(crate::clickhouse::transaction_error()),
other => {
let sql = sqlx::AssertSqlSafe(sql.to_owned());
match other {
Held::Sqlite(c) => sqlx::raw_sql(sql).execute(&mut **c).await.map(drop),
Held::Postgres(c) => sqlx::raw_sql(sql).execute(&mut **c).await.map(drop),
Held::Mysql(c) => sqlx::raw_sql(sql).execute(&mut **c).await.map(drop),
Held::SqlServer(_) => unreachable!(),
Held::SqlServer(_) | Held::Clickhouse(_) => unreachable!(),
}
.map_err(db_err)
}
@@ -833,6 +862,7 @@ impl Held {
Held::SqlServer(mut c) => {
c.0.take();
}
Held::Clickhouse(_) => {}
}
}
@@ -845,6 +875,9 @@ impl Held {
max_rows: usize,
cancel: &CancellationToken,
) -> IdeResult<QueryResult> {
if let Held::Clickhouse(c) = self {
return c.execute(sql, max_rows, cancel).await;
}
let started = Instant::now();
// `$id`: the query naming the server process, `$kill`: how another connection stops it.
macro_rules! run {
@@ -887,6 +920,7 @@ impl Held {
() = cancel.cancelled() => return Err(IdeError::cancelled()),
}
}
Held::Clickhouse(_) => unreachable!(),
};
let (columns, rows, affected, has_more, first) = fetched;
let elapsed_ms = started.elapsed().as_secs_f64() * 1000.0;
@@ -907,7 +941,7 @@ pub fn begin_statement(driver: DbDriver) -> &'static str {
match driver {
DbDriver::Mysql => "START TRANSACTION",
DbDriver::SqlServer => "BEGIN TRAN",
DbDriver::Sqlite | DbDriver::Postgres => "BEGIN",
DbDriver::Sqlite | DbDriver::Postgres | DbDriver::Clickhouse => "BEGIN",
}
}
+76 -12
View File
@@ -2,6 +2,7 @@
//! execution ([`conn`]), introspection, table data paging / editing ([`sql`])
//! and statement splitting ([`split`]), for SQLite, PostgreSQL and MySQL/MariaDB.
mod clickhouse;
pub mod conn;
pub mod history;
pub mod plan;
@@ -547,12 +548,14 @@ impl DbManager {
})?;
let version = match source.driver {
DbDriver::Sqlite => "SELECT 'SQLite ' || sqlite_version()",
DbDriver::Postgres => "SELECT version()",
DbDriver::Postgres | DbDriver::Clickhouse => "SELECT version()",
DbDriver::Mysql => "SELECT CONCAT(@@version_comment, ' ', VERSION())",
DbDriver::SqlServer => "SELECT @@VERSION",
};
let v = pool.rows(version, 1).await.ok().and_then(|r| r.into_iter().next()?.into_iter().next()?);
let result = pool.rows(version, 1).await;
pool.close().await;
let rows = if source.driver == DbDriver::Clickhouse { result? } else { result.unwrap_or_default() };
let v = rows.into_iter().next().and_then(|r| r.into_iter().next().flatten());
Ok(v.unwrap_or_else(|| "Connected".into()))
}
@@ -701,8 +704,11 @@ impl DbManager {
_ => indexes.push(IndexInfo { name, unique: flag(&unique), columns: vec![col] }),
}
}
let fk_rows =
pool.rows(&sql::table_foreign_keys(driver, schema, table), MAX_META_ROWS).await.unwrap_or_default();
let fk_rows = if driver == DbDriver::Clickhouse {
Vec::new()
} else {
pool.rows(&sql::table_foreign_keys(driver, schema, table), MAX_META_ROWS).await.unwrap_or_default()
};
let mut foreign_keys = group_foreign_keys(fk_rows, &[]);
// SQLite keys that reference the primary key implicitly: look it up.
for k in foreign_keys.iter_mut().filter(|k| k.ref_columns.is_empty()) {
@@ -715,13 +721,15 @@ impl DbManager {
.collect();
}
foreign_keys.retain(|k| k.columns.len() == k.ref_columns.len());
let triggers = pool
.rows(&sql::triggers(driver, schema, table), MAX_META_ROWS)
.await
.unwrap_or_else(|e| {
let trigger_rows = if driver == DbDriver::Clickhouse {
Vec::new()
} else {
pool.rows(&sql::triggers(driver, schema, table), MAX_META_ROWS).await.unwrap_or_else(|e| {
tracing::warn!("triggers: {}", e.message);
Vec::new()
})
};
let triggers = trigger_rows
.into_iter()
.filter_map(|r| {
let mut it = r.into_iter();
@@ -822,6 +830,11 @@ impl DbManager {
let ddl = cells(pool.rows(&q, 1).await?, 1).into_iter().next();
ddl.map(|d| format!("{d};\n")).ok_or_else(missing)
}
DbDriver::Clickhouse => {
let q = format!("SHOW CREATE TABLE {}", sql::qualified(driver, schema, name));
let ddl = cells(pool.rows(&q, 1).await?, 0).into_iter().next();
ddl.map(|d| format!("{};\n", d.trim_end().trim_end_matches(';'))).ok_or_else(missing)
}
DbDriver::SqlServer => {
let obj = format!("{}.{}", schema, name).replace('\'', "''");
let ddl = cells(pool.rows(&format!("SELECT OBJECT_DEFINITION(OBJECT_ID('{}'))", obj), 1).await?, 0)
@@ -902,10 +915,14 @@ impl DbManager {
}
}
// Foreign keys are a nicety (JOIN completion): a server that refuses the catalog query still completes.
let fk_rows = pool.rows(sql::foreign_keys(driver), MAX_META_ROWS).await.unwrap_or_else(|e| {
tracing::warn!("foreign keys: {}", e.message);
let fk_rows = if driver == DbDriver::Clickhouse {
Vec::new()
});
} else {
pool.rows(sql::foreign_keys(driver), MAX_META_ROWS).await.unwrap_or_else(|e| {
tracing::warn!("foreign keys: {}", e.message);
Vec::new()
})
};
let foreign_keys = group_foreign_keys(fk_rows, &tables);
let default_schema = match sql::current_schema(driver) {
None => "main".to_owned(),
@@ -980,6 +997,9 @@ impl DbManager {
let Some(session) = session else {
return pool.execute(statement, max_rows, cancel).await;
};
if pool.driver() == DbDriver::Clickhouse {
return Err(IdeError::invalid("ClickHouse manual transactions are unsupported"));
}
let mut held = session.lock().await;
if held.is_none() {
let mut conn = Held::acquire(&pool).await?;
@@ -998,8 +1018,11 @@ impl DbManager {
/// Switches `id` between auto-commit and manual commit. Leaving manual
/// mode rolls back what was not committed.
pub async fn set_manual_commit(&self, id: &str, manual: bool) {
pub async fn set_manual_commit(&self, id: &str, manual: bool) -> IdeResult<()> {
if manual {
if self.source(id)?.driver == DbDriver::Clickhouse {
return Err(IdeError::invalid("ClickHouse manual transactions are unsupported"));
}
self.sessions.lock().entry(id.to_owned()).or_default();
} else {
let session = self.sessions.lock().remove(id);
@@ -1007,10 +1030,14 @@ impl DbManager {
end_session(&s).await;
}
}
Ok(())
}
/// Commits (or rolls back) the open transaction of a manual-commit data source.
pub async fn finish_transaction(&self, id: &str, commit: bool) -> IdeResult<()> {
if self.source(id).is_ok_and(|s| s.driver == DbDriver::Clickhouse) {
return Err(IdeError::invalid("ClickHouse manual transactions are unsupported"));
}
let session = self.sessions.lock().get(id).cloned();
let Some(session) = session else { return Ok(()) };
let mut held = session.lock().await;
@@ -1052,6 +1079,9 @@ impl DbManager {
/// Applies the table editor's pending changes in one transaction.
pub async fn apply_edits(&self, id: &str, schema: &str, table: &str, edits: &[DbRowEdit]) -> IdeResult<u64> {
if self.source(id)?.driver == DbDriver::Clickhouse {
return Err(IdeError::invalid("ClickHouse table editing is unsupported"));
}
let pool = self.pool(id).await?;
let driver = pool.driver();
let statements = edits
@@ -1064,3 +1094,37 @@ impl DbManager {
pool.transaction(&statements).await
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn clickhouse_rejects_manual_transactions_and_table_edits_without_connecting() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path().to_path_buf();
let ssh = Arc::new(SshManager::new(root.join("ssh-passwords.json")));
let db = DbManager::new(root.clone(), root.join("passwords.json"), root.join("history.json"), ssh);
db.save_sources(vec![DataSource {
id: "clickhouse".into(),
name: "clickhouse".into(),
driver: DbDriver::Clickhouse,
host: "127.0.0.1".into(),
port: Some(1),
..Default::default()
}])
.await
.unwrap();
let error = db.set_manual_commit("clickhouse", true).await.unwrap_err();
assert!(error.message.contains("manual transactions are unsupported"));
assert!(db.sessions.lock().is_empty());
for commit in [false, true] {
let error = db.finish_transaction("clickhouse", commit).await.unwrap_err();
assert!(error.message.contains("manual transactions are unsupported"));
}
let error = db.apply_edits("clickhouse", "default", "events", &[]).await.unwrap_err();
assert!(error.message.contains("table editing is unsupported"));
assert!(!db.is_connected("clickhouse"));
db.set_manual_commit("clickhouse", false).await.unwrap();
}
}
+18 -1
View File
@@ -35,6 +35,7 @@ pub fn explain_statement(driver: DbDriver, statement: &str) -> String {
match driver {
DbDriver::Sqlite => format!("EXPLAIN QUERY PLAN {s}"),
DbDriver::Postgres => format!("EXPLAIN (FORMAT JSON) {s}"),
DbDriver::Clickhouse => format!("EXPLAIN json = 1 {s}"),
DbDriver::Mysql => format!("EXPLAIN {s}"),
DbDriver::SqlServer => format!("SET SHOWPLAN_TEXT ON; {s}"),
}
@@ -44,7 +45,7 @@ pub fn explain_statement(driver: DbDriver, statement: &str) -> String {
pub fn parse(driver: DbDriver, result: &QueryResult) -> IdeResult<Vec<PlanNode>> {
match driver {
DbDriver::Sqlite => Ok(sqlite(&result.rows)),
DbDriver::Postgres => {
DbDriver::Postgres | DbDriver::Clickhouse => {
let text: String =
result.rows.iter().filter_map(|r| r.first().cloned().flatten()).collect::<Vec<_>>().join("\n");
let json: Value = serde_json::from_str(&text).map_err(|e| IdeError::invalid(format!("plan: {e}")))?;
@@ -199,6 +200,7 @@ mod tests {
assert_eq!(explain_statement(DbDriver::Sqlite, "select 1;"), "EXPLAIN QUERY PLAN select 1");
assert_eq!(explain_statement(DbDriver::Postgres, " select 1 "), "EXPLAIN (FORMAT JSON) select 1");
assert_eq!(explain_statement(DbDriver::Mysql, "select 1"), "EXPLAIN select 1");
assert_eq!(explain_statement(DbDriver::Clickhouse, " select 1;"), "EXPLAIN json = 1 select 1");
assert_eq!(explain_statement(DbDriver::SqlServer, "select 1"), "SET SHOWPLAN_TEXT ON; select 1");
}
@@ -254,6 +256,21 @@ mod tests {
assert_eq!((plan[3].parent, plan[3].operation.as_str()), (Some(2), "Seq Scan"));
}
#[test]
fn reads_clickhouse_json_plans_and_rejects_invalid_json() {
let json = r#"[{"Plan": {"Node Type": "Expression", "Node Id": "Expression_5",
"Description": "Project names", "Plans": [
{"Node Type": "ReadFromMergeTree", "Node Id": "ReadFromMergeTree_0",
"Description": "default.events"}
]}}]"#;
let plan = parse(DbDriver::Clickhouse, &result(&["explain"], vec![vec![Some(json)]])).unwrap();
assert_eq!(plan.len(), 2);
assert_eq!((plan[0].parent, plan[0].operation.as_str()), (None, "Expression"));
assert_eq!((plan[1].parent, plan[1].operation.as_str()), (Some(0), "ReadFromMergeTree"));
assert!(plan[1].properties.iter().any(|p| p.name == "Description" && p.value == "default.events"));
assert!(parse(DbDriver::Clickhouse, &result(&["explain"], vec![vec![Some("invalid")]])).is_err());
}
#[test]
fn reads_mysql_rows() {
let r = result(
+40 -1
View File
@@ -18,6 +18,7 @@ pub enum DbDriver {
Mysql,
/// Microsoft SQL Server.
SqlServer,
Clickhouse,
}
impl DbDriver {
@@ -27,6 +28,7 @@ impl DbDriver {
DbDriver::Postgres => Some(5432),
DbDriver::Mysql => Some(3306),
DbDriver::SqlServer => Some(1433),
DbDriver::Clickhouse => Some(8123),
}
}
}
@@ -253,6 +255,7 @@ pub fn scheme_driver(scheme: &str) -> Option<DbDriver> {
"mysql" | "mariadb" | "tidb" | "singlestore" | "memsql" | "oceanbase" => Some(DbDriver::Mysql),
"sqlserver" | "mssql" | "mssqlserver" => Some(DbDriver::SqlServer),
"sqlite" | "sqlite3" => Some(DbDriver::Sqlite),
"clickhouse" | "http" | "https" => Some(DbDriver::Clickhouse),
_ => None,
}
}
@@ -260,7 +263,7 @@ pub fn scheme_driver(scheme: &str) -> Option<DbDriver> {
/// Parses `scheme://[user[:password]@]host[:port][/database][?k=v&…]`, `jdbc:` forms
/// included, and `sqlite:path`.
pub fn parse_url(raw: &str) -> IdeResult<ParsedUrl> {
let bad = || IdeError::invalid(format!("not a database URL: {raw}"));
let bad = || IdeError::invalid("invalid database URL");
let url = raw.trim();
let url = url.strip_prefix("jdbc:").or_else(|| url.strip_prefix("JDBC:")).unwrap_or(url);
let (scheme, rest) = url.split_once(':').ok_or_else(bad)?;
@@ -367,6 +370,15 @@ impl DataSource {
}
properties.append(&mut s.properties);
s.properties = properties;
if s.driver == DbDriver::Clickhouse {
let raw_url = self.url.trim().to_ascii_lowercase();
let url = raw_url.strip_prefix("jdbc:").unwrap_or(&raw_url);
if url.starts_with("https://") {
s.ssl = DbSslMode::VerifyFull;
} else if url.starts_with("http://") {
s.ssl = DbSslMode::Disable;
}
}
Ok(s)
}
@@ -519,4 +531,31 @@ mod tests {
assert!(DataSource { url: "mysql://h/db".into(), ..s.clone() }.resolved().is_err());
assert_eq!(DataSource { url_only: false, ..s }.resolved().unwrap().host, "localhost");
}
#[test]
fn clickhouse_urls_resolve_http_https_and_credentials() {
assert_eq!(DbDriver::Clickhouse.default_port(), Some(8123));
let source = DataSource {
driver: DbDriver::Clickhouse,
url_only: true,
url: "https://reader:s%[email protected]:8443/analytics?max_execution_time=30".into(),
..Default::default()
};
let resolved = source.resolved().unwrap();
assert_eq!(resolved.host, "db.example");
assert_eq!(resolved.port, Some(8443));
assert_eq!(resolved.database, "analytics");
assert_eq!(resolved.user, "reader");
assert_eq!(resolved.ssl, DbSslMode::VerifyFull);
assert_eq!(source.url_password().as_deref(), Some("s@cret"));
assert_eq!(resolved.properties[0].name, "max_execution_time");
let http = DataSource { url: "http://localhost/default".into(), ..source.clone() }.resolved().unwrap();
assert_eq!(http.ssl, DbSslMode::Disable);
let jdbc = DataSource { url: "jdbc:clickhouse://localhost/analytics".into(), ..source.clone() };
assert_eq!(jdbc.resolved().unwrap().database, "analytics");
let secure = DataSource { url: "https://db.example/default?sslmode=disable".into(), ..source };
assert_eq!(secure.resolved().unwrap().ssl, DbSslMode::VerifyFull);
let error = parse_url("clickhouse://reader:[email protected]:invalid/default").unwrap_err();
assert!(!error.message.contains("secret"));
}
}
+15 -4
View File
@@ -28,6 +28,7 @@ enum State {
/// Every statement of `text` that contains more than whitespace and comments.
pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
let mysql = driver == DbDriver::Mysql;
let clickhouse = driver == DbDriver::Clickhouse;
let chars: Vec<(usize, char)> = text.char_indices().collect();
// UTF-16 offset of each char index (plus the end).
let mut u16_at = Vec::with_capacity(chars.len() + 1);
@@ -81,7 +82,7 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
state = State::Double;
has_code = true;
}
'`' if mysql => {
'`' if mysql || clickhouse => {
state = State::Backtick;
has_code = true;
}
@@ -117,7 +118,7 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
_ => {}
},
State::Single => {
if mysql && c == '\\' {
if (mysql || clickhouse) && c == '\\' {
i += 1;
} else if c == '\'' {
if next == Some('\'') {
@@ -128,7 +129,9 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
}
}
State::Double => {
if c == '"' {
if clickhouse && c == '\\' {
i += 1;
} else if c == '"' {
if next == Some('"') {
i += 1;
} else {
@@ -137,7 +140,9 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
}
}
State::Backtick => {
if c == '`' {
if clickhouse && (c == '\\' || (c == '`' && next == Some('`'))) {
i += 1;
} else if c == '`' {
state = State::Code;
}
}
@@ -224,6 +229,12 @@ mod tests {
assert_eq!(texts(sql, DbDriver::Mysql), vec!["select `a;b`, 'it\\'s;' from t # x;y", "select 2"]);
}
#[test]
fn clickhouse_backticks_and_escaped_quotes_keep_statements_intact() {
let query = r#"SELECT `a\`;b`, "c\";d", 'it\'s;safe'; SELECT 2"#;
assert_eq!(texts(query, DbDriver::Clickhouse), vec![r#"SELECT `a\`;b`, "c\";d", 'it\'s;safe'"#, "SELECT 2"]);
}
#[test]
fn postgres_dollar_quotes() {
let sql =
+106 -2
View File
@@ -7,6 +7,7 @@ use specta::Type;
pub fn quote_ident(driver: DbDriver, name: &str) -> String {
match driver {
DbDriver::Clickhouse => format!("`{}`", name.replace('\\', "\\\\").replace('`', "\\`")),
DbDriver::Mysql => format!("`{}`", name.replace('`', "``")),
DbDriver::SqlServer => format!("[{}]", name.replace(']', "]]")),
_ => format!("\"{}\"", name.replace('"', "\"\"")),
@@ -15,7 +16,7 @@ pub fn quote_ident(driver: DbDriver, name: &str) -> String {
pub fn quote_literal(driver: DbDriver, value: &str) -> String {
let mut s = value.replace('\'', "''");
if driver == DbDriver::Mysql {
if matches!(driver, DbDriver::Mysql | DbDriver::Clickhouse) {
s = s.replace('\\', "\\\\");
}
format!("'{s}'")
@@ -48,6 +49,10 @@ pub const INTERNAL_DATABASES: &[&str] = &["rdsadmin", "azure_maintenance", "azur
pub fn schemas(driver: DbDriver, system: bool) -> &'static str {
match driver {
DbDriver::Sqlite => "SELECT name FROM pragma_database_list WHERE name <> 'temp' ORDER BY seq",
DbDriver::Clickhouse if system => "SELECT name FROM system.databases ORDER BY name",
DbDriver::Clickhouse => {
"SELECT name FROM system.databases WHERE lower(name) NOT IN ('system', 'information_schema') ORDER BY name"
}
DbDriver::Postgres if system => {
"SELECT schema_name FROM information_schema.schemata \
WHERE schema_name NOT LIKE 'pg\\_toast%' AND schema_name NOT LIKE 'pg\\_temp%' ORDER BY 1"
@@ -73,6 +78,12 @@ pub fn schemas(driver: DbDriver, system: bool) -> &'static str {
/// Rows: `name, kind` where kind is `table`, `view` or `materialized view`.
pub fn tables(driver: DbDriver, schema: &str) -> String {
match driver {
DbDriver::Clickhouse => format!(
"SELECT name, CASE engine WHEN 'View' THEN 'view' WHEN 'MaterializedView' THEN 'materialized view' \
WHEN 'WindowView' THEN 'view' ELSE 'table' END FROM system.tables \
WHERE database = {} AND is_temporary = 0 ORDER BY name",
quote_literal(driver, schema)
),
DbDriver::Sqlite => format!(
"SELECT name, type FROM {}.sqlite_master WHERE type IN ('table', 'view') \
AND name NOT LIKE 'sqlite\\_%' ESCAPE '\\' ORDER BY name",
@@ -101,6 +112,12 @@ pub fn tables(driver: DbDriver, schema: &str) -> String {
pub fn columns(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
match driver {
DbDriver::Clickhouse => format!(
"SELECT name, type, CASE WHEN startsWith(type, 'Nullable(') \
OR startsWith(type, 'LowCardinality(Nullable(') THEN 'YES' ELSE 'NO' END, \
if(default_kind = '', NULL, default_expression), is_in_primary_key \
FROM system.columns WHERE database = {s} AND table = {t} ORDER BY position"
),
DbDriver::Sqlite => format!(
"SELECT name, type, CASE WHEN \"notnull\" = 0 THEN 'YES' ELSE 'NO' END, dflt_value, \
CASE WHEN pk > 0 THEN 1 ELSE 0 END FROM pragma_table_info({t}, {s}) ORDER BY cid"
@@ -138,6 +155,12 @@ pub fn columns(driver: DbDriver, schema: &str, table: &str) -> String {
pub fn indexes(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
match driver {
DbDriver::Clickhouse => format!(
"SELECT * FROM (SELECT 'PRIMARY KEY' AS index_name, 0 AS is_unique, primary_key AS expression \
FROM system.tables WHERE database = {s} AND name = {t} AND primary_key != '' \
UNION ALL SELECT name, 0, expr FROM system.data_skipping_indices \
WHERE database = {s} AND table = {t}) ORDER BY index_name"
),
DbDriver::Sqlite => format!(
"SELECT il.name, il.\"unique\", ii.name FROM pragma_index_list({t}, {s}) il \
JOIN pragma_index_info(il.name, {s}) ii ORDER BY il.name, ii.seqno"
@@ -169,6 +192,15 @@ pub fn indexes(driver: DbDriver, schema: &str, table: &str) -> String {
/// for SQL completion, grouped by table in column order.
pub fn completion_columns(driver: DbDriver) -> &'static str {
match driver {
DbDriver::Clickhouse => {
"SELECT c.database, c.table, CASE t.engine WHEN 'View' THEN 'view' \
WHEN 'MaterializedView' THEN 'materialized view' WHEN 'WindowView' THEN 'view' ELSE 'table' END, \
c.name, c.type, CASE WHEN startsWith(c.type, 'Nullable(') \
OR startsWith(c.type, 'LowCardinality(Nullable(') THEN 'YES' ELSE 'NO' END, c.is_in_primary_key, 1 \
FROM system.columns c JOIN system.tables t ON t.database = c.database AND t.name = c.table \
WHERE lower(c.database) NOT IN ('system', 'information_schema') AND t.is_temporary = 0 \
ORDER BY c.database, c.table, c.position LIMIT 50000"
}
DbDriver::Sqlite => {
"SELECT 'main', m.name, m.type, p.name, p.type, CASE WHEN p.\"notnull\" = 0 THEN 'YES' ELSE 'NO' END, \
CASE WHEN p.pk > 0 THEN 1 ELSE 0 END, \
@@ -221,6 +253,7 @@ pub fn completion_columns(driver: DbDriver) -> &'static str {
/// when the key references the primary key implicitly.
pub fn foreign_keys(driver: DbDriver) -> &'static str {
match driver {
DbDriver::Clickhouse => "SELECT '', '', '', '', '', '', '' WHERE 0",
DbDriver::Sqlite => {
"SELECT 'main', m.name, f.id, f.\"from\", 'main', f.\"table\", f.\"to\" \
FROM sqlite_master m JOIN pragma_foreign_key_list(m.name) f \
@@ -258,6 +291,7 @@ pub fn foreign_keys(driver: DbDriver) -> &'static str {
pub fn table_foreign_keys(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
match driver {
DbDriver::Clickhouse => foreign_keys(driver).to_owned(),
DbDriver::Sqlite => format!(
"SELECT {s}, {t}, f.id, f.\"from\", {s}, f.\"table\", f.\"to\" FROM pragma_foreign_key_list({t}, {s}) f ORDER BY f.id, f.seq"
),
@@ -276,6 +310,7 @@ pub fn table_foreign_keys(driver: DbDriver, schema: &str, table: &str) -> String
/// The schema unqualified names resolve to (`None`: always `main`).
pub fn current_schema(driver: DbDriver) -> Option<&'static str> {
match driver {
DbDriver::Clickhouse => Some("SELECT currentDatabase()"),
DbDriver::Sqlite => None,
DbDriver::Postgres => Some("SELECT current_schema()"),
DbDriver::Mysql => Some("SELECT DATABASE()"),
@@ -362,6 +397,9 @@ fn where_key(driver: DbDriver, key: &[CellValue]) -> String {
/// The DML statement for `edit`. Values are written as quoted literals, which
/// every supported database coerces to the column type.
pub fn edit_statement(driver: DbDriver, schema: &str, table: &str, edit: &DbRowEdit) -> Result<String, String> {
if driver == DbDriver::Clickhouse {
return Err("ClickHouse table editing is unsupported: primary keys do not uniquely identify rows and edits require transactions".into());
}
let target = qualified(driver, schema, table);
match edit {
DbRowEdit::Update { key, values } => {
@@ -450,6 +488,7 @@ pub fn postgres_ddl_indexes(oid: u32) -> String {
pub fn triggers(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
match driver {
DbDriver::Clickhouse => "SELECT '', '', '' WHERE 0".to_owned(),
DbDriver::Sqlite => format!(
"SELECT name, sql FROM {}.sqlite_master WHERE type = 'trigger' AND tbl_name = {t} ORDER BY name",
quote_ident(driver, schema)
@@ -492,7 +531,7 @@ pub fn sqlite_trigger_event(sql: &str) -> (String, String) {
pub fn routines(driver: DbDriver, schema: &str) -> Option<String> {
let s = quote_literal(driver, schema);
match driver {
DbDriver::Sqlite => None,
DbDriver::Sqlite | DbDriver::Clickhouse => None,
// Functions of extensions are left out, as in IDEA.
DbDriver::Postgres => Some(format!(
"SELECT p.proname, CASE WHEN p.prokind = 'p' THEN 'procedure' ELSE 'function' END, \
@@ -625,6 +664,71 @@ mod tests {
assert_eq!(quote_literal(DbDriver::Mysql, "it's \\"), "'it''s \\\\'");
}
#[test]
fn clickhouse_escapes_backslashes_and_delimiters() {
let driver = DbDriver::Clickhouse;
assert_eq!(quote_literal(driver, r"path\O'Brien"), r"'path\\O''Brien'");
assert_eq!(quote_literal(driver, r"\' OR 1 = 1 --"), r"'\\'' OR 1 = 1 --'");
assert_eq!(quote_ident(driver, r"a\`b"), r"`a\\\`b`");
assert_eq!(qualified(driver, "data base", "event`log"), r"`data base`.`event\`log`");
}
#[test]
fn clickhouse_metadata_preserves_types_defaults_and_nonunique_keys() {
let driver = DbDriver::Clickhouse;
assert_eq!(schemas(driver, true), "SELECT name FROM system.databases ORDER BY name");
assert!(schemas(driver, false).contains("lower(name) NOT IN ('system', 'information_schema')"));
assert_eq!(current_schema(driver), Some("SELECT currentDatabase()"));
let relations = tables(driver, "event's\\db");
assert!(relations.contains("WHEN 'MaterializedView' THEN 'materialized view'"));
assert!(relations.contains(r"WHERE database = 'event''s\\db' AND is_temporary = 0"));
let fields = columns(driver, "default", "events");
assert!(fields.starts_with("SELECT name, type, CASE WHEN startsWith(type, 'Nullable(')"));
assert!(fields.contains("startsWith(type, 'LowCardinality(Nullable(')"));
assert!(fields.contains("if(default_kind = '', NULL, default_expression), is_in_primary_key"));
assert!(fields.ends_with("WHERE database = 'default' AND table = 'events' ORDER BY position"));
let keys = indexes(driver, "default", "events");
assert!(keys.contains("'PRIMARY KEY' AS index_name, 0 AS is_unique, primary_key AS expression"));
assert!(keys.contains("SELECT name, 0, expr FROM system.data_skipping_indices"));
let completion = completion_columns(driver);
assert!(completion.contains("c.is_in_primary_key, 1"));
assert!(completion.contains("ORDER BY c.database, c.table, c.position LIMIT 50000"));
}
#[test]
fn clickhouse_has_no_foreign_key_trigger_or_schema_routine_catalogs() {
let driver = DbDriver::Clickhouse;
assert_eq!(foreign_keys(driver), "SELECT '', '', '', '', '', '', '' WHERE 0");
assert_eq!(table_foreign_keys(driver, "default", "events"), foreign_keys(driver));
assert_eq!(triggers(driver, "default", "events"), "SELECT '', '', '' WHERE 0");
assert!(routines(driver, "default").is_none());
assert!(sequences(driver, "default").is_none());
assert!(object_ddl(driver, "default", crate::DbObjectKind::Trigger, "t").is_none());
}
#[test]
fn clickhouse_pages_quote_keys_and_fetch_one_extra_row() {
let sort = [SortKey { column: "event`id".into(), descending: true }];
assert_eq!(
table_page(DbDriver::Clickhouse, "default", "events", Some("WHERE id > 3"), &sort, 50, 25),
r"SELECT * FROM `default`.`events` WHERE id > 3 ORDER BY `event\`id` DESC LIMIT 26 OFFSET 50"
);
}
#[test]
fn clickhouse_keys_never_enable_transactional_table_edits() {
let key = vec![cv("id", Some("1"))];
for edit in [
DbRowEdit::Update { key: key.clone(), values: vec![cv("name", Some("new"))] },
DbRowEdit::Delete { key },
DbRowEdit::Insert { values: vec![cv("name", Some("new"))] },
DbRowEdit::Insert { values: vec![] },
] {
let error = edit_statement(DbDriver::Clickhouse, "default", "events", &edit).unwrap_err();
assert!(error.contains("primary keys do not uniquely identify rows"));
}
}
#[test]
fn pages() {
let sort = [SortKey { column: "id".into(), descending: true }];
+136
View File
@@ -0,0 +1,136 @@
use ide_core::{CancellationToken, IdeError, IdeResult};
use ide_db::{CellValue, DataSource, DbDriver, DbManager, DbRowEdit, DbSslMode, SortKey, sql};
use std::sync::Arc;
use std::time::{SystemTime, UNIX_EPOCH};
fn check(condition: bool, message: &str) -> IdeResult<()> {
if condition { Ok(()) } else { Err(IdeError::invalid(message)) }
}
#[tokio::test]
#[ignore = "requires CLICKHOUSE_TEST_HOST, CLICKHOUSE_TEST_USER, and CLICKHOUSE_TEST_PASSWORD"]
async fn clickhouse_cloud_manager_catalogs_and_session_table() -> IdeResult<()> {
let tmp = tempfile::tempdir()?;
let root = tmp.path().to_path_buf();
let ssh = Arc::new(ide_ssh::SshManager::new(root.join("ssh-passwords.json")));
let db = DbManager::new(root.clone(), root.join("passwords.json"), root.join("history.json"), ssh);
let mut source = DataSource {
id: "clickhouse".into(),
name: "clickhouse".into(),
driver: DbDriver::Clickhouse,
host: std::env::var("CLICKHOUSE_TEST_HOST").expect("set CLICKHOUSE_TEST_HOST"),
port: Some(
std::env::var("CLICKHOUSE_TEST_PORT")
.map(|value| value.parse().expect("numeric CLICKHOUSE_TEST_PORT"))
.unwrap_or(8443),
),
user: std::env::var("CLICKHOUSE_TEST_USER").expect("set CLICKHOUSE_TEST_USER"),
database: "default".into(),
ssl: DbSslMode::VerifyFull,
..Default::default()
};
source.introspection.show_system = true;
let password = std::env::var("CLICKHOUSE_TEST_PASSWORD").expect("set CLICKHOUSE_TEST_PASSWORD");
check(!db.test(&source, Some(password.clone()), None).await?.is_empty(), "missing server version")?;
db.save_sources(vec![source]).await?;
db.set_password("clickhouse", Some(password), false).await?;
let cancel = CancellationToken::new();
let suffix = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_nanos();
let name = format!("manager_test_{}_{suffix}", std::process::id());
let table = sql::quote_ident(DbDriver::Clickhouse, &name);
db.execute(
"clickhouse",
&format!(
"CREATE TEMPORARY TABLE {table} (id UInt64, label Nullable(String) DEFAULT NULL, \
category LowCardinality(Nullable(String)) DEFAULT NULL, items Array(Nullable(UInt8)), \
label_size UInt64 ALIAS length(ifNull(label, ''))) ENGINE = Memory"
),
0,
&cancel,
)
.await?;
let checked: IdeResult<()> = async {
db.execute(
"clickhouse",
&format!("INSERT INTO {table} (id, label, items) VALUES (1, 'first', [1, NULL]), (1, NULL, [])"),
0,
&cancel,
)
.await?;
let schemas = db.schemas("clickhouse").await?;
check(schemas.iter().any(|schema| schema == "default"), "default database missing")?;
check(schemas.iter().any(|schema| schema == "system"), "system database missing")?;
let tables = db.tables("clickhouse", "system").await?;
check(tables.iter().any(|table| table.name == "one"), "system table listing missing one")?;
let details = db.table_details("clickhouse", "", &name).await?;
check(details.columns.len() == 5, "temporary table columns missing")?;
for (name, data_type, nullable) in [
("id", "UInt64", false),
("label", "Nullable(String)", true),
("category", "LowCardinality(Nullable(String))", true),
("items", "Array(Nullable(UInt8))", false),
] {
check(
details
.columns
.iter()
.any(|column| column.name == name && column.data_type == data_type && column.nullable == nullable),
"column type or nullability mismatch",
)?;
}
check(
details.columns.iter().any(|column| column.name == "label" && column.default.as_deref() == Some("NULL")),
"explicit default expression missing",
)?;
check(details.indexes.iter().all(|index| !index.unique), "ClickHouse index reported unique")?;
check(details.foreign_keys.is_empty() && details.triggers.is_empty(), "unsupported table objects returned")?;
let sort = [SortKey { column: "id".into(), descending: false }];
let first = db.table_page("clickhouse", "", &name, None, &sort, 0, 1, &cancel).await?;
check(first.result.rows.len() == 1 && first.result.has_more, "first page limit failed")?;
let second = db.table_page("clickhouse", "", &name, None, &sort, 1, 1, &cancel).await?;
check(second.result.rows.len() == 1 && !second.result.has_more, "page offset failed")?;
let ddl = db.ddl("clickhouse", "", &name).await?;
check(ddl.contains(&name) && ddl.contains("Memory"), "temporary table DDL missing")?;
let completion = db.completion("clickhouse").await?;
check(completion.default_schema == "default", "completion default database mismatch")?;
check(completion.foreign_keys.is_empty(), "completion returned unsupported foreign keys")?;
if let Some(table) = completion.tables.first() {
let details = db.table_details("clickhouse", &table.schema, &table.name).await?;
check(
table.columns.iter().all(|column| {
details.columns.iter().any(|field| field.name == column.name && field.data_type == column.data_type)
}),
"completion columns differ from catalog",
)?;
}
let plan = db.explain("clickhouse", &format!("SELECT id FROM {table}"), &cancel).await?;
check(!plan.is_empty(), "query plan missing")?;
let value = r"path\O'Brien";
let result = db
.execute("clickhouse", &format!("SELECT {}", sql::quote_literal(DbDriver::Clickhouse, value)), 1, &cancel)
.await?;
check(result.rows == vec![vec![Some(value.to_owned())]], "quoted literal changed value")?;
let edits = [DbRowEdit::Update {
key: vec![CellValue { column: "id".into(), value: Some("1".into()) }],
values: vec![CellValue { column: "label".into(), value: Some("changed".into()) }],
}];
check(
db.apply_edits("clickhouse", "", &name, &edits)
.await
.err()
.is_some_and(|error| error.message.contains("table editing is unsupported")),
"supplied row key authorized table editing",
)?;
check(db.set_manual_commit("clickhouse", true).await.is_err(), "manual transactions were accepted")?;
for commit in [false, true] {
check(db.finish_transaction("clickhouse", commit).await.is_err(), "transaction control was accepted")?;
}
Ok(())
}
.await;
let cleanup = db.execute("clickhouse", &format!("DROP TEMPORARY TABLE {table}"), 0, &cancel).await;
db.disconnect("clickhouse").await;
checked?;
cleanup?;
Ok(())
}
+3 -3
View File
@@ -197,7 +197,7 @@ async fn count(db: &DbManager) -> String {
async fn manual_commit_keeps_a_transaction_until_commit_or_rollback() -> IdeResult<()> {
let (_tmp, db) = setup().await;
let c = CancellationToken::new();
db.set_manual_commit("s1", true).await;
db.set_manual_commit("s1", true).await.unwrap();
db.execute("s1", "INSERT INTO users (name) VALUES ('dee')", 1, &c).await?;
// The session sees its own change.
assert_eq!(count(&db).await, "4");
@@ -208,10 +208,10 @@ async fn manual_commit_keeps_a_transaction_until_commit_or_rollback() -> IdeResu
db.finish_transaction("s1", true).await?;
// Leaving manual mode rolls back what is still open.
db.execute("s1", "DELETE FROM users", 1, &c).await?;
db.set_manual_commit("s1", false).await;
db.set_manual_commit("s1", false).await.unwrap();
assert_eq!(count(&db).await, "4");
// Disconnecting with an open transaction does not hang.
db.set_manual_commit("s1", true).await;
db.set_manual_commit("s1", true).await.unwrap();
db.execute("s1", "DELETE FROM users", 1, &c).await?;
db.disconnect("s1").await;
assert_eq!(count(&db).await, "4");
+24
View File
@@ -0,0 +1,24 @@
[package]
name = "ide-extension"
description = "Plugin manifest, extension points, contribution registry and capability model (no Wasm runtime)"
version.workspace = true
edition.workspace = true
license.workspace = true
[dependencies]
ide-core.workspace = true
serde.workspace = true
serde_json.workspace = true
specta.workspace = true
specta-typescript.workspace = true
toml.workspace = true
tracing.workspace = true
parking_lot.workspace = true
semver = "1"
blake3.workspace = true
zip.workspace = true
base64.workspace = true
ed25519-dalek = "3"
[dev-dependencies]
tempfile.workspace = true
+277
View File
@@ -0,0 +1,277 @@
//! What a plugin may do beyond pure computation, and how risky each grant is.
//!
//! Capabilities are written as strings in the manifest (`project.read`,
//! `net:crates.io`, `process:cargo`) and stored the same way in the grant
//! ledger. The 0.1 permission names (`project-read`, …) parse as their 0.2
//! equivalents.
use serde::{Deserialize, Deserializer, Serialize, Serializer};
use specta::Type;
use std::fmt;
use std::str::FromStr;
#[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum Capability {
/// Read files inside the project root; the project root itself.
ProjectRead,
/// The focused editor's file, selection and text.
EditorRead,
/// Edit the focused editor (versioned edits).
EditorWrite,
VcsRead,
VcsWrite,
/// The plugin's own key-value store and data directory.
Storage,
/// The plugin's own secrets (kept in the client's OS keychain).
Secrets,
/// Workbench events (file opened, saved, focus…), not keystrokes.
Events,
/// Keystrokes and editor input events.
InputEvents,
/// The system clipboard, from the plugin's web views.
Clipboard,
/// HTTP(S) to one host (`*.example.com` matches subdomains).
Net(String),
/// Run one external program. Full code execution: never fine-grained.
Process(String),
/// Read files outside the project under one path.
FsRead(String),
}
/// How a grant is presented (plan 4.2).
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum Risk {
/// Listed on the install page, one confirmation.
Low,
/// Listed separately with its purpose.
Medium,
/// Separate dialog: "equivalent to running arbitrary code on this machine".
High,
}
impl Capability {
pub fn risk(&self) -> Risk {
match self {
Self::ProjectRead | Self::EditorRead | Self::Storage | Self::Events | Self::VcsRead => Risk::Low,
Self::EditorWrite | Self::VcsWrite | Self::Net(_) | Self::Secrets | Self::InputEvents | Self::Clipboard => {
Risk::Medium
}
Self::Process(_) | Self::FsRead(_) => Risk::High,
}
}
/// Low-risk capabilities that the one-time migration of the 0.1 "declared
/// means granted" behaviour may grant automatically (plan 4.6).
pub fn auto_migrates(&self) -> bool {
self.risk() == Risk::Low
}
/// Short user-facing description for install and grant pages.
pub fn describe(&self) -> String {
match self {
Self::ProjectRead => "Read files in the open project".into(),
Self::EditorRead => "Read the focused editor's file and selection".into(),
Self::EditorWrite => "Change text in the editor".into(),
Self::VcsRead => "Read version control state".into(),
Self::VcsWrite => "Change version control state (commit, fetch, checkout)".into(),
Self::Storage => "Store its own data".into(),
Self::Secrets => "Store secrets in the system keychain".into(),
Self::Events => "Observe workbench events (files opened, saved, focus)".into(),
Self::InputEvents => "Observe keystrokes and editor input".into(),
Self::Clipboard => "Read and write the clipboard from its web views".into(),
Self::Net(host) => format!("Connect to {host} (from the machine the project is on)"),
Self::Process(bin) => format!("Run {bin} — equivalent to running arbitrary code on this machine"),
Self::FsRead(path) => format!("Read files under {path}, outside the project"),
}
}
/// Whether `self`, granted, also covers `wanted` (`net:*.x.com` covers `net:a.x.com`).
pub fn covers(&self, wanted: &Capability) -> bool {
match (self, wanted) {
(Self::Net(pattern), Self::Net(host)) => host_matches(pattern, host),
(Self::FsRead(base), Self::FsRead(path)) => {
std::path::Path::new(path).starts_with(std::path::Path::new(base))
}
_ => self == wanted,
}
}
}
/// `example.com` matches exactly; `*.example.com` matches any subdomain (not the apex).
pub fn host_matches(pattern: &str, host: &str) -> bool {
let host = host.trim_end_matches('.').to_ascii_lowercase();
let pattern = pattern.to_ascii_lowercase();
match pattern.strip_prefix("*.") {
Some(suffix) => host.len() > suffix.len() + 1 && host.ends_with(&format!(".{suffix}")),
None => host == pattern,
}
}
impl fmt::Display for Capability {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::ProjectRead => f.write_str("project.read"),
Self::EditorRead => f.write_str("editor.read"),
Self::EditorWrite => f.write_str("editor.write"),
Self::VcsRead => f.write_str("vcs.read"),
Self::VcsWrite => f.write_str("vcs.write"),
Self::Storage => f.write_str("storage"),
Self::Secrets => f.write_str("secrets"),
Self::Events => f.write_str("events"),
Self::InputEvents => f.write_str("events.input"),
Self::Clipboard => f.write_str("clipboard"),
Self::Net(h) => write!(f, "net:{h}"),
Self::Process(b) => write!(f, "process:{b}"),
Self::FsRead(p) => write!(f, "fs.read:{p}"),
}
}
}
impl FromStr for Capability {
type Err = String;
fn from_str(s: &str) -> Result<Self, String> {
let s = s.trim();
if let Some((kind, arg)) = s.split_once(':') {
let arg = arg.trim();
if arg.is_empty() {
return Err(format!("capability {s:?} needs a value after ':'"));
}
return match kind {
"net" => {
let host = arg.strip_prefix("*.").unwrap_or(arg);
let valid = !host.is_empty()
&& host.len() <= 253
&& host.split('.').all(|label| {
!label.is_empty()
&& label.len() <= 63
&& label.bytes().all(|b| b.is_ascii_alphanumeric() || b == b'-')
});
if valid { Ok(Self::Net(arg.to_ascii_lowercase())) } else { Err(format!("invalid host in {s:?}")) }
}
"process" => {
if arg.contains(['/', '\\']) && !std::path::Path::new(arg).is_absolute() {
return Err(format!("{s:?}: use a program name or an absolute path"));
}
Ok(Self::Process(arg.to_owned()))
}
"fs.read" => {
if !std::path::Path::new(arg).is_absolute() && !arg.starts_with('~') {
return Err(format!("{s:?}: use an absolute path (or ~/…)"));
}
Ok(Self::FsRead(arg.to_owned()))
}
_ => Err(format!("unknown capability {s:?}")),
};
}
Ok(match s {
"project.read" | "project-read" => Self::ProjectRead,
"editor.read" | "editor-read" => Self::EditorRead,
"editor.write" | "editor-write" => Self::EditorWrite,
"vcs.read" => Self::VcsRead,
"vcs.write" => Self::VcsWrite,
"storage" => Self::Storage,
"secrets" => Self::Secrets,
"events" => Self::Events,
"events.input" => Self::InputEvents,
"clipboard" => Self::Clipboard,
_ => return Err(format!("unknown capability {s:?}")),
})
}
}
impl Serialize for Capability {
fn serialize<S: Serializer>(&self, s: S) -> Result<S::Ok, S::Error> {
s.collect_str(self)
}
}
impl<'de> Deserialize<'de> for Capability {
fn deserialize<D: Deserializer<'de>>(d: D) -> Result<Self, D::Error> {
String::deserialize(d)?.parse().map_err(serde::de::Error::custom)
}
}
/// A capability as the UI shows it.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct CapabilityInfo {
/// `project.read`, `net:crates.io`, …
pub id: String,
pub risk: Risk,
pub description: String,
/// Declared under `optional`: the plugin runs without it.
pub optional: bool,
/// Currently granted.
pub granted: bool,
}
impl CapabilityInfo {
pub fn new(cap: &Capability, optional: bool, granted: bool) -> Self {
Self { id: cap.to_string(), risk: cap.risk(), description: cap.describe(), optional, granted }
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parses_and_prints() {
for s in [
"project.read",
"editor.write",
"vcs.write",
"storage",
"secrets",
"events",
"events.input",
"net:crates.io",
"net:*.github.com",
"process:cargo",
] {
assert_eq!(s.parse::<Capability>().unwrap().to_string(), s);
}
// Absolute paths are the platform's.
let (git, etc) = if cfg!(windows) {
(r"process:C:\Program Files\Git\bin\git.exe", r"fs.read:C:\Windows")
} else {
("process:/usr/bin/git", "fs.read:/etc")
};
for s in [git, etc] {
assert_eq!(s.parse::<Capability>().unwrap().to_string(), s);
}
assert_eq!("project-read".parse::<Capability>().unwrap(), Capability::ProjectRead);
assert!("net:".parse::<Capability>().is_err());
assert!("net:a b".parse::<Capability>().is_err());
assert!("process:./evil".parse::<Capability>().is_err());
assert!("fs.read:rel".parse::<Capability>().is_err());
assert!("network".parse::<Capability>().is_err());
}
#[test]
fn risk_levels() {
assert_eq!(Capability::ProjectRead.risk(), Risk::Low);
assert_eq!(Capability::Net("x".into()).risk(), Risk::Medium);
assert_eq!(Capability::InputEvents.risk(), Risk::Medium);
assert_eq!("clipboard".parse::<Capability>().unwrap(), Capability::Clipboard);
assert_eq!(Capability::Clipboard.risk(), Risk::Medium);
assert_eq!(Capability::Process("cargo".into()).risk(), Risk::High);
assert!(Capability::Storage.auto_migrates() && !Capability::EditorWrite.auto_migrates());
}
#[test]
fn host_patterns() {
assert!(host_matches("crates.io", "crates.io"));
assert!(host_matches("crates.io", "CRATES.IO."));
assert!(!host_matches("crates.io", "static.crates.io"));
assert!(host_matches("*.github.com", "api.github.com"));
assert!(!host_matches("*.github.com", "github.com"));
assert!(!host_matches("*.github.com", "evilgithub.com"));
let any_gh: Capability = "net:*.github.com".parse().unwrap();
assert!(any_gh.covers(&"net:api.github.com".parse().unwrap()));
assert!(Capability::FsRead("/etc".into()).covers(&Capability::FsRead("/etc/hosts".into())));
assert!(!Capability::FsRead("/etc".into()).covers(&Capability::FsRead("/etcx".into())));
}
}
+104
View File
@@ -0,0 +1,104 @@
//! The contribution index: validated UI
//! declarations of installed packages, kept on disk so start-up does not
//! validate them again (sanitising thousands of icon SVGs, say).
//!
//! Only packages with a [`Package::digest`] are indexed: installed copies,
//! which the host never modifies after install. An entry is keyed by that
//! digest, the extension point, the declaration and a salt naming the
//! validation logic (the IDE version), so a new package version, a changed
//! declaration or an IDE upgrade simply misses. A miss or an unreadable entry
//! only costs the validation.
use crate::point::Package;
use crate::ui::UiDecl;
use std::collections::BTreeSet;
use std::path::PathBuf;
/// Bumped when [`UiDecl`]'s serialized shape changes.
const FORMAT: u32 = 1;
pub struct ContributionIndex {
dir: PathBuf,
salt: String,
}
impl ContributionIndex {
/// `salt` names the validators' version (the IDE version).
pub fn new(dir: PathBuf, salt: &str) -> Self {
Self { dir, salt: format!("{FORMAT}:{salt}") }
}
/// Entries start with a prefix of the package digest, for pruning.
fn file(&self, digest: &str, point: &str, decl: &toml::Value) -> PathBuf {
let mut h = blake3::Hasher::new();
for part in [self.salt.as_str(), digest, point, &decl.to_string()] {
h.update(part.as_bytes());
h.update(&[0]);
}
let key = h.finalize().to_hex();
self.dir.join(format!("{}-{}.json", prefix(digest), &key[..32]))
}
pub fn get(&self, pkg: &Package, point: &str, decl: &toml::Value) -> Option<UiDecl> {
let digest = pkg.digest.as_deref()?;
let bytes = std::fs::read(self.file(digest, point, decl)).ok()?;
serde_json::from_slice(&bytes).ok()
}
pub fn put(&self, pkg: &Package, point: &str, decl: &toml::Value, ui: &UiDecl) {
let Some(digest) = pkg.digest.as_deref() else { return };
let Ok(bytes) = serde_json::to_vec(ui) else { return };
let file = self.file(digest, point, decl);
let _ = std::fs::create_dir_all(&self.dir);
let tmp = file.with_extension(format!("tmp{}", std::process::id()));
if std::fs::write(&tmp, bytes).and_then(|()| std::fs::rename(&tmp, &file)).is_err() {
let _ = std::fs::remove_file(&tmp);
}
}
/// Removes entries of packages other than `live` (digests).
pub fn prune<'a>(&self, live: impl IntoIterator<Item = &'a str>) {
let keep: BTreeSet<String> = live.into_iter().map(|d| prefix(d).to_owned()).collect();
let Ok(entries) = std::fs::read_dir(&self.dir) else { return };
for e in entries.flatten() {
let name = e.file_name().to_string_lossy().into_owned();
if !name.split_once('-').is_some_and(|(p, _)| keep.contains(p)) {
let _ = std::fs::remove_file(e.path());
}
}
}
}
fn prefix(digest: &str) -> &str {
let d = digest.strip_prefix("blake3:").unwrap_or(digest);
let end = d.char_indices().nth(16).map_or(d.len(), |(i, _)| i);
&d[..end]
}
#[cfg(test)]
mod tests {
use super::*;
use crate::manifest::Manifest;
#[test]
fn hits_only_the_same_package_point_declaration_and_salt() {
let tmp = tempfile::tempdir().unwrap();
let index = ContributionIndex::new(tmp.path().join("idx"), "1.0");
let manifest =
Manifest::parse("id = \"p\"\nname = \"P\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"\n").unwrap();
let pkg = Package::new(manifest, tmp.path().to_path_buf()).with_digest("blake3:00aa11bb22cc33dd44ee");
let decl: toml::Value = toml::from_str("id = \"x\"").unwrap();
let ui = UiDecl::new("x", "X", serde_json::json!({ "id": "x" }));
assert!(index.get(&pkg, "pt", &decl).is_none());
index.put(&pkg, "pt", &decl, &ui);
assert_eq!(index.get(&pkg, "pt", &decl).unwrap().label, "X");
assert!(index.get(&pkg, "other", &decl).is_none());
assert!(index.get(&pkg, "pt", &toml::from_str("id = \"y\"").unwrap()).is_none());
assert!(ContributionIndex::new(tmp.path().join("idx"), "1.1").get(&pkg, "pt", &decl).is_none());
let unhashed = Package { digest: None, ..pkg.clone() };
assert!(index.get(&unhashed, "pt", &decl).is_none(), "not an installed copy");
index.prune(["blake3:ffff"]);
assert!(index.get(&pkg, "pt", &decl).is_none());
}
}
+483
View File
@@ -0,0 +1,483 @@
//! The grant ledger (plan 4.6): which capabilities the user granted to which
//! plugin, saved as `grants.json` next to the installed plugins.
//!
//! - Installed plugins are keyed by id and remember the version and package
//! hash they were confirmed for. An update that needs no new capability is
//! accepted silently; one that does must be confirmed again.
//! - Linked development plugins and project plugins (`.openidea/plugins/*`)
//! are keyed by their directory, not by hash, so rebuilding does not ask
//! again; new capabilities still do.
//! - `process:<program>` grants record the program's absolute path and file
//! hash at grant time. Calls use that path, never a `PATH` lookup, and fail
//! when the file changed (re-confirmation needed).
use crate::capability::{Capability, Risk};
use serde::{Deserialize, Serialize};
use std::collections::{BTreeMap, BTreeSet};
use std::ffi::OsStr;
use std::path::{Path, PathBuf};
pub const LEDGER_FILE: &str = "grants.json";
const FORMAT: u32 = 1;
/// Whose grant an entry is.
#[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
#[serde(tag = "kind", content = "value", rename_all = "camelCase")]
pub enum Subject {
/// Installed under the plugins directory, by plugin id.
Installed(String),
/// A linked development plugin or a project plugin, by package directory.
Path(PathBuf),
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct BinaryGrant {
pub path: PathBuf,
/// blake3 of the file when granted.
pub hash: String,
}
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Grant {
pub plugin_id: String,
/// The version and package hash the user confirmed (installed plugins).
#[serde(default)]
pub version: String,
#[serde(default)]
pub package_hash: String,
/// The user confirmed the plugin (the install page or the project-plugin prompt).
#[serde(default)]
pub confirmed: bool,
/// Granted capabilities (`project.read`, `net:x`, …).
#[serde(default)]
pub granted: BTreeSet<String>,
/// Optional capabilities the user declined (not asked again until re-confirmed).
#[serde(default)]
pub declined: BTreeSet<String>,
/// `process:<program>` → the program confirmed for it.
#[serde(default)]
pub binaries: BTreeMap<String, BinaryGrant>,
/// Migrated from 0.1: capabilities not granted yet are asked for on first
/// use instead of blocking the plugin from loading.
#[serde(default)]
pub ask_on_use: bool,
}
impl Grant {
pub fn has(&self, cap: &Capability) -> bool {
self.granted.iter().filter_map(|s| s.parse::<Capability>().ok()).any(|g| g.covers(cap))
}
}
/// What a plugin (in its current form) needs, to compare with its grant.
#[derive(Clone, Debug)]
pub struct Request<'a> {
pub subject: Subject,
pub plugin_id: &'a str,
pub version: &'a str,
pub package_hash: &'a str,
pub required: Vec<Capability>,
pub optional: Vec<Capability>,
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum Decision {
/// Load with these capabilities. `pending` required ones are asked for when
/// the plugin is first used (plugins migrated from 0.1 only).
Allowed { granted: Vec<Capability>, pending: Vec<Capability> },
/// Ask the user first. `new` lists what was never granted (all of it on first use).
NeedsConfirmation { new: Vec<Capability>, first_time: bool },
}
#[derive(Debug, Default, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
struct File {
#[serde(default)]
format: u32,
/// The one-time migration from 0.1 ("declared means granted") ran.
#[serde(default)]
migrated: bool,
#[serde(default)]
grants: Vec<(Subject, Grant)>,
}
/// The ledger, loaded into memory; [`Ledger::save`] writes it atomically.
#[derive(Debug, Default)]
pub struct Ledger {
path: Option<PathBuf>,
migrated: bool,
grants: BTreeMap<Subject, Grant>,
}
impl Ledger {
/// Loads `<dir>/grants.json`; a missing or unreadable file gives an empty
/// ledger with `needs_migration()` true.
pub fn load(dir: &Path) -> Self {
let path = dir.join(LEDGER_FILE);
let file: Option<File> = std::fs::read_to_string(&path).ok().and_then(|s| match serde_json::from_str(&s) {
Ok(f) => Some(f),
Err(e) => {
tracing::warn!("{}: {e}; starting with an empty grant ledger", path.display());
None
}
});
let existed = file.is_some();
let file = file.unwrap_or_default();
Self { path: Some(path), migrated: file.migrated || existed, grants: file.grants.into_iter().collect() }
}
pub fn in_memory() -> Self {
Self { path: None, migrated: true, grants: BTreeMap::new() }
}
pub fn save(&self) -> std::io::Result<()> {
let Some(path) = &self.path else { return Ok(()) };
let file = File {
format: FORMAT,
migrated: self.migrated,
grants: self.grants.iter().map(|(s, g)| (s.clone(), g.clone())).collect(),
};
let json = serde_json::to_string_pretty(&file).map_err(std::io::Error::other)?;
let tmp = path.with_extension("json.tmp");
std::fs::write(&tmp, json)?;
std::fs::rename(&tmp, path)
}
pub fn get(&self, subject: &Subject) -> Option<&Grant> {
self.grants.get(subject)
}
pub fn remove(&mut self, subject: &Subject) -> Option<Grant> {
self.grants.remove(subject)
}
/// Puts back an earlier state of `subject`'s grant (`None`: none), e.g.
/// after rolling back an update.
pub fn restore(&mut self, subject: &Subject, grant: Option<Grant>) {
match grant {
Some(g) => {
self.grants.insert(subject.clone(), g);
}
None => {
self.grants.remove(subject);
}
}
}
/// The one-time migration of 0.1's "declared means granted" has not run yet.
pub fn needs_migration(&self) -> bool {
!self.migrated
}
/// Migrates plugins installed before the ledger existed: low-risk
/// capabilities are granted, the rest are asked on first use.
pub fn migrate(&mut self, installed: &[Request<'_>]) {
for req in installed {
let granted: BTreeSet<String> =
req.required.iter().chain(&req.optional).filter(|c| c.auto_migrates()).map(|c| c.to_string()).collect();
self.grants.entry(req.subject.clone()).or_insert_with(|| Grant {
plugin_id: req.plugin_id.to_owned(),
version: req.version.to_owned(),
package_hash: req.package_hash.to_owned(),
confirmed: true,
granted,
ask_on_use: true,
..Grant::default()
});
}
self.migrated = true;
}
/// Compares what the plugin needs with what was granted.
pub fn decide(&mut self, req: &Request<'_>) -> Decision {
let Some(grant) = self.grants.get_mut(&req.subject) else {
return Decision::NeedsConfirmation { new: req.required.clone(), first_time: true };
};
if !grant.confirmed {
return Decision::NeedsConfirmation { new: req.required.clone(), first_time: true };
}
let new: Vec<Capability> = req.required.iter().filter(|c| !grant.has(c)).cloned().collect();
if !new.is_empty() && !grant.ask_on_use {
return Decision::NeedsConfirmation { new, first_time: false };
}
// Same or fewer needs: accept an update silently (plan 4.6).
if grant.version != req.version || grant.package_hash != req.package_hash {
grant.version = req.version.to_owned();
grant.package_hash = req.package_hash.to_owned();
}
let granted = req.required.iter().chain(&req.optional).filter(|c| grant.has(c)).cloned().collect();
Decision::Allowed { granted, pending: new }
}
/// Records the user's confirmation: `required` plus the accepted optional capabilities.
pub fn confirm(
&mut self,
req: &Request<'_>,
accepted_optional: &[Capability],
binaries: BTreeMap<String, BinaryGrant>,
) {
let grant = self.grants.entry(req.subject.clone()).or_default();
grant.plugin_id = req.plugin_id.to_owned();
grant.version = req.version.to_owned();
grant.package_hash = req.package_hash.to_owned();
grant.confirmed = true;
grant.ask_on_use = false;
for cap in req.required.iter().chain(accepted_optional) {
grant.granted.insert(cap.to_string());
}
for cap in &req.optional {
let s = cap.to_string();
if accepted_optional.contains(cap) {
grant.declined.remove(&s);
} else if !grant.granted.contains(&s) {
grant.declined.insert(s);
}
}
grant.binaries.extend(binaries);
}
/// Grants or revokes one capability (Settings › Plugins › permissions).
pub fn set(&mut self, subject: &Subject, cap: &Capability, granted: bool) -> bool {
let Some(grant) = self.grants.get_mut(subject) else { return false };
let s = cap.to_string();
if granted {
grant.declined.remove(&s);
grant.granted.insert(s)
} else {
if let Capability::Process(_) = cap {
grant.binaries.remove(&s);
}
grant.granted.remove(&s)
}
}
/// Records the program approved for a `process:` grant.
pub fn set_binary(&mut self, subject: &Subject, cap: &Capability, binary: BinaryGrant) {
if let Some(grant) = self.grants.get_mut(subject) {
grant.binaries.insert(cap.to_string(), binary);
}
}
/// Checks a `process:` grant at call time: the program confirmed for it,
/// unchanged since. Errors name what to do.
pub fn check_binary(&self, subject: &Subject, cap: &Capability) -> Result<PathBuf, String> {
let Capability::Process(name) = cap else { return Err(format!("{cap} is not a process capability")) };
let grant = self.grants.get(subject).ok_or_else(|| format!("{cap} was not granted"))?;
let key = cap.to_string();
if !grant.granted.contains(&key) {
return Err(format!("{cap} was not granted"));
}
let bin = grant.binaries.get(&key).ok_or_else(|| format!("{cap}: no program was confirmed for {name}"))?;
match file_hash(&bin.path) {
Ok(hash) if hash == bin.hash => Ok(bin.path.clone()),
Ok(_) => Err(format!(
"{} changed since it was approved for {cap}; confirm the plugin's permissions again",
bin.path.display()
)),
Err(e) => Err(format!("{}: {e}; confirm the plugin's permissions again", bin.path.display())),
}
}
/// The program approved for a granted `process:` capability, without
/// checking the file: whoever starts it verifies the hash then
/// ([`check_binary`](Self::check_binary) or the pin's own check).
pub fn pinned_binary(&self, subject: &Subject, cap: &Capability) -> Option<BinaryGrant> {
let grant = self.grants.get(subject)?;
let key = cap.to_string();
if !grant.granted.contains(&key) {
return None;
}
grant.binaries.get(&key).cloned()
}
pub fn subjects(&self) -> impl Iterator<Item = (&Subject, &Grant)> {
self.grants.iter()
}
}
/// What a confirmation page shows: every capability with its risk, high risk first.
pub fn by_risk(caps: &[Capability]) -> Vec<(Risk, Vec<Capability>)> {
let mut groups: BTreeMap<std::cmp::Reverse<Risk>, Vec<Capability>> = BTreeMap::new();
for c in caps {
groups.entry(std::cmp::Reverse(c.risk())).or_default().push(c.clone());
}
groups.into_iter().map(|(r, v)| (r.0, v)).collect()
}
/// Resolves a `process:` program to an absolute path once, at grant time:
/// absolute names are taken as is, others are looked up in `path_var`.
pub fn resolve_program(name: &str, path_var: Option<&OsStr>) -> Result<PathBuf, String> {
let candidate = Path::new(name);
if candidate.is_absolute() {
return if candidate.is_file() { Ok(candidate.to_path_buf()) } else { Err(format!("{name}: no such program")) };
}
let exts: Vec<String> = if cfg!(windows) {
std::env::var("PATHEXT").unwrap_or_else(|_| ".EXE;.CMD;.BAT".into()).split(';').map(str::to_owned).collect()
} else {
vec![String::new()]
};
for dir in std::env::split_paths(path_var.unwrap_or_default()) {
if !dir.is_absolute() {
continue; // `.` or relative entries could point into the project
}
for ext in &exts {
let p = dir.join(format!("{name}{ext}"));
if is_executable(&p) {
return Ok(dunce_canonical(&p));
}
}
}
Err(format!("{name} was not found on PATH"))
}
fn dunce_canonical(p: &Path) -> PathBuf {
std::fs::canonicalize(p).unwrap_or_else(|_| p.to_path_buf())
}
fn is_executable(p: &Path) -> bool {
let Ok(meta) = std::fs::metadata(p) else { return false };
if !meta.is_file() {
return false;
}
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
meta.permissions().mode() & 0o111 != 0
}
#[cfg(not(unix))]
true
}
/// blake3 of a file, hex.
pub fn file_hash(path: &Path) -> std::io::Result<String> {
let mut hasher = blake3::Hasher::new();
hasher.update_reader(std::fs::File::open(path)?)?;
Ok(hasher.finalize().to_hex().to_string())
}
/// blake3 over a package's files (sorted relative paths and contents).
pub fn package_hash(dir: &Path, files: &[PathBuf]) -> std::io::Result<String> {
let mut sorted: Vec<&PathBuf> = files.iter().collect();
sorted.sort();
sorted.dedup();
let mut hasher = blake3::Hasher::new();
for rel in sorted {
hasher.update(rel.to_string_lossy().as_bytes());
hasher.update(&[0]);
hasher.update_reader(std::fs::File::open(dir.join(rel))?)?;
hasher.update(&[0]);
}
Ok(hasher.finalize().to_hex().to_string())
}
#[cfg(test)]
mod tests {
use super::*;
fn caps(list: &[&str]) -> Vec<Capability> {
list.iter().map(|s| s.parse().unwrap()).collect()
}
fn req<'a>(version: &'a str, hash: &'a str, required: &[&str], optional: &[&str]) -> Request<'a> {
Request {
subject: Subject::Installed("p".into()),
plugin_id: "p",
version,
package_hash: hash,
required: caps(required),
optional: caps(optional),
}
}
#[test]
fn confirmation_then_silent_update_then_reconfirm_on_new_capability() {
let mut l = Ledger::in_memory();
let r1 = req("1.0.0", "h1", &["project.read"], &["net:crates.io"]);
assert_eq!(l.decide(&r1), Decision::NeedsConfirmation { new: caps(&["project.read"]), first_time: true });
l.confirm(&r1, &[], BTreeMap::new());
assert_eq!(
l.decide(&r1),
Decision::Allowed { granted: caps(&["project.read"]), pending: vec![] },
"declined optional"
);
assert!(l.get(&r1.subject).unwrap().declined.contains("net:crates.io"));
// Same needs, new version: silent.
let r2 = req("1.1.0", "h2", &["project.read"], &["net:crates.io"]);
assert!(matches!(l.decide(&r2), Decision::Allowed { .. }));
assert_eq!(l.get(&r2.subject).unwrap().version, "1.1.0");
// A new required capability: ask again, only for it.
let r3 = req("1.2.0", "h3", &["project.read", "editor.write"], &[]);
assert_eq!(l.decide(&r3), Decision::NeedsConfirmation { new: caps(&["editor.write"]), first_time: false });
l.confirm(&r3, &[], BTreeMap::new());
assert!(matches!(l.decide(&r3), Decision::Allowed { .. }));
// Revoking a required capability blocks loading again.
assert!(l.set(&r3.subject, &Capability::EditorWrite, false));
assert!(matches!(l.decide(&r3), Decision::NeedsConfirmation { .. }));
}
#[test]
fn migration_grants_low_risk_only() {
let tmp = tempfile::tempdir().unwrap();
let mut l = Ledger::load(tmp.path());
assert!(l.needs_migration());
let r = req("0.1.0", "h", &["project.read", "editor.write"], &[]);
l.migrate(std::slice::from_ref(&r));
// Loads with the low-risk part; the rest is asked for on first use.
assert_eq!(
l.decide(&r),
Decision::Allowed { granted: caps(&["project.read"]), pending: caps(&["editor.write"]) }
);
l.confirm(&r, &[], BTreeMap::new());
assert_eq!(
l.decide(&r),
Decision::Allowed { granted: caps(&["project.read", "editor.write"]), pending: vec![] }
);
l.save().unwrap();
let again = Ledger::load(tmp.path());
assert!(!again.needs_migration());
assert!(again.get(&r.subject).unwrap().has(&Capability::ProjectRead));
}
#[cfg(unix)]
#[test]
fn binaries_are_pinned_by_path_and_hash() {
use std::os::unix::fs::PermissionsExt;
let tmp = tempfile::tempdir().unwrap();
let bin_dir = tmp.path().join("bin");
std::fs::create_dir(&bin_dir).unwrap();
let tool = bin_dir.join("tool");
std::fs::write(&tool, "#!/bin/sh\necho v1\n").unwrap();
std::fs::set_permissions(&tool, std::fs::Permissions::from_mode(0o755)).unwrap();
let path_var = std::env::join_paths([PathBuf::from("relative"), bin_dir.clone()]).unwrap();
let resolved = resolve_program("tool", Some(&path_var)).unwrap();
assert!(resolved.is_absolute());
assert!(resolve_program("missing", Some(&path_var)).is_err());
let mut l = Ledger::in_memory();
let r = req("1.0.0", "h", &["process:tool"], &[]);
let cap: Capability = "process:tool".parse().unwrap();
let bin = BinaryGrant { path: resolved.clone(), hash: file_hash(&resolved).unwrap() };
l.confirm(&r, &[], [(cap.to_string(), bin)].into());
assert_eq!(l.check_binary(&r.subject, &cap).unwrap(), resolved);
std::fs::write(&tool, "#!/bin/sh\necho evil\n").unwrap();
assert!(l.check_binary(&r.subject, &cap).unwrap_err().contains("changed"));
}
#[test]
fn hashes_and_grouping() {
let tmp = tempfile::tempdir().unwrap();
std::fs::write(tmp.path().join("a"), "1").unwrap();
std::fs::write(tmp.path().join("b"), "2").unwrap();
let h1 = package_hash(tmp.path(), &["a".into(), "b".into()]).unwrap();
assert_eq!(h1, package_hash(tmp.path(), &["b".into(), "a".into()]).unwrap());
std::fs::write(tmp.path().join("b"), "3").unwrap();
assert_ne!(h1, package_hash(tmp.path(), &["a".into(), "b".into()]).unwrap());
let groups = by_risk(&caps(&["storage", "process:git", "net:x.io"]));
assert_eq!(groups.iter().map(|g| g.0).collect::<Vec<_>>(), [Risk::High, Risk::Medium, Risk::Low]);
}
}
+36
View File
@@ -0,0 +1,36 @@
//! The plugin system's kernel, independent of the Wasm runtime:
//!
//! - [`Manifest`]: `plugin.toml` (0.2 format; 0.1 still parses);
//! - [`Capability`]: what a plugin may do, with its [`Risk`] level;
//! - [`ExtensionPoint`]: one kind of contribution, implemented by the crate that
//! owns the feature; [`ExtensionRegistry`] validates a package's declarations
//! ([`ExtensionRegistry::prepare`], no side effects), makes them take effect
//! all or nothing ([`ExtensionRegistry::commit`]) and withdraws them per plugin;
//! - [`UiStore`] / [`UiPoint`]: UI contributions, validated here and projected
//! by the client without the possibility to fail;
//! - [`ledger`]: which capabilities the user granted to which plugin;
//! - [`package`]: the package file, its signature and the marketplace index.
//!
//! Dependency direction: `ide-core` ← `ide-extension` ← feature crates (their
//! extension points) and `ide-plugin` (Wasm) ← `ide-host` (assembly).
mod capability;
mod index;
pub mod ledger;
mod manifest;
pub mod package;
mod point;
mod registry;
mod ui;
mod when;
pub use capability::{Capability, CapabilityInfo, Risk, host_matches};
pub use index::ContributionIndex;
pub use manifest::{
ActivationEvent, CATEGORIES, Capabilities, ENGINE_VERSION, EVENT_KINDS, LEGACY_API, MANIFEST_FILE, MAX_CATEGORIES,
Manifest, check_relative, engine_compatible, validate_item_id, validate_plugin_id, validate_point_id,
};
pub use point::{Commit, Disposable, ExtensionPoint, MAX_PACKAGE_FILE, Package, Placement, Prepared, decode};
pub use registry::{ContributionInfo, ExtensionRegistry, PreparedSet};
pub use ui::{UiContribution, UiDecl, UiPoint, UiStore, to_json};
pub use when::validate_when;
+567
View File
@@ -0,0 +1,567 @@
//! `plugin.toml`, the package manifest (plan 4.4).
//!
//! ```toml
//! id = "acme.rust-tools" # optional publisher namespace before the first '.'
//! name = "Rust Tools"
//! version = "1.2.0"
//! engines.openidea = "^1.0"
//! wasm = "plugin.wasm"
//! activation = ["on-language:rust", "on-command:check"]
//!
//! [capabilities]
//! required = ["project.read", "editor.write"]
//! optional = ["net:crates.io", "process:cargo"]
//!
//! [[contributes.commands]]
//! id = "check"
//! title = "Check"
//! ```
//!
//! `contributes` is kept as raw TOML: each `contributes.<point>` is parsed and
//! validated by the extension point registered under that id
//! ([`crate::ExtensionRegistry`]), so new contribution types need no change here.
//!
//! Manifests in the 0.1 format (`openidea-api = "0.1"`, `permissions = [...]`)
//! still parse: permissions become required capabilities and the plugin is
//! activated at start-up, as before.
use crate::capability::Capability;
use serde::Deserialize;
use std::collections::BTreeSet;
use std::path::{Component, Path, PathBuf};
pub const MANIFEST_FILE: &str = "plugin.toml";
/// The plugin API (engine) version this host implements.
pub const ENGINE_VERSION: &str = "1.0.0";
/// The 0.1 API, still accepted for contribution-only packages and 0.1 components.
pub const LEGACY_API: &str = "0.1";
/// Marketplace categories, in the order they are shown.
pub const CATEGORIES: &[&str] =
&["languages", "themes", "icon-themes", "linters", "vcs", "ai", "frameworks", "tools", "other"];
/// Categories a plugin is listed under; later ones are ignored.
pub const MAX_CATEGORIES: usize = 3;
#[derive(Clone, Debug, PartialEq)]
pub struct Manifest {
pub id: String,
pub name: String,
pub version: String,
pub description: String,
pub authors: Vec<String>,
pub homepage: Option<String>,
pub license: Option<String>,
/// Release notes of this version (Settings › Plugins › What's New).
pub change_notes: Option<String>,
/// Marketplace categories as declared (see [`Manifest::categories`]).
pub declared_categories: Vec<String>,
/// `engines.openidea` (a semver requirement), or the legacy `openidea-api`.
pub engine: String,
/// Written in the 0.1 format.
pub legacy: bool,
/// Component file relative to the package directory.
pub wasm: Option<PathBuf>,
pub activation: Vec<ActivationEvent>,
pub capabilities: Capabilities,
/// `contributes.<point>` → its declarations, validated by the extension points.
pub contributes: toml::Table,
}
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct Capabilities {
pub required: Vec<Capability>,
pub optional: Vec<Capability>,
}
impl Capabilities {
pub fn all(&self) -> impl Iterator<Item = &Capability> {
self.required.iter().chain(&self.optional)
}
pub fn is_optional(&self, cap: &Capability) -> bool {
self.optional.contains(cap) && !self.required.contains(cap)
}
}
/// When a plugin's code is instantiated (plan 4.4). Contributions are live
/// without it; code only runs once one of these fires.
#[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum ActivationEvent {
/// One of the plugin's commands is run (`on-command:<id>`; `*` = any).
Command(String),
/// A file of this language is opened.
Language(String),
/// A file matching this glob (project-relative) is opened.
File(String),
/// One of the plugin's views is shown.
View(String),
/// Periodically: `on-schedule:<seconds>`. Does not keep the plugin alive.
Schedule(u32),
/// A workbench event of this kind (`on-event:file-saved`): the plugin is
/// subscribed from the start, and activated to receive it.
Event(String),
/// Shortly after the workbench is ready.
StartupFinished,
}
impl ActivationEvent {
pub fn parse(s: &str) -> Result<Self, String> {
let s = s.trim();
if s == "on-startup-finished" {
return Ok(Self::StartupFinished);
}
let (kind, arg) = s.split_once(':').ok_or_else(|| format!("unknown activation event {s:?}"))?;
let arg = arg.trim();
if arg.is_empty() {
return Err(format!("activation event {s:?} needs a value"));
}
Ok(match kind {
"on-command" => Self::Command(arg.into()),
"on-language" => Self::Language(arg.into()),
"on-file" => Self::File(arg.into()),
"on-view" => Self::View(arg.into()),
"on-schedule" => {
let secs: u32 =
arg.trim_end_matches('s').parse().map_err(|_| format!("{s:?}: expected seconds, e.g. 300"))?;
if secs < 60 {
return Err(format!("{s:?}: the shortest schedule is 60 seconds"));
}
Self::Schedule(secs)
}
"on-event" => {
if !EVENT_KINDS.contains(&arg) {
return Err(format!("{s:?}: unknown event kind (one of {})", EVENT_KINDS.join(", ")));
}
Self::Event(arg.into())
}
_ => return Err(format!("unknown activation event {s:?}")),
})
}
}
/// Workbench event kinds (`on-event:<kind>`, WIT `events.event-kind`).
/// `scheduled` comes from `on-schedule` instead.
pub const EVENT_KINDS: &[&str] = &["file-opened", "file-saved", "editor-focused", "action-performed", "vcs-changed"];
impl std::fmt::Display for ActivationEvent {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Command(c) => write!(f, "on-command:{c}"),
Self::Language(l) => write!(f, "on-language:{l}"),
Self::File(g) => write!(f, "on-file:{g}"),
Self::View(v) => write!(f, "on-view:{v}"),
Self::Schedule(s) => write!(f, "on-schedule:{s}"),
Self::Event(k) => write!(f, "on-event:{k}"),
Self::StartupFinished => f.write_str("on-startup-finished"),
}
}
}
#[derive(Deserialize)]
#[serde(rename_all = "kebab-case", deny_unknown_fields)]
struct Raw {
id: String,
name: String,
version: String,
#[serde(default)]
description: String,
#[serde(default)]
authors: Vec<String>,
#[serde(default)]
homepage: Option<String>,
#[serde(default)]
license: Option<String>,
#[serde(default)]
change_notes: Option<String>,
#[serde(default)]
categories: Vec<String>,
#[serde(default)]
engines: Option<Engines>,
/// 0.1 format.
#[serde(default)]
openidea_api: Option<String>,
#[serde(default)]
wasm: Option<PathBuf>,
#[serde(default)]
activation: Option<Vec<String>>,
#[serde(default)]
capabilities: Option<RawCapabilities>,
/// 0.1 format.
#[serde(default)]
permissions: Option<Vec<String>>,
#[serde(default)]
contributes: toml::Table,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Engines {
openidea: String,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct RawCapabilities {
#[serde(default)]
required: Vec<String>,
#[serde(default)]
optional: Vec<String>,
}
impl Manifest {
pub fn parse(src: &str) -> Result<Self, String> {
let raw: Raw = toml::from_str(src).map_err(|e| e.to_string())?;
let (engine, legacy) = match (raw.engines, raw.openidea_api) {
(Some(e), None) => (e.openidea, false),
(None, Some(api)) => (api, true),
(Some(_), Some(_)) => return Err("use `engines.openidea`, not `openidea-api`".into()),
(None, None) => return Err("missing `engines.openidea` (e.g. \"^1.0\")".into()),
};
if legacy && raw.capabilities.is_some() {
return Err("`capabilities` requires `engines.openidea`; 0.1 manifests use `permissions`".into());
}
if !legacy && raw.permissions.is_some() {
return Err("`permissions` is the 0.1 format; use `[capabilities] required = [...]`".into());
}
let parse_caps = |list: Vec<String>| -> Result<Vec<Capability>, String> {
let mut out: Vec<Capability> = Vec::new();
for s in list {
let cap: Capability = s.parse()?;
if !out.contains(&cap) {
out.push(cap);
}
}
Ok(out)
};
let capabilities = match (raw.capabilities, raw.permissions) {
(Some(c), _) => Capabilities { required: parse_caps(c.required)?, optional: parse_caps(c.optional)? },
(None, Some(p)) => Capabilities { required: parse_caps(p)?, optional: Vec::new() },
(None, None) => Capabilities::default(),
};
let activation = match raw.activation {
Some(list) => list.iter().map(|s| ActivationEvent::parse(s)).collect::<Result<Vec<_>, _>>()?,
// 0.1 plugins were activated at start-up; keep that until they move to 1.0.
None if legacy && raw.wasm.is_some() => vec![ActivationEvent::StartupFinished],
None => Vec::new(),
};
let m = Manifest {
id: raw.id,
name: raw.name,
version: raw.version,
description: raw.description,
authors: raw.authors,
homepage: raw.homepage,
license: raw.license,
change_notes: raw.change_notes,
declared_categories: raw.categories,
engine,
legacy,
wasm: raw.wasm,
activation,
capabilities,
contributes: raw.contributes,
};
m.validate()?;
Ok(m)
}
pub fn load(dir: &Path) -> Result<Self, String> {
let path = dir.join(MANIFEST_FILE);
let src = std::fs::read_to_string(&path).map_err(|e| format!("{}: {e}", path.display()))?;
Self::parse(&src).map_err(|e| format!("{}: {e}", path.display()))
}
fn validate(&self) -> Result<(), String> {
validate_plugin_id(&self.id)?;
if self.name.trim().is_empty() {
return Err("`name` must not be empty".into());
}
if semver::Version::parse(&self.version).is_err() {
return Err(format!("`version` must be a semantic version like 1.2.3, got {:?}", self.version));
}
if !engine_compatible(&self.engine, self.legacy) {
return Err(if self.legacy {
format!(
"unsupported openidea-api {:?} (this IDE implements {LEGACY_API} and {ENGINE_VERSION})",
self.engine
)
} else {
format!("engines.openidea {:?} does not match this IDE's plugin API {ENGINE_VERSION}", self.engine)
});
}
if let Some(wasm) = &self.wasm {
check_relative(wasm, "wasm")?;
}
if self.wasm.is_none() && !self.activation.is_empty() {
return Err("`activation` needs `wasm`: there is no code to activate".into());
}
let mut seen = BTreeSet::new();
for event in &self.activation {
if !seen.insert(event) {
return Err(format!("duplicate activation event {event}"));
}
}
for c in &self.declared_categories {
validate_item_id(c, "category")?;
}
for (point, decls) in &self.contributes {
validate_point_id(point)?;
if !matches!(decls, toml::Value::Array(_) | toml::Value::Table(_)) {
return Err(format!("`contributes.{point}` must be a table or an array of tables"));
}
}
Ok(())
}
/// The declarations of one extension point: `[[contributes.x]]` gives
/// several, `[contributes.x]` one.
pub fn declarations(&self, point: &str) -> Vec<toml::Value> {
match self.contributes.get(point) {
Some(toml::Value::Array(items)) => items.clone(),
Some(single @ toml::Value::Table(_)) => vec![single.clone()],
_ => Vec::new(),
}
}
/// Number of declarations per extension point (for listings before loading).
pub fn declaration_counts(&self) -> Vec<(String, u32)> {
self.contributes.keys().map(|p| (p.clone(), self.declarations(p).len() as u32)).collect()
}
/// Publisher namespace: the id's part before the first `.` (`acme` in `acme.tools`).
pub fn publisher(&self) -> Option<&str> {
self.id.split_once('.').map(|(p, _)| p)
}
/// Whether `cap` is declared (required or optional).
pub fn declares(&self, cap: &Capability) -> bool {
self.capabilities.all().any(|c| c.covers(cap))
}
}
impl Manifest {
/// Its marketplace categories, in [`CATEGORIES`] order: the first
/// [`MAX_CATEGORIES`] declared that this IDE knows; without any, those its
/// contributions imply (themes, icon themes, languages), else `other`.
pub fn categories(&self) -> Vec<String> {
let pick = |f: &dyn Fn(&str) -> bool| CATEGORIES.iter().filter(|c| f(c)).map(|c| c.to_string()).collect();
let mut known: Vec<&str> = Vec::new();
for d in &self.declared_categories {
if CATEGORIES.contains(&d.as_str()) && !known.contains(&d.as_str()) && known.len() < MAX_CATEGORIES {
known.push(d);
}
}
let mut out: Vec<String> = pick(&|c| known.contains(&c));
if out.is_empty() {
out = pick(&|c| match c {
"themes" => self.contributes.contains_key("themes"),
"icon-themes" => self.contributes.contains_key("iconThemes"),
"languages" => self.contributes.contains_key("languages"),
_ => false,
});
}
if out.is_empty() {
out.push("other".into());
}
out
}
/// Declared categories this IDE doesn't know (ignored).
pub fn unknown_categories(&self) -> Vec<&str> {
self.declared_categories.iter().map(String::as_str).filter(|c| !CATEGORIES.contains(c)).collect()
}
}
/// Whether this host can load a plugin written for `engine`.
pub fn engine_compatible(engine: &str, legacy: bool) -> bool {
if legacy {
return engine == LEGACY_API || engine.strip_prefix(LEGACY_API).is_some_and(|rest| rest.starts_with('.'));
}
let Ok(req) = semver::VersionReq::parse(engine) else { return false };
let host = semver::Version::parse(ENGINE_VERSION).expect("valid engine version");
req.matches(&host)
}
/// Segments of `[a-z0-9][a-z0-9_-]*` joined by `.` (an optional publisher
/// namespace first), at most 64 chars.
pub fn validate_plugin_id(id: &str) -> Result<(), String> {
let ok = id.len() <= 64 && !id.is_empty() && id.split('.').all(segment_ok);
if ok {
Ok(())
} else {
Err(format!(
"invalid plugin id {id:?}: use a-z, 0-9, '-', '_' in '.'-separated parts (e.g. acme.tools), max 64"
))
}
}
/// `[a-z0-9][a-z0-9._-]*`, at most 64 chars (commands, themes, views…).
pub fn validate_item_id(id: &str, what: &str) -> Result<(), String> {
let ok_first = id.chars().next().is_some_and(|c| c.is_ascii_lowercase() || c.is_ascii_digit());
let ok_rest = id.chars().all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || matches!(c, '-' | '_' | '.'));
if ok_first && ok_rest && id.len() <= 64 {
Ok(())
} else {
Err(format!("invalid {what} {id:?}: use a-z, 0-9, '.', '-', '_', starting with a letter or digit (max 64)"))
}
}
/// Extension point ids: `camelCase` or `kebab-case` words, dots allowed (`iconThemes`, `menus`).
pub fn validate_point_id(id: &str) -> Result<(), String> {
let ok = !id.is_empty()
&& id.len() <= 64
&& id.starts_with(|c: char| c.is_ascii_lowercase())
&& id.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '-' | '.'));
if ok { Ok(()) } else { Err(format!("invalid contribution point {id:?}")) }
}
fn segment_ok(s: &str) -> bool {
s.chars().next().is_some_and(|c| c.is_ascii_lowercase() || c.is_ascii_digit())
&& s.chars().all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-' || c == '_')
}
/// Package paths must stay inside the package directory.
pub fn check_relative(path: &Path, what: &str) -> Result<(), String> {
let ok = !path.as_os_str().is_empty() && path.components().all(|c| matches!(c, Component::Normal(_)));
if ok { Ok(()) } else { Err(format!("{what} {:?} must be a relative path inside the plugin", path.display())) }
}
#[cfg(test)]
mod tests {
use super::*;
const FULL: &str = r#"
id = "acme.rust-tools"
name = "Rust Tools"
version = "1.2.0"
engines.openidea = "^1.0"
wasm = "plugin.wasm"
activation = ["on-language:rust", "on-command:check", "on-schedule:300"]
[capabilities]
required = ["project.read", "editor.write"]
optional = ["net:crates.io", "process:cargo"]
[[contributes.commands]]
id = "check"
title = "%check.title%"
[[contributes.menus]]
menu = "editor.context"
command = "check"
when = "editorLang == rust"
[contributes.settings]
schema = "settings.schema.json"
"#;
#[test]
fn event_activation() {
assert_eq!(ActivationEvent::parse("on-event:file-saved"), Ok(ActivationEvent::Event("file-saved".into())));
assert_eq!(ActivationEvent::Event("vcs-changed".into()).to_string(), "on-event:vcs-changed");
let err = ActivationEvent::parse("on-event:keystroke").unwrap_err();
assert!(err.contains("unknown event kind") && err.contains("action-performed"), "{err}");
assert!(ActivationEvent::parse("on-event:scheduled").is_err(), "declared with on-schedule");
}
#[test]
fn categories_are_declared_or_implied() {
let head = "id = \"acme.x\"\nname = \"X\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"\n";
let m = |rest: &str| Manifest::parse(&format!("{head}{rest}"));
// Declared, in the catalogue's order; unknown ones (a newer IDE's) are ignored.
let declared = m("categories = [\"tools\", \"vcs\", \"robots\"]\n").unwrap();
assert_eq!(declared.categories(), ["vcs", "tools"]);
assert_eq!(declared.unknown_categories(), ["robots"]);
// None declared: implied by the contributions, else `other`.
let theme = m("[[contributes.themes]]\nid = \"t\"\n[[contributes.languages]]\nid = \"l\"\n").unwrap();
assert_eq!(theme.categories(), ["languages", "themes"]);
let declared_wins = m("categories = [\"tools\"]\n[[contributes.languages]]\nid = \"l\"\n").unwrap();
assert_eq!(declared_wins.categories(), ["tools"]);
assert_eq!(m("").unwrap().categories(), ["other"]);
// At most three: the first ones declared.
let many = m("categories = [\"vcs\", \"ai\", \"vcs\", \"tools\", \"themes\"]\n").unwrap();
assert_eq!(many.categories(), ["vcs", "ai", "tools"]);
assert!(m("categories = [\"Bad Name\"]\n").is_err());
}
#[test]
fn parses_the_02_format() {
let m = Manifest::parse(FULL).unwrap();
assert_eq!(m.publisher(), Some("acme"));
assert!(!m.legacy);
assert_eq!(m.activation.len(), 3);
assert_eq!(m.activation[2], ActivationEvent::Schedule(300));
assert_eq!(m.capabilities.required, vec![Capability::ProjectRead, Capability::EditorWrite]);
assert!(m.capabilities.is_optional(&Capability::Process("cargo".into())));
assert_eq!(m.declarations("commands").len(), 1);
assert_eq!(m.declarations("settings").len(), 1, "a single table is one declaration");
assert!(m.declarations("nope").is_empty());
let counts: Vec<_> = m.declaration_counts();
assert!(counts.contains(&("menus".into(), 1)));
}
#[test]
fn parses_the_01_format() {
let m = Manifest::parse(
"id = \"hello\"\nname = \"Hello\"\nversion = \"0.1.0\"\nopenidea-api = \"0.1\"\nwasm = \"dist/plugin.wasm\"\n\
permissions = [\"project-read\", \"editor-write\"]\n[[contributes.commands]]\nid = \"x\"\ntitle = \"X\"",
)
.unwrap();
assert!(m.legacy);
assert_eq!(m.capabilities.required, vec![Capability::ProjectRead, Capability::EditorWrite]);
assert_eq!(m.activation, vec![ActivationEvent::StartupFinished]);
assert_eq!(m.publisher(), None);
}
fn err(src: &str) -> String {
Manifest::parse(src).unwrap_err()
}
#[test]
fn rejects_invalid_manifests() {
let base = |extra: &str| format!("id = \"ok\"\nname = \"N\"\nversion = \"1.0.0\"\n{extra}");
assert!(err(&base("")).contains("engines.openidea"));
assert!(err(&base("engines.openidea = \"^0.3\"")).contains("does not match"));
assert!(err(&base("engines.openidea = \"^2\"")).contains("does not match"));
assert!(err(&base("engines.openidea = \"^0.2\"")).contains("does not match"));
assert!(err(&base("openidea-api = \"0.2\"")).contains("unsupported"));
assert!(err(&base("engines.openidea = \"^1.0\"\npermissions = [\"project-read\"]")).contains("0.1 format"));
assert!(
err(&base("engines.openidea = \"^1.0\"\n[capabilities]\nrequired = [\"network\"]")).contains("unknown")
);
assert!(err(&base("engines.openidea = \"^1.0\"\nwasm = \"../x.wasm\"")).contains("relative path"));
assert!(err(&base("engines.openidea = \"^1.0\"\nactivation = [\"on-command:x\"]")).contains("needs `wasm`"));
assert!(
err(&base("engines.openidea = \"^1.0\"\nwasm = \"p.wasm\"\nactivation = [\"on-boot\"]"))
.contains("unknown")
);
assert!(
err(&base("engines.openidea = \"^1.0\"\nwasm = \"p.wasm\"\nactivation = [\"on-schedule:5\"]"))
.contains("60 seconds")
);
assert!(err(&base("engines.openidea = \"^1.0\"\nbogus = 1")).contains("unknown field"));
assert!(err(&base("engines.openidea = \"^1.0\"\ncontributes.commands = 3")).contains("table"));
assert!(
err("id = \"Bad\"\nname = \"N\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"").contains("plugin id")
);
assert!(
err("id = \"a..b\"\nname = \"N\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"").contains("plugin id")
);
assert!(err("id = \"ok\"\nname = \"N\"\nversion = \"1.0\"\nengines.openidea = \"^1.0\"").contains("version"));
}
#[test]
fn ids() {
assert!(validate_plugin_id("openidea.material-icons").is_ok());
assert!(validate_plugin_id("hello_2").is_ok());
assert!(validate_plugin_id(".x").is_err());
assert!(validate_item_id("word-count.v2_x", "command id").is_ok());
assert!(validate_item_id("-x", "command id").is_err());
assert!(validate_point_id("iconThemes").is_ok());
assert!(validate_point_id("Icon").is_err());
assert!(engine_compatible("1.0", false));
assert!(engine_compatible(">=1.0, <2", false));
assert!(!engine_compatible("^0.2", false), "0.x components don't run on 1.0");
assert!(engine_compatible("0.1.3", true));
assert!(!engine_compatible("0.10", true));
}
}
+955
View File
@@ -0,0 +1,955 @@
//! Plugin distribution: the package file, its
//! signature, and the static marketplace index.
//!
//! - **Package** (`<id>-<version>.oiplugin`): a zip of the manifest, the files
//! its declarations reference, the component and licence files, plus
//! [`SIGNATURE_FILE`] when signed.
//! - **Signature**: Ed25519 over `openidea-plugin/1\n<id>\n<version>\n<digest>`,
//! `digest` being [`package_digest`] of the unpacked files. A signature
//! proves who published the bytes; whether to trust that publisher is the
//! index's call.
//! - **Index** (`index.json`): the publishers and their keys (the trust root),
//! the published plugin versions, and revocations. A publisher owns the id
//! prefix `<publisher>.`; a revoked key invalidates everything it signed, a
//! revoked plugin version is disabled where it is installed. A plugin may be
//! listed in several versions, each with the plugin API it needs: the IDE
//! offers the newest one it can load.
//! - **Key continuity**: an installed plugin updates silently only to a
//! package signed with the same key, or with a key the installed one
//! endorsed ([`Endorsement`], a publisher rotating keys); any other signer
//! is the user's call. An IDE can pin a publisher to keys it was built with
//! ([`Index::pin_publisher`]): the index then can't replace them.
//! - **Format**: fields are only ever added (IDEs ignore those they don't
//! know); `format` grows only for changes older IDEs can't read, and they
//! still apply such an index's revocations ([`Index::parse`]).
use crate::MANIFEST_FILE;
use base64::Engine as _;
use ed25519_dalek::{Signature, Signer, SigningKey, VerifyingKey};
use serde::{Deserialize, Serialize};
use std::io::{Read, Write};
use std::path::{Component, Path, PathBuf};
pub const SIGNATURE_FILE: &str = "SIGNATURE.json";
pub const PACKAGE_EXTENSION: &str = "oiplugin";
pub const INDEX_FORMAT: u32 = 1;
/// Limits when unpacking an untrusted package.
const MAX_ENTRIES: usize = 10_000;
const MAX_TOTAL: u64 = 512 << 20;
const B64: base64::engine::GeneralPurpose = base64::engine::general_purpose::STANDARD;
/// Content digest of a package: the manifest plus `files` (relative paths).
pub fn package_digest(dir: &Path, files: &[PathBuf]) -> std::io::Result<String> {
let mut all = files.to_vec();
all.push(PathBuf::from(MANIFEST_FILE));
crate::ledger::package_hash(dir, &all)
}
fn signed_message(id: &str, version: &str, digest: &str) -> Vec<u8> {
format!("openidea-plugin/1\n{id}\n{version}\n{digest}").into_bytes()
}
fn endorsement_message(publisher: &str, key: &str) -> Vec<u8> {
format!("openidea-key/1\n{publisher}\n{key}").into_bytes()
}
/// Whether `signature` (base64) is `key`'s (base64) Ed25519 signature of
/// `message`; malformed keys and signatures are errors.
fn verify_ed25519(key: &str, message: &[u8], signature: &str) -> Result<bool, String> {
let key: [u8; 32] = B64
.decode(key)
.ok()
.and_then(|b| b.try_into().ok())
.ok_or_else(|| "signature: malformed public key".to_owned())?;
let key = VerifyingKey::from_bytes(&key).map_err(|_| "signature: invalid public key".to_owned())?;
let sig: [u8; 64] =
B64.decode(signature).ok().and_then(|b| b.try_into().ok()).ok_or_else(|| "signature: malformed".to_owned())?;
Ok(key.verify_strict(message, &Signature::from_bytes(&sig)).is_ok())
}
/// `SIGNATURE.json` of a signed package.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct PackageSignature {
pub publisher: String,
/// Ed25519 public key, base64.
pub key: String,
/// Ed25519 signature, base64.
pub signature: String,
}
/// A publisher's key pair (the 32-byte seed, base64, in a file only the publisher has).
pub struct PublisherKey(SigningKey);
impl PublisherKey {
pub fn from_seed(seed: [u8; 32]) -> Self {
Self(SigningKey::from_bytes(&seed))
}
/// The seed as written by `openidea-plugin keygen`.
pub fn from_base64(text: &str) -> Result<Self, String> {
let bytes = B64.decode(text.trim()).map_err(|e| format!("key file: {e}"))?;
let seed: [u8; 32] = bytes.try_into().map_err(|_| "key file: expected a 32-byte key".to_owned())?;
Ok(Self::from_seed(seed))
}
pub fn seed_base64(&self) -> String {
B64.encode(self.0.to_bytes())
}
pub fn public_base64(&self) -> String {
B64.encode(self.0.verifying_key().to_bytes())
}
/// Vouches for `publisher`'s new key `key` (base64), the one that follows this one.
pub fn endorse(&self, publisher: &str, key: &str) -> Endorsement {
let sig = self.0.sign(&endorsement_message(publisher, key));
Endorsement { key: self.public_base64(), signature: B64.encode(sig.to_bytes()) }
}
pub fn sign(&self, publisher: &str, id: &str, version: &str, digest: &str) -> PackageSignature {
let sig = self.0.sign(&signed_message(id, version, digest));
PackageSignature {
publisher: publisher.into(),
key: self.public_base64(),
signature: B64.encode(sig.to_bytes()),
}
}
}
impl PackageSignature {
/// Whether the signature is valid for `id`, `version` and `digest` (the
/// key's validity is checked against an [`Index`]).
pub fn check(&self, id: &str, version: &str, digest: &str) -> Result<(), String> {
if verify_ed25519(&self.key, &signed_message(id, version, digest), &self.signature)? {
Ok(())
} else {
Err("the signature does not match the package (it was changed after signing)".to_owned())
}
}
}
/// Writes the package `out` from `dir`: the manifest, `files`, and the signature if any.
pub fn pack(dir: &Path, files: &[PathBuf], out: &Path, signature: Option<&PackageSignature>) -> Result<(), String> {
let io = |e: std::io::Error| e.to_string();
let file = std::fs::File::create(out).map_err(io)?;
let mut zip = zip::ZipWriter::new(file);
let options = zip::write::SimpleFileOptions::default().compression_method(zip::CompressionMethod::Deflated);
let mut names: Vec<PathBuf> = std::iter::once(PathBuf::from(MANIFEST_FILE)).chain(files.iter().cloned()).collect();
names.sort();
names.dedup();
for rel in names {
let name = rel.to_str().ok_or("non UTF-8 file name")?.replace('\\', "/");
zip.start_file(name, options).map_err(|e| e.to_string())?;
let bytes = std::fs::read(dir.join(&rel)).map_err(|e| format!("{}: {e}", rel.display()))?;
zip.write_all(&bytes).map_err(io)?;
}
if let Some(sig) = signature {
zip.start_file(SIGNATURE_FILE, options).map_err(|e| e.to_string())?;
zip.write_all(serde_json::to_string_pretty(sig).map_err(|e| e.to_string())?.as_bytes()).map_err(io)?;
}
zip.finish().map_err(|e| e.to_string())?;
Ok(())
}
/// What [`unpack`] wrote.
#[derive(Debug)]
pub struct Unpacked {
/// Every file but the manifest and the signature, relative.
pub files: Vec<PathBuf>,
pub signature: Option<PackageSignature>,
}
/// `name` as a relative path that stays inside the package.
fn safe_path(name: &str) -> Option<PathBuf> {
let p = Path::new(name);
(!name.is_empty() && p.components().all(|c| matches!(c, Component::Normal(_)))).then(|| p.to_path_buf())
}
/// Archiver metadata at the top of an archive, not part of a package
/// (`__MACOSX/…`, `.DS_Store`, AppleDouble `._…` files).
fn archiver_metadata(name: &str) -> bool {
let top = name.split('/').next().unwrap_or_default();
top == "__MACOSX" || top == ".DS_Store" || (top.starts_with("._") && top.len() > 2)
}
/// Where the package is inside an archive: at its root (`.oiplugin`, or a zip
/// of the package's contents) or in its only folder (a zip of the package
/// folder). `None` when neither holds a manifest.
fn package_prefix(names: &[String]) -> Option<String> {
if names.iter().any(|n| n == MANIFEST_FILE) {
return Some(String::new());
}
let mut tops = names.iter().filter(|n| !archiver_metadata(n)).map(|n| n.split('/').next().unwrap_or_default());
let top = tops.next()?;
if tops.any(|t| t != top) {
return None;
}
let prefix = format!("{top}/");
names.iter().any(|n| *n == format!("{prefix}{MANIFEST_FILE}")).then_some(prefix)
}
/// Extracts the package `archive` (an `.oiplugin`, or a `.zip` of a package
/// folder or of its contents) into the empty directory `dest`, refusing
/// paths leaving it, links, and oversized contents.
pub fn unpack(archive: &Path, dest: &Path) -> Result<Unpacked, String> {
let file = std::fs::File::open(archive).map_err(|e| format!("{}: {e}", archive.display()))?;
let mut zip =
zip::ZipArchive::new(file).map_err(|e| format!("{}: not a plugin package ({e})", archive.display()))?;
if zip.len() > MAX_ENTRIES {
return Err(format!("more than {MAX_ENTRIES} files"));
}
let mut names = Vec::with_capacity(zip.len());
for i in 0..zip.len() {
let entry = zip.by_index(i).map_err(|e| e.to_string())?;
if !entry.is_dir() {
names.push(entry.name().map_err(|e| e.to_string())?.into_owned());
}
}
// An IntelliJ plugin or any other zip: say what is missing rather than fail later.
let prefix = package_prefix(&names).ok_or_else(|| {
format!("{} is not an OpenIDEA plugin: no {MANIFEST_FILE} at its root or in its only folder", archive.display())
})?;
std::fs::create_dir_all(dest).map_err(|e| e.to_string())?;
let (mut files, mut signature, mut total, mut manifest) = (Vec::new(), None, 0u64, false);
for i in 0..zip.len() {
let mut entry = zip.by_index(i).map_err(|e| e.to_string())?;
if entry.is_dir() {
continue;
}
let full = entry.name().map_err(|e| e.to_string())?.into_owned();
// Outside the package folder there is only archiver metadata (`package_prefix` checked).
let Some(name) = full.strip_prefix(&prefix).map(str::to_owned) else { continue };
if archiver_metadata(&name) {
continue;
}
if entry.is_symlink() {
return Err(format!("{full}: links are not allowed in packages"));
}
let rel = safe_path(&name).ok_or_else(|| format!("invalid path {full:?} in package"))?;
let mut bytes = Vec::new();
entry.by_ref().take(MAX_TOTAL - total + 1).read_to_end(&mut bytes).map_err(|e| e.to_string())?;
total += bytes.len() as u64;
if total > MAX_TOTAL {
return Err(format!("the package unpacks to more than {} MiB", MAX_TOTAL >> 20));
}
if rel == Path::new(SIGNATURE_FILE) {
signature = Some(serde_json::from_slice(&bytes).map_err(|e| format!("{SIGNATURE_FILE}: {e}"))?);
continue;
}
let to = dest.join(&rel);
if let Some(parent) = to.parent() {
std::fs::create_dir_all(parent).map_err(|e| e.to_string())?;
}
std::fs::write(&to, bytes).map_err(|e| e.to_string())?;
if rel == Path::new(MANIFEST_FILE) {
manifest = true;
} else {
files.push(rel);
}
}
if !manifest {
return Err(format!("the package has no {MANIFEST_FILE}"));
}
files.sort();
Ok(Unpacked { files, signature })
}
// ---- index -------------------------------------------------------------------
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct Index {
pub format: u32,
#[serde(default)]
pub publishers: Vec<Publisher>,
#[serde(default)]
pub plugins: Vec<IndexEntry>,
#[serde(default)]
pub revoked: Vec<Revocation>,
/// The marketplace moved: its index is at this URL (relative to this
/// one, or absolute), and the IDE reads that one instead.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub moved: Option<String>,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct Publisher {
/// Owns plugin ids `<id>.…`.
pub id: String,
pub name: String,
pub keys: Vec<KeyEntry>,
}
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct KeyEntry {
/// Ed25519 public key, base64.
pub key: String,
/// Packages signed with it are no longer trusted.
#[serde(default)]
pub revoked: bool,
/// The publisher's earlier key vouching for this one.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub endorsement: Option<Endorsement>,
}
impl KeyEntry {
pub fn new(key: String) -> Self {
Self { key, ..Self::default() }
}
}
/// A publisher's earlier key vouching for a newer one: plugins installed from
/// packages signed with the earlier key update to ones signed with the newer
/// key without asking the user.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct Endorsement {
/// The endorsing key, base64.
pub key: String,
/// Its Ed25519 signature of `openidea-key/1\n<publisher>\n<endorsed key>`, base64.
pub signature: String,
}
/// A published plugin version.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct IndexEntry {
pub id: String,
pub name: String,
pub version: String,
#[serde(default)]
pub description: String,
#[serde(default)]
pub authors: Vec<String>,
pub publisher: String,
/// The package, relative to the index (or an absolute URL).
pub download: String,
/// [`package_digest`] of its contents.
pub digest: String,
#[serde(default)]
pub capabilities: Vec<String>,
/// The plugin API versions it loads in (a semver requirement): its
/// manifest's `engines.openidea` and the API its component was built for.
/// Required: an entry without one is offered to no IDE.
#[serde(default)]
pub engine: String,
/// Marketplace categories (`CATEGORIES`); an IDE ignores those it doesn't know.
#[serde(default)]
pub categories: Vec<String>,
}
impl IndexEntry {
/// Whether this IDE can load the version.
pub fn compatible(&self) -> bool {
crate::engine_compatible(&self.engine, false)
}
fn semver(&self) -> Option<semver::Version> {
semver::Version::parse(&self.version).ok()
}
}
/// `engines.openidea` of a manifest narrowed to the API its component was built
/// for (`major.minor`, if it has code): the requirement an index lists.
pub fn engine_requirement(manifest_engine: &str, built_for: Option<&semver::Version>) -> String {
match built_for {
Some(v) => format!("{manifest_engine}, ^{}.{}", v.major, v.minor),
None => manifest_engine.to_owned(),
}
}
/// A plugin (all versions when `versions` is empty) that must not run.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct Revocation {
pub id: String,
#[serde(default)]
pub versions: Vec<String>,
pub reason: String,
}
impl Index {
/// Parses an index. One in a newer format keeps what must reach every IDE —
/// revocations, publisher keys (revoked ones) and `moved` — and lists no
/// plugins ([`Index::is_newer`]): the IDE needs updating to install from it.
pub fn parse(text: &str) -> Result<Self, String> {
let value: serde_json::Value = serde_json::from_str(text).map_err(|e| format!("index: {e}"))?;
let format = value.get("format").and_then(serde_json::Value::as_u64).ok_or("index: `format` is missing")?;
if format == u64::from(INDEX_FORMAT) {
return serde_json::from_value(value).map_err(|e| format!("index: {e}"));
}
if format < u64::from(INDEX_FORMAT) {
return Err(format!("index format {format} is not supported (this IDE reads {INDEX_FORMAT})"));
}
fn each<T: serde::de::DeserializeOwned>(value: &serde_json::Value, field: &str) -> Vec<T> {
let items = value.get(field).and_then(serde_json::Value::as_array);
items.into_iter().flatten().filter_map(|v| serde_json::from_value(v.clone()).ok()).collect()
}
Ok(Index {
format: u32::try_from(format).unwrap_or(u32::MAX),
publishers: each(&value, "publishers"),
plugins: Vec::new(),
revoked: each(&value, "revoked"),
moved: value.get("moved").and_then(serde_json::Value::as_str).map(str::to_owned),
})
}
/// Written in a format newer than this IDE reads: it lists no plugins.
pub fn is_newer(&self) -> bool {
self.format > INDEX_FORMAT
}
/// The version of each plugin to offer, and whether this IDE can load it:
/// the newest it can load, else the newest. Revoked versions are skipped;
/// plugins come in the order they are first listed.
pub fn offered(&self) -> Vec<(&IndexEntry, bool)> {
let mut ids: Vec<&str> = Vec::new();
for e in &self.plugins {
if !ids.contains(&e.id.as_str()) {
ids.push(&e.id);
}
}
ids.into_iter()
.filter_map(|id| {
let versions: Vec<&IndexEntry> =
self.plugins.iter().filter(|e| e.id == id && self.revocation(id, &e.version).is_none()).collect();
let newest = |loadable: bool| {
versions.iter().copied().filter(|e| !loadable || e.compatible()).max_by_key(|e| e.semver())
};
newest(true).map(|e| (e, true)).or_else(|| newest(false).map(|e| (e, false)))
})
.collect()
}
/// Drops the versions no IDE would pick: revoked ones, and of each plugin
/// all but the newest version per API requirement.
pub fn prune_versions(&mut self) {
let plugins = std::mem::take(&mut self.plugins);
let mut kept: Vec<IndexEntry> = Vec::new();
for e in plugins.into_iter().filter(|e| self.revocation(&e.id, &e.version).is_none()) {
match kept.iter_mut().find(|k| k.id == e.id && k.engine == e.engine) {
Some(k) if e.semver() > k.semver() => *k = e,
Some(_) => {}
None => kept.push(e),
}
}
self.plugins = kept;
}
/// The revocation that applies to `id` at `version`.
pub fn revocation(&self, id: &str, version: &str) -> Option<&Revocation> {
self.revoked.iter().find(|r| r.id == id && (r.versions.is_empty() || r.versions.iter().any(|v| v == version)))
}
/// Whether `key` (base64) belongs to a publisher and was revoked.
pub fn key_revoked(&self, key: &str) -> bool {
self.publishers.iter().flat_map(|p| &p.keys).any(|k| k.key == key && k.revoked)
}
/// The trusted publisher of a signed package: the signature is valid, its
/// key is one of the publisher's and not revoked, and the publisher owns
/// the plugin's id prefix.
pub fn verify(&self, sig: &PackageSignature, id: &str, version: &str, digest: &str) -> Result<String, String> {
let publisher = self
.publishers
.iter()
.find(|p| p.id == sig.publisher)
.ok_or_else(|| format!("unknown publisher {:?}", sig.publisher))?;
let key = publisher
.keys
.iter()
.find(|k| k.key == sig.key)
.ok_or_else(|| format!("the key is not one of {}'s", publisher.name))?;
if key.revoked {
return Err(format!("{}'s signing key was revoked", publisher.name));
}
if !id.starts_with(&format!("{}.", publisher.id)) {
return Err(format!("{} may only publish plugins named {}.*", publisher.name, publisher.id));
}
sig.check(id, version, digest)?;
Ok(publisher.id.clone())
}
/// Whether `to` is `from`, or a key of `publisher` that `from` endorsed,
/// directly or through keys it endorsed. Endorsements by revoked keys don't
/// count: whoever stole a key could vouch for their own.
pub fn succeeds(&self, publisher: &str, from: &str, to: &str) -> bool {
let keys = self.publishers.iter().find(|p| p.id == publisher).map_or(&[][..], |p| &p.keys[..]);
let mut key = to;
// Each step moves to an earlier key; more steps than keys is a loop.
for _ in 0..=keys.len() {
if key == from {
return true;
}
let Some(e) = keys.iter().find(|k| k.key == key).and_then(|k| k.endorsement.as_ref()) else {
return false;
};
let endorser_valid = keys.iter().any(|k| k.key == e.key && !k.revoked);
if !endorser_valid || verify_ed25519(&e.key, &endorsement_message(publisher, key), &e.signature) != Ok(true)
{
return false;
}
key = &e.key;
}
false
}
/// Makes `roots` (base64) the keys `publisher` signs with, whatever this
/// index says: another key it lists stays only when a root endorsed it (a
/// rotation the roots vouched for). Revocations of the keys are kept.
pub fn pin_publisher(&mut self, publisher: &str, name: &str, roots: &[String]) {
if roots.is_empty() {
return;
}
let i = match self.publishers.iter().position(|p| p.id == publisher) {
Some(i) => i,
None => {
self.publishers.push(Publisher { id: publisher.into(), name: name.into(), keys: Vec::new() });
self.publishers.len() - 1
}
};
for root in roots {
if !self.publishers[i].keys.iter().any(|k| &k.key == root) {
self.publishers[i].keys.push(KeyEntry::new(root.clone()));
}
}
let keys = self.publishers[i].keys.clone();
let trusted = keys
.into_iter()
.filter(|k| roots.iter().any(|r| self.succeeds(publisher, r, &k.key)))
.map(|k| if roots.contains(&k.key) { KeyEntry { endorsement: None, ..k } } else { k })
.collect();
self.publishers[i].keys = trusted;
}
/// Replaces what this index lists for every publisher but `keep` with what
/// `other` lists: its publishers, their plugins and their revocations
/// (`other`'s downloads should already be absolute). `other` may only
/// speak for its own publishers, and none of them may be in `keep`.
pub fn include(&mut self, other: Index, keep: &[&str]) -> Result<(), String> {
let mut theirs: Vec<&str> = Vec::new();
for p in &other.publishers {
if !publisher_id_ok(&p.id) {
return Err(format!("invalid publisher id {:?}: use a-z, 0-9, '-' and '_'", p.id));
}
if keep.contains(&p.id.as_str()) {
return Err(format!("may not list publisher {:?}", p.id));
}
if theirs.contains(&p.id.as_str()) {
return Err(format!("publisher {:?} is listed twice", p.id));
}
theirs.push(&p.id);
}
let owned_by = |id: &str| theirs.iter().any(|p| owns(p, id));
for e in &other.plugins {
if !theirs.contains(&e.publisher.as_str()) || !owns(&e.publisher, &e.id) {
return Err(format!("{}: not a plugin of publisher {:?}", e.id, e.publisher));
}
if semver::VersionReq::parse(&e.engine).is_err() {
return Err(format!("{} {}: `engine` must be the plugin API it needs", e.id, e.version));
}
}
if let Some(r) = other.revoked.iter().find(|r| !owned_by(&r.id)) {
return Err(format!("{}: revoked by an index that doesn't publish it", r.id));
}
let kept = |id: &str| keep.iter().any(|p| owns(p, id));
self.publishers.retain(|p| keep.contains(&p.id.as_str()));
self.plugins.retain(|e| keep.contains(&e.publisher.as_str()));
self.revoked.retain(|r| kept(&r.id));
self.publishers.extend(other.publishers);
self.plugins.extend(other.plugins);
self.revoked.extend(other.revoked);
Ok(())
}
}
/// Whether a package signed `new` (`None`: unsigned) may replace an installed
/// one signed `installed` without asking the user: signed with the same key,
/// or with one the installed key endorsed (per `index`). An unsigned installed
/// copy pins nothing.
pub fn signer_continues(
index: Option<&Index>,
installed: Option<&PackageSignature>,
new: Option<&PackageSignature>,
) -> bool {
match (installed, new) {
(None, _) => true,
(Some(_), None) => false,
(Some(old), Some(new)) => {
old.publisher == new.publisher
&& (old.key == new.key || index.is_some_and(|i| i.succeeds(&new.publisher, &old.key, &new.key)))
}
}
}
/// Whether the plugin `id` is in `publisher`'s namespace.
fn owns(publisher: &str, id: &str) -> bool {
id.strip_prefix(publisher).is_some_and(|rest| rest.starts_with('.'))
}
/// A publisher id is a single segment of a plugin id.
pub fn publisher_id_ok(id: &str) -> bool {
!id.contains('.') && crate::validate_plugin_id(id).is_ok()
}
#[cfg(test)]
mod tests {
use super::*;
fn package(dir: &Path) -> Vec<PathBuf> {
std::fs::create_dir_all(dir.join("dist")).unwrap();
std::fs::write(dir.join(MANIFEST_FILE), "id = \"acme.tools\"\nversion = \"1.0.0\"").unwrap();
std::fs::write(dir.join("dist/plugin.wasm"), b"\0asm").unwrap();
vec![PathBuf::from("dist/plugin.wasm")]
}
fn index(key: &PublisherKey) -> Index {
Index {
format: INDEX_FORMAT,
publishers: vec![Publisher {
id: "acme".into(),
name: "ACME".into(),
keys: vec![KeyEntry::new(key.public_base64())],
}],
plugins: Vec::new(),
revoked: vec![Revocation {
id: "acme.bad".into(),
versions: vec!["1.0.0".into()],
reason: "malware".into(),
}],
moved: None,
}
}
#[test]
fn signed_packages_round_trip_and_tampering_is_caught() {
let tmp = tempfile::tempdir().unwrap();
let src = tmp.path().join("src");
let files = package(&src);
let digest = package_digest(&src, &files).unwrap();
let key = PublisherKey::from_seed([7; 32]);
let sig = key.sign("acme", "acme.tools", "1.0.0", &digest);
let out = tmp.path().join("acme.tools-1.0.0.oiplugin");
pack(&src, &files, &out, Some(&sig)).unwrap();
let dest = tmp.path().join("dest");
let unpacked = unpack(&out, &dest).unwrap();
assert_eq!(unpacked.files, files);
assert_eq!(package_digest(&dest, &unpacked.files).unwrap(), digest);
let index = index(&key);
assert_eq!(index.verify(unpacked.signature.as_ref().unwrap(), "acme.tools", "1.0.0", &digest).unwrap(), "acme");
// Changed bytes, another id, someone else's prefix, a revoked key.
assert!(index.verify(&sig, "acme.tools", "1.0.0", "0000").is_err());
assert!(index.verify(&sig, "acme.tools", "1.0.1", &digest).is_err());
let other = PublisherKey::from_seed([9; 32]).sign("acme", "acme.tools", "1.0.0", &digest);
assert!(index.verify(&other, "acme.tools", "1.0.0", &digest).unwrap_err().contains("not one of"));
let foreign = key.sign("acme", "evil.tools", "1.0.0", &digest);
assert!(index.verify(&foreign, "evil.tools", "1.0.0", &digest).unwrap_err().contains("acme.*"));
let mut revoked = index.clone();
revoked.publishers[0].keys[0].revoked = true;
assert!(revoked.verify(&sig, "acme.tools", "1.0.0", &digest).unwrap_err().contains("revoked"));
assert!(revoked.key_revoked(&sig.key));
assert_eq!(index.revocation("acme.bad", "1.0.0").unwrap().reason, "malware");
assert!(index.revocation("acme.bad", "1.0.1").is_none());
assert_eq!(PublisherKey::from_base64(&key.seed_base64()).unwrap().public_base64(), key.public_base64());
}
fn entry(id: &str, publisher: &str) -> IndexEntry {
IndexEntry {
id: id.into(),
name: id.into(),
version: "1.0.0".into(),
description: String::new(),
authors: Vec::new(),
publisher: publisher.into(),
download: format!("https://example.com/{id}.oiplugin"),
digest: "d".into(),
capabilities: Vec::new(),
engine: "^1.0".into(),
categories: Vec::new(),
}
}
fn version(id: &str, version: &str, engine: &str) -> IndexEntry {
IndexEntry { version: version.into(), engine: engine.into(), ..entry(id, "acme") }
}
#[test]
fn the_ide_is_offered_the_newest_version_it_can_load() {
let host = crate::ENGINE_VERSION;
let newer = semver::Version::parse(host).unwrap().major + 1;
let mut index = Index {
format: INDEX_FORMAT,
plugins: vec![
version("acme.a", "1.0.0", &format!("^{host}")),
version("acme.a", "1.2.0", &format!("^{host}")),
version("acme.a", "2.0.0", &format!("^{newer}.0")),
version("acme.b", "0.1.0", &format!("^{newer}.0")),
version("acme.b", "0.2.0", &format!("^{newer}.1")),
version("acme.c", "1.0.0", ""),
version("acme.c", "1.1.0", ""),
],
revoked: vec![Revocation { id: "acme.c".into(), versions: vec!["1.1.0".into()], reason: "r".into() }],
..Index::default()
};
let offered = |index: &Index| {
index.offered().into_iter().map(|(e, ok)| (e.id.clone(), e.version.clone(), ok)).collect::<Vec<_>>()
};
let expected = [
("acme.a".to_owned(), "1.2.0".to_owned(), true),
("acme.b".to_owned(), "0.2.0".to_owned(), false),
// No `engine`: listed, but no IDE installs it.
("acme.c".to_owned(), "1.0.0".to_owned(), false),
];
assert_eq!(offered(&index), expected);
// Pruning keeps the newest version per requirement, which changes nothing offered.
index.prune_versions();
let left: Vec<_> = index.plugins.iter().map(|e| format!("{} {}", e.id, e.version)).collect();
assert_eq!(left, ["acme.a 1.2.0", "acme.a 2.0.0", "acme.b 0.1.0", "acme.b 0.2.0", "acme.c 1.0.0"]);
assert_eq!(offered(&index), expected);
let built_for = semver::Version::new(1, 3, 0);
assert_eq!(engine_requirement("^1.0", Some(&built_for)), "^1.0, ^1.3");
assert_eq!(engine_requirement("^1.0", None), "^1.0");
assert!(semver::VersionReq::parse(&engine_requirement("^1.0", Some(&built_for))).is_ok());
}
fn publisher(id: &str) -> Publisher {
Publisher { id: id.into(), name: id.into(), keys: vec![KeyEntry::new(format!("{id}-key"))] }
}
fn revocation(id: &str) -> Revocation {
Revocation { id: id.into(), versions: Vec::new(), reason: "r".into() }
}
#[test]
fn including_an_index_replaces_everything_but_the_kept_publishers() {
let mut official = Index {
format: INDEX_FORMAT,
publishers: vec![publisher("openidea"), publisher("gone")],
plugins: vec![entry("openidea.a", "openidea"), entry("gone.x", "gone")],
revoked: vec![revocation("openidea.old"), revocation("gone.x")],
moved: None,
};
let community = Index {
format: INDEX_FORMAT,
publishers: vec![publisher("alice")],
plugins: vec![entry("alice.arthas", "alice")],
revoked: vec![revocation("alice.bad")],
moved: None,
};
official.include(community.clone(), &["openidea"]).unwrap();
let ids = |v: &[IndexEntry]| v.iter().map(|e| e.id.clone()).collect::<Vec<_>>();
assert_eq!(official.publishers.iter().map(|p| p.id.as_str()).collect::<Vec<_>>(), ["openidea", "alice"]);
assert_eq!(ids(&official.plugins), ["openidea.a", "alice.arthas"]);
assert_eq!(official.revoked.iter().map(|r| r.id.as_str()).collect::<Vec<_>>(), ["openidea.old", "alice.bad"]);
// Including again is idempotent.
let once = official.clone();
official.include(community.clone(), &["openidea"]).unwrap();
assert_eq!(official, once);
}
#[test]
fn an_included_index_speaks_only_for_its_own_publishers() {
let base = Index { format: INDEX_FORMAT, publishers: vec![publisher("openidea")], ..Index::default() };
let with = |publishers: Vec<Publisher>, plugins: Vec<IndexEntry>, revoked: Vec<Revocation>| Index {
format: INDEX_FORMAT,
publishers,
plugins,
revoked,
moved: None,
};
let cases = [
(with(vec![publisher("openidea")], vec![], vec![]), "may not list"),
(with(vec![publisher("alice"), publisher("alice")], vec![], vec![]), "listed twice"),
(with(vec![publisher("openidea.x")], vec![], vec![]), "invalid publisher"),
(with(vec![publisher("alice")], vec![entry("openidea.a", "alice")], vec![]), "not a plugin of"),
(with(vec![publisher("alice")], vec![entry("alice.a", "bob")], vec![]), "not a plugin of"),
(with(vec![publisher("alice")], vec![entry("alicex.a", "alice")], vec![]), "not a plugin of"),
(with(vec![publisher("alice")], vec![], vec![revocation("openidea.a")]), "doesn't publish"),
(
with(
vec![publisher("alice")],
vec![IndexEntry { engine: String::new(), ..entry("alice.a", "alice") }],
vec![],
),
"`engine`",
),
];
for (other, error) in cases {
let mut index = base.clone();
let err = index.include(other, &["openidea"]).unwrap_err();
assert!(err.contains(error), "{err}");
assert_eq!(index, base);
}
}
#[test]
fn unpacking_refuses_paths_leaving_the_package() {
let tmp = tempfile::tempdir().unwrap();
let out = tmp.path().join("evil.oiplugin");
let mut zip = zip::ZipWriter::new(std::fs::File::create(&out).unwrap());
let o = zip::write::SimpleFileOptions::default();
zip.start_file(MANIFEST_FILE, o).unwrap();
zip.write_all(b"id = \"x\"").unwrap();
zip.start_file("../outside.txt", o).unwrap();
zip.write_all(b"x").unwrap();
zip.finish().unwrap();
let err = unpack(&out, &tmp.path().join("dest")).unwrap_err();
assert!(err.contains("invalid path"), "{err}");
assert!(!tmp.path().join("outside.txt").exists());
assert!(Index::parse(r#"{"format": 2}"#).unwrap().is_newer());
}
fn zip_of(path: &Path, entries: &[(&str, &str)]) {
let mut zip = zip::ZipWriter::new(std::fs::File::create(path).unwrap());
for (name, body) in entries {
zip.start_file(*name, zip::write::SimpleFileOptions::default()).unwrap();
zip.write_all(body.as_bytes()).unwrap();
}
zip.finish().unwrap();
}
#[test]
fn a_zip_holds_the_package_at_its_root_or_in_its_only_folder() {
let tmp = tempfile::tempdir().unwrap();
let manifest = "id = \"acme.tools\"";
let at_root = tmp.path().join("root.zip");
zip_of(&at_root, &[(MANIFEST_FILE, manifest), ("dist/plugin.wasm", "w"), (".DS_Store", "x")]);
let got = unpack(&at_root, &tmp.path().join("a")).unwrap();
assert_eq!(got.files, [PathBuf::from("dist/plugin.wasm")]);
// What zipping the folder in Finder makes: the folder plus `__MACOSX`.
let folder = tmp.path().join("folder.zip");
zip_of(
&folder,
&[
("acme-tools/plugin.toml", manifest),
("acme-tools/dist/plugin.wasm", "w"),
("__MACOSX/acme-tools/._plugin.toml", "x"),
],
);
let dest = tmp.path().join("b");
let got = unpack(&folder, &dest).unwrap();
assert_eq!(got.files, [PathBuf::from("dist/plugin.wasm")]);
assert!(dest.join(MANIFEST_FILE).is_file());
assert!(!dest.join("__MACOSX").exists() && !dest.join("acme-tools").exists());
// Two folders, a manifest one level too deep, or none (an IntelliJ plugin): not a package.
for (name, entries) in [
("two.zip", &[("a/plugin.toml", manifest), ("b/x.txt", "x")][..]),
("deep.zip", &[("a/b/plugin.toml", manifest)][..]),
("intellij.zip", &[("my-plugin/lib/my-plugin.jar", "jar")][..]),
] {
let path = tmp.path().join(name);
zip_of(&path, entries);
let err = unpack(&path, &tmp.path().join(name).with_extension("out")).unwrap_err();
assert!(err.contains("is not an OpenIDEA plugin"), "{name}: {err}");
}
}
#[test]
fn a_newer_index_still_revokes_and_moves() {
let newer = format!(
r#"{{"format": {}, "plugins": [{{"anything": "new"}}], "shape": "changed",
"publishers": [{{"id": "acme", "name": "ACME", "keys": [{{"key": "k1", "revoked": true}}]}}, {{"id": 5}}],
"revoked": [{{"id": "acme.bad", "reason": "malware"}}, {{"oops": 1}}],
"moved": "https://example.com/next.json"}}"#,
INDEX_FORMAT + 1
);
let index = Index::parse(&newer).unwrap();
assert!(index.is_newer());
assert!(index.plugins.is_empty() && index.offered().is_empty());
assert_eq!(index.revocation("acme.bad", "2.0.0").unwrap().reason, "malware");
assert!(index.key_revoked("k1"));
assert_eq!(index.moved.as_deref(), Some("https://example.com/next.json"));
// Fields this IDE doesn't know are ignored in its own format.
let current = format!(r#"{{"format": {INDEX_FORMAT}, "later": true, "plugins": []}}"#);
assert!(!Index::parse(&current).unwrap().is_newer());
assert!(Index::parse(r#"{"format": 0}"#).unwrap_err().contains("not supported"));
assert!(Index::parse(r#"{"plugins": []}"#).is_err());
}
fn keys<'a>(index: &'a mut Index, publisher: &str) -> &'a mut Vec<KeyEntry> {
&mut index.publishers.iter_mut().find(|p| p.id == publisher).unwrap().keys
}
#[test]
fn rotated_keys_continue_through_endorsements() {
let (k1, k2, k3) =
(PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([2; 32]), PublisherKey::from_seed([3; 32]));
let mut index = index(&k1);
let endorsed = |by: &PublisherKey, key: &PublisherKey| KeyEntry {
endorsement: Some(by.endorse("acme", &key.public_base64())),
..KeyEntry::new(key.public_base64())
};
keys(&mut index, "acme").push(endorsed(&k1, &k2));
keys(&mut index, "acme").push(endorsed(&k2, &k3));
let (p1, p2, p3) = (k1.public_base64(), k2.public_base64(), k3.public_base64());
assert!(index.succeeds("acme", &p1, &p1));
assert!(index.succeeds("acme", &p1, &p3), "through k2");
assert!(!index.succeeds("acme", &p3, &p1), "not backwards");
assert!(!index.succeeds("other", &p1, &p2));
// An endorsement for another publisher, or by a revoked key, doesn't count.
let mut foreign = index.clone();
keys(&mut foreign, "acme")[1].endorsement = Some(k1.endorse("evil", &p2));
assert!(!foreign.succeeds("acme", &p1, &p2));
let mut stolen = index.clone();
keys(&mut stolen, "acme")[0].revoked = true;
assert!(!stolen.succeeds("acme", &p1, &p2));
let sig = |key: &PublisherKey| key.sign("acme", "acme.tools", "1.0.0", "d");
assert!(signer_continues(Some(&index), Some(&sig(&k1)), Some(&sig(&k3))));
assert!(signer_continues(None, Some(&sig(&k1)), Some(&sig(&k1))));
assert!(!signer_continues(None, Some(&sig(&k1)), Some(&sig(&k2))), "no index to vouch");
let stranger = PublisherKey::from_seed([4; 32]);
assert!(!signer_continues(Some(&index), Some(&sig(&k1)), Some(&sig(&stranger))));
assert!(!signer_continues(Some(&index), Some(&sig(&k1)), None), "signed, then unsigned");
assert!(signer_continues(Some(&index), None, Some(&sig(&stranger))), "unsigned pins nothing");
}
#[test]
fn pinned_publishers_keep_their_built_in_keys() {
let (root, next, intruder) =
(PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([2; 32]), PublisherKey::from_seed([5; 32]));
let mut index = index(&intruder);
index.publishers[0].id = "openidea".into();
keys(&mut index, "openidea").push(KeyEntry {
endorsement: Some(root.endorse("openidea", &next.public_base64())),
..KeyEntry::new(next.public_base64())
});
let roots = [root.public_base64()];
let listed = |index: &Index| -> Vec<(String, bool)> {
let mut index = index.clone();
index.pin_publisher("openidea", "OpenIDEA", &roots);
keys(&mut index, "openidea").iter().map(|k| (k.key.clone(), k.revoked)).collect()
};
// The intruder's key goes, the successor the root endorsed stays, the root is added.
assert_eq!(listed(&index), [(next.public_base64(), false), (root.public_base64(), false)]);
let mut pinned = index.clone();
pinned.pin_publisher("openidea", "OpenIDEA", &roots);
let digest = "d";
let signed = |k: &PublisherKey| k.sign("openidea", "openidea.tools", "1.0.0", digest);
assert!(pinned.verify(&signed(&intruder), "openidea.tools", "1.0.0", digest).is_err());
assert!(pinned.verify(&signed(&next), "openidea.tools", "1.0.0", digest).is_ok());
// A revoked root stays revoked, and what it endorsed is no longer trusted.
keys(&mut index, "openidea").push(KeyEntry { revoked: true, ..KeyEntry::new(root.public_base64()) });
assert_eq!(listed(&index), [(root.public_base64(), true)]);
// A publisher missing from the index is added with its roots.
let mut empty = Index { format: INDEX_FORMAT, ..Index::default() };
empty.pin_publisher("openidea", "OpenIDEA", &[root.public_base64()]);
assert_eq!(empty.verify(&signed(&root), "openidea.tools", "1.0.0", digest).unwrap(), "openidea");
// No roots: the index decides.
let mut open = index.clone();
open.pin_publisher("openidea", "OpenIDEA", &[]);
assert_eq!(open, index);
}
}
+175
View File
@@ -0,0 +1,175 @@
//! The extension point contract (plan 4.1).
use crate::capability::Capability;
use crate::manifest::{Manifest, check_relative};
use std::path::{Path, PathBuf};
use std::sync::Arc;
/// Where a contribution takes effect (plan 3, placement rules).
#[derive(Clone, Copy, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub enum Placement {
/// Projected to the client (menus, keybindings, themes, icons).
Ui,
/// Takes effect on the machine the project is on (languages, LSP, run configurations).
Workspace,
}
/// The package a declaration comes from, as extension points see it.
#[derive(Clone, Debug)]
pub struct Package {
pub manifest: Arc<Manifest>,
/// Package directory.
pub dir: PathBuf,
/// Hash of the package's files when they never change (an installed
/// copy): validated declarations are then reused from the
/// [`ContributionIndex`](crate::ContributionIndex).
pub digest: Option<Arc<str>>,
}
/// Largest single package file an extension point reads.
pub const MAX_PACKAGE_FILE: u64 = 8 << 20;
impl Package {
pub fn new(manifest: Manifest, dir: PathBuf) -> Self {
Self { manifest: Arc::new(manifest), dir, digest: None }
}
pub fn with_digest(mut self, digest: &str) -> Self {
self.digest = Some(digest.into());
self
}
pub fn id(&self) -> &str {
&self.manifest.id
}
/// Resolves a package-relative path, refusing anything that leaves the
/// package (`..`, absolute paths, symlinks pointing outside).
pub fn resolve(&self, rel: &Path) -> Result<PathBuf, String> {
check_relative(rel, "path")?;
let joined = self.dir.join(rel);
let real = joined.canonicalize().map_err(|_| format!("{}: no such file in the plugin", rel.display()))?;
let root = self.dir.canonicalize().map_err(|e| format!("plugin directory: {e}"))?;
if !real.starts_with(&root) {
return Err(format!("{}: points outside the plugin", rel.display()));
}
Ok(real)
}
pub fn read_bytes(&self, rel: &Path) -> Result<Vec<u8>, String> {
let real = self.resolve(rel)?;
let meta = std::fs::metadata(&real).map_err(|e| format!("{}: {e}", rel.display()))?;
if !meta.is_file() {
return Err(format!("{}: not a file", rel.display()));
}
if meta.len() > MAX_PACKAGE_FILE {
return Err(format!("{}: larger than {} MiB", rel.display(), MAX_PACKAGE_FILE >> 20));
}
std::fs::read(&real).map_err(|e| format!("{}: {e}", rel.display()))
}
pub fn read_text(&self, rel: &Path) -> Result<String, String> {
String::from_utf8(self.read_bytes(rel)?).map_err(|_| format!("{}: not UTF-8 text", rel.display()))
}
}
/// Undoes one committed contribution. Dropping without [`dispose`](Self::dispose) also disposes.
#[must_use = "dropping a Disposable undoes the contribution"]
pub struct Disposable(Option<Box<dyn FnOnce() + Send>>);
impl Disposable {
pub fn new(f: impl FnOnce() + Send + 'static) -> Self {
Self(Some(Box::new(f)))
}
/// A contribution with nothing to undo.
pub fn noop() -> Self {
Self(None)
}
pub fn dispose(mut self) {
if let Some(f) = self.0.take() {
f();
}
}
}
impl Drop for Disposable {
fn drop(&mut self) {
if let Some(f) = self.0.take() {
f();
}
}
}
impl std::fmt::Debug for Disposable {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("Disposable")
}
}
pub type Commit = Box<dyn FnOnce() -> Result<Disposable, String> + Send>;
/// A validated declaration, ready to take effect.
pub struct Prepared {
/// Capabilities this declaration needs (a language with an LSP → `process:<binary>`).
pub capabilities: Vec<Capability>,
/// Package files the declaration references (copied on install).
pub files: Vec<PathBuf>,
/// Identity within the point: a later declaration with the same key replaces
/// the earlier one of the same plugin (re-registering a command).
pub key: Option<String>,
/// User-facing one-liner for the plugin details page ("Insert Date").
pub label: String,
/// Makes it take effect. Must not fail for reasons `prepare` could have checked.
pub commit: Commit,
}
impl Prepared {
pub fn new(label: impl Into<String>, commit: impl FnOnce() -> Result<Disposable, String> + Send + 'static) -> Self {
Self { capabilities: Vec::new(), files: Vec::new(), key: None, label: label.into(), commit: Box::new(commit) }
}
pub fn key(mut self, key: impl Into<String>) -> Self {
self.key = Some(key.into());
self
}
pub fn file(mut self, path: impl Into<PathBuf>) -> Self {
self.files.push(path.into());
self
}
pub fn capability(mut self, cap: Capability) -> Self {
if !self.capabilities.contains(&cap) {
self.capabilities.push(cap);
}
self
}
}
impl std::fmt::Debug for Prepared {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("Prepared").field("key", &self.key).field("label", &self.label).finish_non_exhaustive()
}
}
/// A kind of contribution (`contributes.<id>` in the manifest). Implemented by
/// the crate that owns the feature; registered once in an [`crate::ExtensionRegistry`].
pub trait ExtensionPoint: Send + Sync {
/// `"languages"`, `"iconThemes"`, …
fn id(&self) -> &'static str;
fn placement(&self) -> Placement;
/// Parses and validates one declaration without side effects, and works
/// out the capabilities it needs. `Ui` points validate completely here: the
/// client's projection is not allowed to fail.
fn prepare(&self, pkg: &Package, decl: toml::Value) -> Result<Prepared, String>;
}
/// Deserializes a declaration with serde, prefixing errors with the point id.
pub fn decode<T: serde::de::DeserializeOwned>(point: &str, decl: toml::Value) -> Result<T, String> {
decl.try_into().map_err(|e: toml::de::Error| format!("contributes.{point}: {}", e.message()))
}
+305
View File
@@ -0,0 +1,305 @@
//! Extension points plus the contributions committed to them, accounted per plugin.
use crate::capability::Capability;
use crate::point::{Disposable, ExtensionPoint, Package, Placement, Prepared};
use parking_lot::{Mutex, RwLock};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::collections::BTreeMap;
use std::path::PathBuf;
use std::sync::Arc;
/// A contribution as the plugin details page lists it.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct ContributionInfo {
pub point: String,
pub label: String,
}
struct Entry {
point: &'static str,
key: Option<String>,
label: String,
disposable: Disposable,
}
/// Every declaration of a package, validated; nothing has taken effect yet.
pub struct PreparedSet {
pub plugin: String,
/// Union of what the declarations need (beyond the manifest's own list).
pub capabilities: Vec<Capability>,
/// Package files the declarations reference.
pub files: Vec<PathBuf>,
/// Declarations for extension points this host does not know (skipped).
pub warnings: Vec<String>,
items: Vec<(&'static str, Prepared)>,
}
impl PreparedSet {
pub fn is_empty(&self) -> bool {
self.items.is_empty()
}
pub fn len(&self) -> usize {
self.items.len()
}
}
impl std::fmt::Debug for PreparedSet {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("PreparedSet")
.field("plugin", &self.plugin)
.field("items", &self.items.len())
.field("warnings", &self.warnings)
.finish()
}
}
#[derive(Default)]
pub struct ExtensionRegistry {
points: RwLock<BTreeMap<&'static str, Arc<dyn ExtensionPoint>>>,
/// plugin id → its contributions in commit order.
entries: Mutex<BTreeMap<String, Vec<Entry>>>,
}
impl ExtensionRegistry {
pub fn new() -> Self {
Self::default()
}
/// Adds an extension point. Ids are unique.
pub fn register(&self, point: Arc<dyn ExtensionPoint>) -> Result<(), String> {
let id = point.id();
crate::manifest::validate_point_id(id)?;
let mut points = self.points.write();
if points.contains_key(id) {
return Err(format!("extension point {id:?} is already registered"));
}
points.insert(id, point);
Ok(())
}
pub fn point_ids(&self) -> Vec<&'static str> {
self.points.read().keys().copied().collect()
}
pub fn placement(&self, point: &str) -> Option<Placement> {
self.points.read().get(point).map(|p| p.placement())
}
fn point(&self, id: &str) -> Option<Arc<dyn ExtensionPoint>> {
self.points.read().get(id).cloned()
}
/// Validates every `contributes.<point>` declaration of `pkg`. Fails as a
/// whole on the first invalid declaration; unknown points are skipped with
/// a warning (forward compatibility).
pub fn prepare(&self, pkg: &Package) -> Result<PreparedSet, String> {
let mut set = PreparedSet {
plugin: pkg.id().to_owned(),
capabilities: Vec::new(),
files: Vec::new(),
warnings: Vec::new(),
items: Vec::new(),
};
for point_id in pkg.manifest.contributes.keys() {
let Some(point) = self.point(point_id) else {
let msg = format!("contributes.{point_id}: not supported by this version of the IDE; skipped");
tracing::warn!(plugin = %pkg.id(), "{msg}");
set.warnings.push(msg);
continue;
};
for (i, decl) in pkg.manifest.declarations(point_id).into_iter().enumerate() {
let prepared = point
.prepare(pkg, decl)
.map_err(|e| format!("contributes.{point_id}[{i}]: {}", strip_prefix(&e, point_id)))?;
for cap in &prepared.capabilities {
if !set.capabilities.contains(cap) {
set.capabilities.push(cap.clone());
}
}
for f in &prepared.files {
if !set.files.contains(f) {
set.files.push(f.clone());
}
}
set.items.push((point.id(), prepared));
}
}
Ok(set)
}
/// Makes a prepared set take effect, all or nothing: if one commit fails,
/// the ones before it are disposed in reverse order.
pub fn commit(&self, set: PreparedSet) -> Result<(), String> {
let mut committed: Vec<Entry> = Vec::with_capacity(set.items.len());
for (point, prepared) in set.items {
let Prepared { key, label, commit, .. } = prepared;
match commit() {
Ok(disposable) => committed.push(Entry { point, key, label, disposable }),
Err(e) => {
tracing::error!(plugin = %set.plugin, "contributes.{point}: commit failed: {e}");
while let Some(entry) = committed.pop() {
entry.disposable.dispose();
}
return Err(format!("contributes.{point}: {e}"));
}
}
}
let replaced = {
let mut entries = self.entries.lock();
let list = entries.entry(set.plugin).or_default();
let mut replaced = Vec::new();
for new in &committed {
if let Some(key) = &new.key
&& let Some(pos) = list.iter().position(|e| e.point == new.point && e.key.as_ref() == Some(key))
{
replaced.push(list.remove(pos));
}
}
list.extend(committed);
replaced
};
for old in replaced.into_iter().rev() {
old.disposable.dispose();
}
Ok(())
}
/// Withdraws everything `plugin` contributed, newest first.
pub fn dispose_plugin(&self, plugin: &str) -> bool {
let list = self.entries.lock().remove(plugin).unwrap_or_default();
let any = !list.is_empty();
for entry in list.into_iter().rev() {
entry.disposable.dispose();
}
any
}
pub fn contributions(&self, plugin: &str) -> Vec<ContributionInfo> {
self.entries
.lock()
.get(plugin)
.map(|list| {
list.iter().map(|e| ContributionInfo { point: e.point.into(), label: e.label.clone() }).collect()
})
.unwrap_or_default()
}
/// How many contributions `plugin` has at `point`.
pub fn count(&self, plugin: &str, point: &str) -> usize {
self.entries.lock().get(plugin).map_or(0, |list| list.iter().filter(|e| e.point == point).count())
}
/// Plugins with at least one contribution.
pub fn plugins(&self) -> Vec<String> {
self.entries.lock().iter().filter(|(_, l)| !l.is_empty()).map(|(id, _)| id.clone()).collect()
}
}
fn strip_prefix<'a>(e: &'a str, point: &str) -> &'a str {
e.strip_prefix(&format!("contributes.{point}: ")).unwrap_or(e)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::manifest::Manifest;
use std::sync::atomic::{AtomicUsize, Ordering};
/// Records live items in a shared list; `fail = true` makes commit fail.
struct Recording {
id: &'static str,
live: Arc<Mutex<Vec<String>>>,
commits: Arc<AtomicUsize>,
}
impl ExtensionPoint for Recording {
fn id(&self) -> &'static str {
self.id
}
fn placement(&self) -> Placement {
Placement::Ui
}
fn prepare(&self, _pkg: &Package, decl: toml::Value) -> Result<Prepared, String> {
let name = decl.get("name").and_then(|v| v.as_str()).ok_or("missing name")?.to_owned();
let fail = decl.get("fail").and_then(|v| v.as_bool()).unwrap_or(false);
let (live, commits) = (self.live.clone(), self.commits.clone());
let label = name.clone();
let mut p = Prepared::new(label, move || {
commits.fetch_add(1, Ordering::SeqCst);
if fail {
return Err("boom".into());
}
live.lock().push(name.clone());
Ok(Disposable::new(move || live.lock().retain(|n| *n != name)))
});
if let Some(f) = decl.get("file").and_then(|v| v.as_str()) {
p = p.file(f);
}
if decl.get("key").and_then(|v| v.as_bool()).unwrap_or(false) {
p = p.key("k");
}
Ok(p)
}
}
fn setup() -> (ExtensionRegistry, Arc<Mutex<Vec<String>>>) {
let reg = ExtensionRegistry::new();
let live = Arc::new(Mutex::new(Vec::new()));
let commits = Arc::new(AtomicUsize::new(0));
reg.register(Arc::new(Recording { id: "things", live: live.clone(), commits: commits.clone() })).unwrap();
reg.register(Arc::new(Recording { id: "others", live: live.clone(), commits })).unwrap();
(reg, live)
}
fn pkg(contributes: &str) -> Package {
let src = format!("id = \"p\"\nname = \"P\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"\n{contributes}");
Package::new(Manifest::parse(&src).unwrap(), PathBuf::from("/nonexistent"))
}
#[test]
fn commits_and_disposes_per_plugin() {
let (reg, live) = setup();
assert!(
reg.register(Arc::new(Recording { id: "things", live: live.clone(), commits: Default::default() }))
.is_err()
);
let p = pkg(
"[[contributes.things]]\nname = \"a\"\nfile = \"x.json\"\n[[contributes.things]]\nname = \"b\"\n[contributes.others]\nname = \"c\"",
);
let set = reg.prepare(&p).unwrap();
assert_eq!(set.len(), 3);
assert_eq!(set.files, [PathBuf::from("x.json")]);
assert!(live.lock().is_empty(), "prepare has no side effects");
reg.commit(set).unwrap();
assert_eq!(*live.lock(), ["c", "a", "b"]);
assert_eq!(reg.count("p", "things"), 2);
assert_eq!(reg.contributions("p").len(), 3);
assert!(reg.dispose_plugin("p"));
assert!(live.lock().is_empty());
}
#[test]
fn a_failed_commit_rolls_back() {
let (reg, live) = setup();
let p = pkg("[[contributes.things]]\nname = \"a\"\n[[contributes.things]]\nname = \"b\"\nfail = true");
let err = reg.commit(reg.prepare(&p).unwrap()).unwrap_err();
assert!(err.contains("boom"));
assert!(live.lock().is_empty(), "a was undone");
assert!(reg.contributions("p").is_empty());
}
#[test]
fn invalid_declarations_fail_prepare_and_unknown_points_warn() {
let (reg, _) = setup();
let err = reg.prepare(&pkg("[[contributes.things]]\nnope = 1")).unwrap_err();
assert_eq!(err, "contributes.things[0]: missing name");
let set =
reg.prepare(&pkg("[[contributes.futureThing]]\nx = 1\n[[contributes.things]]\nname = \"a\"")).unwrap();
assert_eq!(set.len(), 1);
assert_eq!(set.warnings.len(), 1);
assert!(set.warnings[0].contains("futureThing"));
}
}
+247
View File
@@ -0,0 +1,247 @@
//! UI contributions: validated declarations projected to the client.
//!
//! A `Ui` extension point validates a declaration completely in Rust and
//! commits it as JSON into the [`UiStore`]; the client receives the store's
//! snapshot and projects each item onto an existing `register*` function
//! (`defineUiPoint` in `src/features/plugins/projection.ts`). Large payloads
//! (icon SVGs) go into the store's assets and are fetched on demand.
use crate::index::ContributionIndex;
use crate::point::{Disposable, ExtensionPoint, Package, Placement, Prepared};
use parking_lot::RwLock;
use serde::{Deserialize, Serialize};
use specta::Type;
use std::collections::BTreeMap;
use std::path::PathBuf;
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};
/// One projected declaration.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct UiContribution {
pub plugin_id: String,
/// Extension point id (`commands`, `iconThemes`, …).
pub point: String,
/// Stable within (plugin, point): the client keys its registrations on it.
pub key: String,
/// The validated declaration, shaped by the point.
#[specta(type = specta_typescript::Unknown)]
pub decl: serde_json::Value,
}
struct Item {
seq: u64,
contribution: UiContribution,
}
/// Committed UI contributions of all plugins.
#[derive(Default)]
pub struct UiStore {
items: RwLock<Vec<Item>>,
/// (plugin, asset id) → content.
assets: RwLock<BTreeMap<(String, String), Arc<str>>>,
seq: AtomicU64,
revision: AtomicU64,
index: RwLock<Option<Arc<ContributionIndex>>>,
}
impl UiStore {
pub fn new() -> Arc<Self> {
Arc::new(Self::default())
}
/// Adds an item; the returned disposable removes it.
pub fn insert(self: &Arc<Self>, contribution: UiContribution) -> Disposable {
let seq = self.seq.fetch_add(1, Ordering::Relaxed);
self.items.write().push(Item { seq, contribution });
self.revision.fetch_add(1, Ordering::Release);
let store = Arc::downgrade(self);
Disposable::new(move || {
if let Some(store) = store.upgrade() {
store.items.write().retain(|i| i.seq != seq);
store.revision.fetch_add(1, Ordering::Release);
}
})
}
/// Stores assets of a plugin (ids are unique per plugin; later ones replace).
pub fn insert_assets(self: &Arc<Self>, plugin: &str, assets: Vec<(String, String)>) -> Disposable {
let keys: Vec<(String, String)> = assets.iter().map(|(id, _)| (plugin.to_owned(), id.clone())).collect();
{
let mut map = self.assets.write();
for (id, content) in assets {
map.insert((plugin.to_owned(), id), Arc::from(content));
}
}
let store = Arc::downgrade(self);
Disposable::new(move || {
if let Some(store) = store.upgrade() {
let mut map = store.assets.write();
for k in keys {
map.remove(&k);
}
}
})
}
pub fn asset(&self, plugin: &str, id: &str) -> Option<Arc<str>> {
self.assets.read().get(&(plugin.to_owned(), id.to_owned())).cloned()
}
/// Assets of `plugin` whose id starts with `prefix`.
pub fn assets_with_prefix(&self, plugin: &str, prefix: &str) -> Vec<(String, Arc<str>)> {
self.assets
.read()
.range((plugin.to_owned(), prefix.to_owned())..)
.take_while(|((p, id), _)| p == plugin && id.starts_with(prefix))
.map(|((_, id), v)| (id.clone(), v.clone()))
.collect()
}
/// Where [`UiPoint`]s keep validated declarations of installed packages.
pub fn set_index(&self, index: Option<Arc<ContributionIndex>>) {
*self.index.write() = index;
}
/// Bumped on every change: clients compare it to skip re-projection.
pub fn revision(&self) -> u64 {
self.revision.load(Ordering::Acquire)
}
pub fn snapshot(&self) -> Vec<UiContribution> {
self.items.read().iter().map(|i| i.contribution.clone()).collect()
}
/// Items of one point, optionally of one plugin.
pub fn items(&self, point: &str, plugin: Option<&str>) -> Vec<UiContribution> {
self.items
.read()
.iter()
.filter(|i| i.contribution.point == point && plugin.is_none_or(|p| i.contribution.plugin_id == p))
.map(|i| i.contribution.clone())
.collect()
}
}
/// What a [`UiPoint`]'s validator produces for one declaration.
#[derive(Debug, Serialize, Deserialize)]
pub struct UiDecl {
pub key: String,
pub label: String,
pub json: serde_json::Value,
/// Package files referenced (copied on install).
pub files: Vec<PathBuf>,
/// Large validated payloads fetched separately by the client.
pub assets: Vec<(String, String)>,
}
impl UiDecl {
pub fn new(key: impl Into<String>, label: impl Into<String>, json: serde_json::Value) -> Self {
Self { key: key.into(), label: label.into(), json, files: Vec::new(), assets: Vec::new() }
}
}
type Validate = dyn Fn(&Package, toml::Value) -> Result<UiDecl, String> + Send + Sync;
/// A `Ui` extension point defined by a validator: committing stores the
/// validated JSON (and assets) in the [`UiStore`]. Most UI contribution types
/// (menus, keybindings, status bar items, icon themes) are one of these.
pub struct UiPoint {
id: &'static str,
store: Arc<UiStore>,
validate: Box<Validate>,
}
impl UiPoint {
pub fn new(
id: &'static str,
store: Arc<UiStore>,
validate: impl Fn(&Package, toml::Value) -> Result<UiDecl, String> + Send + Sync + 'static,
) -> Self {
Self { id, store, validate: Box::new(validate) }
}
}
impl ExtensionPoint for UiPoint {
fn id(&self) -> &'static str {
self.id
}
fn placement(&self) -> Placement {
Placement::Ui
}
fn prepare(&self, pkg: &Package, decl: toml::Value) -> Result<Prepared, String> {
let index = pkg.digest.as_ref().and_then(|_| self.store.index.read().clone());
let ui = match index.as_ref().and_then(|i| i.get(pkg, self.id, &decl)) {
Some(ui) => ui,
None => match &index {
Some(i) => {
let ui = (self.validate)(pkg, decl.clone())?;
i.put(pkg, self.id, &decl, &ui);
ui
}
None => (self.validate)(pkg, decl)?,
},
};
let UiDecl { key, label, json, files, assets } = ui;
let store = self.store.clone();
let contribution =
UiContribution { plugin_id: pkg.id().to_owned(), point: self.id.to_owned(), key: key.clone(), decl: json };
let plugin = pkg.id().to_owned();
let mut prepared = Prepared::new(label, move || {
let item = store.insert(contribution);
if assets.is_empty() {
return Ok(item);
}
let assets = store.insert_assets(&plugin, assets);
Ok(Disposable::new(move || {
item.dispose();
assets.dispose();
}))
})
.key(key);
prepared.files = files;
Ok(prepared)
}
}
/// TOML declaration → JSON for the client.
pub fn to_json(v: &toml::Value) -> serde_json::Value {
serde_json::to_value(v).unwrap_or(serde_json::Value::Null)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::manifest::Manifest;
use crate::registry::ExtensionRegistry;
#[test]
fn ui_point_projects_into_the_store() {
let store = UiStore::new();
let reg = ExtensionRegistry::new();
reg.register(Arc::new(UiPoint::new("badges", store.clone(), |_pkg, decl| {
let text = decl.get("text").and_then(|v| v.as_str()).ok_or("missing text")?.to_owned();
let mut d = UiDecl::new(text.clone(), text.clone(), serde_json::json!({ "text": text }));
d.assets.push((format!("badge/{text}"), "<svg/>".into()));
Ok(d)
})))
.unwrap();
let src = "id = \"p\"\nname = \"P\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"\n\
[[contributes.badges]]\ntext = \"hi\"";
let pkg = Package::new(Manifest::parse(src).unwrap(), PathBuf::from("/x"));
let rev = store.revision();
reg.commit(reg.prepare(&pkg).unwrap()).unwrap();
assert!(store.revision() > rev);
let items = store.items("badges", Some("p"));
assert_eq!(items.len(), 1);
assert_eq!(items[0].decl["text"], "hi");
assert_eq!(store.asset("p", "badge/hi").as_deref(), Some("<svg/>"));
assert_eq!(store.assets_with_prefix("p", "badge/").len(), 1);
reg.dispose_plugin("p");
assert!(store.snapshot().is_empty());
assert!(store.asset("p", "badge/hi").is_none());
}
}
+136
View File
@@ -0,0 +1,136 @@
//! Syntax check for `when` clauses in UI declarations. The client evaluates
//! them (`src/platform/context.ts`); validating here keeps the client's
//! projection from failing. Grammar (same as the client):
//!
//! ```text
//! expr := or ; or := and ("||" and)* ; and := unary ("&&" unary)*
//! unary := "!" unary | "(" expr ")" | cmp ; cmp := key (("==" | "!=") value)?
//! ```
const MAX_LEN: usize = 512;
#[derive(Debug, PartialEq)]
enum Tok<'a> {
Id(&'a str),
Str,
Op(&'a str),
}
fn tokenize(src: &str) -> Result<Vec<Tok<'_>>, String> {
let mut out = Vec::new();
let b = src.as_bytes();
let mut i = 0;
while i < b.len() {
let c = b[i] as char;
if c.is_whitespace() {
i += 1;
} else if ["&&", "||", "==", "!="].iter().any(|op| src[i..].starts_with(op)) {
out.push(Tok::Op(&src[i..i + 2]));
i += 2;
} else if matches!(c, '!' | '(' | ')') {
out.push(Tok::Op(&src[i..i + 1]));
i += 1;
} else if c == '\'' || c == '"' {
let end = src[i + 1..].find(c).ok_or_else(|| format!("unterminated string in when clause: {src}"))?;
out.push(Tok::Str);
i += end + 2;
} else {
let len = src[i..]
.find(|ch: char| !(ch.is_alphanumeric() || matches!(ch, '_' | '.' | '-' | ':' | '/')))
.unwrap_or(src.len() - i);
if len == 0 {
return Err(format!("unexpected '{c}' in when clause: {src}"));
}
out.push(Tok::Id(&src[i..i + len]));
i += len;
}
}
Ok(out)
}
/// Checks a `when` clause; empty means "always".
pub fn validate_when(src: &str) -> Result<(), String> {
if src.len() > MAX_LEN {
return Err(format!("when clause longer than {MAX_LEN} bytes"));
}
if src.trim().is_empty() {
return Ok(());
}
let toks = tokenize(src)?;
let mut p = 0;
parse_or(&toks, &mut p, src)?;
if p != toks.len() {
return Err(format!("unexpected token in when clause: {src}"));
}
Ok(())
}
fn parse_or(t: &[Tok], p: &mut usize, src: &str) -> Result<(), String> {
parse_and(t, p, src)?;
while t.get(*p) == Some(&Tok::Op("||")) {
*p += 1;
parse_and(t, p, src)?;
}
Ok(())
}
fn parse_and(t: &[Tok], p: &mut usize, src: &str) -> Result<(), String> {
parse_unary(t, p, src)?;
while t.get(*p) == Some(&Tok::Op("&&")) {
*p += 1;
parse_unary(t, p, src)?;
}
Ok(())
}
fn parse_unary(t: &[Tok], p: &mut usize, src: &str) -> Result<(), String> {
match t.get(*p) {
Some(Tok::Op("!")) => {
*p += 1;
parse_unary(t, p, src)
}
Some(Tok::Op("(")) => {
*p += 1;
parse_or(t, p, src)?;
if t.get(*p) != Some(&Tok::Op(")")) {
return Err(format!("missing ')' in when clause: {src}"));
}
*p += 1;
Ok(())
}
Some(Tok::Id(_)) => {
*p += 1;
if matches!(t.get(*p), Some(Tok::Op("==" | "!="))) {
*p += 1;
match t.get(*p) {
Some(Tok::Id(_) | Tok::Str) => *p += 1,
_ => return Err(format!("missing value after comparison in when clause: {src}")),
}
}
Ok(())
}
_ => Err(format!("expected a context key in when clause: {src}")),
}
}
#[cfg(test)]
mod tests {
use super::validate_when;
#[test]
fn accepts_the_client_grammar() {
for ok in [
"",
"editorFocus",
"editorLang == rust",
"editorLang != 'rust' && !readOnly",
"(a || b) && toolWindow.project.active",
"resourceScheme == file:/x",
] {
assert!(validate_when(ok).is_ok(), "{ok}");
}
for bad in ["a &&", "(a", "a == ", "== a", "a b", "a $ b", "'x", "!()"] {
assert!(validate_when(bad).is_err(), "{bad}");
}
}
}
+221 -21
View File
@@ -21,6 +21,18 @@ pub struct FontFamily {
pub monospaced: bool,
}
/// PANOSE bProportion value 9 means "Monospaced"; DirectWrite's
/// `IsMonospacedFont` alone misses fonts that only carry this classification.
const PANOSE_MONOSPACED: u8 = 9;
/// One face counts as monospaced when the font system flags it fixed pitch, its
/// PANOSE proportion is "Monospaced", or its glyph advances are uniform
/// (fonts with wrong/missing metadata, e.g. bitmap-style TTFs).
#[cfg_attr(not(windows), allow(dead_code))]
fn face_is_mono(is_monospaced_font: bool, panose_proportion: u8, advances_equal: bool) -> bool {
is_monospaced_font || panose_proportion == PANOSE_MONOSPACED || advances_equal
}
type Cache = Mutex<Option<Arc<[FontFamily]>>>;
/// The installed families, sorted case-insensitively, read on first use and
@@ -195,7 +207,9 @@ mod imp {
// SAFETY: CoreText descriptors carry documented CFString, CFURL and trait dictionary values.
unsafe {
let collection = CTFontCollection::from_available_fonts(None);
let Some(descriptors) = collection.matching_font_descriptors() else { return Vec::new() };
let Some(descriptors) = collection.matching_font_descriptors() else {
return Vec::new();
};
let mut seen = std::collections::BTreeSet::new();
descriptors
.cast_unchecked::<CTFontDescriptor>()
@@ -271,14 +285,16 @@ mod imp {
#[cfg(windows)]
mod imp {
use std::os::windows::ffi::OsStrExt;
use windows::Win32::Graphics::DirectWrite::IDWriteFontFace1;
use windows::Win32::Graphics::DirectWrite::{
DWRITE_FACTORY_TYPE_SHARED, DWRITE_FONT_STRETCH_NORMAL, DWRITE_FONT_STYLE_NORMAL, DWRITE_FONT_WEIGHT_NORMAL,
DWriteCreateFactory, IDWriteFactory, IDWriteFont1, IDWriteLocalizedStrings,
DWRITE_FACTORY_TYPE_SHARED, DWriteCreateFactory, IDWriteFactory, IDWriteFactory3, IDWriteFactory5, IDWriteFont,
IDWriteFont1, IDWriteFontCollection, IDWriteFontFamily, IDWriteLocalizedStrings,
};
use windows::core::{BOOL, Interface, w};
use windows::core::{BOOL, Interface, PCWSTR, w};
/// The system font collection's families, named in English when the
/// family has an English name (what CSS matches), else its first name.
/// The installed families, named in English when the family has an
/// English name (what CSS matches), else its first name.
pub(crate) fn list(refresh: bool) -> Vec<(String, bool)> {
// SAFETY: plain DirectWrite COM calls on interfaces we own.
unsafe { read(refresh) }.unwrap_or_default()
@@ -286,11 +302,85 @@ mod imp {
unsafe fn read(refresh: bool) -> windows::core::Result<Vec<(String, bool)>> {
let factory: IDWriteFactory = unsafe { DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED) }?;
let mut faces = Vec::new();
// Windows 10+: the newer collection covers registered per-user fonts.
// Exclude downloadable fonts that are not installed on this machine.
// Bitmap .FON fonts (e.g. the stock raster Fixedsys) are invisible to
// DirectWrite either way: it only reads TrueType/OpenType files.
let mut collection = None;
unsafe { factory.GetSystemFontCollection(&mut collection, refresh) }?;
let Some(collection) = collection else {
return Ok(Vec::new());
if let Ok(factory3) = factory.cast::<IDWriteFactory3>() {
let mut local = None;
if unsafe { factory3.GetSystemFontCollection(false, &mut local, refresh) }.is_ok() {
collection = local.map(Into::into);
}
}
if collection.is_none() {
unsafe { factory.GetSystemFontCollection(&mut collection, refresh) }?;
}
if let Some(collection) = collection {
faces.extend(collect(&collection));
}
// DirectWrite enumerates via the font registry, so font files copied
// into the per-user directory without registering (no HKCU entry)
// are invisible to the calls above; scan the directory directly.
faces.extend(unsafe { user_dir_faces(&factory) });
Ok(faces)
}
/// Families from `%LOCALAPPDATA%\Microsoft\Windows\Fonts` files, registered
/// or not, via a font set built from the files on disk.
unsafe fn user_dir_faces(factory: &IDWriteFactory) -> Vec<(String, bool)> {
let Some(local_app_data) = std::env::var_os("LOCALAPPDATA") else {
return Vec::new();
};
let dir = std::path::Path::new(&local_app_data).join(r"Microsoft\Windows\Fonts");
let Ok(entries) = std::fs::read_dir(&dir) else {
return Vec::new();
};
let files: Vec<_> = entries
.filter_map(|e| e.ok())
.map(|e| e.path())
.filter(|p| {
matches!(
p.extension().and_then(|e| e.to_str()).map(|e| e.to_ascii_lowercase()).as_deref(),
Some("ttf" | "otf" | "ttc" | "otc")
)
})
.collect();
unsafe { file_faces(factory, &files) }
}
unsafe fn file_faces(factory: &IDWriteFactory, files: &[std::path::PathBuf]) -> Vec<(String, bool)> {
if files.is_empty() {
return Vec::new();
}
let Ok(factory3) = factory.cast::<IDWriteFactory3>() else {
return Vec::new();
};
let Ok(factory5) = factory.cast::<IDWriteFactory5>() else {
return Vec::new();
};
let Ok(builder) = (unsafe { factory5.CreateFontSetBuilder() }) else {
return Vec::new();
};
for path in files {
let wide: Vec<u16> = path.as_os_str().encode_wide().chain(Some(0)).collect();
// SAFETY: wide is a valid NUL-terminated string for the calls.
let Ok(file) = (unsafe { factory.CreateFontFileReference(PCWSTR(wide.as_ptr()), None) }) else {
continue;
};
let _ = unsafe { builder.AddFontFile(&file) };
}
let Ok(set) = (unsafe { builder.CreateFontSet() }) else {
return Vec::new();
};
let Ok(collection) = (unsafe { factory3.CreateFontCollectionFromFontSet(&set) }) else {
return Vec::new();
};
collect(&collection)
}
fn collect(collection: &IDWriteFontCollection) -> Vec<(String, bool)> {
let mut out = Vec::new();
for i in 0..unsafe { collection.GetFontFamilyCount() } {
let Ok(family) = (unsafe { collection.GetFontFamily(i) }) else {
@@ -299,19 +389,120 @@ mod imp {
let Some(name) = unsafe { family.GetFamilyNames() }.ok().and_then(|n| unsafe { english(&n) }) else {
continue;
};
let mono = unsafe {
family.GetFirstMatchingFont(
DWRITE_FONT_WEIGHT_NORMAL,
DWRITE_FONT_STRETCH_NORMAL,
DWRITE_FONT_STYLE_NORMAL,
)
}
.ok()
.and_then(|f| f.cast::<IDWriteFont1>().ok())
.is_some_and(|f| unsafe { f.IsMonospacedFont() }.as_bool());
out.push((name, mono));
out.push((name, unsafe { family_is_mono(&family) }));
}
out
}
/// A family is monospaced when any face reports fixed pitch, PANOSE
/// proportion 9, or uniform glyph advances; probing every face matches
/// `normalize`'s any-face merge (macOS and Linux already flag every face,
/// not just the regular one).
unsafe fn family_is_mono(family: &IDWriteFontFamily) -> bool {
(0..unsafe { family.GetFontCount() }).any(|i| {
let Ok(font) = (unsafe { family.GetFont(i) }) else {
return false;
};
if let Ok(font1) = font.cast::<IDWriteFont1>() {
let is_monospaced = unsafe { font1.IsMonospacedFont() }.as_bool();
// The proportion byte is defined for text/display fonts only.
let panose = unsafe { font1.GetPanose().values };
let panose_proportion = if panose[0] == 2 { panose[3] } else { 0 };
if super::face_is_mono(is_monospaced, panose_proportion, false) {
return true;
}
}
unsafe { advances_equal(&font) }
})
}
/// Some monospaced fonts carry no fixed-pitch metadata at all (bitmap-style
/// TTFs like Fixedsys Excelsior): compare the advance widths of 'i' and
/// 'W'. Dual-width CJK monospace fonts pass too (both half-width).
unsafe fn advances_equal(font: &IDWriteFont) -> bool {
let Ok(face) = (unsafe { font.CreateFontFace() }) else {
return false;
};
let Ok(face1) = face.cast::<IDWriteFontFace1>() else {
return false;
};
let codepoints = ['i' as u32, 'W' as u32];
let mut indices = [0u16; 2];
if unsafe { face.GetGlyphIndices(codepoints.as_ptr(), 2, indices.as_mut_ptr()) }.is_err()
|| indices.contains(&0)
{
return false;
}
let mut advances = [0i32; 2];
unsafe { face1.GetDesignGlyphAdvances(2, indices.as_ptr(), advances.as_mut_ptr(), false) }.is_ok()
&& advances[0] > 0
&& advances[0] == advances[1]
}
#[test]
fn glyph_probe_distinguishes_monospace_proportional_and_symbol_fonts() {
unsafe {
let factory: IDWriteFactory = DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED).unwrap();
let mut collection = None;
factory.GetSystemFontCollection(&mut collection, false).unwrap();
let collection = collection.unwrap();
for (name, expected) in [
("Consolas", true),
("Arial", false),
("Times New Roman", false),
("Wingdings", false),
("Webdings", false),
("Segoe MDL2 Assets", false),
] {
let wide: Vec<_> = name.encode_utf16().chain(Some(0)).collect();
let mut index = 0;
let mut exists = BOOL(0);
collection.FindFamilyName(PCWSTR(wide.as_ptr()), &mut index, &mut exists).unwrap();
assert!(exists.as_bool(), "{name} ships with Windows");
let family = collection.GetFontFamily(index).unwrap();
for i in 0..family.GetFontCount() {
let font = family.GetFont(i).unwrap();
assert_eq!(advances_equal(&font), expected, "{name} face {i}");
}
}
}
}
#[test]
fn font_files_include_every_family_in_a_collection() {
let path = std::path::PathBuf::from(std::env::var_os("WINDIR").unwrap()).join(r"Fonts\cambria.ttc");
unsafe {
let factory: IDWriteFactory = DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED).unwrap();
let faces = file_faces(&factory, &[path]);
for name in ["Cambria", "Cambria Math"] {
assert!(faces.iter().any(|(family, _)| family == name), "missing {name}: {faces:?}");
}
}
}
#[test]
fn excludes_families_available_only_as_downloads() {
use windows::Win32::Graphics::DirectWrite::{DWRITE_E_REMOTEFONT, DWRITE_LOCALITY_LOCAL, IDWriteFont3};
unsafe {
let factory: IDWriteFactory = DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED).unwrap();
let factory3: IDWriteFactory3 = factory.cast().unwrap();
let set = factory3.GetSystemFontSet().unwrap();
let collection = factory3.CreateFontCollectionFromFontSet(&set).unwrap();
let listed = super::normalize(read(false).unwrap());
let user = super::normalize(user_dir_faces(&factory));
for i in 0..collection.GetFontFamilyCount() {
let family = collection.GetFontFamily(i).unwrap();
let name = english(&family.GetFamilyNames().unwrap()).unwrap();
let remote_only = (0..family.GetFontCount()).all(|j| match family.GetFont(j) {
Ok(font) => font.cast::<IDWriteFont3>().unwrap().GetLocality() != DWRITE_LOCALITY_LOCAL,
Err(error) => error.code() == DWRITE_E_REMOTEFONT,
});
if remote_only && !user.iter().any(|f| f.name == name) {
assert!(!listed.iter().any(|f| f.name == name), "{name} is not installed locally");
}
}
}
Ok(out)
}
unsafe fn english(names: &IDWriteLocalizedStrings) -> Option<String> {
@@ -476,6 +667,15 @@ mod tests {
);
}
#[test]
fn panose_monospaced_proportion_counts_as_mono() {
assert!(face_is_mono(true, 0, false));
assert!(face_is_mono(false, PANOSE_MONOSPACED, false));
assert!(face_is_mono(false, 0, true));
assert!(!face_is_mono(false, 0, false));
assert!(!face_is_mono(false, 3, false));
}
#[test]
fn fc_list_spacing_marks_monospace() {
let out = "DejaVu Sans Mono\t100\nDejaVu Sans\t\nNoto Sans Mono CJK SC\t90\nTerminus\t110\nCantarell\nFira\\-Code\tmono\n\t100\n";
+145
View File
@@ -0,0 +1,145 @@
//! The signed-in GitHub account, independent of any project: its repositories for
//! "Get from Version Control" › GitHub.
use crate::{GhAvailability, QUICK, classify_failure, command, gh_executable, process};
use serde::{Deserialize, Serialize};
use specta::Type;
/// A repository of the signed-in account (owned, collaborator, or organization member).
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct GhAccountRepo {
/// `owner/name`.
pub name_with_owner: String,
pub description: Option<String>,
pub private: bool,
pub fork: bool,
pub archived: bool,
/// What `git clone` takes: HTTPS, or SSH when gh's `git_protocol` is `ssh`.
pub clone_url: String,
}
/// The account's repositories, or why they can't be listed.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase", tag = "kind")]
pub enum GhAccountRepos {
Ready {
login: String,
repos: Vec<GhAccountRepo>,
},
/// No `gh` executable was found.
NotInstalled,
/// `gh auth login` has not been run (or the token expired).
NotLoggedIn {
message: String,
},
Error {
message: String,
},
}
#[derive(Deserialize)]
struct RestRepo {
full_name: String,
description: Option<String>,
#[serde(default)]
private: bool,
#[serde(default)]
fork: bool,
#[serde(default)]
archived: bool,
clone_url: String,
ssh_url: String,
}
/// `gh api --paginate --slurp user/repos` output: one array per page.
pub fn parse_account_repos(slurped: &str, ssh: bool) -> Result<Vec<GhAccountRepo>, serde_json::Error> {
let pages: Vec<Vec<RestRepo>> = serde_json::from_str(slurped)?;
Ok(pages
.into_iter()
.flatten()
.map(|r| GhAccountRepo {
name_with_owner: r.full_name,
description: r.description.filter(|d| !d.trim().is_empty()),
private: r.private,
fork: r.fork,
archived: r.archived,
clone_url: if ssh { r.ssh_url } else { r.clone_url },
})
.collect())
}
/// Lists the repositories of the account gh is signed in to, most recently pushed first.
pub fn account_repositories() -> GhAccountRepos {
if gh_executable().is_none() {
return GhAccountRepos::NotInstalled;
}
let dir = std::env::temp_dir();
let gh = |args: &[&str]| -> Result<String, GhAccountRepos> {
let out = command(&dir, args)
.and_then(|cmd| process::run(cmd, QUICK, None))
.map_err(|e| GhAccountRepos::Error { message: e.message })?;
if out.success {
return Ok(out.stdout);
}
let message = if out.stderr.trim().is_empty() { out.stdout } else { out.stderr };
Err(match classify_failure(&message) {
GhAvailability::NotLoggedIn { message } => GhAccountRepos::NotLoggedIn { message },
_ => GhAccountRepos::Error { message: message.trim().to_owned() },
})
};
let result = (|| {
let login = gh(&["api", "user", "--jq", ".login"])?.trim().to_owned();
let ssh = gh(&["config", "get", "git_protocol", "--host", "github.com"]).is_ok_and(|p| p.trim() == "ssh");
let pages = gh(&[
"api",
"user/repos?per_page=100&sort=pushed&affiliation=owner,collaborator,organization_member",
"--paginate",
"--slurp",
])?;
let repos = parse_account_repos(&pages, ssh)
.map_err(|e| GhAccountRepos::Error { message: format!("unexpected gh output: {e}") })?;
Ok(GhAccountRepos::Ready { login, repos })
})();
result.unwrap_or_else(|e| e)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn reads_every_page_and_picks_the_configured_protocol() {
let pages = r#"[
[{"full_name":"me/app","description":"An app","private":true,"fork":false,"archived":false,
"clone_url":"https://github.com/me/app.git","ssh_url":"[email protected]:me/app.git"}],
[{"full_name":"org/lib","description":"","fork":true,
"clone_url":"https://github.com/org/lib.git","ssh_url":"[email protected]:org/lib.git"}]
]"#;
let https = parse_account_repos(pages, false).unwrap();
assert_eq!(
https,
vec![
GhAccountRepo {
name_with_owner: "me/app".into(),
description: Some("An app".into()),
private: true,
fork: false,
archived: false,
clone_url: "https://github.com/me/app.git".into(),
},
GhAccountRepo {
name_with_owner: "org/lib".into(),
description: None,
private: false,
fork: true,
archived: false,
clone_url: "https://github.com/org/lib.git".into(),
},
]
);
let ssh = parse_account_repos(pages, true).unwrap();
assert_eq!(ssh[1].clone_url, "[email protected]:org/lib.git");
assert!(parse_account_repos("[]", false).unwrap().is_empty());
}
}
+2
View File
@@ -3,9 +3,11 @@
//!
//! Every function blocks; callers run them off the UI / command threads.
mod account;
mod model;
mod process;
pub use account::{GhAccountRepo, GhAccountRepos, account_repositories, parse_account_repos};
pub use model::*;
use ide_core::{CancellationToken, IdeError, IdeResult};
+18
View File
@@ -14,6 +14,7 @@ encoding_rs.workspace = true
ide-core.workspace = true
ide-text.workspace = true
ide-vfs.workspace = true
regex-syntax.workspace = true
ide-workspace.workspace = true
ide-language.workspace = true
ide-index.workspace = true
@@ -30,15 +31,25 @@ ide-terminal.workspace = true
ide-run.workspace = true
ide-sdk.workspace = true
ide-http.workspace = true
reqwest.workspace = true
ignore.workspace = true
base64.workspace = true
getrandom = "0.3"
globset.workspace = true
ide-db.workspace = true
ide-ssh.workspace = true
ide-remote.workspace = true
ide-jsonrpc.workspace = true
ide-agent.workspace = true
ide-notebook.workspace = true
ide-extension.workspace = true
ide-plugin.workspace = true
ide-tools = { path = "../ide-tools" }
ide-docker = { path = "../ide-docker" }
semver = "1"
serde.workspace = true
serde_json.workspace = true
toml.workspace = true
specta.workspace = true
specta-typescript.workspace = true
tokio = { workspace = true, features = ["io-std"] }
@@ -49,13 +60,20 @@ dunce.workspace = true
dirs.workspace = true
uuid.workspace = true
tempfile.workspace = true
zip.workspace = true
quick-xml = "0.42"
[dev-dependencies]
ide-ssh = { workspace = true, features = ["testing"] }
tokio = { workspace = true, features = ["rt-multi-thread", "macros", "io-util", "test-util"] }
criterion = { version = "0.5", default-features = false }
[target.'cfg(unix)'.dependencies]
rustix = { version = "1.1", features = ["fs", "process"] }
[target.'cfg(windows)'.dependencies]
windows-sys = { version = "0.61", features = ["Win32_Storage_FileSystem"] }
[[bench]]
name = "plugins"
harness = false
+149
View File
@@ -0,0 +1,149 @@
//! `cargo bench -p ide-host`: the plugin runtime budgets, checked by `pnpm perf:rust` against
//! `perf/budgets.json` › rust (`plugins/…`).
//!
//! - `load_50`: start-up load of 50 installed plugins (20 with code, 10
//! languages with a syntax provider, 10 icon themes, 10 declarations only),
//! from the contribution index; no Wasm is instantiated.
//! - `cold_activation`: an idle-reclaimed plugin's next command: activation
//! from the precompiled component plus the call.
//! - `syntax_highlights_2k_lines`: an edit-path call (Solidity highlighting).
use criterion::{Criterion, black_box, criterion_group, criterion_main};
use ide_core::LanguageId;
use ide_host::plugins::{Consent, PluginHost};
use ide_plugin::{Invocation, Limits};
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::time::Duration;
fn repo() -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR")).join("../..").canonicalize().unwrap()
}
fn copy_dir(from: &Path, to: &Path) {
std::fs::create_dir_all(to).unwrap();
for e in std::fs::read_dir(from).unwrap().flatten() {
let name = e.file_name();
if matches!(name.to_str(), Some("target" | "src" | "Cargo.lock" | ".gitignore")) {
continue;
}
let dst = to.join(&name);
if e.path().is_dir() { copy_dir(&e.path(), &dst) } else { std::fs::copy(e.path(), &dst).map(|_| ()).unwrap() }
}
}
/// A copy of `src` installed under a new id (and new language / theme ids).
fn variant(work: &Path, src: &Path, n: usize, rename: &[(&str, String)]) -> PathBuf {
let dir = work.join(format!("pkg{n}"));
copy_dir(src, &dir);
for file in ["plugin.toml", "languages/solidity.toml"] {
let path = dir.join(file);
if let Ok(mut text) = std::fs::read_to_string(&path) {
for (from, to) in rename {
text = text.replace(from, to);
}
std::fs::write(&path, text).unwrap();
}
}
dir
}
fn sink() -> Arc<dyn ide_host::plugins::PluginEventSink> {
Arc::new(|_: Option<&str>, _| {})
}
fn install_50(plugins: &Path, work: &Path) {
let host = PluginHost::new(plugins.to_path_buf(), sink(), Limits::default()).unwrap();
host.load_all();
let root = repo();
let mut n = 0;
let mut install = |src: PathBuf, rename: Vec<(&str, String)>| {
let pkg = variant(work, &src, n, &rename);
n += 1;
host.install_from_dir(&pkg, &Consent::default()).unwrap();
};
for i in 0..20 {
install(
root.join("crates/ide-plugin/tests/fixtures/hello"),
vec![("id = \"hello\"", format!("id = \"bench.hello{i}\""))],
);
}
for i in 0..10 {
install(
root.join("plugins/official/solidity"),
vec![
("openidea.solidity", format!("bench.solidity{i}")),
("id = \"solidity\"", format!("id = \"solidity{i}\"")),
("extensions = [\"sol\"]", format!("extensions = [\"sol{i}\"]")),
("on-language:solidity", format!("on-language:solidity{i}")),
("id = \"forge-", format!("id = \"f{i}-")),
("id = \"hardhat-", format!("id = \"h{i}-")),
],
);
}
for i in 0..10 {
install(
root.join("plugins/official/material-icons"),
vec![
("openidea.material-icons", format!("bench.icons{i}")),
("id = \"material\"", format!("id = \"material{i}\"")),
],
);
}
for i in 0..10 {
let dir = work.join(format!("static{i}"));
std::fs::create_dir_all(&dir).unwrap();
std::fs::write(
dir.join("plugin.toml"),
format!(
"id = \"bench.static{i}\"\nname = \"Static {i}\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"\n\
[[contributes.commands]]\nid = \"a\"\ntitle = \"Static {i}: A\"\nwhen = \"editorLang == rust\"\n"
),
)
.unwrap();
host.install_from_dir(&dir, &Consent::default()).unwrap();
}
}
fn bench(c: &mut Criterion) {
let tmp = tempfile::tempdir().unwrap();
let plugins = tmp.path().join("plugins");
install_50(&plugins, &tmp.path().join("work"));
let mut g = c.benchmark_group("plugins");
g.sample_size(20);
g.bench_function("load_50", |b| {
b.iter(|| {
let host = PluginHost::new(plugins.clone(), sink(), Limits::default()).unwrap();
host.load_all();
assert_eq!(host.instances(), 0, "start-up instantiates no Wasm");
black_box(host.list(None).plugins.len())
})
});
let limits = Limits { idle_timeout: Some(Duration::ZERO), ..Limits::default() };
let host = Arc::new(PluginHost::new(plugins.clone(), sink(), limits).unwrap());
host.load_all();
host.warm_component_cache();
g.bench_function("cold_activation", |b| {
b.iter(|| {
host.tick();
// Fails for want of an editor, after activating and running.
let r = host.run_command("bench.hello0", "word-count", None, Invocation::default());
black_box(r.is_err())
})
});
let provider = host.syntax_provider();
let lang = LanguageId::new("solidity0");
let unit = "contract Token {\n uint256 public total;\n mapping(address => uint256) balances;\n event Sent(address to, uint256 v);\n function send(address to, uint256 v) public returns (bool) {\n // move funds\n balances[to] += v;\n emit Sent(to, v);\n return true;\n }\n}\n";
let text = unit.repeat(2000 / unit.lines().count());
assert!(provider.highlights(&lang, &text).is_some());
g.bench_function("syntax_highlights_2k_lines", |b| {
b.iter(|| black_box(provider.highlights(&lang, &text).map(|t| t.len())))
});
g.finish();
}
criterion_group!(benches, bench);
criterion_main!(benches);
@@ -0,0 +1,924 @@
//! `openidea-plugin`: scaffolds, builds, checks and links OpenIDEA plugins.
//!
//! ```text
//! openidea-plugin new <dir> [--template command|theme|icon-theme|language] [--id ID] [--name NAME] [--sdk PATH]
//! openidea-plugin build [dir] [--debug] # cargo → dist/plugin.wasm, then `check`
//! openidea-plugin check [dir|package] # validate the package as the IDE would install it
//! openidea-plugin link [dir] # load the folder in the IDE without installing it
//! openidea-plugin unlink [dir]
//! openidea-plugin links # list linked folders
//! openidea-plugin keygen <key-file> # a publisher's Ed25519 key
//! openidea-plugin endorse <new-public-key> --key OLD --publisher ID # vouch for a new key
//! openidea-plugin pack [dir] [--key F --publisher ID] [--out FILE] # <id>-<version>.oiplugin
//! openidea-plugin index <dir> [--publisher ID --name NAME --key F [--previous-key F]] # marketplace index.json
//! openidea-plugin merge <index> <other-index> --keep ID # list another index's publishers
//! ```
//!
//! `link` writes `links.json` in the IDE's plugins directory
//! (`$OPENIDEA_PLUGINS_DIR`, or the app data directory); the IDE picks the
//! change up when Settings › Plugins is opened or refreshed. Linked plugins
//! are re-read from the folder, so `build` + refresh is the edit loop.
mod templates;
use ide_extension::{Manifest, validate_plugin_id};
use ide_host::plugins::{PluginHost, read_market_url, resolve_market_href};
use ide_plugin::{Limits, PluginEngine};
use std::path::{Path, PathBuf};
use std::process::{Command, ExitCode};
use std::sync::Arc;
const USAGE: &str = "\
usage: openidea-plugin <command> [options]
commands:
new <dir> [--template T] [--id ID] [--name NAME] [--sdk PATH]
create a plugin; T = command (default), theme, icon-theme, language
build [dir] [--debug] compile a Rust plugin to dist/plugin.wasm and check it
check [dir|package] validate the package (folder or .oiplugin) as the IDE would install it
link [dir] load the folder in the IDE without installing it
unlink [dir] stop loading a linked folder
links list linked folders
keygen <key-file> create a publisher signing key (keep the file secret)
endorse <new-public-key> --key F --publisher ID
print an earlier key's endorsement of a new one (JSON), for an
index the publisher doesn't write with `index --previous-key`
pack [dir] [--key F --publisher ID] [--out FILE]
write <id>-<version>.oiplugin, signed with the key
index <dir> [--publisher ID --name NAME --key F [--previous-key F]] [--out FILE]
[--download-base URL]
write the marketplace index.json for the packages in dir: with a
key, register it and list that publisher's plugins (earlier
versions stay for older IDEs); without, list only these packages,
each checked against the index's publishers. A new key replacing
an earlier one is endorsed with --previous-key, so installed
plugins update to it without asking
merge <index> <other-index> --keep ID [--out FILE]
replace every publisher but ID in index with those of other-index
(a path or URL), their plugins and revocations
environment:
OPENIDEA_PLUGINS_DIR the IDE's plugins directory (default: the app data directory)";
fn main() -> ExitCode {
let args: Vec<String> = std::env::args().skip(1).collect();
let result = match args.first().map(String::as_str) {
Some("new") => cmd_new(&args[1..]),
Some("build") => cmd_build(&args[1..]),
Some("check") => dir_arg(&args[1..]).and_then(|d| check_path(&d)),
Some("link") => dir_arg(&args[1..]).and_then(|d| link(&d, true)),
Some("unlink") => dir_arg(&args[1..]).and_then(|d| link(&d, false)),
Some("links") => list_links(),
Some("keygen") => cmd_keygen(&args[1..]),
Some("endorse") => cmd_endorse(&args[1..]),
Some("pack") => cmd_pack(&args[1..]),
Some("index") => cmd_index(&args[1..]),
Some("merge") => cmd_merge(&args[1..]),
Some("--version" | "-V") => {
println!("openidea-plugin {} (plugin API {})", env!("CARGO_PKG_VERSION"), ide_extension::ENGINE_VERSION);
Ok(())
}
Some("help" | "--help" | "-h") | None => {
println!("{USAGE}");
Ok(())
}
Some(other) => Err(format!("unknown command {other:?}\n\n{USAGE}")),
};
match result {
Ok(()) => ExitCode::SUCCESS,
Err(e) => {
eprintln!("error: {e}");
ExitCode::FAILURE
}
}
}
/// Splits `args` into positionals and `--flag value` / `--switch` options.
struct Args {
positional: Vec<String>,
options: Vec<(String, Option<String>)>,
}
impl Args {
fn parse(args: &[String], switches: &[&str], valued: &[&str]) -> Result<Self, String> {
let mut out = Args { positional: Vec::new(), options: Vec::new() };
let mut it = args.iter();
while let Some(a) = it.next() {
if let Some(name) = a.strip_prefix("--") {
let (name, inline) = match name.split_once('=') {
Some((n, v)) => (n, Some(v.to_owned())),
None => (name, None),
};
if switches.contains(&name) && inline.is_none() {
out.options.push((name.into(), None));
} else if valued.contains(&name) {
let v = inline.or_else(|| it.next().cloned()).ok_or_else(|| format!("--{name} needs a value"))?;
out.options.push((name.into(), Some(v)));
} else {
return Err(format!("unknown option --{name}"));
}
} else {
out.positional.push(a.clone());
}
}
Ok(out)
}
fn value(&self, name: &str) -> Option<&str> {
self.options.iter().rev().find(|(n, _)| n == name).and_then(|(_, v)| v.as_deref())
}
fn has(&self, name: &str) -> bool {
self.options.iter().any(|(n, _)| n == name)
}
fn dir(&self) -> Result<PathBuf, String> {
match self.positional.as_slice() {
[] => Ok(PathBuf::from(".")),
[d] => Ok(PathBuf::from(d)),
_ => Err("expected at most one directory".into()),
}
}
}
fn dir_arg(args: &[String]) -> Result<PathBuf, String> {
Args::parse(args, &[], &[])?.dir()
}
fn canonical(dir: &Path) -> Result<PathBuf, String> {
dunce::canonicalize(dir).map_err(|e| format!("{}: {e}", dir.display()))
}
// ---------------------------------------------------------------- new
fn cmd_new(args: &[String]) -> Result<(), String> {
let a = Args::parse(args, &[], &["template", "id", "name", "sdk"])?;
let [dir] = a.positional.as_slice() else {
return Err("usage: openidea-plugin new <dir> [--template T] [--id ID] [--name NAME]".into());
};
let dir = PathBuf::from(dir);
let template = templates::Template::parse(a.value("template").unwrap_or("command"))?;
let base = dir.file_name().and_then(|n| n.to_str()).ok_or("the directory needs a name")?;
let id = match a.value("id") {
Some(id) => id.to_owned(),
None => slugify(base),
};
validate_plugin_id(&id)?;
let name = match a.value("name") {
Some(n) => n.to_owned(),
None => title_case(id.rsplit('.').next().unwrap_or(&id)),
};
if name.trim().is_empty() || name.contains(['"', '\\', '\n']) {
return Err("--name must not be empty or contain quotes, backslashes or newlines".into());
}
if dir.exists() && std::fs::read_dir(&dir).map_err(|e| e.to_string())?.next().is_some() {
return Err(format!("{} already exists and is not empty", dir.display()));
}
std::fs::create_dir_all(&dir).map_err(|e| format!("{}: {e}", dir.display()))?;
let dir = canonical(&dir)?;
let sdk = match template {
templates::Template::Command => Some(sdk_path(a.value("sdk"), &dir)?),
_ => None,
};
let vars = templates::Vars { id: &id, name: &name, sdk: sdk.as_deref() };
let files = templates::render(template, &vars);
for (rel, content) in &files {
let path = dir.join(rel);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent).map_err(|e| format!("{}: {e}", parent.display()))?;
}
std::fs::write(&path, content).map_err(|e| format!("{}: {e}", path.display()))?;
}
// Generated packages must be valid as written.
Manifest::load(&dir).map_err(|e| format!("internal error: the template is invalid: {e}"))?;
println!("created {} ({id}) in {}", template.label(), dir.display());
let shown = std::env::current_dir().ok().and_then(|cwd| dir.strip_prefix(cwd).ok().map(Path::to_path_buf));
let shown = shown.as_deref().unwrap_or(&dir).display().to_string();
println!();
println!("next:");
if template == templates::Template::Command {
println!(" cargo test --manifest-path {shown}/Cargo.toml # unit tests against the mock host");
println!(" openidea-plugin build {shown}");
} else {
println!(" openidea-plugin check {shown}");
}
println!(" openidea-plugin link {shown} # then Settings › Plugins");
Ok(())
}
/// `My Cool_Plugin` → `my-cool-plugin`.
fn slugify(s: &str) -> String {
let mut out = String::new();
for c in s.chars() {
match c {
'a'..='z' | '0'..='9' | '.' => out.push(c),
'A'..='Z' => out.push(c.to_ascii_lowercase()),
_ if !out.ends_with('-') && !out.is_empty() => out.push('-'),
_ => {}
}
}
out.trim_matches(['-', '.']).to_owned()
}
/// `my-cool_plugin` → `My Cool Plugin`.
fn title_case(s: &str) -> String {
s.split(['-', '_'])
.filter(|w| !w.is_empty())
.map(|w| {
let mut c = w.chars();
c.next().map(|f| f.to_uppercase().chain(c).collect::<String>()).unwrap_or_default()
})
.collect::<Vec<_>>()
.join(" ")
}
/// The SDK dependency path written into a new plugin's Cargo.toml: `--sdk`,
/// or the SDK of the source tree this tool was built from.
fn sdk_path(explicit: Option<&str>, plugin_dir: &Path) -> Result<String, String> {
let sdk = match explicit {
Some(p) => PathBuf::from(p),
None => PathBuf::from(concat!(env!("CARGO_MANIFEST_DIR"), "/../../sdk/rust/openidea-plugin")),
};
let sdk = dunce::canonicalize(&sdk).map_err(|_| {
format!("the plugin SDK was not found at {}; pass --sdk <path to sdk/rust/openidea-plugin>", sdk.display())
})?;
if !sdk.join("Cargo.toml").is_file() {
return Err(format!("{} is not the openidea-plugin SDK (no Cargo.toml)", sdk.display()));
}
let path = relative(plugin_dir, &sdk).unwrap_or(sdk);
Ok(path.to_string_lossy().replace('\\', "/"))
}
/// `to` relative to `from` when they share a root (keeps packages in a repo movable).
fn relative(from: &Path, to: &Path) -> Option<PathBuf> {
let (f, t): (Vec<_>, Vec<_>) = (from.components().collect(), to.components().collect());
let common = f.iter().zip(&t).take_while(|(a, b)| a == b).count();
if common <= 1 {
return None;
}
let mut out = PathBuf::new();
for _ in common..f.len() {
out.push("..");
}
for c in &t[common..] {
out.push(c);
}
Some(out)
}
// ---------------------------------------------------------------- build / check
fn cmd_build(args: &[String]) -> Result<(), String> {
let a = Args::parse(args, &["debug"], &[])?;
let dir = canonical(&a.dir()?)?;
let manifest = Manifest::load(&dir)?;
let cargo_toml = dir.join("Cargo.toml");
if !cargo_toml.is_file() {
if manifest.wasm.is_some() {
return Err(format!(
"{} has no Cargo.toml: build the component with your toolchain, then run `check`",
dir.display()
));
}
println!("{} has no code to build", manifest.id);
return check(&dir);
}
let wasm_rel = manifest.wasm.clone().ok_or("plugin.toml has no `wasm = \"dist/plugin.wasm\"`")?;
let crate_name = crate_name(&cargo_toml)?;
ensure_wasm_target();
let release = !a.has("debug");
let target_dir = std::env::var_os("CARGO_TARGET_DIR").map(PathBuf::from).unwrap_or_else(|| dir.join("target"));
let mut cmd = Command::new(std::env::var_os("CARGO").unwrap_or_else(|| "cargo".into()));
cmd.arg("build").arg("--manifest-path").arg(&cargo_toml).args(["--target", "wasm32-wasip2"]);
if release {
cmd.arg("--release");
}
cmd.env("CARGO_TARGET_DIR", &target_dir);
println!("building {crate_name} for wasm32-wasip2 ({})", if release { "release" } else { "debug" });
let status = cmd.status().map_err(|e| format!("cannot run cargo: {e}"))?;
if !status.success() {
return Err("cargo build failed".into());
}
let built = target_dir
.join("wasm32-wasip2")
.join(if release { "release" } else { "debug" })
.join(format!("{}.wasm", crate_name.replace('-', "_")));
if !built.is_file() {
return Err(format!("{} was not produced: is `crate-type = [\"cdylib\"]` set in Cargo.toml?", built.display()));
}
let out = dir.join(&wasm_rel);
if let Some(parent) = out.parent() {
std::fs::create_dir_all(parent).map_err(|e| format!("{}: {e}", parent.display()))?;
}
std::fs::copy(&built, &out).map_err(|e| format!("{}: {e}", out.display()))?;
let size = std::fs::metadata(&out).map(|m| m.len()).unwrap_or(0);
println!(" -> {} ({})", wasm_rel.display(), human_size(size));
check(&dir)
}
fn crate_name(cargo_toml: &Path) -> Result<String, String> {
let src = std::fs::read_to_string(cargo_toml).map_err(|e| format!("{}: {e}", cargo_toml.display()))?;
let doc: toml::Table = src.parse().map_err(|e| format!("{}: {e}", cargo_toml.display()))?;
let package = doc.get("package").and_then(|p| p.as_table());
let lib_name = doc.get("lib").and_then(|l| l.get("name")).and_then(|n| n.as_str());
lib_name
.or_else(|| package.and_then(|p| p.get("name")).and_then(|n| n.as_str()))
.map(str::to_owned)
.ok_or_else(|| format!("{}: no [package] name", cargo_toml.display()))
}
/// Installs the target through rustup when it is missing (no-op without rustup).
fn ensure_wasm_target() {
let Ok(out) = Command::new("rustup").args(["target", "list", "--installed"]).output() else { return };
if String::from_utf8_lossy(&out.stdout).lines().any(|l| l.trim() == "wasm32-wasip2") {
return;
}
println!("installing the wasm32-wasip2 target");
let _ = Command::new("rustup").args(["target", "add", "wasm32-wasip2"]).status();
}
/// Validates `dir` exactly as Install from Disk would, and checks the
/// component imports only the plugin API.
/// [`check`] of a folder, or of a package file unpacked as installing would.
fn check_path(path: &Path) -> Result<(), String> {
if !path.is_file() {
return check(path);
}
let tmp = scratch_dir("unpack")?;
let dir = tmp.path().join("package");
package::unpack(path, &dir).and_then(|_| check(&dir))
}
fn check(dir: &Path) -> Result<(), String> {
let dir = canonical(dir)?;
let tmp = scratch_dir("check")?;
let scratch = tmp.path().join("host");
(|| {
let host = PluginHost::new(scratch.clone(), Arc::new(|_: Option<&str>, _| {}), Limits::default())
.map_err(|e| e.to_string())?;
let preview = host.preview_install(&dir).map_err(|e| e.to_string())?;
let manifest = Manifest::load(&dir)?;
println!("{} {} — {}", preview.id, preview.version, preview.name);
if manifest.legacy {
println!(" note: plugin.toml uses the 0.1 format (`openidea-api`); see plugins/README.md to move to 0.2");
}
if let Some(wasm) = &manifest.wasm {
let engine = PluginEngine::new()?;
let component = engine.compile(&dir.join(wasm))?;
let built_for = engine.check_component(&component)?;
let size = std::fs::metadata(dir.join(wasm)).map(|m| m.len()).unwrap_or(0);
let api = built_for.map(|v| format!(", plugin API {}.{}", v.major, v.minor)).unwrap_or_default();
println!(" code: {} ({}{api})", wasm.display(), human_size(size));
} else {
println!(" code: none (contributions only)");
}
println!(" categories: {}", manifest.categories().join(", "));
for c in manifest.unknown_categories() {
println!(" warning: unknown category {c:?} (ignored); use {}", ide_extension::CATEGORIES.join(", "));
}
if manifest.declared_categories.len() > ide_extension::MAX_CATEGORIES {
println!(
" warning: only the first {} known categories are used; {} are declared",
ide_extension::MAX_CATEGORIES,
manifest.declared_categories.len()
);
}
if !manifest.activation.is_empty() {
let events: Vec<String> = manifest.activation.iter().map(ToString::to_string).collect();
println!(" activation: {}", events.join(", "));
}
for c in &preview.contributions {
println!(" contributes {} × {}", c.count, c.point);
}
for c in &preview.capabilities {
let kind = if c.optional { "optional" } else { "required" };
println!(" capability {} ({kind}, {:?} risk): {}", c.id, c.risk, c.description);
}
for w in &preview.warnings {
println!(" warning: {w}");
}
println!("ok");
Ok(())
})()
}
fn human_size(bytes: u64) -> String {
match bytes {
b if b >= 1 << 20 => format!("{:.1} MiB", b as f64 / (1 << 20) as f64),
b if b >= 1 << 10 => format!("{:.1} KiB", b as f64 / 1024.0),
b => format!("{b} B"),
}
}
// ---------------------------------------------------------------- link
fn plugins_dir() -> Result<PathBuf, String> {
if let Some(dir) = std::env::var_os("OPENIDEA_PLUGINS_DIR") {
return Ok(PathBuf::from(dir));
}
dirs::data_dir()
.map(|d| d.join("ai.l0ng.openidea").join("plugins"))
.ok_or_else(|| "cannot find the app data directory; set OPENIDEA_PLUGINS_DIR".into())
}
#[derive(Default, serde::Serialize, serde::Deserialize)]
struct Links {
#[serde(default)]
paths: Vec<PathBuf>,
}
fn read_links(file: &Path) -> Result<Links, String> {
match std::fs::read_to_string(file) {
Ok(s) => serde_json::from_str(&s).map_err(|e| format!("{}: {e}", file.display())),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(Links::default()),
Err(e) => Err(format!("{}: {e}", file.display())),
}
}
fn link(dir: &Path, add: bool) -> Result<(), String> {
let plugins = plugins_dir()?;
let file = plugins.join("links.json");
let mut links = read_links(&file)?;
let dir = if add {
let dir = canonical(dir)?;
let m = Manifest::load(&dir)?;
if let Some(wasm) = &m.wasm
&& !dir.join(wasm).is_file()
{
println!("note: {} does not exist yet; run `openidea-plugin build` first", wasm.display());
}
if links.paths.contains(&dir) {
println!("{} is already linked", dir.display());
return Ok(());
}
println!("linked {} ({})", m.id, dir.display());
links.paths.push(dir.clone());
dir
} else {
let dir = dunce::canonicalize(dir).unwrap_or_else(|_| dir.to_path_buf());
let before = links.paths.len();
links.paths.retain(|p| p != &dir);
if links.paths.len() == before {
return Err(format!("{} is not linked", dir.display()));
}
println!("unlinked {}", dir.display());
dir
};
let _ = dir;
std::fs::create_dir_all(&plugins).map_err(|e| format!("{}: {e}", plugins.display()))?;
let json = serde_json::to_string_pretty(&links).map_err(|e| e.to_string())?;
let tmp = file.with_extension("json.tmp");
std::fs::write(&tmp, json)
.and_then(|()| std::fs::rename(&tmp, &file))
.map_err(|e| format!("{}: {e}", file.display()))?;
println!("open Settings › Plugins in OpenIDEA to load the change");
Ok(())
}
fn list_links() -> Result<(), String> {
let links = read_links(&plugins_dir()?.join("links.json"))?;
if links.paths.is_empty() {
println!("no linked plugins");
}
for p in links.paths {
let id = Manifest::load(&p).map(|m| m.id).unwrap_or_else(|e| format!("invalid: {e}"));
println!("{id}\t{}", p.display());
}
Ok(())
}
// ---------------------------------------------------------------- distribution
use ide_extension::package::{self, Index, IndexEntry, KeyEntry, PublisherKey};
fn read_key(path: &str) -> Result<PublisherKey, String> {
PublisherKey::from_base64(&std::fs::read_to_string(path).map_err(|e| format!("{path}: {e}"))?)
}
fn cmd_keygen(args: &[String]) -> Result<(), String> {
let a = Args::parse(args, &[], &[])?;
let [file] = a.positional.as_slice() else { return Err("usage: openidea-plugin keygen <key-file>".into()) };
if Path::new(file).exists() {
return Err(format!("{file} exists; not overwriting a key"));
}
let mut seed = [0u8; 32];
getrandom::fill(&mut seed).map_err(|e| format!("random: {e}"))?;
let key = PublisherKey::from_seed(seed);
std::fs::write(file, key.seed_base64()).map_err(|e| format!("{file}: {e}"))?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
let _ = std::fs::set_permissions(file, std::fs::Permissions::from_mode(0o600));
}
println!("public key: {}", key.public_base64());
println!("register it with your publisher id in the marketplace index; keep {file} secret");
Ok(())
}
fn cmd_endorse(args: &[String]) -> Result<(), String> {
let a = Args::parse(args, &[], &["key", "publisher"])?;
let usage = "usage: openidea-plugin endorse <new-public-key> --key <earlier-key-file> --publisher ID";
let ([new], Some(key), Some(publisher)) = (a.positional.as_slice(), a.value("key"), a.value("publisher")) else {
return Err(usage.into());
};
if !package::publisher_id_ok(publisher) {
return Err(format!("invalid publisher id {publisher:?}"));
}
let endorsement = read_key(key)?.endorse(publisher, new.trim());
println!("{}", serde_json::to_string_pretty(&endorsement).map_err(|e| e.to_string())?);
Ok(())
}
/// A fresh scratch directory, removed when dropped.
fn scratch_dir(what: &str) -> Result<tempfile::TempDir, String> {
tempfile::Builder::new().prefix(&format!("openidea-plugin-{what}-")).tempdir().map_err(|e| e.to_string())
}
/// A scratch host to list a package's files as the IDE would.
fn with_scratch_host<T>(f: impl FnOnce(&PluginHost) -> Result<T, String>) -> Result<T, String> {
let tmp = scratch_dir("host")?;
PluginHost::new(tmp.path().join("host"), Arc::new(|_: Option<&str>, _| {}), Limits::default())
.map_err(|e| e.to_string())
.and_then(|h| f(&h))
}
fn cmd_pack(args: &[String]) -> Result<(), String> {
let a = Args::parse(args, &[], &["key", "publisher", "out"])?;
let dir = canonical(&a.dir()?)?;
check(&dir)?;
let manifest = Manifest::load(&dir)?;
let mut files = with_scratch_host(|h| h.package_files_of(&dir).map_err(|e| e.to_string()))?;
let digest = package::package_digest(&dir, &files).map_err(|e| e.to_string())?;
let signature = match (a.value("key"), a.value("publisher")) {
(Some(key), Some(publisher)) => {
if !manifest.id.starts_with(&format!("{publisher}.")) {
return Err(format!("publisher {publisher:?} may only sign plugins named {publisher}.*"));
}
Some(read_key(key)?.sign(publisher, &manifest.id, &manifest.version, &digest))
}
(None, None) => None,
_ => return Err("--key and --publisher go together".into()),
};
for lic in ["LICENSE", "LICENSE.md", "LICENSE.txt", "NOTICE", "NOTICE.md", "COPYING", "README.md"] {
if dir.join(lic).is_file() {
files.push(PathBuf::from(lic));
}
}
let out = a.value("out").map(PathBuf::from).unwrap_or_else(|| {
PathBuf::from(format!("{}-{}.{}", manifest.id, manifest.version, package::PACKAGE_EXTENSION))
});
package::pack(&dir, &files, &out, signature.as_ref())?;
let size = std::fs::metadata(&out).map(|m| m.len()).unwrap_or(0);
let signed = signature.as_ref().map_or("unsigned".to_owned(), |s| format!("signed by {}", s.publisher));
println!("-> {} ({}, {signed})", out.display(), human_size(size));
Ok(())
}
fn cmd_index(args: &[String]) -> Result<(), String> {
let a = Args::parse(args, &[], &["publisher", "name", "key", "previous-key", "out", "download-base"])?;
let dir = canonical(&a.dir()?)?;
let out = a.value("out").map(PathBuf::from).unwrap_or_else(|| dir.join("index.json"));
let base = a.value("download-base").map(|b| b.trim_end_matches('/'));
// Revocations and other publishers of an existing index are kept.
let mut index = match std::fs::read_to_string(&out) {
Ok(text) => current_format(Index::parse(&text)?, &out.to_string_lossy())?,
Err(_) => Index { format: package::INDEX_FORMAT, ..Index::default() },
};
let publisher = match (a.value("publisher"), a.value("key")) {
(Some(publisher), Some(key)) => {
let key = read_key(key)?;
let name = a.value("name").unwrap_or(publisher);
let previous = a.value("previous-key").map(read_key).transpose()?;
index.publishers =
register_key(std::mem::take(&mut index.publishers), publisher, name, &key, previous.as_ref())?;
Some(publisher)
}
(None, None) if a.has("previous-key") => return Err("--previous-key goes with --publisher and --key".into()),
(None, None) => None,
_ => return Err("--publisher and --key go together".into()),
};
let mut packages: Vec<PathBuf> = std::fs::read_dir(&dir)
.map_err(|e| e.to_string())?
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().is_some_and(|e| e == package::PACKAGE_EXTENSION))
.collect();
packages.sort();
let engine = PluginEngine::new()?;
let mut added: Vec<IndexEntry> = Vec::new();
for path in packages {
let tmp = scratch_dir("index")?;
let scratch = tmp.path().join("package");
let entry = (|| {
let unpacked = package::unpack(&path, &scratch)?;
let m = Manifest::load(&scratch)?;
let files = with_scratch_host(|h| h.package_files_of(&scratch).map_err(|e| e.to_string()))?;
let digest = package::package_digest(&scratch, &files).map_err(|e| e.to_string())?;
let sig = unpacked.signature.as_ref().ok_or("not signed")?;
let signer = index.verify(sig, &m.id, &m.version, &digest)?;
if publisher.is_some_and(|p| p != signer) {
return Err(format!("signed by {signer}, not {}", publisher.unwrap_or_default()));
}
if added.iter().any(|e| (&e.id, &e.version) == (&m.id, &m.version)) {
return Err(format!("another package of {} {} is in the folder", m.id, m.version));
}
if m.legacy {
return Err("plugin.toml uses the 0.1 format; see plugins/README.md".into());
}
let built_for = match &m.wasm {
Some(wasm) => engine.check_component(&engine.compile(&scratch.join(wasm))?)?,
None => None,
};
let file = path.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
Ok::<_, String>(IndexEntry {
id: m.id.clone(),
name: m.name.clone(),
version: m.version.clone(),
description: m.description.clone(),
authors: m.authors.clone(),
publisher: signer,
download: base.map_or_else(|| file.clone(), |b| format!("{b}/{file}")),
digest,
capabilities: m.capabilities.all().map(|c| c.to_string()).collect(),
engine: package::engine_requirement(&m.engine, built_for.as_ref()),
categories: m.categories(),
})
})();
let entry = entry.map_err(|e| format!("{}: {e}", path.display()))?;
println!(" {} {} (plugin API {})", entry.id, entry.version, entry.engine);
added.push(entry);
}
// A publisher's earlier versions of these plugins stay for IDEs that can't load the new
// ones (pruned below); plugins no longer in the folder go.
index.plugins.retain(|e| match publisher {
Some(p) => {
e.publisher != p
|| (added.iter().any(|n| n.id == e.id)
&& !added.iter().any(|n| (&n.id, &n.version) == (&e.id, &e.version)))
}
None => false,
});
index.plugins.extend(added);
index.prune_versions();
write_index(&index, &out)
}
/// `publishers` with `key` as `publisher`'s current key. Earlier keys stay listed
/// (packages they signed remain valid until revoked); a key new to the
/// publisher is endorsed by `previous`, one of them, so installed plugins
/// signed with it update without asking.
fn register_key(
mut publishers: Vec<package::Publisher>,
publisher: &str,
name: &str,
key: &PublisherKey,
previous: Option<&PublisherKey>,
) -> Result<Vec<package::Publisher>, String> {
let current = key.public_base64();
let earlier: Vec<KeyEntry> = publishers.iter().filter(|p| p.id == publisher).flat_map(|p| p.keys.clone()).collect();
let entry = match earlier.iter().find(|k| k.key == current) {
Some(listed) => listed.clone(),
None => {
let endorsement = match previous {
Some(prev) => {
let prev_key = prev.public_base64();
if !earlier.iter().any(|k| k.key == prev_key && !k.revoked) {
return Err(format!("--previous-key is not one of {publisher}'s valid keys in the index"));
}
Some(prev.endorse(publisher, &current))
}
None => {
if earlier.iter().any(|k| !k.revoked) {
println!(
" warning: {publisher}'s new key is not endorsed (--previous-key): users who installed \
its plugins will be asked to confirm the change when they update"
);
}
None
}
};
KeyEntry { endorsement, ..KeyEntry::new(current.clone()) }
}
};
let keys = std::iter::once(entry).chain(earlier.into_iter().filter(|k| k.key != current)).collect();
publishers.retain(|p| p.id != publisher);
publishers.push(package::Publisher { id: publisher.into(), name: name.into(), keys });
Ok(publishers)
}
/// `index`, unless a newer `openidea-plugin` wrote it (rewriting it would drop what this one can't read).
fn current_format(index: Index, file: &str) -> Result<Index, String> {
if index.is_newer() {
return Err(format!(
"{file}: index format {} is newer than this tool writes; update openidea-plugin",
index.format
));
}
Ok(index)
}
fn cmd_merge(args: &[String]) -> Result<(), String> {
let a = Args::parse(args, &[], &["keep", "out"])?;
let [base, other] = a.positional.as_slice() else {
return Err("usage: openidea-plugin merge <index> <other-index> --keep ID [--out FILE]".into());
};
let keep = a.value("keep").ok_or("--keep is required")?;
let out = a.value("out").unwrap_or(base);
let mut index =
current_format(Index::parse(&std::fs::read_to_string(base).map_err(|e| format!("{base}: {e}"))?)?, base)?;
let bytes = read_market_url(other, 16 << 20)?;
let theirs = Index::parse(&String::from_utf8(bytes).map_err(|_| format!("{other}: not UTF-8"))?)?;
let mut theirs = current_format(theirs, other)?;
// Their downloads are relative to their index, not to ours.
for e in &mut theirs.plugins {
e.download = resolve_market_href(other, &e.download);
}
let count = theirs.plugins.len();
index.include(theirs, &[keep]).map_err(|e| format!("{other}: {e}"))?;
println!(" {count} plugins from {other}");
write_index(&index, Path::new(out))
}
fn write_index(index: &Index, out: &Path) -> Result<(), String> {
let json = serde_json::to_string_pretty(index).map_err(|e| e.to_string())?;
std::fs::write(out, json).map_err(|e| format!("{}: {e}", out.display()))?;
println!("-> {} ({} plugins)", out.display(), index.plugins.len());
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_new_key_is_endorsed_by_the_previous_one() {
let (k1, k2, k3) =
(PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([2; 32]), PublisherKey::from_seed([3; 32]));
let first = register_key(Vec::new(), "acme", "ACME", &k1, None).unwrap();
let rotated = register_key(first.clone(), "acme", "ACME", &k2, Some(&k1)).unwrap();
let index = Index { format: package::INDEX_FORMAT, publishers: rotated.clone(), ..Index::default() };
assert!(index.succeeds("acme", &k1.public_base64(), &k2.public_base64()));
assert_eq!(rotated[0].keys.len(), 2, "the earlier key stays listed");
// Registering the current key again keeps its endorsement.
let again = register_key(rotated.clone(), "acme", "ACME", &k2, None).unwrap();
assert_eq!(again, rotated);
// Only a key the publisher has can endorse.
assert!(register_key(first.clone(), "acme", "ACME", &k2, Some(&k3)).unwrap_err().contains("--previous-key"));
// Without one the new key is listed unendorsed.
let unendorsed = register_key(first, "acme", "ACME", &k3, None).unwrap();
assert!(unendorsed[0].keys[0].endorsement.is_none());
}
#[test]
fn names() {
assert_eq!(slugify("My Cool_Plugin"), "my-cool-plugin");
assert_eq!(slugify("acme.Tools"), "acme.tools");
assert_eq!(title_case("material-icons"), "Material Icons");
assert_eq!(
relative(Path::new("/a/b/plugins/x"), Path::new("/a/b/sdk/rust/p")).unwrap(),
PathBuf::from("../../sdk/rust/p")
);
assert_eq!(relative(Path::new("/x/y"), Path::new("/z")), None);
}
/// A theme package `id` signed by `publisher` with `key`, packed into `out`.
fn signed_theme(tmp: &Path, id: &str, publisher: &str, key: &str, out: &Path) {
signed_theme_version(tmp, id, "0.1.0", "^1.0", publisher, key, out);
}
fn signed_theme_version(tmp: &Path, id: &str, version: &str, engine: &str, publisher: &str, key: &str, out: &Path) {
let src = tmp.join("src").join(format!("{id}-{version}"));
let vars = templates::Vars { id, name: id, sdk: None };
for (rel, content) in templates::render(templates::Template::Theme, &vars) {
let path = src.join(rel);
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
let content = content
.replace("version = \"0.1.0\"", &format!("version = \"{version}\""))
.replace("engines.openidea = \"^1.0\"", &format!("engines.openidea = \"{engine}\""));
std::fs::write(path, content).unwrap();
}
std::fs::create_dir_all(out).unwrap();
let args = [src.to_str().unwrap(), "--key", key, "--publisher", publisher, "--out"]
.into_iter()
.map(String::from)
.chain([out.join(format!("{id}-{version}.oiplugin")).to_string_lossy().into_owned()])
.collect::<Vec<_>>();
cmd_pack(&args).unwrap();
}
#[test]
fn a_release_keeps_the_versions_older_ides_need() {
let tmp = tempfile::tempdir().unwrap();
let t = tmp.path();
let key = t.join("acme.key");
std::fs::write(&key, PublisherKey::from_seed([5; 32]).seed_base64()).unwrap();
let key = key.to_string_lossy().into_owned();
let index_file = t.join("index.json");
let release = |n: usize, packages: &[(&str, &str, &str)]| {
let dir = t.join(format!("release-{n}"));
for (id, version, engine) in packages {
signed_theme_version(t, id, version, engine, "acme", &key, &dir);
}
let base = format!("https://example.com/download/v{n}");
let args = [dir.to_str().unwrap(), "--publisher", "acme", "--key", &key, "--download-base", &base];
cmd_index(&strings(&[&args[..], &["--out", index_file.to_str().unwrap()]].concat())).unwrap();
let index = Index::parse(&std::fs::read_to_string(&index_file).unwrap()).unwrap();
index.plugins.iter().map(|e| format!("{} {} {}", e.id, e.version, e.download)).collect::<Vec<_>>()
};
release(1, &[("acme.a", "1.0.0", "^1.0"), ("acme.b", "1.0.0", "^1.0")]);
// Same requirement: the newer version replaces it; acme.b is gone from the release.
assert_eq!(
release(2, &[("acme.a", "1.1.0", "^1.0")]),
["acme.a 1.1.0 https://example.com/download/v2/acme.a-1.1.0.oiplugin"]
);
// Another API requirement: the last version for the earlier one stays, where it was published.
assert_eq!(
release(3, &[("acme.a", "2.0.0", ">=1.0, <2")]),
[
"acme.a 1.1.0 https://example.com/download/v2/acme.a-1.1.0.oiplugin",
"acme.a 2.0.0 https://example.com/download/v3/acme.a-2.0.0.oiplugin",
]
);
let index = Index::parse(&std::fs::read_to_string(&index_file).unwrap()).unwrap();
assert_eq!(index.offered()[0].0.version, "2.0.0", "the newest this IDE can load");
}
fn strings(args: &[&str]) -> Vec<String> {
args.iter().map(|a| a.to_string()).collect()
}
#[test]
fn official_and_community_indexes_merge() {
let tmp = tempfile::tempdir().unwrap();
let t = tmp.path();
let mut keys = Vec::new();
for (name, seed) in [("openidea", 1), ("alice", 2), ("carol", 3)] {
let file = t.join(format!("{name}.key"));
let key = PublisherKey::from_seed([seed; 32]);
std::fs::write(&file, key.seed_base64()).unwrap();
keys.push((file.to_string_lossy().into_owned(), key.public_base64()));
}
let (official, community) = (t.join("official"), t.join("community"));
signed_theme(t, "openidea.night", "openidea", &keys[0].0, &official);
signed_theme(t, "alice.dawn", "alice", &keys[1].0, &community);
let official_index = official.join("index.json");
let community_index = community.join("index.json");
let release = strings(&[official.to_str().unwrap(), "--publisher", "openidea", "--key", &keys[0].0]);
cmd_index(&release).unwrap();
// The community index lists its publishers; `index` without a key checks packages against them.
let skeleton = Index {
format: package::INDEX_FORMAT,
publishers: vec![package::Publisher {
id: "alice".into(),
name: "Alice".into(),
keys: vec![KeyEntry::new(keys[1].1.clone())],
}],
..Index::default()
};
write_index(&skeleton, &community_index).unwrap();
cmd_index(&strings(&[community.to_str().unwrap()])).unwrap();
cmd_merge(&strings(&[
official_index.to_str().unwrap(),
community_index.to_str().unwrap(),
"--keep",
"openidea",
]))
.unwrap();
// A release rebuilds the official plugins and keeps the community's.
cmd_index(&release).unwrap();
let index = Index::parse(&std::fs::read_to_string(&official_index).unwrap()).unwrap();
let listed: Vec<_> = index.plugins.iter().map(|e| (e.id.as_str(), e.publisher.as_str())).collect();
assert_eq!(listed, [("alice.dawn", "alice"), ("openidea.night", "openidea")]);
assert_eq!(index.plugins[1].categories, ["themes"], "implied by the theme");
assert_eq!(index.plugins[1].download, "openidea.night-0.1.0.oiplugin");
assert!(Path::new(&index.plugins[0].download).is_absolute(), "{}", index.plugins[0].download);
// A package by a publisher the index doesn't know is refused.
signed_theme(t, "carol.noon", "carol", &keys[2].0, &community);
let err = cmd_index(&strings(&[community.to_str().unwrap()])).unwrap_err();
assert!(err.contains("unknown publisher"), "{err}");
}
#[test]
fn every_template_is_a_valid_package() {
for t in templates::Template::ALL {
let dir =
std::env::temp_dir().join(format!("openidea-plugin-template-{}-{}", t.label(), std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
let vars = templates::Vars { id: "acme.sample", name: "Sample", sdk: Some("../sdk") };
for (rel, content) in templates::render(t, &vars) {
let path = dir.join(rel);
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
std::fs::write(path, content).unwrap();
}
let m = Manifest::load(&dir).unwrap_or_else(|e| panic!("{}: {e}", t.label()));
assert_eq!(m.id, "acme.sample");
if m.wasm.is_none() {
check(&dir).unwrap_or_else(|e| panic!("{}: {e}", t.label()));
}
std::fs::remove_dir_all(&dir).unwrap();
}
}
}
@@ -0,0 +1,281 @@
//! The files of `openidea-plugin new`. Placeholders: `__ID__` (plugin id),
//! `__SLUG__` (last id segment), `__NAME__`, `__CRATE__`, `__SDK__`.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Template {
Command,
Theme,
IconTheme,
Language,
}
impl Template {
pub const ALL: [Template; 4] = [Self::Command, Self::Theme, Self::IconTheme, Self::Language];
pub fn parse(s: &str) -> Result<Self, String> {
Self::ALL
.into_iter()
.find(|t| t.label() == s)
.ok_or_else(|| format!("unknown template {s:?} (command, theme, icon-theme, language)"))
}
pub fn label(self) -> &'static str {
match self {
Self::Command => "command",
Self::Theme => "theme",
Self::IconTheme => "icon-theme",
Self::Language => "language",
}
}
}
pub struct Vars<'a> {
pub id: &'a str,
pub name: &'a str,
pub sdk: Option<&'a str>,
}
/// `(relative path, contents)` of every file of the template.
pub fn render(template: Template, vars: &Vars) -> Vec<(String, String)> {
let slug = vars.id.rsplit('.').next().unwrap_or(vars.id).to_owned();
let crate_name = format!("openidea-plugin-{}", vars.id.replace('.', "-"));
let fill = |s: &str| {
s.replace("__ID__", vars.id)
.replace("__SLUG__", &slug)
.replace("__NAME__", vars.name)
.replace("__CRATE__", &crate_name)
.replace("__SDK__", vars.sdk.unwrap_or(""))
// The theme files come from the Harbor Night plugin.
.replace("harbor-night", &slug)
.replace("Harbor Night", vars.name)
};
let files: Vec<(String, &str)> = match template {
Template::Command => vec![
("plugin.toml".into(), COMMAND_MANIFEST),
("Cargo.toml".into(), COMMAND_CARGO),
("src/lib.rs".into(), COMMAND_LIB),
(".gitignore".into(), "/target\n"),
("README.md".into(), COMMAND_README),
],
Template::Theme => vec![
("plugin.toml".into(), THEME_MANIFEST),
(format!("themes/{slug}.theme.json"), THEME_UI),
(format!("themes/{slug}.scheme.json"), THEME_SCHEME),
],
Template::IconTheme => vec![
("plugin.toml".into(), ICON_MANIFEST),
("icons/theme.json".into(), ICON_THEME),
("icons/file.svg".into(), ICON_FILE),
("icons/folder.svg".into(), ICON_FOLDER),
("icons/folder-open.svg".into(), ICON_FOLDER_OPEN),
("icons/markdown.svg".into(), ICON_MARKDOWN),
],
Template::Language => {
vec![("plugin.toml".into(), LANGUAGE_MANIFEST), (format!("languages/{slug}.toml"), LANGUAGE_DEF)]
}
};
files.into_iter().map(|(path, content)| (path, fill(content))).collect()
}
const COMMAND_MANIFEST: &str = r#"id = "__ID__"
name = "__NAME__"
version = "0.1.0"
description = "Adds two sample commands."
authors = []
# Marketplace categories (up to 3): languages, themes, icon-themes, linters,
# vcs, ai, frameworks, tools, other.
categories = ["tools"]
engines.openidea = "^1.0"
# Built by `openidea-plugin build`. The code is loaded the first time one of
# its commands runs; the commands below show in Find Action before that.
wasm = "dist/plugin.wasm"
# What the code may do. The user approves these when installing.
[capabilities]
required = ["editor.read"]
[[contributes.commands]]
id = "say-hello"
title = "__NAME__: Say Hello"
category = "__NAME__"
[[contributes.commands]]
id = "line-count"
title = "__NAME__: Line Count of Active File"
category = "__NAME__"
"#;
const COMMAND_CARGO: &str = r#"[package]
name = "__CRATE__"
version = "0.1.0"
edition = "2024"
publish = false
# A plugin builds on its own (target wasm32-wasip2), outside any IDE workspace.
[workspace]
[lib]
crate-type = ["cdylib", "rlib"]
[dependencies]
openidea-plugin = { path = "__SDK__" }
[profile.release]
opt-level = "s"
lto = true
strip = true
codegen-units = 1
panic = "abort"
"#;
const COMMAND_LIB: &str = r#"//! __NAME__: an OpenIDEA plugin.
use openidea_plugin::{Invocation, NotificationLevel, Plugin, host, register_plugin};
#[derive(Default)]
pub struct Main {
greetings: u32,
}
impl Plugin for Main {
fn run_command(&mut self, id: &str, ctx: &Invocation, _arg: Option<&str>) -> Result<Option<String>, String> {
match id {
"say-hello" => {
// Instance state lasts until the plugin is idle for a while;
// keep what must survive in `host::storage`.
self.greetings += 1;
let body = format!("Hello #{} from __NAME__!", self.greetings);
host::notify(NotificationLevel::Info, "__NAME__", &body);
Ok(None)
}
"line-count" => {
let editor = ctx.editor().ok_or("no file is open")?;
let text = editor.document()?.text;
Ok(Some(format!("{} lines in {}", text.lines().count(), editor.path())))
}
_ => Err(format!("unknown command {id}")),
}
}
}
register_plugin!(Main);
#[cfg(test)]
mod tests {
use super::*;
use openidea_plugin::mock::MockHost;
#[test]
fn says_hello() {
let host = MockHost::install();
let mut plugin = Main::default();
plugin.run_command("say-hello", &host.invocation(), None).unwrap();
plugin.run_command("say-hello", &host.invocation(), None).unwrap();
assert_eq!(host.notifications()[1].body, "Hello #2 from __NAME__!");
}
#[test]
fn counts_lines() {
let host = MockHost::install();
host.open_editor("/project/src/main.rs", "rust", "fn main() {\n}\n", &[]);
let out = Main::default().run_command("line-count", &host.invocation(), None).unwrap();
assert_eq!(out.as_deref(), Some("2 lines in /project/src/main.rs"));
}
}
"#;
const COMMAND_README: &str = r#"# __NAME__
An OpenIDEA plugin.
```sh
cargo test # logic, against the SDK's mock host
openidea-plugin build # → dist/plugin.wasm, validated like an install
openidea-plugin link # load this folder in the IDE (Settings › Plugins)
```
`plugin.toml` declares the commands, so they appear in Find Action before the
code is loaded, and the capabilities the code needs. `src/lib.rs` implements
them with the `openidea-plugin` SDK.
"#;
const THEME_MANIFEST: &str = r#"id = "__ID__"
name = "__NAME__"
version = "0.1.0"
description = "A dark theme."
authors = []
engines.openidea = "^1.0"
# One entry per theme. `editor-scheme` is optional: its tokens are merged over
# the theme's (IntelliJ's .theme.json + .icls pair). Reference: docs/plugins/themes.md.
[[contributes.themes]]
path = "themes/__SLUG__.theme.json"
editor-scheme = "themes/__SLUG__.scheme.json"
"#;
const THEME_UI: &str = include_str!("../../../../../plugins/official/harbor-night/themes/harbor-night.theme.json");
const THEME_SCHEME: &str = include_str!("../../../../../plugins/official/harbor-night/themes/harbor-night.scheme.json");
const ICON_MANIFEST: &str = r#"id = "__ID__"
name = "__NAME__"
version = "0.1.0"
description = "File and folder icons."
authors = []
engines.openidea = "^1.0"
# Apply it in Settings › Plugins › __NAME__. SVGs are sanitised on install:
# no scripts, event handlers, external references or entities.
[[contributes.iconThemes]]
id = "__SLUG__"
label = "__NAME__"
path = "icons/theme.json"
"#;
const ICON_THEME: &str = r#"{
"iconDefinitions": {
"file": "icons/file.svg",
"folder": "icons/folder.svg",
"folder-open": "icons/folder-open.svg",
"markdown": "icons/markdown.svg"
},
"file": "file",
"folder": "folder",
"folderExpanded": "folder-open",
"fileExtensions": { "md": "markdown", "markdown": "markdown" },
"fileNames": { "readme": "markdown" },
"folderNames": {}
}
"#;
const ICON_FILE: &str = r##"<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16"><path fill="#90a4ae" d="M4 1.5h5.5L13 5v9.5H4z"/><path fill="#cfd8dc" d="M9.5 1.5V5H13z"/></svg>
"##;
const ICON_FOLDER: &str = r##"<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16"><path fill="#90a4ae" d="M1.5 3h5l1.5 1.5h6.5v9h-13z"/></svg>
"##;
const ICON_FOLDER_OPEN: &str = r##"<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16"><path fill="#90a4ae" d="M1.5 3h5l1.5 1.5h6v2h-10l-2 7h-.5z"/><path fill="#b0bec5" d="M4 7.5h11.5l-2 6H2z"/></svg>
"##;
const ICON_MARKDOWN: &str = r##"<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16"><rect x="1" y="3.5" width="14" height="9" rx="1.5" fill="#42a5f5"/><path fill="#fff" d="M3 10.5v-5h1.5L6 7.5l1.5-2H9v5H7.5V8L6 10 4.5 8v2.5zm8 0L9.5 8h1V5.5H12V8h1z"/></svg>
"##;
const LANGUAGE_MANIFEST: &str = r#"id = "__ID__"
name = "__NAME__"
version = "0.1.0"
description = "Language support for .__SLUG__ files: detection, comments and indentation."
authors = []
engines.openidea = "^1.0"
# Same format as languages/<id>.toml in the IDE repository.
[[contributes.languages]]
path = "languages/__SLUG__.toml"
"#;
const LANGUAGE_DEF: &str = r##"id = "__SLUG__"
name = "__NAME__"
extensions = ["__SLUG__"]
line-comment = "#"
indent = { tab-size = 2, insert-spaces = true }
# Borrow an existing CodeMirror mode for instant highlighting.
ui-mode = "Properties files"
"##;
+30 -1
View File
@@ -1,6 +1,8 @@
//! `openidea-server --stdio`: the IDE backend for remote development, serving
//! one desktop client over stdin / stdout (started by the client over SSH).
use ide_host::plugins::remote::{RemotePluginLanguages, SYNC_COMMAND};
use ide_host::plugins::remote_host::RemotePlugins;
use ide_host::rpc::{self, Dispatcher};
use ide_host::{Host, HostConfig};
use ide_language::LanguageRegistry;
@@ -51,8 +53,12 @@ async fn run(base: PathBuf) -> i32 {
return 1;
}
};
// The desktop's plugin languages (`plugin_sync_languages`); programs pinned on this machine.
let plugin_languages = RemotePluginLanguages::new(Some(&data_dir));
languages.attach_plugins(plugin_languages.store().clone());
let peer = Arc::new(OnceLock::new());
let sink_peer = peer.clone();
let plugins_data = data_dir.clone();
let host = Arc::new(Host::new(HostConfig {
config_dir,
local_data_dir: data_dir.clone(),
@@ -64,8 +70,31 @@ async fn run(base: PathBuf) -> i32 {
runtime: tokio::runtime::Handle::current(),
project_environment: true,
}));
// The desktop's code plugins run here, next to the project (`plugin_sync_packages`).
let plugins = match RemotePlugins::start(&plugins_data, &host, peer.clone()) {
Ok(p) => Some(p),
Err(e) => {
tracing::warn!("plugins are unavailable: {e}");
None
}
};
tracing::info!("openidea-server {} ready", env!("CARGO_PKG_VERSION"));
rpc::serve(host.clone(), Dispatcher::standard(), peer, tokio::io::stdin(), tokio::io::stdout()).await;
let mut dispatcher = Dispatcher::standard();
if let Some(plugins) = &plugins {
plugins.register(&mut dispatcher);
}
dispatcher.add(SYNC_COMMAND, move |c| {
let remote = plugin_languages.clone();
async move {
let report = remote.sync(c.arg("languages")?);
c.host.plugin_languages_changed();
Ok(report)
}
});
rpc::serve(host.clone(), dispatcher, peer, tokio::io::stdin(), tokio::io::stdout()).await;
if let Some(plugins) = &plugins {
plugins.host().shutdown();
}
// `process::exit` skips destructors: nothing started for the client may outlive it.
if tokio::time::timeout(std::time::Duration::from_secs(10), host.shutdown()).await.is_err() {
tracing::warn!("language servers did not shut down in time");
+13 -1
View File
@@ -1,6 +1,6 @@
//! The IDE backend without a UI toolkit: open projects per window, terminals,
//! processes, settings and the remote services. The desktop shell (`src-tauri`)
//! wraps one [`Host`] and adds windows, menus and plugins; `openidea-server`
//! wraps one [`Host`] and adds windows and menus; `openidea-server`
//! runs one on a remote machine and serves it over JSON-RPC ([`rpc`]).
pub mod data_dirs;
@@ -10,6 +10,7 @@ mod java_runtime;
mod kotlin_bridge;
pub mod notebook;
pub mod ops;
pub mod plugins;
mod project;
mod proxy;
mod requests;
@@ -133,6 +134,17 @@ impl Host {
self.syntax.get_or_init(|| Arc::new(Syntax::new())).clone()
}
/// Plugin languages or the servers they may start changed (they are part
/// of [`Self::languages`] and every project's fork): open documents are
/// detected again and language servers follow their new configuration.
pub fn plugin_languages_changed(&self) {
for project in self.projects() {
project.documents.redetect_languages();
let lsp = project.lsp.clone();
self.runtime.spawn(async move { lsp.reconfigure().await });
}
}
/// Every open project (all windows).
pub fn projects(&self) -> Vec<Arc<Project>> {
self.projects.read().values().cloned().collect()
+12
View File
@@ -30,6 +30,17 @@ pub fn set_command(host: &Host, window: &str, id: String, command: Option<String
host.project(window)?.agents.set_builtin_command(&id, command)
}
/// Environment variables built-in agent `id` starts with; empty clears them.
pub fn set_env(
host: &Host,
window: &str,
id: String,
env: std::collections::BTreeMap<String, String>,
) -> IdeResult<()> {
host.require_trust(window)?;
host.project(window)?.agents.set_builtin_env(&id, env)
}
/// Where ACP Registry agents are saved (`<config>/agents`, like the user's own) and installed.
pub(crate) fn registry_paths(host: &Host) -> RegistryPaths {
RegistryPaths::new(&host.config_dir, &host.local_data_dir)
@@ -195,6 +206,7 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("agent_subscribe", |c| async move { subscribe(&c.host, &c.window, c.channel("onEvent")?) });
d.add("agent_providers", |c| async move { providers(&c.host, &c.window) });
d.add("agent_set_command", |c| async move { set_command(&c.host, &c.window, c.arg("id")?, c.arg("command")?) });
d.add("agent_set_env", |c| async move { set_env(&c.host, &c.window, c.arg("id")?, c.arg("env")?) });
d.add("agent_registry_list", |c| async move { registry_list(&c.host, &c.window, c.arg("request")?).await });
d.add("agent_registry_install", |c| async move {
registry_install(&c.host, &c.window, c.arg("id")?, c.arg("request")?, c.sender("onEvent")?).await
+73 -2
View File
@@ -116,6 +116,11 @@ pub async fn build_sync(
tracing::warn!("Maven profiles not saved: {e}");
false
});
if profiles_changed {
// The modules of the chosen profiles are read from the POMs with them from now on (the
// project tree's module folders too), whether or not Maven itself can run.
project.sink.emit(ide_core::IdeEvent::ProjectRootsChanged { root: project.root.clone().into() });
}
let token = host.begin_project_request(window, request);
let result = tokio::task::spawn_blocking(move || {
let current = load(&root, &cache, &settings);
@@ -217,6 +222,52 @@ fn declaring_pom(
)))
}
/// The `.toml` of a version catalog the build imports as `group:artifact:version`, once a
/// build has downloaded it (`ide_sdk::libraries::published_version_catalog`).
pub async fn gradle_published_catalog(
host: &Host,
window: &str,
coordinates: String,
) -> IdeResult<Option<ide_core::IpcPath>> {
let project = host.project(window)?;
tokio::task::spawn_blocking(move || {
ide_sdk::libraries::published_version_catalog(&project.root, &project.settings.effective(), &coordinates)
.map(Into::into)
})
.await
.map_err(join_err)
}
/// The resolved dependency tree of the Gradle project `gradle_path` (`:`, `:app`) for
/// `configuration`, from Gradle's `dependencies` report; cancellable with `cancel_request(request)`.
pub async fn gradle_dependencies(
host: &Host,
window: &str,
gradle_path: String,
configuration: String,
offline: bool,
request: RequestId,
) -> IdeResult<Vec<ide_build::GradleDependency>> {
host.require_trust(window)?;
let project = host.project(window)?;
let (root, cache, settings) = (project.root.clone(), project.build_cache_dir(), project.settings.effective());
let token = host.begin_project_request(window, request);
let result = tokio::task::spawn_blocking(move || {
let gradle = load(&root, &cache, &settings).gradle;
let options = SyncOptions {
offline,
env: gradle.as_ref().map(|g| g.env.clone()).unwrap_or_default(),
gradle_command: gradle.as_ref().map(|g| g.command.clone()),
..Default::default()
};
ide_build::gradle_dependencies(&root, &options, &gradle_path, &configuration, &token)
})
.await
.map_err(join_err);
host.end_request(request);
result?
}
/// Where the direct dependency `dependency` of the Maven module `pom` is declared.
pub async fn maven_dependency_declaration(
host: &Host,
@@ -296,7 +347,7 @@ pub async fn build_run(
let settings = ide_workspace::SettingsStore::load(&config_dir, Some(member)).effective();
let member_cache = cache.join(crate::project::project_key(member));
let model = with_profiles(load(member, &member_cache, &settings), &member_cache);
let plan = compile::plan(member, &model, kind, file.as_deref())?;
let plan = compile::plan(member, &model, kind, file.as_deref(), settings.cmake.profiles.as_deref())?;
match &mut combined {
Some(combined) => combined.steps.extend(plan.steps),
None => combined = Some(plan),
@@ -306,7 +357,13 @@ pub async fn build_run(
plan.title = workspace.name;
plan
} else {
compile::plan(&root, &with_profiles(load(&root, &cache, &settings), &cache), kind, file.as_deref())?
compile::plan(
&root,
&with_profiles(load(&root, &cache, &settings), &cache),
kind,
file.as_deref(),
settings.cmake.profiles.as_deref(),
)?
};
let result = compile::run(&root, &plan, &token, &|e| {
let _ = on_event.send(e);
@@ -337,6 +394,20 @@ pub(crate) fn register(d: &mut Dispatcher) {
)
.await
});
d.add("gradle_published_catalog", |c| async move {
gradle_published_catalog(&c.host, &c.window, c.arg("coordinates")?).await
});
d.add("gradle_dependencies", |c| async move {
gradle_dependencies(
&c.host,
&c.window,
c.arg("gradlePath")?,
c.arg("configuration")?,
c.arg("offline")?,
c.arg("request")?,
)
.await
});
d.add("maven_dependency_declaration", |c| async move {
maven_dependency_declaration(&c.host, &c.window, c.arg("pom")?, c.arg("dependency")?).await
});
+1 -2
View File
@@ -216,8 +216,7 @@ pub async fn connect(host: &Host, window: &str, id: &str) -> IdeResult<()> {
/// manual mode rolls back what was not committed).
pub async fn set_manual_commit(host: &Host, window: &str, id: &str, manual: bool) -> IdeResult<()> {
let db = host.project(window)?.db.clone();
db.set_manual_commit(id, manual).await;
Ok(())
db.set_manual_commit(id, manual).await
}
/// Commits or rolls back the open transaction of a manual-commit data source.
+25
View File
@@ -188,6 +188,19 @@ pub async fn debug_stop(host: &Host, _window: &str, session: DebugSessionId) ->
host.debugger.stop(session).await
}
/// The adapter's completions for an expression being typed in the frame `frame_id`.
pub async fn debug_completions(
host: &Host,
_window: &str,
session: DebugSessionId,
frame_id: Option<i64>,
text: String,
line: u32,
column: u32,
) -> IdeResult<Vec<ide_dap::DebugCompletion>> {
host.debugger.completions(session, frame_id, text, line, column).await
}
/// Evaluate with an explicit DAP context: `repl`, `watch`, `hover` or `clipboard`.
pub async fn debug_evaluate_in(
host: &Host,
@@ -376,6 +389,18 @@ pub(crate) fn register(d: &mut Dispatcher) {
.await
});
d.add("debug_stop", |c| async move { debug_stop(&c.host, &c.window, c.arg("session")?).await });
d.add("debug_completions", |c| async move {
debug_completions(
&c.host,
&c.window,
c.arg("session")?,
c.arg("frameId")?,
c.arg("text")?,
c.arg("line")?,
c.arg("column")?,
)
.await
});
d.add("debug_evaluate_in", |c| async move {
debug_evaluate_in(
&c.host,
+17
View File
@@ -0,0 +1,17 @@
//! Docker run configurations: the environment their `docker` command line runs with.
use crate::Host;
use crate::rpc::Dispatcher;
use ide_core::IdeResult;
use ide_docker::DockerEndpoint;
use std::collections::BTreeMap;
/// The endpoint's variables plus a `PATH` that finds this host's `docker`. A remote project
/// gets them from the server, since its run configurations start on the remote host.
pub async fn run_env(host: &Host, endpoint: DockerEndpoint) -> IdeResult<BTreeMap<String, String>> {
endpoint.validate()?;
host.blocking(move || Ok(ide_docker::run_env(&endpoint))).await
}
pub(crate) fn register(d: &mut Dispatcher) {
d.add("docker_run_env", |c| async move { run_env(&c.host, c.arg("endpoint")?).await });
}
+276 -6
View File
@@ -1,9 +1,11 @@
use super::doc::blocking;
use crate::Host;
use crate::apply_workspace_edit as apply_edit;
use crate::plugins::{ProviderKind, editor_providers_for as editor_providers};
use crate::rpc::Dispatcher;
use ide_core::{DocId, IdeResult, Location, Pos, Range, RequestId, ServerId, TextEdit, WorkspaceEdit};
use ide_lsp::types::*;
use ide_plugin::ProviderDocument;
async fn prepare_actions(
project: std::sync::Arc<crate::Project>,
@@ -38,9 +40,99 @@ pub async fn lsp_completion(
) -> IdeResult<CompletionList> {
let lsp = host.project(window)?.lsp.clone();
let cancel = host.begin_request(request);
let result = lsp.completion(doc, pos, trigger, cancel).await;
let result = lsp.completion(doc, pos, trigger.clone(), cancel).await;
host.end_request(request);
result
// Plugin providers (`[[contributes.providers]]`) add theirs.
let Some((plugins, d)) = plugin_providers(host, window, doc, ProviderKind::Completion) else { return result };
let items = plugin_call(move || plugins.provide_completion(&d, pos, trigger.as_deref())).await.unwrap_or_default();
if items.is_empty() {
return result;
}
let mut list = result.unwrap_or(CompletionList { is_incomplete: false, items: Vec::new() });
list.items.extend(items.into_iter().map(|(_, item)| plugin_completion(item)));
Ok(list)
}
/// The plugin host and the document, when some plugin provides `kind`.
fn plugin_providers(
host: &Host,
window: &str,
doc: DocId,
kind: ProviderKind,
) -> Option<(std::sync::Arc<crate::plugins::PluginHost>, ProviderDocument)> {
let plugins = editor_providers(kind)?;
let snap = host.project(window).ok()?.documents.snapshot(doc).ok()?;
let d = ProviderDocument {
path: snap.path.0,
language: snap.language.0,
version: u64::from(snap.version),
text: snap.text,
};
Some((plugins, d))
}
/// Plugin calls block (the plugin's actor thread does the work).
async fn plugin_call<T: Send + 'static>(f: impl FnOnce() -> T + Send + 'static) -> Option<T> {
tokio::task::spawn_blocking(f).await.ok()
}
/// Items from plugins carry no server (resolve and commands don't apply).
const PLUGIN_SERVER: ServerId = ServerId(0);
/// A plugin's `completion-item.kind` (wit/providers.wit lists them).
fn plugin_completion_kind(kind: &str) -> Option<CompletionKind> {
Some(match kind {
"text" => CompletionKind::Text,
"method" => CompletionKind::Method,
"function" => CompletionKind::Function,
"constructor" => CompletionKind::Constructor,
"field" => CompletionKind::Field,
"variable" => CompletionKind::Variable,
"class" => CompletionKind::Class,
"interface" => CompletionKind::Interface,
"module" => CompletionKind::Module,
"property" => CompletionKind::Property,
"unit" => CompletionKind::Unit,
"value" => CompletionKind::Value,
"enum" => CompletionKind::Enum,
"keyword" => CompletionKind::Keyword,
"snippet" => CompletionKind::Snippet,
"color" => CompletionKind::Color,
"file" => CompletionKind::File,
"reference" => CompletionKind::Reference,
"folder" => CompletionKind::Folder,
"enum-member" => CompletionKind::EnumMember,
"constant" => CompletionKind::Constant,
"struct" => CompletionKind::Struct,
"event" => CompletionKind::Event,
"operator" => CompletionKind::Operator,
"type-parameter" => CompletionKind::TypeParameter,
_ => return None,
})
}
fn plugin_completion(item: ide_plugin::CompletionItem) -> CompletionItem {
let kind = item.kind.as_deref().and_then(plugin_completion_kind);
CompletionItem {
label: item.label,
label_detail: None,
detail: item.detail,
kind,
documentation: item.documentation.map(|value| Markup { kind: "markdown".into(), value }),
sort_text: item.sort_text,
filter_text: None,
preselect: false,
deprecated: false,
insert_text: item.insert_text,
is_snippet: false,
replace_range: item.replace,
replace_range_tab: None,
additional_edits: Vec::new(),
commit_characters: Vec::new(),
command: None,
data: None,
server: PLUGIN_SERVER,
}
}
/// Inline (ghost text) suggestions at `pos`; `automatic` when triggered by typing.
@@ -54,9 +146,26 @@ pub async fn lsp_inline_completion(
) -> IdeResult<Vec<InlineCompletionItem>> {
let lsp = host.project(window)?.lsp.clone();
let cancel = host.begin_request(request);
let result = lsp.inline_completion(doc, pos, automatic, cancel).await;
let result = lsp.inline_completion(doc, pos, automatic, cancel.clone()).await;
// Plugins: cancelled (cooperatively) when the editor cancels the request.
let plugin = match plugin_providers(host, window, doc, ProviderKind::InlineCompletion) {
Some((plugins, d)) if !matches!(&result, Ok(items) if !items.is_empty()) => {
let cancel = cancel.clone();
plugin_call(move || plugins.provide_inline_completion(&d, pos, &cancel)).await.flatten()
}
_ => None,
};
host.end_request(request);
result
match plugin {
Some((_, s)) => Ok(vec![InlineCompletionItem {
insert_text: s.insert_text,
range: s.replace,
filter_text: None,
command: None,
server: PLUGIN_SERVER,
}]),
None => result,
}
}
pub async fn lsp_resolve_completion(
@@ -87,7 +196,40 @@ pub async fn lsp_signature_help(host: &Host, window: &str, doc: DocId, pos: Pos)
}
pub async fn lsp_goto(host: &Host, window: &str, doc: DocId, pos: Pos, kind: GotoKind) -> IdeResult<Vec<Location>> {
host.project(window)?.lsp.goto(doc, pos, kind).await
let result = host.project(window)?.lsp.goto(doc, pos, kind).await;
if kind != GotoKind::Definition {
return result;
}
let Some((plugins, d)) = plugin_providers(host, window, doc, ProviderKind::Definition) else { return result };
let found = plugin_call(move || plugins.provide_definition(&d, pos)).await.unwrap_or_default();
if found.is_empty() {
return result;
}
let mut out = result.unwrap_or_default();
out.extend(found.into_iter().map(|l| Location { path: l.path.into(), range: l.range, uri: None }));
Ok(out)
}
/// Gutter markers from plugins (`line-markers` providers) for `doc`.
pub async fn plugin_line_markers(
host: &Host,
window: &str,
doc: DocId,
) -> IdeResult<Vec<crate::plugins::LineMarkerInfo>> {
let Some((plugins, d)) = plugin_providers(host, window, doc, ProviderKind::LineMarkers) else {
return Ok(Vec::new());
};
let markers = plugin_call(move || plugins.provide_line_markers(&d)).await.unwrap_or_default();
Ok(markers
.into_iter()
.map(|(plugin_id, m)| crate::plugins::LineMarkerInfo {
plugin_id,
line: m.line,
icon: m.icon,
tooltip: m.tooltip,
target: m.target.map(|l| Location { path: l.path.into(), range: l.range, uri: None }),
})
.collect())
}
/// Implementations of the symbols at `positions`, one list per position (the
@@ -165,10 +307,47 @@ pub async fn lsp_code_actions(
let project = host.project(window)?;
let documents = project.documents.clone();
let version = blocking(move || documents.version(doc)).await?;
let actions = project.lsp.code_actions(doc, range, only).await?;
let result = project.lsp.code_actions(doc, range, only.clone()).await;
let plugin_actions = match plugin_providers(host, window, doc, ProviderKind::CodeActions) {
Some((plugins, d)) => plugin_call(move || plugins.provide_code_actions(&d, range)).await.unwrap_or_default(),
None => Vec::new(),
};
let plugin_actions: Vec<CodeAction> = plugin_actions
.into_iter()
.map(|(_, a)| plugin_code_action(a))
.filter(|a| {
only.is_empty()
|| a.kind.as_ref().is_some_and(|k| only.iter().any(|o| k == o || k.starts_with(&format!("{o}."))))
})
.collect();
let actions = if plugin_actions.is_empty() {
result?
} else {
let mut out = result.unwrap_or_default();
out.extend(plugin_actions);
out
};
prepare_actions(project, doc, version, actions).await
}
fn plugin_code_action(a: ide_plugin::CodeAction) -> CodeAction {
let files = a
.edits
.into_iter()
.map(|(path, edits)| ide_core::FileEdit::Edit { path: path.into(), edits, version: None })
.collect();
CodeAction {
title: a.title,
kind: a.kind,
is_preferred: a.preferred,
disabled_reason: None,
edit: Some(WorkspaceEdit { files, preconditions: Vec::new() }),
command: None,
data: None,
server: PLUGIN_SERVER,
}
}
/// Cancellable code actions with the kinds the servers advertise (Refactor This, refactoring availability).
pub async fn lsp_code_action_query(
host: &Host,
@@ -233,6 +412,56 @@ pub async fn lsp_prepare_rename(host: &Host, window: &str, doc: DocId, pos: Pos)
}
/// Computes the rename edit without applying it, so the UI can preview conflicts.
/// The class at `pos` and the members a class-hierarchy refactoring can move.
pub async fn lsp_hierarchy_members(
host: &Host,
window: &str,
request: RequestId,
doc: DocId,
pos: Pos,
kind: HierarchyRefactoring,
) -> IdeResult<HierarchyMembers> {
let project = host.project(window)?;
let cancel = host.begin_request(request);
let result = project.lsp.hierarchy_members(doc, pos, kind, cancel).await;
host.end_request(request);
result
}
/// Checks a class-hierarchy refactoring and computes its edit without applying it.
#[allow(clippy::too_many_arguments)]
pub async fn lsp_hierarchy_refactor(
host: &Host,
window: &str,
request: RequestId,
doc: DocId,
pos: Pos,
kind: HierarchyRefactoring,
target: Option<String>,
choices: Vec<HierarchyChoice>,
) -> IdeResult<HierarchyRefactored> {
let project = host.project(window)?;
let documents = project.documents.clone();
let version = blocking(move || documents.version(doc)).await?;
let cancel = host.begin_request(request);
let result = project.lsp.hierarchy_refactor(doc, pos, kind, target, choices, cancel).await;
host.end_request(request);
let mut refactored = result?;
blocking(move || {
if project.documents.version(doc)? != version {
return Err(ide_core::IdeError::new(
ide_core::error::ErrorKind::OutOfSync,
"document changed while computing the refactoring; request it again",
));
}
if let Some(edit) = &mut refactored.edit {
project.documents.prepare_workspace_edit(edit)?;
}
Ok(refactored)
})
.await
}
pub async fn lsp_rename(
host: &Host,
window: &str,
@@ -497,6 +726,22 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("lsp_rename", |c| async move {
lsp_rename(&c.host, &c.window, c.arg("request")?, c.arg("doc")?, c.arg("pos")?, c.arg("newName")?).await
});
d.add("lsp_hierarchy_members", |c| async move {
lsp_hierarchy_members(&c.host, &c.window, c.arg("request")?, c.arg("doc")?, c.arg("pos")?, c.arg("kind")?).await
});
d.add("lsp_hierarchy_refactor", |c| async move {
lsp_hierarchy_refactor(
&c.host,
&c.window,
c.arg("request")?,
c.arg("doc")?,
c.arg("pos")?,
c.arg("kind")?,
c.arg("target")?,
c.arg("choices")?,
)
.await
});
d.add("lsp_format", |c| async move {
lsp_format(&c.host, &c.window, c.arg("doc")?, c.arg("range")?, c.arg("options")?).await
});
@@ -551,3 +796,28 @@ pub(crate) fn register(d: &mut Dispatcher) {
lsp_execute_command(&c.host, &c.window, c.arg("doc")?, c.arg("command")?).await
});
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn every_completion_kind_the_plugin_api_documents_is_understood() {
let wit = include_str!("../../../../wit/providers.wit");
let doc: String = wit
.lines()
.map(|l| l.trim().trim_start_matches("///").trim())
.skip_while(|l| !l.starts_with("Icon and sort group:"))
.take_while(|l| !l.starts_with("kind:"))
.collect::<Vec<_>>()
.join(" ");
let list = doc.strip_prefix("Icon and sort group:").unwrap().split(". Unknown").next().unwrap();
let list = list.replace(" or ", ", ");
let kinds: Vec<&str> = list.split(',').map(str::trim).filter(|k| !k.is_empty()).collect();
assert_eq!(kinds.len(), 25, "{kinds:?}");
for k in kinds {
assert!(plugin_completion_kind(k).is_some(), "{k:?}");
}
assert_eq!(plugin_completion_kind("nonsense"), None);
}
}
+2
View File
@@ -12,6 +12,7 @@ pub mod debug;
pub mod dependencies;
pub mod deploy;
pub mod doc;
pub mod docker;
pub mod env;
pub mod fs;
pub mod history;
@@ -87,6 +88,7 @@ pub fn register_all(d: &mut crate::rpc::Dispatcher) {
run::register(d);
env::register(d);
deploy::register(d);
docker::register(d);
db::register(d);
}
+61 -14
View File
@@ -41,7 +41,7 @@ pub fn configs(host: &Host, window: &str) -> IdeResult<Vec<RunConfig>> {
}
/// A project's detected configurations (minus removed ones and unlinked Maven projects), then
/// its saved ones; toolchains from Languages & Frameworks applied. Gradle choices stay as saved:
/// its saved ones; toolchains from Languages & Frameworks applied. Build-tool choices stay as saved:
/// derived defaults must not be written back as explicit environment.
fn detected_and_saved(
root: &std::path::Path,
@@ -56,7 +56,7 @@ fn detected_and_saved(
.filter(|d| !suppressed.contains(&d.id) && !saved.iter().any(|u| u.id == d.id))
.collect();
all.extend(saved.iter().cloned());
for config in all.iter_mut().filter(|c| c.kind != "gradle") {
for config in all.iter_mut().filter(|c| !matches!(c.kind.as_str(), "gradle" | "maven")) {
ide_sdk::apply::apply_to_run_configs(std::slice::from_mut(config), root, settings);
}
all
@@ -75,13 +75,65 @@ pub fn save_configs(
let configs: Vec<RunConfig> = configs
.iter()
.map(|c| match c.kind.as_str() {
"gradle" => c.clone(),
"gradle" | "maven" => c.clone(),
_ => ide_sdk::apply::without_derived(c, &project.root, &settings),
})
.collect();
ide_run::save_user_configs_with_removals(&project.root, &configs, removed_detected_ids)
}
/// The project `config` runs in (a workspace member, by its working directory) and its settings.
fn settings_for(
host: &Host,
project: &crate::project::Project,
config: &RunConfig,
) -> IdeResult<(std::path::PathBuf, ide_core::settings::Settings)> {
let member = ide_workspace::workspace::Workspace::load(&project.root)?.and_then(|workspace| {
let cwd = config.cwd.as_deref()?;
workspace.projects.into_iter().filter(|root| cwd.starts_with(root)).max_by_key(|root| root.components().count())
});
Ok(match member {
Some(root) => {
let settings = ide_workspace::SettingsStore::load(&host.config_dir, Some(&root)).effective();
(root, settings)
}
None => (project.root.clone(), project.settings.effective()),
})
}
/// Temporary configurations as they are kept between sessions: without what settings derived
/// (toolchain commands, Maven options, `JAVA_HOME`…), so they follow later settings changes.
pub fn without_derived(host: &Host, window: &str, configs: Vec<RunConfig>) -> IdeResult<Vec<RunConfig>> {
host.require_trust(window)?;
let project = host.project(window)?;
configs
.into_iter()
.map(|c| {
if c.kind == "gradle" {
return Ok(c);
}
let (root, settings) = settings_for(host, &project, &c)?;
Ok(ide_sdk::apply::without_derived(&c, &root, &settings))
})
.collect()
}
/// Kept temporary configurations with today's settings applied, as [`configs`] lists the others.
pub fn apply_settings(host: &Host, window: &str, configs: Vec<RunConfig>) -> IdeResult<Vec<RunConfig>> {
host.require_trust(window)?;
let project = host.project(window)?;
configs
.into_iter()
.map(|mut c| {
if c.kind != "gradle" {
let (root, settings) = settings_for(host, &project, &c)?;
ide_sdk::apply::apply_to_run_configs(std::slice::from_mut(&mut c), &root, &settings);
}
Ok(c)
})
.collect()
}
pub async fn start(host: &Host, window: &str, config: RunConfig, on_output: Emit<OutputChunk>) -> IdeResult<RunInfo> {
host.require_trust(window)?;
let project = host.project(window)?;
@@ -100,18 +152,13 @@ pub async fn start(host: &Host, window: &str, config: RunConfig, on_output: Emit
.run(&settings, &project.root, &config, project.sink.clone(), Box::new(move |c| out(c)))
.await;
}
let member = ide_workspace::workspace::Workspace::load(&project.root)?.and_then(|workspace| {
let cwd = config.cwd.as_deref()?;
workspace.projects.into_iter().filter(|root| cwd.starts_with(root)).max_by_key(|root| root.components().count())
});
let root = member.as_ref().unwrap_or(&project.root);
let settings = match &member {
Some(root) => ide_workspace::SettingsStore::load(&host.config_dir, Some(root)).effective(),
None => project.settings.effective(),
};
let (root, settings) = settings_for(host, &project, &config)?;
let root = &root;
// Configurations made in the UI name bare tools (`godot`) that may live outside PATH.
let mut config = config;
if matches!(config.kind.as_str(), "gradle" | "tomcat") {
if config.kind == "maven" {
ide_sdk::apply::prepare_maven_run(&mut config, root, &settings);
} else if matches!(config.kind.as_str(), "gradle" | "tomcat") {
ide_sdk::apply::apply_to_run_configs(std::slice::from_mut(&mut config), root, &settings);
} else {
ide_sdk::apply::resolve_run_command(&mut config, root, &settings);
@@ -147,7 +194,7 @@ pub async fn start(host: &Host, window: &str, config: RunConfig, on_output: Emit
let encoding = settings.console.encoding;
let options = StartOptions {
encoding: Some(encoding).filter(|e| !e.trim().is_empty()),
python_paths: if member.is_some() { vec![root.clone()] } else { super::python::paths(&project) },
python_paths: if *root != project.root { vec![root.clone()] } else { super::python::paths(&project) },
};
project.require_trust()?;
let mut sink = project.sink.clone();
+40
View File
@@ -241,6 +241,35 @@ pub async fn run_tomcat_update(host: &Host, window: &str, config: RunConfig, mod
.map_err(|e| IdeError::internal(e.to_string()))?
}
/// Updates a running Tomcat artifact deployment from the module's compiled output and web
/// resources (`ide_run::tomcat::sync_outputs`). `false`: rebuild the WAR instead.
pub async fn run_tomcat_sync(
host: &Host,
window: &str,
config: RunConfig,
module: std::path::PathBuf,
class_paths: Vec<String>,
resources_only: bool,
) -> IdeResult<bool> {
host.require_trust(window)?;
let project = host.project(window)?;
if !host.processes.list().iter().any(|r| r.running && r.config_id == config.id) {
return Err(IdeError::invalid("Start this Tomcat configuration before updating it"));
}
if config.extras.params.get("deployment").is_some_and(|v| v != "artifact") {
return Ok(false);
}
tokio::task::spawn_blocking(move || {
let module = dunce::canonicalize(project.root.join(module))?;
let class_paths: Vec<_> =
class_paths.iter().filter_map(|p| dunce::canonicalize(project.root.join(p)).ok()).collect();
let base = ide_run::tomcat::base_dir(&project.cache_dir, &config.id);
ide_run::tomcat::sync_outputs(&base, &module, &class_paths, resources_only)
})
.await
.map_err(|e| IdeError::internal(e.to_string()))?
}
/// Commands of this module the remote server answers.
pub(crate) fn register(d: &mut Dispatcher) {
d.add("run_dotnet_resolve", |c| async move {
@@ -255,6 +284,17 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("run_tomcat_home", |c| async move { run_tomcat_home(&c.host, &c.window, c.arg("home")?).await });
d.add("run_tomcat_artifacts", |c| async move { run_tomcat_artifacts(&c.host, &c.window).await });
d.add("run_tomcat_build", |c| async move { run_tomcat_build(&c.host, &c.window, c.arg("config")?).await });
d.add("run_tomcat_sync", |c| async move {
run_tomcat_sync(
&c.host,
&c.window,
c.arg("config")?,
c.arg("module")?,
c.arg("classPaths")?,
c.arg("resourcesOnly")?,
)
.await
});
d.add("run_tomcat_update", |c| async move {
run_tomcat_update(&c.host, &c.window, c.arg("config")?, c.arg("mode")?).await
});
+803
View File
@@ -0,0 +1,803 @@
//! [`HostApi`] for one plugin's code: what its imports reach in the IDE.
//! Capabilities were checked by the runtime before any of this runs.
use super::{PluginEvent, PluginEventSink};
use ide_core::{CancellationToken, TextEdit};
use ide_extension::Package;
use ide_language::ProgramPin;
use ide_plugin::{
EventKind, HostApi, HostError, HttpRequest, HttpResponse, NotificationLevel, ProcessOutput, RepositoryStatus,
StatusItem,
};
use parking_lot::{Mutex, RwLock};
use std::collections::{BTreeMap, BTreeSet, HashMap};
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};
use std::time::{Duration, Instant};
/// Largest storage of one plugin (keys + values).
const MAX_STORAGE: usize = 1 << 20;
const MAX_KEY: usize = 256;
/// Most status bar items per plugin.
const MAX_STATUS_ITEMS: usize = 8;
/// What plugins see of the open projects; provided by the app
/// ([`super::PluginHost::set_workspace`]), over [`crate::Host`] on the desktop.
pub trait PluginWorkspace: Send + Sync {
/// Roots of the open projects.
fn projects(&self) -> Vec<PathBuf>;
/// Unsaved text of the open document at `path`.
fn buffer_text(&self, path: &Path) -> Option<String>;
/// Version and text of the open document at `path`.
fn document(&self, path: &Path) -> Option<(u64, String)>;
/// Applies `edits` to the open document at `path` if it is still at
/// `version`, as one change (one undo step in the editor).
fn apply_edits(&self, path: &Path, version: u64, edits: &[TextEdit]) -> Result<u64, HostError>;
/// Code from the project may run (git hooks and config on fetch).
fn trusted(&self, root: &Path) -> bool;
/// Root of the project open in `window`.
fn project_of(&self, window: &str) -> Option<PathBuf>;
/// Shows `items` as producer `source`'s diagnostics of `path` (replacing earlier ones).
fn publish_diagnostics(&self, source: &str, path: &Path, items: Vec<ide_core::Diagnostic>)
-> Result<(), HostError>;
/// The user's values of `plugin`'s settings: the project's at `root`
/// (over global), or global.
fn plugin_settings(&self, _plugin: &str, _root: Option<&Path>) -> serde_json::Map<String, serde_json::Value> {
serde_json::Map::new()
}
}
/// Where plugin secrets live (the OS keychain on the desktop). Plugins can
/// only ask whether one is set; values go out only inside `net` requests.
pub trait SecretStore: Send + Sync {
fn get(&self, plugin: &str, name: &str) -> Option<String>;
/// Sets, or (`None`) deletes.
fn set(&self, plugin: &str, name: &str, value: Option<&str>) -> Result<(), String>;
}
/// Fallback [`SecretStore`]: one file per plugin, readable only by the user.
pub struct FileSecretStore {
pub dir: PathBuf,
}
impl FileSecretStore {
fn file(&self, plugin: &str) -> PathBuf {
self.dir.join(plugin).join("secrets.json")
}
fn load(&self, plugin: &str) -> BTreeMap<String, String> {
std::fs::read(self.file(plugin)).ok().and_then(|b| serde_json::from_slice(&b).ok()).unwrap_or_default()
}
}
impl SecretStore for FileSecretStore {
fn get(&self, plugin: &str, name: &str) -> Option<String> {
self.load(plugin).remove(name)
}
fn set(&self, plugin: &str, name: &str, value: Option<&str>) -> Result<(), String> {
let mut map = self.load(plugin);
match value {
Some(v) => map.insert(name.to_owned(), v.to_owned()),
None => map.remove(name),
};
let file = self.file(plugin);
std::fs::create_dir_all(file.parent().expect("has parent")).map_err(|e| e.to_string())?;
let bytes = serde_json::to_vec(&map).map_err(|e| e.to_string())?;
let tmp = file.with_extension("tmp");
std::fs::write(&tmp, bytes).map_err(|e| e.to_string())?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
let _ = std::fs::set_permissions(&tmp, std::fs::Permissions::from_mode(0o600));
}
std::fs::rename(&tmp, &file).map_err(|e| e.to_string())
}
}
/// Largest response body a plugin receives.
const MAX_RESPONSE: usize = 8 << 20;
/// Largest stdout / stderr kept from a process.
const MAX_OUTPUT: usize = 4 << 20;
/// Most files `find-files` walks before giving up.
const MAX_WALK: usize = 200_000;
/// Runtime for plugin HTTP requests (actor threads block on it).
pub(super) fn net_runtime() -> &'static tokio::runtime::Runtime {
static RT: std::sync::OnceLock<tokio::runtime::Runtime> = std::sync::OnceLock::new();
RT.get_or_init(|| {
tokio::runtime::Builder::new_multi_thread()
.worker_threads(2)
.thread_name("plugin-net")
.enable_all()
.build()
.expect("plugin network runtime")
})
}
/// Replaces `{{secret:<name>}}` in `value` with the plugin's secrets.
fn fill_secrets(value: &str, lookup: impl Fn(&str) -> Option<String>) -> Result<String, HostError> {
let mut out = String::with_capacity(value.len());
let mut rest = value;
while let Some(i) = rest.find("{{secret:") {
out.push_str(&rest[..i]);
let after = &rest[i + "{{secret:".len()..];
let end = after.find("}}").ok_or_else(|| HostError::Invalid("unterminated {{secret:…}}".into()))?;
let name = &after[..end];
let secret = lookup(name).ok_or_else(|| HostError::NotFound(format!("secret {name:?} is not set")))?;
out.push_str(&secret);
rest = &after[end + 2..];
}
out.push_str(rest);
Ok(out)
}
/// A plugin task shown in the UI.
struct TaskEntry {
plugin: String,
cancel: CancellationToken,
cancelled: bool,
}
/// State shared by the host and every plugin's [`PluginApi`].
pub(super) struct Shared {
pub sink: Arc<dyn PluginEventSink>,
pub workspace: RwLock<Option<Arc<dyn PluginWorkspace>>>,
/// `<plugins>/.data`: one directory per plugin id.
pub data_dir: PathBuf,
tasks: Mutex<HashMap<u64, TaskEntry>>,
next_task: AtomicU64,
/// (plugin, item id) → item, for windows opened later.
status: Mutex<BTreeMap<(String, String), StatusItem>>,
/// plugin → kinds subscribed at run time (kept across idle reclaim).
pub subscriptions: Mutex<HashMap<String, BTreeSet<EventKind>>>,
/// plugin → files it shows problems for, withdrawn when it stops.
diagnostics: Mutex<HashMap<String, BTreeSet<PathBuf>>>,
pub secrets: RwLock<Arc<dyn SecretStore>>,
/// (plugin, `process:` capability) → the approved program.
pub pins: Arc<RwLock<HashMap<(String, String), ProgramPin>>>,
/// Validated UI declarations (settings schemas, declared views).
pub ui: Arc<ide_extension::UiStore>,
}
impl Shared {
pub fn new(
sink: Arc<dyn PluginEventSink>,
data_dir: PathBuf,
pins: Arc<RwLock<HashMap<(String, String), ProgramPin>>>,
ui: Arc<ide_extension::UiStore>,
) -> Arc<Self> {
Arc::new(Self {
ui,
sink,
workspace: RwLock::new(None),
secrets: RwLock::new(Arc::new(FileSecretStore { dir: data_dir.clone() })),
pins,
data_dir,
tasks: Mutex::default(),
next_task: AtomicU64::new(1),
status: Mutex::default(),
subscriptions: Mutex::default(),
diagnostics: Mutex::default(),
})
}
fn workspace(&self) -> Option<Arc<dyn PluginWorkspace>> {
self.workspace.read().clone()
}
pub fn project_of(&self, window: &str) -> Option<PathBuf> {
self.workspace()?.project_of(window)
}
/// Shows `plugin`'s problems of `path`, remembering the file until they are
/// cleared (by the plugin, or when it stops).
fn publish_diagnostics(
&self,
plugin: &str,
path: &Path,
items: Vec<ide_core::Diagnostic>,
) -> Result<(), HostError> {
let workspace = self.workspace().ok_or_else(|| HostError::NotFound("no project is open".into()))?;
let empty = items.is_empty();
workspace.publish_diagnostics(plugin, path, items)?;
let mut published = self.diagnostics.lock();
if empty {
if let Some(paths) = published.get_mut(plugin) {
paths.remove(path);
if paths.is_empty() {
published.remove(plugin);
}
}
} else {
published.entry(plugin.to_owned()).or_default().insert(path.to_path_buf());
}
Ok(())
}
pub fn status_items(&self) -> Vec<(String, String, StatusItem)> {
self.status.lock().iter().map(|((p, id), item)| (p.clone(), id.clone(), item.clone())).collect()
}
/// Withdraws a plugin's run-time state (it stopped or failed). Its
/// subscriptions go only with `subscriptions`, as they survive restarts.
pub fn clear_plugin(&self, plugin: &str, subscriptions: bool) {
let removed: Vec<String> = {
let mut status = self.status.lock();
let ids: Vec<String> = status.keys().filter(|(p, _)| p == plugin).map(|(_, id)| id.clone()).collect();
for id in &ids {
status.remove(&(plugin.to_owned(), id.clone()));
}
ids
};
for id in removed {
self.sink.emit(None, PluginEvent::StatusItemChanged { plugin_id: plugin.to_owned(), id, item: None });
}
let tasks: Vec<u64> = self.tasks.lock().iter().filter(|(_, t)| t.plugin == plugin).map(|(id, _)| *id).collect();
for id in tasks {
self.end_task(id);
}
// Its problems go with it (an idle reclaim keeps them: it isn't cleared then).
let paths = self.diagnostics.lock().remove(plugin).unwrap_or_default();
if let Some(workspace) = self.workspace() {
for path in paths {
let _ = workspace.publish_diagnostics(plugin, &path, Vec::new());
}
}
if subscriptions && self.subscriptions.lock().remove(plugin).is_some() {
self.sink.emit(None, PluginEvent::SubscriptionsChanged);
}
}
/// The user cancelled a task in the UI.
pub fn cancel_task(&self, id: u64) -> bool {
let mut tasks = self.tasks.lock();
let Some(t) = tasks.get_mut(&id) else { return false };
t.cancelled = true;
t.cancel.cancel();
true
}
fn end_task(&self, id: u64) {
if let Some(t) = self.tasks.lock().remove(&id) {
self.sink.emit(None, PluginEvent::TaskEnded { plugin_id: t.plugin, task_id: id.to_string() });
}
}
pub fn storage_file(&self, plugin: &str) -> PathBuf {
self.data_dir.join(plugin).join("storage.json")
}
}
pub(super) struct PluginApi {
pub pkg: Package,
pub shared: Arc<Shared>,
}
impl PluginApi {
fn id(&self) -> &str {
self.pkg.id()
}
fn load_storage(&self) -> Result<BTreeMap<String, String>, HostError> {
match std::fs::read(self.shared.storage_file(self.id())) {
Ok(bytes) => serde_json::from_slice(&bytes).map_err(|e| HostError::Failed(format!("storage: {e}"))),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(BTreeMap::new()),
Err(e) => Err(HostError::Failed(format!("storage: {e}"))),
}
}
fn save_storage(&self, map: &BTreeMap<String, String>) -> Result<(), HostError> {
let file = self.shared.storage_file(self.id());
let bytes = serde_json::to_vec(map).map_err(|e| HostError::Failed(e.to_string()))?;
let io = |e: std::io::Error| HostError::Failed(format!("storage: {e}"));
std::fs::create_dir_all(file.parent().expect("has parent")).map_err(io)?;
let tmp = file.with_extension("tmp");
std::fs::write(&tmp, bytes).and_then(|()| std::fs::rename(&tmp, &file)).map_err(io)
}
/// The program approved for the (already checked) `process:` grant,
/// verified right before it starts.
fn approved_program(&self, program: &str) -> Result<PathBuf, HostError> {
let cap = format!("process:{program}");
// Granted without a pinned file: the program wasn't found when it was
// allowed, and installing it later approves nothing by itself.
let pin = self.shared.pins.read().get(&(self.id().to_owned(), cap.clone())).cloned().ok_or_else(|| {
HostError::Failed(format!(
"{cap}: `{program}` was not found when this permission was allowed; \
install it, then allow {cap} again in Settings › Plugins"
))
})?;
pin.verify().map(Path::to_path_buf).map_err(|e| HostError::Denied(format!("{cap}: {e}")))
}
}
fn failed(e: impl std::fmt::Display) -> HostError {
HostError::Failed(e.to_string())
}
impl HostApi for PluginApi {
fn notify(&self, client: Option<&str>, level: NotificationLevel, title: String, body: String) {
self.shared
.sink
.emit(client, PluginEvent::Notification { plugin_id: self.id().to_owned(), level, title, body });
}
fn status_item(&self, id: &str, item: Option<StatusItem>) {
let key = (self.id().to_owned(), id.to_owned());
{
let mut status = self.shared.status.lock();
match &item {
Some(item) => {
let count = status.keys().filter(|(p, _)| p == self.id()).count();
if !status.contains_key(&key) && count >= MAX_STATUS_ITEMS {
tracing::warn!(plugin = %self.id(), "more than {MAX_STATUS_ITEMS} status items; {id:?} ignored");
return;
}
if status.get(&key) == Some(item) {
return;
}
status.insert(key, item.clone());
}
None => {
if status.remove(&key).is_none() {
return;
}
}
}
}
self.shared
.sink
.emit(None, PluginEvent::StatusItemChanged { plugin_id: self.id().to_owned(), id: id.to_owned(), item });
}
fn projects(&self) -> Vec<PathBuf> {
self.shared.workspace().map(|w| w.projects()).unwrap_or_default()
}
fn buffer_text(&self, path: &Path) -> Option<String> {
self.shared.workspace()?.buffer_text(path)
}
fn document(&self, path: &Path) -> Option<(u64, String)> {
self.shared.workspace()?.document(path)
}
fn apply_edits(
&self,
_client: Option<&str>,
path: &Path,
version: u64,
edits: Vec<TextEdit>,
) -> Result<u64, HostError> {
let workspace = self.shared.workspace().ok_or_else(|| HostError::NotFound("no project is open".into()))?;
workspace.apply_edits(path, version, &edits)
}
fn storage_get(&self, key: &str) -> Result<Option<String>, HostError> {
Ok(self.load_storage()?.get(key).cloned())
}
fn storage_set(&self, key: &str, value: Option<String>) -> Result<(), HostError> {
if key.is_empty() || key.len() > MAX_KEY {
return Err(HostError::Invalid(format!("keys are 1–{MAX_KEY} bytes")));
}
let mut map = self.load_storage()?;
let changed = match value {
Some(v) => map.insert(key.to_owned(), v.clone()).as_ref() != Some(&v),
None => map.remove(key).is_some(),
};
if !changed {
return Ok(());
}
let size: usize = map.iter().map(|(k, v)| k.len() + v.len()).sum();
if size > MAX_STORAGE {
return Err(HostError::Invalid(format!("storage is limited to {} KiB", MAX_STORAGE >> 10)));
}
self.save_storage(&map)
}
fn storage_keys(&self, prefix: &str) -> Result<Vec<String>, HostError> {
Ok(self.load_storage()?.into_keys().filter(|k| k.starts_with(prefix)).collect())
}
fn vcs_status(&self, root: &Path) -> Result<Option<RepositoryStatus>, HostError> {
let Some(repo) = ide_vcs::Repo::discover(root).map_err(failed)? else { return Ok(None) };
let info = repo.branch_info().map_err(failed)?;
let changed = repo.status().map_err(failed)?.len();
Ok(Some(RepositoryStatus {
root: repo.root().to_string_lossy().into_owned(),
branch: info.head,
commit: info.head_commit,
upstream: info.upstream,
ahead: info.ahead,
behind: info.behind,
operation: info.operation,
changed_files: u32::try_from(changed).unwrap_or(u32::MAX),
}))
}
fn vcs_fetch(&self, root: &Path, cancel: &CancellationToken) -> Result<(), HostError> {
let workspace = self.shared.workspace();
if !workspace.is_some_and(|w| w.trusted(root)) {
return Err(HostError::Failed("the project is in Restricted Mode: git does not run there".into()));
}
let Some(repo) = ide_vcs::Repo::discover(root).map_err(failed)? else {
return Err(HostError::NotFound("the project is not in a git repository".into()));
};
match repo.fetch(cancel, &|_| {}) {
Ok(()) => Ok(()),
Err(_) if cancel.is_cancelled() => Err(HostError::Cancelled),
Err(e) => Err(failed(e.message)),
}
}
fn find_files(&self, root: &Path, glob: &str, limit: usize) -> Result<Vec<String>, HostError> {
let matcher = globset::GlobBuilder::new(glob)
.literal_separator(true)
.build()
.map_err(|e| HostError::Invalid(format!("{glob:?}: {e}")))?
.compile_matcher();
let mut out = Vec::new();
for (n, entry) in ignore::WalkBuilder::new(root).hidden(true).build().flatten().enumerate() {
if n >= MAX_WALK || out.len() >= limit {
break;
}
if !entry.file_type().is_some_and(|t| t.is_file()) {
continue;
}
let Ok(rel) = entry.path().strip_prefix(root) else { continue };
if matcher.is_match(rel) {
out.push(entry.path().to_string_lossy().into_owned());
}
}
out.sort();
Ok(out)
}
fn publish_diagnostics(&self, path: &Path, items: Vec<ide_core::Diagnostic>) -> Result<(), HostError> {
self.shared.publish_diagnostics(self.id(), path, items)
}
fn fetch(&self, request: HttpRequest, cancel: &CancellationToken) -> Result<HttpResponse, HostError> {
let secrets = self.shared.secrets.read().clone();
let id = self.id().to_owned();
let mut headers = reqwest::header::HeaderMap::new();
for (name, value) in &request.headers {
let value = fill_secrets(value, |n| secrets.get(&id, n))?;
let name = reqwest::header::HeaderName::from_bytes(name.as_bytes())
.map_err(|e| HostError::Invalid(format!("header {name:?}: {e}")))?;
let mut value = reqwest::header::HeaderValue::from_str(&value)
.map_err(|e| HostError::Invalid(format!("header {name}: {e}")))?;
value.set_sensitive(true);
headers.append(name, value);
}
let method = reqwest::Method::from_bytes(request.method.to_ascii_uppercase().as_bytes())
.map_err(|_| HostError::Invalid(format!("method {:?}", request.method)))?;
let client = ide_http::client::shared().map_err(|e| HostError::Failed(e.message))?;
// A redirect to another host is refused after the fact (the shared
// client follows redirects).
let builder = client.request(method, &request.url).headers(headers).timeout(request.timeout);
let builder = match request.body {
Some(body) => builder.body(body),
None => builder,
};
let cancel = cancel.clone();
let host = ide_plugin::net_host(&request.url).map_err(HostError::Invalid)?;
net_runtime().block_on(async move {
let send = async {
let response = builder.send().await.map_err(|e| HostError::Failed(e.to_string()))?;
if ide_plugin::net_host(response.url().as_str()).ok().as_deref() != Some(host.as_str()) {
return Err(HostError::Denied(format!(
"net:{} (redirected)",
response.url().host_str().unwrap_or("?")
)));
}
let status = response.status().as_u16();
let headers = response
.headers()
.iter()
.map(|(k, v)| (k.as_str().to_owned(), String::from_utf8_lossy(v.as_bytes()).into_owned()))
.collect();
let mut body = Vec::new();
let mut response = response;
while let Some(chunk) = response.chunk().await.map_err(|e| HostError::Failed(e.to_string()))? {
if body.len() + chunk.len() > MAX_RESPONSE {
return Err(HostError::Failed("response larger than 8 MiB".into()));
}
body.extend_from_slice(&chunk);
}
Ok(HttpResponse { status, headers, body })
};
tokio::select! {
r = send => r,
() = cancel.cancelled() => Err(HostError::Cancelled),
}
})
}
fn secret_is_set(&self, name: &str) -> bool {
self.shared.secrets.read().get(self.id(), name).is_some()
}
fn run_process(
&self,
program: &str,
args: Vec<String>,
cwd: Option<&Path>,
stdin: Option<String>,
cancel: &CancellationToken,
timeout: Duration,
) -> Result<ProcessOutput, HostError> {
let binary = self.approved_program(program)?;
let cwd = cwd.map(Path::to_path_buf).unwrap_or_else(|| self.shared.data_dir.join(self.id()));
let _ = std::fs::create_dir_all(&cwd);
let mut child = std::process::Command::new(&binary)
.args(&args)
.current_dir(&cwd)
.stdin(if stdin.is_some() { std::process::Stdio::piped() } else { std::process::Stdio::null() })
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
.map_err(|e| HostError::Failed(format!("{}: {e}", binary.display())))?;
if let (Some(input), Some(mut pipe)) = (stdin, child.stdin.take()) {
std::thread::spawn(move || {
use std::io::Write;
let _ = pipe.write_all(input.as_bytes());
});
}
let read = |pipe: Option<Box<dyn std::io::Read + Send>>| {
std::thread::spawn(move || {
use std::io::Read;
let mut out = Vec::new();
if let Some(p) = pipe {
let _ = p.take(MAX_OUTPUT as u64).read_to_end(&mut out);
}
String::from_utf8_lossy(&out).into_owned()
})
};
let stdout = read(child.stdout.take().map(|p| Box::new(p) as Box<dyn std::io::Read + Send>));
let stderr = read(child.stderr.take().map(|p| Box::new(p) as Box<dyn std::io::Read + Send>));
let deadline = Instant::now() + timeout;
let status = loop {
if let Some(status) = child.try_wait().map_err(|e| HostError::Failed(e.to_string()))? {
break status;
}
if cancel.is_cancelled() || Instant::now() >= deadline {
let _ = child.kill();
let _ = child.wait();
return Err(if cancel.is_cancelled() {
HostError::Cancelled
} else {
HostError::Failed(format!("{program} timed out after {timeout:?}"))
});
}
std::thread::sleep(Duration::from_millis(10));
};
Ok(ProcessOutput {
status: status.code(),
stdout: stdout.join().unwrap_or_default(),
stderr: stderr.join().unwrap_or_default(),
})
}
fn subscribe(&self, kind: EventKind, on: bool) {
let changed = {
let mut subs = self.shared.subscriptions.lock();
let set = subs.entry(self.id().to_owned()).or_default();
if on { set.insert(kind) } else { set.remove(&kind) }
};
if changed {
self.shared.sink.emit(None, PluginEvent::SubscriptionsChanged);
}
}
fn task_begin(&self, client: Option<&str>, title: String, cancellable: bool, cancel: CancellationToken) -> u64 {
let id = self.shared.next_task.fetch_add(1, Ordering::Relaxed);
self.shared.tasks.lock().insert(id, TaskEntry { plugin: self.id().to_owned(), cancel, cancelled: false });
self.shared.sink.emit(
client,
PluginEvent::TaskStarted { plugin_id: self.id().to_owned(), task_id: id.to_string(), title, cancellable },
);
id
}
fn task_report(&self, id: u64, done: u32, total: Option<u32>, message: Option<String>) {
if !self.shared.tasks.lock().contains_key(&id) {
return;
}
self.shared.sink.emit(
None,
PluginEvent::TaskProgress {
plugin_id: self.id().to_owned(),
task_id: id.to_string(),
done,
total,
message,
},
);
}
fn task_cancelled(&self, id: u64) -> bool {
self.shared.tasks.lock().get(&id).is_some_and(|t| t.cancelled)
}
fn task_end(&self, id: u64) {
self.shared.end_task(id);
}
fn refresh_view(&self, view: &str) {
self.shared
.sink
.emit(None, PluginEvent::ViewChanged { plugin_id: self.id().to_owned(), view: view.to_owned() });
}
fn post_view_message(&self, view: &str, message: String) -> Result<(), HostError> {
let declared = [super::ui_points::VIEWS, super::ui_points::CUSTOM_EDITORS]
.iter()
.any(|point| self.shared.ui.items(point, Some(self.id())).iter().any(|i| i.key == view));
if !declared {
return Err(HostError::NotFound(format!("the plugin declares no view {view:?}")));
}
let event = PluginEvent::ViewMessage { plugin_id: self.id().to_owned(), view: view.to_owned(), message };
self.shared.sink.emit(None, event);
Ok(())
}
fn open_terminal(
&self,
client: Option<&str>,
title: String,
program: &str,
args: Vec<String>,
cwd: Option<&Path>,
) -> Result<(), HostError> {
let binary = self.approved_program(program)?;
let cwd = cwd.map(Path::to_path_buf).unwrap_or_else(|| self.shared.data_dir.join(self.id()));
let _ = std::fs::create_dir_all(&cwd);
let event = PluginEvent::OpenTerminal {
plugin_id: self.id().to_owned(),
title,
program: binary.into(),
args,
cwd: cwd.into(),
};
self.shared.sink.emit(client, event);
Ok(())
}
fn setting(&self, root: Option<&Path>, key: &str) -> Option<String> {
let schema: Vec<super::ui_points::SettingSchema> = self
.shared
.ui
.items(super::ui_points::SETTINGS, Some(self.id()))
.into_iter()
.find_map(|i| serde_json::from_value(i.decl.get("properties")?.clone()).ok())?;
let setting = schema.into_iter().find(|s| s.key == key)?;
let user = self.shared.workspace().map(|w| w.plugin_settings(self.id(), root)).unwrap_or_default();
let value = user.get(key).filter(|v| setting.accepts(v)).cloned().unwrap_or(setting.default);
serde_json::to_string(&value).ok()
}
}
/// [`PluginWorkspace`] over the desktop host's open projects.
pub struct HostWorkspace(pub std::sync::Weak<crate::Host>);
impl HostWorkspace {
fn projects(&self) -> Vec<Arc<crate::project::Project>> {
self.0.upgrade().map(|h| h.projects()).unwrap_or_default()
}
/// The project holding an open document at `path`, and its id.
fn open_document(&self, path: &Path) -> Option<(Arc<crate::project::Project>, ide_core::DocId)> {
self.projects().into_iter().find_map(|p| p.documents.by_path(path).map(|doc| (p, doc)))
}
}
impl PluginWorkspace for HostWorkspace {
fn projects(&self) -> Vec<PathBuf> {
let mut roots: Vec<PathBuf> = HostWorkspace::projects(self).iter().map(|p| p.root.clone()).collect();
roots.sort();
roots.dedup();
roots
}
fn buffer_text(&self, path: &Path) -> Option<String> {
HostWorkspace::projects(self).iter().find_map(|p| p.documents.text_of(path))
}
fn document(&self, path: &Path) -> Option<(u64, String)> {
let (project, doc) = self.open_document(path)?;
let snap = project.documents.snapshot(doc).ok()?;
Some((u64::from(snap.version), snap.text))
}
fn apply_edits(&self, path: &Path, version: u64, edits: &[TextEdit]) -> Result<u64, HostError> {
let (project, doc) =
self.open_document(path).ok_or_else(|| HostError::NotFound(format!("{} is not open", path.display())))?;
let version = u32::try_from(version).map_err(|_| HostError::Stale)?;
match project.documents.apply_edits_at(doc, version, edits) {
Ok(Some(v)) => Ok(u64::from(v)),
Ok(None) => Err(HostError::Stale),
Err(e) if e.kind == ide_core::error::ErrorKind::InvalidInput => Err(HostError::Invalid(e.message)),
Err(e) => Err(HostError::Failed(e.message)),
}
}
fn trusted(&self, root: &Path) -> bool {
HostWorkspace::projects(self).iter().any(|p| p.root == root && p.is_trusted())
}
fn project_of(&self, window: &str) -> Option<PathBuf> {
self.0.upgrade()?.project(window).ok().map(|p| p.root.clone())
}
fn publish_diagnostics(
&self,
source: &str,
path: &Path,
items: Vec<ide_core::Diagnostic>,
) -> Result<(), HostError> {
let projects = HostWorkspace::projects(self);
let project = projects
.iter()
.filter(|p| path.starts_with(&p.root))
.max_by_key(|p| p.root.as_os_str().len())
.ok_or_else(|| HostError::Invalid(format!("{} is not in an open project", path.display())))?;
project.sink.emit(ide_core::IdeEvent::Diagnostics {
path: path.to_path_buf().into(),
source: format!("plugin:{source}"),
items,
});
Ok(())
}
fn plugin_settings(&self, plugin: &str, root: Option<&Path>) -> serde_json::Map<String, serde_json::Value> {
let Some(host) = self.0.upgrade() else { return serde_json::Map::new() };
let settings = match root.and_then(|r| host.projects().into_iter().find(|p| p.root == r)) {
Some(p) => p.settings.effective(),
None => host.global_settings.effective(),
};
settings.plugins.get(plugin).cloned().unwrap_or_default()
}
}
#[cfg(test)]
mod tests {
use super::*;
fn api(dir: &Path, pins: Arc<RwLock<HashMap<(String, String), ProgramPin>>>) -> PluginApi {
let manifest = ide_extension::Manifest::parse(
"id = \"acme.lint\"\nname = \"Lint\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"\n",
)
.unwrap();
let sink: Arc<dyn PluginEventSink> = Arc::new(|_: Option<&str>, _: PluginEvent| {});
let shared = Shared::new(sink, dir.join(".data"), pins, ide_extension::UiStore::new());
PluginApi { pkg: Package::new(manifest, dir.to_path_buf()), shared }
}
#[test]
fn programs_run_only_as_approved() {
let tmp = tempfile::tempdir().unwrap();
let bin = tmp.path().join("lint");
std::fs::write(&bin, "#!/bin/sh\n").unwrap();
let pins: Arc<RwLock<HashMap<(String, String), ProgramPin>>> = Arc::default();
let api = api(tmp.path(), pins.clone());
// Allowed while the program was missing: say how to fix it rather than "not granted".
let err = api.approved_program("lint").unwrap_err().to_string();
assert!(
err.contains("was not found when this permission was allowed") && err.contains("allow process:lint again"),
"{err}"
);
pins.write().insert(("acme.lint".into(), "process:lint".into()), ProgramPin::approve(&bin).unwrap());
assert_eq!(api.approved_program("lint").unwrap(), bin);
std::fs::write(&bin, "#!/bin/sh\necho replaced\n").unwrap();
assert!(
matches!(api.approved_program("lint"), Err(HostError::Denied(e)) if e.contains("changed since it was approved"))
);
}
}
@@ -0,0 +1,207 @@
use super::icon_patterns::validate_pattern;
use ide_extension::validate_item_id;
use serde::{Deserialize, Serialize};
use std::collections::{BTreeMap, BTreeSet};
use std::path::PathBuf;
const MAX_GROUPS: usize = 64;
const MAX_CHOICES: usize = 16;
const MAX_CHOICE_OPTIONS: usize = 16;
const MAX_NAME_LEN: usize = 255;
pub(super) const DARK_SUFFIX: &str = "_dark";
#[derive(Deserialize)]
#[serde(untagged)]
pub(super) enum IconDefinition {
Path(PathBuf),
Variants(IconVariants),
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
pub(super) struct IconVariants {
light: PathBuf,
#[serde(default)]
dark: Option<PathBuf>,
}
impl IconDefinition {
pub(super) fn light(&self) -> &PathBuf {
match self {
IconDefinition::Path(p) => p,
IconDefinition::Variants(v) => &v.light,
}
}
pub(super) fn dark(&self) -> Option<&PathBuf> {
match self {
IconDefinition::Path(_) => None,
IconDefinition::Variants(v) => v.dark.as_ref(),
}
}
}
#[derive(Clone, Copy, Default, PartialEq, Eq, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub(super) enum Target {
#[default]
File,
Folder,
}
#[derive(Deserialize, Serialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub(super) struct Pattern {
pattern: String,
icon: String,
#[serde(default)]
target: Target,
#[serde(default)]
path: bool,
#[serde(default)]
ignore_case: bool,
#[serde(default)]
group: Option<String>,
#[serde(default)]
open_icon: Option<String>,
}
fn enabled_by_default() -> bool {
true
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub(super) struct Group {
pub(super) id: String,
name: String,
#[serde(default = "enabled_by_default")]
enabled: bool,
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub(super) struct ChoiceOption {
pub(super) id: String,
pub(super) name: String,
#[serde(default)]
overrides: BTreeMap<String, String>,
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub(super) struct Choice {
pub(super) id: String,
pub(super) name: String,
pub(super) default: String,
pub(super) options: Vec<ChoiceOption>,
}
#[derive(Default, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct Extensions {
#[serde(default)]
pub(super) patterns: Vec<Pattern>,
#[serde(default)]
pub(super) overrides: BTreeMap<String, String>,
#[serde(default)]
pub(super) groups: Vec<Group>,
#[serde(default)]
pub(super) choices: Vec<Choice>,
#[serde(default)]
pub(super) accent: bool,
}
pub(super) fn check_icon_id(id: &str) -> Result<(), String> {
validate_item_id(id, "icon id")?;
if id.ends_with(DARK_SUFFIX) {
return Err(format!("icon id {id:?}: the {DARK_SUFFIX:?} suffix is reserved for dark variants"));
}
Ok(())
}
fn check_named(id: &str, name: &str, what: &str) -> Result<(), String> {
validate_item_id(id, what)?;
if name.trim().is_empty() || name.len() > MAX_NAME_LEN {
return Err(format!("{what} {id:?}: name must have 1 to {MAX_NAME_LEN} characters"));
}
Ok(())
}
fn check_unique<'a>(ids: impl Iterator<Item = &'a String>, what: &str) -> Result<(), String> {
let mut seen = BTreeSet::new();
match ids.into_iter().find(|id| !seen.insert(*id)) {
Some(id) => Err(format!("duplicate {what} {id:?}")),
None => Ok(()),
}
}
impl Extensions {
pub(super) fn mappings(&self) -> usize {
let chosen: usize = self.choices.iter().flat_map(|c| &c.options).map(|o| o.overrides.len()).sum();
self.patterns.len() + self.overrides.len() + chosen
}
pub(super) fn validate(&self, check_ref: &dyn Fn(&str, &str) -> Result<(), String>) -> Result<(), String> {
self.validate_groups()?;
for p in &self.patterns {
validate_pattern(&p.pattern)?;
check_ref(&p.icon, "patterns")?;
if let Some(group) = &p.group
&& !self.groups.iter().any(|g| &g.id == group)
{
return Err(format!("pattern {:?}: unknown group {group:?}", p.pattern));
}
if let Some(open) = &p.open_icon {
if p.target != Target::Folder {
return Err(format!("pattern {:?}: openIcon is only for folder patterns", p.pattern));
}
check_ref(open, "patterns")?;
}
}
check_overrides(&self.overrides, check_ref, "overrides")?;
for choice in &self.choices {
for option in &choice.options {
check_overrides(&option.overrides, check_ref, "choices")?;
}
}
Ok(())
}
fn validate_groups(&self) -> Result<(), String> {
if self.groups.len() > MAX_GROUPS || self.choices.len() > MAX_CHOICES {
return Err(format!("too many groups or choices (max {MAX_GROUPS} and {MAX_CHOICES})"));
}
for g in &self.groups {
check_named(&g.id, &g.name, "group")?;
}
check_unique(self.groups.iter().map(|g| &g.id), "group")?;
check_unique(self.choices.iter().map(|c| &c.id), "choice")?;
for choice in &self.choices {
check_named(&choice.id, &choice.name, "choice")?;
if choice.options.is_empty() || choice.options.len() > MAX_CHOICE_OPTIONS {
return Err(format!("choice {:?}: expected 1 to {MAX_CHOICE_OPTIONS} options", choice.id));
}
for option in &choice.options {
check_named(&option.id, &option.name, "choice option")?;
}
check_unique(choice.options.iter().map(|o| &o.id), "choice option")?;
if !choice.options.iter().any(|o| o.id == choice.default) {
return Err(format!("choice {:?}: default {:?} is not one of its options", choice.id, choice.default));
}
}
Ok(())
}
}
fn check_overrides(
overrides: &BTreeMap<String, String>,
check_ref: &dyn Fn(&str, &str) -> Result<(), String>,
what: &str,
) -> Result<(), String> {
for (name, icon) in overrides {
validate_item_id(name, "overridden icon name")?;
check_ref(icon, what)?;
}
Ok(())
}
@@ -0,0 +1,105 @@
use regex_syntax::hir::{Hir, HirKind};
pub(crate) const MAX_PATTERN_LEN: usize = 512;
const NAME_LEN: u64 = 256;
const MAX_COST: u64 = 4 * NAME_LEN * NAME_LEN;
const UNSUPPORTED: &[(&str, &str)] = &[
("\\A", "\\A"),
("\\z", "\\z"),
("\\Z", "\\Z"),
("\\p", "\\p"),
("\\P", "\\P"),
("[[:", "a POSIX class"),
("&&", "class intersection"),
("~~", "class symmetric difference"),
];
pub(crate) fn validate_pattern(pattern: &str) -> Result<(), String> {
if pattern.len() > MAX_PATTERN_LEN {
return Err(format!("pattern {pattern:?} is too long (max {MAX_PATTERN_LEN})"));
}
if let Some(construct) = unsupported_construct(pattern) {
return Err(format!("pattern {pattern:?}: {construct} is not supported"));
}
let hir = regex_syntax::Parser::new().parse(pattern).map_err(|e| format!("invalid pattern {pattern:?}: {e}"))?;
match cost(&hir) {
Err(why) => Err(format!("pattern {pattern:?}: {why} is not supported")),
Ok(c) if c > MAX_COST => Err(format!("pattern {pattern:?} is too complex: too many ways to match one name")),
Ok(_) => Ok(()),
}
}
fn unsupported_construct(pattern: &str) -> Option<&'static str> {
if pattern.match_indices("(?").any(|(at, _)| !pattern[at + 2..].starts_with(':')) {
return Some("an inline flag, lookaround or named group");
}
UNSUPPORTED.iter().find(|(needle, _)| pattern.contains(needle)).map(|(_, what)| *what)
}
fn cost(hir: &Hir) -> Result<u64, &'static str> {
match hir.kind() {
HirKind::Empty | HirKind::Literal(_) | HirKind::Class(_) | HirKind::Look(_) => Ok(1),
HirKind::Capture(c) => cost(&c.sub),
HirKind::Concat(items) => items.iter().try_fold(1u64, |acc, h| Ok(acc.saturating_mul(cost(h)?))),
HirKind::Alternation(items) => items.iter().try_fold(0u64, |acc, h| Ok(acc.saturating_add(cost(h)?))),
HirKind::Repetition(r) => match r.max {
None if matches!(r.sub.kind(), HirKind::Literal(_) | HirKind::Class(_)) => Ok(NAME_LEN),
None => Err("a repeated group (`(…)*`, `(…)+`)"),
Some(max) => Ok(cost(&r.sub)?.saturating_add(1).saturating_pow(max)),
},
}
}
#[cfg(test)]
mod tests {
use super::*;
fn err(pattern: &str) -> String {
validate_pattern(pattern).unwrap_err()
}
#[test]
fn accepts_typical_file_name_patterns() {
for pattern in [
r"^(architecture|ARCHITECTURE)(\.(md|rst|txt))?$",
r"(main|workflow|ci|release|build|config)\.ya?ml$",
r"^[a-z]+\.d\.ts$",
r"^\.?eslintrc(\.(js|cjs|json|ya?ml))?$",
r"^.*\.config\.[cm]?[jt]s$",
r"^.*-(test|spec)\..*$",
r"^[0-9]{4}-[0-9]{2}\.log$",
] {
assert!(validate_pattern(pattern).is_ok(), "{pattern}: {:?}", validate_pattern(pattern));
}
}
#[test]
fn rejects_patterns_that_backtrack_catastrophically() {
assert!(err(r"(a+)+$").contains("repeated group"));
assert!(err(r"(?:a*)*").contains("repeated group"));
assert!(err(r"^(\w|\w\w)*!$").contains("repeated group"));
assert!(err(r"^(a|aa)+$").contains("repeated group"));
assert!(err(r"^.*.*.*x$").contains("too complex"));
assert!(err(r"^(a?){30}a{30}$").contains("too complex"));
assert!(err(&format!("^{}b$", "(a|aa)".repeat(20))).contains("too complex"));
assert!(err(r"x{0,40}").contains("too complex"));
}
#[test]
fn rejects_constructs_javascript_reads_differently() {
assert!(err(r"(?i)readme").contains("not supported"));
assert!(err(r"(?P<n>a)").contains("not supported"));
assert!(err(r"\Areadme").contains("not supported"));
assert!(err(r"readme\z").contains("not supported"));
assert!(err(r"\p{L}").contains("not supported"));
assert!(err(r"[[:alpha:]]").contains("not supported"));
assert!(err(r"[a-z&&[^x]]").contains("not supported"));
}
#[test]
fn rejects_invalid_and_oversized_patterns() {
assert!(err("(").contains("invalid pattern"));
assert!(err(&"a".repeat(MAX_PATTERN_LEN + 1)).contains("too long"));
}
}
@@ -0,0 +1,185 @@
use super::icon_extensions::Extensions;
use serde::{Deserialize, Serialize};
const MAX_SECTIONS: usize = 16;
const MAX_ITEMS: usize = 32;
const MAX_TEXT_LEN: usize = 255;
const MAX_HINT_LEN: usize = 512;
const MAX_COLUMNS: u8 = 4;
#[derive(Clone, Copy, PartialEq, Eq, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub(super) enum SettingOption {
HideFiles,
HideFolders,
HideUiIcons,
HidePsiIcons,
HollowFolders,
Groups,
Choice,
Accent,
Monochrome,
Saturation,
Opacity,
IconSize,
RowHeight,
}
#[derive(Deserialize, Serialize)]
#[serde(untagged)]
pub(super) enum SettingDefault {
Number(u8),
Color(String),
}
#[derive(Deserialize, Serialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub(super) struct SettingItem {
option: SettingOption,
#[serde(default)]
label: Option<String>,
#[serde(default)]
hint: Option<String>,
#[serde(default)]
invert: bool,
#[serde(default)]
choice: Option<String>,
#[serde(default)]
columns: Option<u8>,
#[serde(default)]
default: Option<SettingDefault>,
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub(super) struct SettingsSection {
title: String,
items: Vec<SettingItem>,
}
#[derive(Serialize)]
struct ChoiceSummary<'a> {
id: &'a str,
name: &'a str,
default: &'a str,
options: Vec<OptionSummary<'a>>,
}
#[derive(Serialize)]
struct OptionSummary<'a> {
id: &'a str,
name: &'a str,
}
fn check_text(text: &str, max: usize, what: &str) -> Result<(), String> {
if text.trim().is_empty() || text.len() > max {
return Err(format!("settings: {what} must have 1 to {max} characters"));
}
Ok(())
}
fn range(option: SettingOption) -> Option<(u8, u8)> {
match option {
SettingOption::Saturation => Some((0, 20)),
SettingOption::Opacity => Some((10, 100)),
SettingOption::IconSize => Some((12, 32)),
SettingOption::RowHeight => Some((16, 48)),
_ => None,
}
}
fn is_color(text: &str) -> bool {
text.len() == 7 && text.starts_with('#') && text[1..].chars().all(|c| c.is_ascii_hexdigit())
}
fn is_flag(option: SettingOption) -> bool {
matches!(
option,
SettingOption::HideFiles
| SettingOption::HideFolders
| SettingOption::HideUiIcons
| SettingOption::HidePsiIcons
| SettingOption::HollowFolders
)
}
impl SettingItem {
fn validate(&self, ext: &Extensions) -> Result<(), String> {
let named = !matches!(self.option, SettingOption::Groups | SettingOption::Choice);
match &self.label {
Some(label) => check_text(label, MAX_TEXT_LEN, "label")?,
None if named => return Err("settings: label is required".into()),
None => {}
}
if let Some(hint) = &self.hint {
check_text(hint, MAX_HINT_LEN, "hint")?;
}
if self.invert && !is_flag(self.option) {
return Err("settings: invert is only for on/off options".into());
}
if self.columns.is_some_and(|c| c == 0 || c > MAX_COLUMNS)
|| (self.columns.is_some() && self.option != SettingOption::Groups)
{
return Err(format!("settings: columns are 1 to {MAX_COLUMNS}, for groups only"));
}
match (self.option, &self.choice) {
(SettingOption::Choice, Some(id)) if !ext.choices.iter().any(|c| &c.id == id) => {
return Err(format!("settings: unknown choice {id:?}"));
}
(SettingOption::Choice, None) => return Err("settings: choice items name their choice".into()),
(SettingOption::Choice, Some(_)) | (_, None) => {}
(_, Some(_)) => return Err("settings: only choice items name a choice".into()),
}
if self.option == SettingOption::Accent && !ext.accent {
return Err("settings: the accent option needs \"accent\": true".into());
}
self.validate_default()
}
fn validate_default(&self) -> Result<(), String> {
let colored = matches!(self.option, SettingOption::Accent | SettingOption::Monochrome);
match (range(self.option), &self.default) {
(Some((min, max)), Some(SettingDefault::Number(n))) if (min..=max).contains(n) => Ok(()),
(Some((min, max)), _) => Err(format!("settings: a default from {min} to {max} is required")),
(None, Some(SettingDefault::Color(c))) if colored && is_color(c) => Ok(()),
(None, _) if colored => Err("settings: a #rrggbb default is required".into()),
(None, None) => Ok(()),
(None, Some(_)) => Err("settings: this option takes no default".into()),
}
}
}
pub(super) fn validate(sections: &[SettingsSection], ext: &Extensions) -> Result<(), String> {
if sections.len() > MAX_SECTIONS {
return Err(format!("settings: more than {MAX_SECTIONS} sections"));
}
for section in sections {
check_text(&section.title, MAX_TEXT_LEN, "title")?;
if section.items.len() > MAX_ITEMS {
return Err(format!("settings: more than {MAX_ITEMS} items in {:?}", section.title));
}
for item in &section.items {
item.validate(ext)?;
}
}
Ok(())
}
pub(super) fn summary(sections: &[SettingsSection], ext: &Extensions) -> serde_json::Value {
let choices: Vec<ChoiceSummary> = ext
.choices
.iter()
.map(|c| ChoiceSummary {
id: &c.id,
name: &c.name,
default: &c.default,
options: c.options.iter().map(|o| OptionSummary { id: &o.id, name: &o.name }).collect(),
})
.collect();
serde_json::json!({
"groups": ext.groups,
"choices": choices,
"accent": ext.accent,
"settings": sections,
})
}
+844
View File
@@ -0,0 +1,844 @@
//! The `iconThemes` extension point: file and folder icons.
//!
//! ```toml
//! [[contributes.iconThemes]]
//! id = "material"
//! label = "Material Icons"
//! path = "icons/theme.json"
//! ```
//!
//! `theme.json` (paths relative to the package):
//!
//! ```json
//! {
//! "iconDefinitions": { "rust": "icons/rust.svg", "folder-src": "icons/folder-src.svg" },
//! "file": "file", "folder": "folder", "folderExpanded": "folder-open",
//! "fileExtensions": { "rs": "rust" },
//! "fileNames": { "cargo.toml": "cargo" },
//! "folderNames": { "src": "folder-src" }
//! }
//! ```
//!
//! Every SVG is sanitised here (allow-listed elements and attributes, no
//! scripts, event handlers, external references or entities) and stored as an
//! asset; the client fetches the mapping and the icons only when the theme is
//! selected, so installed icon themes cost nothing at start-up.
use super::icon_extensions::{Choice, DARK_SUFFIX, Extensions, Group, IconDefinition, Pattern, check_icon_id};
use super::icon_settings::{self, SettingsSection};
use ide_extension::{Package, UiDecl, check_relative, decode, validate_item_id};
use serde::Deserialize;
use std::collections::{BTreeMap, BTreeSet};
use std::path::PathBuf;
pub const POINT_ID: &str = "iconThemes";
const MAX_ICONS: usize = 5000;
const MAX_SVG: usize = 64 << 10;
const MAX_MAPPINGS: usize = 20_000;
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Decl {
id: String,
label: String,
path: PathBuf,
}
#[derive(Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
struct ThemeFile {
icon_definitions: BTreeMap<String, IconDefinition>,
#[serde(default)]
file: Option<String>,
#[serde(default)]
folder: Option<String>,
#[serde(default)]
folder_expanded: Option<String>,
#[serde(default)]
folder_open: Option<String>,
#[serde(default)]
file_extensions: BTreeMap<String, String>,
#[serde(default)]
file_names: BTreeMap<String, String>,
#[serde(default)]
folder_names: BTreeMap<String, String>,
#[serde(default)]
patterns: Vec<Pattern>,
#[serde(default)]
overrides: BTreeMap<String, String>,
#[serde(default)]
groups: Vec<Group>,
#[serde(default)]
choices: Vec<Choice>,
#[serde(default)]
accent: bool,
#[serde(default)]
settings: Vec<SettingsSection>,
}
/// Asset id of a theme's mapping (JSON).
pub fn map_asset(theme: &str) -> String {
format!("iconTheme/{theme}/map")
}
pub fn settings_asset(theme: &str) -> String {
format!("iconTheme/{theme}/settings")
}
/// Asset id prefix of a theme's SVGs (`…/svg/<icon id>`).
pub fn svg_prefix(theme: &str) -> String {
format!("iconTheme/{theme}/svg/")
}
/// An SVG icon of the package, size-checked and sanitised.
pub(super) fn read_icon(pkg: &Package, path: &std::path::Path) -> Result<String, String> {
check_relative(path, "icon")?;
let svg = pkg.read_text(path)?;
if svg.len() > MAX_SVG {
return Err(format!("{}: larger than {} KiB", path.display(), MAX_SVG >> 10));
}
sanitize_svg(&svg).map_err(|e| format!("{}: {e}", path.display()))
}
pub(super) fn icon_theme(pkg: &Package, decl: toml::Value) -> Result<UiDecl, String> {
let d: Decl = decode(POINT_ID, decl)?;
validate_item_id(&d.id, "icon theme id")?;
if d.label.trim().is_empty() {
return Err(format!("icon theme {:?} has an empty label", d.id));
}
let src = pkg.read_text(&d.path)?;
let mut theme: ThemeFile = serde_json::from_str(&src).map_err(|e| format!("{}: {e}", d.path.display()))?;
let extensions = Extensions {
patterns: std::mem::take(&mut theme.patterns),
overrides: std::mem::take(&mut theme.overrides),
groups: std::mem::take(&mut theme.groups),
choices: std::mem::take(&mut theme.choices),
accent: theme.accent,
};
let at = |what: &str| format!("{} ({what})", d.path.display());
if theme.icon_definitions.len() > MAX_ICONS {
return Err(format!("{}: more than {MAX_ICONS} icons", d.path.display()));
}
let mappings =
theme.file_extensions.len() + theme.file_names.len() + theme.folder_names.len() + extensions.mappings();
if mappings > MAX_MAPPINGS {
return Err(format!("{}: more than {MAX_MAPPINGS} mappings", d.path.display()));
}
let defined: BTreeSet<&str> = theme.icon_definitions.keys().map(String::as_str).collect();
let check_ref = |icon: &str, what: &str| -> Result<(), String> {
if defined.contains(icon) { Ok(()) } else { Err(format!("{}: unknown icon {icon:?}", at(what))) }
};
for icon in theme.icon_definitions.keys() {
check_icon_id(icon)?;
}
extensions.validate(&check_ref)?;
icon_settings::validate(&theme.settings, &extensions).map_err(|e| format!("{}: {e}", d.path.display()))?;
for (what, icon) in [
("file", &theme.file),
("folder", &theme.folder),
("folderExpanded", &theme.folder_expanded),
("folderOpen", &theme.folder_open),
] {
if let Some(icon) = icon {
check_ref(icon, what)?;
}
}
// Extensions may be written with their dot (`.rs`); names keep theirs (`.github`, `.gitignore`).
let lower = |map: &BTreeMap<String, String>, what: &str| -> Result<BTreeMap<String, String>, String> {
let mut out = BTreeMap::new();
for (k, icon) in map {
check_ref(icon, what)?;
let key = k.trim();
let key = if what == "fileExtensions" { key.trim_start_matches('.') } else { key }.to_lowercase();
if key.is_empty() || key.contains(['/', '\\']) {
return Err(format!("{}: invalid key {k:?}", at(what)));
}
out.insert(key, icon.clone());
}
Ok(out)
};
let file_extensions = lower(&theme.file_extensions, "fileExtensions")?;
let file_names = lower(&theme.file_names, "fileNames")?;
let folder_names = lower(&theme.folder_names, "folderNames")?;
let mut assets = Vec::with_capacity(theme.icon_definitions.len() + 1);
let mut files = vec![d.path.clone()];
for (icon, def) in &theme.icon_definitions {
let dark = def.dark().map(|path| (path, DARK_SUFFIX));
for (path, suffix) in std::iter::once((def.light(), "")).chain(dark) {
let clean = read_icon(pkg, path)?;
assets.push((format!("{}{icon}{suffix}", svg_prefix(&d.id)), clean));
files.push(path.clone());
}
}
let map = serde_json::json!({
"file": theme.file,
"folder": theme.folder,
"folderExpanded": theme.folder_expanded,
"folderOpen": theme.folder_open,
"fileExtensions": file_extensions,
"fileNames": file_names,
"folderNames": folder_names,
"patterns": extensions.patterns,
"overrides": extensions.overrides,
"groups": extensions.groups,
"choices": extensions.choices,
"accent": extensions.accent,
});
if !theme.settings.is_empty() {
assets.push((settings_asset(&d.id), icon_settings::summary(&theme.settings, &extensions).to_string()));
}
assets.push((map_asset(&d.id), map.to_string()));
let json = serde_json::json!({ "id": d.id, "label": d.label, "icons": theme.icon_definitions.len() });
let mut out = UiDecl::new(d.id.clone(), d.label, json);
out.files = files;
out.assets = assets;
Ok(out)
}
// ---------------------------------------------------------------------------
// SVG sanitiser
const ELEMENTS: &[&str] = &[
"svg",
"g",
"path",
"circle",
"ellipse",
"line",
"polyline",
"polygon",
"rect",
"defs",
"lineargradient",
"radialgradient",
"stop",
"clippath",
"mask",
"use",
"symbol",
"pattern",
"text",
"tspan",
"filter",
"fegaussianblur",
"feoffset",
"feblend",
"fecolormatrix",
"feflood",
"fecomposite",
"femerge",
"femergenode",
"fedropshadow",
];
/// Dropped with their content: harmless metadata.
const STRIPPED: &[&str] = &["title", "desc", "metadata"];
const ATTRIBUTES: &[&str] = &[
"id",
"class",
"viewbox",
"width",
"height",
"x",
"y",
"x1",
"y1",
"x2",
"y2",
"cx",
"cy",
"r",
"rx",
"ry",
"fx",
"fy",
"d",
"points",
"transform",
"fill",
"fill-opacity",
"fill-rule",
"clip-rule",
"clip-path",
"mask",
"stroke",
"stroke-width",
"stroke-linecap",
"stroke-linejoin",
"stroke-miterlimit",
"stroke-dasharray",
"stroke-dashoffset",
"stroke-opacity",
"opacity",
"offset",
"stop-color",
"stop-opacity",
"gradientunits",
"gradienttransform",
"spreadmethod",
"patternunits",
"patterncontentunits",
"patterntransform",
"clippathunits",
"maskunits",
"maskcontentunits",
"preserveaspectratio",
"style",
"href",
"xlink:href",
"xmlns",
"xmlns:xlink",
"version",
"font-family",
"font-size",
"font-weight",
"text-anchor",
"dominant-baseline",
"letter-spacing",
"display",
"visibility",
"color",
"filter",
"in",
"in2",
"result",
"stddeviation",
"dx",
"dy",
"mode",
"operator",
"values",
"type",
"flood-color",
"flood-opacity",
"k1",
"k2",
"k3",
"k4",
"filterunits",
"primitiveunits",
"vector-effect",
"shape-rendering",
"isolation",
"mix-blend-mode",
];
/// Parses `src` and re-serialises the allow-listed subset. Fails on anything
/// that could execute, fetch or escape (scripts, `foreignObject`, event
/// handlers, external `href`s, `url()` to anything but a local fragment,
/// DTDs, entities, CDATA).
pub fn sanitize_svg(src: &str) -> Result<String, String> {
let mut out = String::with_capacity(src.len());
let mut stack: Vec<String> = Vec::new();
// Depth inside a stripped element (its content is skipped).
let mut skip_depth = 0usize;
let mut seen_root = false;
let mut rest = src;
while !rest.is_empty() {
let Some(lt) = rest.find('<') else {
if skip_depth == 0 && !stack.is_empty() {
out.push_str(&text(rest)?);
} else if !rest.trim().is_empty() && stack.is_empty() {
return Err("text outside the <svg> element".into());
}
break;
};
if lt > 0 {
let t = &rest[..lt];
if skip_depth == 0 && !stack.is_empty() {
out.push_str(&text(t)?);
} else if stack.is_empty() && !t.trim().is_empty() {
return Err("text outside the <svg> element".into());
}
}
rest = &rest[lt..];
if let Some(after) = rest.strip_prefix("<!--") {
let end = after.find("-->").ok_or("unterminated comment")?;
rest = &after[end + 3..];
continue;
}
if rest.starts_with("<?") {
let end = rest.find("?>").ok_or("unterminated processing instruction")?;
rest = &rest[end + 2..];
continue;
}
if rest.starts_with("<!") {
return Err("DTDs, entities and CDATA are not allowed".into());
}
if let Some(after) = rest.strip_prefix("</") {
let end = after.find('>').ok_or("unterminated closing tag")?;
let name = after[..end].trim().to_ascii_lowercase();
rest = &after[end + 1..];
let open = stack.pop().ok_or_else(|| format!("unexpected </{name}>"))?;
if open != name {
return Err(format!("</{name}> does not close <{open}>"));
}
if skip_depth > 0 {
skip_depth -= 1;
} else {
out.push_str(&format!("</{name}>"));
}
continue;
}
// Start tag.
let end = tag_end(rest).ok_or("unterminated tag")?;
let inner = &rest[1..end];
rest = &rest[end + 1..];
let self_closing = inner.trim_end().ends_with('/');
let inner = inner.trim_end().trim_end_matches('/');
let name_end = inner.find(|c: char| c.is_whitespace()).unwrap_or(inner.len());
let raw_name = &inner[..name_end];
let name = raw_name.to_ascii_lowercase();
if name.is_empty() {
return Err("empty tag".into());
}
if stack.is_empty() {
if seen_root || name != "svg" {
return Err("the file must contain exactly one <svg> element".into());
}
seen_root = true;
}
if !self_closing {
stack.push(name.clone());
}
if skip_depth > 0 {
if !self_closing {
skip_depth += 1;
}
continue;
}
if STRIPPED.contains(&name.as_str()) || name.contains(':') {
// Editor metadata (sodipodi:namedview) or descriptive text.
if !self_closing {
skip_depth = 1;
}
continue;
}
if !ELEMENTS.contains(&name.as_str()) {
return Err(format!("<{raw_name}> is not allowed"));
}
out.push('<');
out.push_str(raw_name);
for (attr, value) in attributes(&inner[name_end..])? {
let key = attr.to_ascii_lowercase();
if key.starts_with("on") {
return Err(format!("event handler {attr} is not allowed"));
}
if key.contains(':') && !matches!(key.as_str(), "xlink:href" | "xmlns:xlink") {
continue; // inkscape:label, sodipodi:docname, …
}
if !ATTRIBUTES.contains(&key.as_str()) && !key.starts_with("data-") && !key.starts_with("aria-") {
return Err(format!("attribute {attr} is not allowed"));
}
check_value(&key, &value).map_err(|e| format!("{attr}: {e}"))?;
out.push(' ');
out.push_str(&attr);
out.push_str("=\"");
out.push_str(&escape(&value));
out.push('"');
}
out.push_str(if self_closing { "/>" } else { ">" });
}
if !stack.is_empty() {
return Err(format!("<{}> is not closed", stack.last().map(String::as_str).unwrap_or("?")));
}
if !seen_root {
return Err("no <svg> element".into());
}
Ok(out)
}
/// Index of the `>` ending the tag at the start of `s`, skipping quoted values.
fn tag_end(s: &str) -> Option<usize> {
let mut quote: Option<char> = None;
for (i, c) in s.char_indices().skip(1) {
match (quote, c) {
(Some(q), c) if c == q => quote = None,
(Some(_), _) => {}
(None, '"' | '\'') => quote = Some(c),
(None, '>') => return Some(i),
(None, '<') => return None,
_ => {}
}
}
None
}
fn attributes(s: &str) -> Result<Vec<(String, String)>, String> {
let mut out = Vec::new();
let mut rest = s.trim_start();
while !rest.is_empty() {
let eq = rest.find('=').ok_or_else(|| format!("attribute without value near {:?}", clip(rest)))?;
let name = rest[..eq].trim();
if name.is_empty() || name.contains(|c: char| c.is_whitespace() || c == '"' || c == '\'') {
return Err(format!("malformed attribute near {:?}", clip(rest)));
}
let after = rest[eq + 1..].trim_start();
let quote = after.chars().next().filter(|c| *c == '"' || *c == '\'').ok_or("unquoted attribute value")?;
let close = after[1..].find(quote).ok_or("unterminated attribute value")?;
let value = decode_entities(&after[1..1 + close])?;
if out.iter().any(|(n, _): &(String, String)| n.eq_ignore_ascii_case(name)) {
return Err(format!("duplicate attribute {name}"));
}
out.push((name.to_owned(), value));
rest = after[close + 2..].trim_start();
}
Ok(out)
}
fn check_value(key: &str, value: &str) -> Result<(), String> {
let lower = value.to_ascii_lowercase();
if matches!(key, "href" | "xlink:href") {
return if value.starts_with('#') { Ok(()) } else { Err("only local references (#id) are allowed".into()) };
}
if key == "xmlns" || key == "xmlns:xlink" {
return Ok(());
}
if lower.contains("javascript:")
|| lower.contains("expression(")
|| lower.contains("@import")
|| value.contains('\\')
{
return Err("unsafe value".into());
}
let mut rest = lower.as_str();
while let Some(i) = rest.find("url(") {
let inner = rest[i + 4..].trim_start().trim_start_matches(['"', '\'']);
if !inner.starts_with('#') {
return Err("url() may only reference a local fragment".into());
}
rest = &rest[i + 4..];
}
Ok(())
}
fn decode_entities(s: &str) -> Result<String, String> {
if !s.contains('&') {
return Ok(s.to_owned());
}
let mut out = String::with_capacity(s.len());
let mut rest = s;
while let Some(i) = rest.find('&') {
out.push_str(&rest[..i]);
let semi = rest[i..].find(';').ok_or("unterminated entity")?;
let ent = &rest[i + 1..i + semi];
let c = match ent {
"amp" => '&',
"lt" => '<',
"gt" => '>',
"quot" => '"',
"apos" => '\'',
_ => {
let code = if let Some(hex) = ent.strip_prefix("#x").or_else(|| ent.strip_prefix("#X")) {
u32::from_str_radix(hex, 16).ok()
} else if let Some(dec) = ent.strip_prefix('#') {
dec.parse().ok()
} else {
None
};
code.and_then(char::from_u32).ok_or_else(|| format!("entity &{ent}; is not allowed"))?
}
};
out.push(c);
rest = &rest[i + semi + 1..];
}
out.push_str(rest);
Ok(out)
}
fn text(t: &str) -> Result<String, String> {
Ok(escape(&decode_entities(t)?))
}
fn escape(s: &str) -> String {
s.replace('&', "&amp;").replace('<', "&lt;").replace('>', "&gt;").replace('"', "&quot;")
}
fn clip(s: &str) -> &str {
let mut end = s.len().min(24);
while !s.is_char_boundary(end) {
end -= 1;
}
&s[..end]
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn keeps_ordinary_icons() {
let src = r##"<?xml version="1.0"?>
<!-- Material icon -->
<svg xmlns="http://www.w3.org/2000/svg" xmlns:inkscape="x" viewBox="0 0 24 24" inkscape:version="1">
<title>rust</title>
<defs><linearGradient id="g"><stop offset="0" stop-color="#fff"/></linearGradient></defs>
<path fill="url(#g)" d="M1 2h3 &amp; 4" style="opacity:.5"/>
<use href="#g"/>
</svg>"##;
let out = sanitize_svg(src).unwrap();
assert!(out.starts_with("<svg xmlns="), "{out}");
assert!(!out.contains("title") && !out.contains("inkscape") && !out.contains("<!--"));
assert!(out.contains(r#"fill="url(#g)""#));
assert!(out.contains(r#"d="M1 2h3 &amp; 4""#));
assert!(out.ends_with("</svg>"));
}
#[test]
fn rejects_active_content() {
let bad = [
r##"<svg><script>alert(1)</script></svg>"##,
r##"<svg onload="alert(1)"></svg>"##,
r##"<svg><foreignObject/></svg>"##,
r##"<svg><use href="https://evil/x.svg#a"/></svg>"##,
r##"<svg><use xlink:href="data:image/svg+xml,..."/></svg>"##,
r##"<svg><rect style="fill:url(https://evil)"/></svg>"##,
r##"<svg><rect fill="url( 'http://x' )"/></svg>"##,
r##"<!DOCTYPE svg [<!ENTITY x "y">]><svg/>"##,
r##"<svg><![CDATA[x]]></svg>"##,
r##"<svg><a href="#x"/></svg>"##,
r##"<svg><rect width="&xx;"/></svg>"##,
r##"<svg><rect width=10/></svg>"##,
r##"<svg><g></svg>"##,
r##"<svg/><svg/>"##,
r##"<g/>"##,
r##"hello"##,
r##"<svg><rect style="behavior:url(#x);x:expression(1)"/></svg>"##,
];
for src in bad {
assert!(sanitize_svg(src).is_err(), "accepted: {src}");
}
}
#[test]
fn theme_files_validate_and_produce_assets() {
let tmp = tempfile::tempdir().unwrap();
let dir = tmp.path();
std::fs::create_dir_all(dir.join("icons")).unwrap();
std::fs::write(dir.join("icons/rust.svg"), r#"<svg viewBox="0 0 16 16"><path d="M0 0h16v16z"/></svg>"#)
.unwrap();
std::fs::write(dir.join("icons/file.svg"), r#"<svg viewBox="0 0 16 16"><rect width="16" height="16"/></svg>"#)
.unwrap();
std::fs::write(
dir.join("theme.json"),
r#"{"iconDefinitions":{"rust":"icons/rust.svg","file":"icons/file.svg"},"file":"file",
"fileExtensions":{".RS":"rust"},"fileNames":{"Cargo.toml":"rust",".gitignore":"file"},
"folderNames":{".GitHub":"file","src":"rust"}}"#,
)
.unwrap();
let manifest = ide_extension::Manifest::parse(
"id = \"p\"\nname = \"P\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"",
)
.unwrap();
let pkg = Package::new(manifest, dir.to_path_buf());
let decl = |path: &str| {
let mut t = toml::Table::new();
t.insert("id".into(), "m".into());
t.insert("label".into(), "M".into());
t.insert("path".into(), path.into());
toml::Value::Table(t)
};
let d = icon_theme(&pkg, decl("theme.json")).unwrap();
assert_eq!(d.json["icons"], 2);
assert_eq!(d.files.len(), 3);
let map: serde_json::Value =
serde_json::from_str(&d.assets.iter().find(|(k, _)| *k == map_asset("m")).unwrap().1).unwrap();
assert_eq!(map["fileExtensions"]["rs"], "rust");
assert_eq!(map["fileNames"]["cargo.toml"], "rust");
// Names keep their leading dot (only extensions drop it).
assert_eq!(map["fileNames"][".gitignore"], "file");
assert_eq!(map["folderNames"][".github"], "file");
assert_eq!(map["folderNames"]["src"], "rust");
assert!(map["folderNames"].get("github").is_none());
assert!(d.assets.iter().any(|(k, _)| *k == format!("{}rust", svg_prefix("m"))));
std::fs::write(dir.join("bad.json"), r#"{"iconDefinitions":{},"fileExtensions":{"rs":"nope"}}"#).unwrap();
assert!(icon_theme(&pkg, decl("bad.json")).unwrap_err().contains("unknown icon"));
std::fs::write(dir.join("icons/evil.svg"), r#"<svg><script/></svg>"#).unwrap();
std::fs::write(dir.join("evil.json"), r#"{"iconDefinitions":{"e":"icons/evil.svg"}}"#).unwrap();
assert!(icon_theme(&pkg, decl("evil.json")).unwrap_err().contains("evil.svg"));
assert!(icon_theme(&pkg, decl("../x.json")).is_err());
}
const SVG: &str = r#"<svg viewBox="0 0 16 16"><path d="M0 0h16v16z"/></svg>"#;
fn theme_package(theme: &str) -> (tempfile::TempDir, Package) {
let tmp = tempfile::tempdir().unwrap();
let dir = tmp.path();
std::fs::create_dir_all(dir.join("icons")).unwrap();
for icon in ["file", "folder", "folder-src", "folder-src-open", "rust", "rust-dark", "tests", "arrow"] {
std::fs::write(dir.join(format!("icons/{icon}.svg")), SVG).unwrap();
}
std::fs::write(dir.join("theme.json"), theme).unwrap();
let manifest = ide_extension::Manifest::parse(
"id = \"p\"\nname = \"P\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"",
)
.unwrap();
let pkg = Package::new(manifest, dir.to_path_buf());
(tmp, pkg)
}
fn load(theme: &str) -> Result<UiDecl, String> {
let (_tmp, pkg) = theme_package(theme);
let mut t = toml::Table::new();
t.insert("id".into(), "m".into());
t.insert("label".into(), "M".into());
t.insert("path".into(), "theme.json".into());
icon_theme(&pkg, toml::Value::Table(t))
}
fn with(extra: &str) -> String {
format!(
r#"{{"iconDefinitions":{{"file":"icons/file.svg","folder":"icons/folder.svg",
"folder-src":"icons/folder-src.svg","folder-src-open":"icons/folder-src-open.svg",
"rust":{{"light":"icons/rust.svg","dark":"icons/rust-dark.svg"}},
"tests":"icons/tests.svg","arrow":"icons/arrow.svg"}},
"file":"file","folder":"folder"{extra}}}"#
)
}
const EXTENSIONS: &str = r#","patterns":[
{"pattern":"^.*\\.rs$","icon":"rust","ignoreCase":true},
{"pattern":"/tests?/","icon":"tests","path":true,"group":"tests"},
{"pattern":"^src$","icon":"folder-src","target":"folder","openIcon":"folder-src-open"}],
"overrides":{"chevron-right":"arrow","symbol-class":"rust"},
"groups":[{"id":"tests","name":"Tests","enabled":false}],
"choices":[{"id":"arrows","name":"Arrows","default":"plain","options":[
{"id":"plain","name":"Default"},{"id":"material","name":"Material","overrides":{"chevron-down":"arrow"}}]}],
"accent":true,"folderOpen":"folder-src-open""#;
fn map_of(d: &UiDecl) -> serde_json::Value {
serde_json::from_str(&d.assets.iter().find(|(k, _)| *k == map_asset("m")).unwrap().1).unwrap()
}
#[test]
fn theme_extensions_reach_the_mapping() {
let d = load(&with(EXTENSIONS)).unwrap();
let map = map_of(&d);
assert_eq!(
map["patterns"],
serde_json::json!([
{"pattern":"^.*\\.rs$","icon":"rust","target":"file","path":false,"ignoreCase":true,"group":null,"openIcon":null},
{"pattern":"/tests?/","icon":"tests","target":"file","path":true,"ignoreCase":false,"group":"tests","openIcon":null},
{"pattern":"^src$","icon":"folder-src","target":"folder","path":false,"ignoreCase":false,"group":null,"openIcon":"folder-src-open"}
])
);
assert_eq!(map["overrides"], serde_json::json!({"chevron-right":"arrow","symbol-class":"rust"}));
assert_eq!(map["groups"], serde_json::json!([{"id":"tests","name":"Tests","enabled":false}]));
assert_eq!(map["choices"][0]["default"], "plain");
assert_eq!(map["choices"][0]["options"][1]["overrides"], serde_json::json!({"chevron-down":"arrow"}));
assert_eq!(map["choices"][0]["options"][0]["overrides"], serde_json::json!({}));
assert_eq!(map["accent"], true);
assert_eq!(map["folderOpen"], "folder-src-open");
assert_eq!(d.json["icons"], 7);
assert!(d.assets.iter().any(|(k, _)| *k == format!("{}rust", svg_prefix("m"))));
assert!(d.assets.iter().any(|(k, _)| *k == format!("{}rust_dark", svg_prefix("m"))));
assert!(d.files.iter().any(|f| f.ends_with("icons/rust-dark.svg")));
}
#[test]
fn themes_without_extensions_keep_their_mapping() {
let map = map_of(&load(&with("")).unwrap());
assert_eq!(map["patterns"], serde_json::json!([]));
assert_eq!(map["overrides"], serde_json::json!({}));
assert_eq!(map["groups"], serde_json::json!([]));
assert_eq!(map["choices"], serde_json::json!([]));
assert_eq!(map["accent"], false);
}
#[test]
fn theme_extensions_are_validated() {
let cases = [
(r#","patterns":[{"pattern":"x","icon":"nope"}]"#, "unknown icon"),
(r#","patterns":[{"pattern":"(a+)+$","icon":"rust"}]"#, "repeated group"),
(r#","patterns":[{"pattern":"(","icon":"rust"}]"#, "invalid pattern"),
(r#","patterns":[{"pattern":"x","icon":"rust","group":"nope"}]"#, "unknown group"),
(r#","patterns":[{"pattern":"x","icon":"rust","openIcon":"tests"}]"#, "openIcon"),
(r#","patterns":[{"pattern":"x","icon":"folder","target":"folder","openIcon":"nope"}]"#, "unknown icon"),
(r#","patterns":[{"pattern":"x","icon":"rust","priority":1}]"#, "unknown field"),
(r#","overrides":{"chevron-right":"nope"}"#, "unknown icon"),
(r#","folderOpen":"nope""#, "unknown icon"),
(r#","overrides":{"Chevron":"rust"}"#, "invalid overridden icon"),
(r#","groups":[{"id":"a","name":" "}]"#, "name"),
(r#","groups":[{"id":"a","name":"A"},{"id":"a","name":"B"}]"#, "duplicate"),
(r#","choices":[{"id":"c","name":"C","default":"x","options":[{"id":"y","name":"Y"}]}]"#, "default"),
(r#","choices":[{"id":"c","name":"C","default":"y","options":[]}]"#, "options"),
(
r#","choices":[{"id":"c","name":"C","default":"y","options":[{"id":"y","name":"Y","overrides":{"a":"nope"}}]}]"#,
"unknown icon",
),
];
for (extra, needle) in cases {
let err = load(&with(extra)).err().unwrap_or_else(|| panic!("accepted: {extra}"));
assert!(err.contains(needle), "{extra}: {err}");
}
let dark_id = r#"{"iconDefinitions":{"x_dark":"icons/rust.svg"}}"#;
assert!(load(dark_id).unwrap_err().contains("_dark"));
}
const SETTINGS: &str = r##","settings":[
{"title":"Toggle","items":[
{"option":"hideFiles","label":"Enable file icons","hint":"Files","invert":true},
{"option":"hollowFolders","label":"Hollow"}]},
{"title":"Packs","items":[{"option":"groups","columns":2}]},
{"title":"Other","items":[
{"option":"choice","choice":"arrows"},
{"option":"accent","label":"Accent","default":"#009688"},
{"option":"saturation","label":"Saturation","default":10}]}]"##;
fn settings_of(d: &UiDecl) -> Option<serde_json::Value> {
d.assets.iter().find(|(k, _)| *k == settings_asset("m")).map(|(_, v)| serde_json::from_str(v).unwrap())
}
#[test]
fn theme_settings_get_their_own_asset() {
let d = load(&with(&format!("{EXTENSIONS}{SETTINGS}"))).unwrap();
let s = settings_of(&d).unwrap();
assert_eq!(s["accent"], true);
assert_eq!(s["groups"], serde_json::json!([{"id":"tests","name":"Tests","enabled":false}]));
assert_eq!(
s["choices"],
serde_json::json!([{"id":"arrows","name":"Arrows","default":"plain","options":[
{"id":"plain","name":"Default"},{"id":"material","name":"Material"}]}])
);
assert_eq!(s["settings"][0]["title"], "Toggle");
assert_eq!(
s["settings"][0]["items"][0],
serde_json::json!({"option":"hideFiles","label":"Enable file icons","hint":"Files","invert":true,
"choice":null,"columns":null,"default":null})
);
assert_eq!(s["settings"][2]["items"][1]["default"], "#009688");
assert_eq!(s["settings"][2]["items"][2]["default"], 10);
assert!(settings_of(&load(&with(EXTENSIONS)).unwrap()).is_none());
}
#[test]
fn theme_settings_are_validated() {
let section = |items: &str| with(&format!(r#"{EXTENSIONS},"settings":[{{"title":"T","items":[{items}]}}]"#));
let cases = [
(section(r#"{"option":"nope","label":"X"}"#), "unknown variant"),
(section(r#"{"option":"hideFiles"}"#), "label"),
(section(r#"{"option":"hideFiles","label":" "}"#), "label"),
(section(r#"{"option":"choice","choice":"nope"}"#), "unknown choice"),
(section(r#"{"option":"choice"}"#), "choice"),
(section(r#"{"option":"groups","columns":9}"#), "columns"),
(section(r#"{"option":"saturation","label":"S"}"#), "default"),
(section(r#"{"option":"saturation","label":"S","default":50}"#), "default"),
(section(r##"{"option":"opacity","label":"O","default":"#000000"}"##), "default"),
(section(r#"{"option":"monochrome","label":"M","default":"red"}"#), "default"),
(section(r#"{"option":"saturation","label":"S","default":5,"invert":true}"#), "invert"),
(section(r#"{"option":"hideFiles","label":"F","extra":1}"#), "unknown field"),
(with(&format!(r#"{EXTENSIONS},"settings":[{{"title":"","items":[]}}]"#)), "title"),
(
with(r##","settings":[{"title":"T","items":[{"option":"accent","label":"A","default":"#000000"}]}]"##),
"accent",
),
];
for (theme, needle) in cases {
let err = load(&theme).err().unwrap_or_else(|| panic!("accepted: {theme}"));
assert!(err.contains(needle), "{theme}: {err}");
}
}
}
+288
View File
@@ -0,0 +1,288 @@
//! Copying a package into the plugins directory without ever leaving the
//! plugin missing. Installed plugins are versioned:
//! `<plugins>/<id>/<version>/` plus an `active` pointer file (and `previous`,
//! the version to roll back to). A new version is written to a staging
//! directory, renamed into place, and only then made active by replacing the
//! pointer; a swap interrupted by a crash is undone on the next start
//! ([`recover`]). Older installs used `<plugins>/<id>/` directly; they are
//! moved into the versioned layout on their next update.
use std::io;
use std::path::{Path, PathBuf};
use std::time::{SystemTime, UNIX_EPOCH};
/// Top-level files copied with a package besides those its declarations reference.
pub const LICENSE_FILES: &[&str] =
&["LICENSE", "LICENSE.md", "LICENSE.txt", "NOTICE", "NOTICE.md", "NOTICE.txt", "COPYING", "README.md"];
/// Written into an installed package: the package hash the grant ledger compares.
pub const HASH_FILE: &str = ".package-hash";
const STAGING: &str = ".staging-";
const OLD: &str = ".old-";
/// `<root>/active`: the version in use.
pub const ACTIVE_FILE: &str = "active";
/// `<root>/previous`: the version an update replaced (for rollback).
pub const PREVIOUS_FILE: &str = "previous";
fn nonce() -> String {
let n = SystemTime::now().duration_since(UNIX_EPOCH).map(|d| d.as_nanos()).unwrap_or_default();
format!("{n:x}-{}", std::process::id())
}
/// Copies `files` (plus manifest and licence files) from `src` into `target`
/// atomically, recording `hash`.
pub fn install_package(src: &Path, target: &Path, files: &[PathBuf], hash: &str) -> io::Result<()> {
let parent = target.parent().ok_or_else(|| io::Error::other("plugin target has no parent"))?;
let name = target.file_name().and_then(|n| n.to_str()).ok_or_else(|| io::Error::other("bad target"))?;
std::fs::create_dir_all(parent)?;
let staging = parent.join(format!("{STAGING}{name}-{}", nonce()));
let result = (|| {
std::fs::create_dir_all(&staging)?;
std::fs::copy(src.join(ide_extension::MANIFEST_FILE), staging.join(ide_extension::MANIFEST_FILE))?;
for file in files {
let to = staging.join(file);
if let Some(dir) = to.parent() {
std::fs::create_dir_all(dir)?;
}
std::fs::copy(src.join(file), to)?;
}
for lic in LICENSE_FILES {
if src.join(lic).is_file() {
std::fs::copy(src.join(lic), staging.join(lic))?;
}
}
std::fs::write(staging.join(HASH_FILE), hash)?;
swap(&staging, target, |from, to| std::fs::rename(from, to))
})();
if result.is_err() {
let _ = std::fs::remove_dir_all(&staging);
}
result
}
/// A version as a directory name (semver: `[0-9A-Za-z.+-]`).
fn version_dir(version: &str) -> io::Result<&str> {
let ok = !version.is_empty()
&& version.len() <= 64
&& !version.starts_with('.')
&& version.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '+' | '-'));
if ok { Ok(version) } else { Err(io::Error::other(format!("version {version:?} cannot name a directory"))) }
}
fn read_pointer(root: &Path, name: &str) -> Option<String> {
let v = std::fs::read_to_string(root.join(name)).ok()?;
let v = v.trim();
(!v.is_empty() && version_dir(v).is_ok()).then(|| v.to_owned())
}
/// Replaces the pointer file `name` atomically.
fn write_pointer(root: &Path, name: &str, version: &str) -> io::Result<()> {
let tmp = root.join(format!(".{name}.tmp"));
std::fs::write(&tmp, version)?;
std::fs::rename(&tmp, root.join(name))
}
/// The directory of a plugin's active version under its `root`
/// (`<plugins>/<id>`): `<root>/<active>`, or `root` itself in the flat layout.
pub fn active_dir(root: &Path) -> Option<PathBuf> {
if let Some(v) = read_pointer(root, ACTIVE_FILE) {
let dir = root.join(v);
return dir.join(ide_extension::MANIFEST_FILE).is_file().then_some(dir);
}
root.join(ide_extension::MANIFEST_FILE).is_file().then(|| root.to_path_buf())
}
/// The version an update replaced, if it is still there.
pub fn previous_version(root: &Path) -> Option<String> {
read_pointer(root, PREVIOUS_FILE).filter(|v| root.join(v).join(ide_extension::MANIFEST_FILE).is_file())
}
/// Moves a flat install (`<root>/plugin.toml`) to `<root>/<version>/`.
fn migrate_flat(root: &Path, version_of: impl Fn(&Path) -> Option<String>) -> io::Result<()> {
if read_pointer(root, ACTIVE_FILE).is_some() || !root.join(ide_extension::MANIFEST_FILE).is_file() {
return Ok(());
}
let version = version_of(root).unwrap_or_else(|| "0.0.0".into());
let version = version_dir(&version).map(str::to_owned).unwrap_or_else(|_| "0.0.0".into());
let parent = root.parent().unwrap_or(Path::new("."));
let name = root.file_name().and_then(|n| n.to_str()).ok_or_else(|| io::Error::other("bad plugin directory"))?;
// `.old-<name>-…` is restored by `recover` if this is interrupted.
let old = parent.join(format!("{OLD}{name}-{}", nonce()));
std::fs::rename(root, &old)?;
std::fs::create_dir_all(root)?;
std::fs::rename(&old, root.join(&version))?;
write_pointer(root, ACTIVE_FILE, &version)
}
/// Installs `files` of `src` as `<root>/<version>` and makes it the active
/// version. Returns the version it replaced (now `previous`).
pub fn install_version(
src: &Path,
root: &Path,
version: &str,
files: &[PathBuf],
hash: &str,
version_of: impl Fn(&Path) -> Option<String>,
) -> io::Result<Option<String>> {
let version = version_dir(version)?;
std::fs::create_dir_all(root)?;
migrate_flat(root, version_of)?;
install_package(src, &root.join(version), files, hash)?;
let previous = read_pointer(root, ACTIVE_FILE).filter(|v| v != version);
if let Some(p) = &previous {
write_pointer(root, PREVIOUS_FILE, p)?;
}
write_pointer(root, ACTIVE_FILE, version)?;
Ok(previous)
}
/// Makes `version` active again (rollback). The replaced version stays until [`prune`].
pub fn activate_version(root: &Path, version: &str) -> io::Result<()> {
let version = version_dir(version)?;
if !root.join(version).join(ide_extension::MANIFEST_FILE).is_file() {
return Err(io::Error::other(format!("version {version} is not installed")));
}
let _ = std::fs::remove_file(root.join(PREVIOUS_FILE));
write_pointer(root, ACTIVE_FILE, version)
}
/// Removes versions other than the active and the previous one.
pub fn prune(root: &Path) {
let keep: Vec<String> = [ACTIVE_FILE, PREVIOUS_FILE].iter().filter_map(|f| read_pointer(root, f)).collect();
if keep.is_empty() {
return;
}
let Ok(entries) = std::fs::read_dir(root) else { return };
for e in entries.flatten() {
let path = e.path();
let name = e.file_name().to_string_lossy().into_owned();
if path.is_dir() && !keep.contains(&name) {
let _ = std::fs::remove_dir_all(&path);
}
}
}
/// Moves `staging` to `target`, keeping the old `target` until the new one is in place.
fn swap(staging: &Path, target: &Path, rename: impl Fn(&Path, &Path) -> io::Result<()>) -> io::Result<()> {
if !target.exists() {
return rename(staging, target);
}
let parent = target.parent().unwrap_or(Path::new("."));
let name = target.file_name().and_then(|n| n.to_str()).unwrap_or("plugin");
let old = parent.join(format!("{OLD}{name}-{}", nonce()));
rename(target, &old)?;
if let Err(e) = rename(staging, target) {
// Put the previous version back.
rename(&old, target)?;
return Err(e);
}
if let Err(e) = std::fs::remove_dir_all(&old) {
tracing::warn!("removing the previous version {}: {e}", old.display());
}
Ok(())
}
/// Undoes interrupted installs in `dir`: restores `.old-<name>-*` when
/// `<name>` is missing, and removes leftover staging and old directories.
pub fn recover(dir: &Path) {
let Ok(entries) = std::fs::read_dir(dir) else { return };
for entry in entries.flatten() {
let path = entry.path();
// Inside a versioned plugin: staging and old versions of a version swap.
if path.is_dir() && read_pointer(&path, ACTIVE_FILE).is_some() {
recover(&path);
continue;
}
let Some(file) = path.file_name().and_then(|n| n.to_str()).map(str::to_owned) else { continue };
if file.starts_with(STAGING) {
let _ = std::fs::remove_dir_all(&path);
} else if let Some(rest) = file.strip_prefix(OLD) {
// `<name>-<nonce>-<pid>`: the name is everything before the last two parts.
let parts: Vec<&str> = rest.rsplitn(3, '-').collect();
let name = parts.get(2).copied().unwrap_or_default();
let target = dir.join(name);
// A layout migration interrupted after creating the new root: it is still empty.
if !name.is_empty() && target.is_dir() && active_dir(&target).is_none() {
let _ = std::fs::remove_dir_all(&target);
}
if !name.is_empty() && !target.exists() {
tracing::warn!("restoring {} after an interrupted install", target.display());
let _ = std::fs::rename(&path, &target);
} else {
let _ = std::fs::remove_dir_all(&path);
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn failed_swap_keeps_the_old_version() {
let tmp = tempfile::tempdir().unwrap();
let target = tmp.path().join("p");
let staging = tmp.path().join(".staging-p-1");
std::fs::create_dir_all(&target).unwrap();
std::fs::write(target.join("v"), "1").unwrap();
std::fs::create_dir_all(&staging).unwrap();
std::fs::write(staging.join("v"), "2").unwrap();
let calls = std::cell::Cell::new(0);
let err = swap(&staging, &target, |a, b| {
calls.set(calls.get() + 1);
if calls.get() == 2 { Err(io::Error::other("disk full")) } else { std::fs::rename(a, b) }
});
assert!(err.is_err());
assert_eq!(std::fs::read_to_string(target.join("v")).unwrap(), "1");
swap(&staging, &target, |a, b| std::fs::rename(a, b)).unwrap();
assert_eq!(std::fs::read_to_string(target.join("v")).unwrap(), "2");
assert_eq!(std::fs::read_dir(tmp.path()).unwrap().count(), 1, "old version removed");
}
#[test]
fn versions_are_installed_side_by_side_and_rolled_back() {
let tmp = tempfile::tempdir().unwrap();
let src = tmp.path().join("src");
std::fs::create_dir_all(&src).unwrap();
let root = tmp.path().join("plugins/acme.tools");
let version_of = |_: &Path| Some("0.9.0".to_owned());
// A flat install from before versioning.
std::fs::create_dir_all(&root).unwrap();
std::fs::write(root.join(ide_extension::MANIFEST_FILE), "v0.9").unwrap();
assert_eq!(active_dir(&root), Some(root.clone()));
std::fs::write(src.join(ide_extension::MANIFEST_FILE), "v1").unwrap();
assert_eq!(install_version(&src, &root, "1.0.0", &[], "h1", version_of).unwrap().as_deref(), Some("0.9.0"));
assert_eq!(active_dir(&root), Some(root.join("1.0.0")));
std::fs::write(src.join(ide_extension::MANIFEST_FILE), "v2").unwrap();
assert_eq!(install_version(&src, &root, "2.0.0", &[], "h2", version_of).unwrap().as_deref(), Some("1.0.0"));
assert_eq!(previous_version(&root).as_deref(), Some("1.0.0"));
prune(&root);
assert!(!root.join("0.9.0").exists(), "older than previous: pruned");
activate_version(&root, "1.0.0").unwrap();
assert_eq!(
std::fs::read_to_string(active_dir(&root).unwrap().join(ide_extension::MANIFEST_FILE)).unwrap(),
"v1"
);
assert_eq!(previous_version(&root), None);
assert!(install_version(&src, &root, "../x", &[], "h", version_of).is_err());
}
#[test]
fn recover_restores_an_interrupted_swap() {
let tmp = tempfile::tempdir().unwrap();
std::fs::create_dir_all(tmp.path().join(".old-acme.tools-1a2b-77")).unwrap();
std::fs::write(tmp.path().join(".old-acme.tools-1a2b-77/v"), "1").unwrap();
std::fs::create_dir_all(tmp.path().join(".staging-acme.tools-1a2c-77")).unwrap();
std::fs::create_dir_all(tmp.path().join("x")).unwrap();
std::fs::create_dir_all(tmp.path().join(".old-x-1-2")).unwrap();
recover(tmp.path());
assert_eq!(std::fs::read_to_string(tmp.path().join("acme.tools/v")).unwrap(), "1");
let mut names: Vec<_> =
std::fs::read_dir(tmp.path()).unwrap().flatten().map(|e| e.file_name().into_string().unwrap()).collect();
names.sort();
assert_eq!(names, ["acme.tools", "x"]);
}
}
+874
View File
@@ -0,0 +1,874 @@
//! IntelliJ theme plugins, read for import as OpenIDEA theme plugins: a
//! Marketplace download (a `.jar`, or a `.zip` holding the plugin's
//! `lib/*.jar`), an installed plugin's folder or a loose `.theme.json`; and the
//! theme plugins of the JetBrains IDEs on this machine, with the theme each IDE
//! is set to (`options/laf.xml`).
//!
//! Nothing is converted here: the UI maps IntelliJ keys onto design tokens
//! (`src/features/plugins/intellijTheme.ts`, shared with `pnpm
//! theme:import-intellij`) and [`super::PluginHost::stage_package`] writes the
//! result for the install page, which validates it like any package.
use ide_core::{IdeError, IdeResult, IpcPath};
use quick_xml::{Reader, XmlVersion, events::Event};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::ffi::OsString;
use std::io::{Cursor, Read, Seek};
use std::path::{Path, PathBuf};
/// A descriptor, theme or editor scheme larger than this is not one.
const MAX_FILE: u64 = 4 << 20;
/// A plugin jar read into memory from a Marketplace zip.
const MAX_JAR: u64 = 64 << 20;
/// Jars of one plugin looked into.
const MAX_JARS: usize = 64;
/// All the jars read into memory from one Marketplace zip.
const MAX_NESTED: u64 = 256 << 20;
/// Descriptors read for one plugin: `plugin.xml` and those it includes.
const MAX_DESCRIPTORS: usize = 64;
const PLUGIN_XML: &str = "META-INF/plugin.xml";
/// A theme plugin's themes, as read from its files.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct IntellijThemeSource {
/// The plugin's `<name>`, or the theme's for a loose file.
pub name: String,
pub plugin_id: Option<String>,
pub version: Option<String>,
pub vendor: Option<String>,
pub themes: Vec<IntellijThemeFile>,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct IntellijThemeFile {
/// The `themeProvider` id (what `laf.xml` names), or the file name without `.theme.json`.
pub id: String,
/// The `.theme.json`, comments removed.
pub theme: String,
/// The editor scheme (`.icls` XML) its `editorScheme` names, when the plugin has it.
pub editor_scheme: Option<String>,
}
/// A theme of a plugin installed in a JetBrains IDE.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct InstalledIntellijTheme {
/// The IDE and its version: `IntelliJ IDEA 2025.2`.
pub ide: String,
/// The plugin's name.
pub plugin: String,
/// What to import: the plugin's folder or jar.
pub path: IpcPath,
pub theme_id: String,
pub theme_name: String,
pub dark: bool,
/// The IDE is set to this theme.
pub active: bool,
}
/// Reads the themes of an IntelliJ theme plugin (a `.jar`, a Marketplace
/// `.zip`, an installed plugin's folder) or of a `.theme.json`.
pub fn read_source(path: &Path) -> IdeResult<IntellijThemeSource> {
read(path).map_err(IdeError::invalid)
}
/// The theme plugins of the JetBrains IDEs (and Android Studio) of this user,
/// the themes their IDEs are set to first.
pub fn installed_themes() -> Vec<InstalledIntellijTheme> {
let Some(home) = dirs::home_dir() else { return Vec::new() };
let layout = if cfg!(target_os = "macos") {
Layout::Mac
} else if cfg!(windows) {
Layout::Windows
} else {
Layout::Xdg
};
find_installed(&ide_dirs(layout, &home, |key| std::env::var_os(key)))
}
fn read(path: &Path) -> Result<IntellijThemeSource, String> {
if path.extension().is_some_and(|e| e.eq_ignore_ascii_case("json")) {
return loose_theme(path);
}
let mut jars = plugin_jars(path)?;
let plugin = descriptor(&mut jars)?;
let shown = || path.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
if plugin.themes.is_empty() {
return Err(format!("{} is not a theme plugin: its plugin.xml declares no themeProvider", shown()));
}
let mut themes = Vec::new();
let mut problem = None;
for (id, theme_path) in &plugin.themes {
match plugin_theme(&mut jars, id, theme_path) {
Ok(theme) => themes.push(theme),
Err(e) => problem = problem.or(Some(e)),
}
}
if themes.is_empty() {
return Err(problem.unwrap_or_else(|| format!("{} has no readable theme", shown())));
}
Ok(IntellijThemeSource {
name: plugin.name.unwrap_or_else(shown),
plugin_id: plugin.id,
version: plugin.version,
vendor: plugin.vendor,
themes,
})
}
fn plugin_theme(jars: &mut [Jar], id: &str, path: &str) -> Result<IntellijThemeFile, String> {
let bytes = find(jars, entry_name(path), MAX_FILE)?.ok_or_else(|| format!("{path} is missing from the plugin"))?;
let (theme, fields) = theme_json(&bytes).map_err(|e| format!("{path}: {e}"))?;
let editor_scheme = match string_field(&fields, "editorScheme") {
Some(scheme) => find(jars, entry_name(scheme), MAX_FILE)?.map(|b| text(&b)),
None => None,
};
Ok(IntellijThemeFile { id: id.to_owned(), theme, editor_scheme })
}
/// A `.theme.json` with the editor scheme beside it (its `editorScheme` names a path in the plugin's jar).
fn loose_theme(path: &Path) -> Result<IntellijThemeSource, String> {
let (theme, fields) = theme_json(&read_file(path)?).map_err(|e| format!("{}: {e}", path.display()))?;
// Any JSON object parses: an IntelliJ theme has UI colours or an editor scheme.
let scheme_ref = string_field(&fields, "editorScheme");
if !fields.get("ui").is_some_and(serde_json::Value::is_object) && scheme_ref.is_none() {
return Err(format!("{} is not an IntelliJ theme: it has no \"ui\" or \"editorScheme\"", path.display()));
}
let editor_scheme = scheme_ref
.and_then(|p| Path::new(p).file_name().map(|n| path.with_file_name(n)))
.filter(|p| p.is_file())
.map(|p| read_file(&p).map(|b| text(&b)))
.transpose()?;
let file = path.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
let id = file.strip_suffix(".theme.json").or_else(|| file.strip_suffix(".json")).unwrap_or(&file).to_owned();
Ok(IntellijThemeSource {
name: string_field(&fields, "name").map_or_else(|| id.clone(), str::to_owned),
plugin_id: None,
version: None,
vendor: string_field(&fields, "author").map(str::to_owned),
themes: vec![IntellijThemeFile { id, theme, editor_scheme }],
})
}
type JsonObject = serde_json::Map<String, serde_json::Value>;
/// The theme JSON, comments removed, and its fields, if it is an object.
fn theme_json(bytes: &[u8]) -> Result<(String, JsonObject), String> {
let json = super::theme::strip_json_comments(&text(bytes));
match serde_json::from_str::<serde_json::Value>(&json) {
Ok(serde_json::Value::Object(fields)) => Ok((json, fields)),
Ok(_) => Err("not a theme: expected a JSON object".into()),
Err(e) => Err(format!("invalid theme JSON: {e}")),
}
}
fn string_field<'a>(fields: &'a JsonObject, key: &str) -> Option<&'a str> {
fields.get(key)?.as_str().filter(|s| !s.trim().is_empty())
}
/// `/themes/x.theme.json` → `themes/x.theme.json`.
fn entry_name(path: &str) -> &str {
path.trim().trim_start_matches('/')
}
fn text(bytes: &[u8]) -> String {
let bytes = bytes.strip_prefix(b"\xEF\xBB\xBF").unwrap_or(bytes);
String::from_utf8_lossy(bytes).into_owned()
}
fn read_file(path: &Path) -> Result<Vec<u8>, String> {
let shown = |e: std::io::Error| format!("{}: {e}", path.display());
let file = std::fs::File::open(path).map_err(shown)?;
let mut buf = Vec::new();
file.take(MAX_FILE + 1).read_to_end(&mut buf).map_err(shown)?;
if buf.len() as u64 > MAX_FILE {
return Err(format!("{} is too large for a theme", path.display()));
}
Ok(buf)
}
trait ReadSeek: Read + Seek {}
impl<T: Read + Seek> ReadSeek for T {}
type Jar = zip::ZipArchive<Box<dyn ReadSeek>>;
fn open_jar(path: &Path) -> Result<Jar, String> {
let file = std::fs::File::open(path).map_err(|e| format!("{}: {e}", path.display()))?;
let reader: Box<dyn ReadSeek> = Box::new(std::io::BufReader::new(file));
zip::ZipArchive::new(reader).map_err(|e| format!("{} is not a jar or zip file ({e})", path.display()))
}
/// The jars of the plugin at `path`: a jar, a Marketplace zip (`<plugin>/lib/*.jar`)
/// or an installed plugin's folder (`lib/*.jar`).
fn plugin_jars(path: &Path) -> Result<Vec<Jar>, String> {
let not_plugin = || format!("{} is not an IntelliJ plugin: no {PLUGIN_XML} and no lib/*.jar", path.display());
if path.is_dir() {
let mut found: Vec<PathBuf> = [path.join("lib"), path.join("lib").join("modules")]
.iter()
.flat_map(|dir| std::fs::read_dir(dir).into_iter().flatten().flatten())
.map(|e| e.path())
.filter(|p| p.is_file() && is_jar(p))
.collect();
found.sort();
let jars: Vec<Jar> = found.iter().take(MAX_JARS).filter_map(|p| open_jar(p).ok()).collect();
return if jars.is_empty() { Err(not_plugin()) } else { Ok(jars) };
}
let mut archive = open_jar(path)?;
if archive.index_for_name(PLUGIN_XML).is_some() {
return Ok(vec![archive]);
}
let jars = nested_jars(&mut archive, MAX_JAR, MAX_NESTED);
if jars.is_empty() { Err(not_plugin()) } else { Ok(jars) }
}
/// The `lib` jars of a Marketplace zip, smallest first (a theme's own jar is small)
/// while they fit `max_total`; library jars too large or not jars at all are skipped.
fn nested_jars(archive: &mut Jar, max_jar: u64, max_total: u64) -> Vec<Jar> {
let mut nested: Vec<(u64, String)> = entry_names(archive)
.into_iter()
.filter(|n| is_lib_jar(n))
.filter_map(|n| Some((archive.by_name(&n).ok()?.size(), n)))
.filter(|(size, _)| *size <= max_jar)
.collect();
nested.sort();
let (mut jars, mut loaded) = (Vec::new(), 0u64);
for (size, name) in nested.into_iter().take(MAX_JARS) {
if loaded.saturating_add(size) > max_total {
break;
}
let Ok(Some(bytes)) = read_entry(archive, &name, max_jar) else { continue };
loaded += size;
let reader: Box<dyn ReadSeek> = Box::new(Cursor::new(bytes));
if let Ok(jar) = zip::ZipArchive::new(reader) {
jars.push(jar);
}
}
jars
}
fn entry_names(jar: &Jar) -> Vec<String> {
jar.file_names().filter_map(Result::ok).map(|n| n.into_owned()).collect()
}
fn is_jar(path: &Path) -> bool {
path.extension().is_some_and(|e| e.eq_ignore_ascii_case("jar"))
}
/// `lib/x.jar`, `<plugin>/lib/x.jar` or `<plugin>/lib/modules/x.jar` in a Marketplace zip.
fn is_lib_jar(name: &str) -> bool {
let parts: Vec<&str> = name.split('/').collect();
let n = parts.len();
name.to_ascii_lowercase().ends_with(".jar")
&& (2..=4).contains(&n)
&& (parts[n - 2] == "lib" || (n >= 3 && parts[n - 2] == "modules" && parts[n - 3] == "lib"))
}
fn read_entry(jar: &mut Jar, name: &str, max: u64) -> Result<Option<Vec<u8>>, String> {
let Ok(file) = jar.by_name(name) else { return Ok(None) };
if file.is_dir() {
return Ok(None);
}
if file.size() > max {
return Err(format!("{name} is too large"));
}
let mut buf = Vec::with_capacity(file.size() as usize);
file.take(max.saturating_add(1)).read_to_end(&mut buf).map_err(|e| format!("{name}: {e}"))?;
if buf.len() as u64 > max {
return Err(format!("{name} is too large"));
}
Ok(Some(buf))
}
fn find(jars: &mut [Jar], name: &str, max: u64) -> Result<Option<Vec<u8>>, String> {
for jar in jars.iter_mut() {
if let Some(bytes) = read_entry(jar, name, max)? {
return Ok(Some(bytes));
}
}
Ok(None)
}
/// What the plugin's descriptors say: `<idea-plugin>`'s `id`, `name`, `version`,
/// `vendor`, and its `themeProvider`s (id, path in the jar).
#[derive(Debug, Default, PartialEq)]
struct Descriptor {
id: Option<String>,
name: Option<String>,
version: Option<String>,
vendor: Option<String>,
themes: Vec<(String, String)>,
}
/// Reads `META-INF/plugin.xml` and the descriptors it includes (`<xi:include href>`), in any jar.
fn descriptor(jars: &mut [Jar]) -> Result<Descriptor, String> {
let main = find(jars, PLUGIN_XML, MAX_FILE)
.ok()
.flatten()
.ok_or_else(|| format!("not an IntelliJ plugin: no {PLUGIN_XML}"))?;
let mut out = Descriptor::default();
let mut includes = Vec::new();
parse_descriptor(&text(&main), true, &mut out, &mut includes).map_err(|e| format!("{PLUGIN_XML}: {e}"))?;
let mut read = std::collections::HashSet::from([PLUGIN_XML.to_owned()]);
while let Some(href) = includes.pop() {
let name = include_entry(&href);
if read.len() >= MAX_DESCRIPTORS || !read.insert(name.clone()) {
continue;
}
let Ok(Some(bytes)) = find(jars, &name, MAX_FILE) else { continue };
let _ = parse_descriptor(&text(&bytes), false, &mut out, &mut includes);
}
let mut seen = std::collections::HashSet::new();
out.themes.retain(|(id, _)| seen.insert(id.clone()));
Ok(out)
}
/// An `<xi:include href>` as a jar entry: `/META-INF/x.xml` from the jar's root, `x.xml` beside plugin.xml.
fn include_entry(href: &str) -> String {
let href = href.trim();
href.strip_prefix('/').map_or_else(|| format!("META-INF/{href}"), str::to_owned)
}
fn parse_descriptor(xml: &str, main: bool, out: &mut Descriptor, includes: &mut Vec<String>) -> Result<(), String> {
let mut reader = Reader::from_str(xml);
let mut stack: Vec<String> = Vec::new();
let mut text = String::new();
loop {
let event = reader.read_event().map_err(|e| e.to_string())?;
match event {
Event::Start(ref tag) | Event::Empty(ref tag) => {
let name = tag.local_name().as_ref().to_owned();
if name == "include" {
let href = tag.attributes().flatten().find(|a| a.key.local_name().as_ref() == "href");
if let Some(href) = href.and_then(|a| a.normalized_value(XmlVersion::Implicit1_0).ok()) {
includes.push(href.into_owned());
}
}
if name == "themeProvider" {
let (mut id, mut path) = (None, None);
for a in tag.attributes() {
let a = a.map_err(|e| e.to_string())?;
let value = a.normalized_value(XmlVersion::Implicit1_0).map_err(|e| e.to_string())?;
match a.key.local_name().as_ref() {
"id" => id = Some(value.trim().to_owned()),
"path" => path = Some(value.trim().to_owned()),
_ => {}
}
}
if let (Some(id), Some(path)) = (id, path)
&& !id.is_empty()
&& !path.is_empty()
{
out.themes.push((id, path));
}
}
if matches!(event, Event::Start(_)) {
stack.push(name);
text.clear();
}
}
Event::Text(t) => text.push_str(&t.xml_content(XmlVersion::Implicit1_0)),
Event::CData(t) => text.push_str(t.as_ref()),
Event::GeneralRef(r) => {
let name = r.as_ref();
let escaped = format!("&{name};");
text.push_str(&quick_xml::escape::unescape(&escaped).map_err(|e| e.to_string())?);
}
Event::End(_) => {
let name = stack.pop().ok_or("unbalanced element")?;
// Direct children of <idea-plugin>.
if main && stack.len() == 1 && stack[0] == "idea-plugin" {
let value = text.trim();
let slot = match name.as_str() {
"id" => Some(&mut out.id),
"name" => Some(&mut out.name),
"version" => Some(&mut out.version),
"vendor" => Some(&mut out.vendor),
_ => None,
};
if let Some(slot) = slot
&& slot.is_none()
&& !value.is_empty()
{
*slot = Some(value.to_owned());
}
}
text.clear();
}
Event::Eof => return if stack.is_empty() { Ok(()) } else { Err("unclosed element".into()) },
_ => {}
}
}
}
/// Where an OS keeps the IDEs' settings and plugins.
#[derive(Clone, Copy, Debug)]
enum Layout {
/// `~/Library/Application Support/<vendor>/<IDE>`, plugins in its `plugins/`.
Mac,
/// `%APPDATA%\<vendor>\<IDE>`, plugins in its `plugins\`.
Windows,
/// `$XDG_CONFIG_HOME/<vendor>/<IDE>`, plugins in `$XDG_DATA_HOME/<vendor>/<IDE>`.
Xdg,
}
#[derive(Debug, PartialEq)]
struct IdeDir {
label: String,
product: String,
version: (u32, u32),
config: PathBuf,
plugins: PathBuf,
}
/// The settings folders of JetBrains IDEs and Android Studio (2020.1 and later), newest first.
fn ide_dirs(layout: Layout, home: &Path, env: impl Fn(&str) -> Option<OsString>) -> Vec<IdeDir> {
let var = |key: &str| env(key).filter(|v| !v.is_empty()).map(PathBuf::from);
let (config_root, data_root) = match layout {
Layout::Mac => {
let root = home.join("Library").join("Application Support");
(root.clone(), root)
}
Layout::Windows => {
let root = var("APPDATA").unwrap_or_else(|| home.join("AppData").join("Roaming"));
(root.clone(), root)
}
Layout::Xdg => (
var("XDG_CONFIG_HOME").unwrap_or_else(|| home.join(".config")),
var("XDG_DATA_HOME").unwrap_or_else(|| home.join(".local").join("share")),
),
};
let mut out = Vec::new();
for vendor in ["JetBrains", "Google"] {
for entry in std::fs::read_dir(config_root.join(vendor)).into_iter().flatten().flatten() {
let name = entry.file_name().to_string_lossy().into_owned();
let Some((product, version, shown)) = product_dir(&name) else { continue };
let config = entry.path();
if !config.is_dir() {
continue;
}
let plugins = match layout {
Layout::Xdg => data_root.join(vendor).join(&name),
Layout::Mac | Layout::Windows => config.join("plugins"),
};
out.push(IdeDir {
label: format!("{} {shown}", product_label(product)),
product: product.to_owned(),
version,
config,
plugins,
});
}
}
out.sort_by(|a, b| b.version.cmp(&a.version).then_with(|| a.label.cmp(&b.label)));
out
}
/// `IntelliJIdea2025.2` → (`IntelliJIdea`, (2025, 2), `2025.2`).
fn product_dir(name: &str) -> Option<(&str, (u32, u32), &str)> {
let at = name.find(|c: char| c.is_ascii_digit())?;
let (product, version) = name.split_at(at);
if product.is_empty() || !product.chars().all(|c| c.is_ascii_alphabetic()) {
return None;
}
let (year, minor) = version.split_once('.')?;
let ok = |s: &str| !s.is_empty() && s.len() <= 4 && s.chars().all(|c| c.is_ascii_digit());
if year.len() != 4 || !ok(year) || !ok(minor) {
return None;
}
Some((product, (year.parse().ok()?, minor.parse().ok()?), version))
}
fn product_label(product: &str) -> &str {
match product {
"IntelliJIdea" => "IntelliJ IDEA",
"IdeaIC" => "IntelliJ IDEA Community Edition",
"PyCharmCE" => "PyCharm Community Edition",
"AndroidStudio" => "Android Studio",
"AndroidStudioPreview" => "Android Studio Preview",
other => other,
}
}
/// The theme ids `options/laf.xml` selects: the current one, and with "Sync with OS" the preferred light and dark.
fn active_theme_ids(config: &Path) -> Vec<String> {
let Ok(bytes) = read_file(&config.join("options").join("laf.xml")) else { return Vec::new() };
let xml = text(&bytes);
let mut reader = Reader::from_str(&xml);
let (mut ids, mut preferred, mut autodetect) = (Vec::new(), Vec::new(), false);
loop {
match reader.read_event() {
Ok(Event::Start(tag) | Event::Empty(tag)) => {
let attr = |key: &str| {
tag.attributes()
.flatten()
.find(|a| a.key.as_ref() == key)
.and_then(|a| a.normalized_value(XmlVersion::Implicit1_0).ok().map(|v| v.trim().to_owned()))
};
match tag.local_name().as_ref() {
"component" if attr("name").as_deref() == Some("LafManager") => {
autodetect = attr("autodetect").as_deref() == Some("true");
}
"laf" => ids.extend(attr("themeId")),
"preferred-light-laf" | "preferred-dark-laf" => preferred.extend(attr("themeId")),
_ => {}
}
}
Ok(Event::Eof) | Err(_) => break,
_ => {}
}
}
if autodetect {
ids.extend(preferred);
}
ids.retain(|id| !id.is_empty());
ids
}
fn find_installed(ides: &[IdeDir]) -> Vec<InstalledIntellijTheme> {
let mut out = Vec::new();
for ide in ides {
let active = active_theme_ids(&ide.config);
let mut plugins: Vec<PathBuf> = std::fs::read_dir(&ide.plugins)
.into_iter()
.flatten()
.flatten()
.map(|e| e.path())
.filter(|p| p.is_dir() || (p.is_file() && is_jar(p)))
.collect();
plugins.sort();
for plugin in plugins {
let Ok(mut jars) = plugin_jars(&plugin) else { continue };
let Ok(found) = descriptor(&mut jars) else { continue };
for (id, path) in &found.themes {
let Ok(Some(bytes)) = find(&mut jars, entry_name(path), MAX_FILE) else { continue };
let Ok((_, fields)) = theme_json(&bytes) else { continue };
let file = || plugin.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
out.push(InstalledIntellijTheme {
ide: ide.label.clone(),
plugin: found.name.clone().unwrap_or_else(file),
path: plugin.clone().into(),
theme_id: id.clone(),
theme_name: string_field(&fields, "name").unwrap_or(id).to_owned(),
dark: fields.get("dark").and_then(serde_json::Value::as_bool).unwrap_or(true),
active: active.contains(id),
});
}
}
}
// Stable: the IDEs' order (newest first) within each group.
out.sort_by_key(|t| !t.active);
out
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
const PLUGIN: &str = r#"<idea-plugin>
<id>com.example.deepsea</id>
<name>Deep Sea &amp; Friends</name>
<version>2.3</version>
<vendor email="[email protected]">Someone</vendor>
<description><![CDATA[<b>Dark</b> themes]]></description>
<depends>com.intellij.modules.platform</depends>
<extensions defaultExtensionNs="com.intellij">
<themeProvider id="deep-sea-id" path="/themes/DeepSea.theme.json"/>
<themeProvider id="shallow-id" path="themes/Shallow.theme.json" />
</extensions>
</idea-plugin>"#;
const DEEP: &str = r##"{
// comment
"name": "Deep Sea",
"dark": true,
"editorScheme": "/themes/DeepSea.xml",
"ui": { "*": { "background": "#101820" } }
}"##;
const SHALLOW: &str = r#"{ "name": "Shallow", "dark": false, "ui": {} }"#;
const SCHEME: &str = r#"<scheme name="Deep Sea" version="142" parent_scheme="Darcula"/>"#;
fn jar_bytes(entries: &[(&str, &[u8])]) -> Vec<u8> {
let mut zip = zip::ZipWriter::new(Cursor::new(Vec::new()));
for (name, data) in entries {
zip.start_file(*name, zip::write::SimpleFileOptions::default()).unwrap();
zip.write_all(data).unwrap();
}
zip.finish().unwrap().into_inner()
}
fn theme_jar() -> Vec<u8> {
jar_bytes(&[
("META-INF/MANIFEST.MF", b"Manifest-Version: 1.0\n"),
(PLUGIN_XML, PLUGIN.as_bytes()),
("themes/DeepSea.theme.json", DEEP.as_bytes()),
("themes/DeepSea.xml", SCHEME.as_bytes()),
("themes/Shallow.theme.json", SHALLOW.as_bytes()),
])
}
fn write(path: &Path, bytes: &[u8]) {
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
std::fs::write(path, bytes).unwrap();
}
fn expected_source() -> IntellijThemeSource {
IntellijThemeSource {
name: "Deep Sea & Friends".into(),
plugin_id: Some("com.example.deepsea".into()),
version: Some("2.3".into()),
vendor: Some("Someone".into()),
themes: vec![
IntellijThemeFile {
id: "deep-sea-id".into(),
theme: super::super::theme::strip_json_comments(DEEP),
editor_scheme: Some(SCHEME.into()),
},
IntellijThemeFile { id: "shallow-id".into(), theme: SHALLOW.into(), editor_scheme: None },
],
}
}
#[test]
fn reads_a_theme_jar_with_its_editor_scheme() {
let tmp = tempfile::tempdir().unwrap();
let jar = tmp.path().join("deep-sea.jar");
write(&jar, &theme_jar());
let source = read_source(&jar).unwrap();
assert_eq!(source, expected_source());
assert!(!source.themes[0].theme.contains("comment"));
}
#[test]
fn reads_a_marketplace_zip_and_an_installed_folder() {
let tmp = tempfile::tempdir().unwrap();
let other = jar_bytes(&[("META-INF/MANIFEST.MF", b"x"), ("a/B.class", b"\xCA\xFE")]);
let zip = tmp.path().join("DeepSea-2.3.zip");
write(&zip, &jar_bytes(&[("DeepSea/lib/a-util.jar", &other), ("DeepSea/lib/deep-sea.jar", &theme_jar())]));
assert_eq!(read_source(&zip).unwrap(), expected_source());
let dir = tmp.path().join("plugins").join("DeepSea");
write(&dir.join("lib").join("deep-sea.jar"), &theme_jar());
write(&dir.join("lib").join("a-util.jar"), &other);
assert_eq!(read_source(&dir).unwrap(), expected_source());
}
#[test]
fn skips_library_jars_it_cannot_or_need_not_read() {
let tmp = tempfile::tempdir().unwrap();
// Stored, not deflated: as large in the zip as its content.
let mut big = zip::ZipWriter::new(Cursor::new(Vec::new()));
let stored = zip::write::SimpleFileOptions::default().compression_method(zip::CompressionMethod::Stored);
big.start_file("lib/Data.class", stored).unwrap();
big.write_all(&[b'x'; 8192]).unwrap();
let big = big.finish().unwrap().into_inner();
let zip = tmp.path().join("DeepSea.zip");
write(
&zip,
&jar_bytes(&[
("DeepSea/lib/a-broken.jar", b"not a jar"),
("DeepSea/lib/b-big.jar", &big),
("DeepSea/lib/deep-sea.jar", &theme_jar()),
]),
);
assert_eq!(read_source(&zip).unwrap(), expected_source());
// The broken jar is skipped; a jar over the size limit, or past the total
// (smallest first), is not read, while the theme's small jar still is.
let mut archive = open_jar(&zip).unwrap();
let theme_len = theme_jar().len() as u64;
assert_eq!(nested_jars(&mut archive, u64::MAX, u64::MAX).len(), 2);
let one = nested_jars(&mut archive, theme_len + 64, u64::MAX);
assert_eq!(one.len(), 1);
assert!(one[0].index_for_name(PLUGIN_XML).is_some());
assert_eq!(nested_jars(&mut archive, u64::MAX, theme_len + 64).len(), 1);
assert!(nested_jars(&mut archive, 8, u64::MAX).is_empty());
}
#[test]
fn reads_a_loose_theme_json_with_the_scheme_beside_it() {
let tmp = tempfile::tempdir().unwrap();
let theme = tmp.path().join("DeepSea.theme.json");
write(&theme, DEEP.as_bytes());
write(&tmp.path().join("DeepSea.xml"), SCHEME.as_bytes());
let source = read_source(&theme).unwrap();
assert_eq!(source.name, "Deep Sea");
assert_eq!(source.themes.len(), 1);
assert_eq!(source.themes[0].id, "DeepSea");
assert_eq!(source.themes[0].editor_scheme.as_deref(), Some(SCHEME));
std::fs::remove_file(tmp.path().join("DeepSea.xml")).unwrap();
assert_eq!(read_source(&theme).unwrap().themes[0].editor_scheme, None);
}
#[test]
fn says_what_is_wrong_with_other_files() {
let tmp = tempfile::tempdir().unwrap();
let not_zip = tmp.path().join("notes.jar");
write(&not_zip, b"hello");
assert!(read_source(&not_zip).unwrap_err().message.contains("not a jar or zip"));
let other_plugin = tmp.path().join("tools.jar");
let xml = "<idea-plugin><name>Tools</name><actions/></idea-plugin>";
write(&other_plugin, &jar_bytes(&[(PLUGIN_XML, xml.as_bytes())]));
assert!(read_source(&other_plugin).unwrap_err().message.contains("not a theme plugin"));
let package = tmp.path().join("package.zip");
write(&package, &jar_bytes(&[("plugin.toml", b"id = \"x\"")]));
assert!(read_source(&package).unwrap_err().message.contains("not an IntelliJ plugin"));
let other = tmp.path().join("package.json");
write(&other, br#"{ "name": "app", "version": "1.0.0" }"#);
assert!(read_source(&other).unwrap_err().message.contains("is not an IntelliJ theme"));
let broken = tmp.path().join("broken.theme.json");
write(&broken, b"{ \"name\": ");
assert!(read_source(&broken).unwrap_err().message.contains("invalid theme JSON"));
let missing = tmp.path().join("missing.jar");
let xml = r#"<idea-plugin><name>M</name><themeProvider id="m" path="/m.theme.json"/></idea-plugin>"#;
write(&missing, &jar_bytes(&[(PLUGIN_XML, xml.as_bytes())]));
assert!(read_source(&missing).unwrap_err().message.contains("/m.theme.json is missing"));
}
#[test]
fn finds_theme_providers_in_included_descriptors() {
let tmp = tempfile::tempdir().unwrap();
let jar = tmp.path().join("split.jar");
let main = r#"<idea-plugin xmlns:xi="http://www.w3.org/2001/XInclude"><name>Split</name>
<xi:include href="/META-INF/themes.xml"/></idea-plugin>"#;
let part = r#"<idea-plugin xmlns:xi="http://www.w3.org/2001/XInclude"><extensions defaultExtensionNs="com.intellij">
<themeProvider id="s" path="/s.theme.json"/></extensions><name>Not this</name>
<xi:include href="more.xml"/><xi:include href="/META-INF/themes.xml"/></idea-plugin>"#;
let more = r#"<idea-plugin><themeProvider id="m" path="/m.theme.json"/></idea-plugin>"#;
let unrelated = r#"<idea-plugin><themeProvider id="x" path="/x.theme.json"/></idea-plugin>"#;
write(
&jar,
&jar_bytes(&[
("META-INF/themes.xml", part.as_bytes()),
("META-INF/more.xml", more.as_bytes()),
("META-INF/unrelated.xml", unrelated.as_bytes()),
(PLUGIN_XML, main.as_bytes()),
("s.theme.json", br#"{"name":"S"}"#),
("m.theme.json", br#"{"name":"M"}"#),
("x.theme.json", br#"{"name":"X"}"#),
]),
);
let source = read_source(&jar).unwrap();
assert_eq!(source.name, "Split");
// Only what plugin.xml includes (relative or from the root, once each) counts.
assert_eq!(source.themes.iter().map(|t| t.id.as_str()).collect::<Vec<_>>(), ["s", "m"]);
}
#[test]
fn parses_ide_folder_names() {
assert_eq!(product_dir("IntelliJIdea2025.2"), Some(("IntelliJIdea", (2025, 2), "2025.2")));
assert_eq!(product_dir("PyCharmCE2023.10"), Some(("PyCharmCE", (2023, 10), "2023.10")));
assert_eq!(product_dir("AndroidStudio2024.3"), Some(("AndroidStudio", (2024, 3), "2024.3")));
for name in
["Toolbox", "consentOptions", "IntelliJIdea2025", "Idea-2025.1", "2025.1", "Rider25.1", "Go Land2025.1"]
{
assert_eq!(product_dir(name), None, "{name}");
}
}
#[test]
fn locates_the_settings_and_plugins_of_each_os() {
let tmp = tempfile::tempdir().unwrap();
let home = tmp.path();
let mk = |p: PathBuf| {
std::fs::create_dir_all(&p).unwrap();
p
};
let mac = mk(home.join("Library/Application Support/JetBrains/IntelliJIdea2025.2"));
mk(home.join("Library/Application Support/JetBrains/Toolbox"));
let dirs = ide_dirs(Layout::Mac, home, |_| None);
assert_eq!(dirs.len(), 1);
assert_eq!(dirs[0].label, "IntelliJ IDEA 2025.2");
assert_eq!((dirs[0].config.clone(), dirs[0].plugins.clone()), (mac.clone(), mac.join("plugins")));
let appdata = mk(home.join("Roaming"));
let win = mk(appdata.join("JetBrains/PyCharm2024.3"));
let studio = mk(appdata.join("Google/AndroidStudio2025.1"));
let env = |k: &str| (k == "APPDATA").then(|| appdata.clone().into_os_string());
let dirs = ide_dirs(Layout::Windows, home, env);
assert_eq!(
dirs.iter().map(|d| d.label.as_str()).collect::<Vec<_>>(),
["Android Studio 2025.1", "PyCharm 2024.3"]
);
assert_eq!(dirs[1].plugins, win.join("plugins"));
assert_eq!(dirs[0].config, studio);
let config = mk(home.join(".config/JetBrains/IdeaIC2024.1"));
mk(home.join(".config/JetBrains/GoLand2025.2"));
let dirs = ide_dirs(Layout::Xdg, home, |_| None);
assert_eq!(dirs.iter().map(|d| d.product.as_str()).collect::<Vec<_>>(), ["GoLand", "IdeaIC"]);
assert_eq!(dirs[1].label, "IntelliJ IDEA Community Edition 2024.1");
assert_eq!(dirs[1].config, config);
assert_eq!(dirs[1].plugins, home.join(".local/share/JetBrains/IdeaIC2024.1"));
let xdg_data = home.join("data");
let env = |k: &str| (k == "XDG_DATA_HOME").then(|| xdg_data.clone().into_os_string());
assert_eq!(ide_dirs(Layout::Xdg, home, env)[1].plugins, xdg_data.join("JetBrains/IdeaIC2024.1"));
}
#[test]
fn reads_the_selected_theme_from_laf_xml() {
let tmp = tempfile::tempdir().unwrap();
let laf = tmp.path().join("options/laf.xml");
write(
&laf,
br#"<application>
<component name="LafManager">
<laf class-name="com.intellij.ide.ui.laf.UIThemeBasedLookAndFeelInfo" themeId="deep-sea-id" />
<preferred-light-laf themeId="shallow-id" />
</component>
</application>"#,
);
assert_eq!(active_theme_ids(tmp.path()), ["deep-sea-id"]);
let synced = std::fs::read_to_string(&laf)
.unwrap()
.replace(r#"name="LafManager""#, r#"name="LafManager" autodetect="true""#);
write(&laf, synced.as_bytes());
assert_eq!(active_theme_ids(tmp.path()), ["deep-sea-id", "shallow-id"]);
assert!(active_theme_ids(&tmp.path().join("missing")).is_empty());
}
#[test]
fn lists_installed_theme_plugins_the_ides_current_first() {
let tmp = tempfile::tempdir().unwrap();
let config = tmp.path().join("IntelliJIdea2025.2");
let plugins = config.join("plugins");
write(&plugins.join("DeepSea/lib/deep-sea.jar"), &theme_jar());
write(
&plugins.join("Tools/lib/tools.jar"),
&jar_bytes(&[(PLUGIN_XML, b"<idea-plugin><name>T</name></idea-plugin>")]),
);
write(&plugins.join("broken.jar"), b"not a jar");
write(
&config.join("options/laf.xml"),
br#"<application><component name="LafManager"><laf themeId="shallow-id"/></component></application>"#,
);
let ide = IdeDir {
label: "IntelliJ IDEA 2025.2".into(),
product: "IntelliJIdea".into(),
version: (2025, 2),
config: config.clone(),
plugins: plugins.clone(),
};
let found = find_installed(&[ide]);
let theme = |id: &str, name: &str, dark: bool, active: bool| InstalledIntellijTheme {
ide: "IntelliJ IDEA 2025.2".into(),
plugin: "Deep Sea & Friends".into(),
path: plugins.join("DeepSea").into(),
theme_id: id.into(),
theme_name: name.into(),
dark,
active,
};
assert_eq!(found, [theme("shallow-id", "Shallow", false, true), theme("deep-sea-id", "Deep Sea", true, false)]);
assert!(find_installed(&[]).is_empty());
}
}
+463
View File
@@ -0,0 +1,463 @@
//! The marketplace: a static index (`index.json` +
//! packages, served from any URL or folder) lists publishers and their keys,
//! plugins, and revocations. The host keeps the last index it read
//! (`market-index.json`), checks signatures against it, and disables what it
//! revokes — at start-up and on every refresh (the desktop refreshes daily).
//! Release builds pin the official publisher to the keys they were built
//! with, so no index can sign `openidea.*` plugins with another key.
use super::{PackageFile, PluginEvent, PluginHost, PluginSource, PluginStatus, PluginUpdate, install};
use ide_core::{IdeError, IdeResult};
use ide_extension::package::{self, Index, IndexEntry, PackageSignature, SIGNATURE_FILE};
use ide_extension::{MANIFEST_FILE, Manifest};
use semver::Version;
use serde::{Deserialize, Serialize};
use std::path::{Path, PathBuf};
/// The official marketplace: the index the release workflow publishes with
/// the app, next to the signed packages of the official plugins.
pub const OFFICIAL_INDEX: &str = "https://github.com/l0ng-ai/openidea/releases/latest/download/plugins-index.json";
/// The publisher of the official plugins.
pub const OFFICIAL_PUBLISHER: &str = "openidea";
/// The last index read, in the plugins directory.
pub const INDEX_FILE: &str = "market-index.json";
const DOWNLOADS: &str = ".downloads";
const MAX_INDEX: usize = 16 << 20;
const MAX_PACKAGE: usize = 256 << 20;
const MAX_STAGED_FILES: usize = 256;
const MAX_STAGED_BYTES: usize = 16 << 20;
/// `moved` links followed before giving up (a loop, or a misconfiguration).
const MAX_MOVES: usize = 3;
/// A marketplace entry with what is installed.
#[derive(Clone, Debug, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct MarketPlugin {
#[serde(flatten)]
pub entry: IndexEntry,
/// Installed version, if any.
pub installed: Option<String>,
/// Whether this IDE can load `entry` (the newest version it can, else the newest).
pub compatible: bool,
}
impl MarketPlugin {
/// A version this IDE can load, newer than the one installed.
pub fn is_update(&self) -> bool {
let newer = |installed: &String| match (Version::parse(&self.entry.version), Version::parse(installed)) {
(Ok(offered), Ok(installed)) => offered > installed,
_ => false,
};
self.compatible && self.installed.as_ref().is_some_and(newer)
}
}
/// The official publisher's keys this IDE was built with
/// (`OPENIDEA_OFFICIAL_PLUGIN_KEYS`: base64, comma-separated). Development
/// builds have none: the index decides.
pub(super) fn built_in_official_keys() -> Vec<String> {
let keys = option_env!("OPENIDEA_OFFICIAL_PLUGIN_KEYS").unwrap_or_default();
keys.split(',').map(str::trim).filter(|k| !k.is_empty()).map(str::to_owned).collect()
}
fn pinned(mut index: Index, official_keys: &[String]) -> Index {
index.pin_publisher(OFFICIAL_PUBLISHER, "OpenIDEA", official_keys);
index
}
/// Reads `url`: `http(s)://…`, `file://…` or a local path.
pub fn read_url(url: &str, max: usize) -> Result<Vec<u8>, String> {
if url.starts_with("https://") || url.starts_with("http://") {
let client = ide_http::client::shared().map_err(|e| e.message)?;
let url = url.to_owned();
return super::api::net_runtime().block_on(async move {
let response = client.get(&url).send().await.map_err(|e| format!("{url}: {e}"))?;
if !response.status().is_success() {
return Err(format!("{url}: HTTP {}", response.status()));
}
let bytes = response.bytes().await.map_err(|e| format!("{url}: {e}"))?;
if bytes.len() > max {
return Err(format!("{url}: larger than {} MiB", max >> 20));
}
Ok(bytes.to_vec())
});
}
let path = url.strip_prefix("file://").unwrap_or(url);
let bytes = std::fs::read(path).map_err(|e| format!("{path}: {e}"))?;
if bytes.len() > max {
return Err(format!("{path}: larger than {} MiB", max >> 20));
}
Ok(bytes)
}
/// The index at `url`, following `moved`, with downloads made absolute against
/// where it was read from.
fn read_index(url: &str) -> IdeResult<Index> {
let mut url = url.to_owned();
for _ in 0..=MAX_MOVES {
let bytes = read_url(&url, MAX_INDEX).map_err(IdeError::unavailable)?;
let text = String::from_utf8(bytes).map_err(|_| IdeError::invalid("the index is not UTF-8"))?;
let mut index = Index::parse(&text).map_err(IdeError::invalid)?;
match index.moved.take().filter(|m| !m.trim().is_empty()) {
Some(next) => {
let next = resolve(&url, &next);
if url.starts_with("https://") && !next.starts_with("https://") {
return Err(IdeError::invalid(format!("the marketplace moved to {next}, which isn't https")));
}
url = next;
}
None => {
for e in &mut index.plugins {
e.download = resolve(&url, &e.download);
}
return Ok(index);
}
}
}
Err(IdeError::unavailable(format!("the marketplace index moved more than {MAX_MOVES} times")))
}
/// `href` relative to the index at `base`: a URL, or a local path (with the
/// platform's separators, e.g. `C:\\market\\index.json`).
pub fn resolve(base: &str, href: &str) -> String {
if href.contains("://") || href.starts_with('/') || Path::new(href).is_absolute() {
return href.to_owned();
}
if base.contains("://") {
return match base.rfind('/') {
Some(i) => format!("{}/{href}", &base[..i]),
None => href.to_owned(),
};
}
match Path::new(base).parent() {
Some(dir) => dir.join(href).to_string_lossy().into_owned(),
None => href.to_owned(),
}
}
impl PluginHost {
pub(super) fn load_market_index(dir: &Path, official_keys: &[String]) -> Option<Index> {
let text = std::fs::read_to_string(dir.join(INDEX_FILE)).ok()?;
let index = Index::parse(&text).map_err(|e| tracing::warn!("{INDEX_FILE}: {e}")).ok()?;
Some(pinned(index, official_keys))
}
/// Pins the official publisher to `keys` (base64) instead of the built-in
/// ones, from the next index read.
pub fn set_official_keys(&self, keys: Vec<String>) {
*self.official_keys.write() = keys;
}
/// The marketplace index URL: the user's, else the default (none: no marketplace).
pub fn market_url(&self) -> Option<String> {
self.state.lock().market_url.clone().or_else(|| self.default_market_url.read().clone())
}
/// The marketplace used while the user has set none.
pub fn set_default_market_url(&self, url: Option<String>) {
*self.default_market_url.write() = url.filter(|u| !u.trim().is_empty());
}
pub fn set_market_url(&self, url: Option<String>) {
self.state.lock().market_url = url.filter(|u| !u.trim().is_empty());
self.save_state();
}
/// Reads the index again, keeps it, and applies its revocations. Announces
/// updates of installed plugins it hasn't announced yet.
pub fn refresh_market(&self) -> IdeResult<Vec<MarketPlugin>> {
let url = self.market_url().ok_or_else(|| IdeError::unavailable("no plugin marketplace is configured"))?;
let index = read_index(&url)?;
if let Ok(json) = serde_json::to_string(&index) {
let _ = std::fs::write(self.plugins_dir.join(INDEX_FILE), json);
}
let newer = index.is_newer().then_some(index.format);
*self.market.write() = Some(pinned(index, &self.official_keys.read()));
// Trust may have changed: who signed what, what is revoked.
let installed: Vec<_> =
self.plugins.read().values().filter(|p| p.source == PluginSource::Installed).cloned().collect();
for plugin in installed {
let publisher = self.verified_publisher(plugin.dir());
plugin.meta.lock().publisher = publisher;
}
self.apply_revocations();
if let Some(format) = newer {
return Err(IdeError::unavailable(format!(
"the marketplace needs a newer OpenIDEA (index format {format}); its revocations still apply"
)));
}
let plugins = self.market_plugins();
self.announce_updates(&plugins);
Ok(plugins)
}
/// Tells the windows about updates once per set of versions: a later
/// refresh offering the same ones says nothing.
fn announce_updates(&self, plugins: &[MarketPlugin]) {
let updates: Vec<PluginUpdate> = plugins
.iter()
.filter(|p| p.is_update())
.map(|p| PluginUpdate {
id: p.entry.id.clone(),
name: p.entry.name.clone(),
version: p.entry.version.clone(),
})
.collect();
let key: Vec<String> = updates.iter().map(|u| format!("{}@{}", u.id, u.version)).collect();
if std::mem::replace(&mut *self.announced_updates.lock(), key.clone()) != key && !updates.is_empty() {
self.emit(None, PluginEvent::UpdatesAvailable { updates });
}
}
/// Whether the package in `src` would replace the installed `id`'s copy
/// with one signed by another key than that copy, without that key's
/// endorsement: the user confirms the change of signer.
pub(super) fn signer_changed(&self, id: &str, src: &Path) -> bool {
let Some(current) = install::active_dir(&self.plugins_dir.join(id)) else { return false };
let market = self.market.read();
!package::signer_continues(market.as_ref(), read_signature(&current).as_ref(), read_signature(src).as_ref())
}
/// The plugins of the last index read.
pub fn market_plugins(&self) -> Vec<MarketPlugin> {
let Some(index) = self.market.read().clone() else { return Vec::new() };
let plugins = self.plugins.read();
index
.offered()
.into_iter()
.map(|(entry, compatible)| {
let installed = plugins
.get(&entry.id)
.filter(|p| p.source == PluginSource::Installed)
.map(|p| p.manifest().version.clone());
MarketPlugin { entry: entry.clone(), installed, compatible }
})
.collect()
}
/// Downloads a marketplace plugin, checks it against the index (digest and
/// signature), and unpacks it for the install page. Returns the folder to
/// install from.
pub fn download_market_plugin(&self, id: &str) -> IdeResult<PathBuf> {
let url = self.market_url().ok_or_else(|| IdeError::unavailable("no plugin marketplace is configured"))?;
let index = self.market.read().clone().ok_or_else(|| IdeError::unavailable("refresh the marketplace first"))?;
let (entry, compatible) = index
.offered()
.into_iter()
.find(|(e, _)| e.id == id)
.map(|(e, ok)| (e.clone(), ok))
.ok_or_else(|| IdeError::not_found(format!("{id} is not in the marketplace")))?;
if !compatible {
return Err(IdeError::unavailable(format!(
"{} {} needs plugin API {}: update OpenIDEA",
entry.name, entry.version, entry.engine
)));
}
let bytes = read_url(&resolve(&url, &entry.download), MAX_PACKAGE).map_err(IdeError::unavailable)?;
let downloads = self.plugins_dir.join(DOWNLOADS);
std::fs::create_dir_all(&downloads)?;
let archive = downloads.join(format!("{}-{}.{}", entry.id, entry.version, package::PACKAGE_EXTENSION));
std::fs::write(&archive, bytes)?;
let dir = self.unpack_package(&archive)?;
let _ = std::fs::remove_file(&archive);
let manifest = Manifest::load(&dir).map_err(IdeError::invalid)?;
let files = self.package_files_in(&dir)?;
let digest = package::package_digest(&dir, &files)?;
if (manifest.id.as_str(), manifest.version.as_str(), digest.as_str())
!= (&entry.id, &entry.version, &entry.digest)
{
let _ = std::fs::remove_dir_all(&dir);
return Err(IdeError::invalid("the download does not match the marketplace's entry"));
}
if self.verified_publisher(&dir).is_none() {
let _ = std::fs::remove_dir_all(&dir);
return Err(IdeError::invalid("the marketplace package is not signed by a trusted publisher"));
}
Ok(dir)
}
/// Unpacks a package file (`.oiplugin`, or a `.zip` of a package) next to the downloads; returns the folder.
pub fn unpack_package(&self, archive: &Path) -> IdeResult<PathBuf> {
let stem = archive.file_stem().and_then(|s| s.to_str()).unwrap_or("package");
let dir = self.plugins_dir.join(DOWNLOADS).join(format!("{stem}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
let unpacked = package::unpack(archive, &dir).map_err(IdeError::invalid)?;
// Kept with the files: install checks it and keeps it with the installed copy.
if let Some(sig) = unpacked.signature {
let json = serde_json::to_string_pretty(&sig).map_err(|e| IdeError::internal(e.to_string()))?;
std::fs::write(dir.join(SIGNATURE_FILE), json)?;
}
Ok(dir)
}
/// Writes a package the UI generated (an imported IntelliJ theme) into a
/// folder next to the downloads, named after `name` (its plugin id); returns the folder.
/// The install page validates it like any other package.
pub fn stage_package(&self, name: &str, files: &[PackageFile]) -> IdeResult<PathBuf> {
ide_extension::validate_item_id(name, "package name").map_err(IdeError::invalid)?;
if !files.iter().any(|f| f.path == MANIFEST_FILE) {
return Err(IdeError::invalid(format!("a package needs a {MANIFEST_FILE}")));
}
let total: usize = files.iter().map(|f| f.text.len()).sum();
if files.len() > MAX_STAGED_FILES || total > MAX_STAGED_BYTES {
return Err(IdeError::invalid("the package is too large"));
}
let mut seen = std::collections::HashSet::new();
for f in files {
let valid = !f.path.is_empty()
&& f.path.len() <= 255
&& f.path.split('/').all(|part| {
!matches!(part, "" | "." | "..") && !part.contains(['\\', ':', '\0']) && !part.starts_with('.')
});
if !valid {
return Err(IdeError::invalid(format!("invalid package path {:?}", f.path)));
}
if !seen.insert(f.path.to_ascii_lowercase()) {
return Err(IdeError::invalid(format!("{} is in the package twice", f.path)));
}
}
// One folder per package, replaced by the next import of it: they don't pile up.
let dir = self.plugins_dir.join(DOWNLOADS).join(format!("{name}-generated"));
let _ = std::fs::remove_dir_all(&dir);
for f in files {
let path = dir.join(&f.path);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(&path, &f.text)?;
}
Ok(dir)
}
/// Files of the package in `dir` (what its declarations reference, and
/// the component): what a package file holds besides the manifest.
pub fn package_files_of(&self, dir: &Path) -> IdeResult<Vec<PathBuf>> {
self.package_files_in(dir)
}
pub(super) fn package_files_in(&self, dir: &Path) -> IdeResult<Vec<PathBuf>> {
let manifest = Manifest::load(dir).map_err(IdeError::invalid)?;
let pkg = ide_extension::Package::new(manifest, dir.to_path_buf());
let set = self.registry.prepare(&pkg).map_err(IdeError::invalid)?;
Ok(set.files.into_iter().chain(pkg.manifest.wasm.clone()).collect())
}
/// The publisher whose valid, trusted signature the package in `dir` carries.
pub(super) fn verified_publisher(&self, dir: &Path) -> Option<String> {
let sig = read_signature(dir)?;
let index = self.market.read().clone()?;
let manifest = Manifest::load(dir).ok()?;
let files = self.package_files_in(dir).ok()?;
let digest = package::package_digest(dir, &files).ok()?;
index
.verify(&sig, &manifest.id, &manifest.version, &digest)
.map_err(|e| tracing::warn!("{}: {e}", dir.display()))
.ok()
}
/// A signature that is present but doesn't verify (tampered, foreign prefix,
/// revoked or unknown key): installing is refused.
pub(super) fn signature_problem(&self, dir: &Path) -> Option<String> {
let sig = read_signature(dir)?;
let manifest = Manifest::load(dir).ok()?;
let files = self.package_files_in(dir).ok()?;
let digest = package::package_digest(dir, &files).ok()?;
if let Err(e) = sig.check(&manifest.id, &manifest.version, &digest) {
return Some(e);
}
let index = self.market.read().clone()?;
index.verify(&sig, &manifest.id, &manifest.version, &digest).err()
}
/// Why an installed plugin must not run (index revocations, revoked signing key).
pub(super) fn revoked_reason(&self, id: &str, version: &str, dir: &Path) -> Option<String> {
let market = self.market.read();
let index = market.as_ref()?;
if let Some(r) = index.revocation(id, version) {
return Some(r.reason.clone());
}
let sig = read_signature(dir)?;
index.key_revoked(&sig.key).then(|| "its publisher's signing key was revoked".to_owned())
}
/// Disables installed plugins the index revokes (telling the user once),
/// and starts again those it no longer revokes.
pub(super) fn apply_revocations(&self) {
let installed: Vec<_> =
self.plugins.read().values().filter(|p| p.source == PluginSource::Installed).cloned().collect();
let mut changed = false;
for plugin in installed {
let reason = self.revoked_reason(plugin.id(), &plugin.manifest().version, plugin.dir());
let current = plugin.meta.lock().error.clone().filter(|e| e.starts_with("revoked: "));
match (reason, current) {
(Some(r), Some(c)) if c == format!("revoked: {r}") => {}
(Some(reason), was) => {
self.stop(&plugin);
{
let mut meta = plugin.meta.lock();
meta.status = PluginStatus::Failed;
meta.error = Some(format!("revoked: {reason}"));
}
if was.is_none() {
self.emit(
None,
super::PluginEvent::Notification {
plugin_id: plugin.id().to_owned(),
level: ide_plugin::NotificationLevel::Warning,
title: format!("{} was disabled", plugin.manifest().name),
body: format!("The marketplace revoked this plugin: {reason}"),
},
);
}
changed = true;
}
(None, Some(_)) => {
{
let mut meta = plugin.meta.lock();
meta.status = PluginStatus::Inactive;
meta.error = None;
}
if !self.state.lock().disabled.contains(plugin.id()) {
self.start(&plugin);
}
changed = true;
}
(None, None) => {}
}
}
if changed {
self.changed();
}
}
}
pub(super) fn read_signature(dir: &Path) -> Option<PackageSignature> {
let text = std::fs::read_to_string(dir.join(SIGNATURE_FILE)).ok()?;
serde_json::from_str(&text).ok()
}
/// Keeps the signature of a package with its installed copy.
pub(super) fn copy_signature(src: &Path, installed: &Path) {
if src.join(SIGNATURE_FILE).is_file() {
let _ = std::fs::copy(src.join(SIGNATURE_FILE), installed.join(SIGNATURE_FILE));
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn downloads_resolve_against_the_index() {
assert_eq!(resolve("https://x.dev/market/index.json", "p/a.oiplugin"), "https://x.dev/market/p/a.oiplugin");
assert_eq!(
Path::new(&resolve("/srv/market/index.json", "a.oiplugin")),
Path::new("/srv/market").join("a.oiplugin")
);
#[cfg(windows)]
assert_eq!(resolve(r"C:\market\index.json", "a.oiplugin"), r"C:\market\a.oiplugin");
assert_eq!(resolve("https://x.dev/index.json", "https://cdn.dev/a"), "https://cdn.dev/a");
}
}
File diff suppressed because it is too large. Load diff
+71
View File
@@ -0,0 +1,71 @@
//! First-party extension points registered by the host, plus
//! [`register_builtin_points`], the one place that lists them. Feature crates
//! own their points (`ide_language::plugin_point`); adding a contribution type
//! means adding a point here or in its crate, and a `defineUiPoint` projection
//! on the client for `Ui` points — nothing else.
use ide_extension::{ExtensionRegistry, Package, UiDecl, UiPoint, UiStore, decode, validate_item_id, validate_when};
use ide_language::plugin_point::{LanguagesPoint, PluginLanguages};
use serde::{Deserialize, Serialize};
use std::sync::Arc;
pub const COMMANDS: &str = "commands";
const MAX_TITLE: usize = 256;
/// `[[contributes.commands]]`: an action `plugin.<plugin-id>.<id>` in Search
/// Everywhere and the keymap editor; shown before the plugin's code runs.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct CommandDecl {
pub id: String,
pub title: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub category: Option<String>,
/// Enablement, evaluated by the client against context keys (`editorLang == rust`).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub when: Option<String>,
}
fn command(_pkg: &Package, decl: toml::Value) -> Result<UiDecl, String> {
let c: CommandDecl = decode(COMMANDS, decl)?;
validate_item_id(&c.id, "command id")?;
if c.title.trim().is_empty() {
return Err(format!("command {:?} has an empty title", c.id));
}
if c.title.len() > MAX_TITLE || c.category.as_ref().is_some_and(|x| x.len() > MAX_TITLE) {
return Err(format!("command {:?}: title or category longer than {MAX_TITLE} bytes", c.id));
}
if let Some(when) = &c.when {
validate_when(when).map_err(|e| format!("command {:?}: {e}", c.id))?;
}
let c = CommandDecl { category: c.category.filter(|x| !x.trim().is_empty()), ..c };
let json = serde_json::to_value(&c).map_err(|e| e.to_string())?;
Ok(UiDecl::new(c.id.clone(), c.title, json))
}
/// Registers every first-party extension point.
pub fn register_builtin_points(
registry: &ExtensionRegistry,
ui: &Arc<UiStore>,
languages: &Arc<PluginLanguages>,
run_detectors: &Arc<super::run_detectors::RunDetectors>,
providers: &Arc<super::providers::Providers>,
) -> Result<(), String> {
registry.register(Arc::new(UiPoint::new(COMMANDS, ui.clone(), command)))?;
registry.register(Arc::new(super::theme::ThemesPoint::new(ui.clone())))?;
registry.register(Arc::new(LanguagesPoint::new(languages.clone())))?;
registry.register(Arc::new(UiPoint::new(super::icons::POINT_ID, ui.clone(), super::icons::icon_theme)))?;
registry.register(Arc::new(super::run_detectors::RunDetectorsPoint::new(run_detectors.clone())))?;
registry.register(Arc::new(super::providers::ProvidersPoint::new(providers.clone())))?;
registry.register(Arc::new(UiPoint::new(super::providers::SECRETS_POINT, ui.clone(), super::providers::secret)))?;
use super::ui_points as p;
registry.register(Arc::new(UiPoint::new(p::MENUS, ui.clone(), p::menu)))?;
registry.register(Arc::new(UiPoint::new(p::KEYMAPS, ui.clone(), p::keymap)))?;
registry.register(Arc::new(UiPoint::new(p::STATUS_BAR, ui.clone(), p::status_item)))?;
registry.register(Arc::new(UiPoint::new(p::SETTINGS, ui.clone(), p::settings)))?;
registry.register(Arc::new(UiPoint::new(p::VIEWS, ui.clone(), p::view)))?;
registry.register(Arc::new(UiPoint::new(p::CUSTOM_EDITORS, ui.clone(), p::custom_editor)))?;
registry.register(Arc::new(UiPoint::new(p::FILE_ICONS, ui.clone(), p::file_icon)))?;
registry.register(Arc::new(UiPoint::new(p::FILE_TEMPLATES, ui.clone(), p::file_template)))?;
Ok(())
}
+342
View File
@@ -0,0 +1,342 @@
//! Editor providers of plugins: completion, inline
//! completion, code actions, go to definition and gutter line markers, for
//! the languages a plugin declares. The host calls only plugins whose
//! declaration matches, so the others stay unloaded.
//!
//! ```toml
//! [[contributes.providers]]
//! kinds = ["definition", "line-markers"]
//! languages = ["java", "xml"] # editor language ids; "*" for any
//! ```
//!
//! And `[[contributes.secrets]]`: secrets the user can store for the plugin
//! (shown in Settings › Plugins), used only through `net` request headers.
//!
//! ```toml
//! [[contributes.secrets]]
//! name = "api-key"
//! label = "API key"
//! description = "Create one at https://example.com/keys"
//! ```
use ide_extension::{Disposable, ExtensionPoint, Package, Placement, Prepared, UiDecl, decode, validate_item_id};
use parking_lot::RwLock;
use serde::{Deserialize, Serialize};
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};
pub const POINT_ID: &str = "providers";
pub const SECRETS_POINT: &str = "secrets";
/// What a plugin can provide in the editor.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "kebab-case")]
pub enum ProviderKind {
Completion,
InlineCompletion,
CodeActions,
Definition,
LineMarkers,
}
#[derive(Clone, Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct Decl {
kinds: Vec<ProviderKind>,
languages: Vec<String>,
}
/// (sequence, plugin, kinds, languages) of one declaration.
type Declared = (u64, String, Vec<ProviderKind>, Vec<String>);
/// Provider declarations of enabled plugins.
#[derive(Default)]
pub struct Providers {
items: RwLock<Vec<Declared>>,
seq: AtomicU64,
}
impl Providers {
fn add(self: &Arc<Self>, plugin: String, decl: Decl) -> Disposable {
let seq = self.seq.fetch_add(1, Ordering::Relaxed);
self.items.write().push((seq, plugin, decl.kinds, decl.languages));
let weak = Arc::downgrade(self);
Disposable::new(move || {
if let Some(this) = weak.upgrade() {
this.items.write().retain(|(s, ..)| *s != seq);
}
})
}
/// Plugins providing `kind` for `language`, in declaration order.
pub fn plugins_for(&self, kind: ProviderKind, language: &str) -> Vec<String> {
let mut out: Vec<String> = Vec::new();
for (_, plugin, kinds, languages) in self.items.read().iter() {
if kinds.contains(&kind) && languages.iter().any(|l| l == "*" || l == language) && !out.contains(plugin) {
out.push(plugin.clone());
}
}
out
}
/// Whether any plugin provides `kind` (for any language).
pub fn any(&self, kind: ProviderKind) -> bool {
self.items.read().iter().any(|(_, _, kinds, _)| kinds.contains(&kind))
}
}
pub struct ProvidersPoint {
store: Arc<Providers>,
}
impl ProvidersPoint {
pub fn new(store: Arc<Providers>) -> Self {
Self { store }
}
}
impl ExtensionPoint for ProvidersPoint {
fn id(&self) -> &'static str {
POINT_ID
}
fn placement(&self) -> Placement {
Placement::Workspace
}
fn prepare(&self, pkg: &Package, decl: toml::Value) -> Result<Prepared, String> {
let d: Decl = decode(POINT_ID, decl)?;
if pkg.manifest.wasm.is_none() {
return Err("contributes.providers: providers need code (`wasm`)".into());
}
if d.kinds.is_empty() || d.languages.is_empty() || d.languages.len() > 64 {
return Err("contributes.providers: give `kinds` and 1–64 `languages`".into());
}
for l in &d.languages {
if l != "*"
&& (l.is_empty()
|| l.len() > 64
|| !l.chars().all(|c| c.is_ascii_alphanumeric() || "-_.+#".contains(c)))
{
return Err(format!("contributes.providers: invalid language id {l:?}"));
}
}
let label = format!(
"{} for {}",
d.kinds
.iter()
.map(|k| serde_json::to_value(k).ok().and_then(|v| v.as_str().map(str::to_owned)).unwrap_or_default())
.collect::<Vec<_>>()
.join(", "),
d.languages.join(", ")
);
let (store, plugin) = (self.store.clone(), pkg.id().to_owned());
Ok(Prepared::new(label, move || Ok(store.add(plugin, d))))
}
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct SecretDecl {
name: String,
label: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
description: Option<String>,
}
/// `[[contributes.secrets]]` (a UI point: Settings › Plugins lists them).
pub(super) fn secret(_pkg: &Package, decl: toml::Value) -> Result<UiDecl, String> {
let d: SecretDecl = decode(SECRETS_POINT, decl)?;
validate_item_id(&d.name, "secret name")?;
if d.label.trim().is_empty() || d.label.len() > 128 || d.description.as_ref().is_some_and(|x| x.len() > 1024) {
return Err(format!("secret {:?}: `label` 1–128 bytes, `description` up to 1 KiB", d.name));
}
let json = serde_json::to_value(&d).map_err(|e| e.to_string())?;
Ok(UiDecl::new(d.name.clone(), d.label, json))
}
// ---- dispatch ---------------------------------------------------------------
use super::{LoadedPlugin, PluginHost, PluginStatus};
use ide_core::CancellationToken;
use ide_plugin::{
ActorError, CallError, CodeAction, CompletionItem, InlineSuggestion, LineMarker, Location, Pending,
ProviderDocument, Runtime,
};
use std::sync::Weak;
use std::time::{Duration, Instant};
/// How long a superseded inline completion may take to return after its
/// request was cancelled before it is interrupted.
const CANCEL_GRACE: Duration = Duration::from_millis(250);
static EDITOR_PROVIDERS: RwLock<Option<Weak<PluginHost>>> = RwLock::new(None);
/// Makes `host`'s plugin providers part of completion, code actions, go to
/// definition and inline completion (`None` removes them).
pub fn set_editor_providers(host: Option<&Arc<PluginHost>>) {
*EDITOR_PROVIDERS.write() = host.map(Arc::downgrade);
}
/// The plugin host whose providers take part, when any declares `kind`.
pub(crate) fn editor_providers(kind: ProviderKind) -> Option<Arc<PluginHost>> {
let host = EDITOR_PROVIDERS.read().as_ref()?.upgrade()?;
host.providers.any(kind).then_some(host)
}
impl PluginHost {
/// Usable plugins declaring `kind` for `language`.
fn provider_plugins(&self, kind: ProviderKind, language: &str) -> Vec<Arc<LoadedPlugin>> {
self.providers
.plugins_for(kind, language)
.into_iter()
.filter_map(|id| self.get(&id).ok())
.filter(|p| {
let m = p.meta.lock();
matches!(m.status, PluginStatus::Active | PluginStatus::Inactive) && m.pending.is_empty()
})
.collect()
}
/// Edit-path calls (soft / hard deadline) to every usable plugin that
/// declares `kind` for `language`, in plugin order, leaving out those that
/// failed or were late. Each plugin runs on its own thread, so the calls
/// run at the same time: the wait is the slowest plugin's, not the sum.
fn edit_calls<R: Send + 'static>(
&self,
kind: ProviderKind,
language: &str,
f: impl FnOnce(&mut Runtime, Duration) -> Result<R, CallError> + Clone + Send + 'static,
) -> Vec<(Arc<LoadedPlugin>, R)> {
let (soft, hard) = (self.limits.edit_soft, self.limits.edit_hard);
let queued: Vec<_> = self
.provider_plugins(kind, language)
.into_iter()
.filter_map(|plugin| {
let (actor, activated) = self.ensure_active(&plugin, None).ok()?;
if activated {
self.changed();
}
let f = f.clone();
let started = Instant::now();
let call = actor.submit(Some(soft), move |rt| f(rt, hard));
Some((plugin, actor, started, call))
})
.collect();
queued
.into_iter()
.filter_map(|(plugin, actor, started, call)| {
let result = call.and_then(Pending::wait);
self.finish_provider_call(&plugin, &actor, started, result).map(|r| (plugin, r))
})
.collect()
}
pub(super) fn finish_provider_call<R>(
&self,
plugin: &Arc<LoadedPlugin>,
actor: &Arc<ide_plugin::Actor>,
started: Instant,
result: Result<R, ActorError>,
) -> Option<R> {
let elapsed = started.elapsed();
match result {
Ok(r) => {
self.record_call(plugin, elapsed, true, true);
Some(r)
}
Err(ActorError::SoftTimeout) => {
plugin.meta.lock().stats.slow_calls += 1;
None
}
Err(ActorError::Cancelled) => None,
Err(ActorError::Call(CallError::Guest(e))) => {
self.record_call(plugin, elapsed, !e.is_failure(), true);
tracing::debug!(plugin = %plugin.id(), "provider: {e}");
None
}
Err(ActorError::Call(CallError::Trap(_)) | ActorError::Stopped) => {
self.record_call(plugin, elapsed, false, false);
self.reap(plugin, actor, None);
None
}
}
}
pub fn provide_completion(
&self,
doc: &ProviderDocument,
at: ide_core::Pos,
trigger: Option<&str>,
) -> Vec<(String, CompletionItem)> {
let (d, t) = (Arc::new(doc.clone()), trigger.map(str::to_owned));
self.edit_calls(ProviderKind::Completion, &doc.language, move |rt, timeout| {
rt.provide_completion(&d, at, t.as_deref(), timeout)
})
.into_iter()
.flat_map(|(p, items)| items.into_iter().map(move |i| (p.id().to_owned(), i)))
.collect()
}
pub fn provide_code_actions(
&self,
doc: &ProviderDocument,
selection: ide_core::Range,
) -> Vec<(String, CodeAction)> {
let d = Arc::new(doc.clone());
self.edit_calls(ProviderKind::CodeActions, &doc.language, move |rt, timeout| {
rt.provide_code_actions(&d, selection, timeout)
})
.into_iter()
.flat_map(|(p, actions)| actions.into_iter().map(move |a| (p.id().to_owned(), a)))
.collect()
}
pub fn provide_definition(&self, doc: &ProviderDocument, at: ide_core::Pos) -> Vec<Location> {
let d = Arc::new(doc.clone());
self.edit_calls(ProviderKind::Definition, &doc.language, move |rt, timeout| {
rt.provide_definition(&d, at, timeout)
})
.into_iter()
.flat_map(|(_, locations)| locations)
.collect()
}
pub fn provide_line_markers(&self, doc: &ProviderDocument) -> Vec<(String, LineMarker)> {
let d = Arc::new(doc.clone());
self.edit_calls(ProviderKind::LineMarkers, &doc.language, move |rt, timeout| {
rt.provide_line_markers(&d, timeout)
})
.into_iter()
.flat_map(|(p, markers)| markers.into_iter().map(move |m| (p.id().to_owned(), m)))
.collect()
}
/// Ghost text from the first plugin that has some. Not bound by the
/// edit-path soft deadline (it may need the network) but by `cancel`:
/// the editor cancels a request as soon as the user types on.
pub fn provide_inline_completion(
&self,
doc: &ProviderDocument,
at: ide_core::Pos,
cancel: &CancellationToken,
) -> Option<(String, InlineSuggestion)> {
for p in self.provider_plugins(ProviderKind::InlineCompletion, &doc.language) {
if cancel.is_cancelled() {
return None;
}
let Ok((actor, activated)) = self.ensure_active(&p, None) else { continue };
if activated {
self.changed();
}
let (d, timeout) = (doc.clone(), self.limits.timeout);
let started = Instant::now();
let result =
actor.call_cancellable(cancel, CANCEL_GRACE, move |rt| rt.provide_inline_completion(&d, at, timeout));
if let Some(Some(s)) = self.finish_provider_call(&p, &actor, started, result) {
return Some((p.id().to_owned(), s));
}
}
None
}
}
+222
View File
@@ -0,0 +1,222 @@
//! Plugin languages of a remote project: `Workspace` contributions take effect where the project is, so the
//! desktop pushes its plugins' language definitions to `openidea-server`
//! (`plugin_sync_languages`) together with the `process:` programs the user
//! granted. The server resolves each granted program on its own `PATH`, pins
//! the file (path + hash, kept in `plugin-pins.json`) the first time and
//! starts only that file afterwards; a program missing from a sync is revoked.
//! Revoking and granting again on the desktop re-pins a replaced program.
use ide_extension::Capability;
use ide_extension::ledger::{self, BinaryGrant};
use ide_language::plugin_point::{PluginLanguages, validate_definition};
use ide_language::{LanguageConfig, ProgramPin};
use parking_lot::Mutex;
use serde::{Deserialize, Serialize};
use std::collections::BTreeMap;
use std::path::PathBuf;
use std::sync::Arc;
pub const PINS_FILE: &str = "plugin-pins.json";
/// The server command the desktop calls.
pub const SYNC_COMMAND: &str = "plugin_sync_languages";
/// One plugin language as the desktop sends it.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct RemoteLanguage {
pub plugin: String,
pub language: LanguageConfig,
/// Program names of the plugin's granted `process:` capabilities.
#[serde(default)]
pub granted: Vec<String>,
}
/// What the server made of a sync.
#[derive(Clone, Debug, Default, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "camelCase")]
pub struct SyncReport {
pub languages: usize,
/// `plugin: program` → the file it starts.
pub pinned: BTreeMap<String, PathBuf>,
/// Granted programs not found on this machine's `PATH`.
pub unresolved: Vec<String>,
/// Definitions that failed validation (not applied).
pub rejected: Vec<String>,
}
#[derive(Default, Serialize, Deserialize)]
struct PinsFile {
/// `plugin\nprocess:program` → the approved file.
#[serde(default)]
pins: BTreeMap<String, BinaryGrant>,
}
type PathVar = Arc<dyn Fn() -> Option<std::ffi::OsString> + Send + Sync>;
/// The server's side: an overlay for its language registry, filled by syncs.
pub struct RemotePluginLanguages {
store: Arc<PluginLanguages>,
pins: Arc<Mutex<BTreeMap<String, BinaryGrant>>>,
file: Option<PathBuf>,
applied: Mutex<Vec<ide_extension::Disposable>>,
path_var: PathVar,
}
fn key(plugin: &str, cap: &Capability) -> String {
format!("{plugin}\n{cap}")
}
impl RemotePluginLanguages {
/// Pins persist in `data_dir` (none: in memory, for tests).
pub fn new(data_dir: Option<&std::path::Path>) -> Arc<Self> {
Self::with_path(data_dir, Arc::new(|| std::env::var_os("PATH")))
}
pub fn with_path(data_dir: Option<&std::path::Path>, path_var: PathVar) -> Arc<Self> {
let file = data_dir.map(|d| d.join(PINS_FILE));
let saved: PinsFile = file
.as_ref()
.and_then(|f| std::fs::read_to_string(f).ok())
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
let pins = Arc::new(Mutex::new(saved.pins));
let store = PluginLanguages::new();
let policy_pins = pins.clone();
store.set_process_policy(Arc::new(move |plugin, cap| {
policy_pins.lock().get(&key(plugin, cap)).map(|b| ProgramPin { path: b.path.clone(), hash: b.hash.clone() })
}));
Arc::new(Self { store, pins, file, applied: Mutex::default(), path_var })
}
/// For [`ide_language::LanguageRegistry::attach_plugins`].
pub fn store(&self) -> &Arc<PluginLanguages> {
&self.store
}
/// Replaces every plugin language with `languages`.
pub fn sync(&self, languages: Vec<RemoteLanguage>) -> SyncReport {
let mut report = SyncReport::default();
let mut wanted: BTreeMap<String, (String, Capability)> = BTreeMap::new();
let mut accepted = Vec::new();
for l in languages {
if let Err(e) = validate_definition(&l.language) {
report.rejected.push(format!("{} ({}): {e}", l.language.id, l.plugin));
continue;
}
for program in &l.granted {
let cap = Capability::Process(program.clone());
// Only programs the definition starts: a grant is not a licence for anything else.
if ide_language::plugin_point::required_capabilities(&l.language).contains(&cap) {
wanted.insert(key(&l.plugin, &cap), (program.clone(), cap));
}
}
accepted.push(l);
}
{
let mut pins = self.pins.lock();
pins.retain(|k, _| wanted.contains_key(k));
for (k, (program, _)) in &wanted {
if pins.contains_key(k) {
continue;
}
let resolved = ledger::resolve_program(program, (self.path_var)().as_deref()).and_then(|path| {
ledger::file_hash(&path).map(|hash| BinaryGrant { path, hash }).map_err(|e| e.to_string())
});
match resolved {
Ok(bin) => {
pins.insert(k.clone(), bin);
}
Err(e) => {
tracing::info!("process:{program}: {e}");
report.unresolved.push(program.clone());
}
}
}
for (k, bin) in pins.iter() {
report.pinned.insert(k.replace("\nprocess:", ": "), bin.path.clone());
}
self.save(&pins);
}
let mut applied = self.applied.lock();
applied.clear(); // disposes the previous sync's languages
for l in accepted {
applied.push(self.store.add(l.plugin, l.language));
}
report.languages = applied.len();
self.store.invalidate();
report
}
fn save(&self, pins: &BTreeMap<String, BinaryGrant>) {
let Some(file) = &self.file else { return };
let json = serde_json::to_string_pretty(&PinsFile { pins: pins.clone() }).unwrap_or_default();
let tmp = file.with_extension("json.tmp");
if let Err(e) = std::fs::write(&tmp, json).and_then(|()| std::fs::rename(&tmp, file)) {
tracing::warn!("saving {}: {e}", file.display());
}
}
}
#[cfg(all(test, unix))]
mod tests {
use super::*;
use ide_core::LanguageId;
use ide_language::LanguageRegistry;
use std::os::unix::fs::PermissionsExt;
fn toy(granted: &[&str]) -> RemoteLanguage {
let language = ide_language::plugin_point::parse_definition(
"id = \"toy\"\nname = \"Toy\"\nextensions = [\"toy\"]\n\
[[language-servers]]\nname = \"toylsp\"\ncommand = \"toylsp\"\n",
)
.unwrap();
RemoteLanguage {
plugin: "acme.toy".into(),
language,
granted: granted.iter().map(|s| (*s).to_owned()).collect(),
}
}
#[test]
fn syncs_languages_and_pins_granted_programs_on_this_machine() {
let tmp = tempfile::tempdir().unwrap();
let bin = tmp.path().join("bin");
std::fs::create_dir(&bin).unwrap();
std::fs::write(bin.join("toylsp"), "#!/bin/sh\n").unwrap();
std::fs::set_permissions(bin.join("toylsp"), std::fs::Permissions::from_mode(0o755)).unwrap();
let path = bin.clone().into_os_string();
let remote = RemotePluginLanguages::with_path(Some(tmp.path()), Arc::new(move || Some(path.clone())));
let registry = LanguageRegistry::load(None).unwrap();
registry.attach_plugins(remote.store().clone());
let project = registry.fork();
let id = LanguageId::new("toy");
// Not granted: the language, no server.
let report = remote.sync(vec![toy(&[])]);
assert_eq!(report.languages, 1);
assert_eq!(project.detect(std::path::Path::new("/r/a.toy"), None), id);
assert!(project.get(&id).unwrap().language_servers.is_empty());
// Granted: resolved here and pinned; persisted.
let report = remote.sync(vec![toy(&["toylsp", "rm"])]);
assert_eq!(report.pinned.len(), 1, "only programs the definition starts: {report:?}");
let pin = project.get(&id).unwrap().language_servers[0].pinned.clone().unwrap();
assert!(pin.verify().is_ok());
assert!(tmp.path().join(PINS_FILE).is_file());
// Replaced: refused at start until revoked and granted again.
std::fs::write(bin.join("toylsp"), "#!/bin/sh\necho x\n").unwrap();
remote.sync(vec![toy(&["toylsp"])]);
assert!(project.get(&id).unwrap().language_servers[0].pinned.as_ref().unwrap().verify().is_err());
remote.sync(vec![toy(&[])]);
remote.sync(vec![toy(&["toylsp"])]);
assert!(project.get(&id).unwrap().language_servers[0].pinned.as_ref().unwrap().verify().is_ok());
// Missing programs are reported; invalid definitions rejected; empty sync removes all.
let mut bad = toy(&[]);
bad.language.id = LanguageId::new("a b");
let report = remote.sync(vec![bad]);
assert_eq!(report.rejected.len(), 1);
assert_eq!(project.detect(std::path::Path::new("/r/a.toy"), None), LanguageId::plain_text());
}
}
+505
View File
@@ -0,0 +1,505 @@
//! Code plugins of remote projects:
//! Wasm runs where the project is, so `openidea-server` hosts the desktop's
//! code plugins and they read, search, run processes and publish diagnostics
//! on the remote machine.
//!
//! The desktop stays the authority over what is installed and granted. It
//! pushes its live code plugins with [`SYNC_PACKAGES`]: first their digests
//! and grants, then the files of those the server doesn't have (one package
//! per request); a package missing from a sync is uninstalled. The server
//! installs them under `<data>/remote-plugins` and records the desktop's
//! grants (a `process:` grant is pinned to the program on this machine's
//! `PATH`).
//!
//! The commands a window needs at run time (`plugin_run_command`,
//! `plugin_post_events`, `plugin_event_kinds`, `plugin_status_items`,
//! `plugin_cancel_task`, `plugin_set_active_file`, `plugin_line_markers`,
//! `plugin_view_render`, `plugin_view_event`)
//! carry the desktop commands' names and arguments, so a remote window's UI
//! routes them here unchanged. Plugin results come back as `plugin`
//! notifications ([`PluginEventMessage`]); secrets stay in the desktop's
//! keychain and are asked for with a `plugin_secret` request
//! ([`SecretRequest`]) when a request header needs one (rule 4).
use super::{
Consent, HostWorkspace, PluginEditor, PluginEvent, PluginHost, PluginSource, PluginStatusItem, SecretStore,
};
use crate::rpc::Dispatcher;
use base64::Engine as _;
use ide_core::{IdeError, IdeResult};
use ide_extension::{MANIFEST_FILE, ledger};
use ide_jsonrpc::Peer;
use ide_plugin::{Invocation, Limits};
use parking_lot::Mutex;
use serde::{Deserialize, Serialize};
use std::collections::{BTreeMap, BTreeSet};
use std::path::{Component, Path, PathBuf};
use std::sync::{Arc, OnceLock, Weak};
use std::time::Duration;
/// The server command the desktop calls with its packages.
pub const SYNC_PACKAGES: &str = "plugin_sync_packages";
/// Server → client notification carrying a [`PluginEventMessage`].
pub const EVENT_NOTIFICATION: &str = "plugin";
/// Server → client request ([`SecretRequest`] → `Option<String>`).
pub const SECRET_REQUEST: &str = "plugin_secret";
/// Under the server's data directory.
pub const PLUGINS_DIR: &str = "remote-plugins";
const INCOMING_DIR: &str = "remote-plugins.incoming";
const SECRET_TIMEOUT: Duration = Duration::from_secs(10);
const EVENT_FLUSH: Duration = Duration::from_millis(250);
const TICKS_PER_SUPERVISION: u32 = 20;
/// One live code plugin of the desktop.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct RemotePackage {
pub id: String,
/// [`package_digest`] of the package.
pub digest: String,
/// Capabilities the user granted on the desktop.
pub granted: Vec<String>,
}
/// One file of a package (`path` relative to the package, `data` base64).
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct PackageFile {
pub path: String,
pub data: String,
}
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct PackageSyncReport {
/// Packages the server needs the files of (send them in another sync).
pub missing: Vec<String>,
pub installed: Vec<String>,
pub removed: Vec<String>,
/// `id: reason`.
pub failed: Vec<String>,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct PluginEventMessage {
/// The window that invoked the plugin (`None`: every window of the connection).
pub window: Option<String>,
pub event: PluginEvent,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct SecretRequest {
pub plugin: String,
pub name: String,
}
/// Identifies a package's content: the manifest, the files its declarations
/// reference and the component.
pub fn package_digest(dir: &Path, files: &[PathBuf]) -> IdeResult<String> {
let mut all = files.to_vec();
all.push(PathBuf::from(MANIFEST_FILE));
Ok(ledger::package_hash(dir, &all)?)
}
/// Whether a plugin event means something to a remote window (the desktop
/// owns the plugin list, themes and consent).
pub fn forwarded(event: &PluginEvent) -> bool {
matches!(
event,
PluginEvent::Notification { .. }
| PluginEvent::StatusItemChanged { .. }
| PluginEvent::TaskStarted { .. }
| PluginEvent::TaskProgress { .. }
| PluginEvent::TaskEnded { .. }
| PluginEvent::SubscriptionsChanged
| PluginEvent::ViewChanged { .. }
| PluginEvent::ViewMessage { .. }
| PluginEvent::OpenTerminal { .. }
)
}
// ---- desktop side -----------------------------------------------------------
impl PluginHost {
fn package_files(&self, plugin: &super::LoadedPlugin) -> IdeResult<Vec<PathBuf>> {
let set = self.registry.prepare(&plugin.pkg).map_err(IdeError::invalid)?;
Ok(set.files.into_iter().chain(plugin.manifest().wasm.clone()).collect())
}
/// The live code plugins (installed or linked) for a remote project's server.
pub fn remote_packages(&self) -> Vec<RemotePackage> {
let plugins: Vec<_> = self.plugins.read().values().cloned().collect();
plugins
.iter()
.filter(|p| p.dev_root.is_none() && !p.invalid && p.manifest().wasm.is_some())
.filter(|p| matches!(p.status(), super::PluginStatus::Active | super::PluginStatus::Inactive))
.filter_map(|p| {
let files = self.package_files(p).ok()?;
let digest = package_digest(p.dir(), &files).ok()?;
let granted = p.meta.lock().granted.iter().map(ToString::to_string).collect();
Some(RemotePackage { id: p.id().to_owned(), digest, granted })
})
.collect()
}
/// The files of a package for [`SYNC_PACKAGES`], the manifest first.
pub fn remote_package_files(&self, id: &str) -> IdeResult<Vec<PackageFile>> {
let plugin = self.get(id)?;
let files = self.package_files(&plugin)?;
std::iter::once(PathBuf::from(MANIFEST_FILE))
.chain(files)
.map(|rel| {
let bytes = std::fs::read(plugin.dir().join(&rel))?;
let path = rel.to_str().ok_or_else(|| IdeError::invalid("non UTF-8 file name"))?.replace('\\', "/");
Ok(PackageFile { path, data: base64::engine::general_purpose::STANDARD.encode(bytes) })
})
.collect()
}
/// A secret a plugin declares, for its remote instance (`plugin_secret`).
pub fn declared_secret(&self, id: &str, name: &str) -> Option<String> {
let plugin = self.get(id).ok()?;
if !self.ui.items(super::providers::SECRETS_POINT, Some(plugin.id())).iter().any(|s| s.key == name) {
return None;
}
let store = self.shared.secrets.read().clone();
store.get(plugin.id(), name)
}
}
// ---- server side ------------------------------------------------------------
/// The plugin host of `openidea-server`.
pub struct RemotePlugins {
host: Arc<PluginHost>,
incoming: PathBuf,
sync: Mutex<()>,
}
/// Secrets from the desktop's keychain, asked for when used.
struct ClientSecrets {
peer: Arc<OnceLock<Peer>>,
runtime: tokio::runtime::Handle,
}
impl SecretStore for ClientSecrets {
fn get(&self, plugin: &str, name: &str) -> Option<String> {
let peer = self.peer.get()?.clone();
let request = SecretRequest { plugin: plugin.to_owned(), name: name.to_owned() };
let ask = async move {
tokio::time::timeout(SECRET_TIMEOUT, peer.request::<_, Option<String>>(SECRET_REQUEST, request, None)).await
};
// Plugin threads block on the runtime; a runtime worker (a host
// command asking) must leave its thread to the other tasks first.
let answer = if tokio::runtime::Handle::try_current().is_ok() {
tokio::task::block_in_place(|| self.runtime.block_on(ask))
} else {
self.runtime.block_on(ask)
};
match answer {
Ok(Ok(value)) => value,
Ok(Err(e)) => {
tracing::warn!("plugin secret {plugin}/{name}: {e}");
None
}
Err(_) => None,
}
}
fn set(&self, _: &str, _: &str, _: Option<&str>) -> Result<(), String> {
Err("secrets are stored on the desktop".into())
}
}
impl RemotePlugins {
/// Creates the host over `<data_dir>/remote-plugins`, loads what earlier
/// sessions installed and starts event delivery and supervision.
pub fn start(data_dir: &Path, projects: &Arc<crate::Host>, peer: Arc<OnceLock<Peer>>) -> IdeResult<Arc<Self>> {
let sink_peer = peer.clone();
let sink = move |client: Option<&str>, event: PluginEvent| {
if forwarded(&event)
&& let Some(p) = sink_peer.get()
{
let _ = p.notify(EVENT_NOTIFICATION, PluginEventMessage { window: client.map(str::to_owned), event });
}
};
let host = Arc::new(PluginHost::new(data_dir.join(PLUGINS_DIR), Arc::new(sink), Limits::default())?);
host.set_workspace(Some(Arc::new(HostWorkspace(Arc::downgrade(projects)))));
host.set_secret_store(Arc::new(ClientSecrets { peer, runtime: projects.runtime.clone() }));
let trust = Arc::downgrade(projects);
host.set_project_policy(Arc::new(move |root| {
trust.upgrade().is_some_and(|h| h.trust.trusted(root).unwrap_or(false))
}));
super::set_editor_providers(Some(&host));
ide_language::set_external_syntax(Some(host.syntax_provider()));
ide_run::set_external_detection(Some(host.run_detection()));
host.load_all();
host.startup_finished();
spawn_supervision(Arc::downgrade(&host))?;
Ok(Arc::new(Self { host, incoming: data_dir.join(INCOMING_DIR), sync: Mutex::new(()) }))
}
pub fn host(&self) -> &Arc<PluginHost> {
&self.host
}
/// Makes the installed packages match `packages`: uninstalls the others,
/// follows changed grants, installs those with `files` and reports the
/// rest as missing.
pub fn sync(
&self,
packages: Vec<RemotePackage>,
mut files: BTreeMap<String, Vec<PackageFile>>,
) -> PackageSyncReport {
let _guard = self.sync.lock();
let mut report = PackageSyncReport::default();
let wanted: BTreeSet<&str> = packages.iter().map(|p| p.id.as_str()).collect();
let installed: Vec<String> = self
.host
.plugins
.read()
.values()
.filter(|p| p.source == PluginSource::Installed)
.map(|p| p.id().to_owned())
.collect();
for id in installed.iter().filter(|id| !wanted.contains(id.as_str())) {
match self.host.uninstall(id) {
Ok(()) => report.removed.push(id.clone()),
Err(e) => report.failed.push(format!("{id}: {e}")),
}
}
for pkg in packages {
if !valid_id(&pkg.id) {
report.failed.push(format!("{}: invalid plugin id", pkg.id));
continue;
}
// The desktop asked the user, change of signer included.
let consent = Consent { optional: pkg.granted.clone(), signer_change: true };
if self.current_digest(&pkg.id).as_deref() == Some(pkg.digest.as_str()) {
if let Err(e) = self.follow_grants(&pkg, &consent) {
report.failed.push(format!("{}: {e}", pkg.id));
}
continue;
}
match files.remove(&pkg.id) {
None => report.missing.push(pkg.id),
Some(list) => match self.install(&pkg, list, &consent) {
Ok(()) => report.installed.push(pkg.id),
Err(e) => report.failed.push(format!("{}: {}", pkg.id, e.message)),
},
}
}
report
}
fn current_digest(&self, id: &str) -> Option<String> {
let plugin = self.host.plugins.read().get(id).cloned()?;
if plugin.source != PluginSource::Installed || plugin.invalid {
return None;
}
let files = self.host.package_files(&plugin).ok()?;
package_digest(plugin.dir(), &files).ok()
}
fn follow_grants(&self, pkg: &RemotePackage, consent: &Consent) -> IdeResult<()> {
let info = self.host.info_of(&pkg.id)?;
let have: BTreeSet<&str> = info.capabilities.iter().filter(|c| c.granted).map(|c| c.id.as_str()).collect();
let want: BTreeSet<&str> = pkg.granted.iter().map(String::as_str).collect();
if have != want || info.status == super::PluginStatus::NeedsConsent {
self.host.grant(&pkg.id, consent)?;
}
Ok(())
}
fn install(&self, pkg: &RemotePackage, files: Vec<PackageFile>, consent: &Consent) -> IdeResult<()> {
let dir = self.incoming.join(&pkg.id);
let _ = std::fs::remove_dir_all(&dir);
let result = (|| {
let mut rels = Vec::new();
for f in files {
let rel = safe_relative(&f.path)?;
let bytes = base64::engine::general_purpose::STANDARD
.decode(f.data.as_bytes())
.map_err(|e| IdeError::invalid(format!("{}: {e}", f.path)))?;
let to = dir.join(&rel);
if let Some(parent) = to.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(&to, bytes)?;
if rel != Path::new(MANIFEST_FILE) {
rels.push(rel);
}
}
if package_digest(&dir, &rels)? != pkg.digest {
return Err(IdeError::invalid("the package arrived incomplete or changed"));
}
let info = self.host.install_from_dir(&dir, consent)?;
if info.id != pkg.id {
let _ = self.host.uninstall(&info.id);
return Err(IdeError::invalid(format!("the package is plugin {:?}", info.id)));
}
Ok(())
})();
let _ = std::fs::remove_dir_all(&dir);
result
}
/// The run-time commands of remote windows plus [`SYNC_PACKAGES`].
pub fn register(self: &Arc<Self>, d: &mut Dispatcher) {
let this = self.clone();
d.add(SYNC_PACKAGES, move |c| {
let this = this.clone();
async move {
let packages: Vec<RemotePackage> = c.arg("packages")?;
let files: Option<BTreeMap<String, Vec<PackageFile>>> = c.arg("files")?;
blocking(move || Ok(this.sync(packages, files.unwrap_or_default()))).await
}
});
let host = Arc::downgrade(&self.host);
let h = host.clone();
d.add("plugin_run_command", move |c| {
let h = h.clone();
async move {
let host = alive(&h)?;
let plugin_id: String = c.arg("pluginId")?;
let command_id: String = c.arg("commandId")?;
let arg: Option<String> = c.arg("arg")?;
let editor: Option<PluginEditor> = c.arg("editor")?;
let invocation = Invocation {
client: Some(c.window.clone()),
workspace_root: c.host.project(&c.window).ok().map(|p| p.root.clone()),
editor: editor.map(PluginEditor::context),
};
blocking(move || host.run_command(&plugin_id, &command_id, arg.as_deref(), invocation)).await
}
});
let h = host.clone();
d.add("plugin_post_events", move |c| {
let h = h.clone();
async move {
alive(&h)?.post_events(Some(&c.window), c.arg("events")?);
Ok(())
}
});
let h = host.clone();
d.add("plugin_event_kinds", move |_| {
let h = h.clone();
async move { Ok(alive(&h)?.event_kinds().into_iter().collect::<Vec<_>>()) }
});
let h = host.clone();
d.add("plugin_status_items", move |_| {
let h = h.clone();
async move {
Ok(alive(&h)?
.status_items()
.into_iter()
.map(|(plugin_id, id, item)| PluginStatusItem { plugin_id, id, item })
.collect::<Vec<_>>())
}
});
let h = host.clone();
d.add("plugin_cancel_task", move |c| {
let h = h.clone();
async move { Ok(alive(&h)?.cancel_task(&c.arg::<String>("taskId")?)) }
});
let h = host.clone();
d.add("plugin_set_active_file", move |c| {
let h = h.clone();
async move {
alive(&h)?.set_active_file(&c.window, c.arg("path")?);
Ok(())
}
});
let h = host.clone();
d.add("plugin_view_render", move |c| {
let h = h.clone();
async move {
let plugins = alive(&h)?;
super::views::render(
&c.host,
plugins,
&c.window,
c.arg("pluginId")?,
c.arg("view")?,
c.arg("viewport")?,
c.arg("doc")?,
c.arg("editor")?,
)
.await
}
});
let h = host.clone();
d.add("plugin_view_event", move |c| {
let h = h.clone();
async move {
let plugins = alive(&h)?;
super::views::event(
&c.host,
plugins,
&c.window,
c.arg("pluginId")?,
c.arg("view")?,
c.arg("event")?,
c.arg("doc")?,
c.arg("editor")?,
)
.await
}
});
d.add("plugin_line_markers", |c| async move {
crate::ops::lsp::plugin_line_markers(&c.host, &c.window, c.arg("doc")?).await
});
}
}
fn alive(host: &Weak<PluginHost>) -> IdeResult<Arc<PluginHost>> {
host.upgrade().ok_or_else(|| IdeError::unavailable("the plugin host has shut down"))
}
async fn blocking<T: Send + 'static>(f: impl FnOnce() -> IdeResult<T> + Send + 'static) -> IdeResult<T> {
tokio::task::spawn_blocking(f).await.map_err(|e| IdeError::internal(e.to_string()))?
}
fn spawn_supervision(host: Weak<PluginHost>) -> IdeResult<()> {
std::thread::Builder::new().name("remote-plugins".into()).spawn(move || {
for n in 1u32.. {
std::thread::sleep(EVENT_FLUSH);
let Some(host) = host.upgrade() else { return };
host.flush_events();
if n % TICKS_PER_SUPERVISION == 0 {
host.tick();
}
}
})?;
Ok(())
}
fn valid_id(id: &str) -> bool {
!id.is_empty()
&& id.len() <= 128
&& !id.starts_with('.')
&& id.chars().all(|c| c.is_ascii_alphanumeric() || "-_.".contains(c))
}
/// `path` as a relative path inside a package (no `..`, no root).
fn safe_relative(path: &str) -> IdeResult<PathBuf> {
let p = Path::new(path);
if path.is_empty() || !p.components().all(|c| matches!(c, Component::Normal(_))) {
return Err(IdeError::invalid(format!("invalid file path {path:?} in package")));
}
Ok(p.to_path_buf())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn package_paths_stay_inside_the_package() {
assert!(safe_relative("dist/plugin.wasm").is_ok());
for bad in ["", "../x", "/etc/passwd", "a/../../b", "./x"] {
assert!(safe_relative(bad).is_err(), "{bad}");
}
assert!(valid_id("ai-completion"));
assert!(!valid_id("../x") && !valid_id(".data") && !valid_id(""));
}
}
@@ -0,0 +1,167 @@
//! The `runDetectors` extension point: run configurations a plugin offers in
//! projects that contain a marker file.
//!
//! ```toml
//! [[contributes.runDetectors]]
//! id = "forge-test"
//! name = "forge test"
//! files = ["foundry.toml"] # any of these at the project root
//! command = "forge"
//! args = ["test"]
//! test = true # results go to the test tree
//! ```
//!
//! The command is a `process:` capability like a language server's: the
//! configuration is offered only while it is granted, and starts the approved
//! file (checked right before it starts).
use ide_extension::{
Capability, Disposable, ExtensionPoint, Package, Placement, Prepared, check_relative, decode, validate_item_id,
};
use ide_language::ProgramPin;
use ide_run::{EXTERNAL_PREFIX, ExternalDetection, RunConfig};
use parking_lot::RwLock;
use serde::Deserialize;
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};
pub const POINT_ID: &str = "runDetectors";
const MAX_FILES: usize = 16;
const MAX_ARGS: usize = 64;
#[derive(Clone, Debug, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct Detector {
pub id: String,
pub name: String,
pub files: Vec<PathBuf>,
pub command: String,
#[serde(default)]
pub args: Vec<String>,
#[serde(default)]
pub test: bool,
}
/// Detectors of enabled plugins.
#[derive(Default)]
pub struct RunDetectors {
items: RwLock<Vec<(u64, String, Detector)>>,
seq: AtomicU64,
}
impl RunDetectors {
fn add(self: &Arc<Self>, plugin: String, detector: Detector) -> Disposable {
let seq = self.seq.fetch_add(1, Ordering::Relaxed);
self.items.write().push((seq, plugin, detector));
let weak = Arc::downgrade(self);
Disposable::new(move || {
if let Some(this) = weak.upgrade() {
this.items.write().retain(|(s, _, _)| *s != seq);
}
})
}
}
pub struct RunDetectorsPoint {
store: Arc<RunDetectors>,
}
impl RunDetectorsPoint {
pub fn new(store: Arc<RunDetectors>) -> Self {
Self { store }
}
}
impl ExtensionPoint for RunDetectorsPoint {
fn id(&self) -> &'static str {
POINT_ID
}
fn placement(&self) -> Placement {
Placement::Workspace
}
fn prepare(&self, pkg: &Package, decl: toml::Value) -> Result<Prepared, String> {
let d: Detector = decode(POINT_ID, decl)?;
validate_item_id(&d.id, "run detector id")?;
if d.name.trim().is_empty() || d.name.len() > 128 {
return Err(format!("run detector {:?}: `name` must be 1–128 bytes", d.id));
}
if d.files.is_empty() || d.files.len() > MAX_FILES {
return Err(format!("run detector {:?}: give 1–{MAX_FILES} marker `files`", d.id));
}
for f in &d.files {
check_relative(f, "files")?;
}
let program = d.command.trim();
if program.is_empty() || program.contains(char::is_whitespace) {
return Err(format!("run detector {:?}: `command` is one program name (arguments go in `args`)", d.id));
}
if d.args.len() > MAX_ARGS {
return Err(format!("run detector {:?}: at most {MAX_ARGS} arguments", d.id));
}
let cap = Capability::Process(program.to_owned());
let (store, plugin, label) = (self.store.clone(), pkg.id().to_owned(), d.name.clone());
let key = d.id.clone();
Ok(Prepared::new(label, move || Ok(store.add(plugin, d))).key(key).capability(cap))
}
}
type Pins = Arc<RwLock<HashMap<(String, String), ProgramPin>>>;
/// [`ExternalDetection`] over the detectors and the approved programs.
pub struct PluginRunDetection {
pub(crate) store: Arc<RunDetectors>,
pub(crate) pins: Pins,
}
fn config_id(plugin: &str, detector: &str) -> String {
format!("{EXTERNAL_PREFIX}{plugin}:{detector}")
}
impl PluginRunDetection {
fn pin(&self, plugin: &str, d: &Detector) -> Option<ProgramPin> {
let cap = Capability::Process(d.command.trim().to_owned());
self.pins.read().get(&(plugin.to_owned(), cap.to_string())).cloned()
}
}
impl ExternalDetection for PluginRunDetection {
fn detect(&self, root: &Path) -> Vec<RunConfig> {
let items = self.store.items.read().clone();
items
.into_iter()
.filter(|(_, plugin, d)| self.pin(plugin, d).is_some() && d.files.iter().any(|f| root.join(f).exists()))
.map(|(_, plugin, d)| RunConfig {
id: config_id(&plugin, &d.id),
name: d.name.clone(),
kind: "plugin".into(),
command: d.command.trim().to_owned(),
args: d.args.clone(),
cwd: None,
env: Default::default(),
detected: true,
is_test: d.test,
debug_adapter: None,
allow_parallel: false,
extras: Default::default(),
})
.collect()
}
fn launch(&self, config: &RunConfig) -> Result<PathBuf, String> {
let items = self.store.items.read();
let (plugin, d) = items
.iter()
.find(|(_, plugin, d)| config_id(plugin, &d.id) == config.id)
.map(|(_, p, d)| (p.clone(), d.clone()))
.ok_or_else(|| "the plugin that detected this configuration is no longer enabled".to_owned())?;
drop(items);
let pin = self.pin(&plugin, &d).ok_or_else(|| {
format!("allow process:{} for plugin {plugin} in Settings › Plugins to run this configuration", d.command)
})?;
pin.verify().map(Path::to_path_buf)
}
}
@@ -1,9 +1,21 @@
//! Theme contributions: design-token overrides applied as CSS variables on `:root`.
//! Theme and editor-scheme files are JSON that may contain `//` and `/* */` comments.
//! The `themes` extension point: design-token overrides applied as CSS
//! variables on `:root`. Theme and editor-scheme files are JSON that may
//! contain `//` and `/* */` comments.
//!
//! ```toml
//! [[contributes.themes]]
//! path = "themes/harbor-night.json"
//! editor-scheme = "themes/harbor-night.scheme.json" # optional
//! ```
use ide_extension::{ExtensionPoint, Package, Placement, Prepared, UiContribution, UiStore, decode};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::collections::BTreeMap;
use std::path::PathBuf;
use std::sync::Arc;
pub const POINT_ID: &str = "themes";
const MAX_TOKENS: usize = 512;
const MAX_VALUE_LEN: usize = 256;
@@ -48,7 +60,7 @@ impl ThemeDef {
}
pub fn validate(&self) -> Result<(), String> {
crate::manifest::validate_command_id(&self.id).map_err(|e| e.replace("command id", "theme id"))?;
ide_extension::validate_item_id(&self.id, "theme id")?;
if self.name.trim().is_empty() {
return Err("theme name must not be empty".into());
}
@@ -71,9 +83,58 @@ impl ThemeDef {
}
}
pub struct ThemesPoint {
store: Arc<UiStore>,
}
impl ThemesPoint {
pub fn new(store: Arc<UiStore>) -> Self {
Self { store }
}
}
#[derive(Deserialize)]
#[serde(rename_all = "kebab-case", deny_unknown_fields)]
struct Decl {
path: PathBuf,
#[serde(default)]
editor_scheme: Option<PathBuf>,
}
impl ExtensionPoint for ThemesPoint {
fn id(&self) -> &'static str {
POINT_ID
}
fn placement(&self) -> Placement {
Placement::Ui
}
fn prepare(&self, pkg: &Package, decl: toml::Value) -> Result<Prepared, String> {
let decl: Decl = decode(POINT_ID, decl)?;
let path = &decl.path;
let mut theme = ThemeDef::parse(&pkg.read_text(path)?).map_err(|e| format!("{}: {e}", path.display()))?;
let mut files = vec![path.clone()];
if let Some(scheme) = &decl.editor_scheme {
files.push(scheme.clone());
theme.merge_editor_scheme(&pkg.read_text(scheme)?).map_err(|e| format!("{}: {e}", scheme.display()))?;
}
let item = UiContribution {
plugin_id: pkg.id().to_owned(),
point: POINT_ID.into(),
key: theme.id.clone(),
decl: serde_json::to_value(&theme).map_err(|e| e.to_string())?,
};
let store = self.store.clone();
let mut prepared = Prepared::new(theme.name.clone(), move || Ok(store.insert(item))).key(theme.id);
prepared.files = files;
Ok(prepared)
}
}
/// Blanks out `//` and `/* */` comments outside strings, keeping newlines so
/// parse errors still point at the right line.
fn strip_json_comments(src: &str) -> String {
pub(super) fn strip_json_comments(src: &str) -> String {
let mut out = String::with_capacity(src.len());
let mut chars = src.chars().peekable();
let (mut in_str, mut escaped) = (false, false);
+256
View File
@@ -0,0 +1,256 @@
//! UI-facing plugin types and the event stream.
use super::theme::ThemeDef;
use ide_extension::{CapabilityInfo, ContributionInfo, UiContribution};
pub use ide_plugin::{ActionPerformed, ActionSource, EventKind, NotificationLevel, StatusItem, WorkbenchEvent};
use serde::{Deserialize, Serialize};
use specta::Type;
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum PluginSource {
/// Installed under `<app-data>/plugins/<id>/`.
Installed,
/// Loaded from `<project>/.openidea/plugins/<dir>/` while that project is open.
Dev,
/// A development plugin linked from any folder (`openidea-plugin link`).
Linked,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum PluginStatus {
/// Code instantiated and activated.
Active,
/// Has code that has not been needed yet: contributions are live, no Wasm instance.
Inactive,
/// No code; contributions only.
Static,
Disabled,
/// Waiting for the user to confirm its permissions (project and new plugins).
NeedsConsent,
/// Failed to load, or failed repeatedly and was stopped for this session (see `error`).
Failed,
}
impl PluginStatus {
/// Contributions are in effect.
pub fn is_live(self) -> bool {
matches!(self, Self::Active | Self::Inactive | Self::Static)
}
}
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PluginInfo {
pub id: String,
pub name: String,
pub version: String,
pub description: String,
pub authors: Vec<String>,
pub homepage: Option<String>,
pub change_notes: Option<String>,
pub source: PluginSource,
/// Package directory.
pub path: ide_core::IpcPath,
pub enabled: bool,
pub status: PluginStatus,
pub error: Option<String>,
pub has_code: bool,
/// Declared capabilities with risk and grant state.
pub capabilities: Vec<CapabilityInfo>,
/// What it contributes while live (point + label).
pub contributions: Vec<ContributionInfo>,
/// `activation` events of the manifest.
pub activation: Vec<String>,
/// Non-fatal problems (unsupported contribution points).
pub warnings: Vec<String>,
/// Statistics of this session.
pub stats: PluginStats,
/// The marketplace publisher whose signature the package carries (none: unsigned or untrusted).
pub publisher: Option<String>,
/// The version an update replaced, to roll back to.
pub previous_version: Option<String>,
}
#[derive(Clone, Copy, Debug, Default, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PluginStats {
pub activations: u32,
/// Last activation (instantiate + `activate`), ms.
pub activation_ms: f64,
pub calls: u32,
pub errors: u32,
/// Slowest call, ms.
pub max_call_ms: f64,
/// 95th percentile of the recent calls, ms.
pub p95_call_ms: f64,
/// Edit-path calls whose result came too late and was dropped.
pub slow_calls: u32,
/// Traps (panics, budgets, permission violations); each drops the instance.
pub crashes: u32,
/// Instances reclaimed after being idle.
pub reclaims: u32,
}
/// Number of declarations of one extension point.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct ContributionCount {
pub point: String,
pub count: u32,
}
/// A plugin package available for installation from a local catalog
/// directory (the samples shipped with the IDE). Not installed by listing it.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct CatalogPlugin {
pub id: String,
pub name: String,
pub version: String,
pub description: String,
pub authors: Vec<String>,
pub homepage: Option<String>,
pub change_notes: Option<String>,
/// Package directory; install with `install_from_dir(path)`.
pub path: ide_core::IpcPath,
pub has_code: bool,
pub capabilities: Vec<CapabilityInfo>,
pub contributions: Vec<ContributionCount>,
/// Why the package cannot be installed as is (e.g. its WASM is not built).
pub problem: Option<String>,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct ActiveTheme {
pub plugin_id: String,
pub theme: ThemeDef,
}
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PluginsSnapshot {
pub plugins: Vec<PluginInfo>,
pub active_theme: Option<ActiveTheme>,
/// UI contributions of live plugins, projected by the client.
pub ui: Vec<UiContribution>,
}
/// A newer version of an installed plugin in the marketplace.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PluginUpdate {
pub id: String,
pub name: String,
pub version: String,
}
/// Backend → UI notifications (Tauri event `ide://plugin`).
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(tag = "type", rename_all = "camelCase")]
pub enum PluginEvent {
/// Plugins, their status or their contributions changed: re-fetch `plugin_list`.
CommandsChanged,
/// The marketplace offers newer versions of installed plugins (announced
/// once per set of versions).
UpdatesAvailable { updates: Vec<PluginUpdate> },
#[serde(rename_all = "camelCase")]
Notification { plugin_id: String, level: NotificationLevel, title: String, body: String },
/// A plugin's status bar item was set (`item`) or removed.
#[serde(rename_all = "camelCase")]
StatusItemChanged { plugin_id: String, id: String, item: Option<StatusItem> },
/// A plugin began a task: show it with the background tasks.
#[serde(rename_all = "camelCase")]
TaskStarted { plugin_id: String, task_id: String, title: String, cancellable: bool },
#[serde(rename_all = "camelCase")]
TaskProgress { plugin_id: String, task_id: String, done: u32, total: Option<u32>, message: Option<String> },
#[serde(rename_all = "camelCase")]
TaskEnded { plugin_id: String, task_id: String },
/// The kinds of workbench events plugins want changed: re-fetch
/// `plugin_event_kinds` (the UI reports only those).
SubscriptionsChanged,
#[serde(rename_all = "camelCase")]
ThemeChanged { theme: Option<ActiveTheme> },
/// A plugin needs capabilities the user has not granted yet (first use
/// after migrating from 0.1): show its consent dialog.
#[serde(rename_all = "camelCase")]
ConsentRequested { plugin_id: String, capabilities: Vec<CapabilityInfo> },
/// A view (or the custom editors) of a plugin must be rendered again.
#[serde(rename_all = "camelCase")]
ViewChanged { plugin_id: String, view: String },
/// For the script of a view's `html` body.
#[serde(rename_all = "camelCase")]
ViewMessage { plugin_id: String, view: String, message: String },
/// Open a terminal running `program` (already approved and verified).
#[serde(rename_all = "camelCase")]
OpenTerminal {
plugin_id: String,
title: String,
program: ide_core::IpcPath,
args: Vec<String>,
cwd: ide_core::IpcPath,
},
/// Plugin languages, or the servers they may start, changed: re-fetch
/// `languages` (the host re-detects open documents: [`crate::Host::plugin_languages_changed`]).
LanguagesChanged,
}
/// Delivers events to the UI. `client` is the window that invoked the plugin
/// (`None` = broadcast, e.g. during activation).
pub trait PluginEventSink: Send + Sync {
fn emit(&self, client: Option<&str>, event: PluginEvent);
}
impl<F: Fn(Option<&str>, PluginEvent) + Send + Sync> PluginEventSink for F {
fn emit(&self, client: Option<&str>, event: PluginEvent) {
self(client, event)
}
}
/// The focused editor when a plugin command is invoked.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PluginEditor {
pub path: String,
pub language: String,
pub selections: Vec<ide_core::Range>,
}
impl PluginEditor {
pub fn context(self) -> ide_plugin::EditorContext {
ide_plugin::EditorContext { path: self.path.into(), language: self.language, selections: self.selections }
}
}
/// A text file of a package the UI generated, at its `/`-separated path in the package.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PackageFile {
pub path: String,
pub text: String,
}
/// A plugin's status bar item.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PluginStatusItem {
pub plugin_id: String,
pub id: String,
pub item: StatusItem,
}
/// A gutter marker from a plugin's `line-markers` provider.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct LineMarkerInfo {
pub plugin_id: String,
/// 0-based.
pub line: u32,
/// `navigate` (default), `implemented`, `implementing`, `run`.
pub icon: Option<String>,
pub tooltip: String,
/// Opened on click.
pub target: Option<ide_core::Location>,
}
+544
View File
@@ -0,0 +1,544 @@
//! Declarative UI contributions: menus, key
//! bindings, status bar items and settings (L0, shown before any code runs),
//! and the views and custom editors whose content plugin code renders (L2).
//! Everything is validated here; the client only projects the JSON.
//!
//! ```toml
//! [[contributes.menus]]
//! menu = "editor.context" # projectView.context, editorTab.context, tools
//! command = "check"
//! when = "editorLang == java"
//!
//! [[contributes.keymaps]]
//! command = "check"
//! key = "Ctrl+Alt+C" # `mac` for a different binding on macOS (`Meta+Alt+C`)
//!
//! [[contributes.statusBar]]
//! id = "branch"
//! text = "…" # code replaces it with `set-status-item("branch", …)`
//! command = "show"
//!
//! [contributes.settings]
//! schema = "settings.schema.json" # {"properties": {"key": {"type": "boolean", "default": true, …}}}
//!
//! [[contributes.views]]
//! id = "jobs"
//! title = "Jobs"
//! icon = "server"
//! location = "right" # left, right, bottom
//!
//! [[contributes.customEditors]]
//! id = "table"
//! title = "Table"
//! patterns = ["*.csv", "*.tsv"]
//! priority = "default" # or "option": only through Open With
//! layout = "tab" # or "split" / "preview": beside the text editor
//! web = "web/index.html" # a web page from the package instead of rendered content
//!
//! [[contributes.fileIcons]]
//! id = "drawing"
//! patterns = ["*.excalidraw"]
//! icon = "icons/drawing.svg"
//! iconDark = "icons/drawing_dark.svg"
//!
//! [[contributes.fileTemplates]]
//! id = "drawing"
//! name = "Excalidraw Drawing"
//! extension = "excalidraw"
//! path = "templates/drawing.excalidraw"
//! ```
use ide_extension::{Package, UiDecl, decode, validate_item_id, validate_when};
use serde::{Deserialize, Serialize};
use serde_json::Value;
use std::collections::BTreeSet;
use std::path::PathBuf;
pub const MENUS: &str = "menus";
pub const KEYMAPS: &str = "keymaps";
pub const STATUS_BAR: &str = "statusBar";
pub const SETTINGS: &str = "settings";
pub const VIEWS: &str = "views";
pub const CUSTOM_EDITORS: &str = "customEditors";
pub const FILE_ICONS: &str = "fileIcons";
pub const FILE_TEMPLATES: &str = "fileTemplates";
const MENU_IDS: [&str; 4] = ["editor.context", "projectView.context", "editorTab.context", "tools"];
const MAX_TITLE: usize = 256;
const MAX_SETTINGS: usize = 200;
const MAX_PATTERNS: usize = 16;
/// Command ids the package declares.
fn commands(pkg: &Package) -> BTreeSet<String> {
pkg.manifest
.declarations(super::COMMANDS)
.iter()
.filter_map(|d| d.get("id").and_then(|v| v.as_str()).map(str::to_owned))
.collect()
}
fn declared_command(pkg: &Package, point: &str, command: &str) -> Result<(), String> {
if commands(pkg).contains(command) {
Ok(())
} else {
Err(format!("contributes.{point}: command {command:?} is not declared under [[contributes.commands]]"))
}
}
fn title(point: &str, t: &str) -> Result<(), String> {
if t.trim().is_empty() || t.len() > MAX_TITLE {
return Err(format!("contributes.{point}: titles are 1–{MAX_TITLE} bytes"));
}
Ok(())
}
fn json<T: Serialize>(v: &T) -> Result<Value, String> {
serde_json::to_value(v).map_err(|e| e.to_string())
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct MenuDecl {
menu: String,
command: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
when: Option<String>,
}
pub(super) fn menu(pkg: &Package, decl: toml::Value) -> Result<UiDecl, String> {
let d: MenuDecl = decode(MENUS, decl)?;
if !MENU_IDS.contains(&d.menu.as_str()) {
return Err(format!("contributes.menus: unknown menu {:?} (one of {})", d.menu, MENU_IDS.join(", ")));
}
declared_command(pkg, MENUS, &d.command)?;
if let Some(w) = &d.when {
validate_when(w).map_err(|e| format!("contributes.menus: {e}"))?;
}
Ok(UiDecl::new(format!("{}/{}", d.menu, d.command), d.command.clone(), json(&d)?))
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct KeymapDecl {
command: String,
key: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
mac: Option<String>,
}
const MODIFIERS: [&str; 4] = ["Meta", "Ctrl", "Alt", "Shift"];
const NAMED_KEYS: [&str; 22] = [
"Enter",
"Escape",
"Tab",
"Space",
"Backspace",
"Delete",
"Insert",
"Home",
"End",
"PageUp",
"PageDown",
"Up",
"Down",
"Left",
"Right",
"Minus",
"Equal",
"Comma",
"Period",
"Slash",
"Semicolon",
"Quote",
];
/// `Ctrl+Alt+C`, `Meta+K Meta+C` (a chord): at least one modifier per
/// stroke but for function keys, so plugins can't take plain typing.
pub fn validate_binding(binding: &str) -> Result<(), String> {
let strokes: Vec<&str> = binding.split(' ').collect();
if strokes.is_empty() || strokes.len() > 2 {
return Err(format!("{binding:?}: one stroke, or two for a chord"));
}
for stroke in strokes {
let parts: Vec<&str> = stroke.split('+').collect();
let (key, mods) = parts.split_last().ok_or_else(|| format!("{binding:?}: empty stroke"))?;
let mut seen = BTreeSet::new();
for m in mods {
if !MODIFIERS.contains(m) || !seen.insert(*m) {
return Err(format!("{binding:?}: unknown or repeated modifier {m:?}"));
}
}
let function = key.strip_prefix('F').and_then(|n| n.parse::<u8>().ok()).is_some_and(|n| (1..=24).contains(&n));
let plain = key.len() == 1 && key.chars().all(|c| c.is_ascii_uppercase() || c.is_ascii_digit());
if !(function || plain || NAMED_KEYS.contains(key)) {
return Err(format!("{binding:?}: unknown key {key:?}"));
}
if !function && seen.iter().all(|m| *m == "Shift") {
return Err(format!("{binding:?}: needs Ctrl, Alt or Meta (plain keys are for typing)"));
}
}
Ok(())
}
pub(super) fn keymap(pkg: &Package, decl: toml::Value) -> Result<UiDecl, String> {
let d: KeymapDecl = decode(KEYMAPS, decl)?;
declared_command(pkg, KEYMAPS, &d.command)?;
validate_binding(&d.key).map_err(|e| format!("contributes.keymaps: {e}"))?;
if let Some(m) = &d.mac {
validate_binding(m).map_err(|e| format!("contributes.keymaps: {e}"))?;
}
Ok(UiDecl::new(d.command.clone(), d.key.clone(), json(&d)?))
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct StatusDecl {
id: String,
text: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
tooltip: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
command: Option<String>,
#[serde(default = "right")]
align: String,
#[serde(default)]
priority: u32,
}
fn right() -> String {
"right".into()
}
pub(super) fn status_item(pkg: &Package, decl: toml::Value) -> Result<UiDecl, String> {
let d: StatusDecl = decode(STATUS_BAR, decl)?;
validate_item_id(&d.id, "status bar item id")?;
if d.text.is_empty() || d.text.len() > 128 || d.tooltip.as_ref().is_some_and(|t| t.len() > 1024) {
return Err(format!("status bar item {:?}: `text` 1–128 bytes, `tooltip` up to 1 KiB", d.id));
}
if d.align != "left" && d.align != "right" {
return Err(format!("status bar item {:?}: `align` is left or right", d.id));
}
if d.priority > 1000 {
return Err(format!("status bar item {:?}: `priority` is 0–1000", d.id));
}
if let Some(c) = &d.command {
declared_command(pkg, STATUS_BAR, c)?;
}
Ok(UiDecl::new(d.id.clone(), d.text.clone(), json(&d)?))
}
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct SettingsDecl {
#[serde(default)]
title: Option<String>,
schema: PathBuf,
}
#[derive(Debug, Deserialize)]
struct SchemaFile {
properties: serde_json::Map<String, Value>,
}
/// One setting of a plugin.
#[derive(Clone, Debug, PartialEq, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct SettingSchema {
pub key: String,
/// `boolean`, `string`, `number` or `integer`.
#[serde(rename = "type")]
pub kind: String,
pub title: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub description: Option<String>,
pub default: Value,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub options: Option<Vec<String>>,
#[serde(default)]
pub multiline: bool,
}
impl SettingSchema {
/// Whether `v` is a valid value of this setting.
pub fn accepts(&self, v: &Value) -> bool {
match self.kind.as_str() {
"boolean" => v.is_boolean(),
"integer" => v.is_i64() || v.is_u64(),
"number" => v.is_number(),
_ => v
.as_str()
.is_some_and(|s| s.len() <= 64 << 10 && self.options.as_ref().is_none_or(|o| o.iter().any(|x| x == s))),
}
}
}
fn setting(key: &str, raw: &Value) -> Result<SettingSchema, String> {
if key.is_empty() || key.len() > 64 || !key.chars().all(|c| c.is_ascii_alphanumeric() || "-_.".contains(c)) {
return Err(format!("setting {key:?}: keys are 1–64 of [A-Za-z0-9._-]"));
}
let o = raw.as_object().ok_or_else(|| format!("setting {key:?}: expected an object"))?;
let kind = o.get("type").and_then(Value::as_str).unwrap_or("");
let default = match kind {
"boolean" => o.get("default").cloned().unwrap_or(Value::Bool(false)),
"string" => o.get("default").cloned().unwrap_or(Value::String(String::new())),
"number" | "integer" => o.get("default").cloned().unwrap_or(Value::from(0)),
_ => return Err(format!("setting {key:?}: `type` is boolean, string, number or integer")),
};
let options = match o.get("enum") {
None => None,
Some(Value::Array(a)) if kind == "string" && !a.is_empty() && a.len() <= 100 => Some(
a.iter()
.map(|v| v.as_str().map(str::to_owned).ok_or_else(|| format!("setting {key:?}: `enum` lists strings")))
.collect::<Result<Vec<_>, _>>()?,
),
Some(_) => return Err(format!("setting {key:?}: `enum` is 1–100 strings of a string setting")),
};
let text = |name: &str, max: usize| -> Result<Option<String>, String> {
match o.get(name) {
None => Ok(None),
Some(Value::String(s)) if s.len() <= max => Ok(Some(s.clone())),
Some(_) => Err(format!("setting {key:?}: `{name}` is a string up to {max} bytes")),
}
};
let s = SettingSchema {
key: key.to_owned(),
kind: kind.to_owned(),
title: text("title", MAX_TITLE)?.unwrap_or_else(|| key.to_owned()),
description: text("description", 4096)?,
default,
options,
multiline: o.get("multiline").and_then(Value::as_bool).unwrap_or(false),
};
if !s.accepts(&s.default) {
return Err(format!("setting {key:?}: the default does not match its type"));
}
Ok(s)
}
/// The validated settings of a `[contributes.settings]` schema file.
pub fn parse_settings_schema(text: &str) -> Result<Vec<SettingSchema>, String> {
let file: SchemaFile = serde_json::from_str(text).map_err(|e| e.to_string())?;
if file.properties.len() > MAX_SETTINGS {
return Err(format!("more than {MAX_SETTINGS} settings"));
}
file.properties.iter().map(|(k, v)| setting(k, v)).collect()
}
pub(super) fn settings(pkg: &Package, decl: toml::Value) -> Result<UiDecl, String> {
let d: SettingsDecl = decode(SETTINGS, decl)?;
let text = pkg.read_text(&d.schema)?;
let properties = parse_settings_schema(&text).map_err(|e| format!("{}: {e}", d.schema.display()))?;
let title = d.title.unwrap_or_else(|| pkg.manifest.name.clone());
title_ok(&title)?;
let mut decl =
UiDecl::new("settings", title.clone(), serde_json::json!({ "title": title, "properties": properties }));
decl.files.push(d.schema);
Ok(decl)
}
fn title_ok(t: &str) -> Result<(), String> {
title(SETTINGS, t)
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct ViewDecl {
id: String,
title: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
icon: Option<String>,
#[serde(default = "right")]
location: String,
/// A web page of the package (`web/index.html`) instead of content the code renders.
#[serde(default, skip_serializing_if = "Option::is_none")]
web: Option<String>,
}
/// The files of a `web` entry, or a check that the code renders the view.
fn web_or_code(pkg: &Package, web: Option<&str>, what: &str) -> Result<Vec<PathBuf>, String> {
match web {
Some(entry) => super::web::web_files(pkg, entry, what),
None if pkg.manifest.wasm.is_none() => {
Err(format!("{what}: give a web page (`web`) or the code that renders it (`wasm`)"))
}
None => Ok(Vec::new()),
}
}
pub(super) fn view(pkg: &Package, decl: toml::Value) -> Result<UiDecl, String> {
let d: ViewDecl = decode(VIEWS, decl)?;
validate_item_id(&d.id, "view id")?;
title(VIEWS, &d.title)?;
let files = web_or_code(pkg, d.web.as_deref(), &format!("view {:?}", d.id))?;
if !["left", "right", "bottom"].contains(&d.location.as_str()) {
return Err(format!("view {:?}: `location` is left, right or bottom", d.id));
}
if d.icon.as_ref().is_some_and(|i| i.len() > 64) {
return Err(format!("view {:?}: icon names are up to 64 bytes", d.id));
}
let mut out = UiDecl::new(d.id.clone(), d.title.clone(), json(&d)?);
out.files = files;
Ok(out)
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct CustomEditorDecl {
id: String,
title: String,
patterns: Vec<String>,
#[serde(default = "default_priority")]
priority: String,
/// `tab`: the file opens in this editor's own tab. `split` / `preview`:
/// the file opens in the text editor with this editor beside it (Editor /
/// Editor and Preview / Preview); the value is the layout files start in.
#[serde(default = "default_layout")]
layout: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
web: Option<String>,
}
fn default_priority() -> String {
"default".into()
}
fn default_layout() -> String {
"tab".into()
}
/// File name globs (`*.csv`, `{a,b}.json`): no directories.
fn file_patterns(patterns: &[String], what: &str) -> Result<(), String> {
if patterns.is_empty() || patterns.len() > MAX_PATTERNS {
return Err(format!("{what}: 1–{MAX_PATTERNS} file name `patterns`"));
}
for p in patterns {
if p.is_empty() || p.len() > 128 || p.contains(['/', '\\']) || globset::Glob::new(p).is_err() {
return Err(format!("{what}: invalid file name pattern {p:?}"));
}
}
Ok(())
}
pub(super) fn custom_editor(pkg: &Package, decl: toml::Value) -> Result<UiDecl, String> {
let d: CustomEditorDecl = decode(CUSTOM_EDITORS, decl)?;
validate_item_id(&d.id, "custom editor id")?;
title(CUSTOM_EDITORS, &d.title)?;
let what = format!("custom editor {:?}", d.id);
let files = web_or_code(pkg, d.web.as_deref(), &what)?;
file_patterns(&d.patterns, &what)?;
if d.priority != "default" && d.priority != "option" {
return Err(format!("{what}: `priority` is default or option"));
}
if !["tab", "split", "preview"].contains(&d.layout.as_str()) {
return Err(format!("{what}: `layout` is tab, split or preview"));
}
let mut out = UiDecl::new(d.id.clone(), d.title.clone(), json(&d)?);
out.files = files;
Ok(out)
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
struct FileIconDecl {
id: String,
patterns: Vec<String>,
icon: PathBuf,
#[serde(default)]
icon_dark: Option<PathBuf>,
}
/// Asset id of a file icon's SVG (`dark`: its dark-theme variant).
pub fn file_icon_asset(id: &str, dark: bool) -> String {
format!("fileIcon/{id}/{}", if dark { "dark" } else { "light" })
}
/// `[[contributes.fileIcons]]`: an icon for the files matching `patterns`,
/// whatever icon theme is in use (the user's own file type icons still win).
pub(super) fn file_icon(pkg: &Package, decl: toml::Value) -> Result<UiDecl, String> {
let d: FileIconDecl = decode(FILE_ICONS, decl)?;
validate_item_id(&d.id, "file icon id")?;
let what = format!("file icon {:?}", d.id);
file_patterns(&d.patterns, &what)?;
let mut files = Vec::new();
let mut assets = Vec::new();
for (path, dark) in [(Some(&d.icon), false), (d.icon_dark.as_ref(), true)] {
let Some(path) = path else { continue };
let svg = super::icons::read_icon(pkg, path)?;
assets.push((file_icon_asset(&d.id, dark), svg));
files.push(path.clone());
}
let json = serde_json::json!({ "id": d.id, "patterns": d.patterns, "dark": d.icon_dark.is_some() });
let mut out = UiDecl::new(d.id.clone(), d.patterns.join(", "), json);
out.files = files;
out.assets = assets;
Ok(out)
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
struct FileTemplateDecl {
id: String,
name: String,
/// Without the dot.
extension: String,
/// The new file's content (`$NAME$`, `$END$` and the other template variables work).
path: PathBuf,
}
const MAX_TEMPLATE: usize = 256 << 10;
/// `[[contributes.fileTemplates]]`: an entry of the New menu.
pub(super) fn file_template(pkg: &Package, decl: toml::Value) -> Result<UiDecl, String> {
let d: FileTemplateDecl = decode(FILE_TEMPLATES, decl)?;
validate_item_id(&d.id, "file template id")?;
title(FILE_TEMPLATES, &d.name)?;
let what = format!("file template {:?}", d.id);
let ext = d.extension.trim_start_matches('.');
if ext.is_empty()
|| ext.len() > 32
|| !ext.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '-' | '_'))
{
return Err(format!("{what}: `extension` is a file extension like \"md\" ({:?})", d.extension));
}
let body = pkg.read_text(&d.path)?;
if body.len() > MAX_TEMPLATE {
return Err(format!("{what}: {} is larger than {} KiB", d.path.display(), MAX_TEMPLATE >> 10));
}
let json = serde_json::json!({ "id": d.id, "name": d.name, "extension": ext, "body": body });
let mut out = UiDecl::new(d.id.clone(), d.name.clone(), json);
out.files = vec![d.path.clone()];
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn bindings_need_a_modifier() {
for ok in ["Ctrl+Alt+C", "Meta+Shift+K", "F5", "Shift+F6", "Meta+K Meta+C", "Alt+Enter"] {
assert!(validate_binding(ok).is_ok(), "{ok}");
}
for bad in ["C", "Shift+C", "Hyper+C", "Ctrl+Ctrl+C", "Ctrl+é", "Ctrl+A B C", ""] {
assert!(validate_binding(bad).is_err(), "{bad}");
}
}
#[test]
fn settings_schemas_are_typed() {
let s = parse_settings_schema(
r#"{"properties": {
"context": {"type": "string", "title": "Context", "default": ""},
"watch": {"type": "boolean", "default": true},
"mode": {"type": "string", "enum": ["a", "b"], "default": "b"},
"limit": {"type": "integer", "default": 100}
}}"#,
)
.unwrap();
assert_eq!(s.len(), 4);
assert!(s.iter().find(|x| x.key == "mode").unwrap().accepts(&Value::from("a")));
assert!(!s.iter().find(|x| x.key == "mode").unwrap().accepts(&Value::from("c")));
assert!(parse_settings_schema(r#"{"properties": {"x": {"type": "string", "default": 3}}}"#).is_err());
assert!(parse_settings_schema(r#"{"properties": {"x": {"type": "object"}}}"#).is_err());
}
}
+92
View File
@@ -0,0 +1,92 @@
//! Declarative views and custom editors as the
//! window commands `plugin_view_render` / `plugin_view_event` serve them —
//! the desktop for local projects, `openidea-server` for remote ones.
use super::{PluginEditor, PluginHost};
use ide_core::{DocId, IdeError, IdeResult};
use ide_plugin::{EditorContext, Invocation, View, ViewEvent, Viewport};
use std::sync::Arc;
/// The window's project, plus the editor the call is about: the document a
/// custom editor shows, else (a tool window) the window's active editor, as
/// a command gets it, so a view's buttons can act on what the user is editing.
pub fn view_invocation(
host: &crate::Host,
window: &str,
doc: Option<DocId>,
editor: Option<PluginEditor>,
) -> IdeResult<Invocation> {
let project = host.project(window).ok();
let editor = match doc {
None => editor.map(PluginEditor::context),
Some(doc) => {
let project = project.as_ref().ok_or_else(|| IdeError::not_found("no project is open in this window"))?;
let info = project.documents.snapshot(doc)?;
Some(EditorContext { path: info.path.0, language: info.language.0, selections: Vec::new() })
}
};
Ok(Invocation { client: Some(window.to_owned()), workspace_root: project.as_ref().map(|p| p.root.clone()), editor })
}
async fn blocking<T: Send + 'static>(f: impl FnOnce() -> IdeResult<T> + Send + 'static) -> IdeResult<T> {
tokio::task::spawn_blocking(f).await.map_err(|e| IdeError::internal(e.to_string()))?
}
#[allow(clippy::too_many_arguments)]
pub async fn render(
host: &crate::Host,
plugins: Arc<PluginHost>,
window: &str,
plugin_id: String,
view: String,
viewport: Option<Viewport>,
doc: Option<DocId>,
editor: Option<PluginEditor>,
) -> IdeResult<View> {
let invocation = view_invocation(host, window, doc, editor)?;
blocking(move || plugins.render_view(&plugin_id, &view, viewport, invocation)).await
}
#[allow(clippy::too_many_arguments)]
pub async fn event(
host: &crate::Host,
plugins: Arc<PluginHost>,
window: &str,
plugin_id: String,
view: String,
event: ViewEvent,
doc: Option<DocId>,
editor: Option<PluginEditor>,
) -> IdeResult<()> {
let invocation = view_invocation(host, window, doc, editor)?;
blocking(move || plugins.view_event(&plugin_id, &view, event, invocation)).await
}
#[cfg(test)]
mod tests {
use super::*;
use crate::{Host, HostConfig};
#[tokio::test]
async fn a_tool_window_call_carries_the_active_editor() {
let tmp = tempfile::tempdir().unwrap();
let host = Host::new(HostConfig {
config_dir: tmp.path().join("config"),
data_dir: tmp.path().join("data"),
local_data_dir: tmp.path().join("data"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new(|_| Arc::new(ide_core::NullSink)),
owner: |w| w,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
});
let range = ide_core::Range::new(ide_core::Pos::new(1, 0), ide_core::Pos::new(1, 4));
let active = PluginEditor { path: "/p/a.rs".into(), language: "rust".into(), selections: vec![range] };
let inv = view_invocation(&host, "main", None, Some(active)).unwrap();
let editor = inv.editor.expect("the active editor");
assert_eq!((editor.path.to_string_lossy().as_ref(), editor.selections.as_slice()), ("/p/a.rs", &[range][..]));
assert_eq!(inv.client.as_deref(), Some("main"));
assert!(view_invocation(&host, "main", None, None).unwrap().editor.is_none());
}
}
+245
View File
@@ -0,0 +1,245 @@
//! Web views: a view or custom editor whose UI is a web page shipped in the
//! package (`web = "web/index.html"`) instead of content the plugin's code
//! renders.
//!
//! The entry's directory is the page's root. Every file under it is part of
//! the declaration: it is checked when the package is prepared (count, size,
//! no links leaving the package), copied on install and covered by the
//! package digest and signature. The client loads the page in a sandboxed
//! frame (an opaque origin with no network and no IDE API) and fetches its
//! files through [`PluginHost::web_file`] (the desktop's `openidea-plugin`
//! URL scheme); the page talks to the IDE only through the frame bridge.
use super::PluginHost;
use ide_core::{IdeError, IdeResult};
use ide_extension::{Package, check_relative};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::path::{Component, Path, PathBuf};
/// Most files under one web root.
const MAX_FILES: usize = 4096;
/// Most bytes under one web root.
const MAX_BYTES: u64 = 256 << 20;
/// Deepest directory nesting under a web root.
const MAX_DEPTH: usize = 16;
/// Checks a declaration's `web` entry: an `.html` file inside a directory of
/// the package (the directory is what the page can load, so it can't be the
/// package root), and everything under that directory. Returns those files,
/// package-relative.
pub(super) fn web_files(pkg: &Package, entry: &str, what: &str) -> Result<Vec<PathBuf>, String> {
let path = Path::new(entry);
check_relative(path, "web")?;
if entry.contains('\\') || !entry.to_ascii_lowercase().ends_with(".html") {
return Err(format!("{what}: `web` names an .html file, `/`-separated ({entry:?})"));
}
let dir = path.parent().filter(|d| !d.as_os_str().is_empty()).ok_or_else(|| {
format!("{what}: put the page in a directory of its own (`web/index.html`): the directory is what it can load")
})?;
let real_entry = pkg.resolve(path)?;
if !real_entry.is_file() {
return Err(format!("{what}: {entry} is not a file"));
}
let root = pkg.resolve(dir)?;
let mut files = Vec::new();
let mut total = 0u64;
let mut stack = vec![(root.clone(), dir.to_path_buf(), 0usize)];
while let Some((real, rel, depth)) = stack.pop() {
let mut entries: Vec<_> =
std::fs::read_dir(&real).map_err(|e| format!("{}: {e}", rel.display()))?.filter_map(Result::ok).collect();
entries.sort_by_key(|e| e.file_name());
for e in entries {
let name = e.file_name();
let rel = rel.join(&name);
// Links could point anywhere: only what is really in the package counts.
let ty = e.file_type().map_err(|err| format!("{}: {err}", rel.display()))?;
if ty.is_symlink() {
return Err(format!("{what}: {} is a link; web files must be real files", rel.display()));
}
if ty.is_dir() {
if depth + 1 > MAX_DEPTH {
return Err(format!("{what}: {} is nested deeper than {MAX_DEPTH} directories", rel.display()));
}
stack.push((e.path(), rel, depth + 1));
continue;
}
if name.to_string_lossy() == ".DS_Store" {
continue;
}
let len = e.metadata().map_err(|err| format!("{}: {err}", rel.display()))?.len();
if len > ide_extension::MAX_PACKAGE_FILE {
return Err(format!(
"{what}: {} is larger than {} MiB",
rel.display(),
ide_extension::MAX_PACKAGE_FILE >> 20
));
}
total += len;
files.push(rel);
if files.len() > MAX_FILES || total > MAX_BYTES {
return Err(format!(
"{what}: {} holds more than {MAX_FILES} files or {} MiB",
dir.display(),
MAX_BYTES >> 20
));
}
}
}
files.sort();
Ok(files)
}
/// The page of a web view, for the client to load in its frame.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct WebPage {
/// `<plugin id>/<revision>/<entry>` under the `openidea-plugin` scheme. The
/// revision changes with the installed package, so a cached file is never
/// stale; relative URLs in the page resolve to the package's files.
pub path: String,
}
/// A file of a web view.
pub struct WebFile {
pub bytes: Vec<u8>,
pub mime: &'static str,
}
impl PluginHost {
/// The web roots `plugin` declares (entry directories, package-relative, `/`-terminated).
fn web_dirs(&self, plugin: &str) -> Vec<String> {
[super::ui_points::VIEWS, super::ui_points::CUSTOM_EDITORS]
.iter()
.flat_map(|point| self.ui.items(point, Some(plugin)))
.filter_map(|item| item.decl.get("web").and_then(|w| w.as_str()).map(str::to_owned))
.filter_map(|entry| entry.rsplit_once('/').map(|(dir, _)| format!("{dir}/")))
.collect()
}
/// Changes whenever the installed package does (cache key of its web files).
fn web_revision(&self, plugin: &super::LoadedPlugin) -> String {
let digest = plugin.pkg.digest.as_deref().unwrap_or_default();
let short: String = digest.chars().filter(char::is_ascii_alphanumeric).take(16).collect();
if short.is_empty() { format!("v{}", plugin.pkg.manifest.version) } else { short }
}
/// The page of `plugin`'s view or custom editor `view`.
pub fn web_page(&self, plugin_id: &str, view: &str) -> IdeResult<WebPage> {
let plugin = self.get(plugin_id)?;
if !plugin.status().is_live() {
return Err(IdeError::unavailable(format!("plugin {plugin_id:?} is not enabled")));
}
let entry = [super::ui_points::VIEWS, super::ui_points::CUSTOM_EDITORS]
.iter()
.flat_map(|point| self.ui.items(point, Some(plugin_id)))
.find(|item| item.key == view)
.and_then(|item| item.decl.get("web").and_then(|w| w.as_str()).map(str::to_owned))
.ok_or_else(|| IdeError::not_found(format!("plugin {plugin_id:?} has no web view {view:?}")))?;
Ok(WebPage { path: format!("{plugin_id}/{}/{entry}", self.web_revision(&plugin)) })
}
/// A file of one of `plugin`'s web views: `path` is `<revision>/<package path>`
/// as [`WebPage::path`] makes them. Only files under a declared web root
/// of a live plugin are served; the revision must be the installed one.
pub fn web_file(&self, plugin_id: &str, path: &str) -> IdeResult<WebFile> {
let plugin = self.get(plugin_id)?;
if !plugin.status().is_live() {
return Err(IdeError::unavailable(format!("plugin {plugin_id:?} is not enabled")));
}
let (revision, rel) = path.split_once('/').ok_or_else(|| IdeError::not_found(path.to_owned()))?;
if revision != self.web_revision(&plugin) {
return Err(IdeError::not_found(format!("{path}: not the installed version")));
}
let rel_path = Path::new(rel);
if !rel_path.components().all(|c| matches!(c, Component::Normal(_))) {
return Err(IdeError::invalid(format!("{rel}: not a plain relative path")));
}
if !self.web_dirs(plugin_id).iter().any(|dir| rel.starts_with(dir.as_str())) {
return Err(IdeError::not_found(format!("{rel}: not part of a web view")));
}
let bytes = plugin.pkg.read_bytes(rel_path).map_err(IdeError::not_found)?;
Ok(WebFile { bytes, mime: mime(rel) })
}
}
/// Content type by extension; unknown files are opaque bytes (never sniffed as script).
pub fn mime(path: &str) -> &'static str {
let ext = path.rsplit_once('.').map(|(_, e)| e.to_ascii_lowercase()).unwrap_or_default();
match ext.as_str() {
"html" | "htm" => "text/html; charset=utf-8",
"js" | "mjs" | "cjs" => "text/javascript; charset=utf-8",
"css" => "text/css; charset=utf-8",
"json" | "map" => "application/json",
"wasm" => "application/wasm",
"svg" => "image/svg+xml",
"png" => "image/png",
"jpg" | "jpeg" => "image/jpeg",
"gif" => "image/gif",
"webp" => "image/webp",
"avif" => "image/avif",
"ico" => "image/x-icon",
"woff2" => "font/woff2",
"woff" => "font/woff",
"ttf" => "font/ttf",
"otf" => "font/otf",
"txt" | "md" => "text/plain; charset=utf-8",
"mp3" => "audio/mpeg",
"mp4" => "video/mp4",
"webm" => "video/webm",
_ => "application/octet-stream",
}
}
#[cfg(test)]
mod tests {
use super::*;
use ide_extension::Manifest;
fn pkg(dir: &Path) -> Package {
let src = "id = \"p\"\nname = \"P\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"\n";
Package::new(Manifest::parse(src).unwrap(), dir.to_path_buf())
}
#[test]
fn a_web_root_is_its_directory_and_nothing_else() {
let tmp = tempfile::tempdir().unwrap();
let d = tmp.path();
std::fs::create_dir_all(d.join("web/assets/fonts")).unwrap();
std::fs::write(d.join("web/index.html"), "<p>hi</p>").unwrap();
std::fs::write(d.join("web/assets/app.js"), "1").unwrap();
std::fs::write(d.join("web/assets/fonts/a.woff2"), "f").unwrap();
std::fs::write(d.join("web/.DS_Store"), "x").unwrap();
std::fs::write(d.join("secret.txt"), "no").unwrap();
let files = web_files(&pkg(d), "web/index.html", "v").unwrap();
let files: Vec<String> = files.iter().map(|f| f.to_string_lossy().replace('\\', "/")).collect();
assert_eq!(files, ["web/assets/app.js", "web/assets/fonts/a.woff2", "web/index.html"]);
std::fs::write(d.join("index.html"), "x").unwrap();
let err = |entry: &str| web_files(&pkg(d), entry, "v").err().unwrap_or_default();
assert!(err("index.html").contains("directory of its own"), "{}", err("index.html"));
assert!(err("web/app.js").contains(".html"));
assert!(err("../web/index.html").contains("relative"));
assert!(err("web/missing.html").contains("no such file"));
}
#[cfg(unix)]
#[test]
fn links_out_of_a_web_root_are_refused() {
let tmp = tempfile::tempdir().unwrap();
let d = tmp.path();
std::fs::create_dir_all(d.join("web")).unwrap();
std::fs::write(d.join("web/index.html"), "x").unwrap();
std::os::unix::fs::symlink("/etc/hosts", d.join("web/hosts")).unwrap();
assert!(web_files(&pkg(d), "web/index.html", "v").unwrap_err().contains("is a link"));
}
#[test]
fn content_types_never_make_unknown_files_scripts() {
assert_eq!(mime("a/b.JS"), "text/javascript; charset=utf-8");
assert_eq!(mime("x.woff2"), "font/woff2");
assert_eq!(mime("x.wasm"), "application/wasm");
assert_eq!(mime("x.bin"), "application/octet-stream");
assert_eq!(mime("noext"), "application/octet-stream");
}
}
+1
View File
@@ -305,6 +305,7 @@ impl Host {
index.update(&changed, &batch.removed);
kotlin_bridge.files_changed(changed.iter().chain(&batch.removed));
lsp.project_files_changed(batch.created.iter().chain(&batch.removed));
lsp.watched_files_changed(&batch.created, &batch.modified, &batch.removed);
// Atomic saves by external tools surface as `created`.
documents.on_disk_changed(&changed);
// A framework's root marker (`angular.json`) created or deleted: its
+6 -1
View File
@@ -9,7 +9,10 @@
//! * server → client notification `event` `{ window, event }` (an `IdeEvent`
//! for that window, or every window when `window` is null);
//! * server → client notification `channel` `{ id, data }`: one value of a
//! streaming argument.
//! streaming argument;
//! * server → client notification `plugin` `{ window, event }` and request
//! `plugin_secret` `{ plugin, name }` → `string | null`: code plugins of the
//! remote project (`plugins::remote_host`).
use crate::Host;
use crate::ops::{self, Emit};
@@ -254,6 +257,8 @@ impl Dispatcher {
&c.arg::<Option<Vec<String>>>("removedDetectedIds")?.unwrap_or_default(),
)
})
.add("run_without_derived", |c| async move { ops::run::without_derived(&c.host, &c.window, c.arg("configs")?) })
.add("run_apply_settings", |c| async move { ops::run::apply_settings(&c.host, &c.window, c.arg("configs")?) })
.add("run_system_env", |_| async move { Ok(ide_run::system_env()) })
.add("run_start", |c| async move {
let on_output = c.channel("onOutput")?;
+12
View File
@@ -177,6 +177,10 @@ pub(crate) fn read_login_env(shell: &str, timeout: std::time::Duration) -> Optio
}
}
}
// Nor what the last read found that this process lacks: children get it, the shell must not.
for (k, _) in ide_core::process::shell_env() {
cmd.env_remove(k);
}
ide_core::process::prepare(&mut cmd);
let mut child = cmd
.args(["-ilc", &format!("printf '{MARK}'; /usr/bin/env -0; printf '{MARK}'")])
@@ -257,6 +261,14 @@ mod tests {
std::fs::write(&rc, "export OPENIDEA_TEST_RC='two words'\n").unwrap();
let second = read_login_env(&shell, Duration::from_secs(10)).unwrap();
assert_eq!(get(&second, "OPENIDEA_TEST_RC").as_deref(), Some("two words"));
// What a read found goes to children, not into the next read: an rc file that no
// longer exports a variable drops it.
ide_core::process::set_shell_env(second);
std::fs::write(&rc, "export OPENIDEA_TEST_SEEN=\"${OPENIDEA_TEST_RC:-unset}\"\n").unwrap();
let third = read_login_env(&shell, Duration::from_secs(10)).unwrap();
assert_eq!(get(&third, "OPENIDEA_TEST_RC"), None);
assert_eq!(get(&third, "OPENIDEA_TEST_SEEN").as_deref(), Some("unset"));
}
#[test]
+1
View File
@@ -273,6 +273,7 @@ pub fn restricted_command_allowed(cmd: &str) -> bool {
| "plugin_set_theme"
| "plugin_set_theme_frame"
| "plugin_catalog"
| "plugin_assets"
| "notebook_open"
| "notebook_get"
| "notebook_edit"
+489
View File
@@ -0,0 +1,489 @@
//! Versioned installs with automatic rollback, signed packages from
//! a static marketplace index, and revocation.
use ide_extension::package::{self, Index, IndexEntry, KeyEntry, Publisher, PublisherKey, Revocation};
use ide_host::plugins::*;
use ide_plugin::{Invocation, Limits};
use parking_lot::Mutex;
use std::path::{Path, PathBuf};
use std::sync::Arc;
type Events = Arc<Mutex<Vec<PluginEvent>>>;
fn repo() -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR")).join("../..")
}
fn copy_dir(from: &Path, to: &Path) {
std::fs::create_dir_all(to).unwrap();
for entry in std::fs::read_dir(from).unwrap().flatten() {
let (src, dst) = (entry.path(), to.join(entry.file_name()));
if entry.file_name() == "target" {
continue;
}
if src.is_dir() { copy_dir(&src, &dst) } else { std::fs::copy(&src, &dst).map(|_| ()).unwrap() }
}
}
/// A copy of `from` (under `dir/name`) whose manifest is rewritten by `f`.
fn variant(from: &Path, dir: &Path, name: &str, f: impl Fn(&str) -> String) -> PathBuf {
let pkg = dir.join(name);
copy_dir(from, &pkg);
let manifest = pkg.join("plugin.toml");
std::fs::write(&manifest, f(&std::fs::read_to_string(&manifest).unwrap())).unwrap();
pkg
}
struct Fixture {
host: PluginHost,
plugins_dir: PathBuf,
events: Events,
_tmp: tempfile::TempDir,
work: PathBuf,
}
fn fixture() -> Fixture {
let tmp = tempfile::tempdir().unwrap();
let plugins_dir = tmp.path().join("plugins");
let events: Events = Arc::default();
let sink = {
let events = events.clone();
move |_: Option<&str>, e: PluginEvent| events.lock().push(e)
};
let host = PluginHost::new(plugins_dir.clone(), Arc::new(sink), Limits::default()).unwrap();
host.load_all();
let work = tmp.path().join("work");
std::fs::create_dir_all(&work).unwrap();
Fixture { host, plugins_dir, events, work, _tmp: tmp }
}
fn inv() -> Invocation {
Invocation { client: Some("main".into()), workspace_root: None, editor: None }
}
fn misbehave() -> PathBuf {
repo().join("crates/ide-plugin/tests/fixtures/misbehave")
}
/// P8 acceptance: an update that fails to start rolls back to the version it replaced.
#[test]
fn a_failing_update_rolls_back_automatically() {
let f = fixture();
let v1 = variant(&misbehave(), &f.work, "v1", |t| t.replace("version = \"0.0.2\"", "version = \"1.0.0\""));
f.host.install_from_dir(&v1, &Consent::all()).unwrap();
assert_eq!(f.host.run_command("misbehave", "echo", Some("hi"), inv()).unwrap().as_deref(), Some("hi"));
let broken =
variant(&misbehave(), &f.work, "v2", |t| t.replace("version = \"0.0.2\"", "version = \"2.0.0-broken\""));
let info = f.host.install_from_dir(&broken, &Consent::all()).unwrap();
assert_eq!((info.version.as_str(), info.previous_version.as_deref()), ("2.0.0-broken", Some("1.0.0")));
let root = f.plugins_dir.join("misbehave");
assert!(root.join("2.0.0-broken").is_dir() && root.join("1.0.0").is_dir(), "versions side by side");
// Its first start fails: back to 1.0.0, without the user doing anything.
f.events.lock().clear();
let err = f.host.run_command("misbehave", "echo", Some("x"), inv()).unwrap_err();
assert!(err.message.contains("version 1.0.0 was restored"), "{}", err.message);
let info = f.host.info_of("misbehave").unwrap();
assert_eq!((info.version.as_str(), info.status), ("1.0.0", PluginStatus::Inactive));
assert!(!root.join("2.0.0-broken").exists(), "the failed version is removed");
assert!(
f.events
.lock()
.iter()
.any(|e| matches!(e, PluginEvent::Notification { title, .. } if title.contains("rolled back")))
);
assert_eq!(f.host.run_command("misbehave", "echo", Some("ok"), inv()).unwrap().as_deref(), Some("ok"));
// A working update is confirmed by its first start; the user can still roll it back.
let v3 = variant(&misbehave(), &f.work, "v3", |t| t.replace("version = \"0.0.2\"", "version = \"3.0.0\""));
f.host.install_from_dir(&v3, &Consent::all()).unwrap();
f.host.run_command("misbehave", "echo", None, inv()).unwrap();
assert_eq!(f.host.info_of("misbehave").unwrap().version, "3.0.0");
let info = f.host.roll_back_update("misbehave").unwrap();
assert_eq!((info.version.as_str(), info.previous_version.as_deref()), ("1.0.0", None));
assert!(f.host.roll_back_update("misbehave").is_err(), "nothing older to go back to");
// A restart loads the active version.
let again = PluginHost::new(f.plugins_dir.clone(), Arc::new(|_: Option<&str>, _| {}), Limits::default()).unwrap();
again.load_all();
assert_eq!(again.info_of("misbehave").unwrap().version, "1.0.0");
f.host.uninstall("misbehave").unwrap();
assert!(!root.exists(), "every version goes");
}
#[test]
fn installs_from_before_versioning_keep_working_and_move_on_update() {
let f = fixture();
// The flat layout of earlier releases.
copy_dir(&misbehave(), &f.plugins_dir.join("misbehave"));
let again = PluginHost::new(f.plugins_dir.clone(), Arc::new(|_: Option<&str>, _| {}), Limits::default()).unwrap();
again.load_all();
assert_eq!(again.info_of("misbehave").unwrap().version, "0.0.2");
let v1 = variant(&misbehave(), &f.work, "v1", |t| t.replace("version = \"0.0.2\"", "version = \"1.0.0\""));
let info = again.install_from_dir(&v1, &Consent::all()).unwrap();
assert_eq!((info.version.as_str(), info.previous_version.as_deref()), ("1.0.0", Some("0.0.2")));
assert!(f.plugins_dir.join("misbehave/0.0.2/plugin.toml").is_file());
assert!(f.plugins_dir.join("misbehave/1.0.0/plugin.toml").is_file());
}
// ---- marketplace ------------------------------------------------------------------
/// `hello` as `acme.hello` at `version`, packed and signed with `key`.
fn signed_hello(dir: &Path, version: &str, key: &PublisherKey, host: &PluginHost) -> (PathBuf, IndexEntry) {
signed(dir, "acme", version, key, host, |t| t.to_owned())
}
/// `hello` as `<publisher>.hello` at `version`, its manifest changed by
/// `edit`, packed and signed with `key`.
fn signed(
dir: &Path,
publisher: &str,
version: &str,
key: &PublisherKey,
host: &PluginHost,
edit: impl Fn(&str) -> String,
) -> (PathBuf, IndexEntry) {
let id = format!("{publisher}.hello");
let src =
variant(&repo().join("crates/ide-plugin/tests/fixtures/hello"), dir, &format!("src-{id}-{version}"), |t| {
edit(
&t.replace("id = \"hello\"", &format!("id = \"{id}\""))
.replace("version = \"0.3.0\"", &format!("version = \"{version}\"")),
)
});
let files = host.package_files_of(&src).unwrap();
let digest = package::package_digest(&src, &files).unwrap();
let sig = key.sign(publisher, &id, version, &digest);
let out = dir.join(format!("{id}-{version}.oiplugin"));
package::pack(&src, &files, &out, Some(&sig)).unwrap();
let entry = IndexEntry {
id: id.clone(),
name: "Hello".into(),
version: version.into(),
description: String::new(),
authors: vec![],
publisher: publisher.into(),
download: out.file_name().unwrap().to_string_lossy().into_owned(),
digest,
capabilities: vec![],
engine: "^1.0".into(),
categories: vec!["tools".into()],
};
(out, entry)
}
fn write_index(
dir: &Path,
key: &PublisherKey,
plugins: Vec<IndexEntry>,
revoked: Vec<Revocation>,
key_revoked: bool,
) -> String {
let index = Index {
format: package::INDEX_FORMAT,
publishers: vec![Publisher {
id: "acme".into(),
name: "ACME".into(),
keys: vec![KeyEntry { revoked: key_revoked, ..KeyEntry::new(key.public_base64()) }],
}],
plugins,
revoked,
moved: None,
};
let path = dir.join("index.json");
std::fs::write(&path, serde_json::to_string(&index).unwrap()).unwrap();
path.to_string_lossy().into_owned()
}
#[test]
fn signed_marketplace_plugins_install_and_revocations_disable_them() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, entry) = signed_hello(&market, "1.0.0", &key, &f.host);
let url = write_index(&market, &key, vec![entry.clone()], vec![], false);
assert!(f.host.refresh_market().is_err(), "no marketplace configured yet");
f.host.set_market_url(Some(url.clone()));
let listed = f.host.refresh_market().unwrap();
assert_eq!((listed[0].entry.id.as_str(), listed[0].installed.as_deref()), ("acme.hello", None));
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert_eq!((preview.unsigned, preview.publisher.as_deref()), (false, Some("acme")));
let info = f.host.install_from_dir(&dir, &Consent::all()).unwrap();
assert_eq!(info.publisher.as_deref(), Some("acme"));
assert_eq!(f.host.market_plugins()[0].installed.as_deref(), Some("1.0.0"));
// A changed file breaks the signature: refused.
let tampered = f.host.download_market_plugin("acme.hello").unwrap();
let manifest = std::fs::read_to_string(tampered.join("plugin.toml")).unwrap();
std::fs::write(tampered.join("plugin.toml"), manifest.replace("Sample plugin", "Totally safe plugin")).unwrap();
let err = f.host.preview_install(&tampered).unwrap_err();
assert!(err.message.contains("changed after signing"), "{}", err.message);
// Someone else's key, or another publisher's prefix: not trusted.
let stranger = PublisherKey::from_seed([4; 32]);
let (foreign, _) = signed_hello(&f.work, "1.0.1", &stranger, &f.host);
let unpacked = f.host.unpack_package(&foreign).unwrap();
assert!(f.host.preview_install(&unpacked).unwrap_err().message.contains("not one of"));
// The index revokes the installed version: disabled, the user told.
f.events.lock().clear();
let revoked =
vec![Revocation { id: "acme.hello".into(), versions: vec!["1.0.0".into()], reason: "steals tokens".into() }];
write_index(&market, &key, vec![entry.clone()], revoked, false);
f.host.refresh_market().unwrap();
let info = f.host.info_of("acme.hello").unwrap();
assert_eq!((info.status, info.error.as_deref()), (PluginStatus::Failed, Some("revoked: steals tokens")));
assert!(
f.events
.lock()
.iter()
.any(|e| matches!(e, PluginEvent::Notification { body, .. } if body.contains("steals tokens")))
);
assert!(f.host.market_plugins().is_empty(), "revoked versions are not offered");
assert!(f.host.install_from_dir(&dir, &Consent::all()).is_err(), "nor installable");
f.host.set_enabled("acme.hello", true).unwrap();
assert_eq!(f.host.info_of("acme.hello").unwrap().status, PluginStatus::Failed, "enabling doesn't bring it back");
// Revocations hold across restarts (the index is kept).
let again = PluginHost::new(f.plugins_dir.clone(), Arc::new(|_: Option<&str>, _| {}), Limits::default()).unwrap();
again.load_all();
assert_eq!(again.info_of("acme.hello").unwrap().status, PluginStatus::Failed);
// A revoked signing key disables what it signed.
write_index(&market, &key, vec![entry.clone()], vec![], true);
f.host.refresh_market().unwrap();
let info = f.host.info_of("acme.hello").unwrap();
assert!(info.error.as_deref().unwrap_or("").contains("signing key was revoked"), "{:?}", info.error);
// Lifted: it runs again.
write_index(&market, &key, vec![entry], vec![], false);
f.host.refresh_market().unwrap();
let info = f.host.info_of("acme.hello").unwrap();
assert_eq!((info.status, info.error), (PluginStatus::Inactive, None));
}
#[test]
fn package_files_install_from_disk_unsigned() {
let f = fixture();
let src = variant(&misbehave(), &f.work, "pkg", |t| t.to_owned());
let files = f.host.package_files_of(&src).unwrap();
let out = f.work.join("misbehave-0.0.2.oiplugin");
package::pack(&src, &files, &out, None).unwrap();
let dir = f.host.unpack_package(&out).unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert_eq!((preview.unsigned, preview.publisher), (true, None));
let info = f.host.install_from_dir(&dir, &Consent::all()).unwrap();
assert_eq!((info.status, info.publisher), (PluginStatus::Inactive, None));
}
#[test]
fn the_marketplace_offers_the_newest_version_this_ide_can_load() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, old) = signed_hello(&market, "1.0.0", &key, &f.host);
let (_, new) = signed_hello(&market, "2.0.0", &key, &f.host);
let next_major = ide_extension::ENGINE_VERSION.split('.').next().unwrap().parse::<u64>().unwrap() + 1;
let new = IndexEntry { engine: format!("^{next_major}.0"), ..new };
f.host.set_market_url(Some(write_index(&market, &key, vec![old.clone(), new.clone()], vec![], false)));
let listed = f.host.refresh_market().unwrap();
assert_eq!((listed.len(), listed[0].entry.version.as_str(), listed[0].compatible), (1, "1.0.0", true));
let dir = f.host.download_market_plugin("acme.hello").unwrap();
assert_eq!(f.host.preview_install(&dir).unwrap().version, "1.0.0");
// Only a version for a newer IDE: shown as such, not downloaded.
write_index(&market, &key, vec![new], vec![], false);
let listed = f.host.refresh_market().unwrap();
assert_eq!((listed[0].entry.version.as_str(), listed[0].compatible), ("2.0.0", false));
let err = f.host.download_market_plugin("acme.hello").unwrap_err();
assert!(err.message.contains("update OpenIDEA"), "{}", err.message);
}
#[test]
fn a_moved_marketplace_is_read_where_it_moved_to() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, entry) = signed_hello(&market, "1.0.0", &key, &f.host);
write_index(&market, &key, vec![entry], vec![], false);
// The old address points at the new one; downloads resolve against where the index is.
let old = f.work.join("old");
std::fs::create_dir_all(&old).unwrap();
std::fs::write(old.join("index.json"), r#"{"format": 1, "moved": "../market/index.json"}"#).unwrap();
f.host.set_market_url(Some(old.join("index.json").to_string_lossy().into_owned()));
let listed = f.host.refresh_market().unwrap();
assert_eq!(listed[0].entry.id, "acme.hello");
let dir = f.host.download_market_plugin("acme.hello").unwrap();
assert_eq!(f.host.preview_install(&dir).unwrap().version, "1.0.0");
// A loop gives up.
std::fs::write(old.join("index.json"), r#"{"format": 1, "moved": "index.json"}"#).unwrap();
let err = f.host.refresh_market().unwrap_err();
assert!(err.message.contains("moved more than"), "{}", err.message);
}
/// `dir/index.json` listing `publishers` and `plugins`; its path.
fn index_of(dir: &Path, publishers: Vec<Publisher>, plugins: Vec<IndexEntry>) -> String {
let index = Index { format: package::INDEX_FORMAT, publishers, plugins, ..Index::default() };
let path = dir.join("index.json");
std::fs::write(&path, serde_json::to_string(&index).unwrap()).unwrap();
path.to_string_lossy().into_owned()
}
fn publisher(id: &str, keys: Vec<KeyEntry>) -> Publisher {
Publisher { id: id.into(), name: id.into(), keys }
}
fn updates_announced(events: &Events) -> Vec<Vec<String>> {
events
.lock()
.iter()
.filter_map(|e| match e {
PluginEvent::UpdatesAvailable { updates } => {
Some(updates.iter().map(|u| format!("{} {}", u.id, u.version)).collect())
}
_ => None,
})
.collect()
}
/// What the install page sends for an update it doesn't need to show: the grants the installed version has.
fn as_granted(preview: &InstallPreview) -> Consent {
let optional = preview.capabilities.iter().filter(|c| c.optional && c.granted).map(|c| c.id.clone()).collect();
Consent { optional, signer_change: false }
}
#[test]
fn updates_keep_their_signer_and_need_no_consent_when_nothing_is_new() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let (k1, k2, k3) =
(PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([2; 32]), PublisherKey::from_seed([3; 32]));
let same = |t: &str| t.to_owned();
let (_, v1) = signed(&market, "acme", "1.0.0", &k1, &f.host, same);
f.host.set_market_url(Some(index_of(
&market,
vec![publisher("acme", vec![KeyEntry::new(k1.public_base64())])],
vec![v1.clone()],
)));
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let first = f.host.preview_install(&dir).unwrap();
assert!(first.needs_consent && !first.signer_changed, "a first install is confirmed");
f.host.install_from_dir(&dir, &Consent::default()).unwrap();
assert!(updates_announced(&f.events).is_empty());
// 1.1.0, same key and needs: announced once, installed with the grants it has.
let (_, v11) = signed(&market, "acme", "1.1.0", &k1, &f.host, same);
let keys1 = vec![KeyEntry::new(k1.public_base64())];
index_of(&market, vec![publisher("acme", keys1.clone())], vec![v1.clone(), v11.clone()]);
f.host.refresh_market().unwrap();
f.host.refresh_market().unwrap();
assert_eq!(updates_announced(&f.events), [["acme.hello 1.1.0"]], "once per set of versions");
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert_eq!(preview.replaces.as_deref(), Some("1.0.0"));
assert!(!preview.needs_consent && !preview.signer_changed);
assert!(preview.capabilities.iter().all(|c| c.granted), "shows what the installed version was granted");
f.host.install_from_dir(&dir, &as_granted(&preview)).unwrap();
assert!(f.host.market_plugins().iter().all(|p| !p.is_update()));
// 1.2.0 asks for something new: the user is asked.
let (_, v12) = signed(&market, "acme", "1.2.0", &k1, &f.host, |t| {
t.replace(
"required = [\"editor.read\", \"editor.write\"]",
"required = [\"editor.read\", \"editor.write\"]\noptional = [\"net:example.com\"]",
)
});
index_of(&market, vec![publisher("acme", keys1.clone())], vec![v12.clone()]);
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert!(preview.needs_consent && !preview.signer_changed);
f.host.install_from_dir(&dir, &Consent::default()).unwrap();
// 1.3.0 signed with a key the installed one didn't endorse: another signer, the user's call.
let (_, v13) = signed(&market, "acme", "1.3.0", &k2, &f.host, same);
let keys12 = vec![KeyEntry::new(k1.public_base64()), KeyEntry::new(k2.public_base64())];
index_of(&market, vec![publisher("acme", keys12)], vec![v13]);
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert!(preview.signer_changed && preview.needs_consent);
let err = f.host.install_from_dir(&dir, &as_granted(&preview)).unwrap_err();
assert!(err.message.contains("another key"), "{}", err.message);
assert_eq!(f.host.info_of("acme.hello").unwrap().version, "1.2.0", "nothing changed");
f.host.install_from_dir(&dir, &Consent { signer_change: true, ..as_granted(&preview) }).unwrap();
// 1.4.0 signed with k3, which k2 endorsed: a rotation the installed signer vouched for.
let (_, v14) = signed(&market, "acme", "1.4.0", &k3, &f.host, same);
let keys = vec![
KeyEntry { endorsement: Some(k2.endorse("acme", &k3.public_base64())), ..KeyEntry::new(k3.public_base64()) },
KeyEntry::new(k2.public_base64()),
];
index_of(&market, vec![publisher("acme", keys)], vec![v14]);
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert!(!preview.signer_changed && !preview.needs_consent);
f.host.install_from_dir(&dir, &as_granted(&preview)).unwrap();
assert_eq!(f.host.info_of("acme.hello").unwrap().version, "1.4.0");
}
#[test]
fn an_index_in_a_newer_format_still_revokes() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, entry) = signed_hello(&market, "1.0.0", &key, &f.host);
f.host.set_market_url(Some(write_index(&market, &key, vec![entry], vec![], false)));
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
f.host.install_from_dir(&dir, &Consent::all()).unwrap();
let newer = format!(
r#"{{"format": {}, "plugins": "a shape this IDE can't read",
"revoked": [{{"id": "acme.hello", "reason": "steals tokens"}}]}}"#,
package::INDEX_FORMAT + 1
);
std::fs::write(market.join("index.json"), newer).unwrap();
let err = f.host.refresh_market().unwrap_err();
assert!(err.message.contains("newer OpenIDEA"), "{}", err.message);
let info = f.host.info_of("acme.hello").unwrap();
assert_eq!((info.status, info.error.as_deref()), (PluginStatus::Failed, Some("revoked: steals tokens")));
assert!(f.host.market_plugins().is_empty());
}
#[test]
fn official_plugins_need_the_key_this_ide_was_built_with() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let (root, intruder) = (PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([6; 32]));
f.host.set_official_keys(vec![root.public_base64()]);
// The index lists the intruder's key for the official publisher.
let (_, forged) = signed(&market, OFFICIAL_PUBLISHER, "1.0.0", &intruder, &f.host, |t| t.to_owned());
let keys = vec![KeyEntry::new(intruder.public_base64())];
f.host.set_market_url(Some(index_of(&market, vec![publisher(OFFICIAL_PUBLISHER, keys.clone())], vec![forged])));
f.host.refresh_market().unwrap();
let err = f.host.download_market_plugin("openidea.hello").unwrap_err();
assert!(err.message.contains("trusted publisher"), "{}", err.message);
// Signed with the built-in key it installs, whatever keys the index lists.
let (_, genuine) = signed(&market, OFFICIAL_PUBLISHER, "1.0.1", &root, &f.host, |t| t.to_owned());
index_of(&market, vec![publisher(OFFICIAL_PUBLISHER, keys)], vec![genuine]);
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("openidea.hello").unwrap();
assert_eq!(f.host.preview_install(&dir).unwrap().publisher.as_deref(), Some(OFFICIAL_PUBLISHER));
}
File diff suppressed because it is too large. Load diff
+304
View File
@@ -0,0 +1,304 @@
//! Code plugins of a remote project run on the
//! server. The desktop's plugin host syncs its live
//! code plugins over the remote protocol; Checkstyle then checks the file on
//! the server's disk when it is saved, and secrets come from the desktop.
#![cfg(unix)]
use ide_core::IdeEvent;
use ide_host::plugins::remote_host::{
EVENT_NOTIFICATION, PackageSyncReport, PluginEventMessage, RemotePlugins, SECRET_REQUEST, SYNC_PACKAGES,
SecretRequest,
};
use ide_host::plugins::{Consent, PluginEvent, PluginHost, PluginStatus, WorkbenchEvent};
use ide_host::rpc::{self, Dispatcher, EventMessage, Invoke};
use ide_host::{Host, HostConfig};
use ide_jsonrpc::{Handler, Peer};
use ide_plugin::Limits;
use parking_lot::Mutex;
use serde_json::{Value, json};
use std::os::unix::fs::PermissionsExt;
use std::path::{Path, PathBuf};
use std::sync::{Arc, OnceLock};
use std::time::Duration;
#[derive(Default)]
struct Client {
events: Mutex<Vec<EventMessage>>,
plugin_events: Mutex<Vec<PluginEventMessage>>,
secret_requests: Mutex<Vec<SecretRequest>>,
}
impl Handler for Client {
fn on_notification(&self, method: &str, params: Value) {
match method {
"event" => self.events.lock().push(serde_json::from_value(params).unwrap()),
EVENT_NOTIFICATION => self.plugin_events.lock().push(serde_json::from_value(params).unwrap()),
_ => {}
}
}
fn on_request(&self, method: &str, params: Value) -> Result<Value, (i64, String)> {
if method == SECRET_REQUEST {
let r: SecretRequest = serde_json::from_value(params).unwrap();
let value = (r.name == "api-key").then(|| "sk-desktop".to_owned());
self.secret_requests.lock().push(r);
return Ok(serde_json::to_value(value).unwrap());
}
Err((ide_jsonrpc::codes::METHOD_NOT_FOUND, method.to_owned()))
}
}
async fn invoke(peer: &Peer, cmd: &str, args: Value) -> Result<Value, ide_core::IdeError> {
let call = Invoke { window: "main".into(), cmd: cmd.into(), args };
peer.request::<_, Value>("invoke", call, None).await.map_err(rpc::error_from_reply)
}
async fn sync(peer: &Peer, desktop: &PluginHost, files: &[&str]) -> PackageSyncReport {
let files: serde_json::Map<String, Value> = files
.iter()
.map(|id| ((*id).to_owned(), serde_json::to_value(desktop.remote_package_files(id).unwrap()).unwrap()))
.collect();
let args = json!({ "packages": desktop.remote_packages(), "files": files });
serde_json::from_value(invoke(peer, SYNC_PACKAGES, args).await.unwrap()).unwrap()
}
fn official(name: &str) -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR")).join("../../plugins/official").join(name)
}
/// A test fixture plugin (`crates/ide-plugin/tests/fixtures/<name>`).
fn fixture(name: &str) -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR")).join("../ide-plugin/tests/fixtures").join(name)
}
/// A `checkstyle` that reports a tab on line 2 of the file it is given.
fn fake_checkstyle(bin: &Path) {
std::fs::create_dir_all(bin).unwrap();
std::fs::write(
bin.join("checkstyle"),
"#!/bin/sh\nfor a; do file=\"$a\"; done\ngrep -q '\t' \"$file\" || { echo '<checkstyle/>'; exit 0; }\n\
echo '<?xml version=\"1.0\"?><checkstyle><file name=\"'\"$file\"'\">'\n\
echo '<error line=\"2\" column=\"1\" severity=\"warning\" message=\"Line contains a tab character.\" source=\"com.puppycrawl.tools.checkstyle.checks.whitespace.FileTabCharacterCheck\"/>'\n\
echo '</file></checkstyle>'\nexit 1\n",
)
.unwrap();
std::fs::set_permissions(bin.join("checkstyle"), std::fs::Permissions::from_mode(0o755)).unwrap();
}
async fn until(what: &str, secs: u64, mut done: impl FnMut() -> bool) {
for _ in 0..secs * 50 {
if done() {
return;
}
tokio::time::sleep(Duration::from_millis(20)).await;
}
panic!("timed out waiting for {what}");
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn remote_projects_run_the_desktops_code_plugins_on_the_server() {
let tmp = tempfile::tempdir().unwrap();
let bin = tmp.path().join("bin");
fake_checkstyle(&bin);
let path_var = {
let bin = bin.clone();
Arc::new(move || Some(bin.clone().into_os_string()))
};
// The desktop: Checkstyle and AI Completion installed and granted.
let desktop =
PluginHost::new(tmp.path().join("desktop/plugins"), Arc::new(|_: Option<&str>, _| {}), Limits::default())
.unwrap();
desktop.set_program_path(path_var.clone());
for name in ["checkstyle", "ai-completion"] {
let info = desktop.install_from_dir(&official(name), &Consent::all()).unwrap();
assert_eq!(info.status, PluginStatus::Inactive, "{:?}", info.error);
}
// The server, with the project on "its" disk.
// Canonical, as the UI reports paths (macOS: /var → /private/var).
let project = tmp.path().join("remote/demo");
std::fs::create_dir_all(project.join("src")).unwrap();
let project = std::fs::canonicalize(project).unwrap();
let base = tmp.path().join("server");
let peer_slot = Arc::new(OnceLock::new());
let sink_peer = peer_slot.clone();
let host = Arc::new(Host::new(HostConfig {
config_dir: base.join("config"),
data_dir: base.join("data"),
local_data_dir: base.join("data"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new(move |w| rpc::event_sink(sink_peer.clone(), Some(w.to_owned()))),
owner: |w| w,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
}));
let plugins = RemotePlugins::start(&base.join("data"), &host, peer_slot.clone()).unwrap();
plugins.host().set_program_path(path_var);
let mut dispatcher = Dispatcher::standard();
plugins.register(&mut dispatcher);
let (client_io, server_io) = tokio::io::duplex(1 << 20);
let (sr, sw) = tokio::io::split(server_io);
tokio::spawn(rpc::serve(host.clone(), dispatcher, peer_slot, sr, sw));
let (cr, cw) = tokio::io::split(client_io);
let client = Arc::new(Client::default());
let peer = Peer::connect(cr, cw, client.clone());
let _: Value = peer.request("initialize", json!({ "version": rpc::PROTOCOL_VERSION }), None).await.unwrap();
invoke(&peer, "open_project", json!({ "path": project })).await.unwrap();
invoke(&peer, "project_trust_set", json!({ "path": project, "trusted": true, "folder": false })).await.unwrap();
// Digests first; then the files of what the server lacks; then nothing to do.
let first = sync(&peer, &desktop, &[]).await;
assert_eq!(first.missing, ["openidea.ai-completion", "openidea.checkstyle"], "{first:?}");
let second = sync(&peer, &desktop, &["openidea.ai-completion", "openidea.checkstyle"]).await;
assert_eq!(second.installed.len(), 2, "{second:?}");
assert!(second.failed.is_empty(), "{second:?}");
assert_eq!(sync(&peer, &desktop, &[]).await, PackageSyncReport::default());
let remote = plugins.host().info_of("openidea.checkstyle").unwrap();
assert_eq!(remote.status, PluginStatus::Inactive, "{:?}", remote.error);
assert!(remote.capabilities.iter().any(|c| c.id == "process:checkstyle" && c.granted));
// The remote window's UI routes the run-time commands to the server.
let kinds = invoke(&peer, "plugin_event_kinds", json!({})).await.unwrap();
assert!(kinds.as_array().unwrap().contains(&json!("file-saved")), "{kinds}");
// Saving a file that exists only on the server: Checkstyle reads that file.
let java = project.join("src/A.java");
std::fs::write(&java, "class A {\n\tint x;\n}\n").unwrap();
let saved = WorkbenchEvent::FileSaved(java.to_string_lossy().into_owned());
invoke(&peer, "plugin_post_events", json!({ "events": [saved] })).await.unwrap();
let published = || {
client.events.lock().iter().find_map(|m| match &m.event {
IdeEvent::Diagnostics { path, source, items } if source == "plugin:openidea.checkstyle" => {
Some((path.0.clone(), items.clone()))
}
_ => None,
})
};
until("remote checkstyle diagnostics", 20, || published().is_some()).await;
let (path, items) = published().unwrap();
assert_eq!(path, java);
assert_eq!((items.len(), items[0].range.start.line), (1, 1));
// Secrets stay on the desktop: the server asks for them when used.
let secrets = plugins.host().secrets("openidea.ai-completion").unwrap();
assert_eq!(secrets, [("api-key".to_owned(), true)]);
assert_eq!(client.secret_requests.lock()[0].plugin, "openidea.ai-completion");
// A command of a remote plugin, with this window as its client.
invoke(
&peer,
"plugin_run_command",
json!({ "pluginId": "openidea.ai-completion", "commandId": "toggle", "arg": null, "editor": null }),
)
.await
.unwrap();
until("the command's notification", 5, || {
client
.plugin_events
.lock()
.iter()
.any(|m| m.window.as_deref() == Some("main") && matches!(m.event, PluginEvent::Notification { .. }))
})
.await;
// Disabled on the desktop: gone from the server.
desktop.set_enabled("openidea.checkstyle", false).unwrap();
let removed = sync(&peer, &desktop, &[]).await;
assert_eq!(removed.removed, ["openidea.checkstyle"]);
assert!(plugins.host().info_of("openidea.checkstyle").is_err());
plugins.host().shutdown();
}
/// CSV Editor and a tool window with a terminal (the `jobs` fixture) in a
/// remote project. The table edits the document on the server; the jobs are
/// the server's `jobs` program's, and the terminal opens in the remote window.
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn remote_projects_get_plugin_views_and_custom_editors() {
let tmp = tempfile::tempdir().unwrap();
let bin = tmp.path().join("bin");
std::fs::create_dir_all(&bin).unwrap();
std::fs::write(bin.join("jobs"), "#!/bin/sh\n[ \"$1\" = list ] && echo remote-job\n").unwrap();
std::fs::set_permissions(bin.join("jobs"), std::fs::Permissions::from_mode(0o755)).unwrap();
let path_var = {
let bin = bin.clone();
Arc::new(move || Some(bin.clone().into_os_string()))
};
let desktop =
PluginHost::new(tmp.path().join("desktop/plugins"), Arc::new(|_: Option<&str>, _| {}), Limits::default())
.unwrap();
desktop.set_program_path(path_var.clone());
for dir in [official("csv-editor"), fixture("jobs")] {
let info = desktop.install_from_dir(&dir, &Consent::all()).unwrap();
assert_eq!(info.status, PluginStatus::Inactive, "{:?}", info.error);
}
let project = tmp.path().join("remote/demo");
std::fs::create_dir_all(&project).unwrap();
let project = std::fs::canonicalize(project).unwrap();
std::fs::write(project.join("people.csv"), "name,city\nAda,London\n").unwrap();
let base = tmp.path().join("server");
let peer_slot = Arc::new(OnceLock::new());
let sink_peer = peer_slot.clone();
let host = Arc::new(Host::new(HostConfig {
config_dir: base.join("config"),
data_dir: base.join("data"),
local_data_dir: base.join("data"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new(move |w| rpc::event_sink(sink_peer.clone(), Some(w.to_owned()))),
owner: |w| w,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
}));
let plugins = RemotePlugins::start(&base.join("data"), &host, peer_slot.clone()).unwrap();
plugins.host().set_program_path(path_var);
let mut dispatcher = Dispatcher::standard();
plugins.register(&mut dispatcher);
let (client_io, server_io) = tokio::io::duplex(1 << 20);
let (sr, sw) = tokio::io::split(server_io);
tokio::spawn(rpc::serve(host.clone(), dispatcher, peer_slot, sr, sw));
let (cr, cw) = tokio::io::split(client_io);
let client = Arc::new(Client::default());
let peer = Peer::connect(cr, cw, client.clone());
let _: Value = peer.request("initialize", json!({ "version": rpc::PROTOCOL_VERSION }), None).await.unwrap();
invoke(&peer, "open_project", json!({ "path": project })).await.unwrap();
invoke(&peer, "project_trust_set", json!({ "path": project, "trusted": true, "folder": false })).await.unwrap();
let report = sync(&peer, &desktop, &["openidea.csv-editor", "jobs"]).await;
assert_eq!(report.installed.len(), 2, "{report:?}");
// CSV Editor: the document is the server's.
let opened = invoke(&peer, "doc_open", json!({ "path": project.join("people.csv") })).await.unwrap();
let doc = opened["id"].as_u64().expect("doc id");
let args = |extra: Value| {
let mut a = json!({ "pluginId": "openidea.csv-editor", "view": "table", "doc": doc });
a.as_object_mut().unwrap().extend(extra.as_object().unwrap().clone());
a
};
let view =
invoke(&peer, "plugin_view_render", args(json!({ "viewport": { "firstRow": 0, "rows": 50 } }))).await.unwrap();
assert_eq!(view["body"]["kind"], "table");
assert_eq!(view["body"]["value"]["rows"], json!([["Ada", "London"]]));
let cell = json!({ "event": { "kind": "cell", "row": 0, "column": 1, "value": "Oslo" } });
invoke(&peer, "plugin_view_event", args(cell)).await.unwrap();
let snap = invoke(&peer, "doc_snapshot", json!({ "doc": doc })).await.unwrap();
assert_eq!(snap["text"], "name,city\nAda,Oslo\n");
// Jobs: the server's program; the terminal opens in this window.
let jobs = |event: Value| json!({ "pluginId": "jobs", "view": "jobs", "doc": null, "event": event });
let view =
invoke(&peer, "plugin_view_render", json!({ "pluginId": "jobs", "view": "jobs", "doc": null })).await.unwrap();
assert_eq!(view["body"]["value"][0]["label"], "remote-job", "{view}");
invoke(&peer, "plugin_view_event", jobs(json!({ "kind": "select", "key": "remote-job" }))).await.unwrap();
invoke(&peer, "plugin_view_event", jobs(json!({ "kind": "button", "id": "follow" }))).await.unwrap();
until("the job's terminal", 5, || {
client.plugin_events.lock().iter().any(|m| {
m.window.as_deref() == Some("main")
&& matches!(&m.event, PluginEvent::OpenTerminal { program, .. } if program.0.starts_with(bin.canonicalize().unwrap()))
})
})
.await;
plugins.host().shutdown();
}
+7
View File
@@ -218,6 +218,13 @@ async fn serves_projects_files_documents_search_and_runs() {
invoke(&peer, "vcs_init", json!({ "dir": project })).await.unwrap();
until("git repository", || project.join(".git").is_dir()).await;
// A Docker run configuration's environment comes from the host its command runs on.
let env =
invoke(&peer, "docker_run_env", json!({ "endpoint": { "kind": "socket", "path": "/run/user/1/docker.sock" } }))
.await
.unwrap();
assert_eq!(env["DOCKER_HOST"], "unix:///run/user/1/docker.sock");
// Real project-host ignore writes travel through the remote command dispatcher.
assert!(Dispatcher::standard().commands().contains(&"vcs_add_to_ignore"));
let ignore_root = dunce::canonicalize(&project).unwrap();
+40
View File
@@ -81,6 +81,46 @@ async fn gradle_defaults_are_not_saved_as_explicit_run_choices() {
assert_eq!(saved.command, gradle.command);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn maven_defaults_are_resolved_at_launch_and_explicit_choices_survive_reload() {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
std::fs::write(root.join("pom.xml"), "<project/>").unwrap();
let home = root.join("maven");
std::fs::create_dir_all(home.join("bin")).unwrap();
let launcher = home.join("bin").join(if cfg!(windows) { "mvn.cmd" } else { "mvn" });
std::fs::write(&launcher, "").unwrap();
let host = host(&root.join("app"));
host.trust.decide(&root, true, false).unwrap();
host.open_project("main", &root).unwrap();
for (key, value) in [
("languages.java.maven.home", home.to_string_lossy().into_owned()),
("languages.java.maven.userSettings", "settings.xml".into()),
] {
ops::project::set_settings(
&host,
"main",
ide_workspace::settings::SettingsScope::Project,
key,
&serde_json::to_string(&value).unwrap(),
)
.unwrap();
}
let mut config = ops::run::configs(&host, "main").unwrap().into_iter().find(|c| c.id == "maven:test").unwrap();
assert_eq!(config.command, "mvn");
assert_eq!(config.args, ["test"]);
assert!(config.env.is_empty());
config.command = launcher.to_string_lossy().into_owned();
config.args = vec!["-s".into(), "own-settings.xml".into(), "test".into()];
config.env.insert("JAVA_HOME".into(), "own-jdk".into());
config.detected = false;
ops::run::save_configs(&host, "main", &[config.clone()], &[]).unwrap();
let saved = ops::run::configs(&host, "main").unwrap().into_iter().find(|c| c.id == config.id).unwrap();
assert_eq!(saved.command, config.command);
assert_eq!(saved.args, config.args);
assert_eq!(saved.env, config.env);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn workspace_cmake_before_launch_tasks_resolve_in_the_member() {
let tmp = tempfile::tempdir().unwrap();
+28 -1
View File
@@ -35,8 +35,16 @@ pub fn build(proxy: &ProxySettings) -> IdeResult<reqwest::Client> {
.with_platform_verifier()
.map_err(|e| fail(e.to_string()))?
.with_no_client_auth();
// `localhost` is tried on IPv4 first: local servers (Ollama, dev servers)
// usually listen there, and on Windows a refused `::1` costs ~2 s before
// the system resolver's order falls back. Ports come from the URL.
let loopback = [
std::net::SocketAddr::from((std::net::Ipv4Addr::LOCALHOST, 0)),
std::net::SocketAddr::from((std::net::Ipv6Addr::LOCALHOST, 0)),
];
let builder = reqwest::Client::builder()
.tls_backend_preconfigured(tls)
.resolve_to_addrs("localhost", &loopback)
.connect_timeout(Duration::from_secs(30))
.user_agent(concat!("OpenIDEA-HttpClient/", env!("CARGO_PKG_VERSION")));
with_proxy(builder, proxy).build().map_err(|e| fail(e.to_string()))
@@ -113,7 +121,11 @@ mod tests {
/// A one-shot HTTP server on loopback: answers `200` and hands back the request head.
fn server() -> (u16, std::thread::JoinHandle<String>) {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
server_on("127.0.0.1:0")
}
fn server_on(addr: &str) -> (u16, std::thread::JoinHandle<String>) {
let listener = TcpListener::bind(addr).unwrap();
let port = listener.local_addr().unwrap().port();
let handle = std::thread::spawn(move || {
let (stream, _) = listener.accept().unwrap();
@@ -141,6 +153,21 @@ mod tests {
check_connection(proxy, url, &CancellationToken::new()).await
}
/// `localhost` reaches a server on either loopback address.
#[tokio::test]
async fn localhost_reaches_ipv4_and_ipv6_servers() {
let direct = ProxySettings::default();
let (port, v4) = server();
assert_eq!(get(&direct, &format!("http://localhost:{port}/")).await.unwrap(), "200 OK");
v4.join().unwrap();
// Hosts without IPv6 loopback skip this half.
if TcpListener::bind("[::1]:0").is_ok() {
let (port, v6) = server_on("[::1]:0");
assert_eq!(get(&direct, &format!("http://localhost:{port}/")).await.unwrap(), "200 OK");
v6.join().unwrap();
}
}
#[tokio::test]
async fn builds_for_every_mode() {
for mode in [ProxyMode::None, ProxyMode::Auto, ProxyMode::Manual] {
+3
View File
@@ -16,3 +16,6 @@ parking_lot.workspace = true
thiserror.workspace = true
tracing.workspace = true
encoding_rs.workspace = true
[dev-dependencies]
tempfile.workspace = true
+15
View File
@@ -243,6 +243,9 @@ struct Inner {
pid: Option<u32>,
/// Fired when the reader task has stopped (EOF, error or kill).
reader_done: CancellationToken,
/// Fired once the child process has exited and been reaped (after [`Peer::kill`],
/// also what it started); at once when there is no child.
exited: CancellationToken,
}
impl Inner {
@@ -432,6 +435,7 @@ impl Peer {
(child.wait().await.ok(), true)
}
};
inner.exited.cancel();
if !killed {
// Bounded: a grandchild may keep the pipes open.
let drained = async {
@@ -510,7 +514,11 @@ impl Peer {
kill: CancellationToken::new(),
pid,
reader_done: CancellationToken::new(),
exited: CancellationToken::new(),
});
if pid.is_none() {
inner.exited.cancel();
}
// Writer: a single task drains the queue so messages never interleave.
{
@@ -704,6 +712,13 @@ impl Peer {
self.inner.close_transport();
self.inner.kill.cancel();
}
/// Resolves once the child process is gone: exited on its own, or killed with what it
/// started and reaped. [`closed`](Self::closed) can come earlier, at EOF. Until then the
/// child may still hold files open, which on Windows can't be deleted.
pub async fn exited(&self) {
self.inner.exited.cancelled().await;
}
}
/// Removes the pending entry and tells the remote side to stop when a request
+59
View File
@@ -413,6 +413,65 @@ async fn kill_terminates_process() {
tokio::time::timeout(Duration::from_secs(5), peer.closed()).await.unwrap();
}
#[cfg(unix)]
#[tokio::test(flavor = "multi_thread")]
async fn exited_resolves_once_the_killed_process_is_reaped() {
let peer =
Peer::spawn("sleep", &["30".into()], &std::env::temp_dir(), &[], Arc::new(NoopHandler), Box::new(|_| {}))
.unwrap();
let pid = peer.pid().unwrap().to_string();
let exited = tokio::time::timeout(Duration::from_millis(200), peer.exited()).await;
assert!(exited.is_err(), "running until killed");
peer.kill();
tokio::time::timeout(Duration::from_secs(5), peer.exited()).await.unwrap();
let alive = std::process::Command::new("kill").args(["-0", &pid]).status().unwrap().success();
assert!(!alive, "reaped, not even a zombie");
}
#[tokio::test(flavor = "multi_thread")]
async fn exited_resolves_at_once_without_a_process() {
let (client, _server) = pair(Arc::new(NoopHandler), Arc::new(Recorder::default()));
tokio::time::timeout(Duration::from_secs(1), client.exited()).await.unwrap();
}
#[cfg(windows)]
#[tokio::test(flavor = "multi_thread")]
async fn exited_waits_for_a_windows_descendant_to_release_its_file() {
use std::os::windows::fs::OpenOptionsExt;
let tmp = tempfile::tempdir().unwrap();
std::fs::write(
tmp.path().join("hold.ps1"),
"$f=[IO.File]::Open('locked.txt','Create','ReadWrite','None'); [IO.File]::WriteAllText('ready.txt','ready'); Start-Sleep 30",
)
.unwrap();
let peer = Peer::spawn(
"cmd",
&[
"/D".into(),
"/C".into(),
"powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File hold.ps1".into(),
],
tmp.path(),
&[],
Arc::new(NoopHandler),
Box::new(|_| {}),
)
.unwrap();
tokio::time::timeout(Duration::from_secs(10), async {
while !tmp.path().join("ready.txt").exists() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await
.unwrap();
let open = || std::fs::OpenOptions::new().read(true).share_mode(0).open(tmp.path().join("locked.txt"));
assert_eq!(open().unwrap_err().raw_os_error(), Some(32));
assert!(tokio::time::timeout(Duration::from_millis(200), peer.exited()).await.is_err());
peer.kill();
tokio::time::timeout(Duration::from_secs(5), peer.exited()).await.unwrap();
drop(open().expect("exited must wait for the descendant's file handle to close"));
}
/// A launcher that prints in the system code page and exits before speaking the
/// protocol: `jdtls.bat` on a Chinese Windows when its Python step fails, then
/// `pause` prints `请按任意键继续. . .` in GBK on stdout.
+1
View File
@@ -7,6 +7,7 @@ license.workspace = true
[dependencies]
ide-core.workspace = true
ide-extension.workspace = true
ide-text.workspace = true
serde.workspace = true
serde_json.workspace = true
+55 -3
View File
@@ -8,16 +8,20 @@ use ide_core::{LanguageId, Range, SymbolKind};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
mod markers;
pub mod plugin_point;
mod registry;
mod syntax;
pub use markers::marked_root;
pub use plugin_point::PluginLanguages;
pub use registry::{FileTypes, LanguageRegistry};
pub use syntax::{
CSS_CLASS, CSS_ID, CSS_VARIABLE, ConfigurationKey, FileReference, Literal, LiteralKind, MAX_CACHED_TREES,
MAX_PARSE_BYTES, ParsedTree, SpellingKind, SpellingRegion, Syntax, SyntaxCache, SyntaxCacheStats,
SyntaxHighlightKind, SyntaxHighlights, TreeError, WebReference, WebReferenceKind, merge_visibility,
CSS_CLASS, CSS_ID, CSS_VARIABLE, ConfigurationKey, ExternalFold, ExternalSymbol, ExternalSyntax, ExternalToken,
FileReference, Literal, LiteralKind, MAX_CACHED_TREES, MAX_PARSE_BYTES, ParsedTree, SpellingKind, SpellingRegion,
Syntax, SyntaxCache, SyntaxCacheStats, SyntaxHighlightKind, SyntaxHighlights, TreeError, WebReference,
WebReferenceKind, merge_visibility, set_external_syntax,
};
/// Parsed form of `languages/<id>.toml`. Backend-only; the UI gets [`LanguageInfo`].
@@ -127,6 +131,16 @@ pub struct LanguageServerConfig {
/// basedpyright): only used while none of those is installed.
#[serde(default)]
pub fallback: bool,
/// What the server is used for (`completion`, `code-action`, `diagnostics`,
/// … `FEATURES` in `ide-lsp/src/caps.rs`); empty means everything. The other
/// capabilities it announces are ignored, so a refactoring-only server next
/// to the language's own one adds no completion, hover or formatting.
#[serde(default)]
pub features: Vec<String>,
/// Code action kinds kept from the server, matched like LSP's `only`
/// (`refactor` keeps `refactor.extract`); empty keeps every kind.
#[serde(default)]
pub code_action_kinds: Vec<String>,
/// Newest Java release the server runs on (kotlin-language-server: 21). A
/// newer `JAVA_HOME` is replaced by the newest installed JDK that fits.
#[serde(default)]
@@ -138,6 +152,9 @@ pub struct LanguageServerConfig {
/// Register source files with tsserver when no project configuration exists.
#[serde(default)]
pub tsserver_project: bool,
/// Set on plugin languages: start exactly this approved program.
#[serde(skip)]
pub pinned: Option<ProgramPin>,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
@@ -157,6 +174,41 @@ pub struct DebuggerConfig {
pub connect: Option<String>,
#[serde(default)]
pub install_hint: Option<String>,
/// Set on plugin languages: start exactly this approved program.
#[serde(skip)]
pub pinned: Option<ProgramPin>,
}
/// The program a plugin's `process:` grant approved:
/// a plugin language's server or debug adapter starts from this exact path,
/// and only while the file still has the approved content. Set by the plugin
/// host, never read from a definition.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct ProgramPin {
pub path: PathBuf,
/// blake3 of the file when approved.
pub hash: String,
}
impl ProgramPin {
/// Approves `path` as it is now.
pub fn approve(path: impl Into<PathBuf>) -> std::io::Result<Self> {
let path = path.into();
let hash = ide_extension::ledger::file_hash(&path)?;
Ok(Self { path, hash })
}
/// Checks the file right before it is started; the error tells the user what to do.
pub fn verify(&self) -> Result<&Path, String> {
match ide_extension::ledger::file_hash(&self.path) {
Ok(hash) if hash == self.hash => Ok(&self.path),
Ok(_) => Err(format!(
"{} changed since it was approved; review the plugin's permissions in Settings › Plugins",
self.path.display()
)),
Err(e) => Err(format!("{}: {e}", self.path.display())),
}
}
}
fn default_transport() -> String {
+294
View File
@@ -0,0 +1,294 @@
//! The `languages` extension point: language definitions contributed by
//! plugins, in the same format as `languages/<id>.toml`.
//!
//! ```toml
//! [[contributes.languages]]
//! path = "languages/solidity.toml"
//! ```
//!
//! A definition that starts programs (`language-servers.command`,
//! `debugger.command`) needs `process:<program>` for each: a declaration is a
//! capability too, or registering languages would bypass the sandbox.
//! Committed definitions live in [`PluginLanguages`], an overlay over the
//! built-in registry (built-in definitions win on id clashes): see
//! [`crate::LanguageRegistry::attach_plugins`]. A server or debugger whose
//! program is not granted is left out; the language itself still works.
use crate::{LanguageConfig, ProgramPin};
use ide_extension::{Capability, Disposable, ExtensionPoint, Package, Placement, Prepared, decode};
use parking_lot::RwLock;
use serde::Deserialize;
use std::path::PathBuf;
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};
pub const POINT_ID: &str = "languages";
/// Decides whether a plugin may start a program: the approved file for a
/// granted `process:` capability, or `None`.
pub type ProcessPolicy = dyn Fn(&str, &Capability) -> Option<ProgramPin> + Send + Sync;
/// Languages of enabled plugins, in commit order.
#[derive(Default)]
pub struct PluginLanguages {
items: RwLock<Vec<(u64, String, LanguageConfig)>>,
seq: AtomicU64,
revision: AtomicU64,
policy: RwLock<Option<Arc<ProcessPolicy>>>,
}
impl PluginLanguages {
pub fn new() -> Arc<Self> {
Arc::new(Self::default())
}
pub fn all(&self) -> Vec<LanguageConfig> {
self.items.read().iter().map(|(_, _, c)| c.clone()).collect()
}
/// `(plugin id, definition)` pairs.
pub fn with_owners(&self) -> Vec<(String, LanguageConfig)> {
self.items.read().iter().map(|(_, p, c)| (p.clone(), c.clone())).collect()
}
/// The plugin that contributes language `id`.
pub fn owner_of(&self, id: &ide_core::LanguageId) -> Option<String> {
self.items.read().iter().find(|(_, _, c)| &c.id == id).map(|(_, p, _)| p.clone())
}
/// Changes whenever [`effective`](Self::effective) may give something else.
pub fn revision(&self) -> u64 {
self.revision.load(Ordering::Acquire)
}
/// Who may start what. Without a policy no plugin program starts.
pub fn set_process_policy(&self, policy: Arc<ProcessPolicy>) {
*self.policy.write() = Some(policy);
self.invalidate();
}
/// Grants changed: registries recompute [`effective`](Self::effective).
pub fn invalidate(&self) {
self.revision.fetch_add(1, Ordering::Release);
}
/// The definitions as they take effect: servers and the debugger whose
/// program is granted are pinned to the approved file, the others are
/// left out.
pub fn effective(&self) -> Vec<LanguageConfig> {
let policy = self.policy.read().clone();
let pin = |plugin: &str, program: &str| {
policy.as_ref().and_then(|p| p(plugin, &Capability::Process(program.to_owned())))
};
self.items
.read()
.iter()
.map(|(_, plugin, cfg)| {
let mut cfg = cfg.clone();
cfg.language_servers.retain_mut(|s| {
s.pinned = pin(plugin, &s.command);
s.pinned.is_some()
});
let debugger_pin = cfg.debugger.as_ref().and_then(|d| pin(plugin, &d.command));
match (&mut cfg.debugger, debugger_pin) {
(Some(d), Some(p)) => d.pinned = Some(p),
(debugger, _) => *debugger = None,
}
cfg
})
.collect()
}
/// Adds a definition until the returned handle is dropped.
pub fn add(self: &Arc<Self>, plugin: String, cfg: LanguageConfig) -> Disposable {
let seq = self.seq.fetch_add(1, Ordering::Relaxed);
self.items.write().push((seq, plugin, cfg));
self.revision.fetch_add(1, Ordering::Release);
let weak = Arc::downgrade(self);
Disposable::new(move || {
if let Some(this) = weak.upgrade() {
this.items.write().retain(|(s, _, _)| *s != seq);
this.revision.fetch_add(1, Ordering::Release);
}
})
}
}
pub struct LanguagesPoint {
store: Arc<PluginLanguages>,
}
impl LanguagesPoint {
pub fn new(store: Arc<PluginLanguages>) -> Self {
Self { store }
}
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Decl {
/// A `languages/<id>.toml`-style file in the package.
path: PathBuf,
}
/// Parses and checks one language definition.
pub fn parse_definition(src: &str) -> Result<LanguageConfig, String> {
let cfg: LanguageConfig = toml::from_str(src).map_err(|e| format!("invalid language definition: {e}"))?;
validate_definition(&cfg)?;
Ok(cfg)
}
/// Checks a definition that did not come from a package file (a remote sync).
pub fn validate_definition(cfg: &LanguageConfig) -> Result<(), String> {
let id = cfg.id.as_str();
if id.is_empty() || cfg.name.trim().is_empty() {
return Err("language `id` and `name` must not be empty".into());
}
if !id.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.' | '+')) || id.len() > 64 {
return Err(format!("invalid language id {id:?}"));
}
if let Some(first) = &cfg.first_line {
regex_lite::Regex::new(first).map_err(|e| format!("invalid first-line pattern: {e}"))?;
}
Ok(())
}
/// The programs a definition starts, as `process:` capabilities.
pub fn required_capabilities(cfg: &LanguageConfig) -> Vec<Capability> {
let mut caps: Vec<Capability> = Vec::new();
let servers = cfg.language_servers.iter().map(|s| s.command.as_str());
for program in servers.chain(cfg.debugger.iter().map(|d| d.command.as_str())) {
let cap = Capability::Process(program.to_owned());
if !program.trim().is_empty() && !caps.contains(&cap) {
caps.push(cap);
}
}
caps
}
impl ExtensionPoint for LanguagesPoint {
fn id(&self) -> &'static str {
POINT_ID
}
fn placement(&self) -> Placement {
Placement::Workspace
}
fn prepare(&self, pkg: &Package, decl: toml::Value) -> Result<Prepared, String> {
let decl: Decl = decode(POINT_ID, decl)?;
let src = pkg.read_text(&decl.path)?;
let cfg = parse_definition(&src).map_err(|e| format!("{}: {e}", decl.path.display()))?;
let caps = required_capabilities(&cfg);
let (store, plugin, key, label) = (
self.store.clone(),
pkg.id().to_owned(),
cfg.id.to_string(),
format!("{} (.{})", cfg.name, cfg.extensions.join(", .")),
);
let mut prepared = Prepared::new(label, move || Ok(store.add(plugin, cfg))).key(key).file(decl.path);
for cap in caps {
prepared = prepared.capability(cap);
}
Ok(prepared)
}
}
#[cfg(test)]
mod tests {
use super::*;
use ide_extension::{ExtensionRegistry, Manifest};
#[test]
fn languages_point_registers_and_requires_process_capabilities() {
let tmp = tempfile::tempdir().unwrap();
std::fs::create_dir(tmp.path().join("l")).unwrap();
std::fs::write(
tmp.path().join("l/sol.toml"),
"id = \"solidity\"\nname = \"Solidity\"\nextensions = [\"sol\"]\n\
[[language-servers]]\nname = \"solc\"\ncommand = \"solc\"\nargs = [\"--lsp\"]\n",
)
.unwrap();
let src = "id = \"p\"\nname = \"P\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"\n\
[[contributes.languages]]\npath = \"l/sol.toml\"";
let pkg = Package::new(Manifest::parse(src).unwrap(), tmp.path().to_path_buf());
let store = PluginLanguages::new();
let reg = ExtensionRegistry::new();
reg.register(Arc::new(LanguagesPoint::new(store.clone()))).unwrap();
let set = reg.prepare(&pkg).unwrap();
assert_eq!(set.capabilities, [Capability::Process("solc".into())]);
assert_eq!(set.files, [PathBuf::from("l/sol.toml")]);
reg.commit(set).unwrap();
assert_eq!(store.all()[0].id.as_str(), "solidity");
reg.dispose_plugin("p");
assert!(store.all().is_empty());
let bad = |def: &str| parse_definition(def).unwrap_err();
assert!(bad("not toml [").contains("invalid language"));
assert!(bad("id = \"\"\nname = \"x\"").contains("must not be empty"));
assert!(bad("id = \"a b\"\nname = \"x\"").contains("invalid language id"));
}
#[test]
fn registry_overlay_follows_the_store_and_the_grants() {
use crate::LanguageRegistry;
use ide_core::LanguageId;
use std::path::Path;
let base = LanguageRegistry::load(None).unwrap();
let store = PluginLanguages::new();
base.attach_plugins(store.clone());
// Projects fork before plugins load: the fork follows the store anyway.
let project = base.fork();
assert_eq!(project.detect(Path::new("a.sol"), None), LanguageId::plain_text());
let sol = parse_definition(
"id = \"solidity\"\nname = \"Solidity\"\nextensions = [\"sol\"]\n\
[[language-servers]]\nname = \"solc\"\ncommand = \"solc\"\n\
[debugger]\nadapter = \"soldbg\"\ncommand = \"soldbg\"\n",
)
.unwrap();
let rust_clash = parse_definition("id = \"rust\"\nname = \"Evil Rust\"\nextensions = [\"rs\"]").unwrap();
let ext_clash = parse_definition(
"id = \"rusty\"\nname = \"Rusty\"\nextensions = [\"rs\", \"rsy\"]\nfilenames = [\"Cargo.lock\"]",
)
.unwrap();
let _sol = store.add("acme.sol".into(), sol);
let _clash = store.add("acme.evil".into(), rust_clash);
let _ext = store.add("acme.evil".into(), ext_clash);
assert_eq!(project.detect(Path::new("a.sol"), None).as_str(), "solidity");
let rust = project.get(&LanguageId::new("rust")).unwrap();
assert_eq!(rust.name, "Rust", "built-in definitions win on id clashes");
assert_eq!(project.detect(Path::new("main.rs"), None).as_str(), "rust", "and keep their extensions");
assert_ne!(project.detect(Path::new("Cargo.lock"), None).as_str(), "rusty", "and file names");
assert_eq!(project.detect(Path::new("x.rsy"), None).as_str(), "rusty");
// No policy: the language works, its programs do not start.
let cfg = project.get(&LanguageId::new("solidity")).unwrap();
assert!(cfg.language_servers.is_empty() && cfg.debugger.is_none());
let pin = ProgramPin { path: "/opt/solc".into(), hash: "h".into() };
let granted = pin.clone();
store.set_process_policy(Arc::new(move |plugin, cap| {
(plugin == "acme.sol" && *cap == Capability::Process("solc".into())).then(|| granted.clone())
}));
let cfg = project.get(&LanguageId::new("solidity")).unwrap();
assert_eq!(cfg.language_servers[0].pinned.as_ref(), Some(&pin));
assert!(cfg.debugger.is_none(), "soldbg was not granted");
drop(_sol);
assert_eq!(project.detect(Path::new("a.sol"), None), LanguageId::plain_text());
assert!(base.get(&LanguageId::new("solidity")).is_none());
}
#[test]
fn program_pins_check_the_file() {
let tmp = tempfile::tempdir().unwrap();
let bin = tmp.path().join("tool");
std::fs::write(&bin, "v1").unwrap();
let pin = ProgramPin { path: bin.clone(), hash: ide_extension::ledger::file_hash(&bin).unwrap() };
assert_eq!(pin.verify().unwrap(), bin);
std::fs::write(&bin, "v2").unwrap();
assert!(pin.verify().unwrap_err().contains("changed since it was approved"));
}
}
Loaded 100 of 852 files, more files were not shown because too many files have changed in this diff. Show more