Compare commits

..
Author SHA1 Message Date
l0ng-ai 434aacc64a chore(plugins): regenerate bindings on top of main 2026-11-04 02:41:46 +08:00
l0ng-ai 58f8929509 test(plugins): the MyBatis test asks again while the first call builds the index 2026-11-04 02:41:46 +08:00
l0ng-ai 14ddd948f8 test(plugins): capability paths are the platform's in the parse test 2026-11-04 02:41:46 +08:00
l0ng-ai 8df7731fa6 chore(plugins): ide-extension at the workspace version in Cargo.lock 2026-11-04 02:41:46 +08:00
l0ng-ai 20281e9b66 fix(plugins): Windows: local marketplace paths, localhost connections, tests
- A marketplace index given as a local Windows path (`C:\market\index.json`)
  resolves its packages next to it; only URLs are joined with `/`.
- HTTP requests to `localhost` try IPv4 loopback first, then IPv6: local
  servers such as Ollama usually listen on IPv4, and on Windows a refused
  `::1` cost about two seconds per request.
- Tests that assumed Unix: an absolute program path is the platform's, a
  manifest rewritten by a test may have CRLF line ends, the precompiled-cache
  test replaces the cache file only after the host that maps it is gone, and
  the MyBatis test lets its first call build the index before asserting.
- The icon theme test's time limits guard against work growing faster than
  the icon count rather than measure the runner.
2026-11-04 02:41:46 +08:00
l0ng-ai 20960c1d96 fix(plugins): soft deadlines hold under load, unique cache writes, CI on every platform
- A provider result produced after its soft deadline is dropped even when it
  is already waiting (the caller was scheduled late or waited on another
  plugin first), so the deadline holds on a busy machine. The result is
  timestamped when the plugin's thread produces it, so a result that came in
  time is still used when it is read late.
- Each precompiled-component write uses its own temporary file: two threads
  storing the same component could rename a half-written file into place.
- Tests: the cache-corruption case replaces the file instead of overwriting
  the one a running component maps; the yield test no longer counts on a
  tick rate; a test that runs a shell script is Unix-only like its
  neighbours.
- Plugin web pages declare `@types/node` for their Vite configs instead of
  finding the repository's.
2026-11-04 02:41:46 +08:00
l0ng-ai 9cc83c77b8 feat(plugins): plugin contributions in the IDE's own settings and menus; official plugins published with releases
- Themes: a plugin theme is an `appearance.theme` value
  (`plugin:<id>/<theme>`), offered by Settings › Appearance (and the
  preferred themes of Sync with OS), Quick Switch, the toolbar gear menu and
  Welcome › Customize. The project window selects it in the plugin host;
  picking a built-in theme clears it. While its plugin is off, the built-in
  theme of its kind shows and the choice is kept.
- File icon themes: chosen in Settings › Appearance › File icon theme and
  kept in `appearance.fileIconTheme`, so every window follows.
- Keymap: plugins' default shortcuts show in Settings › Keymap as the
  defaults of their actions: removable, changeable, resettable and part of
  conflict checks.
- Problems: a plugin's problems are withdrawn when it stops, fails, is
  disabled or uninstalled. Settings › Inspections lists them under Plugins by
  plugin name; stored overrides of `plugin:` producers parse correctly.
- Status bar items of plugins are named in the status bar menu and can be
  hidden.
- Languages added or removed by plugins refresh Override File Type, the
  Code Style pages and Inspections without a restart.
- Uninstalling a plugin deletes the secrets it declared.
- The Settings window follows plugin changes made elsewhere.
- The plugin system is marked Preview (Settings › Plugins, README); Harbor
  Night is versioned 0.1.0 like the other official plugins.
- Release: the official plugins are packed, signed (secret
  PLUGIN_SIGNING_KEY) and indexed into the release as
  `plugins-index.json`; released builds use it as the default marketplace.
  Without the key the step is skipped.
2026-11-04 02:41:46 +08:00
l0ng-ai ac71b7008e feat(plugins): Graphviz, Regex Tester, License Header and Harbor Night official plugins
- Graphviz: `.dot` / `.gv` files open beside a live preview (a web view
  rendering with Graphviz compiled to WebAssembly) in any layout engine,
  with zoom, pan, Fit and SVG / PNG export through the save dialog. What
  Graphviz reports goes from the page to the plugin's code, which publishes
  it as problems on the named line. A DOT language, file icons and a
  New Graphviz Graph template.
- Regex Tester: a tool window whose html view sends pattern, flags, text and
  replacement as you type; the plugin matches with Rust's regex (linear time)
  and answers with matches, groups and the replacement in UTF-16 offsets.
  Use Selection and Test Selection take the editor's selection.
- License Header: files without a copyright / SPDX header are reported when
  they open and on save; a quick fix and a command add one in the file's
  comment syntax (after a shebang or encoding line); a project scan in a
  background task reports the rest. Holder and license are settings.
- Harbor Night becomes an official plugin and the one source of the theme
  files: `openidea-plugin new --template theme`, the theme docs and tests
  start from it.
- Tool windows' renders and events carry the window's active editor, as
  commands do, so a view's buttons can act on what the user is editing.
- knip leaves plugins' web pages (projects of their own) alone.
2026-11-04 02:41:46 +08:00
l0ng-ai 3b78d15d6d refactor(plugins): drop the Kubernetes plugin, samples become test fixtures
- Remove the Kubernetes official plugin.
- `hello` and `language-toy` move from `plugins/` to
  `crates/ide-plugin/tests/fixtures/`: they stay what the host's tests run
  against, but are no longer listed as plugins of the repository.
- A `jobs` fixture takes over what only Kubernetes exercised: a tree view
  filled by a pinned program, a terminal for the activated entry in the
  invoking window, string and number settings, locally and in remote
  projects.
- `plugins/` now holds only the official plugins; `build.sh`, CI and the
  debug plugin catalog follow.
2026-11-04 02:41:46 +08:00
l0ng-ai 3f52439703 chore(plugins): regenerate bindings and reconcile with main
- Regenerated bindings for the merged settings, edits and plugin commands.
- Mouse shortcuts run their action through `executeAction` again.
- Dropped Chinese entries whose templates main already translates.
2026-11-04 02:41:46 +08:00
l0ng-aiandflorentvdry a7b7616da9 feat(plugins): install packages from a .zip or a folder's plugin.toml
Install Plugin from Package File now takes an `.oiplugin`, a `.zip` or a
package folder's `plugin.toml`, and goes through the same install page
(capabilities, signature, consent) as every other install.

- A zip may hold the package at its root or in its only folder, as zipping
  the folder makes; archiver metadata (`__MACOSX`, `.DS_Store`, `._*`) is
  left out. A zip without a `plugin.toml` there, such as an IntelliJ plugin,
  is refused with a message saying so. Unsafe paths, links and size limits
  are checked as for `.oiplugin` files.
- Picking a `plugin.toml` installs its folder.

Co-authored-by: florentvdry <[email protected]>
2026-11-04 02:41:46 +08:00
l0ng-aiandDeng Liu 900f2e6819 feat(plugins): web views, document bridge and the Excalidraw official plugin
Plugins can now ship a web page as the UI of a view or custom editor
(`web = "web/index.html"`), so editors built on web libraries are plugins
rather than part of the core.

- Web views: every file under the page's directory is part of the package
  (checked, copied on install, covered by the digest and signature) and
  served through the `openidea-plugin` URL scheme, under a per-revision
  path with a CSP of its own, into an `<iframe sandbox="allow-scripts">`.
  An update or reinstall reloads open pages.
- Frame bridge (`window.openidea`), shared by html bodies and web pages:
  theme, messages to the plugin's code, `document.get/edit/replace/onChange`
  (versioned edits that join the text editor's undo history, Save and dirty
  state), `saveFile`/`openFile` through the IDE's dialogs, and IDE shortcuts
  the page leaves alone.
- Custom editor layouts: tab, split (Editor and X) or preview over the text
  editor, switchable from the tab's actions.
- New points `fileIcons` and `fileTemplates`; new optional capability
  `clipboard` for web views.
- Excalidraw official plugin: `.excalidraw` / `.excalidraw.json` open as an
  interactive canvas kept in step with the file's text, with self-hosted
  fonts, PNG/SVG export, images, file icons, a New Drawing template and
  JSON highlighting. Closes #732.
- Fixes: a lock-order deadlock between listing plugins and refreshing
  project pins; the dev server ignored edits under `src/features/plugins`.
- `plugins/build.sh` builds and checks web pages; CI typechecks and tests
  them.

Co-authored-by: Deng Liu <[email protected]>
2026-11-04 02:41:46 +08:00
l0ng-ai 78e0c7a23e feat(plugins): finalize plugin API before release, concurrent providers
Plugin API (breaking; the plugin system is unreleased):
- Remove the runtime `contributions` interface (register-command,
  register-language, register-theme). Commands, languages and themes are
  declared in plugin.toml only, so everything a plugin contributes goes
  through the install preview and consent. The registry drops runtime
  scope, `contribute` and `dispose_runtime`; inline `definition` in
  language and theme declarations is gone.
- `completion-item.kind` is a string (the 25 LSP kinds; unknown kinds show
  no icon), like the other open-ended kinds.
- `lifecycle.activate` and `commands.run` return `types.error` instead of a
  string. The host maps it to the matching error kind, and only `failed`
  counts against the plugin: cancelling, a missing grant or an unknown
  command does not. The SDK's `Plugin` returns `Result<_, Error>`; plain
  messages convert to `Error::Failed`.

Runtime:
- Editor providers of several plugins are called at the same time
  (`Actor::submit` + `Pending::wait`): the wait is the slowest plugin's,
  not the sum, so each gets the full edit-path budget.

Versioning and tooling:
- Components are checked at install and by `openidea-plugin check` for the
  plugin API they were built for; another major or a newer minor is
  refused with what to do.
- Tests: the SDK's WIT matches the host's, a simulated next-minor host
  links every shipped component, every host function has an SDK wrapper,
  released APIs only grow (no snapshot until the first release).
- plugins/build.sh builds every checked-in component, records a source
  fingerprint next to it, and `--check` fails on stale components; CI runs
  it plus fmt, clippy (host and wasm) and tests for the SDK, the official
  plugins and the fixtures.
- The API evolution rules and the checklist for a new host function are in
  plugins/README.md; the design notes under docs/ are removed.
2026-11-04 02:41:46 +08:00
l0ng-ai fa003016ef feat(plugins): theme variables for html view frames, kept current without reload
- Inject `data-theme` and stable `--openidea-*` colour and font variables
  (concrete values; the frame has no IDE stylesheet) into html view bodies.
- Post theme changes to the loaded frame instead of rebuilding srcdoc, so a
  theme switch keeps the frame's state; re-send on load to close the race.
- Expose `openidea.theme` / `openidea.onThemeChange` to frame scripts.
- Move `subscribeTheme` / `themeKey` from the Markdown mermaid module to
  `src/platform/themeWatch.ts`, shared by both.
- Document the variables in plugins/README.md.
2026-11-04 02:41:46 +08:00
l0ng-ai fd1f465aeb feat(plugins): extension points, capability consent, async runtime and plugin API 1.0
Rework the plugin system per docs/plugin_plan.md (P0–P8):

- Extension-point registry (ide-extension) so new contribution types register
  without touching the kernel; languages, run detectors, providers, views and
  UI contributions go through it.
- Per-plugin capability consent with a risk-graded ledger; process
  capabilities are pinned to resolved binary paths and hashes.
- Async, actor-per-plugin Wasm runtime with lazy activation, cancellation,
  events, long-running tasks and idle reclamation.
- Plugin host moved into ide-host, shared by local and remote projects.
- Distribution: install/package tooling (openidea-plugin bin) and a static
  market index.
- WIT frozen as openidea:[email protected]; Rust SDK with mock host.
- Ten official validation plugins under plugins/official.
2026-11-04 02:41:46 +08:00
536 changed files with 2105 additions and 42226 deletions

No files matched your search

-46
View File
@@ -1,46 +0,0 @@
name: Plugin index
# Refreshes the community plugins in the latest release's plugins-index.json
# (what released apps read) from the registry, l0ng-ai/openidea-plugins. The
# official plugins, their key and revocations stay as the release wrote them.
on:
schedule:
- cron: "17 * * * *"
workflow_dispatch:
# Sent by the registry when its index changes.
repository_dispatch:
types: [community-plugins]
permissions:
contents: write
concurrency:
group: plugins-index
env:
COMMUNITY_PLUGINS_INDEX: https://github.com/l0ng-ai/openidea-plugins/releases/download/index/index.json
jobs:
merge:
runs-on: ubuntu-latest
steps:
- name: Merge the community index
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
cd "$RUNNER_TEMP"
tag="$(gh release view --repo "$GITHUB_REPOSITORY" --json tagName -q .tagName)"
tool=openidea-plugin-linux-x86_64
if ! gh release download "$tag" --repo "$GITHUB_REPOSITORY" -p plugins-index.json -p "$tool.gz"; then
echo "::notice::$tag has no plugin index or openidea-plugin tool yet"
exit 0
fi
gunzip "$tool.gz" && chmod +x "$tool"
cp plugins-index.json before.json
"./$tool" merge plugins-index.json "$COMMUNITY_PLUGINS_INDEX" --keep openidea
if cmp -s before.json plugins-index.json; then
echo "unchanged"
else
gh release upload "$tag" plugins-index.json --repo "$GITHUB_REPOSITORY" --clobber
fi
+4 -32
View File
@@ -18,7 +18,6 @@ env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
TAG: ${{ github.event.inputs.tag || github.ref_name }}
COMMUNITY_PLUGINS_INDEX: https://github.com/l0ng-ai/openidea-plugins/releases/download/index/index.json
jobs:
# Creates (or reuses) one draft release that every platform build uploads into.
@@ -26,7 +25,6 @@ jobs:
runs-on: ubuntu-latest
outputs:
release_id: ${{ steps.release.outputs.id }}
official_plugin_keys: ${{ steps.plugin-keys.outputs.keys }}
steps:
- uses: actions/checkout@v4
with: { ref: "${{ env.TAG }}" }
@@ -37,20 +35,6 @@ jobs:
echo "::error::tag $TAG does not match tauri.conf.json version $app"
exit 1
fi
# The official plugins' public key (from PLUGIN_SIGNING_KEY, the seed
# `openidea-plugin keygen` writes), built into the app: their packages
# must carry its signature whatever the marketplace index lists. Without
# the secret the app takes the index's word, as development builds do.
- id: plugin-keys
env:
SIGNING_KEY: ${{ secrets.PLUGIN_SIGNING_KEY }}
run: |
if [ -n "$SIGNING_KEY" ]; then
seed="$(printf %s "$SIGNING_KEY" | base64 -d | od -An -tx1 | tr -d ' \n')"
key="$(printf '302e020100300506032b657004220420%s' "$seed" | xxd -r -p |
openssl pkey -inform DER -pubout -outform DER | tail -c 32 | base64)"
echo "keys=$key" >> "$GITHUB_OUTPUT"
fi
- id: release
uses: actions/github-script@v7
with:
@@ -73,8 +57,6 @@ jobs:
build:
needs: create-release
name: Build (${{ matrix.name }})
env:
OPENIDEA_OFFICIAL_PLUGIN_KEYS: ${{ needs.create-release.outputs.official_plugin_keys }}
strategy:
fail-fast: false
matrix:
@@ -202,11 +184,7 @@ jobs:
# (releases/latest/download/plugins-index.json). They are not in the
# installers: users install them from Settings › Plugins. Without the
# signing key (secret PLUGIN_SIGNING_KEY: the file `openidea-plugin keygen`
# writes) nothing is published and the release goes on. The index also lists
# the community plugins of the registry (COMMUNITY_PLUGINS_INDEX; refreshed
# between releases by plugins-index.yml), and the release carries the
# openidea-plugin tool for Linux x64, which that workflow and the registry's
# checks run.
# writes) nothing is published and the release goes on.
plugins:
needs: create-release
name: Official plugins
@@ -259,17 +237,11 @@ jobs:
version="$(sed -n 's/^version *= *"\(.*\)".*/\1/p' "$dir/plugin.toml" | head -1)"
"$tool" pack "$dir" --key "$key" --publisher openidea --out "$out/$id-$version.oiplugin"
done
# The published index's revocations, earlier keys and community plugins carry over.
# The published index's revocations and earlier keys carry over.
curl -fsSL "https://github.com/$GITHUB_REPOSITORY/releases/latest/download/plugins-index.json" \
-o "$out/plugins-index.json" || rm -f "$out/plugins-index.json"
# Downloads point at this release: older versions kept for older IDEs stay where they were published.
"$tool" index "$out" --publisher openidea --name OpenIDEA --key "$key" --out "$out/plugins-index.json" \
--download-base "https://github.com/$GITHUB_REPOSITORY/releases/download/$TAG"
"$tool" merge "$out/plugins-index.json" "$COMMUNITY_PLUGINS_INDEX" --keep openidea ||
echo "::warning::the community plugins were not refreshed"
gzip -9 -c "$tool" > "$out/openidea-plugin-linux-x86_64.gz"
gh release upload "$TAG" "$out"/*.oiplugin "$out/plugins-index.json" \
"$out/openidea-plugin-linux-x86_64.gz" --clobber
"$tool" index "$out" --publisher openidea --name OpenIDEA --key "$key" --out "$out/plugins-index.json"
gh release upload "$TAG" "$out"/*.oiplugin "$out/plugins-index.json" --clobber
# The updater manifest (latest.json), built once from the signed bundles every build uploaded.
# Fails when a platform is missing, so a release never ships a manifest without it.
-1
View File
@@ -39,7 +39,6 @@ Visual style: dark, low-contrast greys, one cool accent
icons from `src/ui/icons.tsx`, never emoji. Real `<button>`/`<input>`; `aria-label` on icon buttons.
- User-visible text goes through the text catalog: add every new string to both
`src/platform/locales/text/en.json` and `zh.json` (`textCatalog.test.ts` enforces it).
Other locales may lack entries; their UI shows the English text until a translator adds it.
- Keep comments sparse and purposeful; match surrounding style.
- For GUI verification, prefer Tauri MCP; use Computer Use only when Tauri MCP is
unavailable or cannot cover the required checks.
-5
View File
@@ -31,10 +31,5 @@ Project layout, coding rules, commands and checks are in [AGENTS.md](AGENTS.md).
- If a change needs a product decision you can't make, ask in the issue and stop
rather than guessing.
Plugins: pull requests here add or change official plugins (`plugins/official/`)
and the plugin API. Community plugins live in their authors' repositories and
are listed through [l0ng-ai/openidea-plugins](https://github.com/l0ng-ai/openidea-plugins);
see [`plugins/README.md`](plugins/README.md) § 8.1.
A check on each PR adds `needs-info` when the issue link or the verification
section is missing, and removes it once they're filled in.
Generated
-28
View File
@@ -4154,11 +4154,8 @@ dependencies = [
"dirs 6.0.0",
"futures",
"ide-core",
"ide-http",
"ide-ssh",
"parking_lot",
"reqwest",
"rustls",
"serde",
"serde_json",
"specta",
@@ -4169,7 +4166,6 @@ dependencies = [
"tokio-util",
"toml 0.9.12+spec-1.1.0",
"tracing",
"uuid",
]
[[package]]
@@ -4248,7 +4244,6 @@ dependencies = [
"ide-coverage",
"ide-dap",
"ide-db",
"ide-docker",
"ide-extension",
"ide-http",
"ide-index",
@@ -4272,11 +4267,8 @@ dependencies = [
"ide-workspace",
"ignore",
"parking_lot",
"quick-xml",
"regex-syntax",
"reqwest",
"rustix",
"semver",
"serde",
"serde_json",
"specta",
@@ -4341,7 +4333,6 @@ dependencies = [
"parking_lot",
"serde",
"serde_json",
"tempfile",
"thiserror 2.0.21",
"tokio",
"tokio-util",
@@ -7141,7 +7132,6 @@ dependencies = [
"rustls-platform-verifier",
"serde",
"serde_json",
"serde_urlencoded",
"sync_wrapper",
"tokio",
"tokio-rustls",
@@ -7500,12 +7490,6 @@ dependencies = [
"wait-timeout",
]
[[package]]
name = "ryu"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f"
[[package]]
name = "saa"
version = "5.6.2"
@@ -7810,18 +7794,6 @@ dependencies = [
"serde_core",
]
[[package]]
name = "serde_urlencoded"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3491c14715ca2294c4d6a88f15e84739788c1d030eed8c110436aafdaa2f3fd"
dependencies = [
"form_urlencoded",
"itoa",
"ryu",
"serde",
]
[[package]]
name = "serde_with"
version = "3.24.0"
-1
View File
@@ -86,7 +86,6 @@ zip = { version = "9", default-features = false, features = ["deflate"] }
# search, index, language
grep-searcher = "0.1"
grep-regex = "0.1"
regex-syntax = "0.8"
grep-matcher = "0.1"
nucleo-matcher = "0.3"
redb = "4"
+5 -21
View File
@@ -58,8 +58,6 @@ pub(crate) const LIMITS_UPDATE: &str = "openidea_limits";
const MAX_EMBED: u64 = 256 * 1024;
/// How long a probe waits for the agent to answer `initialize`.
const PROBE_TIMEOUT: Duration = Duration::from_secs(30);
/// How long deleting a task waits for its agent to exit before removing the worktree.
const EXIT_WAIT: Duration = Duration::from_secs(5);
/// The agent tasks of one project.
pub struct AgentManager {
@@ -347,14 +345,6 @@ impl AgentManager {
provider::set_builtin_command(&dir, id, command.as_deref())
}
/// Sets the environment variables built-in `id` starts with; empty clears them.
pub fn set_builtin_env(&self, id: &str, env: std::collections::BTreeMap<String, String>) -> IdeResult<()> {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::set_builtin_env(&dir, id, env)
}
/// Re-reads configured/installed agents, drops cached probes, and notifies subscribers.
pub fn reload_providers(&self) {
let specs = provider::load(self.shared.paths.custom_agents_dir.as_deref());
@@ -841,7 +831,7 @@ impl AgentManager {
/// Forgets the task; with `remove_worktree` also deletes its worktree (the branch stays).
pub async fn delete(&self, id: &str, remove_worktree: bool) -> IdeResult<()> {
let task = self.task(id)?;
let live = task.kill();
task.kill();
let (isolation, cwd, root) = {
let s = task.state.lock();
(
@@ -853,11 +843,6 @@ impl AgentManager {
// A copy of the task (a fork) may work in the same worktree.
let shared = self.tasks.read().values().any(|t| t.id != id && t.state.lock().task.cwd.0 == cwd);
if remove_worktree && !shared && isolation == Isolation::Worktree && cwd.exists() {
// On Windows the agent's processes lock the folder (their working directory,
// open files) until they have exited.
if let Some(peer) = live {
let _ = tokio::time::timeout(EXIT_WAIT, peer.exited()).await;
}
let worktrees = self.shared.paths.worktrees_dir.clone();
blocking(move || worktree::remove(&root, &worktrees, &cwd)).await?;
}
@@ -1612,12 +1597,11 @@ impl Task {
}
}
/// Stops the agent; returns its connection, to wait for its process to exit.
fn kill(&self) -> Option<acp::Conn> {
fn kill(&self) {
self.cancel_permissions();
let live = self.live.lock().take()?;
live.peer.kill();
Some(live.peer)
if let Some(live) = self.live.lock().take() {
live.peer.kill();
}
}
}
-3
View File
@@ -3,7 +3,6 @@
use ide_core::IpcPath;
use serde::{Deserialize, Serialize};
use specta::Type;
use std::collections::BTreeMap;
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
@@ -358,8 +357,6 @@ pub struct AgentProvider {
pub command_read_only_reason: Option<AgentCommandReadOnlyReason>,
/// Whether the command was found (on `PATH`).
pub installed: bool,
/// Environment variables it starts with (from its agent file).
pub env: BTreeMap<String, String>,
/// Settings (models, modes…) the agent offered last time it ran; empty before its first session.
pub config: Vec<AgentConfigOption>,
/// Slash commands it offered last time.
+2 -30
View File
@@ -124,7 +124,6 @@ pub(crate) fn describe(spec: &ProviderSpec) -> AgentProvider {
command_editable: reason.is_none(),
command_read_only_reason: reason,
installed: installed(&spec.command),
env: spec.env.iter().cloned().collect(),
config: vec![],
commands: vec![],
source: spec.source.clone(),
@@ -134,16 +133,6 @@ pub(crate) fn describe(spec: &ProviderSpec) -> AgentProvider {
/// Overrides built-in `id`'s command through its agent file, preserving other
/// customizations. `None` restores the built-in command and removes a redundant file.
pub(crate) fn set_builtin_command(custom_dir: &Path, id: &str, command: Option<&str>) -> IdeResult<()> {
update_builtin(custom_dir, id, |agent, builtin| agent.command = command.unwrap_or(&builtin.command).to_owned())
}
/// The environment built-in `id` starts with, through its agent file (empty clears it).
pub(crate) fn set_builtin_env(custom_dir: &Path, id: &str, env: BTreeMap<String, String>) -> IdeResult<()> {
update_builtin(custom_dir, id, |agent, _| agent.env = env)
}
/// Changes built-in `id`'s agent file; a file left with only the built-in's values is removed.
fn update_builtin(custom_dir: &Path, id: &str, change: impl FnOnce(&mut CustomAgent, &ProviderSpec)) -> IdeResult<()> {
let Some(builtin) = builtin_spec(id) else {
return Err(IdeError::invalid(format!("{id} is not a built-in agent")));
};
@@ -163,8 +152,7 @@ fn update_builtin(custom_dir: &Path, id: &str, change: impl FnOnce(&mut CustomAg
if agent.registry.is_some() {
return Err(IdeError::invalid(format!("{id} is installed from the ACP Registry")));
}
change(&mut agent, &builtin);
if agent.command == builtin.command
if command.is_none()
&& agent.name.as_deref() == Some(builtin.name.as_str())
&& agent.driver == builtin.driver
&& agent.args == builtin.args
@@ -176,6 +164,7 @@ fn update_builtin(custom_dir: &Path, id: &str, change: impl FnOnce(&mut CustomAg
}
return Ok(());
}
agent.command = command.unwrap_or(&builtin.command).to_owned();
agent.write(&file)?;
Ok(())
}
@@ -379,23 +368,6 @@ mod tests {
assert!(installed("sh"));
}
#[test]
fn set_builtin_env_writes_and_clears_the_environment() {
let dir = tempfile::tempdir().unwrap();
let env = BTreeMap::from([("CLAUDE_CODE_ENTRYPOINT".to_owned(), "openidea".to_owned())]);
set_builtin_env(dir.path(), "claude", env.clone()).unwrap();
let claude = load(Some(dir.path())).into_iter().find(|s| s.id == "claude").unwrap();
assert_eq!(claude.env, vec![("CLAUDE_CODE_ENTRYPOINT".to_owned(), "openidea".to_owned())]);
assert_eq!(describe(&claude).env, env);
// The command override and the environment are kept independently.
set_builtin_command(dir.path(), "claude", Some("/opt/claude")).unwrap();
set_builtin_command(dir.path(), "claude", None).unwrap();
assert!(dir.path().join("claude.toml").is_file());
set_builtin_env(dir.path(), "claude", BTreeMap::new()).unwrap();
assert!(!dir.path().join("claude.toml").exists());
set_builtin_env(dir.path(), "not-a-builtin", env).unwrap_err();
}
#[test]
fn set_builtin_command_overrides_and_clears_a_builtin() {
let dir = tempfile::tempdir().unwrap();
-111
View File
@@ -1109,114 +1109,3 @@ async fn idle_agents_stop_and_resume_their_session_on_the_next_message() {
assert!(live(&mgr, &task.id));
mgr.shutdown();
}
/// Deleting a task whose agent still runs in the worktree removes both and leaves no
/// process behind, whether it is waiting for permission or idle without session/load.
#[tokio::test(flavor = "multi_thread")]
async fn deleting_a_task_waits_for_its_agent_to_exit_before_removing_the_worktree() {
for no_load in [false, true] {
delete_running_agent(no_load).await;
}
}
async fn delete_running_agent(no_load: bool) {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
let git = |args: &[&str]| {
let out = std::process::Command::new("git")
.args(["-c", "user.name=t", "-c", "user.email=t@t", "-c", "commit.gpgsign=false"])
.args(args)
.current_dir(&root)
.output()
.unwrap();
assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr));
};
git(&["init", "-q", "-b", "main"]);
std::fs::write(root.join("file.txt"), "original\n").unwrap();
git(&["add", "."]);
git(&["commit", "-qm", "initial"]);
let agent = Arc::new(FakeAgent { no_load, ..FakeAgent::default() });
let pids: Arc<Mutex<Vec<u32>>> = Arc::default();
let (fake, started) = (agent.clone(), pids.clone());
// The fake agent, served on behalf of a real process that exits only when killed.
let connector: Connector = Arc::new(move |_spec, cwd, client, _stderr| {
#[cfg(unix)]
let mut cmd = {
let mut cmd = tokio::process::Command::new("sleep");
cmd.arg("30");
cmd
};
#[cfg(windows)]
let mut cmd = {
let mut cmd = tokio::process::Command::new("cmd");
cmd.args(["/D", "/C", "echo ready>agent-ready & ping -n 30 127.0.0.1 >nul"]);
cmd
};
ide_core::process::prepare(cmd.as_std_mut());
ide_core::process::own_group(cmd.as_std_mut());
cmd.current_dir(cwd).kill_on_drop(true);
let child = cmd.spawn()?;
started.lock().push(child.id().unwrap());
let (a, b) = duplex(1 << 20);
let (ar, aw) = tokio::io::split(a);
let (br, bw) = tokio::io::split(b);
// `connect_with_child` frames with Content-Length.
fake.servers.lock().push(Peer::connect(br, bw, fake.clone()));
Ok(Peer::connect_with_child(ar, aw, client, child))
});
let mgr = AgentManager::with_connector(
AgentPaths {
project_root: root.clone(),
store_dir: root.join("store"),
worktrees_dir: root.join("worktrees"),
models_file: root.join("models.json"),
custom_agents_dir: None,
home_dir: None,
},
tokio::runtime::Handle::current(),
connector,
);
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text(if no_load { "again" } else { "Fix the lexer" }),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Worktree,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, if no_load { AgentStatus::Ready } else { AgentStatus::NeedsInput }).await;
let pid = *pids.lock().last().expect("the agent was started");
#[cfg(windows)]
{
tokio::time::timeout(Duration::from_secs(5), async {
while !task.cwd.0.join("agent-ready").exists() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await
.unwrap();
assert_eq!(
std::fs::rename(&task.cwd.0, task.cwd.0.with_extension("moved")).unwrap_err().raw_os_error(),
Some(32),
"the running agent really locks its worktree"
);
}
mgr.delete(&task.id, true).await.unwrap();
assert!(!task.cwd.0.exists());
// Killed and reaped: not even a zombie is left.
#[cfg(unix)]
let alive = std::process::Command::new("kill").args(["-0", &pid.to_string()]).status().unwrap().success();
#[cfg(windows)]
let alive = {
let mut cmd = std::process::Command::new("tasklist");
ide_core::process::prepare(&mut cmd).args(["/FI", &format!("PID eq {pid}"), "/FO", "CSV", "/NH"]);
String::from_utf8_lossy(&cmd.output().unwrap().stdout).contains(&format!("\"{pid}\""))
};
assert!(!alive, "the agent (pid {pid}) is still running");
}
+2 -70
View File
@@ -5,7 +5,7 @@ use ide_core::{IdeError, IdeResult, IpcPath};
use ide_vcs::{ChangeKind, Repo};
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use std::time::{Duration, UNIX_EPOCH};
use std::time::UNIX_EPOCH;
pub(crate) struct Created {
pub path: PathBuf,
@@ -65,27 +65,7 @@ pub(crate) fn remove(project_root: &Path, worktrees_dir: &Path, path: &Path) ->
if !path.starts_with(worktrees_dir) || path == worktrees_dir {
return Err(err);
}
retrying(REMOVE_ATTEMPTS, REMOVE_PAUSE, || std::fs::remove_dir_all(path))
.map_err(|e| IdeError::internal(format!("{}: {e}", path.display())))
}
/// On Windows a folder can't be deleted while a process still has a file in it open
/// (os error 32): an agent's helper that outlived it, an antivirus scan, the search indexer.
/// Those let go within moments, so removing is retried for a few seconds.
const REMOVE_ATTEMPTS: u32 = 10;
const REMOVE_PAUSE: Duration = Duration::from_millis(300);
fn retrying(attempts: u32, pause: Duration, mut op: impl FnMut() -> std::io::Result<()>) -> std::io::Result<()> {
for _ in 1..attempts {
match op() {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => std::thread::sleep(pause),
_ => return Ok(()),
}
}
match op() {
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()),
r => r,
}
std::fs::remove_dir_all(path).map_err(|e| IdeError::internal(e.to_string()))
}
/// Files under `cwd` that differ from `base` (untracked files included), with line counts.
@@ -169,54 +149,6 @@ fn unique_slug(base: &str, taken: impl Fn(&str) -> bool) -> String {
mod tests {
use super::*;
#[test]
fn removing_is_retried_while_a_file_is_still_in_use() {
use std::io::{Error, ErrorKind};
// A sharing violation as Windows reports it (os error 32), a vanished folder, other errors.
let in_use = || Error::from_raw_os_error(32);
for (fails, outcome) in [(0, true), (3, true), (REMOVE_ATTEMPTS - 1, true), (REMOVE_ATTEMPTS, false)] {
let mut calls = 0;
let r = retrying(REMOVE_ATTEMPTS, Duration::ZERO, || {
calls += 1;
if calls <= fails { Err(in_use()) } else { Ok(()) }
});
assert_eq!(r.is_ok(), outcome, "{fails} failures");
assert_eq!(calls, (fails + 1).min(REMOVE_ATTEMPTS));
}
let mut calls = 0;
let gone = retrying(REMOVE_ATTEMPTS, Duration::ZERO, || {
calls += 1;
if calls == 1 { Err(in_use()) } else { Err(Error::from(ErrorKind::NotFound)) }
});
assert!(gone.is_ok(), "deleted meanwhile: done");
assert_eq!(calls, 2);
}
#[cfg(windows)]
#[test]
fn removing_waits_for_a_windows_file_lock_to_be_released() {
use std::os::windows::fs::OpenOptionsExt;
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
std::fs::write(repo.join("a.txt"), "one\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
let worktrees = tmp.path().join("wt");
let created = create(&repo, &worktrees, "Locked file", None).unwrap();
let file = std::fs::OpenOptions::new().read(true).share_mode(0).open(created.path.join("a.txt")).unwrap();
assert_eq!(std::fs::remove_file(created.path.join("a.txt")).unwrap_err().raw_os_error(), Some(32));
let release = std::thread::spawn(move || {
std::thread::sleep(Duration::from_millis(800));
drop(file);
});
let result = remove(&repo, &worktrees, &created.path);
release.join().unwrap();
result.unwrap();
assert!(!created.path.exists());
}
#[test]
fn slugs_are_branch_safe() {
assert_eq!(slugify("Fix lexer UTF-16 columns!"), "fix-lexer-utf-16-columns");
+13 -104
View File
@@ -10,7 +10,6 @@ pub use fingerprint::{LaunchFingerprint, launch_fingerprint};
use crate::process::{self, Stream};
use crate::{BuildModel, maven};
use ide_core::settings::CMakeProfile;
use ide_core::{CancellationToken, Diagnostic, IdeError, IdeResult, Pos, Range, Severity};
use parse::{ColUnit, Format, OutputParser, RawMessage};
use serde::{Deserialize, Serialize};
@@ -42,8 +41,6 @@ pub enum BuildSystem {
Go,
#[serde(rename = "typescript")]
TypeScript,
#[serde(rename = "cmake")]
CMake,
}
/// One command of a build.
@@ -155,8 +152,6 @@ fn detect_dir(dir: &Path, model: &BuildModel, root: &Path) -> Option<BuildSystem
Some(BuildSystem::Go)
} else if has(dir, &["tsconfig.json", "package.json"]) {
Some(BuildSystem::TypeScript)
} else if has(dir, &["CMakeLists.txt"]) {
Some(BuildSystem::CMake)
} else {
None
}
@@ -181,14 +176,7 @@ fn ancestors_within<'a>(root: &'a Path, file: &'a Path) -> impl Iterator<Item =
}
/// The build of `root`, or of the module containing `file` (Build Module / Recompile).
/// `cmake`: the CMake profiles of the settings (the first one is built), if any.
pub fn plan(
root: &Path,
model: &BuildModel,
kind: BuildKind,
file: Option<&Path>,
cmake: Option<&[CMakeProfile]>,
) -> IdeResult<BuildPlan> {
pub fn plan(root: &Path, model: &BuildModel, kind: BuildKind, file: Option<&Path>) -> IdeResult<BuildPlan> {
let project_name = root.file_name().map_or_else(|| "project".to_owned(), |n| n.to_string_lossy().into_owned());
let module_dir = match file {
Some(f) => {
@@ -204,9 +192,7 @@ pub fn plan(
let system = detect_dir(&dir, model, root)
.or_else(|| if model.gradle.is_some() { Some(BuildSystem::Gradle) } else { None })
.or_else(|| model.maven.as_ref().filter(|m| !m.projects.is_empty()).map(|_| BuildSystem::Maven))
.ok_or_else(|| {
IdeError::not_found("no supported build system (Maven, Gradle, Cargo, Go, TypeScript, CMake) found")
})?;
.ok_or_else(|| IdeError::not_found("no supported build system (Maven, Gradle, Cargo, Go, TypeScript) found"))?;
let module = module_dir.as_deref().filter(|d| *d != root);
let title = match module {
Some(d) => format!(
@@ -223,7 +209,6 @@ pub fn plan(
// Go builds the file's package (a module holds many).
BuildSystem::Go => go_steps(file.and_then(Path::parent).unwrap_or(&dir), kind),
BuildSystem::TypeScript => ts_steps(&dir, kind),
BuildSystem::CMake => cmake_steps(&dir, kind, cmake.and_then(<[_]>::first)),
};
Ok(BuildPlan { system, title, steps })
}
@@ -317,50 +302,6 @@ fn cargo_steps(dir: &Path, kind: BuildKind) -> Vec<BuildStep> {
}
}
/// CLion's Build Project for a CMake project: configures the build directory when it has no
/// `CMakeCache.txt` yet, then `cmake --build` (`--clean-first` to rebuild). The directory and
/// options come from the CMake profile; without one, a configured directory of the project
/// (the first by name), else `build`, as the generated run configurations use.
fn cmake_steps(dir: &Path, kind: BuildKind, profile: Option<&CMakeProfile>) -> Vec<BuildStep> {
let build_dir = match profile {
Some(p) if !p.build_dir.trim().is_empty() => p.build_dir.trim().to_owned(),
Some(p) => format!("cmake-build-{}", p.build_type.to_lowercase()),
None => {
let mut configured: Vec<String> = std::fs::read_dir(dir)
.into_iter()
.flatten()
.flatten()
.filter(|e| e.path().join("CMakeCache.txt").is_file())
.map(|e| e.file_name().to_string_lossy().into_owned())
.collect();
configured.sort();
configured.into_iter().next().unwrap_or_else(|| "build".into())
}
};
let build_type = profile.map_or("Debug", |p| p.build_type.as_str());
let env = profile.map(|p| p.env.clone()).unwrap_or_default();
let mut steps = Vec::new();
if !dir.join(&build_dir).join("CMakeCache.txt").is_file() {
let mut configure = BuildStep::new("cmake", &["-S", ".", "-B", &build_dir], dir, Format::Gcc);
if let Some(p) = profile.filter(|p| !p.generator.trim().is_empty()) {
configure.args.extend(["-G".into(), p.generator.clone()]);
}
configure.args.push(format!("-DCMAKE_BUILD_TYPE={build_type}"));
configure.args.push("-DCMAKE_EXPORT_COMPILE_COMMANDS=ON".into());
configure.args.extend(profile.map(|p| p.cmake_args.clone()).unwrap_or_default());
configure.env = env.clone();
steps.push(configure);
}
let mut build = BuildStep::new("cmake", &["--build", &build_dir, "--config", build_type], dir, Format::Gcc);
if kind == BuildKind::Rebuild {
build.args.push("--clean-first".into());
}
build.args.extend(profile.map(|p| p.build_args.clone()).unwrap_or_default());
build.env = env;
steps.push(build);
steps
}
fn go_steps(dir: &Path, kind: BuildKind) -> Vec<BuildStep> {
let module_root = dir.ancestors().find(|d| d.join("go.mod").is_file()).unwrap_or(dir);
let target = if module_root == dir { "./...".to_owned() } else { format!("./{}/...", rel_str(module_root, dir)) };
@@ -746,7 +687,7 @@ fn launch_plan(root: &Path, model: &BuildModel, module: &Path, test: bool) -> Id
} else {
BuildKind::Build
};
let mut plan = plan(root, model, kind, Some(module), None)?;
let mut plan = plan(root, model, kind, Some(module))?;
if plan.system == BuildSystem::Maven && !test {
for step in &mut plan.steps {
if let Some(goal) = step.args.last_mut() {
@@ -855,12 +796,12 @@ mod tests {
write(&root, "core/pom.xml", "<project><artifactId>core</artifactId></project>");
write(&root, "core/src/main/java/A.java", "class A {}");
let model = crate::load(&root, &tmp.path().join("cache"), None);
let p = plan(&root, &model, BuildKind::Build, None, None).unwrap();
let p = plan(&root, &model, BuildKind::Build, None).unwrap();
assert_eq!(p.system, BuildSystem::Maven);
assert_eq!(p.title, "demo");
assert_eq!(p.steps.len(), 1);
assert_eq!(p.steps[0].args, ["-B", "-Dstyle.color=never", "test-compile"]);
let p = plan(&root, &model, BuildKind::Rebuild, Some(&root.join("core/src/main/java/A.java")), None).unwrap();
let p = plan(&root, &model, BuildKind::Rebuild, Some(&root.join("core/src/main/java/A.java"))).unwrap();
assert_eq!(p.steps[0].args, ["-B", "-Dstyle.color=never", "-pl", "core", "-am", "clean", "test-compile"]);
assert_eq!(p.steps[0].cwd, root);
assert_eq!(p.title, "demo: build module 'core'");
@@ -869,24 +810,24 @@ mod tests {
write(&g, "settings.gradle", "include 'app'");
write(&g, "app/build.gradle", "apply plugin: 'java'");
let model = crate::load(&g, &tmp.path().join("cache2"), None);
let p = plan(&g, &model, BuildKind::Build, Some(&g.join("app/src/A.java")), None).unwrap();
let p = plan(&g, &model, BuildKind::Build, Some(&g.join("app/src/A.java"))).unwrap();
assert_eq!(p.system, BuildSystem::Gradle);
assert_eq!(p.steps[0].args, ["--console=plain", ":app:classes", ":app:testClasses"]);
assert_eq!(
plan(&g, &model, BuildKind::Artifacts, None, None).unwrap().steps[0].args,
plan(&g, &model, BuildKind::Artifacts, None).unwrap().steps[0].args,
["--console=plain", "assemble"]
);
let c = tmp.path().join("c");
write(&c, "Cargo.toml", "[package]\nname='c'");
let p = plan(&c, &BuildModel::default(), BuildKind::Rebuild, None, None).unwrap();
let p = plan(&c, &BuildModel::default(), BuildKind::Rebuild, None).unwrap();
assert_eq!(p.system, BuildSystem::Cargo);
assert_eq!(p.steps.len(), 2);
assert_eq!(p.steps[0].args[0], "clean");
let go = tmp.path().join("go");
write(&go, "go.mod", "module x");
let p = plan(&go, &BuildModel::default(), BuildKind::Build, Some(&go.join("pkg/a/a.go")), None).unwrap();
let p = plan(&go, &BuildModel::default(), BuildKind::Build, Some(&go.join("pkg/a/a.go"))).unwrap();
assert_eq!(p.steps[0].args, ["build", "./pkg/a/..."]);
assert_eq!(p.steps[0].cwd, go);
@@ -894,51 +835,19 @@ mod tests {
write(&ts, "package.json", r#"{"scripts":{"build":"vite build"}}"#);
write(&ts, "tsconfig.json", "{}");
write(&ts, "pnpm-lock.yaml", "");
let p = plan(&ts, &BuildModel::default(), BuildKind::Build, None, None).unwrap();
let p = plan(&ts, &BuildModel::default(), BuildKind::Build, None).unwrap();
assert_eq!(p.steps[0].args, ["--noEmit", "--pretty", "false", "-p", "."]);
let p = plan(&ts, &BuildModel::default(), BuildKind::Artifacts, None, None).unwrap();
let p = plan(&ts, &BuildModel::default(), BuildKind::Artifacts, None).unwrap();
assert_eq!(
(p.steps[0].program.as_str(), p.steps[0].args.as_slice()),
("pnpm", &["run".to_owned(), "build".to_owned()][..])
);
let cm = tmp.path().join("cm");
write(&cm, "CMakeLists.txt", "project(app)");
let args = |p: &BuildPlan| p.steps.iter().map(|s| s.args.join(" ")).collect::<Vec<_>>();
let p = plan(&cm, &BuildModel::default(), BuildKind::Build, None, None).unwrap();
assert_eq!(p.system, BuildSystem::CMake);
assert_eq!(
args(&p),
[
"-S . -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_EXPORT_COMPILE_COMMANDS=ON",
"--build build --config Debug"
],
"an unconfigured project is configured first"
);
write(&cm, "cmake-build-release/CMakeCache.txt", "");
let p = plan(&cm, &BuildModel::default(), BuildKind::Rebuild, None, None).unwrap();
assert_eq!(
args(&p),
["--build cmake-build-release --config Debug --clean-first"],
"a configured directory is reused"
);
let profile = CMakeProfile {
build_type: "Release".into(),
generator: "Ninja".into(),
build_args: vec!["-j".into(), "8".into()],
..CMakeProfile::default()
};
let p =
plan(&cm, &BuildModel::default(), BuildKind::Build, None, Some(std::slice::from_ref(&profile))).unwrap();
assert_eq!(args(&p), ["--build cmake-build-release --config Release -j 8"]);
let p = plan(&cm, &BuildModel::default(), BuildKind::Build, None, Some(&[CMakeProfile::default()])).unwrap();
assert_eq!(p.steps[0].args[..4], ["-S", ".", "-B", "cmake-build-debug"]);
let siblings = tmp.path().join("siblings");
write(&siblings, "core/pom.xml", "<project><artifactId>core</artifactId></project>");
write(&siblings, "system/pom.xml", "<project><artifactId>system</artifactId></project>");
let model = crate::load(&siblings, &tmp.path().join("cache3"), None);
let p = plan(&siblings, &model, BuildKind::Build, None, None).unwrap();
let p = plan(&siblings, &model, BuildKind::Build, None).unwrap();
assert_eq!(p.system, BuildSystem::Maven);
assert_eq!(
p.steps.iter().map(|s| s.cwd.clone()).collect::<Vec<_>>(),
@@ -947,7 +856,7 @@ mod tests {
let none = tmp.path().join("none");
std::fs::create_dir_all(&none).unwrap();
assert!(plan(&none, &BuildModel::default(), BuildKind::Build, None, None).is_err());
assert!(plan(&none, &BuildModel::default(), BuildKind::Build, None).is_err());
}
#[test]
-32
View File
@@ -22,8 +22,6 @@ pub(crate) enum Format {
Tsc,
/// `go build`: `./a.go:12:5: msg`.
Go,
/// GCC / Clang (through `cmake --build`): `src/a.cpp:12:5: error: msg`.
Gcc,
}
/// What a printed column counts.
@@ -298,17 +296,6 @@ fn go_line(line: &str) -> Option<RawMessage> {
(!msg.is_empty()).then(|| RawMessage::new(Severity::Error, msg, ColUnit::Byte).at(path, ln, col))
}
/// A GCC / Clang diagnostic line; notes and the context lines after it are left as output.
fn gcc_line(line: &str) -> Option<RawMessage> {
let (path, ln, col, rest) = path_line_col(line)?;
let rest = rest.strip_prefix(':')?.trim_start();
let (severity, msg) =
[("fatal error:", Severity::Error), ("error:", Severity::Error), ("warning:", Severity::Warning)]
.into_iter()
.find_map(|(prefix, severity)| rest.strip_prefix(prefix).map(|m| (severity, m.trim())))?;
(!msg.is_empty()).then(|| RawMessage::new(severity, msg, ColUnit::Byte).at(path, ln, col))
}
/// rustc's JSON diagnostic (`reason: compiler-message`) → message and its rendered text.
fn cargo_json(v: &Value) -> Option<(Option<RawMessage>, String)> {
if v.get("reason")?.as_str()? != "compiler-message" {
@@ -423,7 +410,6 @@ impl OutputParser {
Format::Gradle => javac_header(&line).or_else(|| kotlin_line(&line)),
Format::Tsc => tsc_line(&line),
Format::Go => go_line(&line),
Format::Gcc => gcc_line(&line),
Format::Cargo => None,
};
if let Some(msg) = parsed {
@@ -577,24 +563,6 @@ mod tests {
assert_eq!(out[2].text, "Cannot find name 'foo'.");
}
#[test]
fn gcc_errors_and_warnings() {
let out = parse(
Format::Gcc,
"[1/2] Building CXX object CMakeFiles/app.dir/main.cpp.o\n\
/p/src/main.cpp:12:5: error: use of undeclared identifier 'foo'\n\
\x20 12 | foo();\n\
/p/src/util.h:3:10: warning: unused variable 'x' [-Wunused-variable]\n\
/p/src/main.cpp:4:10: fatal error: 'missing.h' file not found\n\
/p/src/main.cpp:2:1: note: in file included from here\n",
);
assert_eq!(out.len(), 3, "{out:?}");
assert_eq!(out[0].severity, Severity::Error);
assert_eq!(out[0].text, "use of undeclared identifier 'foo'");
assert_eq!(out[1].severity, Severity::Warning);
assert_eq!(out[2].text, "'missing.h' file not found");
}
#[test]
fn go_errors() {
let out = parse(
-257
View File
@@ -1,257 +0,0 @@
//! A Gradle project's resolved dependencies for the Dependency Analyzer:
//! `gradle <project>:dependencies --configuration <name>`, parsed from its tree report.
use crate::gradle;
use crate::process;
use crate::sync::{SyncOptions, error_summary};
use ide_core::{CancellationToken, IdeError, IdeResult};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::path::Path;
/// One line of the report: an occurrence of a dependency where it is requested.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct GradleDependency {
/// `group`, empty for a project dependency.
pub group: String,
/// Artifact name, or the project path (`:core`) of a project dependency.
pub name: String,
/// The version asked for here; `None` when a platform or constraint supplies it.
pub requested: Option<String>,
/// The version Gradle selected (`-> 1.2`), when it differs from the request.
pub selected: Option<String>,
/// A project of the build (`project :core`, or a module substituted by one).
pub project: bool,
/// `(*)`: listed with its dependencies earlier in the tree.
pub repeated: bool,
/// `(c)`: a dependency constraint rather than a dependency.
pub constraint: bool,
/// `(n)` / `FAILED`: not resolved.
pub unresolved: bool,
pub children: Vec<GradleDependency>,
}
/// Parses the tree of a `dependencies --configuration` report (lines `+--- g:a:1.0 -> 1.1 (*)`).
pub fn parse_report(output: &str) -> Vec<GradleDependency> {
let mut roots: Vec<GradleDependency> = Vec::new();
// Path of child indexes from the roots to the last node of each depth.
let mut path: Vec<usize> = Vec::new();
for line in output.lines() {
let Some((depth, text)) = tree_line(line) else { continue };
let node = parse_entry(text);
path.truncate(depth);
if depth > path.len() {
continue; // A line deeper than its parent allows: not part of a well-formed tree.
}
let siblings = if depth == 0 {
&mut roots
} else {
let mut list = &mut roots;
for &i in &path[..depth - 1] {
list = &mut list[i].children;
}
let Some(parent) = list.get_mut(path[depth - 1]) else { continue };
&mut parent.children
};
siblings.push(node);
path.push(siblings.len() - 1);
}
roots
}
/// `| \--- g:a:1.0` → (1, "g:a:1.0").
fn tree_line(line: &str) -> Option<(usize, &str)> {
let mut rest = line;
let mut depth = 0;
loop {
if let Some(r) = rest.strip_prefix("+--- ").or_else(|| rest.strip_prefix("\\--- ")) {
return Some((depth, r.trim()));
}
rest = rest.strip_prefix("| ").or_else(|| rest.strip_prefix(" "))?;
depth += 1;
}
}
fn parse_entry(text: &str) -> GradleDependency {
let mut d = GradleDependency::default();
let mut text = text.trim();
loop {
if let Some(t) = text.strip_suffix(" (*)") {
d.repeated = true;
text = t;
} else if let Some(t) = text.strip_suffix(" (c)") {
d.constraint = true;
text = t;
} else if let Some(t) = text.strip_suffix(" (n)").or_else(|| text.strip_suffix(" FAILED")) {
d.unresolved = true;
text = t;
} else {
break;
}
text = text.trim_end();
}
let (left, right) = match text.split_once(" -> ") {
Some((l, r)) => (l.trim(), Some(r.trim())),
None => (text, None),
};
if let Some(project) = left.strip_prefix("project ") {
d.project = true;
d.name = project_path(project);
} else {
let mut parts = left.splitn(3, ':');
d.group = parts.next().unwrap_or_default().to_owned();
d.name = parts.next().unwrap_or_default().to_owned();
d.requested = parts.next().filter(|v| !v.is_empty()).map(str::to_owned);
}
if let Some(right) = right {
if let Some(project) = right.strip_prefix("project ") {
// A module substituted by a project of the build.
d.project = true;
d.selected = Some(project_path(project));
} else {
d.selected = Some(right.to_owned());
}
}
d
}
/// `:core`, or `':core'` as Gradle 9 quotes it.
fn project_path(text: &str) -> String {
let t = text.trim();
t.strip_prefix('\'').and_then(|t| t.strip_suffix('\'')).unwrap_or(t).to_owned()
}
/// Runs the `dependencies` report of `project` (a Gradle path, `:` for the root) for
/// `configuration` and returns its tree.
pub fn gradle_dependencies(
root: &Path,
options: &SyncOptions,
project: &str,
configuration: &str,
cancel: &CancellationToken,
) -> IdeResult<Vec<GradleDependency>> {
if !configuration.chars().all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '-') || configuration.is_empty() {
return Err(IdeError::invalid(format!("not a configuration name: {configuration}")));
}
if !project.starts_with(':') || project.chars().any(char::is_whitespace) {
return Err(IdeError::invalid(format!("not a Gradle project path: {project}")));
}
let model = gradle::load(root).ok_or_else(|| IdeError::not_found("no Gradle build in this project"))?;
let task = if project == ":" { ":dependencies".to_owned() } else { format!("{project}:dependencies") };
let mut args: Vec<String> =
vec!["--console=plain".into(), "-q".into(), task, "--configuration".into(), configuration.into()];
if options.offline {
args.push("--offline".into());
}
let out = process::run(
options.gradle_command.as_deref().unwrap_or(&model.command),
&args,
&root.join(&model.dir),
&options.env,
cancel,
|_, _| {},
)?;
if !out.success {
return Err(IdeError::unavailable(error_summary(&out.text)));
}
Ok(parse_report(&out.text))
}
#[cfg(test)]
mod tests {
use super::*;
fn dep(group: &str, name: &str, requested: Option<&str>) -> GradleDependency {
GradleDependency {
group: group.into(),
name: name.into(),
requested: requested.map(Into::into),
..Default::default()
}
}
#[test]
fn parses_the_tree_with_selected_versions_and_markers() {
let report = "\
------------------------------------------------------------
Project ':app'
------------------------------------------------------------
runtimeClasspath - Runtime classpath of source set 'main'.
+--- org.springframework.boot:spring-boot-starter-web -> 3.2.0
| +--- org.springframework.boot:spring-boot-starter:3.2.0
| | \\--- org.yaml:snakeyaml:2.2
| \\--- com.fasterxml.jackson.core:jackson-databind:2.15.0 -> 2.15.3 (*)
+--- project ':core'
| \\--- com.google.guava:guava:{strictly 32.1.3-jre} -> 32.1.3-jre (c)
+--- org.example:legacy:1.0 -> project :legacy
\\--- org.missing:gone:9 FAILED
(c) - A dependency constraint, not a dependency. The dependency affected by the constraint occurs elsewhere in the tree.
(*) - Indicates repeated occurrences of a transitive dependency subtree. Gradle expands transitive dependency subtrees only once per project; repeat occurrences only display the root of the subtree, followed by this annotation.
";
let tree = parse_report(report);
assert_eq!(tree.len(), 4);
let web = &tree[0];
assert_eq!((web.group.as_str(), web.name.as_str()), ("org.springframework.boot", "spring-boot-starter-web"));
assert_eq!((web.requested.as_deref(), web.selected.as_deref()), (None, Some("3.2.0")));
assert_eq!(web.children.len(), 2);
assert_eq!(web.children[0].children, vec![dep("org.yaml", "snakeyaml", Some("2.2"))]);
let databind = &web.children[1];
assert_eq!((databind.requested.as_deref(), databind.selected.as_deref()), (Some("2.15.0"), Some("2.15.3")));
assert!(databind.repeated);
let core = &tree[1];
assert!(core.project && core.group.is_empty());
assert_eq!(core.name, ":core");
let guava = &core.children[0];
assert!(guava.constraint);
assert_eq!(guava.requested.as_deref(), Some("{strictly 32.1.3-jre}"));
let legacy = &tree[2];
assert!(legacy.project);
assert_eq!((legacy.name.as_str(), legacy.selected.as_deref()), ("legacy", Some(":legacy")));
assert!(tree[3].unresolved);
assert_eq!(tree[3].requested.as_deref(), Some("9"));
}
#[test]
fn an_empty_configuration_has_no_dependencies() {
assert!(
parse_report("compileClasspath - Compile classpath for source set 'main'.\nNo dependencies\n").is_empty()
);
}
#[test]
fn rejects_names_that_are_not_configurations_or_projects() {
let tmp = tempfile::tempdir().unwrap();
let options = SyncOptions::default();
let cancel = CancellationToken::new();
assert!(gradle_dependencies(tmp.path(), &options, ":", "x --init-script y", &cancel).is_err());
assert!(gradle_dependencies(tmp.path(), &options, "app", "runtimeClasspath", &cancel).is_err());
}
#[cfg(unix)]
#[test]
fn runs_the_report_of_the_project_and_configuration() {
use std::os::unix::fs::PermissionsExt;
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
std::fs::write(root.join("settings.gradle"), "include 'app'").unwrap();
std::fs::write(root.join("build.gradle"), "").unwrap();
let launcher = root.join("fake-gradle");
std::fs::write(&launcher, "#!/bin/sh\necho \"$@\" > args\necho '\\--- org.a:b:1.0'\n").unwrap();
std::fs::set_permissions(&launcher, std::fs::Permissions::from_mode(0o755)).unwrap();
let options = SyncOptions {
gradle_command: Some(launcher.to_string_lossy().into_owned()),
offline: true,
..Default::default()
};
let tree =
gradle_dependencies(root, &options, ":app", "testRuntimeClasspath", &CancellationToken::new()).unwrap();
assert_eq!(tree, vec![dep("org.a", "b", Some("1.0"))]);
assert_eq!(
std::fs::read_to_string(root.join("args")).unwrap().trim(),
"--console=plain -q :app:dependencies --configuration testRuntimeClasspath --offline"
);
}
}
+1 -1
View File
@@ -81,7 +81,7 @@ pub fn build(
cancel: &CancellationToken,
) -> IdeResult<Outcome> {
std::fs::create_dir_all(generation)?;
let mut plan = compile::plan(root, model, BuildKind::Build, None, None)?;
let mut plan = compile::plan(root, model, BuildKind::Build, None)?;
if plan.system != compile::BuildSystem::Maven {
return Err(IdeError::invalid("Kotlin bridge requires a Maven build root"));
}
+1 -4
View File
@@ -8,7 +8,6 @@
pub mod application_update;
pub mod compile;
pub mod gradle;
mod gradle_deps;
pub mod kotlin_bridge;
pub mod maven;
pub mod pom_edit;
@@ -18,7 +17,6 @@ pub mod web;
mod xml;
pub use gradle::{GradleModel, GradleProject, GradleTask};
pub use gradle_deps::{GradleDependency, gradle_dependencies};
pub use maven::{MavenDependency, MavenModel, MavenPlugin, MavenProfile, MavenProject, Omission};
pub use sync::{SyncOptions, SyncProgress, maven_profiles, save_maven_profiles, sync_gradle, sync_maven};
@@ -55,8 +53,7 @@ pub fn load_linked(
settings: &ide_core::settings::MavenSettings,
) -> BuildModel {
let maven = maven::is_maven_project(root).then(|| {
let profiles = sync::maven_profiles(cache_dir);
let mut m = maven::load_with(root, maven_repo.or_else(maven::default_local_repo), &profiles);
let mut m = maven::load_with_repo(root, maven_repo.or_else(maven::default_local_repo));
maven::UnlinkedProjects::from_model(&m, settings).filter(&mut m);
sync::apply_maven_cache(root, cache_dir, &mut m);
m
+6 -86
View File
@@ -460,8 +460,6 @@ const MAX_PARENT_DEPTH: usize = 10;
struct Resolver {
local_repo: Option<PathBuf>,
/// The chosen profiles (`-P` entries; `!id` / `-id` deactivates).
profiles: Vec<String>,
poms: HashMap<PathBuf, Option<RawPom>>,
contexts: HashMap<PathBuf, Inherited>,
}
@@ -486,16 +484,8 @@ pub fn local_repository_of(settings: &Path) -> Option<PathBuf> {
}
impl Resolver {
fn new(local_repo: Option<PathBuf>, profiles: &[String]) -> Self {
Self { local_repo, profiles: profiles.to_vec(), poms: HashMap::new(), contexts: HashMap::new() }
}
/// The active profiles of a POM: the chosen ones, and the `activeByDefault` ones not turned
/// off while none of the POM's other profiles is chosen (Maven's rule).
fn active<'a>(&self, profiles: &'a [RawProfile]) -> impl Iterator<Item = &'a RawProfile> {
let chosen = |id: &str| profile_choice(&self.profiles, id);
let any_chosen = profiles.iter().any(|p| chosen(&p.id) == Some(true));
profiles.iter().filter(move |p| chosen(&p.id).unwrap_or(p.active_by_default && !any_chosen))
fn new(local_repo: Option<PathBuf>) -> Self {
Self { local_repo, poms: HashMap::new(), contexts: HashMap::new() }
}
fn pom(&mut self, path: &Path) -> Option<RawPom> {
@@ -560,7 +550,7 @@ impl Resolver {
if pom.compiler_release.is_some() {
ctx.compiler_release = pom.compiler_release.clone();
}
for p in self.active(&pom.profiles) {
for p in pom.profiles.iter().filter(|p| p.active_by_default) {
merge_by(&mut ctx.properties, &p.properties, |(k, _)| k.clone());
merge_by(&mut ctx.deps, &p.deps, RawDep::key);
merge_plugins(&mut ctx.plugins, &p.plugins);
@@ -676,7 +666,7 @@ pub(crate) fn launch_inputs(
}
Some(())
}
let mut resolver = Resolver::new(local_repo.or_else(default_local_repo), &[]);
let mut resolver = Resolver::new(local_repo.or_else(default_local_repo));
let mut out = Vec::new();
for project in projects {
let dir = root.join(&project.dir);
@@ -1065,32 +1055,8 @@ pub fn load(root: &Path) -> MavenModel {
load_with_repo(root, default_local_repo())
}
/// [`load`] with the chosen `profiles` (`-P` entries; `!id` deactivates): their modules,
/// dependencies and properties are part of the model, those turned off are not.
pub fn load_with_profiles(root: &Path, profiles: &[String]) -> MavenModel {
load_with(root, default_local_repo(), profiles)
}
pub(crate) fn load_with_repo(root: &Path, local_repo: Option<PathBuf>) -> MavenModel {
load_with(root, local_repo, &[])
}
/// The `-P` entries' choice for profile `id`: on, off (`!id` / `-id`), or none (the last wins).
fn profile_choice(chosen: &[String], id: &str) -> Option<bool> {
chosen.iter().rev().find_map(|c| {
let c = c.trim();
if c == id {
Some(true)
} else if c.strip_prefix('!').or_else(|| c.strip_prefix('-')) == Some(id) {
Some(false)
} else {
None
}
})
}
pub(crate) fn load_with(root: &Path, local_repo: Option<PathBuf>, profiles: &[String]) -> MavenModel {
let mut resolver = Resolver::new(local_repo, profiles);
let mut resolver = Resolver::new(local_repo);
let mut model = MavenModel::default();
let mut seen: HashSet<PathBuf> = HashSet::new();
// (pom, aggregated)
@@ -1110,7 +1076,7 @@ pub(crate) fn load_with(root: &Path, local_repo: Option<PathBuf>, profiles: &[St
let i = |s: &str| interpolate(s, &props);
let mut module_names: Vec<String> = pom.modules.clone();
for p in resolver.active(&pom.profiles) {
for p in pom.profiles.iter().filter(|p| p.active_by_default) {
module_names.extend(p.modules.iter().cloned());
}
let mut modules: Vec<PathBuf> = Vec::new();
@@ -1733,52 +1699,6 @@ mod tests {
assert!(model.projects.iter().all(|p| p.command == if cfg!(windows) { "mvn.cmd" } else { "mvn" }));
}
#[test]
fn chosen_profiles_bring_their_modules_and_dependencies() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
write(
root,
"pom.xml",
r#"<project><groupId>demo</groupId><artifactId>parent</artifactId><version>1.0</version><packaging>pom</packaging>
<modules><module>core</module></modules>
<profiles>
<profile><id>flink-1.17</id><activation><activeByDefault>true</activeByDefault></activation>
<modules><module>flink-1.17</module></modules></profile>
<profile><id>flink-1.20</id><modules><module>flink-1.20</module></modules>
<dependencies><dependency><groupId>org.apache.flink</groupId><artifactId>flink-core</artifactId><version>1.20.0</version></dependency></dependencies>
</profile>
</profiles>
</project>"#,
);
for m in ["core", "flink-1.17", "flink-1.20"] {
write(
root,
&format!("{m}/pom.xml"),
&format!(
"<project><parent><groupId>demo</groupId><artifactId>parent</artifactId><version>1.0</version></parent><artifactId>{m}</artifactId></project>"
),
);
}
let dirs = |m: &MavenModel| m.projects.iter().map(|p| p.dir.clone()).collect::<Vec<_>>();
let flink = |m: &MavenModel, dir: &str| {
m.projects.iter().find(|p| p.dir == dir).unwrap().dependencies.iter().any(|d| d.artifact_id == "flink-core")
};
let default = load_with(root, None, &[]);
assert_eq!(dirs(&default), ["", "core", "flink-1.17"]);
assert!(!flink(&default, "core"));
// Choosing another profile of the POM turns its activeByDefault one off (as `-P` does).
let chosen = load_with(root, None, &["flink-1.20".into()]);
assert_eq!(dirs(&chosen), ["", "core", "flink-1.20"]);
assert!(flink(&chosen, "core"), "a module inherits the chosen profile's dependencies");
let both = load_with(root, None, &["flink-1.20".into(), "flink-1.17".into()]);
assert_eq!(dirs(&both), ["", "core", "flink-1.17", "flink-1.20"]);
assert_eq!(dirs(&load_with(root, None, &["!flink-1.17".into()])), ["", "core"]);
}
#[test]
fn profiles_of_a_parent_pom_are_inherited() {
let tmp = tempfile::tempdir().unwrap();
+1 -1
View File
@@ -283,7 +283,7 @@ pub fn sync_maven(
cancel: &CancellationToken,
progress: &dyn Fn(SyncProgress),
) -> IdeResult<()> {
let mut model = maven::load_with_profiles(root, &options.profiles);
let mut model = maven::load(root);
maven::UnlinkedProjects::from_model(&model, &options.maven_settings).filter(&mut model);
std::fs::create_dir_all(cache_dir)?;
let mut cache = MavenCache {
+6 -21
View File
@@ -123,14 +123,12 @@ fn taskkill_tree(child: &Child) {
/// inherits them loads the image's (older) libraries and plugins next to its own, e.g.
/// `curl: symbol lookup error ... nghttp2_...`. The IDE process keeps its environment
/// (WebKit and GTK still need it). Then applies the [`set_env_overlay`] variables and
/// those of a manual HTTP proxy ([`crate::proxy_settings::ProxySettings::child_env`]),
/// and the login shell's variables this process lacks ([`shell_env`]: an app started from
/// Finder/Dock doesn't have what rc files export, such as `GOPATH`, `JAVA_HOME` or
/// `MAVEN_OPTS`). Variables the call site has already set are kept.
/// those of a manual HTTP proxy ([`crate::proxy_settings::ProxySettings::child_env`]).
/// Variables the call site has already set are kept. A no-op outside an AppImage
/// without an overlay.
pub fn host_env(cmd: &mut Command) -> &mut Command {
let fixes = child_env_changes();
let shell: Vec<_> = shell_env().collect();
if fixes.is_empty() && shell.is_empty() {
if fixes.is_empty() {
return cmd;
}
let explicit: Vec<OsString> = cmd.get_envs().map(|(k, _)| k.to_owned()).collect();
@@ -140,9 +138,6 @@ pub fn host_env(cmd: &mut Command) -> &mut Command {
None => cmd.env_remove(key),
};
}
for (key, value) in shell.into_iter().filter(|(k, _)| !explicit.contains(k)) {
cmd.env(key, value);
}
cmd
}
@@ -159,13 +154,12 @@ pub fn host_vars() -> impl Iterator<Item = (OsString, OsString)> {
apply_changes(std::env::vars_os().collect(), &child_env_changes()).into_iter()
}
/// [`std::env::var_os`] as a child should see it (see [`host_env`]), the login shell's value
/// included when this process lacks the variable.
/// [`std::env::var_os`] as a child should see it (see [`host_env`]).
pub fn host_var_os(key: impl AsRef<OsStr>) -> Option<OsString> {
let key = key.as_ref();
match child_env_changes().into_iter().find(|(k, _)| k == key) {
Some((_, fixed)) => fixed,
None => std::env::var_os(key).or_else(|| shell_env().find(|(k, _)| k == key).map(|(_, v)| v)),
None => std::env::var_os(key),
}
}
@@ -578,15 +572,6 @@ mod tests {
assert_eq!(get("HOME"), Some(None));
let shell: Vec<_> = shell_env().map(|(k, _)| k).collect();
assert_eq!(shell, [OsString::from("OPENIDEA_OVERLAY_B")]);
// Every child gets what the login shell exports and the IDE process lacks (a Finder
// launch: `GOPATH`, `JAVA_HOME`...), unless the call site sets it.
assert_eq!(get("OPENIDEA_OVERLAY_B"), Some(Some("b".into())));
let mut own = Command::new("true");
own.env("OPENIDEA_OVERLAY_B", "call site");
host_env(&mut own);
let value = own.get_envs().find(|(n, _)| *n == "OPENIDEA_OVERLAY_B").and_then(|(_, v)| v);
assert_eq!(value, Some(OsStr::new("call site")));
assert_eq!(host_var_os("OPENIDEA_OVERLAY_B"), Some("b".into()));
assert_eq!(host_var_os("PATH"), Some("/reloaded".into()));
assert!(host_vars().all(|(k, _)| k != "HOME"));
assert!(host_vars().any(|(k, v)| k == "OPENIDEA_OVERLAY_A" && v == "project"));
-62
View File
@@ -157,16 +157,6 @@ pub struct LanguageServerSettings {
pub languages: Vec<String>,
/// Files marking a workspace root for a server defined here (`pyproject.toml`).
pub root_markers: Vec<String>,
/// Replaces the definition's `features`: what the server is used for
/// (`code-action`, `hover`, …); an empty list means everything.
#[serde(skip_serializing_if = "Option::is_none")]
#[specta(optional)]
pub features: Option<Vec<String>>,
/// Replaces the definition's `code-action-kinds` (`refactor`); an empty
/// list keeps every kind.
#[serde(skip_serializing_if = "Option::is_none")]
#[specta(optional)]
pub code_action_kinds: Option<Vec<String>>,
}
impl Default for LanguageServerSettings {
@@ -180,8 +170,6 @@ impl Default for LanguageServerSettings {
settings: None,
languages: Vec::new(),
root_markers: Vec::new(),
features: None,
code_action_kinds: None,
}
}
}
@@ -351,7 +339,6 @@ pub struct AppearanceSettings {
pub preferred_dark_theme: String,
/// A plugin's file icon theme (`<plugin-id>/<theme-id>`); empty = the built-in icons.
pub file_icon_theme: String,
pub file_icon_theme_options: BTreeMap<String, FileIconThemeOptions>,
/// IDE zoom in percent (50–300).
pub zoom: u32,
/// Apply `ui_font_family` / `ui_font_size` (otherwise the theme's).
@@ -387,24 +374,6 @@ pub struct AppearanceSettings {
/// Main toolbar entries per side, in order: title widget ids (`project`,
/// `vcs.branch`…) or `action:<id>`. Widgets in neither list nor `hidden`
/// (registered later) keep their default place.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct FileIconThemeOptions {
pub groups: BTreeMap<String, bool>,
pub choices: BTreeMap<String, String>,
pub hide_files: bool,
pub hide_folders: bool,
pub hollow_folders: bool,
pub hide_ui_icons: bool,
pub hide_psi_icons: bool,
pub monochrome: Option<String>,
pub saturation: Option<u8>,
pub opacity: Option<u8>,
pub icon_size: Option<u8>,
pub row_height: Option<u8>,
pub accent: Option<String>,
}
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct MainToolbarSettings {
@@ -425,7 +394,6 @@ impl Default for AppearanceSettings {
preferred_light_theme: "light".into(),
preferred_dark_theme: "dark".into(),
file_icon_theme: String::new(),
file_icon_theme_options: BTreeMap::new(),
zoom: 100,
use_custom_font: false,
ui_font_family: "Inter".into(),
@@ -951,36 +919,6 @@ mod tests {
assert_eq!(serde_json::from_value::<Settings>(value).unwrap(), settings);
}
#[test]
fn file_icon_theme_options_default_empty_and_round_trip() {
let old: Settings = serde_json::from_str(r#"{"appearance":{"fileIconTheme":"p/t"}}"#).unwrap();
assert!(old.appearance.file_icon_theme_options.is_empty());
let json = serde_json::json!({"appearance":{"fileIconThemeOptions":{"p/t":{
"groups":{"tests":true},"choices":{"arrows":"plusminus"},"hideFiles":true,"hideFolders":false,"hollowFolders":true,"hideUiIcons":true,"hidePsiIcons":false,
"monochrome":"#546e7a","saturation":5,"opacity":80,"iconSize":20,"rowHeight":30,"accent":"#009688"}}}});
let settings: Settings = serde_json::from_value(json.clone()).unwrap();
let options = &settings.appearance.file_icon_theme_options["p/t"];
assert!(options.hide_files && options.hollow_folders && !options.hide_folders);
assert!(options.hide_ui_icons && !options.hide_psi_icons);
assert!(options.groups["tests"]);
assert_eq!(options.choices["arrows"], "plusminus");
assert_eq!(
(options.saturation, options.opacity, options.icon_size, options.row_height),
(Some(5), Some(80), Some(20), Some(30))
);
assert_eq!(options.monochrome.as_deref(), Some("#546e7a"));
assert_eq!(options.accent.as_deref(), Some("#009688"));
let value = serde_json::to_value(&settings).unwrap();
let restored: Settings = serde_json::from_value(value).unwrap();
assert_eq!(restored, settings);
let plain = serde_json::to_value(FileIconThemeOptions::default()).unwrap();
assert_eq!(plain["hideFiles"], false);
assert_eq!(plain["hideUiIcons"], false);
assert_eq!(plain["hidePsiIcons"], false);
assert_eq!(plain["monochrome"], serde_json::Value::Null);
assert_eq!(plain["groups"], serde_json::json!({}));
}
#[test]
fn ui_font_fallback_defaults_empty_and_round_trips() {
let old: Settings = serde_json::from_str(r#"{"appearance":{"useCustomFont":true}}"#).unwrap();
-54
View File
@@ -212,19 +212,6 @@ pub struct Variable {
pub evaluate_name: Option<String>,
}
/// A completion the adapter offers for an expression (DAP `completions`).
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct DebugCompletion {
pub label: String,
/// What to insert, when it differs from `label`.
pub text: Option<String>,
/// DAP item type: `method`, `field`, `variable`, `class`, `keyword`, …
pub kind: Option<String>,
pub detail: Option<String>,
pub sort_text: Option<String>,
}
#[derive(Clone, Copy, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum StepKind {
@@ -1170,47 +1157,6 @@ impl DebugManager {
})
}
/// Completions for `text` at `line` / `column` (0-based, UTF-16) in the frame's scope; none
/// when the adapter doesn't offer them.
pub async fn completions(
&self,
s: DebugSessionId,
frame_id: Option<i64>,
text: String,
line: u32,
column: u32,
) -> IdeResult<Vec<DebugCompletion>> {
let session = self.session(s)?;
if !session.supports("supportsCompletionsRequest") {
return Ok(Vec::new());
}
let mut args = json!({ "text": text, "line": line + 1, "column": column + 1 });
if let Some(f) = frame_id {
args["frameId"] = f.into();
}
let body = session.conn.request_timeout("completions", args, REQUEST_TIMEOUT).await?;
let text =
|v: &Value, key: &str| v.get(key).and_then(Value::as_str).filter(|t| !t.is_empty()).map(str::to_owned);
Ok(body
.get("targets")
.and_then(Value::as_array)
.map(|targets| {
targets
.iter()
.filter_map(|t| {
Some(DebugCompletion {
label: text(t, "label")?,
text: text(t, "text"),
kind: text(t, "type"),
detail: text(t, "detail"),
sort_text: text(t, "sortText"),
})
})
.collect()
})
.unwrap_or_default())
}
pub async fn resume(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
let body = session.conn.request_timeout("continue", json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await?;
-17
View File
@@ -73,7 +73,6 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
msg,
json!({
"supportsConfigurationDoneRequest": true,
"supportsCompletionsRequest": true,
"supportsTerminateRequest": true,
"supportsSetVariable": true,
"supportsFunctionBreakpoints": true,
@@ -173,14 +172,6 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
]})
),
"evaluate" => ok!(msg, json!({"result": "42", "type": "int", "variablesReference": 0})),
"completions" => ok!(
msg,
json!({"targets": [
{"label": "getName()", "text": "getName", "type": "method", "detail": "String"},
{"label": "name", "type": "field"},
{"label": ""},
]})
),
"continue" => {
ok!(msg, json!({"allThreadsContinued": true}));
send!(json!({"type": "event", "event": "exited", "body": {"exitCode": 3}}));
@@ -377,14 +368,6 @@ async fn full_protocol_flow() {
let eval = h.request("evaluate");
assert_eq!(eval["arguments"]["context"], "watch");
assert_eq!(eval["arguments"]["frameId"], 10);
let items = h.manager.completions(id, Some(10), "req.na".into(), 0, 6).await.unwrap();
assert_eq!(
items.iter().map(|c| (c.label.as_str(), c.text.as_deref(), c.kind.as_deref())).collect::<Vec<_>>(),
[("getName()", Some("getName"), Some("method")), ("name", None, Some("field"))]
);
let asked = h.request("completions");
assert_eq!((asked["arguments"]["line"].clone(), asked["arguments"]["column"].clone()), (json!(1), json!(7)));
assert_eq!(asked["arguments"]["frameId"], 10);
// Live breakpoint updates reach the session.
let states = h.manager.set_breakpoints(path.clone(), vec![bp(0)]).await.unwrap();
+1 -5
View File
@@ -8,7 +8,7 @@ license.workspace = true
[dependencies]
ide-core.workspace = true
serde.workspace = true
serde_json = { workspace = true, features = ["raw_value"] }
serde_json.workspace = true
specta.workspace = true
toml.workspace = true
tokio.workspace = true
@@ -21,10 +21,6 @@ bb8.workspace = true
tokio-util = { workspace = true, features = ["compat"] }
dirs.workspace = true
ide-ssh.workspace = true
ide-http.workspace = true
reqwest = { workspace = true, features = ["query"] }
rustls.workspace = true
uuid.workspace = true
[dev-dependencies]
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
-844
View File
@@ -1,844 +0,0 @@
use crate::conn::{QueryResult, ResultColumn, Setup};
use crate::source::{DataSource, DbSslMode, DbTxMode};
use ide_core::{CancellationToken, IdeError, IdeResult, error::ErrorKind};
use parking_lot::RwLock;
use reqwest::{Client, RequestBuilder, StatusCode, Url};
use std::sync::Arc;
use std::time::{Duration, Instant};
use tokio::sync::Mutex;
#[derive(Clone)]
pub struct ClickhousePool(Arc<State>);
struct State {
client: RwLock<Option<Client>>,
endpoint: Url,
user: String,
password: String,
database: String,
settings: Vec<(String, String)>,
read_only: bool,
session_id: String,
session: Mutex<bool>,
closed: CancellationToken,
}
/// The response format: a names line, a types line, then one JSON array per row, so rows past
/// the grid's limit are skipped without being parsed.
const FORMAT: &str = "JSONCompactEachRowWithNamesAndTypes";
pub(crate) fn transaction_error() -> IdeError {
IdeError::invalid("ClickHouse HTTP does not support SQL transactions; use auto-commit and run writes directly")
}
impl ClickhousePool {
pub(crate) async fn connect(source: &DataSource, password: Option<&str>, setup: Setup) -> IdeResult<Self> {
if source.options.transaction == DbTxMode::Manual {
return Err(transaction_error());
}
for (name, _) in &setup.params {
if reserved_setting(name) {
return Err(IdeError::invalid(format!("ClickHouse HTTP manages the {name} setting")));
}
}
let secure = matches!(source.ssl, DbSslMode::Require | DbSslMode::VerifyCa | DbSslMode::VerifyFull);
let tls_files = [&source.ssl_ca_file, &source.ssl_cert_file, &source.ssl_key_file];
if !secure && tls_files.iter().any(|p| !p.trim().is_empty()) {
return Err(IdeError::invalid(
"ClickHouse TLS certificate files require an SSL mode of Require, Verify CA, or Verify Full",
));
}
let host = source.host.trim();
if host.is_empty() || host.contains(['/', '@', '?', '#']) {
return Err(IdeError::invalid("ClickHouse host must contain only a hostname or IP address"));
}
let host = if host.contains(':') && !host.starts_with('[') { format!("[{host}]") } else { host.to_owned() };
let scheme = if secure { "https" } else { "http" };
let port = source.port.unwrap_or(if secure { 8443 } else { 8123 });
let endpoint = Url::parse(&format!("{scheme}://{host}:{port}/"))
.map_err(|_| IdeError::invalid("invalid ClickHouse HTTP host or port"))?;
let timeout = setup.connect_timeout.unwrap_or(Duration::from_secs(30));
let _ = rustls::crypto::ring::default_provider().install_default();
let mut builder = Client::builder()
.connect_timeout(timeout)
.pool_max_idle_per_host(4)
.redirect(reqwest::redirect::Policy::none())
.user_agent(concat!("OpenIDEA/", env!("CARGO_PKG_VERSION")));
let ca = source.ssl_ca_file.trim();
if !ca.is_empty() {
let pem = read_pem(ca, "CA certificate")?;
let certificates = reqwest::Certificate::from_pem_bundle(&pem)
.map_err(|_| IdeError::invalid("invalid ClickHouse CA certificate PEM"))?;
if certificates.is_empty() {
return Err(IdeError::invalid("ClickHouse CA certificate PEM contains no certificates"));
}
builder = builder.tls_certs_only(certificates);
}
if source.ssl == DbSslMode::VerifyCa {
if ca.is_empty() {
return Err(IdeError::invalid("ClickHouse Verify CA requires a CA certificate file"));
}
builder = builder.tls_danger_accept_invalid_hostnames(true);
}
let cert = source.ssl_cert_file.trim();
let key = source.ssl_key_file.trim();
if cert.is_empty() != key.is_empty() {
return Err(IdeError::invalid("ClickHouse client TLS requires both certificate and key files"));
}
if !cert.is_empty() {
let mut pem = read_pem(cert, "client certificate")?;
pem.push(b'\n');
pem.extend(read_pem(key, "client key")?);
let identity = reqwest::Identity::from_pem(&pem)
.map_err(|_| IdeError::invalid("invalid ClickHouse client certificate or key PEM"))?;
builder = builder.identity(identity);
}
let proxy = ide_core::proxy_settings::proxy();
let client = ide_http::client::with_proxy(builder, &proxy).build().map_err(http_error)?;
let database = if source.options.default_schema.trim().is_empty() {
source.database.trim()
} else {
source.options.default_schema.trim()
};
let pool = Self(Arc::new(State {
client: RwLock::new(Some(client)),
endpoint,
user: if source.user.is_empty() { "default".into() } else { source.user.clone() },
password: password.unwrap_or_default().to_owned(),
database: if database.is_empty() { "default".into() } else { database.into() },
settings: setup.params,
read_only: source.read_only,
session_id: uuid::Uuid::new_v4().to_string(),
session: Mutex::new(false),
closed: CancellationToken::new(),
}));
let cancel = CancellationToken::new();
let initialize = async {
pool.execute("SELECT 1", 1, &cancel).await?;
for sql in setup.statements {
pool.execute(&sql, 0, &cancel).await?;
}
Ok(pool.clone())
};
tokio::pin!(initialize);
tokio::select! {
result = &mut initialize => {
if result.is_err() {
pool.close();
}
result
},
() = tokio::time::sleep(timeout) => {
cancel.cancel();
let _ = initialize.await;
pool.close();
Err(IdeError::new(ErrorKind::TimedOut, "timed out connecting to ClickHouse HTTP"))
}
}
}
pub fn close(&self) {
self.0.closed.cancel();
self.0.client.write().take();
}
pub fn is_closed(&self) -> bool {
self.0.closed.is_cancelled()
}
fn client(&self) -> IdeResult<Client> {
self.0.client.read().clone().ok_or_else(|| IdeError::new(ErrorKind::Io, "ClickHouse connection is closed"))
}
fn request(&self, client: &Client, query_id: &str, max_rows: usize, initialized: bool) -> RequestBuilder {
let mut request = client
.post(self.0.endpoint.clone())
.basic_auth(&self.0.user, Some(&self.0.password))
.header(reqwest::header::CONTENT_TYPE, "text/plain; charset=utf-8")
.query(&self.0.settings)
.query(&[
("session_id", self.0.session_id.clone()),
("session_timeout", "3600".into()),
("query_id", query_id.to_owned()),
("default_format", FORMAT.into()),
("output_format_json_quote_denormals", "1".into()),
("max_result_rows", max_rows.saturating_add(1).to_string()),
("result_overflow_mode", "break".into()),
]);
if initialized {
request = request.query(&[("session_check", "1")]);
} else {
request = request.query(&[("database", &self.0.database)]);
}
if self.0.read_only {
request = request.query(&[("readonly", "1")]);
}
request
}
pub async fn execute(&self, sql: &str, max_rows: usize, cancel: &CancellationToken) -> IdeResult<QueryResult> {
if transaction_statement(sql) {
return Err(transaction_error());
}
let mut session = tokio::select! {
biased;
() = cancel.cancelled() => return Err(IdeError::cancelled()),
() = self.0.closed.cancelled() => return Err(IdeError::new(ErrorKind::Io, "ClickHouse connection is closed")),
session = self.0.session.lock() => session,
};
let client = self.client()?;
let query_id = uuid::Uuid::new_v4().to_string();
let request = self.request(&client, &query_id, max_rows, *session).body(sql.to_owned());
let started = Instant::now();
let fetch = async {
let mut response = request.send().await.map_err(http_error)?;
let execution_ms = started.elapsed().as_secs_f64() * 1000.0;
let status = response.status();
let header = |name: &str| response.headers().get(name).and_then(|v| v.to_str().ok()).map(str::to_owned);
let exception_code = header("X-ClickHouse-Exception-Code");
let format = header("X-ClickHouse-Format");
let affected = header("X-ClickHouse-Summary")
.and_then(|v| serde_json::from_str::<serde_json::Value>(&v).ok())
.and_then(|v| {
v.get("written_rows")
.and_then(|n| n.as_str().and_then(|n| n.parse::<u64>().ok()).or_else(|| n.as_u64()))
});
if !status.is_success() || exception_code.as_deref().is_some_and(|code| code != "0") {
let body = response.bytes().await.map_err(http_error)?;
return Err(self.server_error(status, exception_code.as_deref(), &body));
}
let (columns, rows, has_more) = if format.as_deref().is_some_and(|f| f != FORMAT) {
let body = response.bytes().await.map_err(http_error)?;
if !body.iter().all(u8::is_ascii_whitespace) {
return Err(IdeError::invalid(
"ClickHouse result grids need the default output format; remove the query's FORMAT clause",
));
}
(Vec::new(), Vec::new(), false)
} else {
let mut reader = RowReader::new(max_rows);
while let Some(chunk) = response.chunk().await.map_err(http_error)? {
reader.push(&chunk)?;
}
reader.finish().map_err(|error| self.redact_error(error))?
};
let affected = if columns.is_empty() { affected.map(|n| n as f64) } else { None };
Ok(QueryResult {
statement: sql.to_owned(),
columns,
rows,
affected,
has_more,
elapsed_ms: started.elapsed().as_secs_f64() * 1000.0,
execution_ms,
})
};
let result = tokio::select! {
biased;
() = cancel.cancelled() => None,
() = self.0.closed.cancelled() => None,
result = fetch => Some(result),
};
match result {
Some(result) => {
if result.is_ok() {
*session = true;
}
result
}
None => {
let killed = self.kill(&client, &query_id).await;
match killed {
Ok(()) => Err(IdeError::cancelled()),
Err(error) => Err(IdeError::new(
ErrorKind::Cancelled,
format!("cancelled locally; ClickHouse server cancellation failed: {}", error.message),
)),
}
}
}
}
async fn kill(&self, client: &Client, query_id: &str) -> IdeResult<()> {
let response = client
.post(self.0.endpoint.clone())
.basic_auth(&self.0.user, Some(&self.0.password))
.timeout(Duration::from_secs(3))
.body(format!("KILL QUERY WHERE query_id = '{query_id}' SYNC"))
.send()
.await
.map_err(http_error)?;
let status = response.status();
let code = response
.headers()
.get("X-ClickHouse-Exception-Code")
.and_then(|value| value.to_str().ok())
.map(str::to_owned);
let body = response.bytes().await.map_err(http_error)?;
if status.is_success() && !code.as_deref().is_some_and(|code| code != "0") && !body.starts_with(b"Code:") {
Ok(())
} else {
Err(self.server_error(status, code.as_deref(), &body))
}
}
fn redact_error(&self, mut error: IdeError) -> IdeError {
if !self.0.password.is_empty() {
error.message = error.message.replace(&self.0.password, "[redacted]");
}
error
}
fn server_error(&self, status: StatusCode, code: Option<&str>, body: &[u8]) -> IdeError {
let message = String::from_utf8_lossy(body);
let message: String = exception_text(&message).chars().take(16_384).collect();
let auth = matches!(status, StatusCode::UNAUTHORIZED | StatusCode::FORBIDDEN)
|| matches!(code, Some("193" | "194" | "195" | "516" | "497"));
let detail = if message.is_empty() { status.canonical_reason().unwrap_or("request failed") } else { &message };
let detail = if detail.contains("SESSION_NOT_FOUND") || detail.contains("Session not found") {
"ClickHouse HTTP session expired; reconnect to restore session settings"
} else {
detail
};
self.redact_error(IdeError::new(
if auth { ErrorKind::PermissionDenied } else { ErrorKind::InvalidInput },
format!("ClickHouse HTTP {}: {detail}", status.as_u16()),
))
}
}
type DecodedResult = (Vec<ResultColumn>, Vec<Vec<Option<String>>>, bool);
/// Decodes a streamed [`FORMAT`] body. Keeps the first `max_rows` rows; later rows only set
/// `has_more` and are read past unparsed, so a grid limit bounds memory however large the
/// block `result_overflow_mode=break` lets the server send. A line that is not a JSON array
/// starts the exception ClickHouse writes when a query fails mid-stream.
struct RowReader {
max_rows: usize,
pending: Vec<u8>,
names: Option<Vec<String>>,
types: Option<Vec<String>>,
rows: Vec<Vec<Option<String>>>,
more: bool,
exception: Option<String>,
}
impl RowReader {
fn new(max_rows: usize) -> Self {
Self { max_rows, pending: Vec::new(), names: None, types: None, rows: Vec::new(), more: false, exception: None }
}
fn push(&mut self, chunk: &[u8]) -> IdeResult<()> {
self.pending.extend_from_slice(chunk);
let buffer = std::mem::take(&mut self.pending);
let mut start = 0;
while let Some(end) = buffer[start..].iter().position(|&b| b == b'\n') {
self.line(&buffer[start..start + end])?;
start += end + 1;
}
self.pending = buffer[start..].to_vec();
Ok(())
}
fn line(&mut self, line: &[u8]) -> IdeResult<()> {
if let Some(exception) = &mut self.exception {
if exception.len() < 16_384 {
exception.push_str(&String::from_utf8_lossy(line));
exception.push('\n');
}
return Ok(());
}
let text = line.trim_ascii();
if text.is_empty() {
return Ok(());
}
if text[0] != b'[' {
self.exception = Some(format!("{}\n", String::from_utf8_lossy(line)));
return Ok(());
}
if self.names.is_some() && self.types.is_some() && self.rows.len() >= self.max_rows {
self.more = true;
return Ok(());
}
let Ok(values) = serde_json::from_slice::<Vec<Box<serde_json::value::RawValue>>>(text) else {
self.exception = Some(format!("{}\n", String::from_utf8_lossy(line)));
return Ok(());
};
let protocol = |message: String| IdeError::new(ErrorKind::Protocol, message);
let header = |values: Vec<Box<serde_json::value::RawValue>>| {
values
.iter()
.map(|v| serde_json::from_str::<String>(v.get()))
.collect::<Result<Vec<_>, _>>()
.map_err(|e| protocol(format!("invalid ClickHouse result header: {e}")))
};
let Some(names) = &self.names else {
self.names = Some(header(values)?);
return Ok(());
};
if self.types.is_none() {
self.types = Some(header(values)?);
return Ok(());
}
if values.len() != names.len() {
return Err(protocol("ClickHouse result row width does not match its metadata".into()));
}
let row = values
.into_iter()
.map(|value| {
let text = value.get();
if text == "null" {
Ok(None)
} else if text.starts_with('"') {
serde_json::from_str::<String>(text).map(Some).map_err(|e| protocol(e.to_string()))
} else {
Ok(Some(text.to_owned()))
}
})
.collect::<IdeResult<_>>()?;
self.rows.push(row);
Ok(())
}
fn finish(mut self) -> IdeResult<DecodedResult> {
let rest = std::mem::take(&mut self.pending);
self.line(&rest)?;
if let Some(exception) = self.exception {
return Err(IdeError::invalid(format!("ClickHouse: {}", exception_text(&exception))));
}
let Some(names) = self.names else { return Ok((Vec::new(), Vec::new(), false)) };
let types = self.types.unwrap_or_default();
if types.len() != names.len() {
return Err(IdeError::new(ErrorKind::Protocol, "ClickHouse result types do not match its columns"));
}
let columns = names.into_iter().zip(types).map(|(name, type_name)| ResultColumn { name, type_name }).collect();
Ok((columns, self.rows, self.more))
}
}
/// The server's message in a failed response: rows already streamed and the
/// `__exception__` block markers ClickHouse wraps a mid-stream error in are dropped.
fn exception_text(body: &str) -> &str {
let Some(at) = body.find("Code:") else { return body.trim() };
let text = &body[at..];
match text.find("\n__exception__") {
// The line before the closing marker is `<length> <tag>`.
Some(end) => text[..end].trim_end().rsplit_once('\n').map_or(text[..end].trim(), |(message, _)| message.trim()),
None => text.trim(),
}
}
fn http_error(error: reqwest::Error) -> IdeError {
use std::error::Error;
let kind = if error.is_timeout() { ErrorKind::TimedOut } else { ErrorKind::Io };
let error = error.without_url();
let mut message = error.to_string();
let mut source = error.source();
while let Some(error) = source {
let detail = error.to_string();
if !message.contains(&detail) {
message.push_str(": ");
message.push_str(&detail);
}
source = error.source();
}
IdeError::new(kind, format!("ClickHouse HTTP: {message}"))
}
fn read_pem(path: &str, label: &str) -> IdeResult<Vec<u8>> {
std::fs::read(ide_ssh::expand(path))
.map_err(|error| IdeError::new(ErrorKind::Io, format!("cannot read ClickHouse {label}: {error}")))
}
fn reserved_setting(name: &str) -> bool {
let name = name.to_ascii_lowercase();
name.starts_with("output_format_json_")
|| matches!(
name.as_str(),
"user"
| "password"
| "query"
| "database"
| "session_id"
| "session_check"
| "session_timeout"
| "query_id"
| "default_format"
| "max_result_rows"
| "result_overflow_mode"
| "readonly"
)
}
fn transaction_statement(sql: &str) -> bool {
let mut sql = sql.trim_start();
loop {
if let Some(rest) = sql.strip_prefix("--").or_else(|| sql.strip_prefix('#')) {
sql = rest.split_once('\n').map_or("", |(_, rest)| rest).trim_start();
} else if let Some(rest) = sql.strip_prefix("/*") {
sql = rest.split_once("*/").map_or("", |(_, rest)| rest).trim_start();
} else {
break;
}
}
let mut words = sql.split(|c: char| !c.is_ascii_alphanumeric() && c != '_').filter(|s| !s.is_empty());
let first = words.next().unwrap_or_default();
["BEGIN", "COMMIT", "ROLLBACK", "SAVEPOINT", "RELEASE"].iter().any(|s| first.eq_ignore_ascii_case(s))
|| (first.eq_ignore_ascii_case("START") && words.next().is_some_and(|s| s.eq_ignore_ascii_case("TRANSACTION")))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::conn::Pool;
use crate::source::{DbDriver, DbOptions, DbProperty};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
const ONE: &str = "[\"1\"]\n[\"UInt8\"]\n[1]\n";
async fn read_request(stream: &mut tokio::net::TcpStream) -> String {
let mut bytes = Vec::new();
let mut buffer = [0; 4096];
loop {
let count = tokio::time::timeout(Duration::from_secs(5), stream.read(&mut buffer)).await.unwrap().unwrap();
assert!(count > 0, "request ended early");
bytes.extend_from_slice(&buffer[..count]);
if let Some(end) = bytes.windows(4).position(|s| s == b"\r\n\r\n") {
let header = String::from_utf8_lossy(&bytes[..end]);
let length: usize = header
.lines()
.find_map(|line| {
line.to_ascii_lowercase().strip_prefix("content-length:").map(|v| v.trim().parse().unwrap())
})
.unwrap_or(0);
if bytes.len() >= end + 4 + length {
return String::from_utf8(bytes).unwrap();
}
}
}
}
async fn reply(stream: &mut tokio::net::TcpStream, status: &str, headers: &str, body: &str) {
let response =
format!("HTTP/1.1 {status}\r\nContent-Length: {}\r\nConnection: close\r\n{headers}\r\n{body}", body.len());
stream.write_all(response.as_bytes()).await.unwrap();
}
async fn server(
responses: Vec<(&'static str, &'static str, &'static str)>,
) -> (u16, tokio::task::JoinHandle<Vec<String>>) {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let port = listener.local_addr().unwrap().port();
let task = tokio::spawn(async move {
let mut requests = Vec::new();
for (status, headers, body) in responses {
let (mut stream, _) =
tokio::time::timeout(Duration::from_secs(5), listener.accept()).await.unwrap().unwrap();
requests.push(read_request(&mut stream).await);
reply(&mut stream, status, headers, body).await;
}
requests
});
(port, task)
}
fn source(port: u16) -> DataSource {
DataSource { driver: DbDriver::Clickhouse, host: "127.0.0.1".into(), port: Some(port), ..Default::default() }
}
fn decode(body: &[u8], max_rows: usize, chunk: usize) -> IdeResult<DecodedResult> {
let mut reader = RowReader::new(max_rows);
for part in body.chunks(chunk) {
reader.push(part)?;
}
reader.finish()
}
#[test]
fn rows_preserve_precision_nulls_and_strings_across_chunks() {
let body = concat!(
r#"["n", "d", "v", "v", "v"]"#,
"\n",
r#"["UInt64", "Decimal(38, 9)", "Nullable(String)", "Nullable(String)", "Nullable(String)"]"#,
"\n",
r#"[18446744073709551615, 12345678901234567890.123456789, null, "null", "ᴺᵁᴸᴸ"]"#,
);
for chunk in [1, 7, body.len()] {
let (columns, rows, more) = decode(body.as_bytes(), 1, chunk).unwrap();
assert_eq!(columns.len(), 5);
assert_eq!(columns[1].type_name, "Decimal(38, 9)");
assert_eq!(
rows[0],
vec![
Some("18446744073709551615".into()),
Some("12345678901234567890.123456789".into()),
None,
Some("null".into()),
Some("ᴺᵁᴸᴸ".into()),
]
);
assert!(!more);
}
let (columns, rows, more) = decode(b"[\"x\"]\n[\"String\"]\n", 0, 64).unwrap();
assert_eq!(columns[0].name, "x");
assert!(rows.is_empty() && !more);
assert_eq!(decode(b"", 5, 64).unwrap().0.len(), 0);
assert!(decode(b"[\"x\"]\n[\"UInt8\"]\n[1, 2]\n", 1, 64).is_err());
}
#[test]
fn rows_past_the_limit_are_skipped_but_a_later_exception_still_fails() {
let mut body = String::from("[\"n\"]\n[\"UInt64\"]\n");
for n in 0..1000 {
body.push_str(&format!("[{n}]\n"));
}
let (_, rows, more) = decode(body.as_bytes(), 2, 4096).unwrap();
assert_eq!(rows, vec![vec![Some("0".into())], vec![Some("1".into())]]);
assert!(more);
body.push_str("\n__exception__\nabcdefgh\nCode: 395. DB::Exception: Value passed to 'throwIf' is non-zero\n");
body.push_str("62 abcdefgh\n__exception__\n");
let error = decode(body.as_bytes(), 2, 4096).unwrap_err();
assert_eq!(error.message, "ClickHouse: Code: 395. DB::Exception: Value passed to 'throwIf' is non-zero");
let multiline = "Code: 62. DB::Exception: Syntax error\nat line 2";
assert_eq!(exception_text(multiline), multiline);
}
#[tokio::test]
async fn http_requests_use_auth_settings_limits_and_a_shared_session() {
let (port, requests) = server(vec![
("200 OK", "X-ClickHouse-Format: JSONCompactEachRowWithNamesAndTypes\r\n", ONE),
("200 OK", "", ""),
(
"200 OK",
"X-ClickHouse-Format: JSONCompactEachRowWithNamesAndTypes\r\n",
"[\"n\"]\n[\"UInt64\"]\n[\"1\"]\n[\"2\"]\n[\"3\"]\n",
),
])
.await;
let source = DataSource {
user: "reader".into(),
read_only: true,
options: DbOptions { default_schema: "analytics".into(), time_zone: "UTC".into(), ..Default::default() },
properties: vec![DbProperty { name: "max_threads".into(), value: "2".into() }],
..source(port)
};
let pool = Pool::connect(&source, Some("mock-password"), None).await.unwrap();
pool.execute("USE other", 0, &CancellationToken::new()).await.unwrap();
let result = pool.execute("SELECT number FROM numbers(3)", 1, &CancellationToken::new()).await.unwrap();
assert_eq!(result.rows, vec![vec![Some("1".into())]]);
assert!(result.has_more && result.affected.is_none());
let requests = requests.await.unwrap();
let target = |request: &String| {
Url::parse(&format!(
"http://localhost{}",
request.lines().next().unwrap().split_whitespace().nth(1).unwrap()
))
.unwrap()
};
let urls: Vec<_> = requests.iter().map(target).collect();
let value = |url: &Url, key: &str| url.query_pairs().find(|(k, _)| k == key).map(|(_, v)| v.into_owned());
assert_eq!(value(&urls[0], "database").as_deref(), Some("analytics"));
assert_eq!(value(&urls[1], "database"), None);
assert_eq!(value(&urls[1], "session_check").as_deref(), Some("1"));
assert_eq!(value(&urls[0], "session_id"), value(&urls[2], "session_id"));
assert_ne!(value(&urls[0], "query_id"), value(&urls[2], "query_id"));
assert_eq!(value(&urls[2], "max_result_rows").as_deref(), Some("2"));
for (request, url) in requests.iter().zip(&urls) {
assert!(request.starts_with("POST /?"));
assert!(request.contains("Basic cmVhZGVyOm1vY2stcGFzc3dvcmQ="));
assert!(!url.as_str().contains("reader") && !url.as_str().contains("mock-password"));
assert_eq!(value(url, "readonly").as_deref(), Some("1"));
assert_eq!(value(url, "session_timezone").as_deref(), Some("UTC"));
assert_eq!(value(url, "max_threads").as_deref(), Some("2"));
}
assert!(requests[0].ends_with("SELECT 1"));
assert!(pool.transaction(&[]).await.is_err());
let cancel = CancellationToken::new();
cancel.cancel();
assert_eq!(pool.execute("SELECT 1", 1, &cancel).await.unwrap_err().kind, ErrorKind::Cancelled);
pool.close().await;
assert!(pool.is_closed());
assert!(pool.execute("SELECT 1", 1, &CancellationToken::new()).await.is_err());
}
#[tokio::test]
async fn sequential_queries_reuse_the_http_connection() {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let port = listener.local_addr().unwrap().port();
let task = tokio::spawn(async move {
let (mut stream, _) = listener.accept().await.unwrap();
for _ in 0..2 {
read_request(&mut stream).await;
let response = format!("HTTP/1.1 200 OK\r\nContent-Length: {}\r\n\r\n{ONE}", ONE.len());
stream.write_all(response.as_bytes()).await.unwrap();
}
});
let pool = Pool::connect(&source(port), None, None).await.unwrap();
assert_eq!(
pool.execute("SELECT 1", 1, &CancellationToken::new()).await.unwrap().rows,
vec![vec![Some("1".into())]]
);
task.await.unwrap();
}
#[tokio::test]
async fn server_errors_keep_details_and_hide_passwords() {
let (port, requests) = server(vec![(
"403 Forbidden",
"X-ClickHouse-Exception-Code: 516\r\n",
"Code: 516. DB::Exception: bad mock-password",
)])
.await;
let error = match Pool::connect(&source(port), Some("mock-password"), None).await {
Ok(_) => panic!("expected login failure"),
Err(error) => error,
};
assert_eq!(error.kind, ErrorKind::PermissionDenied);
assert!(error.message.contains("516") && error.message.contains("[redacted]"));
assert!(!error.message.contains("mock-password"));
requests.await.unwrap();
let (port, requests) = server(vec![("200 OK", "", "Code: 241. DB::Exception: memory limit")]).await;
let error = match Pool::connect(&source(port), None, None).await {
Ok(_) => panic!("expected late query failure"),
Err(error) => error,
};
assert!(error.message.contains("memory limit"));
requests.await.unwrap();
let (port, requests) = server(vec![(
"500 Internal Server Error",
"X-ClickHouse-Exception-Code: 395\r\n",
"[\"1\"]\n[\"UInt8\"]\n[1]\n\n__exception__\ntag\nCode: 395. DB::Exception: thrown\n32 tag\n__exception__\n",
)])
.await;
let error = match Pool::connect(&source(port), None, None).await {
Ok(_) => panic!("expected mid-stream failure"),
Err(error) => error,
};
assert_eq!(error.message, "ClickHouse HTTP 500: Code: 395. DB::Exception: thrown");
requests.await.unwrap();
}
#[tokio::test]
async fn cancellation_sends_kill_outside_the_busy_session() {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let port = listener.local_addr().unwrap().port();
let (started, received) = tokio::sync::oneshot::channel();
let server = tokio::spawn(async move {
let (mut health, _) = listener.accept().await.unwrap();
read_request(&mut health).await;
reply(&mut health, "200 OK", "", ONE).await;
drop(health);
let (mut query, _) = listener.accept().await.unwrap();
let request = read_request(&mut query).await;
started.send(()).unwrap();
let (mut kill, _) = tokio::time::timeout(Duration::from_secs(5), listener.accept()).await.unwrap().unwrap();
let kill_request = read_request(&mut kill).await;
reply(&mut kill, "200 OK", "", "").await;
(request, kill_request)
});
let pool = Pool::connect(&source(port), None, None).await.unwrap();
let cancel = CancellationToken::new();
let query = pool.execute("SELECT sleep(30)", 1, &cancel);
let cancelled = async {
received.await.unwrap();
cancel.cancel();
};
let (result, ()) = tokio::join!(query, cancelled);
assert_eq!(result.unwrap_err().kind, ErrorKind::Cancelled);
let (query, kill) = server.await.unwrap();
let target = query.lines().next().unwrap().split_whitespace().nth(1).unwrap();
let url = Url::parse(&format!("http://localhost{target}")).unwrap();
let query_id = url.query_pairs().find(|(k, _)| k == "query_id").unwrap().1.into_owned();
assert!(kill.ends_with(&format!("KILL QUERY WHERE query_id = '{query_id}' SYNC")));
assert!(!kill.contains("session_id="));
}
#[tokio::test]
async fn unsupported_settings_and_transactions_fail_before_connecting() {
for statement in ["BEGIN", "-- comment\n COMMIT", "/* comment */ ROLLBACK", "START TRANSACTION", "SAVEPOINT s"]
{
assert!(transaction_statement(statement));
}
assert!(!transaction_statement("SELECT 'BEGIN'"));
let sources = [
DataSource { options: DbOptions { transaction: DbTxMode::Manual, ..Default::default() }, ..source(1) },
DataSource {
properties: vec![DbProperty { name: "statement_cache_capacity".into(), value: "4".into() }],
..source(1)
},
DataSource { properties: vec![DbProperty { name: "password".into(), value: "x".into() }], ..source(1) },
DataSource { ssl_ca_file: "unused.pem".into(), ..source(1) },
DataSource { ssl: DbSslMode::VerifyCa, ..source(1) },
DataSource { ssl: DbSslMode::Require, ssl_cert_file: "unused.pem".into(), ..source(1) },
];
for source in sources {
let error = match Pool::connect(&source, None, None).await {
Ok(_) => panic!("unsupported settings were accepted"),
Err(error) => error,
};
assert_eq!(error.kind, ErrorKind::InvalidInput);
}
}
#[tokio::test]
#[ignore = "requires CLICKHOUSE_TEST_HOST, CLICKHOUSE_TEST_USER, and CLICKHOUSE_TEST_PASSWORD"]
async fn clickhouse_cloud_transport() {
let source = DataSource {
driver: DbDriver::Clickhouse,
host: std::env::var("CLICKHOUSE_TEST_HOST").expect("CLICKHOUSE_TEST_HOST"),
port: Some(std::env::var("CLICKHOUSE_TEST_PORT").ok().and_then(|v| v.parse().ok()).unwrap_or(8443)),
user: std::env::var("CLICKHOUSE_TEST_USER").expect("CLICKHOUSE_TEST_USER"),
ssl: DbSslMode::Require,
options: DbOptions {
time_zone: "UTC".into(),
startup_script: "SET max_threads = 2".into(),
..Default::default()
},
properties: vec![DbProperty { name: "max_block_size".into(), value: "50".into() }],
..Default::default()
};
let password = std::env::var("CLICKHOUSE_TEST_PASSWORD").expect("CLICKHOUSE_TEST_PASSWORD");
let pool = Pool::connect(&source, Some(&password), None).await.unwrap();
let cancel = CancellationToken::new();
let settings = pool
.execute("SELECT getSetting('max_threads'), getSetting('max_block_size'), timezone()", 1, &cancel)
.await
.unwrap();
assert_eq!(settings.rows, vec![vec![Some("2".into()), Some("50".into()), Some("UTC".into())]]);
let result = pool.execute("SELECT toUInt64('18446744073709551615') AS n, CAST(NULL AS Nullable(String)) AS missing, CAST('ᴺᵁᴸᴸ' AS Nullable(String)) AS text, 'a\\nb' AS lines", 1, &cancel).await.unwrap();
assert_eq!(
result.rows,
vec![vec![Some("18446744073709551615".into()), None, Some("ᴺᵁᴸᴸ".into()), Some("a\nb".into())]]
);
assert_eq!(result.columns[1].type_name, "Nullable(String)");
let limited = pool.execute("SELECT number FROM numbers(100)", 2, &cancel).await.unwrap();
assert_eq!(limited.rows.len(), 2);
assert!(limited.has_more);
let marker = format!("transport_cancel_{}", uuid::Uuid::new_v4().simple());
let sql = format!("SELECT sleepEachRow(0.05) FROM numbers(50) /* {marker} */");
let stop = CancellationToken::new();
let query = pool.execute(&sql, 1, &stop);
let cancel_query = async {
tokio::time::sleep(Duration::from_millis(500)).await;
stop.cancel();
};
let (cancelled, ()) = tokio::join!(query, cancel_query);
let cancelled = cancelled.unwrap_err();
assert_eq!(cancelled.kind, ErrorKind::Cancelled);
assert_eq!(cancelled.message, "cancelled");
let active = pool.execute(&format!("SELECT count() FROM system.processes WHERE position(query, '{marker}') > 0 AND query_id != currentQueryID()"), 1, &cancel).await.unwrap();
assert_eq!(active.rows, vec![vec![Some("0".into())]]);
let table = format!("transport_test_{}", uuid::Uuid::new_v4().simple());
pool.execute(&format!("CREATE TEMPORARY TABLE {table} (n UInt8)"), 0, &cancel).await.unwrap();
let continuity = async {
pool.execute(&format!("INSERT INTO {table} VALUES (7)"), 0, &cancel).await?;
pool.execute(&format!("SELECT n FROM {table}"), 1, &cancel).await
}
.await;
let cleanup = pool.execute(&format!("DROP TEMPORARY TABLE {table}"), 0, &cancel).await;
assert_eq!(continuity.unwrap().rows, vec![vec![Some("7".into())]]);
cleanup.unwrap();
pool.close().await;
assert!(pool.is_closed());
}
}
+3 -37
View File
@@ -106,7 +106,6 @@ pub enum Pool {
Postgres(PgPool),
Mysql(MySqlPool),
SqlServer(Bb8Pool<MssqlConnectionManager>),
Clickhouse(crate::clickhouse::ClickhousePool),
}
/// Runs `$body` with `$p` bound to the concrete pool of each driver.
@@ -116,7 +115,7 @@ macro_rules! with_pool {
Pool::Sqlite($p) => $body,
Pool::Postgres($p) => $body,
Pool::Mysql($p) => $body,
Pool::SqlServer(_) | Pool::Clickhouse(_) => unreachable!(),
Pool::SqlServer(_) => unreachable!(),
}
};
}
@@ -504,9 +503,6 @@ impl Setup {
value.parse::<u64>().map_err(|_| IdeError::invalid(format!("{what} must be a whole number: {value}")))
};
match (source.driver, name.to_ascii_lowercase().as_str()) {
(DbDriver::Clickhouse, "statement_cache_capacity") => {
return Err(IdeError::invalid("ClickHouse HTTP does not support statement caching"));
}
(_, "statement_cache_capacity") => s.statement_cache = Some(number(&name)? as usize),
(_, "connect_timeout") => s.connect_timeout = Some(Duration::from_secs(number(&name)?)),
(DbDriver::Postgres, "application_name") => s.application_name = Some(value.to_owned()),
@@ -516,9 +512,6 @@ impl Setup {
_ => s.params.push((name, value.to_owned())),
}
}
if source.driver == DbDriver::Clickhouse && !o.time_zone.trim().is_empty() {
s.params.push(("session_timezone".into(), o.time_zone.trim().into()));
}
let script = o.startup_script.trim();
if !script.is_empty() {
s.statements.extend(crate::split::split(script, source.driver).into_iter().map(|st| st.text));
@@ -534,7 +527,6 @@ impl Pool {
Pool::Postgres(_) => DbDriver::Postgres,
Pool::Mysql(_) => DbDriver::Mysql,
Pool::SqlServer(_) => DbDriver::SqlServer,
Pool::Clickhouse(_) => DbDriver::Clickhouse,
}
}
@@ -542,9 +534,6 @@ impl Pool {
pub async fn connect(source: &DataSource, password: Option<&str>, root: Option<&Path>) -> IdeResult<Self> {
const MAX: u32 = 4;
let setup = Setup::of(source)?;
if source.driver == DbDriver::Clickhouse {
return crate::clickhouse::ClickhousePool::connect(source, password, setup).await.map(Pool::Clickhouse);
}
let timeout = setup.connect_timeout.unwrap_or(Duration::from_secs(30));
let init = Arc::new(setup.statements);
// Runs the session statements (startup script…) on every new connection.
@@ -706,14 +695,12 @@ impl Pool {
conn.query("SELECT 1", &[]).await.map_err(mssql_err)?.into_results().await.map_err(mssql_err)?;
Ok(Pool::SqlServer(pool))
}
DbDriver::Clickhouse => unreachable!(),
}
}
pub async fn close(&self) {
match self {
Pool::SqlServer(_) => {}
Pool::Clickhouse(p) => p.close(),
_ => with_pool!(self, p => p.close().await),
}
}
@@ -721,16 +708,12 @@ impl Pool {
pub fn is_closed(&self) -> bool {
match self {
Pool::SqlServer(_) => false,
Pool::Clickhouse(p) => p.is_closed(),
_ => with_pool!(self, p => p.is_closed()),
}
}
/// Runs `sql` (no row limit beyond `max`), for introspection: cells are text (see [`Cells::Text`]).
pub async fn rows(&self, sql: &str, max: usize) -> IdeResult<Vec<Vec<Option<String>>>> {
if let Pool::Clickhouse(p) = self {
return Ok(p.execute(sql, max, &CancellationToken::new()).await?.rows);
}
let fetched: IdeResult<Fetched> = match self {
Pool::SqlServer(p) => {
let mut conn = p.get_owned().await.map_err(mssql_pool_err)?;
@@ -744,9 +727,6 @@ impl Pool {
/// Runs one statement and keeps at most `max_rows` rows. Cancelling stops
/// the statement on the server (PostgreSQL, MySQL) and drops the connection.
pub async fn execute(&self, sql: &str, max_rows: usize, cancel: &CancellationToken) -> IdeResult<QueryResult> {
if let Pool::Clickhouse(p) = self {
return p.execute(sql, max_rows, cancel).await;
}
let mut conn = Held::acquire(self).await?;
let r = conn.execute(self, sql, max_rows, cancel).await;
if cancel.is_cancelled() {
@@ -760,7 +740,6 @@ impl Pool {
/// row when flagged (table editor submits).
pub async fn transaction(&self, statements: &[(String, bool)]) -> IdeResult<u64> {
match self {
Pool::Clickhouse(_) => Err(crate::clickhouse::transaction_error()),
Pool::SqlServer(p) => {
let mut conn = p.get_owned().await.map_err(mssql_pool_err)?;
let result = async {
@@ -811,7 +790,6 @@ pub enum Held {
Postgres(PoolConnection<Postgres>),
Mysql(PoolConnection<MySql>),
SqlServer(Box<bb8::PooledConnection<'static, MssqlConnectionManager>>),
Clickhouse(crate::clickhouse::ClickhousePool),
}
impl Held {
@@ -821,12 +799,6 @@ impl Held {
Pool::Postgres(p) => Held::Postgres(p.acquire().await.map_err(db_err)?),
Pool::Mysql(p) => Held::Mysql(p.acquire().await.map_err(db_err)?),
Pool::SqlServer(p) => Held::SqlServer(Box::new(p.get_owned().await.map_err(mssql_pool_err)?)),
Pool::Clickhouse(p) => {
if p.is_closed() {
return Err(IdeError::new(ErrorKind::Io, "ClickHouse connection is closed"));
}
Held::Clickhouse(p.clone())
}
})
}
@@ -834,14 +806,13 @@ impl Held {
pub async fn command(&mut self, sql: &str) -> IdeResult<()> {
match self {
Held::SqlServer(c) => mssql_command(c, sql).await,
Held::Clickhouse(_) => Err(crate::clickhouse::transaction_error()),
other => {
let sql = sqlx::AssertSqlSafe(sql.to_owned());
match other {
Held::Sqlite(c) => sqlx::raw_sql(sql).execute(&mut **c).await.map(drop),
Held::Postgres(c) => sqlx::raw_sql(sql).execute(&mut **c).await.map(drop),
Held::Mysql(c) => sqlx::raw_sql(sql).execute(&mut **c).await.map(drop),
Held::SqlServer(_) | Held::Clickhouse(_) => unreachable!(),
Held::SqlServer(_) => unreachable!(),
}
.map_err(db_err)
}
@@ -862,7 +833,6 @@ impl Held {
Held::SqlServer(mut c) => {
c.0.take();
}
Held::Clickhouse(_) => {}
}
}
@@ -875,9 +845,6 @@ impl Held {
max_rows: usize,
cancel: &CancellationToken,
) -> IdeResult<QueryResult> {
if let Held::Clickhouse(c) = self {
return c.execute(sql, max_rows, cancel).await;
}
let started = Instant::now();
// `$id`: the query naming the server process, `$kill`: how another connection stops it.
macro_rules! run {
@@ -920,7 +887,6 @@ impl Held {
() = cancel.cancelled() => return Err(IdeError::cancelled()),
}
}
Held::Clickhouse(_) => unreachable!(),
};
let (columns, rows, affected, has_more, first) = fetched;
let elapsed_ms = started.elapsed().as_secs_f64() * 1000.0;
@@ -941,7 +907,7 @@ pub fn begin_statement(driver: DbDriver) -> &'static str {
match driver {
DbDriver::Mysql => "START TRANSACTION",
DbDriver::SqlServer => "BEGIN TRAN",
DbDriver::Sqlite | DbDriver::Postgres | DbDriver::Clickhouse => "BEGIN",
DbDriver::Sqlite | DbDriver::Postgres => "BEGIN",
}
}
+12 -76
View File
@@ -2,7 +2,6 @@
//! execution ([`conn`]), introspection, table data paging / editing ([`sql`])
//! and statement splitting ([`split`]), for SQLite, PostgreSQL and MySQL/MariaDB.
mod clickhouse;
pub mod conn;
pub mod history;
pub mod plan;
@@ -548,14 +547,12 @@ impl DbManager {
})?;
let version = match source.driver {
DbDriver::Sqlite => "SELECT 'SQLite ' || sqlite_version()",
DbDriver::Postgres | DbDriver::Clickhouse => "SELECT version()",
DbDriver::Postgres => "SELECT version()",
DbDriver::Mysql => "SELECT CONCAT(@@version_comment, ' ', VERSION())",
DbDriver::SqlServer => "SELECT @@VERSION",
};
let result = pool.rows(version, 1).await;
let v = pool.rows(version, 1).await.ok().and_then(|r| r.into_iter().next()?.into_iter().next()?);
pool.close().await;
let rows = if source.driver == DbDriver::Clickhouse { result? } else { result.unwrap_or_default() };
let v = rows.into_iter().next().and_then(|r| r.into_iter().next().flatten());
Ok(v.unwrap_or_else(|| "Connected".into()))
}
@@ -704,11 +701,8 @@ impl DbManager {
_ => indexes.push(IndexInfo { name, unique: flag(&unique), columns: vec![col] }),
}
}
let fk_rows = if driver == DbDriver::Clickhouse {
Vec::new()
} else {
pool.rows(&sql::table_foreign_keys(driver, schema, table), MAX_META_ROWS).await.unwrap_or_default()
};
let fk_rows =
pool.rows(&sql::table_foreign_keys(driver, schema, table), MAX_META_ROWS).await.unwrap_or_default();
let mut foreign_keys = group_foreign_keys(fk_rows, &[]);
// SQLite keys that reference the primary key implicitly: look it up.
for k in foreign_keys.iter_mut().filter(|k| k.ref_columns.is_empty()) {
@@ -721,15 +715,13 @@ impl DbManager {
.collect();
}
foreign_keys.retain(|k| k.columns.len() == k.ref_columns.len());
let trigger_rows = if driver == DbDriver::Clickhouse {
Vec::new()
} else {
pool.rows(&sql::triggers(driver, schema, table), MAX_META_ROWS).await.unwrap_or_else(|e| {
let triggers = pool
.rows(&sql::triggers(driver, schema, table), MAX_META_ROWS)
.await
.unwrap_or_else(|e| {
tracing::warn!("triggers: {}", e.message);
Vec::new()
})
};
let triggers = trigger_rows
.into_iter()
.filter_map(|r| {
let mut it = r.into_iter();
@@ -830,11 +822,6 @@ impl DbManager {
let ddl = cells(pool.rows(&q, 1).await?, 1).into_iter().next();
ddl.map(|d| format!("{d};\n")).ok_or_else(missing)
}
DbDriver::Clickhouse => {
let q = format!("SHOW CREATE TABLE {}", sql::qualified(driver, schema, name));
let ddl = cells(pool.rows(&q, 1).await?, 0).into_iter().next();
ddl.map(|d| format!("{};\n", d.trim_end().trim_end_matches(';'))).ok_or_else(missing)
}
DbDriver::SqlServer => {
let obj = format!("{}.{}", schema, name).replace('\'', "''");
let ddl = cells(pool.rows(&format!("SELECT OBJECT_DEFINITION(OBJECT_ID('{}'))", obj), 1).await?, 0)
@@ -915,14 +902,10 @@ impl DbManager {
}
}
// Foreign keys are a nicety (JOIN completion): a server that refuses the catalog query still completes.
let fk_rows = if driver == DbDriver::Clickhouse {
let fk_rows = pool.rows(sql::foreign_keys(driver), MAX_META_ROWS).await.unwrap_or_else(|e| {
tracing::warn!("foreign keys: {}", e.message);
Vec::new()
} else {
pool.rows(sql::foreign_keys(driver), MAX_META_ROWS).await.unwrap_or_else(|e| {
tracing::warn!("foreign keys: {}", e.message);
Vec::new()
})
};
});
let foreign_keys = group_foreign_keys(fk_rows, &tables);
let default_schema = match sql::current_schema(driver) {
None => "main".to_owned(),
@@ -997,9 +980,6 @@ impl DbManager {
let Some(session) = session else {
return pool.execute(statement, max_rows, cancel).await;
};
if pool.driver() == DbDriver::Clickhouse {
return Err(IdeError::invalid("ClickHouse manual transactions are unsupported"));
}
let mut held = session.lock().await;
if held.is_none() {
let mut conn = Held::acquire(&pool).await?;
@@ -1018,11 +998,8 @@ impl DbManager {
/// Switches `id` between auto-commit and manual commit. Leaving manual
/// mode rolls back what was not committed.
pub async fn set_manual_commit(&self, id: &str, manual: bool) -> IdeResult<()> {
pub async fn set_manual_commit(&self, id: &str, manual: bool) {
if manual {
if self.source(id)?.driver == DbDriver::Clickhouse {
return Err(IdeError::invalid("ClickHouse manual transactions are unsupported"));
}
self.sessions.lock().entry(id.to_owned()).or_default();
} else {
let session = self.sessions.lock().remove(id);
@@ -1030,14 +1007,10 @@ impl DbManager {
end_session(&s).await;
}
}
Ok(())
}
/// Commits (or rolls back) the open transaction of a manual-commit data source.
pub async fn finish_transaction(&self, id: &str, commit: bool) -> IdeResult<()> {
if self.source(id).is_ok_and(|s| s.driver == DbDriver::Clickhouse) {
return Err(IdeError::invalid("ClickHouse manual transactions are unsupported"));
}
let session = self.sessions.lock().get(id).cloned();
let Some(session) = session else { return Ok(()) };
let mut held = session.lock().await;
@@ -1079,9 +1052,6 @@ impl DbManager {
/// Applies the table editor's pending changes in one transaction.
pub async fn apply_edits(&self, id: &str, schema: &str, table: &str, edits: &[DbRowEdit]) -> IdeResult<u64> {
if self.source(id)?.driver == DbDriver::Clickhouse {
return Err(IdeError::invalid("ClickHouse table editing is unsupported"));
}
let pool = self.pool(id).await?;
let driver = pool.driver();
let statements = edits
@@ -1094,37 +1064,3 @@ impl DbManager {
pool.transaction(&statements).await
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn clickhouse_rejects_manual_transactions_and_table_edits_without_connecting() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path().to_path_buf();
let ssh = Arc::new(SshManager::new(root.join("ssh-passwords.json")));
let db = DbManager::new(root.clone(), root.join("passwords.json"), root.join("history.json"), ssh);
db.save_sources(vec![DataSource {
id: "clickhouse".into(),
name: "clickhouse".into(),
driver: DbDriver::Clickhouse,
host: "127.0.0.1".into(),
port: Some(1),
..Default::default()
}])
.await
.unwrap();
let error = db.set_manual_commit("clickhouse", true).await.unwrap_err();
assert!(error.message.contains("manual transactions are unsupported"));
assert!(db.sessions.lock().is_empty());
for commit in [false, true] {
let error = db.finish_transaction("clickhouse", commit).await.unwrap_err();
assert!(error.message.contains("manual transactions are unsupported"));
}
let error = db.apply_edits("clickhouse", "default", "events", &[]).await.unwrap_err();
assert!(error.message.contains("table editing is unsupported"));
assert!(!db.is_connected("clickhouse"));
db.set_manual_commit("clickhouse", false).await.unwrap();
}
}
+1 -18
View File
@@ -35,7 +35,6 @@ pub fn explain_statement(driver: DbDriver, statement: &str) -> String {
match driver {
DbDriver::Sqlite => format!("EXPLAIN QUERY PLAN {s}"),
DbDriver::Postgres => format!("EXPLAIN (FORMAT JSON) {s}"),
DbDriver::Clickhouse => format!("EXPLAIN json = 1 {s}"),
DbDriver::Mysql => format!("EXPLAIN {s}"),
DbDriver::SqlServer => format!("SET SHOWPLAN_TEXT ON; {s}"),
}
@@ -45,7 +44,7 @@ pub fn explain_statement(driver: DbDriver, statement: &str) -> String {
pub fn parse(driver: DbDriver, result: &QueryResult) -> IdeResult<Vec<PlanNode>> {
match driver {
DbDriver::Sqlite => Ok(sqlite(&result.rows)),
DbDriver::Postgres | DbDriver::Clickhouse => {
DbDriver::Postgres => {
let text: String =
result.rows.iter().filter_map(|r| r.first().cloned().flatten()).collect::<Vec<_>>().join("\n");
let json: Value = serde_json::from_str(&text).map_err(|e| IdeError::invalid(format!("plan: {e}")))?;
@@ -200,7 +199,6 @@ mod tests {
assert_eq!(explain_statement(DbDriver::Sqlite, "select 1;"), "EXPLAIN QUERY PLAN select 1");
assert_eq!(explain_statement(DbDriver::Postgres, " select 1 "), "EXPLAIN (FORMAT JSON) select 1");
assert_eq!(explain_statement(DbDriver::Mysql, "select 1"), "EXPLAIN select 1");
assert_eq!(explain_statement(DbDriver::Clickhouse, " select 1;"), "EXPLAIN json = 1 select 1");
assert_eq!(explain_statement(DbDriver::SqlServer, "select 1"), "SET SHOWPLAN_TEXT ON; select 1");
}
@@ -256,21 +254,6 @@ mod tests {
assert_eq!((plan[3].parent, plan[3].operation.as_str()), (Some(2), "Seq Scan"));
}
#[test]
fn reads_clickhouse_json_plans_and_rejects_invalid_json() {
let json = r#"[{"Plan": {"Node Type": "Expression", "Node Id": "Expression_5",
"Description": "Project names", "Plans": [
{"Node Type": "ReadFromMergeTree", "Node Id": "ReadFromMergeTree_0",
"Description": "default.events"}
]}}]"#;
let plan = parse(DbDriver::Clickhouse, &result(&["explain"], vec![vec![Some(json)]])).unwrap();
assert_eq!(plan.len(), 2);
assert_eq!((plan[0].parent, plan[0].operation.as_str()), (None, "Expression"));
assert_eq!((plan[1].parent, plan[1].operation.as_str()), (Some(0), "ReadFromMergeTree"));
assert!(plan[1].properties.iter().any(|p| p.name == "Description" && p.value == "default.events"));
assert!(parse(DbDriver::Clickhouse, &result(&["explain"], vec![vec![Some("invalid")]])).is_err());
}
#[test]
fn reads_mysql_rows() {
let r = result(
+1 -40
View File
@@ -18,7 +18,6 @@ pub enum DbDriver {
Mysql,
/// Microsoft SQL Server.
SqlServer,
Clickhouse,
}
impl DbDriver {
@@ -28,7 +27,6 @@ impl DbDriver {
DbDriver::Postgres => Some(5432),
DbDriver::Mysql => Some(3306),
DbDriver::SqlServer => Some(1433),
DbDriver::Clickhouse => Some(8123),
}
}
}
@@ -255,7 +253,6 @@ pub fn scheme_driver(scheme: &str) -> Option<DbDriver> {
"mysql" | "mariadb" | "tidb" | "singlestore" | "memsql" | "oceanbase" => Some(DbDriver::Mysql),
"sqlserver" | "mssql" | "mssqlserver" => Some(DbDriver::SqlServer),
"sqlite" | "sqlite3" => Some(DbDriver::Sqlite),
"clickhouse" | "http" | "https" => Some(DbDriver::Clickhouse),
_ => None,
}
}
@@ -263,7 +260,7 @@ pub fn scheme_driver(scheme: &str) -> Option<DbDriver> {
/// Parses `scheme://[user[:password]@]host[:port][/database][?k=v&…]`, `jdbc:` forms
/// included, and `sqlite:path`.
pub fn parse_url(raw: &str) -> IdeResult<ParsedUrl> {
let bad = || IdeError::invalid("invalid database URL");
let bad = || IdeError::invalid(format!("not a database URL: {raw}"));
let url = raw.trim();
let url = url.strip_prefix("jdbc:").or_else(|| url.strip_prefix("JDBC:")).unwrap_or(url);
let (scheme, rest) = url.split_once(':').ok_or_else(bad)?;
@@ -370,15 +367,6 @@ impl DataSource {
}
properties.append(&mut s.properties);
s.properties = properties;
if s.driver == DbDriver::Clickhouse {
let raw_url = self.url.trim().to_ascii_lowercase();
let url = raw_url.strip_prefix("jdbc:").unwrap_or(&raw_url);
if url.starts_with("https://") {
s.ssl = DbSslMode::VerifyFull;
} else if url.starts_with("http://") {
s.ssl = DbSslMode::Disable;
}
}
Ok(s)
}
@@ -531,31 +519,4 @@ mod tests {
assert!(DataSource { url: "mysql://h/db".into(), ..s.clone() }.resolved().is_err());
assert_eq!(DataSource { url_only: false, ..s }.resolved().unwrap().host, "localhost");
}
#[test]
fn clickhouse_urls_resolve_http_https_and_credentials() {
assert_eq!(DbDriver::Clickhouse.default_port(), Some(8123));
let source = DataSource {
driver: DbDriver::Clickhouse,
url_only: true,
url: "https://reader:s%[email protected]:8443/analytics?max_execution_time=30".into(),
..Default::default()
};
let resolved = source.resolved().unwrap();
assert_eq!(resolved.host, "db.example");
assert_eq!(resolved.port, Some(8443));
assert_eq!(resolved.database, "analytics");
assert_eq!(resolved.user, "reader");
assert_eq!(resolved.ssl, DbSslMode::VerifyFull);
assert_eq!(source.url_password().as_deref(), Some("s@cret"));
assert_eq!(resolved.properties[0].name, "max_execution_time");
let http = DataSource { url: "http://localhost/default".into(), ..source.clone() }.resolved().unwrap();
assert_eq!(http.ssl, DbSslMode::Disable);
let jdbc = DataSource { url: "jdbc:clickhouse://localhost/analytics".into(), ..source.clone() };
assert_eq!(jdbc.resolved().unwrap().database, "analytics");
let secure = DataSource { url: "https://db.example/default?sslmode=disable".into(), ..source };
assert_eq!(secure.resolved().unwrap().ssl, DbSslMode::VerifyFull);
let error = parse_url("clickhouse://reader:[email protected]:invalid/default").unwrap_err();
assert!(!error.message.contains("secret"));
}
}
+4 -15
View File
@@ -28,7 +28,6 @@ enum State {
/// Every statement of `text` that contains more than whitespace and comments.
pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
let mysql = driver == DbDriver::Mysql;
let clickhouse = driver == DbDriver::Clickhouse;
let chars: Vec<(usize, char)> = text.char_indices().collect();
// UTF-16 offset of each char index (plus the end).
let mut u16_at = Vec::with_capacity(chars.len() + 1);
@@ -82,7 +81,7 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
state = State::Double;
has_code = true;
}
'`' if mysql || clickhouse => {
'`' if mysql => {
state = State::Backtick;
has_code = true;
}
@@ -118,7 +117,7 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
_ => {}
},
State::Single => {
if (mysql || clickhouse) && c == '\\' {
if mysql && c == '\\' {
i += 1;
} else if c == '\'' {
if next == Some('\'') {
@@ -129,9 +128,7 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
}
}
State::Double => {
if clickhouse && c == '\\' {
i += 1;
} else if c == '"' {
if c == '"' {
if next == Some('"') {
i += 1;
} else {
@@ -140,9 +137,7 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
}
}
State::Backtick => {
if clickhouse && (c == '\\' || (c == '`' && next == Some('`'))) {
i += 1;
} else if c == '`' {
if c == '`' {
state = State::Code;
}
}
@@ -229,12 +224,6 @@ mod tests {
assert_eq!(texts(sql, DbDriver::Mysql), vec!["select `a;b`, 'it\\'s;' from t # x;y", "select 2"]);
}
#[test]
fn clickhouse_backticks_and_escaped_quotes_keep_statements_intact() {
let query = r#"SELECT `a\`;b`, "c\";d", 'it\'s;safe'; SELECT 2"#;
assert_eq!(texts(query, DbDriver::Clickhouse), vec![r#"SELECT `a\`;b`, "c\";d", 'it\'s;safe'"#, "SELECT 2"]);
}
#[test]
fn postgres_dollar_quotes() {
let sql =
+2 -106
View File
@@ -7,7 +7,6 @@ use specta::Type;
pub fn quote_ident(driver: DbDriver, name: &str) -> String {
match driver {
DbDriver::Clickhouse => format!("`{}`", name.replace('\\', "\\\\").replace('`', "\\`")),
DbDriver::Mysql => format!("`{}`", name.replace('`', "``")),
DbDriver::SqlServer => format!("[{}]", name.replace(']', "]]")),
_ => format!("\"{}\"", name.replace('"', "\"\"")),
@@ -16,7 +15,7 @@ pub fn quote_ident(driver: DbDriver, name: &str) -> String {
pub fn quote_literal(driver: DbDriver, value: &str) -> String {
let mut s = value.replace('\'', "''");
if matches!(driver, DbDriver::Mysql | DbDriver::Clickhouse) {
if driver == DbDriver::Mysql {
s = s.replace('\\', "\\\\");
}
format!("'{s}'")
@@ -49,10 +48,6 @@ pub const INTERNAL_DATABASES: &[&str] = &["rdsadmin", "azure_maintenance", "azur
pub fn schemas(driver: DbDriver, system: bool) -> &'static str {
match driver {
DbDriver::Sqlite => "SELECT name FROM pragma_database_list WHERE name <> 'temp' ORDER BY seq",
DbDriver::Clickhouse if system => "SELECT name FROM system.databases ORDER BY name",
DbDriver::Clickhouse => {
"SELECT name FROM system.databases WHERE lower(name) NOT IN ('system', 'information_schema') ORDER BY name"
}
DbDriver::Postgres if system => {
"SELECT schema_name FROM information_schema.schemata \
WHERE schema_name NOT LIKE 'pg\\_toast%' AND schema_name NOT LIKE 'pg\\_temp%' ORDER BY 1"
@@ -78,12 +73,6 @@ pub fn schemas(driver: DbDriver, system: bool) -> &'static str {
/// Rows: `name, kind` where kind is `table`, `view` or `materialized view`.
pub fn tables(driver: DbDriver, schema: &str) -> String {
match driver {
DbDriver::Clickhouse => format!(
"SELECT name, CASE engine WHEN 'View' THEN 'view' WHEN 'MaterializedView' THEN 'materialized view' \
WHEN 'WindowView' THEN 'view' ELSE 'table' END FROM system.tables \
WHERE database = {} AND is_temporary = 0 ORDER BY name",
quote_literal(driver, schema)
),
DbDriver::Sqlite => format!(
"SELECT name, type FROM {}.sqlite_master WHERE type IN ('table', 'view') \
AND name NOT LIKE 'sqlite\\_%' ESCAPE '\\' ORDER BY name",
@@ -112,12 +101,6 @@ pub fn tables(driver: DbDriver, schema: &str) -> String {
pub fn columns(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
match driver {
DbDriver::Clickhouse => format!(
"SELECT name, type, CASE WHEN startsWith(type, 'Nullable(') \
OR startsWith(type, 'LowCardinality(Nullable(') THEN 'YES' ELSE 'NO' END, \
if(default_kind = '', NULL, default_expression), is_in_primary_key \
FROM system.columns WHERE database = {s} AND table = {t} ORDER BY position"
),
DbDriver::Sqlite => format!(
"SELECT name, type, CASE WHEN \"notnull\" = 0 THEN 'YES' ELSE 'NO' END, dflt_value, \
CASE WHEN pk > 0 THEN 1 ELSE 0 END FROM pragma_table_info({t}, {s}) ORDER BY cid"
@@ -155,12 +138,6 @@ pub fn columns(driver: DbDriver, schema: &str, table: &str) -> String {
pub fn indexes(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
match driver {
DbDriver::Clickhouse => format!(
"SELECT * FROM (SELECT 'PRIMARY KEY' AS index_name, 0 AS is_unique, primary_key AS expression \
FROM system.tables WHERE database = {s} AND name = {t} AND primary_key != '' \
UNION ALL SELECT name, 0, expr FROM system.data_skipping_indices \
WHERE database = {s} AND table = {t}) ORDER BY index_name"
),
DbDriver::Sqlite => format!(
"SELECT il.name, il.\"unique\", ii.name FROM pragma_index_list({t}, {s}) il \
JOIN pragma_index_info(il.name, {s}) ii ORDER BY il.name, ii.seqno"
@@ -192,15 +169,6 @@ pub fn indexes(driver: DbDriver, schema: &str, table: &str) -> String {
/// for SQL completion, grouped by table in column order.
pub fn completion_columns(driver: DbDriver) -> &'static str {
match driver {
DbDriver::Clickhouse => {
"SELECT c.database, c.table, CASE t.engine WHEN 'View' THEN 'view' \
WHEN 'MaterializedView' THEN 'materialized view' WHEN 'WindowView' THEN 'view' ELSE 'table' END, \
c.name, c.type, CASE WHEN startsWith(c.type, 'Nullable(') \
OR startsWith(c.type, 'LowCardinality(Nullable(') THEN 'YES' ELSE 'NO' END, c.is_in_primary_key, 1 \
FROM system.columns c JOIN system.tables t ON t.database = c.database AND t.name = c.table \
WHERE lower(c.database) NOT IN ('system', 'information_schema') AND t.is_temporary = 0 \
ORDER BY c.database, c.table, c.position LIMIT 50000"
}
DbDriver::Sqlite => {
"SELECT 'main', m.name, m.type, p.name, p.type, CASE WHEN p.\"notnull\" = 0 THEN 'YES' ELSE 'NO' END, \
CASE WHEN p.pk > 0 THEN 1 ELSE 0 END, \
@@ -253,7 +221,6 @@ pub fn completion_columns(driver: DbDriver) -> &'static str {
/// when the key references the primary key implicitly.
pub fn foreign_keys(driver: DbDriver) -> &'static str {
match driver {
DbDriver::Clickhouse => "SELECT '', '', '', '', '', '', '' WHERE 0",
DbDriver::Sqlite => {
"SELECT 'main', m.name, f.id, f.\"from\", 'main', f.\"table\", f.\"to\" \
FROM sqlite_master m JOIN pragma_foreign_key_list(m.name) f \
@@ -291,7 +258,6 @@ pub fn foreign_keys(driver: DbDriver) -> &'static str {
pub fn table_foreign_keys(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
match driver {
DbDriver::Clickhouse => foreign_keys(driver).to_owned(),
DbDriver::Sqlite => format!(
"SELECT {s}, {t}, f.id, f.\"from\", {s}, f.\"table\", f.\"to\" FROM pragma_foreign_key_list({t}, {s}) f ORDER BY f.id, f.seq"
),
@@ -310,7 +276,6 @@ pub fn table_foreign_keys(driver: DbDriver, schema: &str, table: &str) -> String
/// The schema unqualified names resolve to (`None`: always `main`).
pub fn current_schema(driver: DbDriver) -> Option<&'static str> {
match driver {
DbDriver::Clickhouse => Some("SELECT currentDatabase()"),
DbDriver::Sqlite => None,
DbDriver::Postgres => Some("SELECT current_schema()"),
DbDriver::Mysql => Some("SELECT DATABASE()"),
@@ -397,9 +362,6 @@ fn where_key(driver: DbDriver, key: &[CellValue]) -> String {
/// The DML statement for `edit`. Values are written as quoted literals, which
/// every supported database coerces to the column type.
pub fn edit_statement(driver: DbDriver, schema: &str, table: &str, edit: &DbRowEdit) -> Result<String, String> {
if driver == DbDriver::Clickhouse {
return Err("ClickHouse table editing is unsupported: primary keys do not uniquely identify rows and edits require transactions".into());
}
let target = qualified(driver, schema, table);
match edit {
DbRowEdit::Update { key, values } => {
@@ -488,7 +450,6 @@ pub fn postgres_ddl_indexes(oid: u32) -> String {
pub fn triggers(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
match driver {
DbDriver::Clickhouse => "SELECT '', '', '' WHERE 0".to_owned(),
DbDriver::Sqlite => format!(
"SELECT name, sql FROM {}.sqlite_master WHERE type = 'trigger' AND tbl_name = {t} ORDER BY name",
quote_ident(driver, schema)
@@ -531,7 +492,7 @@ pub fn sqlite_trigger_event(sql: &str) -> (String, String) {
pub fn routines(driver: DbDriver, schema: &str) -> Option<String> {
let s = quote_literal(driver, schema);
match driver {
DbDriver::Sqlite | DbDriver::Clickhouse => None,
DbDriver::Sqlite => None,
// Functions of extensions are left out, as in IDEA.
DbDriver::Postgres => Some(format!(
"SELECT p.proname, CASE WHEN p.prokind = 'p' THEN 'procedure' ELSE 'function' END, \
@@ -664,71 +625,6 @@ mod tests {
assert_eq!(quote_literal(DbDriver::Mysql, "it's \\"), "'it''s \\\\'");
}
#[test]
fn clickhouse_escapes_backslashes_and_delimiters() {
let driver = DbDriver::Clickhouse;
assert_eq!(quote_literal(driver, r"path\O'Brien"), r"'path\\O''Brien'");
assert_eq!(quote_literal(driver, r"\' OR 1 = 1 --"), r"'\\'' OR 1 = 1 --'");
assert_eq!(quote_ident(driver, r"a\`b"), r"`a\\\`b`");
assert_eq!(qualified(driver, "data base", "event`log"), r"`data base`.`event\`log`");
}
#[test]
fn clickhouse_metadata_preserves_types_defaults_and_nonunique_keys() {
let driver = DbDriver::Clickhouse;
assert_eq!(schemas(driver, true), "SELECT name FROM system.databases ORDER BY name");
assert!(schemas(driver, false).contains("lower(name) NOT IN ('system', 'information_schema')"));
assert_eq!(current_schema(driver), Some("SELECT currentDatabase()"));
let relations = tables(driver, "event's\\db");
assert!(relations.contains("WHEN 'MaterializedView' THEN 'materialized view'"));
assert!(relations.contains(r"WHERE database = 'event''s\\db' AND is_temporary = 0"));
let fields = columns(driver, "default", "events");
assert!(fields.starts_with("SELECT name, type, CASE WHEN startsWith(type, 'Nullable(')"));
assert!(fields.contains("startsWith(type, 'LowCardinality(Nullable(')"));
assert!(fields.contains("if(default_kind = '', NULL, default_expression), is_in_primary_key"));
assert!(fields.ends_with("WHERE database = 'default' AND table = 'events' ORDER BY position"));
let keys = indexes(driver, "default", "events");
assert!(keys.contains("'PRIMARY KEY' AS index_name, 0 AS is_unique, primary_key AS expression"));
assert!(keys.contains("SELECT name, 0, expr FROM system.data_skipping_indices"));
let completion = completion_columns(driver);
assert!(completion.contains("c.is_in_primary_key, 1"));
assert!(completion.contains("ORDER BY c.database, c.table, c.position LIMIT 50000"));
}
#[test]
fn clickhouse_has_no_foreign_key_trigger_or_schema_routine_catalogs() {
let driver = DbDriver::Clickhouse;
assert_eq!(foreign_keys(driver), "SELECT '', '', '', '', '', '', '' WHERE 0");
assert_eq!(table_foreign_keys(driver, "default", "events"), foreign_keys(driver));
assert_eq!(triggers(driver, "default", "events"), "SELECT '', '', '' WHERE 0");
assert!(routines(driver, "default").is_none());
assert!(sequences(driver, "default").is_none());
assert!(object_ddl(driver, "default", crate::DbObjectKind::Trigger, "t").is_none());
}
#[test]
fn clickhouse_pages_quote_keys_and_fetch_one_extra_row() {
let sort = [SortKey { column: "event`id".into(), descending: true }];
assert_eq!(
table_page(DbDriver::Clickhouse, "default", "events", Some("WHERE id > 3"), &sort, 50, 25),
r"SELECT * FROM `default`.`events` WHERE id > 3 ORDER BY `event\`id` DESC LIMIT 26 OFFSET 50"
);
}
#[test]
fn clickhouse_keys_never_enable_transactional_table_edits() {
let key = vec![cv("id", Some("1"))];
for edit in [
DbRowEdit::Update { key: key.clone(), values: vec![cv("name", Some("new"))] },
DbRowEdit::Delete { key },
DbRowEdit::Insert { values: vec![cv("name", Some("new"))] },
DbRowEdit::Insert { values: vec![] },
] {
let error = edit_statement(DbDriver::Clickhouse, "default", "events", &edit).unwrap_err();
assert!(error.contains("primary keys do not uniquely identify rows"));
}
}
#[test]
fn pages() {
let sort = [SortKey { column: "id".into(), descending: true }];
-136
View File
@@ -1,136 +0,0 @@
use ide_core::{CancellationToken, IdeError, IdeResult};
use ide_db::{CellValue, DataSource, DbDriver, DbManager, DbRowEdit, DbSslMode, SortKey, sql};
use std::sync::Arc;
use std::time::{SystemTime, UNIX_EPOCH};
fn check(condition: bool, message: &str) -> IdeResult<()> {
if condition { Ok(()) } else { Err(IdeError::invalid(message)) }
}
#[tokio::test]
#[ignore = "requires CLICKHOUSE_TEST_HOST, CLICKHOUSE_TEST_USER, and CLICKHOUSE_TEST_PASSWORD"]
async fn clickhouse_cloud_manager_catalogs_and_session_table() -> IdeResult<()> {
let tmp = tempfile::tempdir()?;
let root = tmp.path().to_path_buf();
let ssh = Arc::new(ide_ssh::SshManager::new(root.join("ssh-passwords.json")));
let db = DbManager::new(root.clone(), root.join("passwords.json"), root.join("history.json"), ssh);
let mut source = DataSource {
id: "clickhouse".into(),
name: "clickhouse".into(),
driver: DbDriver::Clickhouse,
host: std::env::var("CLICKHOUSE_TEST_HOST").expect("set CLICKHOUSE_TEST_HOST"),
port: Some(
std::env::var("CLICKHOUSE_TEST_PORT")
.map(|value| value.parse().expect("numeric CLICKHOUSE_TEST_PORT"))
.unwrap_or(8443),
),
user: std::env::var("CLICKHOUSE_TEST_USER").expect("set CLICKHOUSE_TEST_USER"),
database: "default".into(),
ssl: DbSslMode::VerifyFull,
..Default::default()
};
source.introspection.show_system = true;
let password = std::env::var("CLICKHOUSE_TEST_PASSWORD").expect("set CLICKHOUSE_TEST_PASSWORD");
check(!db.test(&source, Some(password.clone()), None).await?.is_empty(), "missing server version")?;
db.save_sources(vec![source]).await?;
db.set_password("clickhouse", Some(password), false).await?;
let cancel = CancellationToken::new();
let suffix = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_nanos();
let name = format!("manager_test_{}_{suffix}", std::process::id());
let table = sql::quote_ident(DbDriver::Clickhouse, &name);
db.execute(
"clickhouse",
&format!(
"CREATE TEMPORARY TABLE {table} (id UInt64, label Nullable(String) DEFAULT NULL, \
category LowCardinality(Nullable(String)) DEFAULT NULL, items Array(Nullable(UInt8)), \
label_size UInt64 ALIAS length(ifNull(label, ''))) ENGINE = Memory"
),
0,
&cancel,
)
.await?;
let checked: IdeResult<()> = async {
db.execute(
"clickhouse",
&format!("INSERT INTO {table} (id, label, items) VALUES (1, 'first', [1, NULL]), (1, NULL, [])"),
0,
&cancel,
)
.await?;
let schemas = db.schemas("clickhouse").await?;
check(schemas.iter().any(|schema| schema == "default"), "default database missing")?;
check(schemas.iter().any(|schema| schema == "system"), "system database missing")?;
let tables = db.tables("clickhouse", "system").await?;
check(tables.iter().any(|table| table.name == "one"), "system table listing missing one")?;
let details = db.table_details("clickhouse", "", &name).await?;
check(details.columns.len() == 5, "temporary table columns missing")?;
for (name, data_type, nullable) in [
("id", "UInt64", false),
("label", "Nullable(String)", true),
("category", "LowCardinality(Nullable(String))", true),
("items", "Array(Nullable(UInt8))", false),
] {
check(
details
.columns
.iter()
.any(|column| column.name == name && column.data_type == data_type && column.nullable == nullable),
"column type or nullability mismatch",
)?;
}
check(
details.columns.iter().any(|column| column.name == "label" && column.default.as_deref() == Some("NULL")),
"explicit default expression missing",
)?;
check(details.indexes.iter().all(|index| !index.unique), "ClickHouse index reported unique")?;
check(details.foreign_keys.is_empty() && details.triggers.is_empty(), "unsupported table objects returned")?;
let sort = [SortKey { column: "id".into(), descending: false }];
let first = db.table_page("clickhouse", "", &name, None, &sort, 0, 1, &cancel).await?;
check(first.result.rows.len() == 1 && first.result.has_more, "first page limit failed")?;
let second = db.table_page("clickhouse", "", &name, None, &sort, 1, 1, &cancel).await?;
check(second.result.rows.len() == 1 && !second.result.has_more, "page offset failed")?;
let ddl = db.ddl("clickhouse", "", &name).await?;
check(ddl.contains(&name) && ddl.contains("Memory"), "temporary table DDL missing")?;
let completion = db.completion("clickhouse").await?;
check(completion.default_schema == "default", "completion default database mismatch")?;
check(completion.foreign_keys.is_empty(), "completion returned unsupported foreign keys")?;
if let Some(table) = completion.tables.first() {
let details = db.table_details("clickhouse", &table.schema, &table.name).await?;
check(
table.columns.iter().all(|column| {
details.columns.iter().any(|field| field.name == column.name && field.data_type == column.data_type)
}),
"completion columns differ from catalog",
)?;
}
let plan = db.explain("clickhouse", &format!("SELECT id FROM {table}"), &cancel).await?;
check(!plan.is_empty(), "query plan missing")?;
let value = r"path\O'Brien";
let result = db
.execute("clickhouse", &format!("SELECT {}", sql::quote_literal(DbDriver::Clickhouse, value)), 1, &cancel)
.await?;
check(result.rows == vec![vec![Some(value.to_owned())]], "quoted literal changed value")?;
let edits = [DbRowEdit::Update {
key: vec![CellValue { column: "id".into(), value: Some("1".into()) }],
values: vec![CellValue { column: "label".into(), value: Some("changed".into()) }],
}];
check(
db.apply_edits("clickhouse", "", &name, &edits)
.await
.err()
.is_some_and(|error| error.message.contains("table editing is unsupported")),
"supplied row key authorized table editing",
)?;
check(db.set_manual_commit("clickhouse", true).await.is_err(), "manual transactions were accepted")?;
for commit in [false, true] {
check(db.finish_transaction("clickhouse", commit).await.is_err(), "transaction control was accepted")?;
}
Ok(())
}
.await;
let cleanup = db.execute("clickhouse", &format!("DROP TEMPORARY TABLE {table}"), 0, &cancel).await;
db.disconnect("clickhouse").await;
checked?;
cleanup?;
Ok(())
}
+3 -3
View File
@@ -197,7 +197,7 @@ async fn count(db: &DbManager) -> String {
async fn manual_commit_keeps_a_transaction_until_commit_or_rollback() -> IdeResult<()> {
let (_tmp, db) = setup().await;
let c = CancellationToken::new();
db.set_manual_commit("s1", true).await.unwrap();
db.set_manual_commit("s1", true).await;
db.execute("s1", "INSERT INTO users (name) VALUES ('dee')", 1, &c).await?;
// The session sees its own change.
assert_eq!(count(&db).await, "4");
@@ -208,10 +208,10 @@ async fn manual_commit_keeps_a_transaction_until_commit_or_rollback() -> IdeResu
db.finish_transaction("s1", true).await?;
// Leaving manual mode rolls back what is still open.
db.execute("s1", "DELETE FROM users", 1, &c).await?;
db.set_manual_commit("s1", false).await.unwrap();
db.set_manual_commit("s1", false).await;
assert_eq!(count(&db).await, "4");
// Disconnecting with an open transaction does not hang.
db.set_manual_commit("s1", true).await.unwrap();
db.set_manual_commit("s1", true).await;
db.execute("s1", "DELETE FROM users", 1, &c).await?;
db.disconnect("s1").await;
assert_eq!(count(&db).await, "4");
+2 -2
View File
@@ -27,8 +27,8 @@ mod when;
pub use capability::{Capability, CapabilityInfo, Risk, host_matches};
pub use index::ContributionIndex;
pub use manifest::{
ActivationEvent, CATEGORIES, Capabilities, ENGINE_VERSION, EVENT_KINDS, LEGACY_API, MANIFEST_FILE, MAX_CATEGORIES,
Manifest, check_relative, engine_compatible, validate_item_id, validate_plugin_id, validate_point_id,
ActivationEvent, Capabilities, ENGINE_VERSION, EVENT_KINDS, LEGACY_API, MANIFEST_FILE, Manifest, check_relative,
engine_compatible, validate_item_id, validate_plugin_id, validate_point_id,
};
pub use point::{Commit, Disposable, ExtensionPoint, MAX_PACKAGE_FILE, Package, Placement, Prepared, decode};
pub use registry::{ContributionInfo, ExtensionRegistry, PreparedSet};
-66
View File
@@ -35,11 +35,6 @@ pub const MANIFEST_FILE: &str = "plugin.toml";
pub const ENGINE_VERSION: &str = "1.0.0";
/// The 0.1 API, still accepted for contribution-only packages and 0.1 components.
pub const LEGACY_API: &str = "0.1";
/// Marketplace categories, in the order they are shown.
pub const CATEGORIES: &[&str] =
&["languages", "themes", "icon-themes", "linters", "vcs", "ai", "frameworks", "tools", "other"];
/// Categories a plugin is listed under; later ones are ignored.
pub const MAX_CATEGORIES: usize = 3;
#[derive(Clone, Debug, PartialEq)]
pub struct Manifest {
@@ -52,8 +47,6 @@ pub struct Manifest {
pub license: Option<String>,
/// Release notes of this version (Settings › Plugins › What's New).
pub change_notes: Option<String>,
/// Marketplace categories as declared (see [`Manifest::categories`]).
pub declared_categories: Vec<String>,
/// `engines.openidea` (a semver requirement), or the legacy `openidea-api`.
pub engine: String,
/// Written in the 0.1 format.
@@ -173,8 +166,6 @@ struct Raw {
#[serde(default)]
change_notes: Option<String>,
#[serde(default)]
categories: Vec<String>,
#[serde(default)]
engines: Option<Engines>,
/// 0.1 format.
#[serde(default)]
@@ -252,7 +243,6 @@ impl Manifest {
homepage: raw.homepage,
license: raw.license,
change_notes: raw.change_notes,
declared_categories: raw.categories,
engine,
legacy,
wasm: raw.wasm,
@@ -300,9 +290,6 @@ impl Manifest {
return Err(format!("duplicate activation event {event}"));
}
}
for c in &self.declared_categories {
validate_item_id(c, "category")?;
}
for (point, decls) in &self.contributes {
validate_point_id(point)?;
if !matches!(decls, toml::Value::Array(_) | toml::Value::Table(_)) {
@@ -338,39 +325,6 @@ impl Manifest {
}
}
impl Manifest {
/// Its marketplace categories, in [`CATEGORIES`] order: the first
/// [`MAX_CATEGORIES`] declared that this IDE knows; without any, those its
/// contributions imply (themes, icon themes, languages), else `other`.
pub fn categories(&self) -> Vec<String> {
let pick = |f: &dyn Fn(&str) -> bool| CATEGORIES.iter().filter(|c| f(c)).map(|c| c.to_string()).collect();
let mut known: Vec<&str> = Vec::new();
for d in &self.declared_categories {
if CATEGORIES.contains(&d.as_str()) && !known.contains(&d.as_str()) && known.len() < MAX_CATEGORIES {
known.push(d);
}
}
let mut out: Vec<String> = pick(&|c| known.contains(&c));
if out.is_empty() {
out = pick(&|c| match c {
"themes" => self.contributes.contains_key("themes"),
"icon-themes" => self.contributes.contains_key("iconThemes"),
"languages" => self.contributes.contains_key("languages"),
_ => false,
});
}
if out.is_empty() {
out.push("other".into());
}
out
}
/// Declared categories this IDE doesn't know (ignored).
pub fn unknown_categories(&self) -> Vec<&str> {
self.declared_categories.iter().map(String::as_str).filter(|c| !CATEGORIES.contains(c)).collect()
}
}
/// Whether this host can load a plugin written for `engine`.
pub fn engine_compatible(engine: &str, legacy: bool) -> bool {
if legacy {
@@ -463,26 +417,6 @@ mod tests {
assert!(ActivationEvent::parse("on-event:scheduled").is_err(), "declared with on-schedule");
}
#[test]
fn categories_are_declared_or_implied() {
let head = "id = \"acme.x\"\nname = \"X\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"\n";
let m = |rest: &str| Manifest::parse(&format!("{head}{rest}"));
// Declared, in the catalogue's order; unknown ones (a newer IDE's) are ignored.
let declared = m("categories = [\"tools\", \"vcs\", \"robots\"]\n").unwrap();
assert_eq!(declared.categories(), ["vcs", "tools"]);
assert_eq!(declared.unknown_categories(), ["robots"]);
// None declared: implied by the contributions, else `other`.
let theme = m("[[contributes.themes]]\nid = \"t\"\n[[contributes.languages]]\nid = \"l\"\n").unwrap();
assert_eq!(theme.categories(), ["languages", "themes"]);
let declared_wins = m("categories = [\"tools\"]\n[[contributes.languages]]\nid = \"l\"\n").unwrap();
assert_eq!(declared_wins.categories(), ["tools"]);
assert_eq!(m("").unwrap().categories(), ["other"]);
// At most three: the first ones declared.
let many = m("categories = [\"vcs\", \"ai\", \"vcs\", \"tools\", \"themes\"]\n").unwrap();
assert_eq!(many.categories(), ["vcs", "ai", "tools"]);
assert!(m("categories = [\"Bad Name\"]\n").is_err());
}
#[test]
fn parses_the_02_format() {
let m = Manifest::parse(FULL).unwrap();
+23 -513
View File
@@ -9,19 +9,9 @@
//! proves who published the bytes; whether to trust that publisher is the
//! index's call.
//! - **Index** (`index.json`): the publishers and their keys (the trust root),
//! the published plugin versions, and revocations. A publisher owns the id
//! prefix `<publisher>.`; a revoked key invalidates everything it signed, a
//! revoked plugin version is disabled where it is installed. A plugin may be
//! listed in several versions, each with the plugin API it needs: the IDE
//! offers the newest one it can load.
//! - **Key continuity**: an installed plugin updates silently only to a
//! package signed with the same key, or with a key the installed one
//! endorsed ([`Endorsement`], a publisher rotating keys); any other signer
//! is the user's call. An IDE can pin a publisher to keys it was built with
//! ([`Index::pin_publisher`]): the index then can't replace them.
//! - **Format**: fields are only ever added (IDEs ignore those they don't
//! know); `format` grows only for changes older IDEs can't read, and they
//! still apply such an index's revocations ([`Index::parse`]).
//! the published plugins, and revocations. A publisher owns the id prefix
//! `<publisher>.`; a revoked key invalidates everything it signed, a
//! revoked plugin version is disabled where it is installed.
use crate::MANIFEST_FILE;
use base64::Engine as _;
@@ -50,24 +40,6 @@ fn signed_message(id: &str, version: &str, digest: &str) -> Vec<u8> {
format!("openidea-plugin/1\n{id}\n{version}\n{digest}").into_bytes()
}
fn endorsement_message(publisher: &str, key: &str) -> Vec<u8> {
format!("openidea-key/1\n{publisher}\n{key}").into_bytes()
}
/// Whether `signature` (base64) is `key`'s (base64) Ed25519 signature of
/// `message`; malformed keys and signatures are errors.
fn verify_ed25519(key: &str, message: &[u8], signature: &str) -> Result<bool, String> {
let key: [u8; 32] = B64
.decode(key)
.ok()
.and_then(|b| b.try_into().ok())
.ok_or_else(|| "signature: malformed public key".to_owned())?;
let key = VerifyingKey::from_bytes(&key).map_err(|_| "signature: invalid public key".to_owned())?;
let sig: [u8; 64] =
B64.decode(signature).ok().and_then(|b| b.try_into().ok()).ok_or_else(|| "signature: malformed".to_owned())?;
Ok(key.verify_strict(message, &Signature::from_bytes(&sig)).is_ok())
}
/// `SIGNATURE.json` of a signed package.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
@@ -102,12 +74,6 @@ impl PublisherKey {
B64.encode(self.0.verifying_key().to_bytes())
}
/// Vouches for `publisher`'s new key `key` (base64), the one that follows this one.
pub fn endorse(&self, publisher: &str, key: &str) -> Endorsement {
let sig = self.0.sign(&endorsement_message(publisher, key));
Endorsement { key: self.public_base64(), signature: B64.encode(sig.to_bytes()) }
}
pub fn sign(&self, publisher: &str, id: &str, version: &str, digest: &str) -> PackageSignature {
let sig = self.0.sign(&signed_message(id, version, digest));
PackageSignature {
@@ -122,11 +88,19 @@ impl PackageSignature {
/// Whether the signature is valid for `id`, `version` and `digest` (the
/// key's validity is checked against an [`Index`]).
pub fn check(&self, id: &str, version: &str, digest: &str) -> Result<(), String> {
if verify_ed25519(&self.key, &signed_message(id, version, digest), &self.signature)? {
Ok(())
} else {
Err("the signature does not match the package (it was changed after signing)".to_owned())
}
let key: [u8; 32] = B64
.decode(&self.key)
.ok()
.and_then(|b| b.try_into().ok())
.ok_or_else(|| "signature: malformed public key".to_owned())?;
let key = VerifyingKey::from_bytes(&key).map_err(|_| "signature: invalid public key".to_owned())?;
let sig: [u8; 64] = B64
.decode(&self.signature)
.ok()
.and_then(|b| b.try_into().ok())
.ok_or_else(|| "signature: malformed".to_owned())?;
key.verify_strict(&signed_message(id, version, digest), &Signature::from_bytes(&sig))
.map_err(|_| "the signature does not match the package (it was changed after signing)".to_owned())
}
}
@@ -268,10 +242,6 @@ pub struct Index {
pub plugins: Vec<IndexEntry>,
#[serde(default)]
pub revoked: Vec<Revocation>,
/// The marketplace moved: its index is at this URL (relative to this
/// one, or absolute), and the IDE reads that one instead.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub moved: Option<String>,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
@@ -283,7 +253,7 @@ pub struct Publisher {
pub keys: Vec<KeyEntry>,
}
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct KeyEntry {
/// Ed25519 public key, base64.
@@ -291,27 +261,6 @@ pub struct KeyEntry {
/// Packages signed with it are no longer trusted.
#[serde(default)]
pub revoked: bool,
/// The publisher's earlier key vouching for this one.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub endorsement: Option<Endorsement>,
}
impl KeyEntry {
pub fn new(key: String) -> Self {
Self { key, ..Self::default() }
}
}
/// A publisher's earlier key vouching for a newer one: plugins installed from
/// packages signed with the earlier key update to ones signed with the newer
/// key without asking the user.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct Endorsement {
/// The endorsing key, base64.
pub key: String,
/// Its Ed25519 signature of `openidea-key/1\n<publisher>\n<endorsed key>`, base64.
pub signature: String,
}
/// A published plugin version.
@@ -332,34 +281,6 @@ pub struct IndexEntry {
pub digest: String,
#[serde(default)]
pub capabilities: Vec<String>,
/// The plugin API versions it loads in (a semver requirement): its
/// manifest's `engines.openidea` and the API its component was built for.
/// Required: an entry without one is offered to no IDE.
#[serde(default)]
pub engine: String,
/// Marketplace categories (`CATEGORIES`); an IDE ignores those it doesn't know.
#[serde(default)]
pub categories: Vec<String>,
}
impl IndexEntry {
/// Whether this IDE can load the version.
pub fn compatible(&self) -> bool {
crate::engine_compatible(&self.engine, false)
}
fn semver(&self) -> Option<semver::Version> {
semver::Version::parse(&self.version).ok()
}
}
/// `engines.openidea` of a manifest narrowed to the API its component was built
/// for (`major.minor`, if it has code): the requirement an index lists.
pub fn engine_requirement(manifest_engine: &str, built_for: Option<&semver::Version>) -> String {
match built_for {
Some(v) => format!("{manifest_engine}, ^{}.{}", v.major, v.minor),
None => manifest_engine.to_owned(),
}
}
/// A plugin (all versions when `versions` is empty) that must not run.
@@ -373,71 +294,12 @@ pub struct Revocation {
}
impl Index {
/// Parses an index. One in a newer format keeps what must reach every IDE —
/// revocations, publisher keys (revoked ones) and `moved` — and lists no
/// plugins ([`Index::is_newer`]): the IDE needs updating to install from it.
pub fn parse(text: &str) -> Result<Self, String> {
let value: serde_json::Value = serde_json::from_str(text).map_err(|e| format!("index: {e}"))?;
let format = value.get("format").and_then(serde_json::Value::as_u64).ok_or("index: `format` is missing")?;
if format == u64::from(INDEX_FORMAT) {
return serde_json::from_value(value).map_err(|e| format!("index: {e}"));
let index: Index = serde_json::from_str(text).map_err(|e| format!("index: {e}"))?;
if index.format != INDEX_FORMAT {
return Err(format!("index format {} is not supported (this IDE reads {INDEX_FORMAT})", index.format));
}
if format < u64::from(INDEX_FORMAT) {
return Err(format!("index format {format} is not supported (this IDE reads {INDEX_FORMAT})"));
}
fn each<T: serde::de::DeserializeOwned>(value: &serde_json::Value, field: &str) -> Vec<T> {
let items = value.get(field).and_then(serde_json::Value::as_array);
items.into_iter().flatten().filter_map(|v| serde_json::from_value(v.clone()).ok()).collect()
}
Ok(Index {
format: u32::try_from(format).unwrap_or(u32::MAX),
publishers: each(&value, "publishers"),
plugins: Vec::new(),
revoked: each(&value, "revoked"),
moved: value.get("moved").and_then(serde_json::Value::as_str).map(str::to_owned),
})
}
/// Written in a format newer than this IDE reads: it lists no plugins.
pub fn is_newer(&self) -> bool {
self.format > INDEX_FORMAT
}
/// The version of each plugin to offer, and whether this IDE can load it:
/// the newest it can load, else the newest. Revoked versions are skipped;
/// plugins come in the order they are first listed.
pub fn offered(&self) -> Vec<(&IndexEntry, bool)> {
let mut ids: Vec<&str> = Vec::new();
for e in &self.plugins {
if !ids.contains(&e.id.as_str()) {
ids.push(&e.id);
}
}
ids.into_iter()
.filter_map(|id| {
let versions: Vec<&IndexEntry> =
self.plugins.iter().filter(|e| e.id == id && self.revocation(id, &e.version).is_none()).collect();
let newest = |loadable: bool| {
versions.iter().copied().filter(|e| !loadable || e.compatible()).max_by_key(|e| e.semver())
};
newest(true).map(|e| (e, true)).or_else(|| newest(false).map(|e| (e, false)))
})
.collect()
}
/// Drops the versions no IDE would pick: revoked ones, and of each plugin
/// all but the newest version per API requirement.
pub fn prune_versions(&mut self) {
let plugins = std::mem::take(&mut self.plugins);
let mut kept: Vec<IndexEntry> = Vec::new();
for e in plugins.into_iter().filter(|e| self.revocation(&e.id, &e.version).is_none()) {
match kept.iter_mut().find(|k| k.id == e.id && k.engine == e.engine) {
Some(k) if e.semver() > k.semver() => *k = e,
Some(_) => {}
None => kept.push(e),
}
}
self.plugins = kept;
Ok(index)
}
/// The revocation that applies to `id` at `version`.
@@ -473,127 +335,6 @@ impl Index {
sig.check(id, version, digest)?;
Ok(publisher.id.clone())
}
/// Whether `to` is `from`, or a key of `publisher` that `from` endorsed,
/// directly or through keys it endorsed. Endorsements by revoked keys don't
/// count: whoever stole a key could vouch for their own.
pub fn succeeds(&self, publisher: &str, from: &str, to: &str) -> bool {
let keys = self.publishers.iter().find(|p| p.id == publisher).map_or(&[][..], |p| &p.keys[..]);
let mut key = to;
// Each step moves to an earlier key; more steps than keys is a loop.
for _ in 0..=keys.len() {
if key == from {
return true;
}
let Some(e) = keys.iter().find(|k| k.key == key).and_then(|k| k.endorsement.as_ref()) else {
return false;
};
let endorser_valid = keys.iter().any(|k| k.key == e.key && !k.revoked);
if !endorser_valid || verify_ed25519(&e.key, &endorsement_message(publisher, key), &e.signature) != Ok(true)
{
return false;
}
key = &e.key;
}
false
}
/// Makes `roots` (base64) the keys `publisher` signs with, whatever this
/// index says: another key it lists stays only when a root endorsed it (a
/// rotation the roots vouched for). Revocations of the keys are kept.
pub fn pin_publisher(&mut self, publisher: &str, name: &str, roots: &[String]) {
if roots.is_empty() {
return;
}
let i = match self.publishers.iter().position(|p| p.id == publisher) {
Some(i) => i,
None => {
self.publishers.push(Publisher { id: publisher.into(), name: name.into(), keys: Vec::new() });
self.publishers.len() - 1
}
};
for root in roots {
if !self.publishers[i].keys.iter().any(|k| &k.key == root) {
self.publishers[i].keys.push(KeyEntry::new(root.clone()));
}
}
let keys = self.publishers[i].keys.clone();
let trusted = keys
.into_iter()
.filter(|k| roots.iter().any(|r| self.succeeds(publisher, r, &k.key)))
.map(|k| if roots.contains(&k.key) { KeyEntry { endorsement: None, ..k } } else { k })
.collect();
self.publishers[i].keys = trusted;
}
/// Replaces what this index lists for every publisher but `keep` with what
/// `other` lists: its publishers, their plugins and their revocations
/// (`other`'s downloads should already be absolute). `other` may only
/// speak for its own publishers, and none of them may be in `keep`.
pub fn include(&mut self, other: Index, keep: &[&str]) -> Result<(), String> {
let mut theirs: Vec<&str> = Vec::new();
for p in &other.publishers {
if !publisher_id_ok(&p.id) {
return Err(format!("invalid publisher id {:?}: use a-z, 0-9, '-' and '_'", p.id));
}
if keep.contains(&p.id.as_str()) {
return Err(format!("may not list publisher {:?}", p.id));
}
if theirs.contains(&p.id.as_str()) {
return Err(format!("publisher {:?} is listed twice", p.id));
}
theirs.push(&p.id);
}
let owned_by = |id: &str| theirs.iter().any(|p| owns(p, id));
for e in &other.plugins {
if !theirs.contains(&e.publisher.as_str()) || !owns(&e.publisher, &e.id) {
return Err(format!("{}: not a plugin of publisher {:?}", e.id, e.publisher));
}
if semver::VersionReq::parse(&e.engine).is_err() {
return Err(format!("{} {}: `engine` must be the plugin API it needs", e.id, e.version));
}
}
if let Some(r) = other.revoked.iter().find(|r| !owned_by(&r.id)) {
return Err(format!("{}: revoked by an index that doesn't publish it", r.id));
}
let kept = |id: &str| keep.iter().any(|p| owns(p, id));
self.publishers.retain(|p| keep.contains(&p.id.as_str()));
self.plugins.retain(|e| keep.contains(&e.publisher.as_str()));
self.revoked.retain(|r| kept(&r.id));
self.publishers.extend(other.publishers);
self.plugins.extend(other.plugins);
self.revoked.extend(other.revoked);
Ok(())
}
}
/// Whether a package signed `new` (`None`: unsigned) may replace an installed
/// one signed `installed` without asking the user: signed with the same key,
/// or with one the installed key endorsed (per `index`). An unsigned installed
/// copy pins nothing.
pub fn signer_continues(
index: Option<&Index>,
installed: Option<&PackageSignature>,
new: Option<&PackageSignature>,
) -> bool {
match (installed, new) {
(None, _) => true,
(Some(_), None) => false,
(Some(old), Some(new)) => {
old.publisher == new.publisher
&& (old.key == new.key || index.is_some_and(|i| i.succeeds(&new.publisher, &old.key, &new.key)))
}
}
}
/// Whether the plugin `id` is in `publisher`'s namespace.
fn owns(publisher: &str, id: &str) -> bool {
id.strip_prefix(publisher).is_some_and(|rest| rest.starts_with('.'))
}
/// A publisher id is a single segment of a plugin id.
pub fn publisher_id_ok(id: &str) -> bool {
!id.contains('.') && crate::validate_plugin_id(id).is_ok()
}
#[cfg(test)]
@@ -613,7 +354,7 @@ mod tests {
publishers: vec![Publisher {
id: "acme".into(),
name: "ACME".into(),
keys: vec![KeyEntry::new(key.public_base64())],
keys: vec![KeyEntry { key: key.public_base64(), revoked: false }],
}],
plugins: Vec::new(),
revoked: vec![Revocation {
@@ -621,7 +362,6 @@ mod tests {
versions: vec!["1.0.0".into()],
reason: "malware".into(),
}],
moved: None,
}
}
@@ -659,138 +399,6 @@ mod tests {
assert_eq!(PublisherKey::from_base64(&key.seed_base64()).unwrap().public_base64(), key.public_base64());
}
fn entry(id: &str, publisher: &str) -> IndexEntry {
IndexEntry {
id: id.into(),
name: id.into(),
version: "1.0.0".into(),
description: String::new(),
authors: Vec::new(),
publisher: publisher.into(),
download: format!("https://example.com/{id}.oiplugin"),
digest: "d".into(),
capabilities: Vec::new(),
engine: "^1.0".into(),
categories: Vec::new(),
}
}
fn version(id: &str, version: &str, engine: &str) -> IndexEntry {
IndexEntry { version: version.into(), engine: engine.into(), ..entry(id, "acme") }
}
#[test]
fn the_ide_is_offered_the_newest_version_it_can_load() {
let host = crate::ENGINE_VERSION;
let newer = semver::Version::parse(host).unwrap().major + 1;
let mut index = Index {
format: INDEX_FORMAT,
plugins: vec![
version("acme.a", "1.0.0", &format!("^{host}")),
version("acme.a", "1.2.0", &format!("^{host}")),
version("acme.a", "2.0.0", &format!("^{newer}.0")),
version("acme.b", "0.1.0", &format!("^{newer}.0")),
version("acme.b", "0.2.0", &format!("^{newer}.1")),
version("acme.c", "1.0.0", ""),
version("acme.c", "1.1.0", ""),
],
revoked: vec![Revocation { id: "acme.c".into(), versions: vec!["1.1.0".into()], reason: "r".into() }],
..Index::default()
};
let offered = |index: &Index| {
index.offered().into_iter().map(|(e, ok)| (e.id.clone(), e.version.clone(), ok)).collect::<Vec<_>>()
};
let expected = [
("acme.a".to_owned(), "1.2.0".to_owned(), true),
("acme.b".to_owned(), "0.2.0".to_owned(), false),
// No `engine`: listed, but no IDE installs it.
("acme.c".to_owned(), "1.0.0".to_owned(), false),
];
assert_eq!(offered(&index), expected);
// Pruning keeps the newest version per requirement, which changes nothing offered.
index.prune_versions();
let left: Vec<_> = index.plugins.iter().map(|e| format!("{} {}", e.id, e.version)).collect();
assert_eq!(left, ["acme.a 1.2.0", "acme.a 2.0.0", "acme.b 0.1.0", "acme.b 0.2.0", "acme.c 1.0.0"]);
assert_eq!(offered(&index), expected);
let built_for = semver::Version::new(1, 3, 0);
assert_eq!(engine_requirement("^1.0", Some(&built_for)), "^1.0, ^1.3");
assert_eq!(engine_requirement("^1.0", None), "^1.0");
assert!(semver::VersionReq::parse(&engine_requirement("^1.0", Some(&built_for))).is_ok());
}
fn publisher(id: &str) -> Publisher {
Publisher { id: id.into(), name: id.into(), keys: vec![KeyEntry::new(format!("{id}-key"))] }
}
fn revocation(id: &str) -> Revocation {
Revocation { id: id.into(), versions: Vec::new(), reason: "r".into() }
}
#[test]
fn including_an_index_replaces_everything_but_the_kept_publishers() {
let mut official = Index {
format: INDEX_FORMAT,
publishers: vec![publisher("openidea"), publisher("gone")],
plugins: vec![entry("openidea.a", "openidea"), entry("gone.x", "gone")],
revoked: vec![revocation("openidea.old"), revocation("gone.x")],
moved: None,
};
let community = Index {
format: INDEX_FORMAT,
publishers: vec![publisher("alice")],
plugins: vec![entry("alice.arthas", "alice")],
revoked: vec![revocation("alice.bad")],
moved: None,
};
official.include(community.clone(), &["openidea"]).unwrap();
let ids = |v: &[IndexEntry]| v.iter().map(|e| e.id.clone()).collect::<Vec<_>>();
assert_eq!(official.publishers.iter().map(|p| p.id.as_str()).collect::<Vec<_>>(), ["openidea", "alice"]);
assert_eq!(ids(&official.plugins), ["openidea.a", "alice.arthas"]);
assert_eq!(official.revoked.iter().map(|r| r.id.as_str()).collect::<Vec<_>>(), ["openidea.old", "alice.bad"]);
// Including again is idempotent.
let once = official.clone();
official.include(community.clone(), &["openidea"]).unwrap();
assert_eq!(official, once);
}
#[test]
fn an_included_index_speaks_only_for_its_own_publishers() {
let base = Index { format: INDEX_FORMAT, publishers: vec![publisher("openidea")], ..Index::default() };
let with = |publishers: Vec<Publisher>, plugins: Vec<IndexEntry>, revoked: Vec<Revocation>| Index {
format: INDEX_FORMAT,
publishers,
plugins,
revoked,
moved: None,
};
let cases = [
(with(vec![publisher("openidea")], vec![], vec![]), "may not list"),
(with(vec![publisher("alice"), publisher("alice")], vec![], vec![]), "listed twice"),
(with(vec![publisher("openidea.x")], vec![], vec![]), "invalid publisher"),
(with(vec![publisher("alice")], vec![entry("openidea.a", "alice")], vec![]), "not a plugin of"),
(with(vec![publisher("alice")], vec![entry("alice.a", "bob")], vec![]), "not a plugin of"),
(with(vec![publisher("alice")], vec![entry("alicex.a", "alice")], vec![]), "not a plugin of"),
(with(vec![publisher("alice")], vec![], vec![revocation("openidea.a")]), "doesn't publish"),
(
with(
vec![publisher("alice")],
vec![IndexEntry { engine: String::new(), ..entry("alice.a", "alice") }],
vec![],
),
"`engine`",
),
];
for (other, error) in cases {
let mut index = base.clone();
let err = index.include(other, &["openidea"]).unwrap_err();
assert!(err.contains(error), "{err}");
assert_eq!(index, base);
}
}
#[test]
fn unpacking_refuses_paths_leaving_the_package() {
let tmp = tempfile::tempdir().unwrap();
@@ -805,7 +413,7 @@ mod tests {
let err = unpack(&out, &tmp.path().join("dest")).unwrap_err();
assert!(err.contains("invalid path"), "{err}");
assert!(!tmp.path().join("outside.txt").exists());
assert!(Index::parse(r#"{"format": 2}"#).unwrap().is_newer());
assert!(Index::parse(r#"{"format": 2}"#).is_err());
}
fn zip_of(path: &Path, entries: &[(&str, &str)]) {
@@ -854,102 +462,4 @@ mod tests {
assert!(err.contains("is not an OpenIDEA plugin"), "{name}: {err}");
}
}
#[test]
fn a_newer_index_still_revokes_and_moves() {
let newer = format!(
r#"{{"format": {}, "plugins": [{{"anything": "new"}}], "shape": "changed",
"publishers": [{{"id": "acme", "name": "ACME", "keys": [{{"key": "k1", "revoked": true}}]}}, {{"id": 5}}],
"revoked": [{{"id": "acme.bad", "reason": "malware"}}, {{"oops": 1}}],
"moved": "https://example.com/next.json"}}"#,
INDEX_FORMAT + 1
);
let index = Index::parse(&newer).unwrap();
assert!(index.is_newer());
assert!(index.plugins.is_empty() && index.offered().is_empty());
assert_eq!(index.revocation("acme.bad", "2.0.0").unwrap().reason, "malware");
assert!(index.key_revoked("k1"));
assert_eq!(index.moved.as_deref(), Some("https://example.com/next.json"));
// Fields this IDE doesn't know are ignored in its own format.
let current = format!(r#"{{"format": {INDEX_FORMAT}, "later": true, "plugins": []}}"#);
assert!(!Index::parse(&current).unwrap().is_newer());
assert!(Index::parse(r#"{"format": 0}"#).unwrap_err().contains("not supported"));
assert!(Index::parse(r#"{"plugins": []}"#).is_err());
}
fn keys<'a>(index: &'a mut Index, publisher: &str) -> &'a mut Vec<KeyEntry> {
&mut index.publishers.iter_mut().find(|p| p.id == publisher).unwrap().keys
}
#[test]
fn rotated_keys_continue_through_endorsements() {
let (k1, k2, k3) =
(PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([2; 32]), PublisherKey::from_seed([3; 32]));
let mut index = index(&k1);
let endorsed = |by: &PublisherKey, key: &PublisherKey| KeyEntry {
endorsement: Some(by.endorse("acme", &key.public_base64())),
..KeyEntry::new(key.public_base64())
};
keys(&mut index, "acme").push(endorsed(&k1, &k2));
keys(&mut index, "acme").push(endorsed(&k2, &k3));
let (p1, p2, p3) = (k1.public_base64(), k2.public_base64(), k3.public_base64());
assert!(index.succeeds("acme", &p1, &p1));
assert!(index.succeeds("acme", &p1, &p3), "through k2");
assert!(!index.succeeds("acme", &p3, &p1), "not backwards");
assert!(!index.succeeds("other", &p1, &p2));
// An endorsement for another publisher, or by a revoked key, doesn't count.
let mut foreign = index.clone();
keys(&mut foreign, "acme")[1].endorsement = Some(k1.endorse("evil", &p2));
assert!(!foreign.succeeds("acme", &p1, &p2));
let mut stolen = index.clone();
keys(&mut stolen, "acme")[0].revoked = true;
assert!(!stolen.succeeds("acme", &p1, &p2));
let sig = |key: &PublisherKey| key.sign("acme", "acme.tools", "1.0.0", "d");
assert!(signer_continues(Some(&index), Some(&sig(&k1)), Some(&sig(&k3))));
assert!(signer_continues(None, Some(&sig(&k1)), Some(&sig(&k1))));
assert!(!signer_continues(None, Some(&sig(&k1)), Some(&sig(&k2))), "no index to vouch");
let stranger = PublisherKey::from_seed([4; 32]);
assert!(!signer_continues(Some(&index), Some(&sig(&k1)), Some(&sig(&stranger))));
assert!(!signer_continues(Some(&index), Some(&sig(&k1)), None), "signed, then unsigned");
assert!(signer_continues(Some(&index), None, Some(&sig(&stranger))), "unsigned pins nothing");
}
#[test]
fn pinned_publishers_keep_their_built_in_keys() {
let (root, next, intruder) =
(PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([2; 32]), PublisherKey::from_seed([5; 32]));
let mut index = index(&intruder);
index.publishers[0].id = "openidea".into();
keys(&mut index, "openidea").push(KeyEntry {
endorsement: Some(root.endorse("openidea", &next.public_base64())),
..KeyEntry::new(next.public_base64())
});
let roots = [root.public_base64()];
let listed = |index: &Index| -> Vec<(String, bool)> {
let mut index = index.clone();
index.pin_publisher("openidea", "OpenIDEA", &roots);
keys(&mut index, "openidea").iter().map(|k| (k.key.clone(), k.revoked)).collect()
};
// The intruder's key goes, the successor the root endorsed stays, the root is added.
assert_eq!(listed(&index), [(next.public_base64(), false), (root.public_base64(), false)]);
let mut pinned = index.clone();
pinned.pin_publisher("openidea", "OpenIDEA", &roots);
let digest = "d";
let signed = |k: &PublisherKey| k.sign("openidea", "openidea.tools", "1.0.0", digest);
assert!(pinned.verify(&signed(&intruder), "openidea.tools", "1.0.0", digest).is_err());
assert!(pinned.verify(&signed(&next), "openidea.tools", "1.0.0", digest).is_ok());
// A revoked root stays revoked, and what it endorsed is no longer trusted.
keys(&mut index, "openidea").push(KeyEntry { revoked: true, ..KeyEntry::new(root.public_base64()) });
assert_eq!(listed(&index), [(root.public_base64(), true)]);
// A publisher missing from the index is added with its roots.
let mut empty = Index { format: INDEX_FORMAT, ..Index::default() };
empty.pin_publisher("openidea", "OpenIDEA", &[root.public_base64()]);
assert_eq!(empty.verify(&signed(&root), "openidea.tools", "1.0.0", digest).unwrap(), "openidea");
// No roots: the index decides.
let mut open = index.clone();
open.pin_publisher("openidea", "OpenIDEA", &[]);
assert_eq!(open, index);
}
}
+20 -220
View File
@@ -21,18 +21,6 @@ pub struct FontFamily {
pub monospaced: bool,
}
/// PANOSE bProportion value 9 means "Monospaced"; DirectWrite's
/// `IsMonospacedFont` alone misses fonts that only carry this classification.
const PANOSE_MONOSPACED: u8 = 9;
/// One face counts as monospaced when the font system flags it fixed pitch, its
/// PANOSE proportion is "Monospaced", or its glyph advances are uniform
/// (fonts with wrong/missing metadata, e.g. bitmap-style TTFs).
#[cfg_attr(not(windows), allow(dead_code))]
fn face_is_mono(is_monospaced_font: bool, panose_proportion: u8, advances_equal: bool) -> bool {
is_monospaced_font || panose_proportion == PANOSE_MONOSPACED || advances_equal
}
type Cache = Mutex<Option<Arc<[FontFamily]>>>;
/// The installed families, sorted case-insensitively, read on first use and
@@ -207,9 +195,7 @@ mod imp {
// SAFETY: CoreText descriptors carry documented CFString, CFURL and trait dictionary values.
unsafe {
let collection = CTFontCollection::from_available_fonts(None);
let Some(descriptors) = collection.matching_font_descriptors() else {
return Vec::new();
};
let Some(descriptors) = collection.matching_font_descriptors() else { return Vec::new() };
let mut seen = std::collections::BTreeSet::new();
descriptors
.cast_unchecked::<CTFontDescriptor>()
@@ -285,16 +271,14 @@ mod imp {
#[cfg(windows)]
mod imp {
use std::os::windows::ffi::OsStrExt;
use windows::Win32::Graphics::DirectWrite::IDWriteFontFace1;
use windows::Win32::Graphics::DirectWrite::{
DWRITE_FACTORY_TYPE_SHARED, DWriteCreateFactory, IDWriteFactory, IDWriteFactory3, IDWriteFactory5, IDWriteFont,
IDWriteFont1, IDWriteFontCollection, IDWriteFontFamily, IDWriteLocalizedStrings,
DWRITE_FACTORY_TYPE_SHARED, DWRITE_FONT_STRETCH_NORMAL, DWRITE_FONT_STYLE_NORMAL, DWRITE_FONT_WEIGHT_NORMAL,
DWriteCreateFactory, IDWriteFactory, IDWriteFont1, IDWriteLocalizedStrings,
};
use windows::core::{BOOL, Interface, PCWSTR, w};
use windows::core::{BOOL, Interface, w};
/// The installed families, named in English when the family has an
/// English name (what CSS matches), else its first name.
/// The system font collection's families, named in English when the
/// family has an English name (what CSS matches), else its first name.
pub(crate) fn list(refresh: bool) -> Vec<(String, bool)> {
// SAFETY: plain DirectWrite COM calls on interfaces we own.
unsafe { read(refresh) }.unwrap_or_default()
@@ -302,85 +286,11 @@ mod imp {
unsafe fn read(refresh: bool) -> windows::core::Result<Vec<(String, bool)>> {
let factory: IDWriteFactory = unsafe { DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED) }?;
let mut faces = Vec::new();
// Windows 10+: the newer collection covers registered per-user fonts.
// Exclude downloadable fonts that are not installed on this machine.
// Bitmap .FON fonts (e.g. the stock raster Fixedsys) are invisible to
// DirectWrite either way: it only reads TrueType/OpenType files.
let mut collection = None;
if let Ok(factory3) = factory.cast::<IDWriteFactory3>() {
let mut local = None;
if unsafe { factory3.GetSystemFontCollection(false, &mut local, refresh) }.is_ok() {
collection = local.map(Into::into);
}
}
if collection.is_none() {
unsafe { factory.GetSystemFontCollection(&mut collection, refresh) }?;
}
if let Some(collection) = collection {
faces.extend(collect(&collection));
}
// DirectWrite enumerates via the font registry, so font files copied
// into the per-user directory without registering (no HKCU entry)
// are invisible to the calls above; scan the directory directly.
faces.extend(unsafe { user_dir_faces(&factory) });
Ok(faces)
}
/// Families from `%LOCALAPPDATA%\Microsoft\Windows\Fonts` files, registered
/// or not, via a font set built from the files on disk.
unsafe fn user_dir_faces(factory: &IDWriteFactory) -> Vec<(String, bool)> {
let Some(local_app_data) = std::env::var_os("LOCALAPPDATA") else {
return Vec::new();
unsafe { factory.GetSystemFontCollection(&mut collection, refresh) }?;
let Some(collection) = collection else {
return Ok(Vec::new());
};
let dir = std::path::Path::new(&local_app_data).join(r"Microsoft\Windows\Fonts");
let Ok(entries) = std::fs::read_dir(&dir) else {
return Vec::new();
};
let files: Vec<_> = entries
.filter_map(|e| e.ok())
.map(|e| e.path())
.filter(|p| {
matches!(
p.extension().and_then(|e| e.to_str()).map(|e| e.to_ascii_lowercase()).as_deref(),
Some("ttf" | "otf" | "ttc" | "otc")
)
})
.collect();
unsafe { file_faces(factory, &files) }
}
unsafe fn file_faces(factory: &IDWriteFactory, files: &[std::path::PathBuf]) -> Vec<(String, bool)> {
if files.is_empty() {
return Vec::new();
}
let Ok(factory3) = factory.cast::<IDWriteFactory3>() else {
return Vec::new();
};
let Ok(factory5) = factory.cast::<IDWriteFactory5>() else {
return Vec::new();
};
let Ok(builder) = (unsafe { factory5.CreateFontSetBuilder() }) else {
return Vec::new();
};
for path in files {
let wide: Vec<u16> = path.as_os_str().encode_wide().chain(Some(0)).collect();
// SAFETY: wide is a valid NUL-terminated string for the calls.
let Ok(file) = (unsafe { factory.CreateFontFileReference(PCWSTR(wide.as_ptr()), None) }) else {
continue;
};
let _ = unsafe { builder.AddFontFile(&file) };
}
let Ok(set) = (unsafe { builder.CreateFontSet() }) else {
return Vec::new();
};
let Ok(collection) = (unsafe { factory3.CreateFontCollectionFromFontSet(&set) }) else {
return Vec::new();
};
collect(&collection)
}
fn collect(collection: &IDWriteFontCollection) -> Vec<(String, bool)> {
let mut out = Vec::new();
for i in 0..unsafe { collection.GetFontFamilyCount() } {
let Ok(family) = (unsafe { collection.GetFontFamily(i) }) else {
@@ -389,120 +299,19 @@ mod imp {
let Some(name) = unsafe { family.GetFamilyNames() }.ok().and_then(|n| unsafe { english(&n) }) else {
continue;
};
out.push((name, unsafe { family_is_mono(&family) }));
}
out
}
/// A family is monospaced when any face reports fixed pitch, PANOSE
/// proportion 9, or uniform glyph advances; probing every face matches
/// `normalize`'s any-face merge (macOS and Linux already flag every face,
/// not just the regular one).
unsafe fn family_is_mono(family: &IDWriteFontFamily) -> bool {
(0..unsafe { family.GetFontCount() }).any(|i| {
let Ok(font) = (unsafe { family.GetFont(i) }) else {
return false;
};
if let Ok(font1) = font.cast::<IDWriteFont1>() {
let is_monospaced = unsafe { font1.IsMonospacedFont() }.as_bool();
// The proportion byte is defined for text/display fonts only.
let panose = unsafe { font1.GetPanose().values };
let panose_proportion = if panose[0] == 2 { panose[3] } else { 0 };
if super::face_is_mono(is_monospaced, panose_proportion, false) {
return true;
}
}
unsafe { advances_equal(&font) }
})
}
/// Some monospaced fonts carry no fixed-pitch metadata at all (bitmap-style
/// TTFs like Fixedsys Excelsior): compare the advance widths of 'i' and
/// 'W'. Dual-width CJK monospace fonts pass too (both half-width).
unsafe fn advances_equal(font: &IDWriteFont) -> bool {
let Ok(face) = (unsafe { font.CreateFontFace() }) else {
return false;
};
let Ok(face1) = face.cast::<IDWriteFontFace1>() else {
return false;
};
let codepoints = ['i' as u32, 'W' as u32];
let mut indices = [0u16; 2];
if unsafe { face.GetGlyphIndices(codepoints.as_ptr(), 2, indices.as_mut_ptr()) }.is_err()
|| indices.contains(&0)
{
return false;
}
let mut advances = [0i32; 2];
unsafe { face1.GetDesignGlyphAdvances(2, indices.as_ptr(), advances.as_mut_ptr(), false) }.is_ok()
&& advances[0] > 0
&& advances[0] == advances[1]
}
#[test]
fn glyph_probe_distinguishes_monospace_proportional_and_symbol_fonts() {
unsafe {
let factory: IDWriteFactory = DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED).unwrap();
let mut collection = None;
factory.GetSystemFontCollection(&mut collection, false).unwrap();
let collection = collection.unwrap();
for (name, expected) in [
("Consolas", true),
("Arial", false),
("Times New Roman", false),
("Wingdings", false),
("Webdings", false),
("Segoe MDL2 Assets", false),
] {
let wide: Vec<_> = name.encode_utf16().chain(Some(0)).collect();
let mut index = 0;
let mut exists = BOOL(0);
collection.FindFamilyName(PCWSTR(wide.as_ptr()), &mut index, &mut exists).unwrap();
assert!(exists.as_bool(), "{name} ships with Windows");
let family = collection.GetFontFamily(index).unwrap();
for i in 0..family.GetFontCount() {
let font = family.GetFont(i).unwrap();
assert_eq!(advances_equal(&font), expected, "{name} face {i}");
}
}
}
}
#[test]
fn font_files_include_every_family_in_a_collection() {
let path = std::path::PathBuf::from(std::env::var_os("WINDIR").unwrap()).join(r"Fonts\cambria.ttc");
unsafe {
let factory: IDWriteFactory = DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED).unwrap();
let faces = file_faces(&factory, &[path]);
for name in ["Cambria", "Cambria Math"] {
assert!(faces.iter().any(|(family, _)| family == name), "missing {name}: {faces:?}");
}
}
}
#[test]
fn excludes_families_available_only_as_downloads() {
use windows::Win32::Graphics::DirectWrite::{DWRITE_E_REMOTEFONT, DWRITE_LOCALITY_LOCAL, IDWriteFont3};
unsafe {
let factory: IDWriteFactory = DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED).unwrap();
let factory3: IDWriteFactory3 = factory.cast().unwrap();
let set = factory3.GetSystemFontSet().unwrap();
let collection = factory3.CreateFontCollectionFromFontSet(&set).unwrap();
let listed = super::normalize(read(false).unwrap());
let user = super::normalize(user_dir_faces(&factory));
for i in 0..collection.GetFontFamilyCount() {
let family = collection.GetFontFamily(i).unwrap();
let name = english(&family.GetFamilyNames().unwrap()).unwrap();
let remote_only = (0..family.GetFontCount()).all(|j| match family.GetFont(j) {
Ok(font) => font.cast::<IDWriteFont3>().unwrap().GetLocality() != DWRITE_LOCALITY_LOCAL,
Err(error) => error.code() == DWRITE_E_REMOTEFONT,
});
if remote_only && !user.iter().any(|f| f.name == name) {
assert!(!listed.iter().any(|f| f.name == name), "{name} is not installed locally");
}
let mono = unsafe {
family.GetFirstMatchingFont(
DWRITE_FONT_WEIGHT_NORMAL,
DWRITE_FONT_STRETCH_NORMAL,
DWRITE_FONT_STYLE_NORMAL,
)
}
.ok()
.and_then(|f| f.cast::<IDWriteFont1>().ok())
.is_some_and(|f| unsafe { f.IsMonospacedFont() }.as_bool());
out.push((name, mono));
}
Ok(out)
}
unsafe fn english(names: &IDWriteLocalizedStrings) -> Option<String> {
@@ -667,15 +476,6 @@ mod tests {
);
}
#[test]
fn panose_monospaced_proportion_counts_as_mono() {
assert!(face_is_mono(true, 0, false));
assert!(face_is_mono(false, PANOSE_MONOSPACED, false));
assert!(face_is_mono(false, 0, true));
assert!(!face_is_mono(false, 0, false));
assert!(!face_is_mono(false, 3, false));
}
#[test]
fn fc_list_spacing_marks_monospace() {
let out = "DejaVu Sans Mono\t100\nDejaVu Sans\t\nNoto Sans Mono CJK SC\t90\nTerminus\t110\nCantarell\nFira\\-Code\tmono\n\t100\n";
-145
View File
@@ -1,145 +0,0 @@
//! The signed-in GitHub account, independent of any project: its repositories for
//! "Get from Version Control" › GitHub.
use crate::{GhAvailability, QUICK, classify_failure, command, gh_executable, process};
use serde::{Deserialize, Serialize};
use specta::Type;
/// A repository of the signed-in account (owned, collaborator, or organization member).
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct GhAccountRepo {
/// `owner/name`.
pub name_with_owner: String,
pub description: Option<String>,
pub private: bool,
pub fork: bool,
pub archived: bool,
/// What `git clone` takes: HTTPS, or SSH when gh's `git_protocol` is `ssh`.
pub clone_url: String,
}
/// The account's repositories, or why they can't be listed.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase", tag = "kind")]
pub enum GhAccountRepos {
Ready {
login: String,
repos: Vec<GhAccountRepo>,
},
/// No `gh` executable was found.
NotInstalled,
/// `gh auth login` has not been run (or the token expired).
NotLoggedIn {
message: String,
},
Error {
message: String,
},
}
#[derive(Deserialize)]
struct RestRepo {
full_name: String,
description: Option<String>,
#[serde(default)]
private: bool,
#[serde(default)]
fork: bool,
#[serde(default)]
archived: bool,
clone_url: String,
ssh_url: String,
}
/// `gh api --paginate --slurp user/repos` output: one array per page.
pub fn parse_account_repos(slurped: &str, ssh: bool) -> Result<Vec<GhAccountRepo>, serde_json::Error> {
let pages: Vec<Vec<RestRepo>> = serde_json::from_str(slurped)?;
Ok(pages
.into_iter()
.flatten()
.map(|r| GhAccountRepo {
name_with_owner: r.full_name,
description: r.description.filter(|d| !d.trim().is_empty()),
private: r.private,
fork: r.fork,
archived: r.archived,
clone_url: if ssh { r.ssh_url } else { r.clone_url },
})
.collect())
}
/// Lists the repositories of the account gh is signed in to, most recently pushed first.
pub fn account_repositories() -> GhAccountRepos {
if gh_executable().is_none() {
return GhAccountRepos::NotInstalled;
}
let dir = std::env::temp_dir();
let gh = |args: &[&str]| -> Result<String, GhAccountRepos> {
let out = command(&dir, args)
.and_then(|cmd| process::run(cmd, QUICK, None))
.map_err(|e| GhAccountRepos::Error { message: e.message })?;
if out.success {
return Ok(out.stdout);
}
let message = if out.stderr.trim().is_empty() { out.stdout } else { out.stderr };
Err(match classify_failure(&message) {
GhAvailability::NotLoggedIn { message } => GhAccountRepos::NotLoggedIn { message },
_ => GhAccountRepos::Error { message: message.trim().to_owned() },
})
};
let result = (|| {
let login = gh(&["api", "user", "--jq", ".login"])?.trim().to_owned();
let ssh = gh(&["config", "get", "git_protocol", "--host", "github.com"]).is_ok_and(|p| p.trim() == "ssh");
let pages = gh(&[
"api",
"user/repos?per_page=100&sort=pushed&affiliation=owner,collaborator,organization_member",
"--paginate",
"--slurp",
])?;
let repos = parse_account_repos(&pages, ssh)
.map_err(|e| GhAccountRepos::Error { message: format!("unexpected gh output: {e}") })?;
Ok(GhAccountRepos::Ready { login, repos })
})();
result.unwrap_or_else(|e| e)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn reads_every_page_and_picks_the_configured_protocol() {
let pages = r#"[
[{"full_name":"me/app","description":"An app","private":true,"fork":false,"archived":false,
"clone_url":"https://github.com/me/app.git","ssh_url":"[email protected]:me/app.git"}],
[{"full_name":"org/lib","description":"","fork":true,
"clone_url":"https://github.com/org/lib.git","ssh_url":"[email protected]:org/lib.git"}]
]"#;
let https = parse_account_repos(pages, false).unwrap();
assert_eq!(
https,
vec![
GhAccountRepo {
name_with_owner: "me/app".into(),
description: Some("An app".into()),
private: true,
fork: false,
archived: false,
clone_url: "https://github.com/me/app.git".into(),
},
GhAccountRepo {
name_with_owner: "org/lib".into(),
description: None,
private: false,
fork: true,
archived: false,
clone_url: "https://github.com/org/lib.git".into(),
},
]
);
let ssh = parse_account_repos(pages, true).unwrap();
assert_eq!(ssh[1].clone_url, "[email protected]:org/lib.git");
assert!(parse_account_repos("[]", false).unwrap().is_empty());
}
}
-2
View File
@@ -3,11 +3,9 @@
//!
//! Every function blocks; callers run them off the UI / command threads.
mod account;
mod model;
mod process;
pub use account::{GhAccountRepo, GhAccountRepos, account_repositories, parse_account_repos};
pub use model::*;
use ide_core::{CancellationToken, IdeError, IdeResult};
+1 -5
View File
@@ -14,7 +14,6 @@ encoding_rs.workspace = true
ide-core.workspace = true
ide-text.workspace = true
ide-vfs.workspace = true
regex-syntax.workspace = true
ide-workspace.workspace = true
ide-language.workspace = true
ide-index.workspace = true
@@ -45,8 +44,6 @@ ide-notebook.workspace = true
ide-extension.workspace = true
ide-plugin.workspace = true
ide-tools = { path = "../ide-tools" }
ide-docker = { path = "../ide-docker" }
semver = "1"
serde.workspace = true
serde_json.workspace = true
toml.workspace = true
@@ -60,13 +57,12 @@ dunce.workspace = true
dirs.workspace = true
uuid.workspace = true
tempfile.workspace = true
zip.workspace = true
quick-xml = "0.42"
[dev-dependencies]
ide-ssh = { workspace = true, features = ["testing"] }
tokio = { workspace = true, features = ["rt-multi-thread", "macros", "io-util", "test-util"] }
criterion = { version = "0.5", default-features = false }
zip.workspace = true
[target.'cfg(unix)'.dependencies]
rustix = { version = "1.1", features = ["fs", "process"] }
+47 -347
View File
@@ -3,15 +3,13 @@
//! ```text
//! openidea-plugin new <dir> [--template command|theme|icon-theme|language] [--id ID] [--name NAME] [--sdk PATH]
//! openidea-plugin build [dir] [--debug] # cargo → dist/plugin.wasm, then `check`
//! openidea-plugin check [dir|package] # validate the package as the IDE would install it
//! openidea-plugin check [dir] # validate the package as the IDE would install it
//! openidea-plugin link [dir] # load the folder in the IDE without installing it
//! openidea-plugin unlink [dir]
//! openidea-plugin links # list linked folders
//! openidea-plugin keygen <key-file> # a publisher's Ed25519 key
//! openidea-plugin endorse <new-public-key> --key OLD --publisher ID # vouch for a new key
//! openidea-plugin pack [dir] [--key F --publisher ID] [--out FILE] # <id>-<version>.oiplugin
//! openidea-plugin index <dir> [--publisher ID --name NAME --key F [--previous-key F]] # marketplace index.json
//! openidea-plugin merge <index> <other-index> --keep ID # list another index's publishers
//! openidea-plugin index <dir> --publisher ID --name NAME --key F # marketplace index.json
//! ```
//!
//! `link` writes `links.json` in the IDE's plugins directory
@@ -22,7 +20,7 @@
mod templates;
use ide_extension::{Manifest, validate_plugin_id};
use ide_host::plugins::{PluginHost, read_market_url, resolve_market_href};
use ide_host::plugins::PluginHost;
use ide_plugin::{Limits, PluginEngine};
use std::path::{Path, PathBuf};
use std::process::{Command, ExitCode};
@@ -35,27 +33,15 @@ commands:
new <dir> [--template T] [--id ID] [--name NAME] [--sdk PATH]
create a plugin; T = command (default), theme, icon-theme, language
build [dir] [--debug] compile a Rust plugin to dist/plugin.wasm and check it
check [dir|package] validate the package (folder or .oiplugin) as the IDE would install it
check [dir] validate the package as the IDE would install it
link [dir] load the folder in the IDE without installing it
unlink [dir] stop loading a linked folder
links list linked folders
keygen <key-file> create a publisher signing key (keep the file secret)
endorse <new-public-key> --key F --publisher ID
print an earlier key's endorsement of a new one (JSON), for an
index the publisher doesn't write with `index --previous-key`
pack [dir] [--key F --publisher ID] [--out FILE]
write <id>-<version>.oiplugin, signed with the key
index <dir> [--publisher ID --name NAME --key F [--previous-key F]] [--out FILE]
[--download-base URL]
write the marketplace index.json for the packages in dir: with a
key, register it and list that publisher's plugins (earlier
versions stay for older IDEs); without, list only these packages,
each checked against the index's publishers. A new key replacing
an earlier one is endorsed with --previous-key, so installed
plugins update to it without asking
merge <index> <other-index> --keep ID [--out FILE]
replace every publisher but ID in index with those of other-index
(a path or URL), their plugins and revocations
index <dir> --publisher ID --name NAME --key F [--out FILE]
write the marketplace index.json for the packages in dir
environment:
OPENIDEA_PLUGINS_DIR the IDE's plugins directory (default: the app data directory)";
@@ -65,15 +51,13 @@ fn main() -> ExitCode {
let result = match args.first().map(String::as_str) {
Some("new") => cmd_new(&args[1..]),
Some("build") => cmd_build(&args[1..]),
Some("check") => dir_arg(&args[1..]).and_then(|d| check_path(&d)),
Some("check") => dir_arg(&args[1..]).and_then(|d| check(&d)),
Some("link") => dir_arg(&args[1..]).and_then(|d| link(&d, true)),
Some("unlink") => dir_arg(&args[1..]).and_then(|d| link(&d, false)),
Some("links") => list_links(),
Some("keygen") => cmd_keygen(&args[1..]),
Some("endorse") => cmd_endorse(&args[1..]),
Some("pack") => cmd_pack(&args[1..]),
Some("index") => cmd_index(&args[1..]),
Some("merge") => cmd_merge(&args[1..]),
Some("--version" | "-V") => {
println!("openidea-plugin {} (plugin API {})", env!("CARGO_PKG_VERSION"), ide_extension::ENGINE_VERSION);
Ok(())
@@ -339,21 +323,10 @@ fn ensure_wasm_target() {
/// Validates `dir` exactly as Install from Disk would, and checks the
/// component imports only the plugin API.
/// [`check`] of a folder, or of a package file unpacked as installing would.
fn check_path(path: &Path) -> Result<(), String> {
if !path.is_file() {
return check(path);
}
let tmp = scratch_dir("unpack")?;
let dir = tmp.path().join("package");
package::unpack(path, &dir).and_then(|_| check(&dir))
}
fn check(dir: &Path) -> Result<(), String> {
let dir = canonical(dir)?;
let tmp = scratch_dir("check")?;
let scratch = tmp.path().join("host");
(|| {
let scratch = std::env::temp_dir().join(format!("openidea-plugin-check-{}", std::process::id()));
let result = (|| {
let host = PluginHost::new(scratch.clone(), Arc::new(|_: Option<&str>, _| {}), Limits::default())
.map_err(|e| e.to_string())?;
let preview = host.preview_install(&dir).map_err(|e| e.to_string())?;
@@ -372,17 +345,6 @@ fn check(dir: &Path) -> Result<(), String> {
} else {
println!(" code: none (contributions only)");
}
println!(" categories: {}", manifest.categories().join(", "));
for c in manifest.unknown_categories() {
println!(" warning: unknown category {c:?} (ignored); use {}", ide_extension::CATEGORIES.join(", "));
}
if manifest.declared_categories.len() > ide_extension::MAX_CATEGORIES {
println!(
" warning: only the first {} known categories are used; {} are declared",
ide_extension::MAX_CATEGORIES,
manifest.declared_categories.len()
);
}
if !manifest.activation.is_empty() {
let events: Vec<String> = manifest.activation.iter().map(ToString::to_string).collect();
println!(" activation: {}", events.join(", "));
@@ -399,7 +361,9 @@ fn check(dir: &Path) -> Result<(), String> {
}
println!("ok");
Ok(())
})()
})();
let _ = std::fs::remove_dir_all(&scratch);
result
}
fn human_size(bytes: u64) -> String {
@@ -515,31 +479,14 @@ fn cmd_keygen(args: &[String]) -> Result<(), String> {
Ok(())
}
fn cmd_endorse(args: &[String]) -> Result<(), String> {
let a = Args::parse(args, &[], &["key", "publisher"])?;
let usage = "usage: openidea-plugin endorse <new-public-key> --key <earlier-key-file> --publisher ID";
let ([new], Some(key), Some(publisher)) = (a.positional.as_slice(), a.value("key"), a.value("publisher")) else {
return Err(usage.into());
};
if !package::publisher_id_ok(publisher) {
return Err(format!("invalid publisher id {publisher:?}"));
}
let endorsement = read_key(key)?.endorse(publisher, new.trim());
println!("{}", serde_json::to_string_pretty(&endorsement).map_err(|e| e.to_string())?);
Ok(())
}
/// A fresh scratch directory, removed when dropped.
fn scratch_dir(what: &str) -> Result<tempfile::TempDir, String> {
tempfile::Builder::new().prefix(&format!("openidea-plugin-{what}-")).tempdir().map_err(|e| e.to_string())
}
/// A scratch host to list a package's files as the IDE would.
fn with_scratch_host<T>(f: impl FnOnce(&PluginHost) -> Result<T, String>) -> Result<T, String> {
let tmp = scratch_dir("host")?;
PluginHost::new(tmp.path().join("host"), Arc::new(|_: Option<&str>, _| {}), Limits::default())
let scratch = std::env::temp_dir().join(format!("openidea-plugin-pack-{}", std::process::id()));
let result = PluginHost::new(scratch.clone(), Arc::new(|_: Option<&str>, _| {}), Limits::default())
.map_err(|e| e.to_string())
.and_then(|h| f(&h))
.and_then(|h| f(&h));
let _ = std::fs::remove_dir_all(&scratch);
result
}
fn cmd_pack(args: &[String]) -> Result<(), String> {
@@ -575,28 +522,29 @@ fn cmd_pack(args: &[String]) -> Result<(), String> {
}
fn cmd_index(args: &[String]) -> Result<(), String> {
let a = Args::parse(args, &[], &["publisher", "name", "key", "previous-key", "out", "download-base"])?;
let a = Args::parse(args, &[], &["publisher", "name", "key", "out"])?;
let dir = canonical(&a.dir()?)?;
let publisher = a.value("publisher").ok_or("--publisher is required")?;
let name = a.value("name").unwrap_or(publisher);
let key = read_key(a.value("key").ok_or("--key is required")?)?;
let out = a.value("out").map(PathBuf::from).unwrap_or_else(|| dir.join("index.json"));
let base = a.value("download-base").map(|b| b.trim_end_matches('/'));
// Revocations and other publishers of an existing index are kept.
let mut index = match std::fs::read_to_string(&out) {
Ok(text) => current_format(Index::parse(&text)?, &out.to_string_lossy())?,
Ok(text) => Index::parse(&text)?,
Err(_) => Index { format: package::INDEX_FORMAT, ..Index::default() },
};
let publisher = match (a.value("publisher"), a.value("key")) {
(Some(publisher), Some(key)) => {
let key = read_key(key)?;
let name = a.value("name").unwrap_or(publisher);
let previous = a.value("previous-key").map(read_key).transpose()?;
index.publishers =
register_key(std::mem::take(&mut index.publishers), publisher, name, &key, previous.as_ref())?;
Some(publisher)
}
(None, None) if a.has("previous-key") => return Err("--previous-key goes with --publisher and --key".into()),
(None, None) => None,
_ => return Err("--publisher and --key go together".into()),
};
// Earlier keys of the publisher stay listed (packages they signed remain valid until revoked).
let current = key.public_base64();
let earlier: Vec<KeyEntry> = index
.publishers
.iter()
.filter(|p| p.id == publisher)
.flat_map(|p| p.keys.iter().filter(|k| k.key != current).cloned())
.collect();
index.publishers.retain(|p| p.id != publisher);
let keys: Vec<KeyEntry> = std::iter::once(KeyEntry { key: current, revoked: false }).chain(earlier).collect();
index.publishers.push(package::Publisher { id: publisher.into(), name: name.into(), keys });
index.plugins.clear();
let mut packages: Vec<PathBuf> = std::fs::read_dir(&dir)
.map_err(|e| e.to_string())?
.flatten()
@@ -604,144 +552,35 @@ fn cmd_index(args: &[String]) -> Result<(), String> {
.filter(|p| p.extension().is_some_and(|e| e == package::PACKAGE_EXTENSION))
.collect();
packages.sort();
let engine = PluginEngine::new()?;
let mut added: Vec<IndexEntry> = Vec::new();
for path in packages {
let tmp = scratch_dir("index")?;
let scratch = tmp.path().join("package");
let scratch = std::env::temp_dir().join(format!("openidea-plugin-index-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&scratch);
let unpacked = package::unpack(&path, &scratch)?;
let entry = (|| {
let unpacked = package::unpack(&path, &scratch)?;
let m = Manifest::load(&scratch)?;
let files = with_scratch_host(|h| h.package_files_of(&scratch).map_err(|e| e.to_string()))?;
let digest = package::package_digest(&scratch, &files).map_err(|e| e.to_string())?;
let sig = unpacked.signature.as_ref().ok_or("not signed")?;
let signer = index.verify(sig, &m.id, &m.version, &digest)?;
if publisher.is_some_and(|p| p != signer) {
return Err(format!("signed by {signer}, not {}", publisher.unwrap_or_default()));
}
if added.iter().any(|e| (&e.id, &e.version) == (&m.id, &m.version)) {
return Err(format!("another package of {} {} is in the folder", m.id, m.version));
}
if m.legacy {
return Err("plugin.toml uses the 0.1 format; see plugins/README.md".into());
}
let built_for = match &m.wasm {
Some(wasm) => engine.check_component(&engine.compile(&scratch.join(wasm))?)?,
None => None,
};
let file = path.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
let sig = unpacked.signature.as_ref().ok_or_else(|| format!("{}: not signed", path.display()))?;
sig.check(&m.id, &m.version, &digest).map_err(|e| format!("{}: {e}", path.display()))?;
Ok::<_, String>(IndexEntry {
id: m.id.clone(),
name: m.name.clone(),
version: m.version.clone(),
description: m.description.clone(),
authors: m.authors.clone(),
publisher: signer,
download: base.map_or_else(|| file.clone(), |b| format!("{b}/{file}")),
publisher: sig.publisher.clone(),
download: path.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default(),
digest,
capabilities: m.capabilities.all().map(|c| c.to_string()).collect(),
engine: package::engine_requirement(&m.engine, built_for.as_ref()),
categories: m.categories(),
})
})();
let entry = entry.map_err(|e| format!("{}: {e}", path.display()))?;
println!(" {} {} (plugin API {})", entry.id, entry.version, entry.engine);
added.push(entry);
let _ = std::fs::remove_dir_all(&scratch);
let entry = entry?;
println!(" {} {}", entry.id, entry.version);
index.plugins.push(entry);
}
// A publisher's earlier versions of these plugins stay for IDEs that can't load the new
// ones (pruned below); plugins no longer in the folder go.
index.plugins.retain(|e| match publisher {
Some(p) => {
e.publisher != p
|| (added.iter().any(|n| n.id == e.id)
&& !added.iter().any(|n| (&n.id, &n.version) == (&e.id, &e.version)))
}
None => false,
});
index.plugins.extend(added);
index.prune_versions();
write_index(&index, &out)
}
/// `publishers` with `key` as `publisher`'s current key. Earlier keys stay listed
/// (packages they signed remain valid until revoked); a key new to the
/// publisher is endorsed by `previous`, one of them, so installed plugins
/// signed with it update without asking.
fn register_key(
mut publishers: Vec<package::Publisher>,
publisher: &str,
name: &str,
key: &PublisherKey,
previous: Option<&PublisherKey>,
) -> Result<Vec<package::Publisher>, String> {
let current = key.public_base64();
let earlier: Vec<KeyEntry> = publishers.iter().filter(|p| p.id == publisher).flat_map(|p| p.keys.clone()).collect();
let entry = match earlier.iter().find(|k| k.key == current) {
Some(listed) => listed.clone(),
None => {
let endorsement = match previous {
Some(prev) => {
let prev_key = prev.public_base64();
if !earlier.iter().any(|k| k.key == prev_key && !k.revoked) {
return Err(format!("--previous-key is not one of {publisher}'s valid keys in the index"));
}
Some(prev.endorse(publisher, &current))
}
None => {
if earlier.iter().any(|k| !k.revoked) {
println!(
" warning: {publisher}'s new key is not endorsed (--previous-key): users who installed \
its plugins will be asked to confirm the change when they update"
);
}
None
}
};
KeyEntry { endorsement, ..KeyEntry::new(current.clone()) }
}
};
let keys = std::iter::once(entry).chain(earlier.into_iter().filter(|k| k.key != current)).collect();
publishers.retain(|p| p.id != publisher);
publishers.push(package::Publisher { id: publisher.into(), name: name.into(), keys });
Ok(publishers)
}
/// `index`, unless a newer `openidea-plugin` wrote it (rewriting it would drop what this one can't read).
fn current_format(index: Index, file: &str) -> Result<Index, String> {
if index.is_newer() {
return Err(format!(
"{file}: index format {} is newer than this tool writes; update openidea-plugin",
index.format
));
}
Ok(index)
}
fn cmd_merge(args: &[String]) -> Result<(), String> {
let a = Args::parse(args, &[], &["keep", "out"])?;
let [base, other] = a.positional.as_slice() else {
return Err("usage: openidea-plugin merge <index> <other-index> --keep ID [--out FILE]".into());
};
let keep = a.value("keep").ok_or("--keep is required")?;
let out = a.value("out").unwrap_or(base);
let mut index =
current_format(Index::parse(&std::fs::read_to_string(base).map_err(|e| format!("{base}: {e}"))?)?, base)?;
let bytes = read_market_url(other, 16 << 20)?;
let theirs = Index::parse(&String::from_utf8(bytes).map_err(|_| format!("{other}: not UTF-8"))?)?;
let mut theirs = current_format(theirs, other)?;
// Their downloads are relative to their index, not to ours.
for e in &mut theirs.plugins {
e.download = resolve_market_href(other, &e.download);
}
let count = theirs.plugins.len();
index.include(theirs, &[keep]).map_err(|e| format!("{other}: {e}"))?;
println!(" {count} plugins from {other}");
write_index(&index, Path::new(out))
}
fn write_index(index: &Index, out: &Path) -> Result<(), String> {
let json = serde_json::to_string_pretty(index).map_err(|e| e.to_string())?;
std::fs::write(out, json).map_err(|e| format!("{}: {e}", out.display()))?;
let json = serde_json::to_string_pretty(&index).map_err(|e| e.to_string())?;
std::fs::write(&out, json).map_err(|e| format!("{}: {e}", out.display()))?;
println!("-> {} ({} plugins)", out.display(), index.plugins.len());
Ok(())
}
@@ -750,25 +589,6 @@ fn write_index(index: &Index, out: &Path) -> Result<(), String> {
mod tests {
use super::*;
#[test]
fn a_new_key_is_endorsed_by_the_previous_one() {
let (k1, k2, k3) =
(PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([2; 32]), PublisherKey::from_seed([3; 32]));
let first = register_key(Vec::new(), "acme", "ACME", &k1, None).unwrap();
let rotated = register_key(first.clone(), "acme", "ACME", &k2, Some(&k1)).unwrap();
let index = Index { format: package::INDEX_FORMAT, publishers: rotated.clone(), ..Index::default() };
assert!(index.succeeds("acme", &k1.public_base64(), &k2.public_base64()));
assert_eq!(rotated[0].keys.len(), 2, "the earlier key stays listed");
// Registering the current key again keeps its endorsement.
let again = register_key(rotated.clone(), "acme", "ACME", &k2, None).unwrap();
assert_eq!(again, rotated);
// Only a key the publisher has can endorse.
assert!(register_key(first.clone(), "acme", "ACME", &k2, Some(&k3)).unwrap_err().contains("--previous-key"));
// Without one the new key is listed unendorsed.
let unendorsed = register_key(first, "acme", "ACME", &k3, None).unwrap();
assert!(unendorsed[0].keys[0].endorsement.is_none());
}
#[test]
fn names() {
assert_eq!(slugify("My Cool_Plugin"), "my-cool-plugin");
@@ -781,126 +601,6 @@ mod tests {
assert_eq!(relative(Path::new("/x/y"), Path::new("/z")), None);
}
/// A theme package `id` signed by `publisher` with `key`, packed into `out`.
fn signed_theme(tmp: &Path, id: &str, publisher: &str, key: &str, out: &Path) {
signed_theme_version(tmp, id, "0.1.0", "^1.0", publisher, key, out);
}
fn signed_theme_version(tmp: &Path, id: &str, version: &str, engine: &str, publisher: &str, key: &str, out: &Path) {
let src = tmp.join("src").join(format!("{id}-{version}"));
let vars = templates::Vars { id, name: id, sdk: None };
for (rel, content) in templates::render(templates::Template::Theme, &vars) {
let path = src.join(rel);
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
let content = content
.replace("version = \"0.1.0\"", &format!("version = \"{version}\""))
.replace("engines.openidea = \"^1.0\"", &format!("engines.openidea = \"{engine}\""));
std::fs::write(path, content).unwrap();
}
std::fs::create_dir_all(out).unwrap();
let args = [src.to_str().unwrap(), "--key", key, "--publisher", publisher, "--out"]
.into_iter()
.map(String::from)
.chain([out.join(format!("{id}-{version}.oiplugin")).to_string_lossy().into_owned()])
.collect::<Vec<_>>();
cmd_pack(&args).unwrap();
}
#[test]
fn a_release_keeps_the_versions_older_ides_need() {
let tmp = tempfile::tempdir().unwrap();
let t = tmp.path();
let key = t.join("acme.key");
std::fs::write(&key, PublisherKey::from_seed([5; 32]).seed_base64()).unwrap();
let key = key.to_string_lossy().into_owned();
let index_file = t.join("index.json");
let release = |n: usize, packages: &[(&str, &str, &str)]| {
let dir = t.join(format!("release-{n}"));
for (id, version, engine) in packages {
signed_theme_version(t, id, version, engine, "acme", &key, &dir);
}
let base = format!("https://example.com/download/v{n}");
let args = [dir.to_str().unwrap(), "--publisher", "acme", "--key", &key, "--download-base", &base];
cmd_index(&strings(&[&args[..], &["--out", index_file.to_str().unwrap()]].concat())).unwrap();
let index = Index::parse(&std::fs::read_to_string(&index_file).unwrap()).unwrap();
index.plugins.iter().map(|e| format!("{} {} {}", e.id, e.version, e.download)).collect::<Vec<_>>()
};
release(1, &[("acme.a", "1.0.0", "^1.0"), ("acme.b", "1.0.0", "^1.0")]);
// Same requirement: the newer version replaces it; acme.b is gone from the release.
assert_eq!(
release(2, &[("acme.a", "1.1.0", "^1.0")]),
["acme.a 1.1.0 https://example.com/download/v2/acme.a-1.1.0.oiplugin"]
);
// Another API requirement: the last version for the earlier one stays, where it was published.
assert_eq!(
release(3, &[("acme.a", "2.0.0", ">=1.0, <2")]),
[
"acme.a 1.1.0 https://example.com/download/v2/acme.a-1.1.0.oiplugin",
"acme.a 2.0.0 https://example.com/download/v3/acme.a-2.0.0.oiplugin",
]
);
let index = Index::parse(&std::fs::read_to_string(&index_file).unwrap()).unwrap();
assert_eq!(index.offered()[0].0.version, "2.0.0", "the newest this IDE can load");
}
fn strings(args: &[&str]) -> Vec<String> {
args.iter().map(|a| a.to_string()).collect()
}
#[test]
fn official_and_community_indexes_merge() {
let tmp = tempfile::tempdir().unwrap();
let t = tmp.path();
let mut keys = Vec::new();
for (name, seed) in [("openidea", 1), ("alice", 2), ("carol", 3)] {
let file = t.join(format!("{name}.key"));
let key = PublisherKey::from_seed([seed; 32]);
std::fs::write(&file, key.seed_base64()).unwrap();
keys.push((file.to_string_lossy().into_owned(), key.public_base64()));
}
let (official, community) = (t.join("official"), t.join("community"));
signed_theme(t, "openidea.night", "openidea", &keys[0].0, &official);
signed_theme(t, "alice.dawn", "alice", &keys[1].0, &community);
let official_index = official.join("index.json");
let community_index = community.join("index.json");
let release = strings(&[official.to_str().unwrap(), "--publisher", "openidea", "--key", &keys[0].0]);
cmd_index(&release).unwrap();
// The community index lists its publishers; `index` without a key checks packages against them.
let skeleton = Index {
format: package::INDEX_FORMAT,
publishers: vec![package::Publisher {
id: "alice".into(),
name: "Alice".into(),
keys: vec![KeyEntry::new(keys[1].1.clone())],
}],
..Index::default()
};
write_index(&skeleton, &community_index).unwrap();
cmd_index(&strings(&[community.to_str().unwrap()])).unwrap();
cmd_merge(&strings(&[
official_index.to_str().unwrap(),
community_index.to_str().unwrap(),
"--keep",
"openidea",
]))
.unwrap();
// A release rebuilds the official plugins and keeps the community's.
cmd_index(&release).unwrap();
let index = Index::parse(&std::fs::read_to_string(&official_index).unwrap()).unwrap();
let listed: Vec<_> = index.plugins.iter().map(|e| (e.id.as_str(), e.publisher.as_str())).collect();
assert_eq!(listed, [("alice.dawn", "alice"), ("openidea.night", "openidea")]);
assert_eq!(index.plugins[1].categories, ["themes"], "implied by the theme");
assert_eq!(index.plugins[1].download, "openidea.night-0.1.0.oiplugin");
assert!(Path::new(&index.plugins[0].download).is_absolute(), "{}", index.plugins[0].download);
// A package by a publisher the index doesn't know is refused.
signed_theme(t, "carol.noon", "carol", &keys[2].0, &community);
let err = cmd_index(&strings(&[community.to_str().unwrap()])).unwrap_err();
assert!(err.contains("unknown publisher"), "{err}");
}
#[test]
fn every_template_is_a_valid_package() {
for t in templates::Template::ALL {
@@ -82,9 +82,6 @@ name = "__NAME__"
version = "0.1.0"
description = "Adds two sample commands."
authors = []
# Marketplace categories (up to 3): languages, themes, icon-themes, linters,
# vcs, ai, frameworks, tools, other.
categories = ["tools"]
engines.openidea = "^1.0"
# Built by `openidea-plugin build`. The code is loaded the first time one of
# its commands runs; the commands below show in Find Action before that.
-12
View File
@@ -30,17 +30,6 @@ pub fn set_command(host: &Host, window: &str, id: String, command: Option<String
host.project(window)?.agents.set_builtin_command(&id, command)
}
/// Environment variables built-in agent `id` starts with; empty clears them.
pub fn set_env(
host: &Host,
window: &str,
id: String,
env: std::collections::BTreeMap<String, String>,
) -> IdeResult<()> {
host.require_trust(window)?;
host.project(window)?.agents.set_builtin_env(&id, env)
}
/// Where ACP Registry agents are saved (`<config>/agents`, like the user's own) and installed.
pub(crate) fn registry_paths(host: &Host) -> RegistryPaths {
RegistryPaths::new(&host.config_dir, &host.local_data_dir)
@@ -206,7 +195,6 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("agent_subscribe", |c| async move { subscribe(&c.host, &c.window, c.channel("onEvent")?) });
d.add("agent_providers", |c| async move { providers(&c.host, &c.window) });
d.add("agent_set_command", |c| async move { set_command(&c.host, &c.window, c.arg("id")?, c.arg("command")?) });
d.add("agent_set_env", |c| async move { set_env(&c.host, &c.window, c.arg("id")?, c.arg("env")?) });
d.add("agent_registry_list", |c| async move { registry_list(&c.host, &c.window, c.arg("request")?).await });
d.add("agent_registry_install", |c| async move {
registry_install(&c.host, &c.window, c.arg("id")?, c.arg("request")?, c.sender("onEvent")?).await
+2 -73
View File
@@ -116,11 +116,6 @@ pub async fn build_sync(
tracing::warn!("Maven profiles not saved: {e}");
false
});
if profiles_changed {
// The modules of the chosen profiles are read from the POMs with them from now on (the
// project tree's module folders too), whether or not Maven itself can run.
project.sink.emit(ide_core::IdeEvent::ProjectRootsChanged { root: project.root.clone().into() });
}
let token = host.begin_project_request(window, request);
let result = tokio::task::spawn_blocking(move || {
let current = load(&root, &cache, &settings);
@@ -222,52 +217,6 @@ fn declaring_pom(
)))
}
/// The `.toml` of a version catalog the build imports as `group:artifact:version`, once a
/// build has downloaded it (`ide_sdk::libraries::published_version_catalog`).
pub async fn gradle_published_catalog(
host: &Host,
window: &str,
coordinates: String,
) -> IdeResult<Option<ide_core::IpcPath>> {
let project = host.project(window)?;
tokio::task::spawn_blocking(move || {
ide_sdk::libraries::published_version_catalog(&project.root, &project.settings.effective(), &coordinates)
.map(Into::into)
})
.await
.map_err(join_err)
}
/// The resolved dependency tree of the Gradle project `gradle_path` (`:`, `:app`) for
/// `configuration`, from Gradle's `dependencies` report; cancellable with `cancel_request(request)`.
pub async fn gradle_dependencies(
host: &Host,
window: &str,
gradle_path: String,
configuration: String,
offline: bool,
request: RequestId,
) -> IdeResult<Vec<ide_build::GradleDependency>> {
host.require_trust(window)?;
let project = host.project(window)?;
let (root, cache, settings) = (project.root.clone(), project.build_cache_dir(), project.settings.effective());
let token = host.begin_project_request(window, request);
let result = tokio::task::spawn_blocking(move || {
let gradle = load(&root, &cache, &settings).gradle;
let options = SyncOptions {
offline,
env: gradle.as_ref().map(|g| g.env.clone()).unwrap_or_default(),
gradle_command: gradle.as_ref().map(|g| g.command.clone()),
..Default::default()
};
ide_build::gradle_dependencies(&root, &options, &gradle_path, &configuration, &token)
})
.await
.map_err(join_err);
host.end_request(request);
result?
}
/// Where the direct dependency `dependency` of the Maven module `pom` is declared.
pub async fn maven_dependency_declaration(
host: &Host,
@@ -347,7 +296,7 @@ pub async fn build_run(
let settings = ide_workspace::SettingsStore::load(&config_dir, Some(member)).effective();
let member_cache = cache.join(crate::project::project_key(member));
let model = with_profiles(load(member, &member_cache, &settings), &member_cache);
let plan = compile::plan(member, &model, kind, file.as_deref(), settings.cmake.profiles.as_deref())?;
let plan = compile::plan(member, &model, kind, file.as_deref())?;
match &mut combined {
Some(combined) => combined.steps.extend(plan.steps),
None => combined = Some(plan),
@@ -357,13 +306,7 @@ pub async fn build_run(
plan.title = workspace.name;
plan
} else {
compile::plan(
&root,
&with_profiles(load(&root, &cache, &settings), &cache),
kind,
file.as_deref(),
settings.cmake.profiles.as_deref(),
)?
compile::plan(&root, &with_profiles(load(&root, &cache, &settings), &cache), kind, file.as_deref())?
};
let result = compile::run(&root, &plan, &token, &|e| {
let _ = on_event.send(e);
@@ -394,20 +337,6 @@ pub(crate) fn register(d: &mut Dispatcher) {
)
.await
});
d.add("gradle_published_catalog", |c| async move {
gradle_published_catalog(&c.host, &c.window, c.arg("coordinates")?).await
});
d.add("gradle_dependencies", |c| async move {
gradle_dependencies(
&c.host,
&c.window,
c.arg("gradlePath")?,
c.arg("configuration")?,
c.arg("offline")?,
c.arg("request")?,
)
.await
});
d.add("maven_dependency_declaration", |c| async move {
maven_dependency_declaration(&c.host, &c.window, c.arg("pom")?, c.arg("dependency")?).await
});
+2 -1
View File
@@ -216,7 +216,8 @@ pub async fn connect(host: &Host, window: &str, id: &str) -> IdeResult<()> {
/// manual mode rolls back what was not committed).
pub async fn set_manual_commit(host: &Host, window: &str, id: &str, manual: bool) -> IdeResult<()> {
let db = host.project(window)?.db.clone();
db.set_manual_commit(id, manual).await
db.set_manual_commit(id, manual).await;
Ok(())
}
/// Commits or rolls back the open transaction of a manual-commit data source.
-25
View File
@@ -188,19 +188,6 @@ pub async fn debug_stop(host: &Host, _window: &str, session: DebugSessionId) ->
host.debugger.stop(session).await
}
/// The adapter's completions for an expression being typed in the frame `frame_id`.
pub async fn debug_completions(
host: &Host,
_window: &str,
session: DebugSessionId,
frame_id: Option<i64>,
text: String,
line: u32,
column: u32,
) -> IdeResult<Vec<ide_dap::DebugCompletion>> {
host.debugger.completions(session, frame_id, text, line, column).await
}
/// Evaluate with an explicit DAP context: `repl`, `watch`, `hover` or `clipboard`.
pub async fn debug_evaluate_in(
host: &Host,
@@ -389,18 +376,6 @@ pub(crate) fn register(d: &mut Dispatcher) {
.await
});
d.add("debug_stop", |c| async move { debug_stop(&c.host, &c.window, c.arg("session")?).await });
d.add("debug_completions", |c| async move {
debug_completions(
&c.host,
&c.window,
c.arg("session")?,
c.arg("frameId")?,
c.arg("text")?,
c.arg("line")?,
c.arg("column")?,
)
.await
});
d.add("debug_evaluate_in", |c| async move {
debug_evaluate_in(
&c.host,
-17
View File
@@ -1,17 +0,0 @@
//! Docker run configurations: the environment their `docker` command line runs with.
use crate::Host;
use crate::rpc::Dispatcher;
use ide_core::IdeResult;
use ide_docker::DockerEndpoint;
use std::collections::BTreeMap;
/// The endpoint's variables plus a `PATH` that finds this host's `docker`. A remote project
/// gets them from the server, since its run configurations start on the remote host.
pub async fn run_env(host: &Host, endpoint: DockerEndpoint) -> IdeResult<BTreeMap<String, String>> {
endpoint.validate()?;
host.blocking(move || Ok(ide_docker::run_env(&endpoint))).await
}
pub(crate) fn register(d: &mut Dispatcher) {
d.add("docker_run_env", |c| async move { run_env(&c.host, c.arg("endpoint")?).await });
}
+11 -133
View File
@@ -76,17 +76,9 @@ async fn plugin_call<T: Send + 'static>(f: impl FnOnce() -> T + Send + 'static)
tokio::task::spawn_blocking(f).await.ok()
}
/// Items from plugins carry no server: commands don't apply, and a code
/// action is resolved by the plugin named in its `data` ([`PluginActionRef`]).
/// Items from plugins carry no server (resolve and commands don't apply).
const PLUGIN_SERVER: ServerId = ServerId(0);
/// `data` of a plugin's code action whose edits it computes when picked.
#[derive(serde::Serialize, serde::Deserialize)]
struct PluginActionRef {
plugin: String,
data: String,
}
/// A plugin's `completion-item.kind` (wit/providers.wit lists them).
fn plugin_completion_kind(kind: &str) -> Option<CompletionKind> {
Some(match kind {
@@ -128,14 +120,14 @@ fn plugin_completion(item: ide_plugin::CompletionItem) -> CompletionItem {
kind,
documentation: item.documentation.map(|value| Markup { kind: "markdown".into(), value }),
sort_text: item.sort_text,
filter_text: item.filter_text,
filter_text: None,
preselect: false,
deprecated: false,
insert_text: item.insert_text,
is_snippet: item.snippet,
is_snippet: false,
replace_range: item.replace,
replace_range_tab: None,
additional_edits: item.additional_edits,
additional_edits: Vec::new(),
commit_characters: Vec::new(),
command: None,
data: None,
@@ -322,7 +314,7 @@ pub async fn lsp_code_actions(
};
let plugin_actions: Vec<CodeAction> = plugin_actions
.into_iter()
.map(|(plugin, a)| plugin_code_action(&plugin, a))
.map(|(_, a)| plugin_code_action(a))
.filter(|a| {
only.is_empty()
|| a.kind.as_ref().is_some_and(|k| only.iter().any(|o| k == o || k.starts_with(&format!("{o}."))))
@@ -338,53 +330,24 @@ pub async fn lsp_code_actions(
prepare_actions(project, doc, version, actions).await
}
fn plugin_code_action(plugin: &str, a: ide_plugin::CodeAction) -> CodeAction {
// With `data` and no edits, the plugin computes them when the action is picked.
let lazy = a.edits.is_empty() && a.data.is_some();
let files: Vec<_> = a
fn plugin_code_action(a: ide_plugin::CodeAction) -> CodeAction {
let files = a
.edits
.into_iter()
.map(|(path, edits)| ide_core::FileEdit::Edit { path: path.into(), edits, version: None })
.collect();
let data = a
.data
.filter(|_| lazy)
.and_then(|data| serde_json::to_value(PluginActionRef { plugin: plugin.to_owned(), data }).ok());
CodeAction {
title: a.title,
kind: a.kind,
is_preferred: a.preferred,
disabled_reason: None,
edit: (!lazy).then(|| WorkspaceEdit { files, preconditions: Vec::new() }),
edit: Some(WorkspaceEdit { files, preconditions: Vec::new() }),
command: None,
data,
data: None,
server: PLUGIN_SERVER,
}
}
/// A code action resolved by whoever offered it: its language server, or
/// the plugin its `data` names.
async fn resolve_action(host: &Host, window: &str, doc: DocId, action: CodeAction) -> IdeResult<CodeAction> {
if action.server != PLUGIN_SERVER {
return host.project(window)?.lsp.resolve_code_action(doc, action).await;
}
let Some(r) = action.data.clone().and_then(|d| serde_json::from_value::<PluginActionRef>(d).ok()) else {
return Ok(action);
};
let (plugins, d) = plugin_providers(host, window, doc, ProviderKind::CodeActions)
.ok_or_else(|| ide_core::IdeError::unavailable("the plugin that offered this action is gone"))?;
let pending = ide_plugin::CodeAction {
title: action.title.clone(),
kind: action.kind.clone(),
preferred: action.is_preferred,
edits: Vec::new(),
data: Some(r.data),
};
let plugin = r.plugin.clone();
let resolved = blocking(move || plugins.resolve_code_action(&plugin, &d, pending)).await?;
Ok(plugin_code_action(&r.plugin, ide_plugin::CodeAction { data: None, ..resolved }))
}
/// Cancellable code actions with the kinds the servers advertise (Refactor This, refactoring availability).
pub async fn lsp_code_action_query(
host: &Host,
@@ -418,7 +381,7 @@ pub async fn lsp_resolve_code_action(
let project = host.project(window)?;
let documents = project.documents.clone();
let version = blocking(move || documents.version(doc)).await?;
let action = resolve_action(host, window, doc, action).await?;
let action = project.lsp.resolve_code_action(doc, action).await?;
Ok(prepare_actions(project, doc, version, vec![action]).await?.remove(0))
}
@@ -428,7 +391,7 @@ pub async fn lsp_apply_code_action(host: &Host, window: &str, doc: DocId, action
let action = if action.edit.is_none() && action.command.is_none() {
let documents = project.documents.clone();
let version = blocking(move || documents.version(doc)).await?;
let action = resolve_action(host, window, doc, action).await?;
let action = project.lsp.resolve_code_action(doc, action).await?;
prepare_actions(project.clone(), doc, version, vec![action]).await?.remove(0)
} else {
action
@@ -449,56 +412,6 @@ pub async fn lsp_prepare_rename(host: &Host, window: &str, doc: DocId, pos: Pos)
}
/// Computes the rename edit without applying it, so the UI can preview conflicts.
/// The class at `pos` and the members a class-hierarchy refactoring can move.
pub async fn lsp_hierarchy_members(
host: &Host,
window: &str,
request: RequestId,
doc: DocId,
pos: Pos,
kind: HierarchyRefactoring,
) -> IdeResult<HierarchyMembers> {
let project = host.project(window)?;
let cancel = host.begin_request(request);
let result = project.lsp.hierarchy_members(doc, pos, kind, cancel).await;
host.end_request(request);
result
}
/// Checks a class-hierarchy refactoring and computes its edit without applying it.
#[allow(clippy::too_many_arguments)]
pub async fn lsp_hierarchy_refactor(
host: &Host,
window: &str,
request: RequestId,
doc: DocId,
pos: Pos,
kind: HierarchyRefactoring,
target: Option<String>,
choices: Vec<HierarchyChoice>,
) -> IdeResult<HierarchyRefactored> {
let project = host.project(window)?;
let documents = project.documents.clone();
let version = blocking(move || documents.version(doc)).await?;
let cancel = host.begin_request(request);
let result = project.lsp.hierarchy_refactor(doc, pos, kind, target, choices, cancel).await;
host.end_request(request);
let mut refactored = result?;
blocking(move || {
if project.documents.version(doc)? != version {
return Err(ide_core::IdeError::new(
ide_core::error::ErrorKind::OutOfSync,
"document changed while computing the refactoring; request it again",
));
}
if let Some(edit) = &mut refactored.edit {
project.documents.prepare_workspace_edit(edit)?;
}
Ok(refactored)
})
.await
}
pub async fn lsp_rename(
host: &Host,
window: &str,
@@ -763,22 +676,6 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("lsp_rename", |c| async move {
lsp_rename(&c.host, &c.window, c.arg("request")?, c.arg("doc")?, c.arg("pos")?, c.arg("newName")?).await
});
d.add("lsp_hierarchy_members", |c| async move {
lsp_hierarchy_members(&c.host, &c.window, c.arg("request")?, c.arg("doc")?, c.arg("pos")?, c.arg("kind")?).await
});
d.add("lsp_hierarchy_refactor", |c| async move {
lsp_hierarchy_refactor(
&c.host,
&c.window,
c.arg("request")?,
c.arg("doc")?,
c.arg("pos")?,
c.arg("kind")?,
c.arg("target")?,
c.arg("choices")?,
)
.await
});
d.add("lsp_format", |c| async move {
lsp_format(&c.host, &c.window, c.arg("doc")?, c.arg("range")?, c.arg("options")?).await
});
@@ -838,25 +735,6 @@ pub(crate) fn register(d: &mut Dispatcher) {
mod tests {
use super::*;
#[test]
fn a_plugin_action_with_data_and_no_edits_is_resolved_by_its_plugin() {
let action = |edits: Vec<(std::path::PathBuf, Vec<TextEdit>)>, data: Option<&str>| ide_plugin::CodeAction {
title: "Fix".into(),
kind: None,
preferred: false,
edits,
data: data.map(str::to_owned),
};
let lazy = plugin_code_action("acme.x", action(Vec::new(), Some("ref")));
assert!(lazy.edit.is_none());
let r: PluginActionRef = serde_json::from_value(lazy.data.unwrap()).unwrap();
assert_eq!((r.plugin.as_str(), r.data.as_str()), ("acme.x", "ref"));
// With edits, `data` is the plugin's business: applied as is.
let edit = TextEdit { range: Range::default(), new_text: "x".into() };
let eager = plugin_code_action("acme.x", action(vec![("/a".into(), vec![edit])], Some("ref")));
assert!(eager.edit.is_some() && eager.data.is_none());
}
#[test]
fn every_completion_kind_the_plugin_api_documents_is_understood() {
let wit = include_str!("../../../../wit/providers.wit");
-2
View File
@@ -12,7 +12,6 @@ pub mod debug;
pub mod dependencies;
pub mod deploy;
pub mod doc;
pub mod docker;
pub mod env;
pub mod fs;
pub mod history;
@@ -88,7 +87,6 @@ pub fn register_all(d: &mut crate::rpc::Dispatcher) {
run::register(d);
env::register(d);
deploy::register(d);
docker::register(d);
db::register(d);
}
+14 -61
View File
@@ -41,7 +41,7 @@ pub fn configs(host: &Host, window: &str) -> IdeResult<Vec<RunConfig>> {
}
/// A project's detected configurations (minus removed ones and unlinked Maven projects), then
/// its saved ones; toolchains from Languages & Frameworks applied. Build-tool choices stay as saved:
/// its saved ones; toolchains from Languages & Frameworks applied. Gradle choices stay as saved:
/// derived defaults must not be written back as explicit environment.
fn detected_and_saved(
root: &std::path::Path,
@@ -56,7 +56,7 @@ fn detected_and_saved(
.filter(|d| !suppressed.contains(&d.id) && !saved.iter().any(|u| u.id == d.id))
.collect();
all.extend(saved.iter().cloned());
for config in all.iter_mut().filter(|c| !matches!(c.kind.as_str(), "gradle" | "maven")) {
for config in all.iter_mut().filter(|c| c.kind != "gradle") {
ide_sdk::apply::apply_to_run_configs(std::slice::from_mut(config), root, settings);
}
all
@@ -75,65 +75,13 @@ pub fn save_configs(
let configs: Vec<RunConfig> = configs
.iter()
.map(|c| match c.kind.as_str() {
"gradle" | "maven" => c.clone(),
"gradle" => c.clone(),
_ => ide_sdk::apply::without_derived(c, &project.root, &settings),
})
.collect();
ide_run::save_user_configs_with_removals(&project.root, &configs, removed_detected_ids)
}
/// The project `config` runs in (a workspace member, by its working directory) and its settings.
fn settings_for(
host: &Host,
project: &crate::project::Project,
config: &RunConfig,
) -> IdeResult<(std::path::PathBuf, ide_core::settings::Settings)> {
let member = ide_workspace::workspace::Workspace::load(&project.root)?.and_then(|workspace| {
let cwd = config.cwd.as_deref()?;
workspace.projects.into_iter().filter(|root| cwd.starts_with(root)).max_by_key(|root| root.components().count())
});
Ok(match member {
Some(root) => {
let settings = ide_workspace::SettingsStore::load(&host.config_dir, Some(&root)).effective();
(root, settings)
}
None => (project.root.clone(), project.settings.effective()),
})
}
/// Temporary configurations as they are kept between sessions: without what settings derived
/// (toolchain commands, Maven options, `JAVA_HOME`…), so they follow later settings changes.
pub fn without_derived(host: &Host, window: &str, configs: Vec<RunConfig>) -> IdeResult<Vec<RunConfig>> {
host.require_trust(window)?;
let project = host.project(window)?;
configs
.into_iter()
.map(|c| {
if c.kind == "gradle" {
return Ok(c);
}
let (root, settings) = settings_for(host, &project, &c)?;
Ok(ide_sdk::apply::without_derived(&c, &root, &settings))
})
.collect()
}
/// Kept temporary configurations with today's settings applied, as [`configs`] lists the others.
pub fn apply_settings(host: &Host, window: &str, configs: Vec<RunConfig>) -> IdeResult<Vec<RunConfig>> {
host.require_trust(window)?;
let project = host.project(window)?;
configs
.into_iter()
.map(|mut c| {
if c.kind != "gradle" {
let (root, settings) = settings_for(host, &project, &c)?;
ide_sdk::apply::apply_to_run_configs(std::slice::from_mut(&mut c), &root, &settings);
}
Ok(c)
})
.collect()
}
pub async fn start(host: &Host, window: &str, config: RunConfig, on_output: Emit<OutputChunk>) -> IdeResult<RunInfo> {
host.require_trust(window)?;
let project = host.project(window)?;
@@ -152,13 +100,18 @@ pub async fn start(host: &Host, window: &str, config: RunConfig, on_output: Emit
.run(&settings, &project.root, &config, project.sink.clone(), Box::new(move |c| out(c)))
.await;
}
let (root, settings) = settings_for(host, &project, &config)?;
let root = &root;
let member = ide_workspace::workspace::Workspace::load(&project.root)?.and_then(|workspace| {
let cwd = config.cwd.as_deref()?;
workspace.projects.into_iter().filter(|root| cwd.starts_with(root)).max_by_key(|root| root.components().count())
});
let root = member.as_ref().unwrap_or(&project.root);
let settings = match &member {
Some(root) => ide_workspace::SettingsStore::load(&host.config_dir, Some(root)).effective(),
None => project.settings.effective(),
};
// Configurations made in the UI name bare tools (`godot`) that may live outside PATH.
let mut config = config;
if config.kind == "maven" {
ide_sdk::apply::prepare_maven_run(&mut config, root, &settings);
} else if matches!(config.kind.as_str(), "gradle" | "tomcat") {
if matches!(config.kind.as_str(), "gradle" | "tomcat") {
ide_sdk::apply::apply_to_run_configs(std::slice::from_mut(&mut config), root, &settings);
} else {
ide_sdk::apply::resolve_run_command(&mut config, root, &settings);
@@ -194,7 +147,7 @@ pub async fn start(host: &Host, window: &str, config: RunConfig, on_output: Emit
let encoding = settings.console.encoding;
let options = StartOptions {
encoding: Some(encoding).filter(|e| !e.trim().is_empty()),
python_paths: if *root != project.root { vec![root.clone()] } else { super::python::paths(&project) },
python_paths: if member.is_some() { vec![root.clone()] } else { super::python::paths(&project) },
};
project.require_trust()?;
let mut sink = project.sink.clone();
-40
View File
@@ -241,35 +241,6 @@ pub async fn run_tomcat_update(host: &Host, window: &str, config: RunConfig, mod
.map_err(|e| IdeError::internal(e.to_string()))?
}
/// Updates a running Tomcat artifact deployment from the module's compiled output and web
/// resources (`ide_run::tomcat::sync_outputs`). `false`: rebuild the WAR instead.
pub async fn run_tomcat_sync(
host: &Host,
window: &str,
config: RunConfig,
module: std::path::PathBuf,
class_paths: Vec<String>,
resources_only: bool,
) -> IdeResult<bool> {
host.require_trust(window)?;
let project = host.project(window)?;
if !host.processes.list().iter().any(|r| r.running && r.config_id == config.id) {
return Err(IdeError::invalid("Start this Tomcat configuration before updating it"));
}
if config.extras.params.get("deployment").is_some_and(|v| v != "artifact") {
return Ok(false);
}
tokio::task::spawn_blocking(move || {
let module = dunce::canonicalize(project.root.join(module))?;
let class_paths: Vec<_> =
class_paths.iter().filter_map(|p| dunce::canonicalize(project.root.join(p)).ok()).collect();
let base = ide_run::tomcat::base_dir(&project.cache_dir, &config.id);
ide_run::tomcat::sync_outputs(&base, &module, &class_paths, resources_only)
})
.await
.map_err(|e| IdeError::internal(e.to_string()))?
}
/// Commands of this module the remote server answers.
pub(crate) fn register(d: &mut Dispatcher) {
d.add("run_dotnet_resolve", |c| async move {
@@ -284,17 +255,6 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("run_tomcat_home", |c| async move { run_tomcat_home(&c.host, &c.window, c.arg("home")?).await });
d.add("run_tomcat_artifacts", |c| async move { run_tomcat_artifacts(&c.host, &c.window).await });
d.add("run_tomcat_build", |c| async move { run_tomcat_build(&c.host, &c.window, c.arg("config")?).await });
d.add("run_tomcat_sync", |c| async move {
run_tomcat_sync(
&c.host,
&c.window,
c.arg("config")?,
c.arg("module")?,
c.arg("classPaths")?,
c.arg("resourcesOnly")?,
)
.await
});
d.add("run_tomcat_update", |c| async move {
run_tomcat_update(&c.host, &c.window, c.arg("config")?, c.arg("mode")?).await
});
@@ -1,207 +0,0 @@
use super::icon_patterns::validate_pattern;
use ide_extension::validate_item_id;
use serde::{Deserialize, Serialize};
use std::collections::{BTreeMap, BTreeSet};
use std::path::PathBuf;
const MAX_GROUPS: usize = 64;
const MAX_CHOICES: usize = 16;
const MAX_CHOICE_OPTIONS: usize = 16;
const MAX_NAME_LEN: usize = 255;
pub(super) const DARK_SUFFIX: &str = "_dark";
#[derive(Deserialize)]
#[serde(untagged)]
pub(super) enum IconDefinition {
Path(PathBuf),
Variants(IconVariants),
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
pub(super) struct IconVariants {
light: PathBuf,
#[serde(default)]
dark: Option<PathBuf>,
}
impl IconDefinition {
pub(super) fn light(&self) -> &PathBuf {
match self {
IconDefinition::Path(p) => p,
IconDefinition::Variants(v) => &v.light,
}
}
pub(super) fn dark(&self) -> Option<&PathBuf> {
match self {
IconDefinition::Path(_) => None,
IconDefinition::Variants(v) => v.dark.as_ref(),
}
}
}
#[derive(Clone, Copy, Default, PartialEq, Eq, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub(super) enum Target {
#[default]
File,
Folder,
}
#[derive(Deserialize, Serialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub(super) struct Pattern {
pattern: String,
icon: String,
#[serde(default)]
target: Target,
#[serde(default)]
path: bool,
#[serde(default)]
ignore_case: bool,
#[serde(default)]
group: Option<String>,
#[serde(default)]
open_icon: Option<String>,
}
fn enabled_by_default() -> bool {
true
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub(super) struct Group {
pub(super) id: String,
name: String,
#[serde(default = "enabled_by_default")]
enabled: bool,
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub(super) struct ChoiceOption {
pub(super) id: String,
pub(super) name: String,
#[serde(default)]
overrides: BTreeMap<String, String>,
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub(super) struct Choice {
pub(super) id: String,
pub(super) name: String,
pub(super) default: String,
pub(super) options: Vec<ChoiceOption>,
}
#[derive(Default, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct Extensions {
#[serde(default)]
pub(super) patterns: Vec<Pattern>,
#[serde(default)]
pub(super) overrides: BTreeMap<String, String>,
#[serde(default)]
pub(super) groups: Vec<Group>,
#[serde(default)]
pub(super) choices: Vec<Choice>,
#[serde(default)]
pub(super) accent: bool,
}
pub(super) fn check_icon_id(id: &str) -> Result<(), String> {
validate_item_id(id, "icon id")?;
if id.ends_with(DARK_SUFFIX) {
return Err(format!("icon id {id:?}: the {DARK_SUFFIX:?} suffix is reserved for dark variants"));
}
Ok(())
}
fn check_named(id: &str, name: &str, what: &str) -> Result<(), String> {
validate_item_id(id, what)?;
if name.trim().is_empty() || name.len() > MAX_NAME_LEN {
return Err(format!("{what} {id:?}: name must have 1 to {MAX_NAME_LEN} characters"));
}
Ok(())
}
fn check_unique<'a>(ids: impl Iterator<Item = &'a String>, what: &str) -> Result<(), String> {
let mut seen = BTreeSet::new();
match ids.into_iter().find(|id| !seen.insert(*id)) {
Some(id) => Err(format!("duplicate {what} {id:?}")),
None => Ok(()),
}
}
impl Extensions {
pub(super) fn mappings(&self) -> usize {
let chosen: usize = self.choices.iter().flat_map(|c| &c.options).map(|o| o.overrides.len()).sum();
self.patterns.len() + self.overrides.len() + chosen
}
pub(super) fn validate(&self, check_ref: &dyn Fn(&str, &str) -> Result<(), String>) -> Result<(), String> {
self.validate_groups()?;
for p in &self.patterns {
validate_pattern(&p.pattern)?;
check_ref(&p.icon, "patterns")?;
if let Some(group) = &p.group
&& !self.groups.iter().any(|g| &g.id == group)
{
return Err(format!("pattern {:?}: unknown group {group:?}", p.pattern));
}
if let Some(open) = &p.open_icon {
if p.target != Target::Folder {
return Err(format!("pattern {:?}: openIcon is only for folder patterns", p.pattern));
}
check_ref(open, "patterns")?;
}
}
check_overrides(&self.overrides, check_ref, "overrides")?;
for choice in &self.choices {
for option in &choice.options {
check_overrides(&option.overrides, check_ref, "choices")?;
}
}
Ok(())
}
fn validate_groups(&self) -> Result<(), String> {
if self.groups.len() > MAX_GROUPS || self.choices.len() > MAX_CHOICES {
return Err(format!("too many groups or choices (max {MAX_GROUPS} and {MAX_CHOICES})"));
}
for g in &self.groups {
check_named(&g.id, &g.name, "group")?;
}
check_unique(self.groups.iter().map(|g| &g.id), "group")?;
check_unique(self.choices.iter().map(|c| &c.id), "choice")?;
for choice in &self.choices {
check_named(&choice.id, &choice.name, "choice")?;
if choice.options.is_empty() || choice.options.len() > MAX_CHOICE_OPTIONS {
return Err(format!("choice {:?}: expected 1 to {MAX_CHOICE_OPTIONS} options", choice.id));
}
for option in &choice.options {
check_named(&option.id, &option.name, "choice option")?;
}
check_unique(choice.options.iter().map(|o| &o.id), "choice option")?;
if !choice.options.iter().any(|o| o.id == choice.default) {
return Err(format!("choice {:?}: default {:?} is not one of its options", choice.id, choice.default));
}
}
Ok(())
}
}
fn check_overrides(
overrides: &BTreeMap<String, String>,
check_ref: &dyn Fn(&str, &str) -> Result<(), String>,
what: &str,
) -> Result<(), String> {
for (name, icon) in overrides {
validate_item_id(name, "overridden icon name")?;
check_ref(icon, what)?;
}
Ok(())
}
@@ -1,105 +0,0 @@
use regex_syntax::hir::{Hir, HirKind};
pub(crate) const MAX_PATTERN_LEN: usize = 512;
const NAME_LEN: u64 = 256;
const MAX_COST: u64 = 4 * NAME_LEN * NAME_LEN;
const UNSUPPORTED: &[(&str, &str)] = &[
("\\A", "\\A"),
("\\z", "\\z"),
("\\Z", "\\Z"),
("\\p", "\\p"),
("\\P", "\\P"),
("[[:", "a POSIX class"),
("&&", "class intersection"),
("~~", "class symmetric difference"),
];
pub(crate) fn validate_pattern(pattern: &str) -> Result<(), String> {
if pattern.len() > MAX_PATTERN_LEN {
return Err(format!("pattern {pattern:?} is too long (max {MAX_PATTERN_LEN})"));
}
if let Some(construct) = unsupported_construct(pattern) {
return Err(format!("pattern {pattern:?}: {construct} is not supported"));
}
let hir = regex_syntax::Parser::new().parse(pattern).map_err(|e| format!("invalid pattern {pattern:?}: {e}"))?;
match cost(&hir) {
Err(why) => Err(format!("pattern {pattern:?}: {why} is not supported")),
Ok(c) if c > MAX_COST => Err(format!("pattern {pattern:?} is too complex: too many ways to match one name")),
Ok(_) => Ok(()),
}
}
fn unsupported_construct(pattern: &str) -> Option<&'static str> {
if pattern.match_indices("(?").any(|(at, _)| !pattern[at + 2..].starts_with(':')) {
return Some("an inline flag, lookaround or named group");
}
UNSUPPORTED.iter().find(|(needle, _)| pattern.contains(needle)).map(|(_, what)| *what)
}
fn cost(hir: &Hir) -> Result<u64, &'static str> {
match hir.kind() {
HirKind::Empty | HirKind::Literal(_) | HirKind::Class(_) | HirKind::Look(_) => Ok(1),
HirKind::Capture(c) => cost(&c.sub),
HirKind::Concat(items) => items.iter().try_fold(1u64, |acc, h| Ok(acc.saturating_mul(cost(h)?))),
HirKind::Alternation(items) => items.iter().try_fold(0u64, |acc, h| Ok(acc.saturating_add(cost(h)?))),
HirKind::Repetition(r) => match r.max {
None if matches!(r.sub.kind(), HirKind::Literal(_) | HirKind::Class(_)) => Ok(NAME_LEN),
None => Err("a repeated group (`(…)*`, `(…)+`)"),
Some(max) => Ok(cost(&r.sub)?.saturating_add(1).saturating_pow(max)),
},
}
}
#[cfg(test)]
mod tests {
use super::*;
fn err(pattern: &str) -> String {
validate_pattern(pattern).unwrap_err()
}
#[test]
fn accepts_typical_file_name_patterns() {
for pattern in [
r"^(architecture|ARCHITECTURE)(\.(md|rst|txt))?$",
r"(main|workflow|ci|release|build|config)\.ya?ml$",
r"^[a-z]+\.d\.ts$",
r"^\.?eslintrc(\.(js|cjs|json|ya?ml))?$",
r"^.*\.config\.[cm]?[jt]s$",
r"^.*-(test|spec)\..*$",
r"^[0-9]{4}-[0-9]{2}\.log$",
] {
assert!(validate_pattern(pattern).is_ok(), "{pattern}: {:?}", validate_pattern(pattern));
}
}
#[test]
fn rejects_patterns_that_backtrack_catastrophically() {
assert!(err(r"(a+)+$").contains("repeated group"));
assert!(err(r"(?:a*)*").contains("repeated group"));
assert!(err(r"^(\w|\w\w)*!$").contains("repeated group"));
assert!(err(r"^(a|aa)+$").contains("repeated group"));
assert!(err(r"^.*.*.*x$").contains("too complex"));
assert!(err(r"^(a?){30}a{30}$").contains("too complex"));
assert!(err(&format!("^{}b$", "(a|aa)".repeat(20))).contains("too complex"));
assert!(err(r"x{0,40}").contains("too complex"));
}
#[test]
fn rejects_constructs_javascript_reads_differently() {
assert!(err(r"(?i)readme").contains("not supported"));
assert!(err(r"(?P<n>a)").contains("not supported"));
assert!(err(r"\Areadme").contains("not supported"));
assert!(err(r"readme\z").contains("not supported"));
assert!(err(r"\p{L}").contains("not supported"));
assert!(err(r"[[:alpha:]]").contains("not supported"));
assert!(err(r"[a-z&&[^x]]").contains("not supported"));
}
#[test]
fn rejects_invalid_and_oversized_patterns() {
assert!(err("(").contains("invalid pattern"));
assert!(err(&"a".repeat(MAX_PATTERN_LEN + 1)).contains("too long"));
}
}
@@ -1,185 +0,0 @@
use super::icon_extensions::Extensions;
use serde::{Deserialize, Serialize};
const MAX_SECTIONS: usize = 16;
const MAX_ITEMS: usize = 32;
const MAX_TEXT_LEN: usize = 255;
const MAX_HINT_LEN: usize = 512;
const MAX_COLUMNS: u8 = 4;
#[derive(Clone, Copy, PartialEq, Eq, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub(super) enum SettingOption {
HideFiles,
HideFolders,
HideUiIcons,
HidePsiIcons,
HollowFolders,
Groups,
Choice,
Accent,
Monochrome,
Saturation,
Opacity,
IconSize,
RowHeight,
}
#[derive(Deserialize, Serialize)]
#[serde(untagged)]
pub(super) enum SettingDefault {
Number(u8),
Color(String),
}
#[derive(Deserialize, Serialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub(super) struct SettingItem {
option: SettingOption,
#[serde(default)]
label: Option<String>,
#[serde(default)]
hint: Option<String>,
#[serde(default)]
invert: bool,
#[serde(default)]
choice: Option<String>,
#[serde(default)]
columns: Option<u8>,
#[serde(default)]
default: Option<SettingDefault>,
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub(super) struct SettingsSection {
title: String,
items: Vec<SettingItem>,
}
#[derive(Serialize)]
struct ChoiceSummary<'a> {
id: &'a str,
name: &'a str,
default: &'a str,
options: Vec<OptionSummary<'a>>,
}
#[derive(Serialize)]
struct OptionSummary<'a> {
id: &'a str,
name: &'a str,
}
fn check_text(text: &str, max: usize, what: &str) -> Result<(), String> {
if text.trim().is_empty() || text.len() > max {
return Err(format!("settings: {what} must have 1 to {max} characters"));
}
Ok(())
}
fn range(option: SettingOption) -> Option<(u8, u8)> {
match option {
SettingOption::Saturation => Some((0, 20)),
SettingOption::Opacity => Some((10, 100)),
SettingOption::IconSize => Some((12, 32)),
SettingOption::RowHeight => Some((16, 48)),
_ => None,
}
}
fn is_color(text: &str) -> bool {
text.len() == 7 && text.starts_with('#') && text[1..].chars().all(|c| c.is_ascii_hexdigit())
}
fn is_flag(option: SettingOption) -> bool {
matches!(
option,
SettingOption::HideFiles
| SettingOption::HideFolders
| SettingOption::HideUiIcons
| SettingOption::HidePsiIcons
| SettingOption::HollowFolders
)
}
impl SettingItem {
fn validate(&self, ext: &Extensions) -> Result<(), String> {
let named = !matches!(self.option, SettingOption::Groups | SettingOption::Choice);
match &self.label {
Some(label) => check_text(label, MAX_TEXT_LEN, "label")?,
None if named => return Err("settings: label is required".into()),
None => {}
}
if let Some(hint) = &self.hint {
check_text(hint, MAX_HINT_LEN, "hint")?;
}
if self.invert && !is_flag(self.option) {
return Err("settings: invert is only for on/off options".into());
}
if self.columns.is_some_and(|c| c == 0 || c > MAX_COLUMNS)
|| (self.columns.is_some() && self.option != SettingOption::Groups)
{
return Err(format!("settings: columns are 1 to {MAX_COLUMNS}, for groups only"));
}
match (self.option, &self.choice) {
(SettingOption::Choice, Some(id)) if !ext.choices.iter().any(|c| &c.id == id) => {
return Err(format!("settings: unknown choice {id:?}"));
}
(SettingOption::Choice, None) => return Err("settings: choice items name their choice".into()),
(SettingOption::Choice, Some(_)) | (_, None) => {}
(_, Some(_)) => return Err("settings: only choice items name a choice".into()),
}
if self.option == SettingOption::Accent && !ext.accent {
return Err("settings: the accent option needs \"accent\": true".into());
}
self.validate_default()
}
fn validate_default(&self) -> Result<(), String> {
let colored = matches!(self.option, SettingOption::Accent | SettingOption::Monochrome);
match (range(self.option), &self.default) {
(Some((min, max)), Some(SettingDefault::Number(n))) if (min..=max).contains(n) => Ok(()),
(Some((min, max)), _) => Err(format!("settings: a default from {min} to {max} is required")),
(None, Some(SettingDefault::Color(c))) if colored && is_color(c) => Ok(()),
(None, _) if colored => Err("settings: a #rrggbb default is required".into()),
(None, None) => Ok(()),
(None, Some(_)) => Err("settings: this option takes no default".into()),
}
}
}
pub(super) fn validate(sections: &[SettingsSection], ext: &Extensions) -> Result<(), String> {
if sections.len() > MAX_SECTIONS {
return Err(format!("settings: more than {MAX_SECTIONS} sections"));
}
for section in sections {
check_text(&section.title, MAX_TEXT_LEN, "title")?;
if section.items.len() > MAX_ITEMS {
return Err(format!("settings: more than {MAX_ITEMS} items in {:?}", section.title));
}
for item in &section.items {
item.validate(ext)?;
}
}
Ok(())
}
pub(super) fn summary(sections: &[SettingsSection], ext: &Extensions) -> serde_json::Value {
let choices: Vec<ChoiceSummary> = ext
.choices
.iter()
.map(|c| ChoiceSummary {
id: &c.id,
name: &c.name,
default: &c.default,
options: c.options.iter().map(|o| OptionSummary { id: &o.id, name: &o.name }).collect(),
})
.collect();
serde_json::json!({
"groups": ext.groups,
"choices": choices,
"accent": ext.accent,
"settings": sections,
})
}
+9 -238
View File
@@ -24,8 +24,6 @@
//! asset; the client fetches the mapping and the icons only when the theme is
//! selected, so installed icon themes cost nothing at start-up.
use super::icon_extensions::{Choice, DARK_SUFFIX, Extensions, Group, IconDefinition, Pattern, check_icon_id};
use super::icon_settings::{self, SettingsSection};
use ide_extension::{Package, UiDecl, check_relative, decode, validate_item_id};
use serde::Deserialize;
use std::collections::{BTreeMap, BTreeSet};
@@ -47,7 +45,7 @@ struct Decl {
#[derive(Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
struct ThemeFile {
icon_definitions: BTreeMap<String, IconDefinition>,
icon_definitions: BTreeMap<String, PathBuf>,
#[serde(default)]
file: Option<String>,
#[serde(default)]
@@ -55,25 +53,11 @@ struct ThemeFile {
#[serde(default)]
folder_expanded: Option<String>,
#[serde(default)]
folder_open: Option<String>,
#[serde(default)]
file_extensions: BTreeMap<String, String>,
#[serde(default)]
file_names: BTreeMap<String, String>,
#[serde(default)]
folder_names: BTreeMap<String, String>,
#[serde(default)]
patterns: Vec<Pattern>,
#[serde(default)]
overrides: BTreeMap<String, String>,
#[serde(default)]
groups: Vec<Group>,
#[serde(default)]
choices: Vec<Choice>,
#[serde(default)]
accent: bool,
#[serde(default)]
settings: Vec<SettingsSection>,
}
/// Asset id of a theme's mapping (JSON).
@@ -81,10 +65,6 @@ pub fn map_asset(theme: &str) -> String {
format!("iconTheme/{theme}/map")
}
pub fn settings_asset(theme: &str) -> String {
format!("iconTheme/{theme}/settings")
}
/// Asset id prefix of a theme's SVGs (`…/svg/<icon id>`).
pub fn svg_prefix(theme: &str) -> String {
format!("iconTheme/{theme}/svg/")
@@ -107,20 +87,12 @@ pub(super) fn icon_theme(pkg: &Package, decl: toml::Value) -> Result<UiDecl, Str
return Err(format!("icon theme {:?} has an empty label", d.id));
}
let src = pkg.read_text(&d.path)?;
let mut theme: ThemeFile = serde_json::from_str(&src).map_err(|e| format!("{}: {e}", d.path.display()))?;
let extensions = Extensions {
patterns: std::mem::take(&mut theme.patterns),
overrides: std::mem::take(&mut theme.overrides),
groups: std::mem::take(&mut theme.groups),
choices: std::mem::take(&mut theme.choices),
accent: theme.accent,
};
let theme: ThemeFile = serde_json::from_str(&src).map_err(|e| format!("{}: {e}", d.path.display()))?;
let at = |what: &str| format!("{} ({what})", d.path.display());
if theme.icon_definitions.len() > MAX_ICONS {
return Err(format!("{}: more than {MAX_ICONS} icons", d.path.display()));
}
let mappings =
theme.file_extensions.len() + theme.file_names.len() + theme.folder_names.len() + extensions.mappings();
let mappings = theme.file_extensions.len() + theme.file_names.len() + theme.folder_names.len();
if mappings > MAX_MAPPINGS {
return Err(format!("{}: more than {MAX_MAPPINGS} mappings", d.path.display()));
}
@@ -128,17 +100,7 @@ pub(super) fn icon_theme(pkg: &Package, decl: toml::Value) -> Result<UiDecl, Str
let check_ref = |icon: &str, what: &str| -> Result<(), String> {
if defined.contains(icon) { Ok(()) } else { Err(format!("{}: unknown icon {icon:?}", at(what))) }
};
for icon in theme.icon_definitions.keys() {
check_icon_id(icon)?;
}
extensions.validate(&check_ref)?;
icon_settings::validate(&theme.settings, &extensions).map_err(|e| format!("{}: {e}", d.path.display()))?;
for (what, icon) in [
("file", &theme.file),
("folder", &theme.folder),
("folderExpanded", &theme.folder_expanded),
("folderOpen", &theme.folder_open),
] {
for (what, icon) in [("file", &theme.file), ("folder", &theme.folder), ("folderExpanded", &theme.folder_expanded)] {
if let Some(icon) = icon {
check_ref(icon, what)?;
}
@@ -163,31 +125,20 @@ pub(super) fn icon_theme(pkg: &Package, decl: toml::Value) -> Result<UiDecl, Str
let mut assets = Vec::with_capacity(theme.icon_definitions.len() + 1);
let mut files = vec![d.path.clone()];
for (icon, def) in &theme.icon_definitions {
let dark = def.dark().map(|path| (path, DARK_SUFFIX));
for (path, suffix) in std::iter::once((def.light(), "")).chain(dark) {
let clean = read_icon(pkg, path)?;
assets.push((format!("{}{icon}{suffix}", svg_prefix(&d.id)), clean));
files.push(path.clone());
}
for (icon, path) in &theme.icon_definitions {
validate_item_id(icon, "icon id")?;
let clean = read_icon(pkg, path)?;
assets.push((format!("{}{icon}", svg_prefix(&d.id)), clean));
files.push(path.clone());
}
let map = serde_json::json!({
"file": theme.file,
"folder": theme.folder,
"folderExpanded": theme.folder_expanded,
"folderOpen": theme.folder_open,
"fileExtensions": file_extensions,
"fileNames": file_names,
"folderNames": folder_names,
"patterns": extensions.patterns,
"overrides": extensions.overrides,
"groups": extensions.groups,
"choices": extensions.choices,
"accent": extensions.accent,
});
if !theme.settings.is_empty() {
assets.push((settings_asset(&d.id), icon_settings::summary(&theme.settings, &extensions).to_string()));
}
assets.push((map_asset(&d.id), map.to_string()));
let json = serde_json::json!({ "id": d.id, "label": d.label, "icons": theme.icon_definitions.len() });
let mut out = UiDecl::new(d.id.clone(), d.label, json);
@@ -661,184 +612,4 @@ mod tests {
assert!(icon_theme(&pkg, decl("evil.json")).unwrap_err().contains("evil.svg"));
assert!(icon_theme(&pkg, decl("../x.json")).is_err());
}
const SVG: &str = r#"<svg viewBox="0 0 16 16"><path d="M0 0h16v16z"/></svg>"#;
fn theme_package(theme: &str) -> (tempfile::TempDir, Package) {
let tmp = tempfile::tempdir().unwrap();
let dir = tmp.path();
std::fs::create_dir_all(dir.join("icons")).unwrap();
for icon in ["file", "folder", "folder-src", "folder-src-open", "rust", "rust-dark", "tests", "arrow"] {
std::fs::write(dir.join(format!("icons/{icon}.svg")), SVG).unwrap();
}
std::fs::write(dir.join("theme.json"), theme).unwrap();
let manifest = ide_extension::Manifest::parse(
"id = \"p\"\nname = \"P\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"",
)
.unwrap();
let pkg = Package::new(manifest, dir.to_path_buf());
(tmp, pkg)
}
fn load(theme: &str) -> Result<UiDecl, String> {
let (_tmp, pkg) = theme_package(theme);
let mut t = toml::Table::new();
t.insert("id".into(), "m".into());
t.insert("label".into(), "M".into());
t.insert("path".into(), "theme.json".into());
icon_theme(&pkg, toml::Value::Table(t))
}
fn with(extra: &str) -> String {
format!(
r#"{{"iconDefinitions":{{"file":"icons/file.svg","folder":"icons/folder.svg",
"folder-src":"icons/folder-src.svg","folder-src-open":"icons/folder-src-open.svg",
"rust":{{"light":"icons/rust.svg","dark":"icons/rust-dark.svg"}},
"tests":"icons/tests.svg","arrow":"icons/arrow.svg"}},
"file":"file","folder":"folder"{extra}}}"#
)
}
const EXTENSIONS: &str = r#","patterns":[
{"pattern":"^.*\\.rs$","icon":"rust","ignoreCase":true},
{"pattern":"/tests?/","icon":"tests","path":true,"group":"tests"},
{"pattern":"^src$","icon":"folder-src","target":"folder","openIcon":"folder-src-open"}],
"overrides":{"chevron-right":"arrow","symbol-class":"rust"},
"groups":[{"id":"tests","name":"Tests","enabled":false}],
"choices":[{"id":"arrows","name":"Arrows","default":"plain","options":[
{"id":"plain","name":"Default"},{"id":"material","name":"Material","overrides":{"chevron-down":"arrow"}}]}],
"accent":true,"folderOpen":"folder-src-open""#;
fn map_of(d: &UiDecl) -> serde_json::Value {
serde_json::from_str(&d.assets.iter().find(|(k, _)| *k == map_asset("m")).unwrap().1).unwrap()
}
#[test]
fn theme_extensions_reach_the_mapping() {
let d = load(&with(EXTENSIONS)).unwrap();
let map = map_of(&d);
assert_eq!(
map["patterns"],
serde_json::json!([
{"pattern":"^.*\\.rs$","icon":"rust","target":"file","path":false,"ignoreCase":true,"group":null,"openIcon":null},
{"pattern":"/tests?/","icon":"tests","target":"file","path":true,"ignoreCase":false,"group":"tests","openIcon":null},
{"pattern":"^src$","icon":"folder-src","target":"folder","path":false,"ignoreCase":false,"group":null,"openIcon":"folder-src-open"}
])
);
assert_eq!(map["overrides"], serde_json::json!({"chevron-right":"arrow","symbol-class":"rust"}));
assert_eq!(map["groups"], serde_json::json!([{"id":"tests","name":"Tests","enabled":false}]));
assert_eq!(map["choices"][0]["default"], "plain");
assert_eq!(map["choices"][0]["options"][1]["overrides"], serde_json::json!({"chevron-down":"arrow"}));
assert_eq!(map["choices"][0]["options"][0]["overrides"], serde_json::json!({}));
assert_eq!(map["accent"], true);
assert_eq!(map["folderOpen"], "folder-src-open");
assert_eq!(d.json["icons"], 7);
assert!(d.assets.iter().any(|(k, _)| *k == format!("{}rust", svg_prefix("m"))));
assert!(d.assets.iter().any(|(k, _)| *k == format!("{}rust_dark", svg_prefix("m"))));
assert!(d.files.iter().any(|f| f.ends_with("icons/rust-dark.svg")));
}
#[test]
fn themes_without_extensions_keep_their_mapping() {
let map = map_of(&load(&with("")).unwrap());
assert_eq!(map["patterns"], serde_json::json!([]));
assert_eq!(map["overrides"], serde_json::json!({}));
assert_eq!(map["groups"], serde_json::json!([]));
assert_eq!(map["choices"], serde_json::json!([]));
assert_eq!(map["accent"], false);
}
#[test]
fn theme_extensions_are_validated() {
let cases = [
(r#","patterns":[{"pattern":"x","icon":"nope"}]"#, "unknown icon"),
(r#","patterns":[{"pattern":"(a+)+$","icon":"rust"}]"#, "repeated group"),
(r#","patterns":[{"pattern":"(","icon":"rust"}]"#, "invalid pattern"),
(r#","patterns":[{"pattern":"x","icon":"rust","group":"nope"}]"#, "unknown group"),
(r#","patterns":[{"pattern":"x","icon":"rust","openIcon":"tests"}]"#, "openIcon"),
(r#","patterns":[{"pattern":"x","icon":"folder","target":"folder","openIcon":"nope"}]"#, "unknown icon"),
(r#","patterns":[{"pattern":"x","icon":"rust","priority":1}]"#, "unknown field"),
(r#","overrides":{"chevron-right":"nope"}"#, "unknown icon"),
(r#","folderOpen":"nope""#, "unknown icon"),
(r#","overrides":{"Chevron":"rust"}"#, "invalid overridden icon"),
(r#","groups":[{"id":"a","name":" "}]"#, "name"),
(r#","groups":[{"id":"a","name":"A"},{"id":"a","name":"B"}]"#, "duplicate"),
(r#","choices":[{"id":"c","name":"C","default":"x","options":[{"id":"y","name":"Y"}]}]"#, "default"),
(r#","choices":[{"id":"c","name":"C","default":"y","options":[]}]"#, "options"),
(
r#","choices":[{"id":"c","name":"C","default":"y","options":[{"id":"y","name":"Y","overrides":{"a":"nope"}}]}]"#,
"unknown icon",
),
];
for (extra, needle) in cases {
let err = load(&with(extra)).err().unwrap_or_else(|| panic!("accepted: {extra}"));
assert!(err.contains(needle), "{extra}: {err}");
}
let dark_id = r#"{"iconDefinitions":{"x_dark":"icons/rust.svg"}}"#;
assert!(load(dark_id).unwrap_err().contains("_dark"));
}
const SETTINGS: &str = r##","settings":[
{"title":"Toggle","items":[
{"option":"hideFiles","label":"Enable file icons","hint":"Files","invert":true},
{"option":"hollowFolders","label":"Hollow"}]},
{"title":"Packs","items":[{"option":"groups","columns":2}]},
{"title":"Other","items":[
{"option":"choice","choice":"arrows"},
{"option":"accent","label":"Accent","default":"#009688"},
{"option":"saturation","label":"Saturation","default":10}]}]"##;
fn settings_of(d: &UiDecl) -> Option<serde_json::Value> {
d.assets.iter().find(|(k, _)| *k == settings_asset("m")).map(|(_, v)| serde_json::from_str(v).unwrap())
}
#[test]
fn theme_settings_get_their_own_asset() {
let d = load(&with(&format!("{EXTENSIONS}{SETTINGS}"))).unwrap();
let s = settings_of(&d).unwrap();
assert_eq!(s["accent"], true);
assert_eq!(s["groups"], serde_json::json!([{"id":"tests","name":"Tests","enabled":false}]));
assert_eq!(
s["choices"],
serde_json::json!([{"id":"arrows","name":"Arrows","default":"plain","options":[
{"id":"plain","name":"Default"},{"id":"material","name":"Material"}]}])
);
assert_eq!(s["settings"][0]["title"], "Toggle");
assert_eq!(
s["settings"][0]["items"][0],
serde_json::json!({"option":"hideFiles","label":"Enable file icons","hint":"Files","invert":true,
"choice":null,"columns":null,"default":null})
);
assert_eq!(s["settings"][2]["items"][1]["default"], "#009688");
assert_eq!(s["settings"][2]["items"][2]["default"], 10);
assert!(settings_of(&load(&with(EXTENSIONS)).unwrap()).is_none());
}
#[test]
fn theme_settings_are_validated() {
let section = |items: &str| with(&format!(r#"{EXTENSIONS},"settings":[{{"title":"T","items":[{items}]}}]"#));
let cases = [
(section(r#"{"option":"nope","label":"X"}"#), "unknown variant"),
(section(r#"{"option":"hideFiles"}"#), "label"),
(section(r#"{"option":"hideFiles","label":" "}"#), "label"),
(section(r#"{"option":"choice","choice":"nope"}"#), "unknown choice"),
(section(r#"{"option":"choice"}"#), "choice"),
(section(r#"{"option":"groups","columns":9}"#), "columns"),
(section(r#"{"option":"saturation","label":"S"}"#), "default"),
(section(r#"{"option":"saturation","label":"S","default":50}"#), "default"),
(section(r##"{"option":"opacity","label":"O","default":"#000000"}"##), "default"),
(section(r#"{"option":"monochrome","label":"M","default":"red"}"#), "default"),
(section(r#"{"option":"saturation","label":"S","default":5,"invert":true}"#), "invert"),
(section(r#"{"option":"hideFiles","label":"F","extra":1}"#), "unknown field"),
(with(&format!(r#"{EXTENSIONS},"settings":[{{"title":"","items":[]}}]"#)), "title"),
(
with(r##","settings":[{"title":"T","items":[{"option":"accent","label":"A","default":"#000000"}]}]"##),
"accent",
),
];
for (theme, needle) in cases {
let err = load(&theme).err().unwrap_or_else(|| panic!("accepted: {theme}"));
assert!(err.contains(needle), "{theme}: {err}");
}
}
}
-874
View File
@@ -1,874 +0,0 @@
//! IntelliJ theme plugins, read for import as OpenIDEA theme plugins: a
//! Marketplace download (a `.jar`, or a `.zip` holding the plugin's
//! `lib/*.jar`), an installed plugin's folder or a loose `.theme.json`; and the
//! theme plugins of the JetBrains IDEs on this machine, with the theme each IDE
//! is set to (`options/laf.xml`).
//!
//! Nothing is converted here: the UI maps IntelliJ keys onto design tokens
//! (`src/features/plugins/intellijTheme.ts`, shared with `pnpm
//! theme:import-intellij`) and [`super::PluginHost::stage_package`] writes the
//! result for the install page, which validates it like any package.
use ide_core::{IdeError, IdeResult, IpcPath};
use quick_xml::{Reader, XmlVersion, events::Event};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::ffi::OsString;
use std::io::{Cursor, Read, Seek};
use std::path::{Path, PathBuf};
/// A descriptor, theme or editor scheme larger than this is not one.
const MAX_FILE: u64 = 4 << 20;
/// A plugin jar read into memory from a Marketplace zip.
const MAX_JAR: u64 = 64 << 20;
/// Jars of one plugin looked into.
const MAX_JARS: usize = 64;
/// All the jars read into memory from one Marketplace zip.
const MAX_NESTED: u64 = 256 << 20;
/// Descriptors read for one plugin: `plugin.xml` and those it includes.
const MAX_DESCRIPTORS: usize = 64;
const PLUGIN_XML: &str = "META-INF/plugin.xml";
/// A theme plugin's themes, as read from its files.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct IntellijThemeSource {
/// The plugin's `<name>`, or the theme's for a loose file.
pub name: String,
pub plugin_id: Option<String>,
pub version: Option<String>,
pub vendor: Option<String>,
pub themes: Vec<IntellijThemeFile>,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct IntellijThemeFile {
/// The `themeProvider` id (what `laf.xml` names), or the file name without `.theme.json`.
pub id: String,
/// The `.theme.json`, comments removed.
pub theme: String,
/// The editor scheme (`.icls` XML) its `editorScheme` names, when the plugin has it.
pub editor_scheme: Option<String>,
}
/// A theme of a plugin installed in a JetBrains IDE.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct InstalledIntellijTheme {
/// The IDE and its version: `IntelliJ IDEA 2025.2`.
pub ide: String,
/// The plugin's name.
pub plugin: String,
/// What to import: the plugin's folder or jar.
pub path: IpcPath,
pub theme_id: String,
pub theme_name: String,
pub dark: bool,
/// The IDE is set to this theme.
pub active: bool,
}
/// Reads the themes of an IntelliJ theme plugin (a `.jar`, a Marketplace
/// `.zip`, an installed plugin's folder) or of a `.theme.json`.
pub fn read_source(path: &Path) -> IdeResult<IntellijThemeSource> {
read(path).map_err(IdeError::invalid)
}
/// The theme plugins of the JetBrains IDEs (and Android Studio) of this user,
/// the themes their IDEs are set to first.
pub fn installed_themes() -> Vec<InstalledIntellijTheme> {
let Some(home) = dirs::home_dir() else { return Vec::new() };
let layout = if cfg!(target_os = "macos") {
Layout::Mac
} else if cfg!(windows) {
Layout::Windows
} else {
Layout::Xdg
};
find_installed(&ide_dirs(layout, &home, |key| std::env::var_os(key)))
}
fn read(path: &Path) -> Result<IntellijThemeSource, String> {
if path.extension().is_some_and(|e| e.eq_ignore_ascii_case("json")) {
return loose_theme(path);
}
let mut jars = plugin_jars(path)?;
let plugin = descriptor(&mut jars)?;
let shown = || path.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
if plugin.themes.is_empty() {
return Err(format!("{} is not a theme plugin: its plugin.xml declares no themeProvider", shown()));
}
let mut themes = Vec::new();
let mut problem = None;
for (id, theme_path) in &plugin.themes {
match plugin_theme(&mut jars, id, theme_path) {
Ok(theme) => themes.push(theme),
Err(e) => problem = problem.or(Some(e)),
}
}
if themes.is_empty() {
return Err(problem.unwrap_or_else(|| format!("{} has no readable theme", shown())));
}
Ok(IntellijThemeSource {
name: plugin.name.unwrap_or_else(shown),
plugin_id: plugin.id,
version: plugin.version,
vendor: plugin.vendor,
themes,
})
}
fn plugin_theme(jars: &mut [Jar], id: &str, path: &str) -> Result<IntellijThemeFile, String> {
let bytes = find(jars, entry_name(path), MAX_FILE)?.ok_or_else(|| format!("{path} is missing from the plugin"))?;
let (theme, fields) = theme_json(&bytes).map_err(|e| format!("{path}: {e}"))?;
let editor_scheme = match string_field(&fields, "editorScheme") {
Some(scheme) => find(jars, entry_name(scheme), MAX_FILE)?.map(|b| text(&b)),
None => None,
};
Ok(IntellijThemeFile { id: id.to_owned(), theme, editor_scheme })
}
/// A `.theme.json` with the editor scheme beside it (its `editorScheme` names a path in the plugin's jar).
fn loose_theme(path: &Path) -> Result<IntellijThemeSource, String> {
let (theme, fields) = theme_json(&read_file(path)?).map_err(|e| format!("{}: {e}", path.display()))?;
// Any JSON object parses: an IntelliJ theme has UI colours or an editor scheme.
let scheme_ref = string_field(&fields, "editorScheme");
if !fields.get("ui").is_some_and(serde_json::Value::is_object) && scheme_ref.is_none() {
return Err(format!("{} is not an IntelliJ theme: it has no \"ui\" or \"editorScheme\"", path.display()));
}
let editor_scheme = scheme_ref
.and_then(|p| Path::new(p).file_name().map(|n| path.with_file_name(n)))
.filter(|p| p.is_file())
.map(|p| read_file(&p).map(|b| text(&b)))
.transpose()?;
let file = path.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
let id = file.strip_suffix(".theme.json").or_else(|| file.strip_suffix(".json")).unwrap_or(&file).to_owned();
Ok(IntellijThemeSource {
name: string_field(&fields, "name").map_or_else(|| id.clone(), str::to_owned),
plugin_id: None,
version: None,
vendor: string_field(&fields, "author").map(str::to_owned),
themes: vec![IntellijThemeFile { id, theme, editor_scheme }],
})
}
type JsonObject = serde_json::Map<String, serde_json::Value>;
/// The theme JSON, comments removed, and its fields, if it is an object.
fn theme_json(bytes: &[u8]) -> Result<(String, JsonObject), String> {
let json = super::theme::strip_json_comments(&text(bytes));
match serde_json::from_str::<serde_json::Value>(&json) {
Ok(serde_json::Value::Object(fields)) => Ok((json, fields)),
Ok(_) => Err("not a theme: expected a JSON object".into()),
Err(e) => Err(format!("invalid theme JSON: {e}")),
}
}
fn string_field<'a>(fields: &'a JsonObject, key: &str) -> Option<&'a str> {
fields.get(key)?.as_str().filter(|s| !s.trim().is_empty())
}
/// `/themes/x.theme.json` → `themes/x.theme.json`.
fn entry_name(path: &str) -> &str {
path.trim().trim_start_matches('/')
}
fn text(bytes: &[u8]) -> String {
let bytes = bytes.strip_prefix(b"\xEF\xBB\xBF").unwrap_or(bytes);
String::from_utf8_lossy(bytes).into_owned()
}
fn read_file(path: &Path) -> Result<Vec<u8>, String> {
let shown = |e: std::io::Error| format!("{}: {e}", path.display());
let file = std::fs::File::open(path).map_err(shown)?;
let mut buf = Vec::new();
file.take(MAX_FILE + 1).read_to_end(&mut buf).map_err(shown)?;
if buf.len() as u64 > MAX_FILE {
return Err(format!("{} is too large for a theme", path.display()));
}
Ok(buf)
}
trait ReadSeek: Read + Seek {}
impl<T: Read + Seek> ReadSeek for T {}
type Jar = zip::ZipArchive<Box<dyn ReadSeek>>;
fn open_jar(path: &Path) -> Result<Jar, String> {
let file = std::fs::File::open(path).map_err(|e| format!("{}: {e}", path.display()))?;
let reader: Box<dyn ReadSeek> = Box::new(std::io::BufReader::new(file));
zip::ZipArchive::new(reader).map_err(|e| format!("{} is not a jar or zip file ({e})", path.display()))
}
/// The jars of the plugin at `path`: a jar, a Marketplace zip (`<plugin>/lib/*.jar`)
/// or an installed plugin's folder (`lib/*.jar`).
fn plugin_jars(path: &Path) -> Result<Vec<Jar>, String> {
let not_plugin = || format!("{} is not an IntelliJ plugin: no {PLUGIN_XML} and no lib/*.jar", path.display());
if path.is_dir() {
let mut found: Vec<PathBuf> = [path.join("lib"), path.join("lib").join("modules")]
.iter()
.flat_map(|dir| std::fs::read_dir(dir).into_iter().flatten().flatten())
.map(|e| e.path())
.filter(|p| p.is_file() && is_jar(p))
.collect();
found.sort();
let jars: Vec<Jar> = found.iter().take(MAX_JARS).filter_map(|p| open_jar(p).ok()).collect();
return if jars.is_empty() { Err(not_plugin()) } else { Ok(jars) };
}
let mut archive = open_jar(path)?;
if archive.index_for_name(PLUGIN_XML).is_some() {
return Ok(vec![archive]);
}
let jars = nested_jars(&mut archive, MAX_JAR, MAX_NESTED);
if jars.is_empty() { Err(not_plugin()) } else { Ok(jars) }
}
/// The `lib` jars of a Marketplace zip, smallest first (a theme's own jar is small)
/// while they fit `max_total`; library jars too large or not jars at all are skipped.
fn nested_jars(archive: &mut Jar, max_jar: u64, max_total: u64) -> Vec<Jar> {
let mut nested: Vec<(u64, String)> = entry_names(archive)
.into_iter()
.filter(|n| is_lib_jar(n))
.filter_map(|n| Some((archive.by_name(&n).ok()?.size(), n)))
.filter(|(size, _)| *size <= max_jar)
.collect();
nested.sort();
let (mut jars, mut loaded) = (Vec::new(), 0u64);
for (size, name) in nested.into_iter().take(MAX_JARS) {
if loaded.saturating_add(size) > max_total {
break;
}
let Ok(Some(bytes)) = read_entry(archive, &name, max_jar) else { continue };
loaded += size;
let reader: Box<dyn ReadSeek> = Box::new(Cursor::new(bytes));
if let Ok(jar) = zip::ZipArchive::new(reader) {
jars.push(jar);
}
}
jars
}
fn entry_names(jar: &Jar) -> Vec<String> {
jar.file_names().filter_map(Result::ok).map(|n| n.into_owned()).collect()
}
fn is_jar(path: &Path) -> bool {
path.extension().is_some_and(|e| e.eq_ignore_ascii_case("jar"))
}
/// `lib/x.jar`, `<plugin>/lib/x.jar` or `<plugin>/lib/modules/x.jar` in a Marketplace zip.
fn is_lib_jar(name: &str) -> bool {
let parts: Vec<&str> = name.split('/').collect();
let n = parts.len();
name.to_ascii_lowercase().ends_with(".jar")
&& (2..=4).contains(&n)
&& (parts[n - 2] == "lib" || (n >= 3 && parts[n - 2] == "modules" && parts[n - 3] == "lib"))
}
fn read_entry(jar: &mut Jar, name: &str, max: u64) -> Result<Option<Vec<u8>>, String> {
let Ok(file) = jar.by_name(name) else { return Ok(None) };
if file.is_dir() {
return Ok(None);
}
if file.size() > max {
return Err(format!("{name} is too large"));
}
let mut buf = Vec::with_capacity(file.size() as usize);
file.take(max.saturating_add(1)).read_to_end(&mut buf).map_err(|e| format!("{name}: {e}"))?;
if buf.len() as u64 > max {
return Err(format!("{name} is too large"));
}
Ok(Some(buf))
}
fn find(jars: &mut [Jar], name: &str, max: u64) -> Result<Option<Vec<u8>>, String> {
for jar in jars.iter_mut() {
if let Some(bytes) = read_entry(jar, name, max)? {
return Ok(Some(bytes));
}
}
Ok(None)
}
/// What the plugin's descriptors say: `<idea-plugin>`'s `id`, `name`, `version`,
/// `vendor`, and its `themeProvider`s (id, path in the jar).
#[derive(Debug, Default, PartialEq)]
struct Descriptor {
id: Option<String>,
name: Option<String>,
version: Option<String>,
vendor: Option<String>,
themes: Vec<(String, String)>,
}
/// Reads `META-INF/plugin.xml` and the descriptors it includes (`<xi:include href>`), in any jar.
fn descriptor(jars: &mut [Jar]) -> Result<Descriptor, String> {
let main = find(jars, PLUGIN_XML, MAX_FILE)
.ok()
.flatten()
.ok_or_else(|| format!("not an IntelliJ plugin: no {PLUGIN_XML}"))?;
let mut out = Descriptor::default();
let mut includes = Vec::new();
parse_descriptor(&text(&main), true, &mut out, &mut includes).map_err(|e| format!("{PLUGIN_XML}: {e}"))?;
let mut read = std::collections::HashSet::from([PLUGIN_XML.to_owned()]);
while let Some(href) = includes.pop() {
let name = include_entry(&href);
if read.len() >= MAX_DESCRIPTORS || !read.insert(name.clone()) {
continue;
}
let Ok(Some(bytes)) = find(jars, &name, MAX_FILE) else { continue };
let _ = parse_descriptor(&text(&bytes), false, &mut out, &mut includes);
}
let mut seen = std::collections::HashSet::new();
out.themes.retain(|(id, _)| seen.insert(id.clone()));
Ok(out)
}
/// An `<xi:include href>` as a jar entry: `/META-INF/x.xml` from the jar's root, `x.xml` beside plugin.xml.
fn include_entry(href: &str) -> String {
let href = href.trim();
href.strip_prefix('/').map_or_else(|| format!("META-INF/{href}"), str::to_owned)
}
fn parse_descriptor(xml: &str, main: bool, out: &mut Descriptor, includes: &mut Vec<String>) -> Result<(), String> {
let mut reader = Reader::from_str(xml);
let mut stack: Vec<String> = Vec::new();
let mut text = String::new();
loop {
let event = reader.read_event().map_err(|e| e.to_string())?;
match event {
Event::Start(ref tag) | Event::Empty(ref tag) => {
let name = tag.local_name().as_ref().to_owned();
if name == "include" {
let href = tag.attributes().flatten().find(|a| a.key.local_name().as_ref() == "href");
if let Some(href) = href.and_then(|a| a.normalized_value(XmlVersion::Implicit1_0).ok()) {
includes.push(href.into_owned());
}
}
if name == "themeProvider" {
let (mut id, mut path) = (None, None);
for a in tag.attributes() {
let a = a.map_err(|e| e.to_string())?;
let value = a.normalized_value(XmlVersion::Implicit1_0).map_err(|e| e.to_string())?;
match a.key.local_name().as_ref() {
"id" => id = Some(value.trim().to_owned()),
"path" => path = Some(value.trim().to_owned()),
_ => {}
}
}
if let (Some(id), Some(path)) = (id, path)
&& !id.is_empty()
&& !path.is_empty()
{
out.themes.push((id, path));
}
}
if matches!(event, Event::Start(_)) {
stack.push(name);
text.clear();
}
}
Event::Text(t) => text.push_str(&t.xml_content(XmlVersion::Implicit1_0)),
Event::CData(t) => text.push_str(t.as_ref()),
Event::GeneralRef(r) => {
let name = r.as_ref();
let escaped = format!("&{name};");
text.push_str(&quick_xml::escape::unescape(&escaped).map_err(|e| e.to_string())?);
}
Event::End(_) => {
let name = stack.pop().ok_or("unbalanced element")?;
// Direct children of <idea-plugin>.
if main && stack.len() == 1 && stack[0] == "idea-plugin" {
let value = text.trim();
let slot = match name.as_str() {
"id" => Some(&mut out.id),
"name" => Some(&mut out.name),
"version" => Some(&mut out.version),
"vendor" => Some(&mut out.vendor),
_ => None,
};
if let Some(slot) = slot
&& slot.is_none()
&& !value.is_empty()
{
*slot = Some(value.to_owned());
}
}
text.clear();
}
Event::Eof => return if stack.is_empty() { Ok(()) } else { Err("unclosed element".into()) },
_ => {}
}
}
}
/// Where an OS keeps the IDEs' settings and plugins.
#[derive(Clone, Copy, Debug)]
enum Layout {
/// `~/Library/Application Support/<vendor>/<IDE>`, plugins in its `plugins/`.
Mac,
/// `%APPDATA%\<vendor>\<IDE>`, plugins in its `plugins\`.
Windows,
/// `$XDG_CONFIG_HOME/<vendor>/<IDE>`, plugins in `$XDG_DATA_HOME/<vendor>/<IDE>`.
Xdg,
}
#[derive(Debug, PartialEq)]
struct IdeDir {
label: String,
product: String,
version: (u32, u32),
config: PathBuf,
plugins: PathBuf,
}
/// The settings folders of JetBrains IDEs and Android Studio (2020.1 and later), newest first.
fn ide_dirs(layout: Layout, home: &Path, env: impl Fn(&str) -> Option<OsString>) -> Vec<IdeDir> {
let var = |key: &str| env(key).filter(|v| !v.is_empty()).map(PathBuf::from);
let (config_root, data_root) = match layout {
Layout::Mac => {
let root = home.join("Library").join("Application Support");
(root.clone(), root)
}
Layout::Windows => {
let root = var("APPDATA").unwrap_or_else(|| home.join("AppData").join("Roaming"));
(root.clone(), root)
}
Layout::Xdg => (
var("XDG_CONFIG_HOME").unwrap_or_else(|| home.join(".config")),
var("XDG_DATA_HOME").unwrap_or_else(|| home.join(".local").join("share")),
),
};
let mut out = Vec::new();
for vendor in ["JetBrains", "Google"] {
for entry in std::fs::read_dir(config_root.join(vendor)).into_iter().flatten().flatten() {
let name = entry.file_name().to_string_lossy().into_owned();
let Some((product, version, shown)) = product_dir(&name) else { continue };
let config = entry.path();
if !config.is_dir() {
continue;
}
let plugins = match layout {
Layout::Xdg => data_root.join(vendor).join(&name),
Layout::Mac | Layout::Windows => config.join("plugins"),
};
out.push(IdeDir {
label: format!("{} {shown}", product_label(product)),
product: product.to_owned(),
version,
config,
plugins,
});
}
}
out.sort_by(|a, b| b.version.cmp(&a.version).then_with(|| a.label.cmp(&b.label)));
out
}
/// `IntelliJIdea2025.2` → (`IntelliJIdea`, (2025, 2), `2025.2`).
fn product_dir(name: &str) -> Option<(&str, (u32, u32), &str)> {
let at = name.find(|c: char| c.is_ascii_digit())?;
let (product, version) = name.split_at(at);
if product.is_empty() || !product.chars().all(|c| c.is_ascii_alphabetic()) {
return None;
}
let (year, minor) = version.split_once('.')?;
let ok = |s: &str| !s.is_empty() && s.len() <= 4 && s.chars().all(|c| c.is_ascii_digit());
if year.len() != 4 || !ok(year) || !ok(minor) {
return None;
}
Some((product, (year.parse().ok()?, minor.parse().ok()?), version))
}
fn product_label(product: &str) -> &str {
match product {
"IntelliJIdea" => "IntelliJ IDEA",
"IdeaIC" => "IntelliJ IDEA Community Edition",
"PyCharmCE" => "PyCharm Community Edition",
"AndroidStudio" => "Android Studio",
"AndroidStudioPreview" => "Android Studio Preview",
other => other,
}
}
/// The theme ids `options/laf.xml` selects: the current one, and with "Sync with OS" the preferred light and dark.
fn active_theme_ids(config: &Path) -> Vec<String> {
let Ok(bytes) = read_file(&config.join("options").join("laf.xml")) else { return Vec::new() };
let xml = text(&bytes);
let mut reader = Reader::from_str(&xml);
let (mut ids, mut preferred, mut autodetect) = (Vec::new(), Vec::new(), false);
loop {
match reader.read_event() {
Ok(Event::Start(tag) | Event::Empty(tag)) => {
let attr = |key: &str| {
tag.attributes()
.flatten()
.find(|a| a.key.as_ref() == key)
.and_then(|a| a.normalized_value(XmlVersion::Implicit1_0).ok().map(|v| v.trim().to_owned()))
};
match tag.local_name().as_ref() {
"component" if attr("name").as_deref() == Some("LafManager") => {
autodetect = attr("autodetect").as_deref() == Some("true");
}
"laf" => ids.extend(attr("themeId")),
"preferred-light-laf" | "preferred-dark-laf" => preferred.extend(attr("themeId")),
_ => {}
}
}
Ok(Event::Eof) | Err(_) => break,
_ => {}
}
}
if autodetect {
ids.extend(preferred);
}
ids.retain(|id| !id.is_empty());
ids
}
fn find_installed(ides: &[IdeDir]) -> Vec<InstalledIntellijTheme> {
let mut out = Vec::new();
for ide in ides {
let active = active_theme_ids(&ide.config);
let mut plugins: Vec<PathBuf> = std::fs::read_dir(&ide.plugins)
.into_iter()
.flatten()
.flatten()
.map(|e| e.path())
.filter(|p| p.is_dir() || (p.is_file() && is_jar(p)))
.collect();
plugins.sort();
for plugin in plugins {
let Ok(mut jars) = plugin_jars(&plugin) else { continue };
let Ok(found) = descriptor(&mut jars) else { continue };
for (id, path) in &found.themes {
let Ok(Some(bytes)) = find(&mut jars, entry_name(path), MAX_FILE) else { continue };
let Ok((_, fields)) = theme_json(&bytes) else { continue };
let file = || plugin.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
out.push(InstalledIntellijTheme {
ide: ide.label.clone(),
plugin: found.name.clone().unwrap_or_else(file),
path: plugin.clone().into(),
theme_id: id.clone(),
theme_name: string_field(&fields, "name").unwrap_or(id).to_owned(),
dark: fields.get("dark").and_then(serde_json::Value::as_bool).unwrap_or(true),
active: active.contains(id),
});
}
}
}
// Stable: the IDEs' order (newest first) within each group.
out.sort_by_key(|t| !t.active);
out
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
const PLUGIN: &str = r#"<idea-plugin>
<id>com.example.deepsea</id>
<name>Deep Sea &amp; Friends</name>
<version>2.3</version>
<vendor email="[email protected]">Someone</vendor>
<description><![CDATA[<b>Dark</b> themes]]></description>
<depends>com.intellij.modules.platform</depends>
<extensions defaultExtensionNs="com.intellij">
<themeProvider id="deep-sea-id" path="/themes/DeepSea.theme.json"/>
<themeProvider id="shallow-id" path="themes/Shallow.theme.json" />
</extensions>
</idea-plugin>"#;
const DEEP: &str = r##"{
// comment
"name": "Deep Sea",
"dark": true,
"editorScheme": "/themes/DeepSea.xml",
"ui": { "*": { "background": "#101820" } }
}"##;
const SHALLOW: &str = r#"{ "name": "Shallow", "dark": false, "ui": {} }"#;
const SCHEME: &str = r#"<scheme name="Deep Sea" version="142" parent_scheme="Darcula"/>"#;
fn jar_bytes(entries: &[(&str, &[u8])]) -> Vec<u8> {
let mut zip = zip::ZipWriter::new(Cursor::new(Vec::new()));
for (name, data) in entries {
zip.start_file(*name, zip::write::SimpleFileOptions::default()).unwrap();
zip.write_all(data).unwrap();
}
zip.finish().unwrap().into_inner()
}
fn theme_jar() -> Vec<u8> {
jar_bytes(&[
("META-INF/MANIFEST.MF", b"Manifest-Version: 1.0\n"),
(PLUGIN_XML, PLUGIN.as_bytes()),
("themes/DeepSea.theme.json", DEEP.as_bytes()),
("themes/DeepSea.xml", SCHEME.as_bytes()),
("themes/Shallow.theme.json", SHALLOW.as_bytes()),
])
}
fn write(path: &Path, bytes: &[u8]) {
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
std::fs::write(path, bytes).unwrap();
}
fn expected_source() -> IntellijThemeSource {
IntellijThemeSource {
name: "Deep Sea & Friends".into(),
plugin_id: Some("com.example.deepsea".into()),
version: Some("2.3".into()),
vendor: Some("Someone".into()),
themes: vec![
IntellijThemeFile {
id: "deep-sea-id".into(),
theme: super::super::theme::strip_json_comments(DEEP),
editor_scheme: Some(SCHEME.into()),
},
IntellijThemeFile { id: "shallow-id".into(), theme: SHALLOW.into(), editor_scheme: None },
],
}
}
#[test]
fn reads_a_theme_jar_with_its_editor_scheme() {
let tmp = tempfile::tempdir().unwrap();
let jar = tmp.path().join("deep-sea.jar");
write(&jar, &theme_jar());
let source = read_source(&jar).unwrap();
assert_eq!(source, expected_source());
assert!(!source.themes[0].theme.contains("comment"));
}
#[test]
fn reads_a_marketplace_zip_and_an_installed_folder() {
let tmp = tempfile::tempdir().unwrap();
let other = jar_bytes(&[("META-INF/MANIFEST.MF", b"x"), ("a/B.class", b"\xCA\xFE")]);
let zip = tmp.path().join("DeepSea-2.3.zip");
write(&zip, &jar_bytes(&[("DeepSea/lib/a-util.jar", &other), ("DeepSea/lib/deep-sea.jar", &theme_jar())]));
assert_eq!(read_source(&zip).unwrap(), expected_source());
let dir = tmp.path().join("plugins").join("DeepSea");
write(&dir.join("lib").join("deep-sea.jar"), &theme_jar());
write(&dir.join("lib").join("a-util.jar"), &other);
assert_eq!(read_source(&dir).unwrap(), expected_source());
}
#[test]
fn skips_library_jars_it_cannot_or_need_not_read() {
let tmp = tempfile::tempdir().unwrap();
// Stored, not deflated: as large in the zip as its content.
let mut big = zip::ZipWriter::new(Cursor::new(Vec::new()));
let stored = zip::write::SimpleFileOptions::default().compression_method(zip::CompressionMethod::Stored);
big.start_file("lib/Data.class", stored).unwrap();
big.write_all(&[b'x'; 8192]).unwrap();
let big = big.finish().unwrap().into_inner();
let zip = tmp.path().join("DeepSea.zip");
write(
&zip,
&jar_bytes(&[
("DeepSea/lib/a-broken.jar", b"not a jar"),
("DeepSea/lib/b-big.jar", &big),
("DeepSea/lib/deep-sea.jar", &theme_jar()),
]),
);
assert_eq!(read_source(&zip).unwrap(), expected_source());
// The broken jar is skipped; a jar over the size limit, or past the total
// (smallest first), is not read, while the theme's small jar still is.
let mut archive = open_jar(&zip).unwrap();
let theme_len = theme_jar().len() as u64;
assert_eq!(nested_jars(&mut archive, u64::MAX, u64::MAX).len(), 2);
let one = nested_jars(&mut archive, theme_len + 64, u64::MAX);
assert_eq!(one.len(), 1);
assert!(one[0].index_for_name(PLUGIN_XML).is_some());
assert_eq!(nested_jars(&mut archive, u64::MAX, theme_len + 64).len(), 1);
assert!(nested_jars(&mut archive, 8, u64::MAX).is_empty());
}
#[test]
fn reads_a_loose_theme_json_with_the_scheme_beside_it() {
let tmp = tempfile::tempdir().unwrap();
let theme = tmp.path().join("DeepSea.theme.json");
write(&theme, DEEP.as_bytes());
write(&tmp.path().join("DeepSea.xml"), SCHEME.as_bytes());
let source = read_source(&theme).unwrap();
assert_eq!(source.name, "Deep Sea");
assert_eq!(source.themes.len(), 1);
assert_eq!(source.themes[0].id, "DeepSea");
assert_eq!(source.themes[0].editor_scheme.as_deref(), Some(SCHEME));
std::fs::remove_file(tmp.path().join("DeepSea.xml")).unwrap();
assert_eq!(read_source(&theme).unwrap().themes[0].editor_scheme, None);
}
#[test]
fn says_what_is_wrong_with_other_files() {
let tmp = tempfile::tempdir().unwrap();
let not_zip = tmp.path().join("notes.jar");
write(&not_zip, b"hello");
assert!(read_source(&not_zip).unwrap_err().message.contains("not a jar or zip"));
let other_plugin = tmp.path().join("tools.jar");
let xml = "<idea-plugin><name>Tools</name><actions/></idea-plugin>";
write(&other_plugin, &jar_bytes(&[(PLUGIN_XML, xml.as_bytes())]));
assert!(read_source(&other_plugin).unwrap_err().message.contains("not a theme plugin"));
let package = tmp.path().join("package.zip");
write(&package, &jar_bytes(&[("plugin.toml", b"id = \"x\"")]));
assert!(read_source(&package).unwrap_err().message.contains("not an IntelliJ plugin"));
let other = tmp.path().join("package.json");
write(&other, br#"{ "name": "app", "version": "1.0.0" }"#);
assert!(read_source(&other).unwrap_err().message.contains("is not an IntelliJ theme"));
let broken = tmp.path().join("broken.theme.json");
write(&broken, b"{ \"name\": ");
assert!(read_source(&broken).unwrap_err().message.contains("invalid theme JSON"));
let missing = tmp.path().join("missing.jar");
let xml = r#"<idea-plugin><name>M</name><themeProvider id="m" path="/m.theme.json"/></idea-plugin>"#;
write(&missing, &jar_bytes(&[(PLUGIN_XML, xml.as_bytes())]));
assert!(read_source(&missing).unwrap_err().message.contains("/m.theme.json is missing"));
}
#[test]
fn finds_theme_providers_in_included_descriptors() {
let tmp = tempfile::tempdir().unwrap();
let jar = tmp.path().join("split.jar");
let main = r#"<idea-plugin xmlns:xi="http://www.w3.org/2001/XInclude"><name>Split</name>
<xi:include href="/META-INF/themes.xml"/></idea-plugin>"#;
let part = r#"<idea-plugin xmlns:xi="http://www.w3.org/2001/XInclude"><extensions defaultExtensionNs="com.intellij">
<themeProvider id="s" path="/s.theme.json"/></extensions><name>Not this</name>
<xi:include href="more.xml"/><xi:include href="/META-INF/themes.xml"/></idea-plugin>"#;
let more = r#"<idea-plugin><themeProvider id="m" path="/m.theme.json"/></idea-plugin>"#;
let unrelated = r#"<idea-plugin><themeProvider id="x" path="/x.theme.json"/></idea-plugin>"#;
write(
&jar,
&jar_bytes(&[
("META-INF/themes.xml", part.as_bytes()),
("META-INF/more.xml", more.as_bytes()),
("META-INF/unrelated.xml", unrelated.as_bytes()),
(PLUGIN_XML, main.as_bytes()),
("s.theme.json", br#"{"name":"S"}"#),
("m.theme.json", br#"{"name":"M"}"#),
("x.theme.json", br#"{"name":"X"}"#),
]),
);
let source = read_source(&jar).unwrap();
assert_eq!(source.name, "Split");
// Only what plugin.xml includes (relative or from the root, once each) counts.
assert_eq!(source.themes.iter().map(|t| t.id.as_str()).collect::<Vec<_>>(), ["s", "m"]);
}
#[test]
fn parses_ide_folder_names() {
assert_eq!(product_dir("IntelliJIdea2025.2"), Some(("IntelliJIdea", (2025, 2), "2025.2")));
assert_eq!(product_dir("PyCharmCE2023.10"), Some(("PyCharmCE", (2023, 10), "2023.10")));
assert_eq!(product_dir("AndroidStudio2024.3"), Some(("AndroidStudio", (2024, 3), "2024.3")));
for name in
["Toolbox", "consentOptions", "IntelliJIdea2025", "Idea-2025.1", "2025.1", "Rider25.1", "Go Land2025.1"]
{
assert_eq!(product_dir(name), None, "{name}");
}
}
#[test]
fn locates_the_settings_and_plugins_of_each_os() {
let tmp = tempfile::tempdir().unwrap();
let home = tmp.path();
let mk = |p: PathBuf| {
std::fs::create_dir_all(&p).unwrap();
p
};
let mac = mk(home.join("Library/Application Support/JetBrains/IntelliJIdea2025.2"));
mk(home.join("Library/Application Support/JetBrains/Toolbox"));
let dirs = ide_dirs(Layout::Mac, home, |_| None);
assert_eq!(dirs.len(), 1);
assert_eq!(dirs[0].label, "IntelliJ IDEA 2025.2");
assert_eq!((dirs[0].config.clone(), dirs[0].plugins.clone()), (mac.clone(), mac.join("plugins")));
let appdata = mk(home.join("Roaming"));
let win = mk(appdata.join("JetBrains/PyCharm2024.3"));
let studio = mk(appdata.join("Google/AndroidStudio2025.1"));
let env = |k: &str| (k == "APPDATA").then(|| appdata.clone().into_os_string());
let dirs = ide_dirs(Layout::Windows, home, env);
assert_eq!(
dirs.iter().map(|d| d.label.as_str()).collect::<Vec<_>>(),
["Android Studio 2025.1", "PyCharm 2024.3"]
);
assert_eq!(dirs[1].plugins, win.join("plugins"));
assert_eq!(dirs[0].config, studio);
let config = mk(home.join(".config/JetBrains/IdeaIC2024.1"));
mk(home.join(".config/JetBrains/GoLand2025.2"));
let dirs = ide_dirs(Layout::Xdg, home, |_| None);
assert_eq!(dirs.iter().map(|d| d.product.as_str()).collect::<Vec<_>>(), ["GoLand", "IdeaIC"]);
assert_eq!(dirs[1].label, "IntelliJ IDEA Community Edition 2024.1");
assert_eq!(dirs[1].config, config);
assert_eq!(dirs[1].plugins, home.join(".local/share/JetBrains/IdeaIC2024.1"));
let xdg_data = home.join("data");
let env = |k: &str| (k == "XDG_DATA_HOME").then(|| xdg_data.clone().into_os_string());
assert_eq!(ide_dirs(Layout::Xdg, home, env)[1].plugins, xdg_data.join("JetBrains/IdeaIC2024.1"));
}
#[test]
fn reads_the_selected_theme_from_laf_xml() {
let tmp = tempfile::tempdir().unwrap();
let laf = tmp.path().join("options/laf.xml");
write(
&laf,
br#"<application>
<component name="LafManager">
<laf class-name="com.intellij.ide.ui.laf.UIThemeBasedLookAndFeelInfo" themeId="deep-sea-id" />
<preferred-light-laf themeId="shallow-id" />
</component>
</application>"#,
);
assert_eq!(active_theme_ids(tmp.path()), ["deep-sea-id"]);
let synced = std::fs::read_to_string(&laf)
.unwrap()
.replace(r#"name="LafManager""#, r#"name="LafManager" autodetect="true""#);
write(&laf, synced.as_bytes());
assert_eq!(active_theme_ids(tmp.path()), ["deep-sea-id", "shallow-id"]);
assert!(active_theme_ids(&tmp.path().join("missing")).is_empty());
}
#[test]
fn lists_installed_theme_plugins_the_ides_current_first() {
let tmp = tempfile::tempdir().unwrap();
let config = tmp.path().join("IntelliJIdea2025.2");
let plugins = config.join("plugins");
write(&plugins.join("DeepSea/lib/deep-sea.jar"), &theme_jar());
write(
&plugins.join("Tools/lib/tools.jar"),
&jar_bytes(&[(PLUGIN_XML, b"<idea-plugin><name>T</name></idea-plugin>")]),
);
write(&plugins.join("broken.jar"), b"not a jar");
write(
&config.join("options/laf.xml"),
br#"<application><component name="LafManager"><laf themeId="shallow-id"/></component></application>"#,
);
let ide = IdeDir {
label: "IntelliJ IDEA 2025.2".into(),
product: "IntelliJIdea".into(),
version: (2025, 2),
config: config.clone(),
plugins: plugins.clone(),
};
let found = find_installed(&[ide]);
let theme = |id: &str, name: &str, dark: bool, active: bool| InstalledIntellijTheme {
ide: "IntelliJ IDEA 2025.2".into(),
plugin: "Deep Sea & Friends".into(),
path: plugins.join("DeepSea").into(),
theme_id: id.into(),
theme_name: name.into(),
dark,
active,
};
assert_eq!(found, [theme("shallow-id", "Shallow", false, true), theme("deep-sea-id", "Deep Sea", true, false)]);
assert!(find_installed(&[]).is_empty());
}
}
+22 -171
View File
@@ -3,14 +3,11 @@
//! plugins, and revocations. The host keeps the last index it read
//! (`market-index.json`), checks signatures against it, and disables what it
//! revokes — at start-up and on every refresh (the desktop refreshes daily).
//! Release builds pin the official publisher to the keys they were built
//! with, so no index can sign `openidea.*` plugins with another key.
use super::{PackageFile, PluginEvent, PluginHost, PluginSource, PluginStatus, PluginUpdate, install};
use super::{PluginHost, PluginSource, PluginStatus};
use ide_core::{IdeError, IdeResult};
use ide_extension::Manifest;
use ide_extension::package::{self, Index, IndexEntry, PackageSignature, SIGNATURE_FILE};
use ide_extension::{MANIFEST_FILE, Manifest};
use semver::Version;
use serde::{Deserialize, Serialize};
use std::path::{Path, PathBuf};
@@ -18,18 +15,11 @@ use std::path::{Path, PathBuf};
/// the app, next to the signed packages of the official plugins.
pub const OFFICIAL_INDEX: &str = "https://github.com/l0ng-ai/openidea/releases/latest/download/plugins-index.json";
/// The publisher of the official plugins.
pub const OFFICIAL_PUBLISHER: &str = "openidea";
/// The last index read, in the plugins directory.
pub const INDEX_FILE: &str = "market-index.json";
const DOWNLOADS: &str = ".downloads";
const MAX_INDEX: usize = 16 << 20;
const MAX_PACKAGE: usize = 256 << 20;
const MAX_STAGED_FILES: usize = 256;
const MAX_STAGED_BYTES: usize = 16 << 20;
/// `moved` links followed before giving up (a loop, or a misconfiguration).
const MAX_MOVES: usize = 3;
/// A marketplace entry with what is installed.
#[derive(Clone, Debug, Serialize, Deserialize, specta::Type)]
@@ -39,32 +29,6 @@ pub struct MarketPlugin {
pub entry: IndexEntry,
/// Installed version, if any.
pub installed: Option<String>,
/// Whether this IDE can load `entry` (the newest version it can, else the newest).
pub compatible: bool,
}
impl MarketPlugin {
/// A version this IDE can load, newer than the one installed.
pub fn is_update(&self) -> bool {
let newer = |installed: &String| match (Version::parse(&self.entry.version), Version::parse(installed)) {
(Ok(offered), Ok(installed)) => offered > installed,
_ => false,
};
self.compatible && self.installed.as_ref().is_some_and(newer)
}
}
/// The official publisher's keys this IDE was built with
/// (`OPENIDEA_OFFICIAL_PLUGIN_KEYS`: base64, comma-separated). Development
/// builds have none: the index decides.
pub(super) fn built_in_official_keys() -> Vec<String> {
let keys = option_env!("OPENIDEA_OFFICIAL_PLUGIN_KEYS").unwrap_or_default();
keys.split(',').map(str::trim).filter(|k| !k.is_empty()).map(str::to_owned).collect()
}
fn pinned(mut index: Index, official_keys: &[String]) -> Index {
index.pin_publisher(OFFICIAL_PUBLISHER, "OpenIDEA", official_keys);
index
}
/// Reads `url`: `http(s)://…`, `file://…` or a local path.
@@ -92,33 +56,6 @@ pub fn read_url(url: &str, max: usize) -> Result<Vec<u8>, String> {
Ok(bytes)
}
/// The index at `url`, following `moved`, with downloads made absolute against
/// where it was read from.
fn read_index(url: &str) -> IdeResult<Index> {
let mut url = url.to_owned();
for _ in 0..=MAX_MOVES {
let bytes = read_url(&url, MAX_INDEX).map_err(IdeError::unavailable)?;
let text = String::from_utf8(bytes).map_err(|_| IdeError::invalid("the index is not UTF-8"))?;
let mut index = Index::parse(&text).map_err(IdeError::invalid)?;
match index.moved.take().filter(|m| !m.trim().is_empty()) {
Some(next) => {
let next = resolve(&url, &next);
if url.starts_with("https://") && !next.starts_with("https://") {
return Err(IdeError::invalid(format!("the marketplace moved to {next}, which isn't https")));
}
url = next;
}
None => {
for e in &mut index.plugins {
e.download = resolve(&url, &e.download);
}
return Ok(index);
}
}
}
Err(IdeError::unavailable(format!("the marketplace index moved more than {MAX_MOVES} times")))
}
/// `href` relative to the index at `base`: a URL, or a local path (with the
/// platform's separators, e.g. `C:\\market\\index.json`).
pub fn resolve(base: &str, href: &str) -> String {
@@ -138,16 +75,9 @@ pub fn resolve(base: &str, href: &str) -> String {
}
impl PluginHost {
pub(super) fn load_market_index(dir: &Path, official_keys: &[String]) -> Option<Index> {
pub(super) fn load_market_index(dir: &Path) -> Option<Index> {
let text = std::fs::read_to_string(dir.join(INDEX_FILE)).ok()?;
let index = Index::parse(&text).map_err(|e| tracing::warn!("{INDEX_FILE}: {e}")).ok()?;
Some(pinned(index, official_keys))
}
/// Pins the official publisher to `keys` (base64) instead of the built-in
/// ones, from the next index read.
pub fn set_official_keys(&self, keys: Vec<String>) {
*self.official_keys.write() = keys;
Index::parse(&text).map_err(|e| tracing::warn!("{INDEX_FILE}: {e}")).ok()
}
/// The marketplace index URL: the user's, else the default (none: no marketplace).
@@ -165,16 +95,14 @@ impl PluginHost {
self.save_state();
}
/// Reads the index again, keeps it, and applies its revocations. Announces
/// updates of installed plugins it hasn't announced yet.
/// Reads the index again, keeps it, and applies its revocations.
pub fn refresh_market(&self) -> IdeResult<Vec<MarketPlugin>> {
let url = self.market_url().ok_or_else(|| IdeError::unavailable("no plugin marketplace is configured"))?;
let index = read_index(&url)?;
if let Ok(json) = serde_json::to_string(&index) {
let _ = std::fs::write(self.plugins_dir.join(INDEX_FILE), json);
}
let newer = index.is_newer().then_some(index.format);
*self.market.write() = Some(pinned(index, &self.official_keys.read()));
let bytes = read_url(&url, MAX_INDEX).map_err(IdeError::unavailable)?;
let text = String::from_utf8(bytes).map_err(|_| IdeError::invalid("the index is not UTF-8"))?;
let index = Index::parse(&text).map_err(IdeError::invalid)?;
let _ = std::fs::write(self.plugins_dir.join(INDEX_FILE), &text);
*self.market.write() = Some(index);
// Trust may have changed: who signed what, what is revoked.
let installed: Vec<_> =
self.plugins.read().values().filter(|p| p.source == PluginSource::Installed).cloned().collect();
@@ -183,41 +111,7 @@ impl PluginHost {
plugin.meta.lock().publisher = publisher;
}
self.apply_revocations();
if let Some(format) = newer {
return Err(IdeError::unavailable(format!(
"the marketplace needs a newer OpenIDEA (index format {format}); its revocations still apply"
)));
}
let plugins = self.market_plugins();
self.announce_updates(&plugins);
Ok(plugins)
}
/// Tells the windows about updates once per set of versions: a later
/// refresh offering the same ones says nothing.
fn announce_updates(&self, plugins: &[MarketPlugin]) {
let updates: Vec<PluginUpdate> = plugins
.iter()
.filter(|p| p.is_update())
.map(|p| PluginUpdate {
id: p.entry.id.clone(),
name: p.entry.name.clone(),
version: p.entry.version.clone(),
})
.collect();
let key: Vec<String> = updates.iter().map(|u| format!("{}@{}", u.id, u.version)).collect();
if std::mem::replace(&mut *self.announced_updates.lock(), key.clone()) != key && !updates.is_empty() {
self.emit(None, PluginEvent::UpdatesAvailable { updates });
}
}
/// Whether the package in `src` would replace the installed `id`'s copy
/// with one signed by another key than that copy, without that key's
/// endorsement: the user confirms the change of signer.
pub(super) fn signer_changed(&self, id: &str, src: &Path) -> bool {
let Some(current) = install::active_dir(&self.plugins_dir.join(id)) else { return false };
let market = self.market.read();
!package::signer_continues(market.as_ref(), read_signature(&current).as_ref(), read_signature(src).as_ref())
Ok(self.market_plugins())
}
/// The plugins of the last index read.
@@ -225,14 +119,16 @@ impl PluginHost {
let Some(index) = self.market.read().clone() else { return Vec::new() };
let plugins = self.plugins.read();
index
.offered()
.into_iter()
.map(|(entry, compatible)| {
.plugins
.iter()
.filter(|e| index.revocation(&e.id, &e.version).is_none())
.cloned()
.map(|entry| {
let installed = plugins
.get(&entry.id)
.filter(|p| p.source == PluginSource::Installed)
.map(|p| p.manifest().version.clone());
MarketPlugin { entry: entry.clone(), installed, compatible }
MarketPlugin { entry, installed }
})
.collect()
}
@@ -243,18 +139,12 @@ impl PluginHost {
pub fn download_market_plugin(&self, id: &str) -> IdeResult<PathBuf> {
let url = self.market_url().ok_or_else(|| IdeError::unavailable("no plugin marketplace is configured"))?;
let index = self.market.read().clone().ok_or_else(|| IdeError::unavailable("refresh the marketplace first"))?;
let (entry, compatible) = index
.offered()
.into_iter()
.find(|(e, _)| e.id == id)
.map(|(e, ok)| (e.clone(), ok))
let entry = index
.plugins
.iter()
.find(|e| e.id == id)
.cloned()
.ok_or_else(|| IdeError::not_found(format!("{id} is not in the marketplace")))?;
if !compatible {
return Err(IdeError::unavailable(format!(
"{} {} needs plugin API {}: update OpenIDEA",
entry.name, entry.version, entry.engine
)));
}
let bytes = read_url(&resolve(&url, &entry.download), MAX_PACKAGE).map_err(IdeError::unavailable)?;
let downloads = self.plugins_dir.join(DOWNLOADS);
std::fs::create_dir_all(&downloads)?;
@@ -292,45 +182,6 @@ impl PluginHost {
Ok(dir)
}
/// Writes a package the UI generated (an imported IntelliJ theme) into a
/// folder next to the downloads, named after `name` (its plugin id); returns the folder.
/// The install page validates it like any other package.
pub fn stage_package(&self, name: &str, files: &[PackageFile]) -> IdeResult<PathBuf> {
ide_extension::validate_item_id(name, "package name").map_err(IdeError::invalid)?;
if !files.iter().any(|f| f.path == MANIFEST_FILE) {
return Err(IdeError::invalid(format!("a package needs a {MANIFEST_FILE}")));
}
let total: usize = files.iter().map(|f| f.text.len()).sum();
if files.len() > MAX_STAGED_FILES || total > MAX_STAGED_BYTES {
return Err(IdeError::invalid("the package is too large"));
}
let mut seen = std::collections::HashSet::new();
for f in files {
let valid = !f.path.is_empty()
&& f.path.len() <= 255
&& f.path.split('/').all(|part| {
!matches!(part, "" | "." | "..") && !part.contains(['\\', ':', '\0']) && !part.starts_with('.')
});
if !valid {
return Err(IdeError::invalid(format!("invalid package path {:?}", f.path)));
}
if !seen.insert(f.path.to_ascii_lowercase()) {
return Err(IdeError::invalid(format!("{} is in the package twice", f.path)));
}
}
// One folder per package, replaced by the next import of it: they don't pile up.
let dir = self.plugins_dir.join(DOWNLOADS).join(format!("{name}-generated"));
let _ = std::fs::remove_dir_all(&dir);
for f in files {
let path = dir.join(&f.path);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(&path, &f.text)?;
}
Ok(dir)
}
/// Files of the package in `dir` (what its declarations reference, and
/// the component): what a package file holds besides the manifest.
pub fn package_files_of(&self, dir: &Path) -> IdeResult<Vec<PathBuf>> {
+6 -53
View File
@@ -16,12 +16,8 @@
//! projects) and folders linked with `openidea-plugin link` (`links.json`).
mod api;
mod icon_extensions;
mod icon_patterns;
mod icon_settings;
mod icons;
mod install;
pub mod intellij;
mod market;
mod points;
mod providers;
@@ -36,10 +32,7 @@ pub mod web;
pub use api::{FileSecretStore, HostWorkspace, PluginWorkspace, SecretStore};
pub use icons::{map_asset as icon_theme_map_asset, sanitize_svg, svg_prefix as icon_theme_svg_prefix};
pub use market::{
INDEX_FILE as MARKET_INDEX_FILE, MarketPlugin, OFFICIAL_INDEX as OFFICIAL_MARKET_INDEX, OFFICIAL_PUBLISHER,
read_url as read_market_url, resolve as resolve_market_href,
};
pub use market::{INDEX_FILE as MARKET_INDEX_FILE, MarketPlugin, OFFICIAL_INDEX as OFFICIAL_MARKET_INDEX};
pub use points::{COMMANDS, CommandDecl, register_builtin_points};
pub(crate) use providers::editor_providers as editor_providers_for;
pub use providers::{ProviderKind, set_editor_providers};
@@ -245,16 +238,12 @@ pub struct Consent {
/// Optional capabilities the user accepted (`net:crates.io`, …). Required
/// ones are implied by consenting at all.
pub optional: Vec<String>,
/// The user accepted a package signed by another key than the installed
/// copy ([`InstallPreview::signer_changed`]).
#[serde(default)]
pub signer_change: bool,
}
impl Consent {
/// Accept everything the plugin asks for (tests, trusted official plugins).
pub fn all() -> Self {
Self { optional: vec!["*".into()], signer_change: true }
Self { optional: vec!["*".into()] }
}
fn accepts(&self, cap: &Capability) -> bool {
@@ -281,14 +270,6 @@ pub struct InstallPreview {
pub unsigned: bool,
/// The verified publisher.
pub publisher: Option<String>,
/// Replaces an installed copy signed with another key, which didn't
/// endorse this package's (or this one is unsigned): the user confirms it
/// ([`Consent::signer_change`]).
pub signer_changed: bool,
/// Something to confirm: a first install, an unsigned package, another
/// signer, or capabilities the installed version wasn't granted or
/// refused. Without, an update installs with the grants it has.
pub needs_consent: bool,
}
type ProjectPolicy = Arc<dyn Fn(&Path) -> bool + Send + Sync>;
@@ -335,10 +316,6 @@ pub struct PluginHost {
pins: Arc<RwLock<HashMap<(String, String), ProgramPin>>>,
/// The last marketplace index read: trust root for signatures, revocations.
market: RwLock<Option<ide_extension::package::Index>>,
/// The official publisher's keys the index is pinned to (none: as listed).
official_keys: RwLock<Vec<String>>,
/// `id@version` of the updates last announced.
announced_updates: Mutex<Vec<String>>,
/// The marketplace used while the user has set none (the app's official one).
default_market_url: RwLock<Option<String>>,
/// Updates not confirmed yet (the new version hasn't started once): what
@@ -377,12 +354,9 @@ impl PluginHost {
policy_pins.read().get(&(plugin.to_owned(), cap.to_string())).cloned()
}));
let shared = api::Shared::new(sink.clone(), plugins_dir.join(".data"), pins.clone(), ui.clone());
let official_keys = market::built_in_official_keys();
let market = RwLock::new(Self::load_market_index(&plugins_dir, &official_keys));
let market = RwLock::new(Self::load_market_index(&plugins_dir));
let host = Self {
market,
official_keys: RwLock::new(official_keys),
announced_updates: Mutex::default(),
default_market_url: RwLock::new(None),
updates: Mutex::default(),
project_policy: RwLock::new(Arc::new(|_| false)),
@@ -725,27 +699,14 @@ impl PluginHost {
}
let publisher = self.verified_publisher(src);
let m = &pkg.manifest;
let replaces = self.plugins.read().get(&m.id).map(|p| p.manifest().version.clone());
// An update shows what the installed version was granted.
let grant = replaces
.as_ref()
.and_then(|_| self.ledger.lock().get(&Subject::Installed(m.id.clone())).cloned())
.filter(|g| g.confirmed);
let granted = |c: &Capability| grant.as_ref().is_some_and(|g| g.has(c));
let mut capabilities: Vec<CapabilityInfo> =
m.capabilities.all().map(|c| CapabilityInfo::new(c, m.capabilities.is_optional(c), granted(c))).collect();
m.capabilities.all().map(|c| CapabilityInfo::new(c, m.capabilities.is_optional(c), false)).collect();
for extra in &set.capabilities {
if !m.declares(extra) {
capabilities.push(CapabilityInfo::new(extra, true, granted(extra)));
capabilities.push(CapabilityInfo::new(extra, true, false));
}
}
capabilities.sort_by(|a, b| b.risk.cmp(&a.risk).then(a.id.cmp(&b.id)));
let signer_changed = self.signer_changed(&m.id, src);
let answered = |c: &CapabilityInfo| {
c.granted || (c.optional && grant.as_ref().is_some_and(|g| g.declined.contains(&c.id)))
};
let needs_consent =
grant.is_none() || publisher.is_none() || signer_changed || !capabilities.iter().all(answered);
Ok(InstallPreview {
id: m.id.clone(),
name: m.name.clone(),
@@ -756,11 +717,9 @@ impl PluginHost {
capabilities,
contributions: counts(m),
warnings: set.warnings.clone(),
replaces,
replaces: self.plugins.read().get(&m.id).map(|p| p.manifest().version.clone()),
unsigned: publisher.is_none(),
publisher,
signer_changed,
needs_consent,
})
}
@@ -783,11 +742,6 @@ impl PluginHost {
if let Some(problem) = self.signature_problem(src) {
return Err(IdeError::invalid(format!("refusing {id}: {problem}")));
}
if !consent.signer_change && self.signer_changed(&id, src) {
return Err(IdeError::invalid(format!(
"refusing {id}: it is signed by another key than the installed version; confirm the change of signer"
)));
}
let version = pkg.manifest.version.clone();
if let Some(r) = self.market.read().as_ref().and_then(|i| i.revocation(&id, &version)) {
return Err(IdeError::invalid(format!("{id} {version} was revoked: {}", r.reason)));
@@ -2253,7 +2207,6 @@ fn placeholder_manifest(dir: &Path) -> Manifest {
homepage: None,
license: None,
change_notes: None,
declared_categories: Vec::new(),
engine: ide_extension::ENGINE_VERSION.into(),
legacy: false,
wasm: None,
-15
View File
@@ -292,21 +292,6 @@ impl PluginHost {
.collect()
}
/// The edits of an action `plugin_id` returned with `data`, now that the
/// user picked it: a call with the command budget, not the edit path's.
pub fn resolve_code_action(
&self,
plugin_id: &str,
doc: &ProviderDocument,
action: CodeAction,
) -> ide_core::IdeResult<CodeAction> {
let d = Arc::new(doc.clone());
let timeout = self.limits.timeout;
self.code_call(plugin_id, ide_plugin::Invocation::default(), "resolving the code action", move |rt, _| {
rt.resolve_code_action(&d, &action, timeout)
})
}
pub fn provide_definition(&self, doc: &ProviderDocument, at: ide_core::Pos) -> Vec<Location> {
let d = Arc::new(doc.clone());
self.edit_calls(ProviderKind::Definition, &doc.language, move |rt, timeout| {
+1 -2
View File
@@ -272,8 +272,7 @@ impl RemotePlugins {
report.failed.push(format!("{}: invalid plugin id", pkg.id));
continue;
}
// The desktop asked the user, change of signer included.
let consent = Consent { optional: pkg.granted.clone(), signer_change: true };
let consent = Consent { optional: pkg.granted.clone() };
if self.current_digest(&pkg.id).as_deref() == Some(pkg.digest.as_str()) {
if let Err(e) = self.follow_grants(&pkg, &consent) {
report.failed.push(format!("{}: {e}", pkg.id));
+1 -1
View File
@@ -134,7 +134,7 @@ impl ExtensionPoint for ThemesPoint {
/// Blanks out `//` and `/* */` comments outside strings, keeping newlines so
/// parse errors still point at the right line.
pub(super) fn strip_json_comments(src: &str) -> String {
fn strip_json_comments(src: &str) -> String {
let mut out = String::with_capacity(src.len());
let mut chars = src.chars().peekable();
let (mut in_str, mut escaped) = (false, false);
-20
View File
@@ -138,24 +138,12 @@ pub struct PluginsSnapshot {
pub ui: Vec<UiContribution>,
}
/// A newer version of an installed plugin in the marketplace.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PluginUpdate {
pub id: String,
pub name: String,
pub version: String,
}
/// Backend → UI notifications (Tauri event `ide://plugin`).
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(tag = "type", rename_all = "camelCase")]
pub enum PluginEvent {
/// Plugins, their status or their contributions changed: re-fetch `plugin_list`.
CommandsChanged,
/// The marketplace offers newer versions of installed plugins (announced
/// once per set of versions).
UpdatesAvailable { updates: Vec<PluginUpdate> },
#[serde(rename_all = "camelCase")]
Notification { plugin_id: String, level: NotificationLevel, title: String, body: String },
/// A plugin's status bar item was set (`item`) or removed.
@@ -224,14 +212,6 @@ impl PluginEditor {
}
}
/// A text file of a package the UI generated, at its `/`-separated path in the package.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PackageFile {
pub path: String,
pub text: String,
}
/// A plugin's status bar item.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
-1
View File
@@ -305,7 +305,6 @@ impl Host {
index.update(&changed, &batch.removed);
kotlin_bridge.files_changed(changed.iter().chain(&batch.removed));
lsp.project_files_changed(batch.created.iter().chain(&batch.removed));
lsp.watched_files_changed(&batch.created, &batch.modified, &batch.removed);
// Atomic saves by external tools surface as `created`.
documents.on_disk_changed(&changed);
// A framework's root marker (`angular.json`) created or deleted: its
-2
View File
@@ -257,8 +257,6 @@ impl Dispatcher {
&c.arg::<Option<Vec<String>>>("removedDetectedIds")?.unwrap_or_default(),
)
})
.add("run_without_derived", |c| async move { ops::run::without_derived(&c.host, &c.window, c.arg("configs")?) })
.add("run_apply_settings", |c| async move { ops::run::apply_settings(&c.host, &c.window, c.arg("configs")?) })
.add("run_system_env", |_| async move { Ok(ide_run::system_env()) })
.add("run_start", |c| async move {
let on_output = c.channel("onOutput")?;
-12
View File
@@ -177,10 +177,6 @@ pub(crate) fn read_login_env(shell: &str, timeout: std::time::Duration) -> Optio
}
}
}
// Nor what the last read found that this process lacks: children get it, the shell must not.
for (k, _) in ide_core::process::shell_env() {
cmd.env_remove(k);
}
ide_core::process::prepare(&mut cmd);
let mut child = cmd
.args(["-ilc", &format!("printf '{MARK}'; /usr/bin/env -0; printf '{MARK}'")])
@@ -261,14 +257,6 @@ mod tests {
std::fs::write(&rc, "export OPENIDEA_TEST_RC='two words'\n").unwrap();
let second = read_login_env(&shell, Duration::from_secs(10)).unwrap();
assert_eq!(get(&second, "OPENIDEA_TEST_RC").as_deref(), Some("two words"));
// What a read found goes to children, not into the next read: an rc file that no
// longer exports a variable drops it.
ide_core::process::set_shell_env(second);
std::fs::write(&rc, "export OPENIDEA_TEST_SEEN=\"${OPENIDEA_TEST_RC:-unset}\"\n").unwrap();
let third = read_login_env(&shell, Duration::from_secs(10)).unwrap();
assert_eq!(get(&third, "OPENIDEA_TEST_RC"), None);
assert_eq!(get(&third, "OPENIDEA_TEST_SEEN").as_deref(), Some("unset"));
}
#[test]
+1 -29
View File
@@ -274,15 +274,6 @@ pub fn restricted_command_allowed(cmd: &str) -> bool {
| "plugin_set_theme_frame"
| "plugin_catalog"
| "plugin_assets"
| "plugin_market"
| "plugin_market_url"
| "plugin_set_market_url"
| "plugin_market_download"
| "plugin_unpack_package"
| "plugin_stage_package"
| "plugin_preview_install"
| "plugin_read_intellij_theme"
| "plugin_installed_intellij_themes"
| "notebook_open"
| "notebook_get"
| "notebook_edit"
@@ -328,31 +319,12 @@ pub fn restricted_command_allowed(cmd: &str) -> bool {
) || cmd.starts_with("plugin:")
}
/// Commands that change which plugin code runs: allowed where no project is open (the
/// welcome screen), otherwise only in a trusted project.
pub fn projectless_command_allowed(cmd: &str) -> bool {
matches!(
cmd,
"plugin_install_dir"
| "plugin_uninstall"
| "plugin_set_enabled"
| "plugin_grant"
| "plugin_set_capability"
| "plugin_roll_back"
| "plugin_reload"
)
}
impl crate::Host {
pub fn authorize_command(&self, window: &str, command: &str) -> IdeResult<()> {
if restricted_command_allowed(command) {
return Ok(());
}
match self.project(window) {
Ok(project) => project.require_trust(),
Err(_) if projectless_command_allowed(command) => Ok(()),
Err(error) => Err(error),
}
self.project(window)?.require_trust()
}
}
+9 -236
View File
@@ -131,44 +131,25 @@ fn installs_from_before_versioning_keep_working_and_move_on_update() {
/// `hello` as `acme.hello` at `version`, packed and signed with `key`.
fn signed_hello(dir: &Path, version: &str, key: &PublisherKey, host: &PluginHost) -> (PathBuf, IndexEntry) {
signed(dir, "acme", version, key, host, |t| t.to_owned())
}
/// `hello` as `<publisher>.hello` at `version`, its manifest changed by
/// `edit`, packed and signed with `key`.
fn signed(
dir: &Path,
publisher: &str,
version: &str,
key: &PublisherKey,
host: &PluginHost,
edit: impl Fn(&str) -> String,
) -> (PathBuf, IndexEntry) {
let id = format!("{publisher}.hello");
let src =
variant(&repo().join("crates/ide-plugin/tests/fixtures/hello"), dir, &format!("src-{id}-{version}"), |t| {
edit(
&t.replace("id = \"hello\"", &format!("id = \"{id}\""))
.replace("version = \"0.3.0\"", &format!("version = \"{version}\"")),
)
});
let src = variant(&repo().join("crates/ide-plugin/tests/fixtures/hello"), dir, &format!("src-{version}"), |t| {
t.replace("id = \"hello\"", "id = \"acme.hello\"")
.replace("version = \"0.3.0\"", &format!("version = \"{version}\""))
});
let files = host.package_files_of(&src).unwrap();
let digest = package::package_digest(&src, &files).unwrap();
let sig = key.sign(publisher, &id, version, &digest);
let out = dir.join(format!("{id}-{version}.oiplugin"));
let sig = key.sign("acme", "acme.hello", version, &digest);
let out = dir.join(format!("acme.hello-{version}.oiplugin"));
package::pack(&src, &files, &out, Some(&sig)).unwrap();
let entry = IndexEntry {
id: id.clone(),
id: "acme.hello".into(),
name: "Hello".into(),
version: version.into(),
description: String::new(),
authors: vec![],
publisher: publisher.into(),
publisher: "acme".into(),
download: out.file_name().unwrap().to_string_lossy().into_owned(),
digest,
capabilities: vec![],
engine: "^1.0".into(),
categories: vec!["tools".into()],
};
(out, entry)
}
@@ -185,11 +166,10 @@ fn write_index(
publishers: vec![Publisher {
id: "acme".into(),
name: "ACME".into(),
keys: vec![KeyEntry { revoked: key_revoked, ..KeyEntry::new(key.public_base64()) }],
keys: vec![KeyEntry { key: key.public_base64(), revoked: key_revoked }],
}],
plugins,
revoked,
moved: None,
};
let path = dir.join("index.json");
std::fs::write(&path, serde_json::to_string(&index).unwrap()).unwrap();
@@ -280,210 +260,3 @@ fn package_files_install_from_disk_unsigned() {
let info = f.host.install_from_dir(&dir, &Consent::all()).unwrap();
assert_eq!((info.status, info.publisher), (PluginStatus::Inactive, None));
}
#[test]
fn the_marketplace_offers_the_newest_version_this_ide_can_load() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, old) = signed_hello(&market, "1.0.0", &key, &f.host);
let (_, new) = signed_hello(&market, "2.0.0", &key, &f.host);
let next_major = ide_extension::ENGINE_VERSION.split('.').next().unwrap().parse::<u64>().unwrap() + 1;
let new = IndexEntry { engine: format!("^{next_major}.0"), ..new };
f.host.set_market_url(Some(write_index(&market, &key, vec![old.clone(), new.clone()], vec![], false)));
let listed = f.host.refresh_market().unwrap();
assert_eq!((listed.len(), listed[0].entry.version.as_str(), listed[0].compatible), (1, "1.0.0", true));
let dir = f.host.download_market_plugin("acme.hello").unwrap();
assert_eq!(f.host.preview_install(&dir).unwrap().version, "1.0.0");
// Only a version for a newer IDE: shown as such, not downloaded.
write_index(&market, &key, vec![new], vec![], false);
let listed = f.host.refresh_market().unwrap();
assert_eq!((listed[0].entry.version.as_str(), listed[0].compatible), ("2.0.0", false));
let err = f.host.download_market_plugin("acme.hello").unwrap_err();
assert!(err.message.contains("update OpenIDEA"), "{}", err.message);
}
#[test]
fn a_moved_marketplace_is_read_where_it_moved_to() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, entry) = signed_hello(&market, "1.0.0", &key, &f.host);
write_index(&market, &key, vec![entry], vec![], false);
// The old address points at the new one; downloads resolve against where the index is.
let old = f.work.join("old");
std::fs::create_dir_all(&old).unwrap();
std::fs::write(old.join("index.json"), r#"{"format": 1, "moved": "../market/index.json"}"#).unwrap();
f.host.set_market_url(Some(old.join("index.json").to_string_lossy().into_owned()));
let listed = f.host.refresh_market().unwrap();
assert_eq!(listed[0].entry.id, "acme.hello");
let dir = f.host.download_market_plugin("acme.hello").unwrap();
assert_eq!(f.host.preview_install(&dir).unwrap().version, "1.0.0");
// A loop gives up.
std::fs::write(old.join("index.json"), r#"{"format": 1, "moved": "index.json"}"#).unwrap();
let err = f.host.refresh_market().unwrap_err();
assert!(err.message.contains("moved more than"), "{}", err.message);
}
/// `dir/index.json` listing `publishers` and `plugins`; its path.
fn index_of(dir: &Path, publishers: Vec<Publisher>, plugins: Vec<IndexEntry>) -> String {
let index = Index { format: package::INDEX_FORMAT, publishers, plugins, ..Index::default() };
let path = dir.join("index.json");
std::fs::write(&path, serde_json::to_string(&index).unwrap()).unwrap();
path.to_string_lossy().into_owned()
}
fn publisher(id: &str, keys: Vec<KeyEntry>) -> Publisher {
Publisher { id: id.into(), name: id.into(), keys }
}
fn updates_announced(events: &Events) -> Vec<Vec<String>> {
events
.lock()
.iter()
.filter_map(|e| match e {
PluginEvent::UpdatesAvailable { updates } => {
Some(updates.iter().map(|u| format!("{} {}", u.id, u.version)).collect())
}
_ => None,
})
.collect()
}
/// What the install page sends for an update it doesn't need to show: the grants the installed version has.
fn as_granted(preview: &InstallPreview) -> Consent {
let optional = preview.capabilities.iter().filter(|c| c.optional && c.granted).map(|c| c.id.clone()).collect();
Consent { optional, signer_change: false }
}
#[test]
fn updates_keep_their_signer_and_need_no_consent_when_nothing_is_new() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let (k1, k2, k3) =
(PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([2; 32]), PublisherKey::from_seed([3; 32]));
let same = |t: &str| t.to_owned();
let (_, v1) = signed(&market, "acme", "1.0.0", &k1, &f.host, same);
f.host.set_market_url(Some(index_of(
&market,
vec![publisher("acme", vec![KeyEntry::new(k1.public_base64())])],
vec![v1.clone()],
)));
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let first = f.host.preview_install(&dir).unwrap();
assert!(first.needs_consent && !first.signer_changed, "a first install is confirmed");
f.host.install_from_dir(&dir, &Consent::default()).unwrap();
assert!(updates_announced(&f.events).is_empty());
// 1.1.0, same key and needs: announced once, installed with the grants it has.
let (_, v11) = signed(&market, "acme", "1.1.0", &k1, &f.host, same);
let keys1 = vec![KeyEntry::new(k1.public_base64())];
index_of(&market, vec![publisher("acme", keys1.clone())], vec![v1.clone(), v11.clone()]);
f.host.refresh_market().unwrap();
f.host.refresh_market().unwrap();
assert_eq!(updates_announced(&f.events), [["acme.hello 1.1.0"]], "once per set of versions");
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert_eq!(preview.replaces.as_deref(), Some("1.0.0"));
assert!(!preview.needs_consent && !preview.signer_changed);
assert!(preview.capabilities.iter().all(|c| c.granted), "shows what the installed version was granted");
f.host.install_from_dir(&dir, &as_granted(&preview)).unwrap();
assert!(f.host.market_plugins().iter().all(|p| !p.is_update()));
// 1.2.0 asks for something new: the user is asked.
let (_, v12) = signed(&market, "acme", "1.2.0", &k1, &f.host, |t| {
t.replace(
"required = [\"editor.read\", \"editor.write\"]",
"required = [\"editor.read\", \"editor.write\"]\noptional = [\"net:example.com\"]",
)
});
index_of(&market, vec![publisher("acme", keys1.clone())], vec![v12.clone()]);
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert!(preview.needs_consent && !preview.signer_changed);
f.host.install_from_dir(&dir, &Consent::default()).unwrap();
// 1.3.0 signed with a key the installed one didn't endorse: another signer, the user's call.
let (_, v13) = signed(&market, "acme", "1.3.0", &k2, &f.host, same);
let keys12 = vec![KeyEntry::new(k1.public_base64()), KeyEntry::new(k2.public_base64())];
index_of(&market, vec![publisher("acme", keys12)], vec![v13]);
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert!(preview.signer_changed && preview.needs_consent);
let err = f.host.install_from_dir(&dir, &as_granted(&preview)).unwrap_err();
assert!(err.message.contains("another key"), "{}", err.message);
assert_eq!(f.host.info_of("acme.hello").unwrap().version, "1.2.0", "nothing changed");
f.host.install_from_dir(&dir, &Consent { signer_change: true, ..as_granted(&preview) }).unwrap();
// 1.4.0 signed with k3, which k2 endorsed: a rotation the installed signer vouched for.
let (_, v14) = signed(&market, "acme", "1.4.0", &k3, &f.host, same);
let keys = vec![
KeyEntry { endorsement: Some(k2.endorse("acme", &k3.public_base64())), ..KeyEntry::new(k3.public_base64()) },
KeyEntry::new(k2.public_base64()),
];
index_of(&market, vec![publisher("acme", keys)], vec![v14]);
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert!(!preview.signer_changed && !preview.needs_consent);
f.host.install_from_dir(&dir, &as_granted(&preview)).unwrap();
assert_eq!(f.host.info_of("acme.hello").unwrap().version, "1.4.0");
}
#[test]
fn an_index_in_a_newer_format_still_revokes() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, entry) = signed_hello(&market, "1.0.0", &key, &f.host);
f.host.set_market_url(Some(write_index(&market, &key, vec![entry], vec![], false)));
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
f.host.install_from_dir(&dir, &Consent::all()).unwrap();
let newer = format!(
r#"{{"format": {}, "plugins": "a shape this IDE can't read",
"revoked": [{{"id": "acme.hello", "reason": "steals tokens"}}]}}"#,
package::INDEX_FORMAT + 1
);
std::fs::write(market.join("index.json"), newer).unwrap();
let err = f.host.refresh_market().unwrap_err();
assert!(err.message.contains("newer OpenIDEA"), "{}", err.message);
let info = f.host.info_of("acme.hello").unwrap();
assert_eq!((info.status, info.error.as_deref()), (PluginStatus::Failed, Some("revoked: steals tokens")));
assert!(f.host.market_plugins().is_empty());
}
#[test]
fn official_plugins_need_the_key_this_ide_was_built_with() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let (root, intruder) = (PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([6; 32]));
f.host.set_official_keys(vec![root.public_base64()]);
// The index lists the intruder's key for the official publisher.
let (_, forged) = signed(&market, OFFICIAL_PUBLISHER, "1.0.0", &intruder, &f.host, |t| t.to_owned());
let keys = vec![KeyEntry::new(intruder.public_base64())];
f.host.set_market_url(Some(index_of(&market, vec![publisher(OFFICIAL_PUBLISHER, keys.clone())], vec![forged])));
f.host.refresh_market().unwrap();
let err = f.host.download_market_plugin("openidea.hello").unwrap_err();
assert!(err.message.contains("trusted publisher"), "{}", err.message);
// Signed with the built-in key it installs, whatever keys the index lists.
let (_, genuine) = signed(&market, OFFICIAL_PUBLISHER, "1.0.1", &root, &f.host, |t| t.to_owned());
index_of(&market, vec![publisher(OFFICIAL_PUBLISHER, keys)], vec![genuine]);
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("openidea.hello").unwrap();
assert_eq!(f.host.preview_install(&dir).unwrap().publisher.as_deref(), Some(OFFICIAL_PUBLISHER));
}
+21 -159
View File
@@ -123,19 +123,6 @@ fn misbehave_pkg() -> PathBuf {
repo().join("crates/ide-plugin/tests/fixtures/misbehave")
}
/// Whether `f` holds within 5 s. A call a slow runner keeps waiting behind
/// another (or behind a new instance starting) holds once it caught up.
fn eventually(mut f: impl FnMut() -> bool) -> bool {
let until = Instant::now() + Duration::from_secs(5);
while !f() {
if Instant::now() > until {
return false;
}
std::thread::sleep(Duration::from_millis(20));
}
true
}
/// A theme package with a light and a dark theme, plus licence files, in `dir`.
fn write_theme_pair(dir: &Path) {
std::fs::create_dir_all(dir.join("t")).unwrap();
@@ -410,11 +397,11 @@ fn crashes_restart_with_backoff_until_the_circuit_opens() {
let info = f.host.info_of("misbehave").unwrap();
assert_eq!((info.status, info.stats.activations, info.error), (PluginStatus::Active, 2, None));
// Three in a row: stopped for the session, backing off between them (the wait
// doubles: `Limits::backoff`, tested where it is defined; wall-clock waits on CI
// are too noisy to compare).
// Three in a row: stopped for the session, with growing waits between.
f.take_events();
for crash in 0..3 {
let mut waits = Vec::new();
for _ in 0..3 {
let started = Instant::now();
let err = loop {
match f.run("misbehave", "panic", None) {
Err(e) if e.contains("restarts in") => std::thread::sleep(Duration::from_millis(20)),
@@ -422,10 +409,9 @@ fn crashes_restart_with_backoff_until_the_circuit_opens() {
}
};
assert!(err.contains("crashed"), "{err}");
if crash < 2 {
assert!(f.run("misbehave", "echo", None).unwrap_err().contains("restarts in"), "backing off");
}
waits.push(started.elapsed());
}
assert!(waits[2] >= backoff * 2 && waits[2] > waits[1], "{waits:?}");
let info = f.host.info_of("misbehave").unwrap();
assert_eq!(info.status, PluginStatus::Failed);
assert!(info.error.as_deref().is_some_and(|e| e.contains("3 times in a row")), "{:?}", info.error);
@@ -493,15 +479,12 @@ fn running_calls_can_be_cancelled() {
let f = f.clone();
std::thread::spawn(move || f.run("misbehave", "spin", None))
};
// A cancel before the call runs (the instance may still be starting) does nothing: repeat it.
let until = Instant::now() + Duration::from_secs(5);
while !spinning.is_finished() {
assert!(Instant::now() < until, "the running call wasn't interrupted");
f.host.cancel("misbehave");
std::thread::sleep(Duration::from_millis(20));
}
std::thread::sleep(Duration::from_millis(100));
let started = Instant::now();
f.host.cancel("misbehave");
let err = spinning.join().unwrap().unwrap_err();
assert!(err.contains("was cancelled"), "{err}");
assert!(started.elapsed() < Duration::from_secs(1));
assert_eq!(f.run("misbehave", "echo", Some("x")).unwrap().as_deref(), Some("x"), "no backoff");
let stats = f.host.info_of("misbehave").unwrap().stats;
assert_eq!((stats.crashes, stats.activations), (0, 2));
@@ -543,22 +526,16 @@ fn edit_path_calls_have_soft_and_hard_deadlines() {
let provider = host.syntax_provider();
let mis = LanguageId::new("mis");
// The first call starts the instance, which may take longer than the soft deadline.
let mut tokens = None;
assert!(eventually(|| {
tokens = provider.highlights(&mis, "fast");
tokens.is_some()
}));
let tokens = tokens.unwrap();
let tokens = provider.highlights(&mis, "fast").unwrap();
assert_eq!((tokens[0].start, tokens[0].end, tokens[0].kind.as_str()), (0, 4, "keyword"));
// Slower than the soft deadline: dropped (however late this thread gets
// to look), but the instance lives on.
assert!(provider.highlights(&mis, "sleep:150").is_none());
assert!(eventually(|| provider.highlights(&mis, "fast").is_some()));
std::thread::sleep(Duration::from_millis(150));
assert!(provider.highlights(&mis, "fast").is_some());
let info = host.info_of("misbehave").unwrap();
assert!(info.stats.slow_calls >= 1, "{:?}", info.stats);
assert_eq!((info.stats.crashes, info.stats.activations), (0, 1));
assert_eq!((info.stats.slow_calls, info.stats.crashes, info.stats.activations), (1, 0, 1));
// Past the hard deadline: interrupted; the next call gets a new instance.
assert!(provider.highlights(&mis, "spin").is_none());
@@ -567,8 +544,7 @@ fn edit_path_calls_have_soft_and_hard_deadlines() {
let info = host.info_of("misbehave").unwrap();
assert_eq!(info.stats.crashes, 1, "{:?}", info.error);
assert!(info.error.as_deref().is_some_and(|e| e.contains("timed out after 400ms")), "{:?}", info.error);
// The new instance may take longer to start than the soft deadline on a slow runner.
assert!(eventually(|| provider.highlights(&mis, "fast").is_some()));
assert!(provider.highlights(&mis, "fast").is_some());
assert_eq!(host.info_of("misbehave").unwrap().stats.activations, 2);
}
@@ -577,10 +553,9 @@ fn edit_path_calls_have_soft_and_hard_deadlines() {
#[test]
fn providers_of_several_plugins_run_concurrently() {
let tmp = tempfile::tempdir().unwrap();
// A budget no runner misses, so only the overlap is measured.
let limits = Limits {
edit_soft: Duration::from_secs(5),
edit_hard: Duration::from_secs(10),
edit_soft: Duration::from_millis(300),
edit_hard: Duration::from_secs(2),
fuel: u64::MAX,
..Limits::default()
};
@@ -613,12 +588,12 @@ fn providers_of_several_plugins_run_concurrently() {
assert_eq!(host.provide_completion(&doc("fast"), at, None).len(), 2);
let started = Instant::now();
let items = host.provide_completion(&doc("sleep:500"), at, None);
let items = host.provide_completion(&doc("sleep:200"), at, None);
let elapsed = started.elapsed();
let mut from: Vec<&str> = items.iter().map(|(p, _)| p.as_str()).collect();
from.sort();
assert_eq!(from, ["slow-a", "slow-b"], "both within the budget");
assert!(elapsed < Duration::from_millis(950), "one after the other would take 1000ms: {elapsed:?}");
assert!(elapsed < Duration::from_millis(350), "one after the other would take 400ms: {elapsed:?}");
assert_eq!(host.info_of("slow-b").unwrap().stats.slow_calls, 0);
}
@@ -638,7 +613,7 @@ fn idle_instances_are_reclaimed_and_schedules_reactivate() {
assert_eq!(f.host.instances(), 0);
let info = f.host.info_of("misbehave").unwrap();
assert_eq!((info.status, info.stats.reclaims), (PluginStatus::Inactive, 1));
assert_eq!(f.command_titles("misbehave").len(), 10, "contributions stay");
assert_eq!(f.command_titles("misbehave").len(), 8, "contributions stay");
assert_eq!(f.run("misbehave", "echo", Some("y")).unwrap().as_deref(), Some("y"));
assert_eq!(f.host.info_of("misbehave").unwrap().stats.activations, 2);
@@ -1089,16 +1064,7 @@ fn plugin_syntax_for_languages_without_a_grammar() {
let sol = LanguageId::new("solidity");
let text = "// c\ncontract A {\n uint256 public x;\n function f() public {}\n}\n";
assert!(provider.highlights(&LanguageId::new("rust"), text).is_none(), "only its own languages");
// The first call of a new instance can run past the edit-path budget on a
// slow machine; its answer is dropped and a later call asks again.
let deadline = Instant::now() + Duration::from_secs(10);
let tokens = loop {
match provider.highlights(&sol, text) {
Some(tokens) => break tokens,
None if Instant::now() < deadline => std::thread::sleep(Duration::from_millis(50)),
None => panic!("no highlights"),
}
};
let tokens = provider.highlights(&sol, text).unwrap();
let kind_at =
|s: &str| tokens.iter().find(|t| &text[t.start as usize..t.end as usize] == s).map(|t| t.kind.as_str());
assert_eq!(kind_at("contract"), Some("keyword"));
@@ -2334,57 +2300,6 @@ fn a_zipped_plugin_folder_installs_like_the_folder() {
assert!(err.contains("is not an OpenIDEA plugin"), "{err}");
}
/// What the UI writes for an imported IntelliJ theme goes through the install page.
#[test]
fn a_generated_package_is_staged_for_the_install_page() {
let f = fixture(Limits::default());
let file = |path: &str, text: &str| PackageFile { path: path.into(), text: text.into() };
let manifest = r#"id = "intellij.deep-sea"
name = "Deep Sea"
version = "2.3.0"
engines.openidea = "^1.0"
[[contributes.themes]]
path = "themes/deep-sea.theme.json"
editor-scheme = "themes/deep-sea.scheme.json"
"#;
let files = [
file("plugin.toml", manifest),
file(
"themes/deep-sea.theme.json",
r##"{"id":"deep-sea","name":"Deep Sea","kind":"dark","tokens":{"--bg-app":"#101820"}}"##,
),
file("themes/deep-sea.scheme.json", r##"{"name":"Deep Sea","tokens":{"--syn-keyword":"#cc7832"}}"##),
];
let dir = f.host.stage_package("intellij.deep-sea", &files).unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert_eq!((preview.id.as_str(), preview.has_code, preview.unsigned), ("intellij.deep-sea", false, true));
assert!(preview.capabilities.is_empty());
f.install(&dir);
let active = f.host.set_theme(Some(("intellij.deep-sea", "deep-sea"))).unwrap().unwrap();
assert_eq!(active.theme.tokens["--bg-app"], "#101820");
assert_eq!(active.theme.tokens["--syn-keyword"], "#cc7832");
// Staging the same package again starts from an empty folder; importing it again replaces it.
assert_eq!(f.host.stage_package("intellij.deep-sea", &files[..1]).unwrap(), dir);
assert!(!dir.join("themes").exists());
let mut changed = files.clone();
changed[2] = file("themes/deep-sea.scheme.json", r##"{"tokens":{"--syn-keyword":"#ff8800"}}"##);
f.install(&f.host.stage_package("intellij.deep-sea", &changed).unwrap());
let active = f.host.set_theme(Some(("intellij.deep-sea", "deep-sea"))).unwrap().unwrap();
assert_eq!(active.theme.tokens["--syn-keyword"], "#ff8800");
for bad in ["../x", "/abs", "a//b", r"a\b", "C:x", ".hidden", "a/./b", ""] {
let err = f.host.stage_package("p", &[file("plugin.toml", manifest), file(bad, "x")]).unwrap_err().message;
assert!(err.contains("invalid package path"), "{bad:?}: {err}");
}
let err = f.host.stage_package("p", &[file("x.json", "{}")]).unwrap_err().message;
assert!(err.contains("needs a plugin.toml"), "{err}");
let err = f.host.stage_package("p", &[file("plugin.toml", ""), file("PLUGIN.toml", "")]).unwrap_err().message;
assert!(err.contains("twice"), "{err}");
assert!(f.host.stage_package("../p", &files).is_err());
}
/// Secrets live outside the plugin's directory (the OS keychain on the
/// desktop): uninstalling the plugin deletes the ones it declares.
#[test]
@@ -2413,56 +2328,3 @@ fn uninstalling_deletes_the_plugins_secrets() {
f.host.uninstall("openidea.ai-completion").unwrap();
assert!(keychain.0.lock().is_empty(), "{:?}", keychain.0.lock().keys().collect::<Vec<_>>());
}
/// Plugin API 1.0 leaves room to grow: event kinds are names, and the editor
/// providers carry snippets, imports, lazily computed quick fixes and tagged
/// problems.
#[test]
fn completion_quick_fixes_problems_and_event_kinds_of_api_1() {
let f = fixture(Limits::default());
let pkg = f._tmp.path().join("misbehave-providers");
copy_dir(&misbehave_pkg(), &pkg);
let manifest = std::fs::read_to_string(pkg.join("plugin.toml")).unwrap();
std::fs::write(
pkg.join("plugin.toml"),
format!("{manifest}\n[[contributes.providers]]\nkinds = [\"completion\", \"code-actions\"]\nlanguages = [\"text\"]\n"),
)
.unwrap();
f.install(&pkg);
let doc = |text: &str| ide_plugin::ProviderDocument {
path: f.project.join("a.txt"),
language: "text".into(),
version: 1,
text: text.into(),
};
let at = ide_core::Pos::new(0, 0);
let items = f.host.provide_completion(&doc("rich"), at, None);
let item = &items[0].1;
assert_eq!(
(item.insert_text.as_str(), item.snippet, item.filter_text.as_deref()),
("f(${1:x})$0", true, Some("fun"))
);
assert_eq!(item.additional_edits[0].new_text, "use f;\n");
// Offered without edits; computed when picked.
let actions = f.host.provide_code_actions(&doc("lazy"), ide_core::Range::default());
let (plugin, action) = actions.into_iter().next().expect("offered");
assert!(action.edits.is_empty() && action.data.as_deref() == Some("fix"));
let resolved = f.host.resolve_code_action(&plugin, &doc("lazy"), action.clone()).unwrap();
assert_eq!(resolved.edits[0].1[0].new_text, "fixed ");
let foreign = ide_plugin::CodeAction { data: Some("other".into()), ..action };
assert!(f.host.resolve_code_action(&plugin, &doc("lazy"), foreign).is_err());
// A kind this IDE doesn't know is refused, not a trap.
let err = f.host.run_command("misbehave", "subscribe", Some("file-deleted"), f.inv()).unwrap_err();
assert!(err.message.contains("no event kind"), "{}", err.message);
let err = f.host.run_command("misbehave", "subscribe", Some("file-saved"), f.inv()).unwrap_err();
assert!(err.message.contains("events"), "known, but not granted: {}", err.message);
// Tags the IDE knows, the tool that found it.
let path = f.project.join("a.txt");
f.host.run_command("misbehave", "diagnose", Some(&path.to_string_lossy()), f.inv()).unwrap();
let d = f.workspace.diagnostics.lock().get(&("misbehave".into(), path)).cloned().unwrap();
assert_eq!((d[0].unnecessary, d[0].deprecated, d[0].source.as_deref()), (true, false, Some("lint")));
}
-7
View File
@@ -218,13 +218,6 @@ async fn serves_projects_files_documents_search_and_runs() {
invoke(&peer, "vcs_init", json!({ "dir": project })).await.unwrap();
until("git repository", || project.join(".git").is_dir()).await;
// A Docker run configuration's environment comes from the host its command runs on.
let env =
invoke(&peer, "docker_run_env", json!({ "endpoint": { "kind": "socket", "path": "/run/user/1/docker.sock" } }))
.await
.unwrap();
assert_eq!(env["DOCKER_HOST"], "unix:///run/user/1/docker.sock");
// Real project-host ignore writes travel through the remote command dispatcher.
assert!(Dispatcher::standard().commands().contains(&"vcs_add_to_ignore"));
let ignore_root = dunce::canonicalize(&project).unwrap();
-40
View File
@@ -81,46 +81,6 @@ async fn gradle_defaults_are_not_saved_as_explicit_run_choices() {
assert_eq!(saved.command, gradle.command);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn maven_defaults_are_resolved_at_launch_and_explicit_choices_survive_reload() {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
std::fs::write(root.join("pom.xml"), "<project/>").unwrap();
let home = root.join("maven");
std::fs::create_dir_all(home.join("bin")).unwrap();
let launcher = home.join("bin").join(if cfg!(windows) { "mvn.cmd" } else { "mvn" });
std::fs::write(&launcher, "").unwrap();
let host = host(&root.join("app"));
host.trust.decide(&root, true, false).unwrap();
host.open_project("main", &root).unwrap();
for (key, value) in [
("languages.java.maven.home", home.to_string_lossy().into_owned()),
("languages.java.maven.userSettings", "settings.xml".into()),
] {
ops::project::set_settings(
&host,
"main",
ide_workspace::settings::SettingsScope::Project,
key,
&serde_json::to_string(&value).unwrap(),
)
.unwrap();
}
let mut config = ops::run::configs(&host, "main").unwrap().into_iter().find(|c| c.id == "maven:test").unwrap();
assert_eq!(config.command, "mvn");
assert_eq!(config.args, ["test"]);
assert!(config.env.is_empty());
config.command = launcher.to_string_lossy().into_owned();
config.args = vec!["-s".into(), "own-settings.xml".into(), "test".into()];
config.env.insert("JAVA_HOME".into(), "own-jdk".into());
config.detected = false;
ops::run::save_configs(&host, "main", &[config.clone()], &[]).unwrap();
let saved = ops::run::configs(&host, "main").unwrap().into_iter().find(|c| c.id == config.id).unwrap();
assert_eq!(saved.command, config.command);
assert_eq!(saved.args, config.args);
assert_eq!(saved.env, config.env);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn workspace_cmake_before_launch_tasks_resolve_in_the_member() {
let tmp = tempfile::tempdir().unwrap();
+2 -11
View File
@@ -58,11 +58,6 @@ async fn application_commands_work_in_restricted_projects_and_without_a_project(
"vcs_write_patch",
"remote_bootstrap_list",
"deploy_test_connection",
"plugin_market",
"plugin_market_url",
"plugin_set_market_url",
"plugin_market_download",
"plugin_preview_install",
] {
assert!(ide_host::trust::restricted_command_allowed(command), "{command}");
for window in ["main", "welcome"] {
@@ -77,6 +72,8 @@ async fn application_commands_work_in_restricted_projects_and_without_a_project(
"tools_list",
"tools_install",
"tools_uninstall",
"plugin_install_dir",
"plugin_reload",
"vcs_git_version",
"docker_compose_op",
"db_execute",
@@ -96,12 +93,6 @@ async fn application_commands_work_in_restricted_projects_and_without_a_project(
assert!(host.authorize_command(window, command).is_err(), "{window}: {command}");
}
}
// Plugin installs run plugin code: from the welcome screen, but not in an untrusted project.
for command in ["plugin_install_dir", "plugin_uninstall", "plugin_set_enabled", "plugin_reload"] {
assert!(!ide_host::trust::restricted_command_allowed(command), "{command}");
assert!(host.authorize_command("main", command).is_err(), "{command}");
assert!(host.authorize_command("welcome", command).is_ok(), "{command}");
}
// These requests validate the vendor before any network access, independently of project trust.
for window in ["main", "welcome"] {
let error = ide_host::ops::tools::jdk_download_releases(
-3
View File
@@ -16,6 +16,3 @@ parking_lot.workspace = true
thiserror.workspace = true
tracing.workspace = true
encoding_rs.workspace = true
[dev-dependencies]
tempfile.workspace = true
-15
View File
@@ -243,9 +243,6 @@ struct Inner {
pid: Option<u32>,
/// Fired when the reader task has stopped (EOF, error or kill).
reader_done: CancellationToken,
/// Fired once the child process has exited and been reaped (after [`Peer::kill`],
/// also what it started); at once when there is no child.
exited: CancellationToken,
}
impl Inner {
@@ -435,7 +432,6 @@ impl Peer {
(child.wait().await.ok(), true)
}
};
inner.exited.cancel();
if !killed {
// Bounded: a grandchild may keep the pipes open.
let drained = async {
@@ -514,11 +510,7 @@ impl Peer {
kill: CancellationToken::new(),
pid,
reader_done: CancellationToken::new(),
exited: CancellationToken::new(),
});
if pid.is_none() {
inner.exited.cancel();
}
// Writer: a single task drains the queue so messages never interleave.
{
@@ -712,13 +704,6 @@ impl Peer {
self.inner.close_transport();
self.inner.kill.cancel();
}
/// Resolves once the child process is gone: exited on its own, or killed with what it
/// started and reaped. [`closed`](Self::closed) can come earlier, at EOF. Until then the
/// child may still hold files open, which on Windows can't be deleted.
pub async fn exited(&self) {
self.inner.exited.cancelled().await;
}
}
/// Removes the pending entry and tells the remote side to stop when a request
-59
View File
@@ -413,65 +413,6 @@ async fn kill_terminates_process() {
tokio::time::timeout(Duration::from_secs(5), peer.closed()).await.unwrap();
}
#[cfg(unix)]
#[tokio::test(flavor = "multi_thread")]
async fn exited_resolves_once_the_killed_process_is_reaped() {
let peer =
Peer::spawn("sleep", &["30".into()], &std::env::temp_dir(), &[], Arc::new(NoopHandler), Box::new(|_| {}))
.unwrap();
let pid = peer.pid().unwrap().to_string();
let exited = tokio::time::timeout(Duration::from_millis(200), peer.exited()).await;
assert!(exited.is_err(), "running until killed");
peer.kill();
tokio::time::timeout(Duration::from_secs(5), peer.exited()).await.unwrap();
let alive = std::process::Command::new("kill").args(["-0", &pid]).status().unwrap().success();
assert!(!alive, "reaped, not even a zombie");
}
#[tokio::test(flavor = "multi_thread")]
async fn exited_resolves_at_once_without_a_process() {
let (client, _server) = pair(Arc::new(NoopHandler), Arc::new(Recorder::default()));
tokio::time::timeout(Duration::from_secs(1), client.exited()).await.unwrap();
}
#[cfg(windows)]
#[tokio::test(flavor = "multi_thread")]
async fn exited_waits_for_a_windows_descendant_to_release_its_file() {
use std::os::windows::fs::OpenOptionsExt;
let tmp = tempfile::tempdir().unwrap();
std::fs::write(
tmp.path().join("hold.ps1"),
"$f=[IO.File]::Open('locked.txt','Create','ReadWrite','None'); [IO.File]::WriteAllText('ready.txt','ready'); Start-Sleep 30",
)
.unwrap();
let peer = Peer::spawn(
"cmd",
&[
"/D".into(),
"/C".into(),
"powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File hold.ps1".into(),
],
tmp.path(),
&[],
Arc::new(NoopHandler),
Box::new(|_| {}),
)
.unwrap();
tokio::time::timeout(Duration::from_secs(10), async {
while !tmp.path().join("ready.txt").exists() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await
.unwrap();
let open = || std::fs::OpenOptions::new().read(true).share_mode(0).open(tmp.path().join("locked.txt"));
assert_eq!(open().unwrap_err().raw_os_error(), Some(32));
assert!(tokio::time::timeout(Duration::from_millis(200), peer.exited()).await.is_err());
peer.kill();
tokio::time::timeout(Duration::from_secs(5), peer.exited()).await.unwrap();
drop(open().expect("exited must wait for the descendant's file handle to close"));
}
/// A launcher that prints in the system code page and exits before speaking the
/// protocol: `jdtls.bat` on a Chinese Windows when its Python step fails, then
/// `pause` prints `请按任意键继续. . .` in GBK on stdout.
-10
View File
@@ -131,16 +131,6 @@ pub struct LanguageServerConfig {
/// basedpyright): only used while none of those is installed.
#[serde(default)]
pub fallback: bool,
/// What the server is used for (`completion`, `code-action`, `diagnostics`,
/// … `FEATURES` in `ide-lsp/src/caps.rs`); empty means everything. The other
/// capabilities it announces are ignored, so a refactoring-only server next
/// to the language's own one adds no completion, hover or formatting.
#[serde(default)]
pub features: Vec<String>,
/// Code action kinds kept from the server, matched like LSP's `only`
/// (`refactor` keeps `refactor.extract`); empty keeps every kind.
#[serde(default)]
pub code_action_kinds: Vec<String>,
/// Newest Java release the server runs on (kotlin-language-server: 21). A
/// newer `JAVA_HOME` is replaced by the newest installed JDK that fits.
#[serde(default)]
+1 -258
View File
@@ -183,8 +183,6 @@ pub struct ServerCaps {
/// `Some(include_text)` when the server wants `didSave`.
pub save: Option<bool>,
pub semantic_legend: Option<(Vec<String>, Vec<String>)>,
/// Applied to what the server announces, now and in later registrations.
pub restriction: Restriction,
}
/// LSP request method → server capability key, for features and dynamic registration.
@@ -217,104 +215,6 @@ pub fn capability_key(method: &str) -> Option<&'static str> {
})
}
/// Names for a server's `features` → the capability keys each one keeps.
pub const FEATURES: &[(&str, &[&str])] = &[
("completion", &["completionProvider"]),
("inline-completion", &["inlineCompletionProvider"]),
("hover", &["hoverProvider"]),
("signature-help", &["signatureHelpProvider"]),
("definition", &["definitionProvider"]),
("declaration", &["declarationProvider"]),
("type-definition", &["typeDefinitionProvider"]),
("implementation", &["implementationProvider"]),
("references", &["referencesProvider"]),
("document-highlight", &["documentHighlightProvider"]),
("document-symbol", &["documentSymbolProvider"]),
("workspace-symbol", &["workspaceSymbolProvider"]),
("code-action", &["codeActionProvider", "executeCommandProvider"]),
("code-lens", &["codeLensProvider", "executeCommandProvider"]),
("rename", &["renameProvider"]),
(
"formatting",
&["documentFormattingProvider", "documentRangeFormattingProvider", "documentOnTypeFormattingProvider"],
),
("inlay-hint", &["inlayHintProvider"]),
("semantic-tokens", &["semanticTokensProvider"]),
("document-link", &["documentLinkProvider"]),
("call-hierarchy", &["callHierarchyProvider"]),
("type-hierarchy", &["typeHierarchyProvider"]),
("folding-range", &["foldingRangeProvider"]),
("selection-range", &["selectionRangeProvider"]),
("diagnostics", &["diagnosticProvider"]),
];
/// A server's `features` and `code-action-kinds`: the part of its capabilities
/// the IDE uses.
#[derive(Clone, Debug, Default, PartialEq)]
pub struct Restriction {
/// Allowed `*Provider` keys; `None` allows all.
keys: Option<Vec<&'static str>>,
/// Code action kind prefixes kept; empty keeps all.
action_kinds: Vec<String>,
}
impl Restriction {
/// Unknown feature names are logged and ignored.
pub fn new(server: &str, features: &[String], action_kinds: &[String]) -> Self {
let keys = (!features.is_empty()).then(|| {
let mut keys = Vec::new();
for name in features {
match FEATURES.iter().find(|(n, _)| n == name) {
Some((_, k)) => keys.extend_from_slice(k),
None => tracing::warn!("{server}: unknown feature `{name}` in `features`"),
}
}
keys
});
Self { keys, action_kinds: action_kinds.to_vec() }
}
fn allows(&self, key: &str) -> bool {
self.keys.as_ref().is_none_or(|k| k.contains(&key))
}
/// Whether the server's `publishDiagnostics` are shown.
pub fn diagnostics(&self) -> bool {
self.allows("diagnosticProvider")
}
/// Whether a code action of `kind` is kept; one without a kind only when
/// every kind is.
pub fn keeps_action(&self, kind: Option<&str>) -> bool {
self.action_kinds.is_empty() || kind.is_some_and(|k| self.action_kinds.iter().any(|p| kind_within(k, p)))
}
/// The kinds the server can still return: those it lists that are kept,
/// narrowed to the kept prefixes below them (`refactor` → `refactor.extract`).
fn advertised_kinds(&self, listed: Option<Vec<String>>) -> Vec<String> {
let Some(listed) = listed else { return self.action_kinds.clone() };
let mut out = Vec::new();
for k in &listed {
let narrowed: Vec<&String> = if self.keeps_action(Some(k)) {
vec![k]
} else {
self.action_kinds.iter().filter(|p| kind_within(p, k)).collect()
};
for k in narrowed {
if !out.contains(k) {
out.push(k.clone());
}
}
}
out
}
}
/// `kind` is `prefix` or below it (`refactor.extract` within `refactor`).
fn kind_within(kind: &str, prefix: &str) -> bool {
kind.strip_prefix(prefix).is_some_and(|rest| rest.is_empty() || rest.starts_with('.'))
}
fn truthy(v: Option<&Value>) -> bool {
!matches!(v, None | Some(Value::Null) | Some(Value::Bool(false)))
}
@@ -344,34 +244,7 @@ impl ServerCaps {
};
(strings("tokenTypes"), strings("tokenModifiers"))
});
Self { raw, encoding, open_close, sync, save, semantic_legend, restriction: Restriction::default() }
}
/// Drops the capabilities outside the server's `features`, and code action
/// kinds outside its `code-action-kinds`. Document sync, workspace options
/// and `experimental` stay.
pub fn restrict(&mut self, restriction: Restriction) {
if let Some(o) = self.raw.as_object_mut() {
o.retain(|key, _| !key.ends_with("Provider") || restriction.allows(key));
}
if self.has("codeActionProvider") && !restriction.action_kinds.is_empty() {
let kinds = restriction.advertised_kinds(self.code_action_kinds());
if kinds.is_empty() {
// It lists only kinds that aren't kept: nothing to ask it for.
if let Some(o) = self.raw.as_object_mut() {
o.remove("codeActionProvider");
}
} else {
if !self.raw["codeActionProvider"].is_object() {
self.raw["codeActionProvider"] = json!({});
}
self.raw["codeActionProvider"]["codeActionKinds"] = json!(kinds);
}
}
if !restriction.allows("semanticTokensProvider") {
self.semantic_legend = None;
}
self.restriction = restriction;
Self { raw, encoding, open_close, sync, save, semantic_legend }
}
/// Whether the server statically advertises the capability under `key`.
@@ -452,18 +325,11 @@ impl ServerCaps {
self.raw["workspace"]["textDocumentContent"] = opts.clone();
return;
}
if let Some(key) = capability_key(method)
&& !self.restriction.allows(key)
{
return;
}
if method == "textDocument/semanticTokens"
&& let Some(opts) = options
{
self.raw["semanticTokensProvider"] = opts.clone();
let restriction = std::mem::take(&mut self.restriction);
*self = Self::parse(std::mem::take(&mut self.raw));
self.restriction = restriction;
return;
}
if method == "textDocument/didSave" {
@@ -483,11 +349,6 @@ impl ServerCaps {
&& !self.has(key)
{
self.raw[key] = options.cloned().filter(|o| o.is_object()).unwrap_or(Value::Bool(true));
if key == "codeActionProvider" {
// Registered kinds go through the same narrowing as announced ones.
let restriction = std::mem::take(&mut self.restriction);
self.restrict(restriction);
}
}
}
@@ -565,122 +426,4 @@ mod tests {
c.unregister("workspace/willRenameFiles");
assert!(c.file_operation("willRename").is_none());
}
fn restriction(features: &[&str], kinds: &[&str]) -> Restriction {
let owned = |v: &[&str]| v.iter().map(|s| s.to_string()).collect::<Vec<_>>();
Restriction::new("test", &owned(features), &owned(kinds))
}
#[test]
fn features_drop_other_capabilities() {
let mut c = ServerCaps::parse(json!({
"textDocumentSync": 2,
"completionProvider": {},
"hoverProvider": true,
"documentFormattingProvider": true,
"foldingRangeProvider": true,
"codeActionProvider": { "resolveProvider": true },
"executeCommandProvider": { "commands": ["rope.extract"] },
"semanticTokensProvider": { "legend": { "tokenTypes": [], "tokenModifiers": [] }, "full": true },
"workspace": { "fileOperations": { "willRename": { "filters": [] } } },
"experimental": { "x": true }
}));
c.restrict(restriction(&["code-action", "nope"], &[]));
assert!(c.has("codeActionProvider") && c.has_sub("codeActionProvider", "resolveProvider"));
assert_eq!(c.commands(), ["rope.extract"]);
for key in ["completionProvider", "hoverProvider", "documentFormattingProvider", "foldingRangeProvider"] {
assert!(!c.has(key), "{key}");
}
assert!(c.semantic_legend.is_none() && !c.semantic_tokens_full());
assert_eq!((c.sync, c.open_close), (SyncKind::Incremental, true));
assert!(c.file_operation("willRename").is_some() && c.raw["experimental"]["x"] == true);
assert!(!c.restriction.diagnostics());
assert_eq!(c.code_action_kinds(), None, "no kind filter: the server may return any kind");
// Later registrations go through the same filter.
c.register("textDocument/hover", None);
c.register("textDocument/semanticTokens", Some(&json!({ "legend": {}, "full": true })));
assert!(!c.has("hoverProvider") && !c.has("semanticTokensProvider"));
c.register("textDocument/didSave", None);
assert_eq!(c.save, Some(false));
let mut c = ServerCaps::parse(json!({ "hoverProvider": true, "semanticTokensProvider": { "full": true } }));
c.restrict(Restriction::default());
assert!(c.has("hoverProvider") && c.restriction.diagnostics(), "no `features`: everything");
c.restrict(restriction(&["semantic-tokens", "diagnostics"], &[]));
c.register("textDocument/semanticTokens", Some(&json!({ "legend": {}, "full": true })));
assert!(!c.has("hoverProvider") && c.semantic_tokens_full() && c.restriction.diagnostics());
c.register("textDocument/hover", None);
assert!(!c.has("hoverProvider"), "the restriction survives re-parsing semantic tokens");
}
#[test]
fn code_action_kinds_narrow_what_is_asked_and_kept() {
let r = restriction(&[], &["refactor", "source.fixAll"]);
assert!(r.keeps_action(Some("refactor")) && r.keeps_action(Some("refactor.extract.function")));
assert!(r.keeps_action(Some("source.fixAll.ruff")));
assert!(!r.keeps_action(Some("refactorx")) && !r.keeps_action(Some("source")) && !r.keeps_action(None));
assert!(Restriction::default().keeps_action(None));
let kinds = |provider: Value| {
let mut c = ServerCaps::parse(json!({ "codeActionProvider": provider }));
c.restrict(r.clone());
c.has("codeActionProvider").then(|| c.code_action_kinds().unwrap())
};
assert_eq!(kinds(json!(true)), Some(vec!["refactor".into(), "source.fixAll".into()]));
assert_eq!(
kinds(json!({ "codeActionKinds": ["quickfix", "refactor.extract", "source"] })),
Some(vec!["refactor.extract".into(), "source.fixAll".into()]),
);
assert_eq!(kinds(json!({ "codeActionKinds": ["quickfix"] })), None, "nothing it lists is kept");
let mut c = ServerCaps::parse(json!({}));
c.restrict(r.clone());
c.register("textDocument/codeAction", Some(&json!({ "codeActionKinds": ["quickfix", "refactor.inline"] })));
assert_eq!(c.code_action_kinds(), Some(vec!["refactor.inline".into()]));
}
#[test]
fn language_definitions_use_known_feature_names() {
for lang in ide_language::LanguageRegistry::load(None).unwrap().all() {
for server in &lang.language_servers {
for name in &server.features {
assert!(FEATURES.iter().any(|(n, _)| n == name), "{}: unknown feature `{name}`", server.name);
}
}
}
}
#[test]
fn feature_names_cover_every_capability_the_ide_asks_for() {
let named: Vec<&str> = FEATURES.iter().flat_map(|(_, keys)| keys.iter().copied()).collect();
for method in [
"textDocument/completion",
"textDocument/hover",
"textDocument/signatureHelp",
"textDocument/definition",
"textDocument/declaration",
"textDocument/typeDefinition",
"textDocument/implementation",
"textDocument/references",
"textDocument/documentHighlight",
"textDocument/documentSymbol",
"workspace/symbol",
"textDocument/codeAction",
"workspace/executeCommand",
"textDocument/rename",
"textDocument/formatting",
"textDocument/rangeFormatting",
"textDocument/inlayHint",
"textDocument/inlineCompletion",
"textDocument/semanticTokens",
"textDocument/codeLens",
"textDocument/documentLink",
"textDocument/prepareCallHierarchy",
"textDocument/prepareTypeHierarchy",
] {
let key = capability_key(method).unwrap();
assert!(named.contains(&key), "{method} ({key}) has no feature name");
}
}
}
-4
View File
@@ -111,10 +111,6 @@ impl ClientHandler {
}
fn publish_diagnostics(&self, mgr: &LspManager, params: Value) {
// Problems come only from servers used for `diagnostics`.
if self.server.running_gen(self.generation).is_some_and(|run| !run.caps.read().restriction.diagnostics()) {
return;
}
let params: lsp::PublishDiagnosticsParams = match serde_json::from_value(params) {
Ok(p) => p,
Err(e) => {
-1
View File
@@ -156,7 +156,6 @@ impl LspManager {
.iter()
.zip(ready)
.filter_map(|(s, run)| Some(Target { server: s.clone(), run: run? }))
.filter(|t| t.caps().restriction.diagnostics())
.collect();
if targets.is_empty() {
return Ok(None);
-1
View File
@@ -20,7 +20,6 @@ const SERVER: &str = "jdtls";
pub(crate) mod configuration;
mod formatter;
mod hierarchy;
pub(crate) mod reserved_names;
pub use formatter::configure_formatter;
/// First start imports the Maven / Gradle model, which can take minutes.
-101
View File
@@ -1,101 +0,0 @@
//! Pull Members Up / Push Members Down through the app-owned JDT extension
//! (`plugins/java-usages`). The server checks the move and computes the edit; the UI
//! shows its problems and applies the edit itself.
use super::COMMAND_TIMEOUT;
use crate::types::{HierarchyChoice, HierarchyMembers, HierarchyRefactored, HierarchyRefactoring, RefactoringProblem};
use crate::{DocSnap, LspManager, Target, convert};
use ide_core::{CancellationToken, DocId, IdeError, IdeResult, Pos};
use serde::Deserialize;
use serde::de::DeserializeOwned;
use serde_json::{Value, json};
const MEMBERS: &str = "openidea.java.hierarchy.members";
const REFACTOR: &str = "openidea.java.hierarchy.refactor";
/// The extension's answer: a result, or why the position can't be refactored.
#[derive(Deserialize)]
#[serde(untagged)]
enum Answer<T> {
Refused { refused: String },
Ok(T),
}
impl<T> Answer<T> {
fn into_result(self) -> IdeResult<T> {
match self {
Answer::Refused { refused } => Err(IdeError::invalid(refused)),
Answer::Ok(value) => Ok(value),
}
}
}
#[derive(Deserialize)]
struct Refactored {
problems: Vec<RefactoringProblem>,
edit: Option<lsp_types::WorkspaceEdit>,
}
fn feature(kind: HierarchyRefactoring) -> &'static str {
match kind {
HierarchyRefactoring::PullUp => "Pull Members Up",
HierarchyRefactoring::PushDown => "Push Members Down",
}
}
impl LspManager {
async fn hierarchy_command<R: DeserializeOwned>(
&self,
doc: DocId,
pos: Pos,
kind: HierarchyRefactoring,
command: &str,
extra: &[Value],
cancel: &CancellationToken,
) -> IdeResult<(R, DocSnap, Target)> {
let (snap, mut targets) =
self.doc_targets(doc, feature(kind), |c| c.commands().iter().any(|c| c == command)).await?;
let t = targets.swap_remove(0);
let at = snap.position(&t, pos);
let mut arguments = vec![json!(snap.uri), json!(at.line), json!(at.character), json!(kind)];
arguments.extend_from_slice(extra);
let params = json!({ "command": command, "arguments": arguments });
let answer: Answer<R> =
self.call(&t, "workspace/executeCommand", params, Some(cancel), COMMAND_TIMEOUT).await?;
if cancel.is_cancelled() {
return Err(IdeError::cancelled());
}
Ok((answer.into_result()?, snap, t))
}
/// The class at `pos` and its movable members; an invalid-input error says why there are none.
pub async fn hierarchy_members(
&self,
doc: DocId,
pos: Pos,
kind: HierarchyRefactoring,
cancel: CancellationToken,
) -> IdeResult<HierarchyMembers> {
let (members, _, _) = self.hierarchy_command(doc, pos, kind, MEMBERS, &[], &cancel).await?;
Ok(members)
}
/// Checks moving `choices` (to `target`, for pull up) and computes the edit without applying it.
pub async fn hierarchy_refactor(
&self,
doc: DocId,
pos: Pos,
kind: HierarchyRefactoring,
target: Option<String>,
choices: Vec<HierarchyChoice>,
cancel: CancellationToken,
) -> IdeResult<HierarchyRefactored> {
let choices: Vec<Value> = choices.iter().map(|c| json!({ "id": c.id, "abstract": c.make_abstract })).collect();
let (refactored, _, t): (Refactored, _, _) =
self.hierarchy_command(doc, pos, kind, REFACTOR, &[json!(target), json!(choices)], &cancel).await?;
let conv = self.conv(t.enc());
Ok(HierarchyRefactored {
problems: refactored.problems,
edit: refactored.edit.map(|e| convert::workspace_edit(&conv, e)),
})
}
}
+2 -34
View File
@@ -570,10 +570,7 @@ impl LspManager {
if let Some(info) = result.get("serverInfo") {
tracing::info!("{} initialized: {info}", server.name);
}
let mut caps = ServerCaps::parse(result.get("capabilities").cloned().unwrap_or_else(|| json!({})));
let cfg = server.effective_config();
caps.restrict(caps::Restriction::new(&server.name, &cfg.features, &cfg.code_action_kinds));
caps
ServerCaps::parse(result.get("capabilities").cloned().unwrap_or_else(|| json!({})))
}
Err(e) => {
tracing::warn!("{}: initialize failed: {e}", server.name);
@@ -1596,14 +1593,8 @@ impl LspManager {
Ok(actions) => {
let conv = self.conv(t.enc());
let acc = out.value.get_or_insert_with(Vec::new);
let caps = t.caps();
acc.extend(
actions
.unwrap_or_default()
.into_iter()
.map(|a| convert::code_action(&conv, a, t.server.id))
// Servers may ignore `only`, and the kinds kept for this one.
.filter(|a| caps.restriction.keeps_action(a.kind.as_deref())),
actions.unwrap_or_default().into_iter().map(|a| convert::code_action(&conv, a, t.server.id)),
);
}
Err(e) => out.fail(e),
@@ -1973,29 +1964,6 @@ impl LspManager {
}
}
/// Files that changed on disk, outside the editor too (a checkout, another tool): each running
/// server hears of those of its languages under its root through `workspace/didChangeWatchedFiles`.
/// Without it jdtls's incremental build never sees them, so a launch runs stale classes.
pub fn watched_files_changed(&self, created: &[PathBuf], modified: &[PathBuf], removed: &[PathBuf]) {
let servers: Vec<Arc<Server>> = self.servers.lock().clone();
for s in servers {
let Some(run) = s.running() else { continue };
let languages = s.languages();
let changes: Vec<Value> = [(created, 1), (modified, 2), (removed, 3)]
.into_iter()
.flat_map(|(paths, kind)| paths.iter().map(move |p| (p, kind)))
.filter(|(p, _)| p.starts_with(&s.root) && languages.contains(&self.languages.detect(p, None)))
.map(|(p, kind)| json!({ "uri": uri::path_to_uri(p), "type": kind }))
.collect();
if changes.is_empty() {
continue;
}
if let Err(e) = run.peer.notify("workspace/didChangeWatchedFiles", json!({ "changes": changes })) {
tracing::debug!("{}: didChangeWatchedFiles: {e}", s.name);
}
}
}
/// Checks initialized capabilities, including dynamic registrations, before choosing a formatter.
pub async fn can_format(&self, doc: DocId) -> IdeResult<bool> {
let snap = self.snapshot(doc)?;
-81
View File
@@ -389,87 +389,6 @@ pub struct Renamed {
pub incomplete: Option<ReferencesIncomplete>,
}
/// IDEA's class-hierarchy refactorings.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum HierarchyRefactoring {
PullUp,
PushDown,
}
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct HierarchyClass {
/// Server-owned identity, passed back as the pull-up destination.
pub id: String,
pub name: String,
pub qualified_name: String,
pub interface: bool,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum HierarchyMemberKind {
Field,
Method,
Type,
}
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct HierarchyMember {
pub id: String,
/// `name(Params) : Type`, as the server renders it.
pub label: String,
pub kind: HierarchyMemberKind,
/// The member at the caret.
pub selected: bool,
/// Can be declared abstract in the destination (pull up) or kept abstract (push down).
pub abstractable: bool,
}
/// The class at a position and what can move: pull-up destinations, or the subclasses
/// members are pushed into.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct HierarchyMembers {
pub class_name: String,
pub targets: Vec<HierarchyClass>,
pub members: Vec<HierarchyMember>,
}
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct HierarchyChoice {
pub id: String,
pub make_abstract: bool,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum ProblemSeverity {
Info,
Warning,
Error,
/// The refactoring can't run at all; there is no edit.
Fatal,
}
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct RefactoringProblem {
pub severity: ProblemSeverity,
pub message: String,
}
/// A checked refactoring: what the server found, and the edit unless a problem is fatal.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct HierarchyRefactored {
pub problems: Vec<RefactoringProblem>,
pub edit: Option<WorkspaceEdit>,
}
#[derive(Clone, Debug, Default, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct CallHierarchy {
-178
View File
@@ -225,8 +225,6 @@ fn server_cfg(name: &str) -> LanguageServerConfig {
install_hint: Some(format!("install {name}")),
connect: None,
fallback: false,
features: Vec::new(),
code_action_kinds: Vec::new(),
max_java: None,
launch_error: None,
tsserver_project: false,
@@ -405,37 +403,6 @@ async fn linked_projects_initialize_separate_language_server_roots() {
h.mgr.shutdown_all().await;
}
#[tokio::test(flavor = "multi_thread")]
async fn files_changed_on_disk_reach_the_servers_of_their_language() {
let h = harness(&[("mock", incremental_caps())]);
h.open("a.mk", "x");
h.wait_status("mock", ServerStatus::Ready).await;
let s = h.launcher.state("mock");
let (created, modified, removed) = (h.root.join("new.mk"), h.root.join("a.mk"), h.root.join("old.mk"));
// Other languages and files outside the server's root are left out.
let other = h.root.join("notes.txt");
let outside = tempfile::tempdir().unwrap().path().join("far.mk");
h.mgr.watched_files_changed(
&[created.clone(), other],
&[modified.clone(), outside],
std::slice::from_ref(&removed),
);
eventually(|| !s.notified("workspace/didChangeWatchedFiles").is_empty()).await;
assert_eq!(
s.notified("workspace/didChangeWatchedFiles")[0]["changes"],
json!([
{ "uri": uri_of(&created), "type": 1 },
{ "uri": uri_of(&modified), "type": 2 },
{ "uri": uri_of(&removed), "type": 3 },
])
);
// Nothing of its languages: no notification.
h.mgr.watched_files_changed(&[h.root.join("b.txt")], &[], &[]);
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
assert_eq!(s.notified("workspace/didChangeWatchedFiles").len(), 1);
h.mgr.shutdown_all().await;
}
#[tokio::test(flavor = "multi_thread")]
async fn initialize_and_incremental_sync() {
let h = harness(&[("mock", incremental_caps())]);
@@ -1782,64 +1749,6 @@ async fn code_action_query_without_listed_kinds() {
assert_eq!(q.advertised_kinds, None, "`codeActionProvider` without `codeActionKinds` may return any kind");
}
#[tokio::test(flavor = "multi_thread")]
async fn features_limit_what_a_server_is_used_for() {
// A refactoring-only server listed first, announcing everything the main one does.
let h = harness_configured("mock", "mk", &[("rope", incremental_caps()), ("main", incremental_caps())], |cfg| {
if cfg.name == "rope" {
cfg.features = vec!["code-action".into()];
cfg.code_action_kinds = vec!["refactor".into()];
}
});
let (doc, path) = h.open("a.mk", "let x = 1;\n");
for name in ["rope", "main"] {
h.wait_status(name, ServerStatus::Ready).await;
}
let (rope, main) = (h.launcher.state("rope"), h.launcher.state("main"));
for s in [&rope, &main] {
s.respond("textDocument/hover", json!({ "contents": "x" }));
s.respond("textDocument/formatting", json!([]));
}
h.mgr.hover(doc, Pos::new(0, 4), Default::default()).await.unwrap();
assert_eq!(main.requested("textDocument/hover").len(), 1);
h.mgr
.format(doc, None, FormatOptions { tab_size: 2, insert_spaces: true, on_save: false, tool: None })
.await
.unwrap();
assert_eq!(main.requested("textDocument/formatting").len(), 1);
assert!(rope.requested("textDocument/hover").is_empty() && rope.requested("textDocument/formatting").is_empty());
rope.respond(
"textDocument/codeAction",
json!([
{ "title": "Organize imports", "kind": "source.organizeImports" },
{ "title": "Extract variable", "kind": "refactor.extract.variable" },
{ "title": "Run", "command": "mock.run" }
]),
);
main.respond("textDocument/codeAction", json!([{ "title": "Fix", "kind": "quickfix" }]));
let q = h.mgr.code_action_query(doc, rng(0, 8, 0, 9), vec![], ide_core::CancellationToken::new()).await.unwrap();
let mut titles: Vec<&str> = q.actions.iter().map(|a| a.title.as_str()).collect();
titles.sort();
assert_eq!(titles, ["Extract variable", "Fix"], "rope's other kinds and bare commands are dropped");
let diag =
json!([{ "range": {"start":{"line":0,"character":4},"end":{"line":0,"character":5}}, "message": "unused" }]);
rope.peer()
.notify("textDocument/publishDiagnostics", json!({ "uri": uri_of(&path), "diagnostics": diag }))
.unwrap();
main.peer()
.notify("textDocument/publishDiagnostics", json!({ "uri": uri_of(&path), "diagnostics": diag }))
.unwrap();
eventually(|| {
h.events.lock().iter().any(|e| matches!(e, IdeEvent::Diagnostics { source, .. } if source == "main"))
})
.await;
tokio::time::sleep(Duration::from_millis(100)).await;
assert!(!h.events.lock().iter().any(|e| matches!(e, IdeEvent::Diagnostics { source, .. } if source == "rope")));
}
#[tokio::test(flavor = "multi_thread")]
async fn shutdown_all_stops_gracefully() {
let h = harness(&[("mock", incremental_caps())]);
@@ -3338,90 +3247,3 @@ async fn jdtls_import_failure_names_reserved_device_files_on_windows() {
}
let _ = std::fs::remove_file(&nul);
}
/// Pull Members Up / Push Members Down: the extension's answers, refusals and edits.
#[tokio::test(flavor = "multi_thread")]
async fn java_hierarchy_refactorings_use_the_extension_commands() {
let mut caps = incremental_caps();
caps["executeCommandProvider"] =
json!({"commands": ["openidea.java.hierarchy.members", "openidea.java.hierarchy.refactor"]});
let h = harness_for("java", "java", &[("jdtls", caps)], &[]);
let s = h.launcher.state("jdtls");
*s.responder.lock() = Some(Arc::new(|_, method, params| {
if method != "workspace/executeCommand" {
return None;
}
let args = params["arguments"].as_array().unwrap();
let uri = args[0].clone();
Some(match (params["command"].as_str().unwrap(), args[1].as_u64().unwrap()) {
(_, 9) => json!({"refused": "Type has no super types."}),
("openidea.java.hierarchy.members", _) => json!({
"className": "Dog",
"targets": [{"id": "=p/src<{Animal.java[Animal", "name": "Animal", "qualifiedName": "Animal", "interface": false}],
"members": [{"id": "=p/src<{Dog.java[Dog~bark", "label": "bark() : String", "kind": "method", "selected": true, "abstractable": true}],
}),
_ => json!({
"problems": [{"severity": "warning", "message": "The visibility of field 'tail' will be changed to package."}],
"edit": {"changes": {uri.as_str().unwrap(): [
{"range": {"start": {"line": 0, "character": 0}, "end": {"line": 0, "character": 5}}, "newText": "class"}
]}},
}),
})
}));
let text =
format!("class Dog extends Animal {{ String bark() {{ return \"\"; }} }}{}class Lonely {{}}", "\n".repeat(9));
let (doc, path) = h.open("Dog.java", &text);
h.wait_status("jdtls", ServerStatus::Ready).await;
eventually(|| s.docs.lock().len() == 1).await;
let info =
h.mgr.hierarchy_members(doc, Pos::new(0, 35), HierarchyRefactoring::PullUp, Default::default()).await.unwrap();
assert_eq!(info.class_name, "Dog");
assert_eq!(info.targets[0].name, "Animal");
assert!(info.members[0].selected && info.members[0].abstractable);
assert_eq!(info.members[0].kind, HierarchyMemberKind::Method);
let sent = s.requested("workspace/executeCommand");
assert_eq!(sent[0]["arguments"], json!([uri_of(&path), 0, 35, "pullUp"]));
let choices = vec![HierarchyChoice { id: info.members[0].id.clone(), make_abstract: true }];
let done = h
.mgr
.hierarchy_refactor(
doc,
Pos::new(0, 35),
HierarchyRefactoring::PushDown,
Some(info.targets[0].id.clone()),
choices,
Default::default(),
)
.await
.unwrap();
assert_eq!(done.problems[0].severity, ProblemSeverity::Warning);
let edit = done.edit.unwrap();
assert!(matches!(&edit.files[..], [FileEdit::Edit { path: p, edits, .. }] if p == &path && edits.len() == 1));
let sent = s.requested("workspace/executeCommand");
assert_eq!(
sent[1]["arguments"],
json!([uri_of(&path), 0, 35, "pushDown", "=p/src<{Animal.java[Animal",
[{"id": "=p/src<{Dog.java[Dog~bark", "abstract": true}]])
);
let refused = h.mgr.hierarchy_members(doc, Pos::new(9, 0), HierarchyRefactoring::PullUp, Default::default()).await;
let e = refused.unwrap_err();
assert_eq!(e.kind, ide_core::error::ErrorKind::InvalidInput);
assert_eq!(e.message, "Type has no super types.");
}
#[tokio::test(flavor = "multi_thread")]
async fn hierarchy_refactorings_name_the_servers_without_them() {
let h = harness(&[("plain", incremental_caps())]);
let (doc, _) = h.open("a.mk", "class A {}");
h.wait_status("plain", ServerStatus::Ready).await;
let e = h
.mgr
.hierarchy_members(doc, Pos::new(0, 6), HierarchyRefactoring::PullUp, Default::default())
.await
.unwrap_err();
assert_eq!(e.kind, ide_core::error::ErrorKind::Unavailable);
assert_eq!(e.message, "plain doesn't support Pull Members Up");
}
-6
View File
@@ -129,8 +129,6 @@ async fn missing_binary_is_reported_with_install_hint() {
install_hint: Some("npm i -g nope-ls".into()),
connect: None,
fallback: false,
features: Vec::new(),
code_action_kinds: Vec::new(),
max_java: None,
launch_error: None,
tsserver_project: false,
@@ -398,8 +396,6 @@ async fn typescript_language_server_end_to_end() {
install_hint: None,
connect: None,
fallback: false,
features: Vec::new(),
code_action_kinds: Vec::new(),
max_java: None,
launch_error: None,
tsserver_project: false,
@@ -447,8 +443,6 @@ fn npm_server(tools: &Path, name: &str, bin: &str, init: Option<serde_json::Valu
install_hint: None,
connect: None,
fallback: false,
features: Vec::new(),
code_action_kinds: Vec::new(),
max_java: None,
launch_error: None,
tsserver_project: false,
+35 -96
View File
@@ -467,6 +467,7 @@ const MAX_SYNTAX_ITEMS: usize = 200_000;
const MAX_BATCH: usize = 1000;
fn event_to_wit(e: WorkbenchEvent) -> wit_events::Event {
use crate::types::ActionSource as S;
match e {
WorkbenchEvent::FileOpened(p) => wit_events::Event::FileOpened(p),
WorkbenchEvent::FileSaved(p) => wit_events::Event::FileSaved(p),
@@ -476,7 +477,14 @@ fn event_to_wit(e: WorkbenchEvent) -> wit_events::Event {
WorkbenchEvent::ActionPerformed(a) => wit_events::Event::ActionPerformed(wit_events::ActionPerformed {
action: a.action,
title: a.title,
source: a.source.name().to_owned(),
source: match a.source {
S::Keyboard => wit_events::ActionSource::Keyboard,
S::Menu => wit_events::ActionSource::Menu,
S::Toolbar => wit_events::ActionSource::Toolbar,
S::ContextMenu => wit_events::ActionSource::ContextMenu,
S::Search => wit_events::ActionSource::Search,
S::Other => wit_events::ActionSource::Other,
},
shortcut: a.shortcut,
}),
}
@@ -662,17 +670,9 @@ impl Runtime {
detail: i.detail.map(|d| clip(d, MAX_TITLE)),
kind: i.kind.map(|k| clip(k, 32)),
insert_text: clip(i.insert_text, 64 << 10),
snippet: i.snippet,
replace: i.replace.map(range_from_wit),
sort_text: i.sort_text.map(|t| clip(t, 64)),
filter_text: i.filter_text.map(|t| clip(t, MAX_TITLE)),
documentation: i.documentation.map(|d| clip(d, 16 << 10)),
additional_edits: i
.additional_edits
.into_iter()
.take(MAX_ADDITIONAL_EDITS)
.map(edit_from_wit)
.collect(),
})
.collect())
}
@@ -699,20 +699,27 @@ impl Runtime {
) -> Result<Vec<CodeAction>, CallError> {
let d = doc_to_wit(doc);
let actions = self.provider_call(timeout, |p, s| p.call_code_actions(s, &d, range_to_wit(selection)))?;
Ok(actions.into_iter().take(MAX_PROVIDER_ITEMS).map(code_action_from_wit).collect())
}
/// The edits of an action `provide_code_actions` returned with `data`.
pub fn resolve_code_action(
&mut self,
doc: &ProviderDocument,
action: &CodeAction,
timeout: Duration,
) -> Result<CodeAction, CallError> {
let d = doc_to_wit(doc);
let a = code_action_to_wit(action);
let resolved = self.provider_call(timeout, |p, s| p.call_resolve_code_action(s, &d, &a))?;
resolved.map(code_action_from_wit).map_err(|e| CallError::Guest(e.into()))
Ok(actions
.into_iter()
.take(MAX_PROVIDER_ITEMS)
.map(|a| CodeAction {
title: clip(a.title, MAX_TITLE),
kind: a.kind.map(|k| clip(k, 64)),
preferred: a.preferred,
edits: a
.edits
.into_iter()
.map(|f| {
let edits = f
.edits
.into_iter()
.map(|e| ide_core::TextEdit { range: range_from_wit(e.range), new_text: e.text })
.collect();
(std::path::PathBuf::from(f.path), edits)
})
.collect(),
})
.collect())
}
pub fn provide_definition(
@@ -961,47 +968,6 @@ const MAX_VIEW_ITEMS: usize = 20_000;
const MAX_CELL: usize = 64 << 10;
const MAX_HTML: usize = 4 << 20;
/// Most additional edits of a completion item.
const MAX_ADDITIONAL_EDITS: usize = 100;
fn edit_from_wit(e: wit_types::TextEdit) -> ide_core::TextEdit {
ide_core::TextEdit { range: range_from_wit(e.range), new_text: e.text }
}
fn code_action_from_wit(a: wit_prov::CodeAction) -> CodeAction {
CodeAction {
title: clip(a.title, MAX_TITLE),
kind: a.kind.map(|k| clip(k, 64)),
preferred: a.preferred,
edits: a
.edits
.into_iter()
.map(|f| (std::path::PathBuf::from(f.path), f.edits.into_iter().map(edit_from_wit).collect()))
.collect(),
data: a.data.map(|d| clip(d, 64 << 10)),
}
}
fn code_action_to_wit(a: &CodeAction) -> wit_prov::CodeAction {
wit_prov::CodeAction {
title: a.title.clone(),
kind: a.kind.clone(),
preferred: a.preferred,
edits: a
.edits
.iter()
.map(|(path, edits)| wit_prov::FileEdit {
path: path.to_string_lossy().into_owned(),
edits: edits
.iter()
.map(|e| wit_types::TextEdit { range: range_to_wit(e.range), text: e.new_text.clone() })
.collect(),
})
.collect(),
data: a.data.clone(),
}
}
fn opt_clip(s: Option<String>, max: usize) -> Option<String> {
s.map(|s| clip(s, max))
}
@@ -1009,9 +975,6 @@ fn opt_clip(s: Option<String>, max: usize) -> Option<String> {
fn view_from_wit(v: wit_vt::View) -> View {
let body = match v.body {
wit_vt::Body::Message(m) => ViewBody::Message(clip(m, 16 << 10)),
wit_vt::Body::Other(e) => {
ViewBody::Message(format!("This view needs a newer version of OpenIDEA ({}).", clip(e.kind, 64)))
}
wit_vt::Body::Tree(items) => ViewBody::Tree(
items
.into_iter()
@@ -1043,8 +1006,10 @@ fn view_from_wit(v: wit_vt::View) -> View {
fields
.into_iter()
.take(MAX_VIEW_ITEMS)
.filter_map(|f| {
let value = match f.value {
.map(|f| ViewField {
key: clip(f.key, 128),
label: clip(f.label, MAX_TITLE),
value: match f.value {
wit_vt::FieldValue::Text(t) => FieldValue::Text(clip(t, MAX_CELL)),
wit_vt::FieldValue::Multiline(t) => FieldValue::Multiline(clip(t, 1 << 20)),
wit_vt::FieldValue::Checkbox(b) => FieldValue::Checkbox(b),
@@ -1052,10 +1017,7 @@ fn view_from_wit(v: wit_vt::View) -> View {
options: options.into_iter().take(1000).map(|o| clip(o, MAX_TITLE)).collect(),
selected: clip(selected, MAX_TITLE),
},
// A field type of a newer API: left out.
wit_vt::FieldValue::Other(_) => return None,
};
Some(ViewField { key: clip(f.key, 128), label: clip(f.label, MAX_TITLE), value })
},
})
.collect(),
),
@@ -1105,26 +1067,3 @@ fn event_to_wit_view(e: ViewEvent) -> wit_vt::ViewEvent {
ViewEvent::Message { text } => W::Message(text),
}
}
#[cfg(test)]
mod tests {
use super::*;
fn ext(kind: &str) -> wit_types::Extension {
wit_types::Extension { kind: kind.into(), data: "{}".into() }
}
#[test]
fn bodies_and_fields_of_a_newer_api_degrade() {
let view = |body| wit_vt::View { toolbar: Vec::new(), body, status: None };
let shown = view_from_wit(view(wit_vt::Body::Other(ext("markdown"))));
assert_eq!(shown.body, ViewBody::Message("This view needs a newer version of OpenIDEA (markdown).".into()));
let field = |key: &str, value| wit_vt::Field { key: key.into(), label: key.into(), value };
let form = view_from_wit(view(wit_vt::Body::Form(vec![
field("color", wit_vt::FieldValue::Other(ext("color"))),
field("name", wit_vt::FieldValue::Text("n".into())),
])));
let ViewBody::Form(fields) = form.body else { panic!("a form") };
assert_eq!(fields.iter().map(|f| f.key.as_str()).collect::<Vec<_>>(), ["name"], "unknown fields are left out");
}
}
-31
View File
@@ -126,20 +126,6 @@ pub enum ActionSource {
Other,
}
impl ActionSource {
/// As plugins see it.
pub fn name(self) -> &'static str {
match self {
Self::Keyboard => "keyboard",
Self::Menu => "menu",
Self::Toolbar => "toolbar",
Self::ContextMenu => "context-menu",
Self::Search => "search",
Self::Other => "other",
}
}
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct ActionPerformed {
@@ -317,16 +303,6 @@ pub fn net_host(url: &str) -> Result<String, String> {
#[cfg(test)]
mod tests {
#[test]
fn the_restart_backoff_doubles_up_to_a_minute() {
let limits = Limits { restart_backoff: Duration::from_millis(150), ..Limits::default() };
let waits: Vec<u128> = (1..=4).map(|n| limits.backoff(n).as_millis()).collect();
assert_eq!(waits, [150, 300, 600, 1200]);
assert_eq!(limits.backoff(30), Duration::from_millis(150 * 64), "the factor stops growing");
let long = Limits { restart_backoff: Duration::from_secs(10), ..Limits::default() };
assert_eq!(long.backoff(5), Duration::from_secs(60));
}
use super::*;
#[test]
@@ -362,14 +338,9 @@ pub struct CompletionItem {
/// WIT `completion-kind` name (`method`, `keyword`, …).
pub kind: Option<String>,
pub insert_text: String,
/// `insert_text` has tab stops and placeholders.
pub snippet: bool,
pub replace: Option<ide_core::Range>,
pub sort_text: Option<String>,
pub filter_text: Option<String>,
pub documentation: Option<String>,
/// Applied with the item elsewhere in the document (an import).
pub additional_edits: Vec<ide_core::TextEdit>,
}
#[derive(Clone, Debug, PartialEq, Eq)]
@@ -384,8 +355,6 @@ pub struct CodeAction {
pub kind: Option<String>,
pub preferred: bool,
pub edits: Vec<(PathBuf, Vec<ide_core::TextEdit>)>,
/// The plugin's reference: the edits come from `resolve_code_action`.
pub data: Option<String>,
}
#[derive(Clone, Debug, PartialEq, Eq)]
+18 -11
View File
@@ -440,11 +440,20 @@ impl vcs::Host for StoreData {
}
}
fn kind_from_wit(k: events::EventKind) -> EventKind {
match k {
events::EventKind::FileOpened => EventKind::FileOpened,
events::EventKind::FileSaved => EventKind::FileSaved,
events::EventKind::EditorFocused => EventKind::EditorFocused,
events::EventKind::ActionPerformed => EventKind::ActionPerformed,
events::EventKind::VcsChanged => EventKind::VcsChanged,
events::EventKind::Scheduled => EventKind::Scheduled,
}
}
impl events::Host for StoreData {
fn subscribe(&mut self, kind: String) -> Fallible<()> {
let Some(kind) = EventKind::parse(&kind) else {
return Ok(Err(wit::Error::Invalid(format!("no event kind {kind:?} in this OpenIDEA"))));
};
fn subscribe(&mut self, kind: events::EventKind) -> Fallible<()> {
let kind = kind_from_wit(kind);
if let Some(cap) = kind.capability()
&& let Err(e) = self.require(cap)
{
@@ -454,10 +463,8 @@ impl events::Host for StoreData {
Ok(Ok(()))
}
fn unsubscribe(&mut self, kind: String) -> wasmtime::Result<()> {
if let Some(kind) = EventKind::parse(&kind) {
self.host.subscribe(kind, false);
}
fn unsubscribe(&mut self, kind: events::EventKind) -> wasmtime::Result<()> {
self.host.subscribe(kind_from_wit(kind), false);
Ok(())
}
}
@@ -535,10 +542,10 @@ impl diagnostics::Host for StoreData {
diagnostics::Severity::Hint => ide_core::Severity::Hint,
},
message: clip(d.message, 4096),
source: Some(d.source.map_or_else(|| source.clone(), |s| clip(s, 128))),
source: Some(source.clone()),
code: d.code.map(|c| clip(c, 128)),
unnecessary: d.tags.iter().any(|t| t == "unnecessary"),
deprecated: d.tags.iter().any(|t| t == "deprecated"),
unnecessary: false,
deprecated: false,
server: Some(source.clone()),
})
.collect();
Binary file not shown.
@@ -1 +1 @@
c9acbc021f3840649823cb7680e13907d429c6fda8271fdb0ae6e5ea89ed15ee
86dfe70a30ab1f7955a16e96c1b32c8be663456b871cc4d3bed7dc6052c062da
Binary file not shown.
@@ -1 +1 @@
2fb6aa879562733bf5b7f11dabe8fc6a415c9ef695868fe7845417a4245d2dc8
0dacb82ac35c1f132dc66e85dbf0663b729f818279d3b32cd06403e502fbf880
Binary file not shown.
@@ -1 +1 @@
25b0f419a68a3b1651d3c2774d834b05e40ede52bc5ed8c0eba6d27a204efe82
0ad2e4c6977a576b7eb27fc1e535fd056dd9da64bdfe6cfda9ca9f57537d0384
@@ -40,11 +40,3 @@ title = "insert"
[[contributes.commands]]
id = "cancel"
title = "cancel"
[[contributes.commands]]
id = "subscribe"
title = "subscribe"
[[contributes.commands]]
id = "diagnose"
title = "diagnose"
+4 -52
View File
@@ -14,9 +14,7 @@ wit_bindgen::generate!({
generate_all,
});
use exports::openidea::plugin::providers::{
self, CodeAction, CompletionItem, Document, FileEdit, InlineSuggestion, LineMarker,
};
use exports::openidea::plugin::providers::{self, CodeAction, CompletionItem, Document, InlineSuggestion, LineMarker};
use exports::openidea::plugin::syntax_provider::{self, Fold, Symbol, Token};
use exports::openidea::plugin::{commands, event_handler, lifecycle};
use openidea::plugin::{invocation::Invocation, types};
@@ -69,23 +67,6 @@ impl commands::Guest for Misbehave {
}
// The user gave up: not the plugin failing.
"cancel" => Err(types::Error::Cancelled),
// Event kinds are names: one this IDE doesn't know is refused.
"subscribe" => openidea::plugin::events::subscribe(&arg.unwrap_or_default()).map(|()| None),
// Tagged problems from another tool on the line 0 of `arg`.
"diagnose" => {
use openidea::plugin::diagnostics::{self, Diagnostic, Severity};
let start = types::Position { line: 0, character: 0 };
let d = Diagnostic {
range: types::Range { start, end: start },
severity: Severity::Warning,
message: "unused".into(),
code: None,
source: Some("lint".into()),
tags: vec!["unnecessary".into(), "glowing".into()],
related: Vec::new(),
};
diagnostics::publish(&arg.unwrap_or_default(), &[d]).map(|()| None)
}
other => Err(types::Error::NotFound(format!("command {other}"))),
}
}
@@ -132,24 +113,14 @@ impl syntax_provider::Guest for Misbehave {
impl providers::Guest for Misbehave {
fn completion(doc: Document, _at: types::Position, _trigger: Option<String>) -> Vec<CompletionItem> {
behave(&doc.text);
// `rich`: a snippet that also adds an import.
let rich = doc.text == "rich";
let start = types::Position { line: 0, character: 0 };
vec![CompletionItem {
label: doc.path.clone(),
detail: None,
kind: Some("keyword".into()),
insert_text: if rich { "f(${1:x})$0".into() } else { doc.path },
snippet: rich,
insert_text: doc.path,
replace: None,
sort_text: None,
filter_text: rich.then(|| "fun".into()),
documentation: None,
additional_edits: if rich {
vec![types::TextEdit { range: types::Range { start, end: start }, text: "use f;\n".into() }]
} else {
Vec::new()
},
}]
}
@@ -157,27 +128,8 @@ impl providers::Guest for Misbehave {
None
}
/// `lazy`: one action whose edits come from `resolve-code-action`.
fn code_actions(doc: Document, _selection: types::Range) -> Vec<CodeAction> {
if doc.text != "lazy" {
return Vec::new();
}
vec![CodeAction {
title: "Fix later".into(),
kind: Some("quickfix".into()),
preferred: false,
edits: Vec::new(),
data: Some("fix".into()),
}]
}
fn resolve_code_action(doc: Document, action: CodeAction) -> Result<CodeAction, types::Error> {
if action.data.as_deref() != Some("fix") {
return Err(types::Error::Invalid("not one of mine".into()));
}
let start = types::Position { line: 0, character: 0 };
let edit = types::TextEdit { range: types::Range { start, end: start }, text: "fixed ".into() };
Ok(CodeAction { edits: vec![FileEdit { path: doc.path, edits: vec![edit] }], ..action })
fn code_actions(_doc: Document, _selection: types::Range) -> Vec<CodeAction> {
Vec::new()
}
fn definition(_doc: Document, _at: types::Position) -> Vec<types::Location> {
+11 -17
View File
@@ -52,16 +52,11 @@ fn local_folder(local: &str, project_root: &Path) -> Option<PathBuf> {
{
return Some(project_root.join(rel));
}
let relative = project_root.join(without_root(path));
let rel = local.trim_matches('/');
let relative = project_root.join(rel);
if !path.is_absolute() || relative.is_dir() || !path.exists() { Some(relative) } else { None }
}
/// `path` without its drive and root (`/a/b`, `C:\a\b` → `a/b`), to read it
/// relative to the project: joining an absolute path would replace the root.
fn without_root(path: &Path) -> PathBuf {
path.components().filter(|c| !matches!(c, Component::Prefix(_) | Component::RootDir)).collect()
}
impl Mappings {
/// `remote_root` is the server root resolved on the host (its login
/// directory when the setting is empty). A server without mappings maps the
@@ -252,15 +247,14 @@ mod tests {
#[test]
fn local_paths_may_be_absolute_inside_the_project() {
let root = Path::new(if cfg!(windows) { r"C:\p" } else { "/p" });
let (inside, statics) = (root.to_str().unwrap(), format!("{}/static/", root.display()));
let m = Mappings::new(&server(vec![(inside, "/", ""), (&statics, "cdn", "")]), root, "/mnt/deploy");
assert_eq!(m.remote(&root.join("src/a.py")).as_deref(), Some("/mnt/deploy/src/a.py"));
assert_eq!(m.remote(&root.join("static/x.css")).as_deref(), Some("/mnt/deploy/cdn/x.css"));
assert_eq!(m.local("/mnt/deploy/cdn/x.css"), Some(root.join("static/x.css")));
let root = Path::new("/p");
let m = Mappings::new(&server(vec![("/p", "/", ""), ("/p/static/", "cdn", "")]), root, "/mnt/deploy");
assert_eq!(m.remote(Path::new("/p/src/a.py")).as_deref(), Some("/mnt/deploy/src/a.py"));
assert_eq!(m.remote(Path::new("/p/static/x.css")).as_deref(), Some("/mnt/deploy/cdn/x.css"));
assert_eq!(m.local("/mnt/deploy/cdn/x.css"), Some(PathBuf::from("/p/static/x.css")));
// A leading `/` that is not the project stays project-relative.
let m = Mappings::new(&server(vec![("/static/", "", "")]), root, "/mnt/deploy");
assert_eq!(m.remote(&root.join("static/x.css")).as_deref(), Some("/mnt/deploy/x.css"));
assert_eq!(m.remote(Path::new("/p/static/x.css")).as_deref(), Some("/mnt/deploy/x.css"));
}
#[test]
@@ -270,7 +264,7 @@ mod tests {
let outside = tmp.path().join("outside");
std::fs::create_dir_all(&root).unwrap();
std::fs::create_dir(&outside).unwrap();
let nested = root.join(without_root(&outside)).join("a.txt");
let nested = root.join(outside.strip_prefix("/").unwrap()).join("a.txt");
std::fs::create_dir_all(nested.parent().unwrap().parent().unwrap()).unwrap();
// A nested project file at the outside path's spelling is not a mapping folder.
std::fs::write(nested.parent().unwrap(), "nested").unwrap();
@@ -294,7 +288,7 @@ mod tests {
assert_eq!(local_folder("/", &root), Some(root.clone()));
let outside = tmp.path().join("outside");
std::fs::create_dir(&outside).unwrap();
let relative = root.join(without_root(&outside));
let relative = root.join(outside.strip_prefix("/").unwrap());
std::fs::create_dir_all(&relative).unwrap();
assert_eq!(local_folder(outside.to_str().unwrap(), &root), Some(relative.clone()));
let nested = relative.join("a.txt");
@@ -303,7 +297,7 @@ mod tests {
assert_eq!(m.remote(&nested).as_deref(), Some("/srv/a.txt"));
assert_eq!(m.local("/srv/a.txt"), Some(nested));
let missing = tmp.path().join("missing");
assert_eq!(local_folder(missing.to_str().unwrap(), &root), Some(root.join(without_root(&missing))));
assert_eq!(local_folder(missing.to_str().unwrap(), &root), Some(root.join(missing.strip_prefix("/").unwrap())));
assert_eq!(local_folder("/src/", &root), Some(root.join("src")));
}
+2 -2
View File
@@ -151,7 +151,7 @@ fn relative(root: &Path, dir: &Path) -> PathBuf {
// ---------------------------------------------------------------- node
pub(crate) fn package_manager(root: &Path, pkg: &Json) -> &'static str {
fn package_manager(root: &Path, pkg: &Json) -> &'static str {
if let Some(pm) = pkg.get("packageManager").and_then(Json::as_str) {
match pm.split('@').next().unwrap_or_default() {
"pnpm" => return "pnpm",
@@ -172,7 +172,7 @@ pub(crate) fn package_manager(root: &Path, pkg: &Json) -> &'static str {
}
}
pub(crate) fn is_test_script(name: &str) -> bool {
fn is_test_script(name: &str) -> bool {
matches!(name, "test" | "vitest" | "jest") || name.starts_with("test:")
}
-922
View File
@@ -1,922 +0,0 @@
//! JetBrains run configurations (PyCharm, WebStorm, IntelliJ IDEA): the shared `.run/*.run.xml`
//! files, `.idea/runConfigurations/*.xml`, and the `RunManager` of `.idea/workspace.xml` without
//! its templates (`default="true"`) and temporary configurations. A configuration found in more
//! than one place is taken from the first, in that order.
//!
//! Python, Django server, Django tests, pytest, npm, Jest, Vitest, Node.js, Shell Script and
//! compound configurations become the matching kinds; other types are skipped with their type.
//! `$PROJECT_DIR$` (and `$MODULE_DIR$`, `$MODULE_WORKING_DIR$`, `$ContentRoot$`,
//! `$ProjectFileDir$`, all the project directory here) and `$USER_HOME$` are replaced; another
//! path variable skips the configuration.
use super::{
Entry, Found, SkipReason, command_name, config, env_var, normalize, path_from, project_python, relative_to,
working_dir,
};
use crate::RunConfig;
use ide_core::IdeResult;
use quick_xml::{Reader, XmlVersion, events::Event};
use std::cell::Cell;
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
/// The files read, in order of precedence, with the name the summary gives them.
pub(crate) fn files(root: &Path) -> Vec<(String, PathBuf)> {
let mut out = Vec::new();
for (dir, suffix) in [(".run", ".run.xml"), (".idea/runConfigurations", ".xml")] {
let mut found: Vec<PathBuf> = std::fs::read_dir(root.join(dir))
.into_iter()
.flatten()
.flatten()
.map(|e| e.path())
.filter(|p| p.is_file() && p.file_name().is_some_and(|n| n.to_string_lossy().ends_with(suffix)))
.collect();
found.sort();
for p in found {
let name = p.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
out.push((format!("{dir}/{name}"), p));
}
}
// IDEA writes a workspace.xml in every project it opens: it counts only with configurations to import.
let workspace = root.join(".idea/workspace.xml");
let has_configurations = std::fs::read_to_string(&workspace).is_ok_and(|text| {
text.contains("RunManager")
&& parse(&text).map_or(true, |doc| configurations(&doc, "RunManager").next().is_some())
});
if has_configurations {
out.push((".idea/workspace.xml".into(), workspace));
}
out
}
/// The configurations of `manager` (`RunManager` in workspace.xml), without IDEA's templates for
/// new configurations and the temporary ones it makes for a single run.
fn configurations<'a>(doc: &'a El, manager: &'a str) -> impl Iterator<Item = &'a El> {
std::iter::once(doc)
.chain(doc.children.iter())
.filter(move |c| c.name == "component" && c.attr("name") == Some(manager))
.flat_map(|component| component.children_named("configuration"))
.filter(|c| c.attr("default") != Some("true") && c.attr("temporary") != Some("true"))
}
pub(crate) fn entries(root: &Path, env: &BTreeMap<String, String>) -> IdeResult<Vec<Entry>> {
let windows = cfg!(windows);
let homes = if windows { ["USERPROFILE", "HOME"] } else { ["HOME", "USERPROFILE"] };
let home = homes.into_iter().find_map(|k| env_var(env, k, windows));
let cx = Cx { root: &normalize(root), home, modules: module_count(root), machine_values: Cell::new(false) };
let mut out = Vec::new();
for (label, path) in files(root) {
let doc = std::fs::read_to_string(&path).map_err(|e| e.to_string()).and_then(|text| parse(&text));
let doc = match doc {
Ok(doc) => doc,
Err(detail) => {
out.push(Entry { name: label, result: Err(SkipReason::InvalidFile { detail }) });
continue;
}
};
let workspace = label == ".idea/workspace.xml";
let manager = if workspace { "RunManager" } else { "ProjectRunConfigurationManager" };
out.extend(configurations(&doc, manager).map(|c| cx.entry(c)));
}
Ok(out)
}
// ------------------------------------------------------------------------- XML
/// An element of the file, with its attributes (values unescaped) and child elements.
#[derive(Debug, Default)]
struct El {
name: String,
attrs: Vec<(String, String)>,
children: Vec<El>,
}
impl El {
fn attr(&self, key: &str) -> Option<&str> {
self.attrs.iter().find(|(k, _)| k == key).map(|(_, v)| v.as_str())
}
fn child(&self, name: &str) -> Option<&El> {
self.children.iter().find(|c| c.name == name)
}
fn children_named<'a>(&'a self, name: &'a str) -> impl Iterator<Item = &'a El> {
self.children.iter().filter(move |c| c.name == name)
}
/// `<option name="KEY" value="…"/>`, a direct child.
fn option(&self, key: &str) -> Option<&str> {
self.children_named("option").find(|o| o.attr("name") == Some(key)).and_then(|o| o.attr("value"))
}
/// `<KEY value="…"/>` (the WebStorm types write their fields so).
fn value(&self, key: &str) -> Option<&str> {
self.child(key).and_then(|c| c.attr("value"))
}
fn flag(&self, key: &str) -> bool {
self.option(key) == Some("true")
}
}
fn parse(xml: &str) -> Result<El, String> {
let mut reader = Reader::from_str(xml);
let mut stack: Vec<El> = vec![El::default()];
loop {
let event = reader.read_event().map_err(|e| e.to_string())?;
match event {
Event::Start(ref tag) | Event::Empty(ref tag) => {
let mut el = El { name: tag.name().as_ref().to_owned(), ..El::default() };
for a in tag.attributes() {
let a = a.map_err(|e| e.to_string())?;
let value = a.normalized_value(XmlVersion::Implicit1_0).map_err(|e| e.to_string())?;
el.attrs.push((a.key.as_ref().to_owned(), value.into_owned()));
}
if matches!(event, Event::Start(_)) {
stack.push(el);
} else if let Some(parent) = stack.last_mut() {
parent.children.push(el);
}
}
Event::End(_) => {
let el = stack.pop().ok_or("unbalanced element")?;
stack.last_mut().ok_or("unbalanced element")?.children.push(el);
}
Event::Eof => break,
_ => {}
}
}
let mut document = stack.pop().filter(|_| stack.is_empty()).ok_or("unclosed element")?;
document.children.pop().ok_or_else(|| "no root element".to_owned())
}
// ------------------------------------------------------------------- mapping
const PYTHON: &str = "PythonConfigurationType";
const DJANGO_SERVER: &str = "Python.DjangoServer";
const DJANGO_TESTS: &str = "DjangoTestsConfigurationType";
const PYTHON_TESTS: &str = "tests";
const NPM: &str = "js.build_tools.npm";
const JEST: &str = "JavaScriptTestRunnerJest";
const VITEST: &str = "JavaScriptTestRunnerVitest";
const NODE: &str = "NodeJSConfigurationType";
const SHELL: &str = "ShConfigurationType";
const COMPOUND: &str = "CompoundRunConfigurationType";
type Res<T> = Result<T, SkipReason>;
/// What a `$NAME$` that isn't a known path macro does.
#[derive(Clone, Copy, PartialEq)]
enum Unknown {
/// Skips the configuration: a path variable defined in the IDE, with no value here.
Skip,
/// Stays as written: script text, where `$` belongs to the shell.
Keep,
}
struct Cx<'a> {
root: &'a Path,
home: Option<String>,
/// Modules `.idea/modules.xml` lists. `$MODULE_DIR$` is the module's content root: the project
/// directory in a single-module project, unknown here with several.
modules: usize,
/// The configuration being converted used `$USER_HOME$`: its values belong to this machine.
machine_values: Cell<bool>,
}
/// A JSON string as the pytest configuration stores its `_new_*` fields (`"\"tests\""`).
fn json_string(raw: &str) -> String {
serde_json::from_str::<String>(raw).unwrap_or_else(|_| raw.to_owned())
}
/// A command line the run configuration forms parse back into `args` (`formatArgs` in the UI):
/// single quotes only where needed.
fn format_args(args: &[String]) -> String {
let quote = |a: &String| {
if a.is_empty() {
"''".to_owned()
} else if a.chars().any(|c| c.is_whitespace() || "'\"\\$`".contains(c)) {
format!("'{}'", a.replace('\'', "'\\''"))
} else {
a.clone()
}
};
args.iter().map(quote).collect::<Vec<_>>().join(" ")
}
/// Regex-escaped for Jest / Vitest `-t`.
fn regex_escape(s: &str) -> String {
let mut out = String::with_capacity(s.len());
for c in s.chars() {
if r"\^$.|?*+()[]{}".contains(c) {
out.push('\\');
}
out.push(c);
}
out
}
impl Cx<'_> {
/// Replaces the path macros in `s`; another `$NAME$` skips the configuration.
fn expand(&self, s: &str) -> Res<String> {
self.expand_with(s, Unknown::Skip)
}
/// Replaces the path macros in `s`, which IDEA writes in every option of a configuration
/// (script text included); `unknown` says what another `$NAME$` does.
fn expand_with(&self, s: &str, unknown: Unknown) -> Res<String> {
let mut out = String::with_capacity(s.len());
let mut rest = s;
while let Some(start) = rest.find('$') {
let after = &rest[start + 1..];
let len =
after.find('$').filter(|&n| n > 0 && after[..n].chars().all(|c| c.is_ascii_alphanumeric() || c == '_'));
let Some(len) = len else {
out.push_str(&rest[..=start]);
rest = after;
continue;
};
out.push_str(&rest[..start]);
let name = &after[..len];
let value = match name {
"PROJECT_DIR" | "ContentRoot" | "ProjectFileDir" => self.root.to_string_lossy().into_owned(),
// Which module's directory is unknown here when there are several.
"MODULE_DIR" | "MODULE_WORKING_DIR" if self.modules > 1 => {
return Err(SkipReason::SeveralModules { variable: format!("${name}$") });
}
"MODULE_DIR" | "MODULE_WORKING_DIR" => self.root.to_string_lossy().into_owned(),
"USER_HOME" => {
self.machine_values.set(true);
self.home.clone().ok_or(SkipReason::PathVariable { variable: format!("${name}$") })?
}
_ => match unknown {
Unknown::Skip => return Err(SkipReason::PathVariable { variable: format!("${name}$") }),
Unknown::Keep => format!("${name}$"),
},
};
out.push_str(&value);
rest = &after[len + 1..];
}
out.push_str(rest);
Ok(out)
}
fn text(&self, value: Option<&str>) -> Res<Option<String>> {
match value.map(str::trim) {
Some(v) if !v.is_empty() => self.expand(v).map(Some),
_ => Ok(None),
}
}
/// Words of a parameters field, split like IDEA's `ParametersListUtil.parse`.
fn words(&self, value: Option<&str>) -> Res<Vec<String>> {
self.words_quoted(value, Quotes::Double)
}
fn words_quoted(&self, value: Option<&str>, quotes: Quotes) -> Res<Vec<String>> {
Ok(self.text(value)?.map(|v| parse_params(&v, quotes)).unwrap_or_default())
}
/// Words of a command line, paths inside the project relative to `dir`.
fn args(&self, value: Option<&str>, dir: &Path) -> Res<Vec<String>> {
self.args_quoted(value, Quotes::Double, dir)
}
fn args_quoted(&self, value: Option<&str>, quotes: Quotes, dir: &Path) -> Res<Vec<String>> {
Ok(self.words_quoted(value, quotes)?.into_iter().map(|a| path_from(self.root, a, dir)).collect())
}
/// The working directory (absolute) and the configuration's `cwd`.
fn cwd(&self, value: Option<&str>) -> Res<(PathBuf, Option<PathBuf>)> {
Ok(match self.text(value)? {
Some(dir) => working_dir(self.root, &dir),
None => (self.root.to_path_buf(), None),
})
}
fn entry(&self, c: &El) -> Entry {
let name = c.attr("name").unwrap_or_default().trim().to_owned();
self.machine_values.set(false);
let result = if name.is_empty() { Err(SkipReason::Invalid) } else { self.convert(&name, c) };
let result = result.map(|mut found| {
// Values of this machine (its home) stay out of the shared run.toml.
found.config.extras.local = self.machine_values.get();
if let Some(folder) = c.attr("folderName").filter(|f| !f.trim().is_empty()) {
found.config.extras.folder = Some(folder.trim().to_owned());
}
// IDEA's "Allow multiple instances".
found.config.allow_parallel = c.attr("singleton") == Some("false");
if c.child("method").is_some_and(|m| !m.children.is_empty()) {
found.ignored.push("before launch".into());
}
found.ignored.sort();
found
});
Entry { name, result }
}
fn convert(&self, name: &str, c: &El) -> Res<Found> {
let ty = c.attr("type").unwrap_or_default();
let factory = c.attr("factoryName").unwrap_or_default();
let mut ignored = Vec::new();
let mut members = Vec::new();
let config = match ty {
PYTHON => self.python(name, c, &mut ignored)?,
DJANGO_SERVER => self.django_server(name, c, &mut ignored)?,
DJANGO_TESTS => self.django_tests(name, c, &mut ignored)?,
PYTHON_TESTS if matches!(factory, "py.test" | "pytest") => self.pytest(name, c, &mut ignored)?,
NPM => self.npm(name, c, &mut ignored)?,
JEST | VITEST => self.js_tests(name, c, ty == VITEST, &mut ignored)?,
NODE => self.node(name, c, &mut ignored)?,
SHELL => self.shell(name, c)?,
COMPOUND => {
members = c.children_named("toRun").filter_map(|t| t.attr("name")).map(str::to_owned).collect();
config(name, "compound", String::new(), Vec::new(), None)
}
PYTHON_TESTS => return Err(SkipReason::UnsupportedType { type_: format!("{ty} ({factory})") }),
other => return Err(SkipReason::UnsupportedType { type_: other.to_owned() }),
};
Ok(Found { config, members, ignored })
}
// ------------------------------------------------------------- Python
/// The interpreter (`SDK_HOME` when the configuration doesn't use the module's), its
/// options, and the working directory.
fn interpreter(&self, c: &El, ignored: &mut Vec<String>) -> Res<(String, Vec<String>, PathBuf, Option<PathBuf>)> {
let (dir, cwd) = self.cwd(c.option("WORKING_DIRECTORY"))?;
let sdk = if c.flag("IS_MODULE_SDK") { None } else { self.text(c.option("SDK_HOME"))? };
let (exe, mut args) = match sdk {
Some(sdk) => (path_from(self.root, sdk, &dir), Vec::new()),
None => project_python(self.root, &dir),
};
args.extend(self.words(c.option("INTERPRETER_OPTIONS"))?);
// A flag (`AbstractPythonRunConfiguration.java:320-321`): `false` leaves nothing out.
if c.option("RUN_TOOL") == Some("true") {
ignored.push("RUN_TOOL".into());
}
Ok((exe, args, dir, cwd))
}
/// `envs`, `PARENT_ENVS`, `ENV_FILES` and the EnvFile plugin's files.
fn python_env(&self, c: &El, rc: &mut RunConfig, ignored: &mut Vec<String>) -> Res<()> {
self.envs(c, rc);
rc.extras.include_system_env = c.option("PARENT_ENVS") != Some("false");
// PyCharm 2024+: "Paths to .env files", joined with `File.pathSeparator`.
for file in split_env_files(c.option("ENV_FILES").unwrap_or_default(), PATH_SEPARATOR) {
if let Some(file) = self.text(Some(file))? {
rc.extras.env_files.push(path_from(self.root, file, self.root));
}
}
self.envfile_plugin(c, rc, ignored)
}
/// Environment variables, as written: IDEA doesn't put path macros there
/// (`RunConfigurationPathMacroFilter`), and the Python and Shell types pass `$NAME$` through as is.
fn envs(&self, c: &El, rc: &mut RunConfig) {
for env in c.child("envs").into_iter().flat_map(|e| e.children_named("env")) {
if let Some(key) = env.attr("name").filter(|k| !k.is_empty()) {
rc.env.insert(key.to_owned(), env.attr("value").unwrap_or_default().to_owned());
}
}
}
/// The EnvFile plugin (`net.ashald.envfile`): its enabled `.env` entries.
fn envfile_plugin(&self, c: &El, rc: &mut RunConfig, ignored: &mut Vec<String>) -> Res<()> {
let Some(ext) = c.children_named("extension").find(|e| e.attr("name") == Some("net.ashald.envfile")) else {
return Ok(());
};
if !ext.flag("IS_ENABLED") {
return Ok(());
}
for entry in ext.child("ENTRIES").into_iter().flat_map(|e| e.children_named("ENTRY")) {
let path = entry.attr("PATH").unwrap_or_default();
if entry.attr("IS_ENABLED") == Some("false") || path.is_empty() {
continue;
}
if entry.attr("PARSER").is_some_and(|p| p != "env") || entry.attr("IS_EXECUTABLE") == Some("true") {
ignored.push(format!("EnvFile {path}"));
continue;
}
if let Some(file) = self.text(Some(path))? {
rc.extras.env_files.push(path_from(self.root, file, self.root));
}
}
Ok(())
}
fn python(&self, name: &str, c: &El, ignored: &mut Vec<String>) -> Res<RunConfig> {
let (exe, mut args, dir, cwd) = self.interpreter(c, ignored)?;
let Some(script) = self.text(c.option("SCRIPT_NAME"))? else {
return Err(SkipReason::NothingToRun);
};
if c.flag("MODULE_MODE") {
args.extend(["-m".to_owned(), script]);
} else {
args.push(path_from(self.root, script, &dir));
}
args.extend(self.args(c.option("PARAMETERS"), &dir)?);
let mut rc = config(name, "python", exe, args, Some("debugpy"));
rc.cwd = cwd.map(Into::into);
rc.extras.emulate_terminal = c.flag("EMULATE_TERMINAL");
if c.flag("REDIRECT_INPUT") {
ignored.push("REDIRECT_INPUT".into());
}
self.python_env(c, &mut rc, ignored)?;
Ok(rc)
}
/// `manage.py` relative to the working directory: there, at the root, or one level down.
fn manage_py(&self, dir: &Path) -> Res<String> {
let mut candidates = vec![dir.join("manage.py"), self.root.join("manage.py")];
if let Ok(read) = std::fs::read_dir(self.root) {
let mut nested: Vec<PathBuf> =
read.flatten().map(|e| e.path().join("manage.py")).filter(|p| p.is_file()).collect();
if nested.len() == 1 {
candidates.append(&mut nested);
}
}
let found =
candidates.into_iter().find(|p| p.is_file()).ok_or(SkipReason::MissingFile { file: "manage.py".into() })?;
Ok(relative_to(&normalize(&found), dir).unwrap_or_else(|| found.to_string_lossy().into_owned()))
}
fn django_server(&self, name: &str, c: &El, ignored: &mut Vec<String>) -> Res<RunConfig> {
let (exe, mut args, dir, cwd) = self.interpreter(c, ignored)?;
args.push(self.manage_py(&dir)?);
let custom = if c.flag("useCustomRunCommand") { self.text(c.option("customRunCommand"))? } else { None };
args.push(custom.unwrap_or_else(|| "runserver".into()));
let port = self.text(c.option("port"))?;
match (self.text(c.option("host"))?, port) {
(Some(host), port) => args.push(format!("{host}:{}", port.as_deref().unwrap_or("8000"))),
(None, Some(port)) => args.push(port),
(None, None) => {}
}
if c.flag("runNoReload") {
args.push("--noreload".into());
}
args.extend(self.words(c.option("additionalOptions"))?);
for key in ["browserUrl", "launchJavascriptDebuger", "runTestServer"] {
if c.option(key).is_some_and(|v| !v.is_empty() && v != "false") {
ignored.push(key.into());
}
}
let mut rc = config(name, "python", exe, args, Some("debugpy"));
rc.cwd = cwd.map(Into::into);
self.python_env(c, &mut rc, ignored)?;
Ok(rc)
}
fn django_tests(&self, name: &str, c: &El, ignored: &mut Vec<String>) -> Res<RunConfig> {
let (exe, mut args, dir, cwd) = self.interpreter(c, ignored)?;
let manage = self.manage_py(&dir)?;
// The Django tests form runs `manage.py` from the working directory; another place stays a Python run.
let kind = if manage == "manage.py" { "django-test" } else { "python" };
args.extend([manage, "test".to_owned()]);
args.extend(self.words(c.option("TARGET"))?);
if c.flag("CUSTOM_SETTINGS")
&& let Some(settings) = self.text(c.option("SETTINGS_FILE"))?
{
args.extend(["--settings".to_owned(), settings_module(&settings, self.root, &dir)]);
}
if c.flag("USE_OPTIONS") {
args.extend(self.words(c.option("OPTIONS"))?);
}
let mut rc = config(name, kind, exe, args, Some("debugpy"));
rc.cwd = cwd.map(Into::into);
// A test configuration either way: the test tree reads `manage.py test` output in a Python run too.
rc.is_test = true;
self.python_env(c, &mut rc, ignored)?;
Ok(rc)
}
fn pytest(&self, name: &str, c: &El, ignored: &mut Vec<String>) -> Res<RunConfig> {
let (exe, mut args, dir, cwd) = self.interpreter(c, ignored)?;
args.extend(["-m".to_owned(), "pytest".to_owned()]);
let field = |key: &str| c.option(key).map(json_string).filter(|v| !v.trim().is_empty());
// As `PyTestConfiguration` (intellij-community `PyTest.kt:151-153`, `:181-183`): parameters only
// follow a Python target (`name[params]`), keywords go to `-k` unless the target is custom.
let target_type = field("_new_targetType");
if let Some(target) = field("_new_target") {
let target = self.expand(&target)?;
match target_type.as_deref() {
Some("PYTHON") => {
let params = field("_new_parameters").map(|p| format!("[{p}]")).unwrap_or_default();
match self.python_target(&target, &dir) {
Some(node_id) => args.push(format!("{node_id}{params}")),
// Not a module of the project (an installed package): pytest finds it by name.
None => {
args.extend(["--pyargs".to_owned(), target]);
if !params.is_empty() {
ignored.push("_new_parameters".into());
}
}
}
}
Some("CUSTOM") => {}
_ => args.push(path_from(self.root, target, &dir)),
}
}
if target_type.as_deref() != Some("CUSTOM")
&& let Some(keywords) =
field("_new_keywords").map(|k| strip_quotes(&k).to_owned()).filter(|k| !k.is_empty())
{
args.extend(["-k".to_owned(), keywords]);
}
// Single quotes group words here too (`PyTestsShared.kt:671`).
args.extend(self.args_quoted(field("_new_additionalArguments").as_deref(), Quotes::Single, &dir)?);
let mut rc = config(name, "pytest", exe, args, Some("debugpy"));
rc.cwd = cwd.map(Into::into);
rc.is_test = true;
self.python_env(c, &mut rc, ignored)?;
Ok(rc)
}
/// A pytest Python target (`tests.test_money.TestRound.test_eur`) as the node id PyCharm
/// passes (`tests/test_money.py::TestRound::test_eur`, intellij-community `PyTestsShared.kt:366-400`):
/// the longest prefix that is a module on disk (`a/b.py` or `a/b/__init__.py`, from the working
/// directory, then the project root), then the rest as `::` symbols. Relative to `dir`.
fn python_target(&self, dotted: &str, dir: &Path) -> Option<String> {
let parts: Vec<&str> = dotted.split('.').collect();
if parts.iter().any(|p| p.is_empty()) {
return None;
}
for base in [dir, self.root] {
for n in (1..=parts.len()).rev() {
let module = parts[..n].join("/");
let (file, node) = if base.join(format!("{module}.py")).is_file() {
(base.join(format!("{module}.py")), format!("{module}.py"))
} else if base.join(&module).join("__init__.py").is_file() {
(base.join(&module), module.clone())
} else {
continue;
};
let path = if base == dir { node } else { path_from(self.root, file.to_string_lossy().into(), dir) };
return Some(
std::iter::once(path)
.chain(parts[n..].iter().map(|s| (*s).to_owned()))
.collect::<Vec<_>>()
.join("::"),
);
}
}
None
}
// ------------------------------------------------------------- JavaScript
/// The configuration's own Node interpreter; `None` for the project's (`project`).
fn node_interpreter(&self, c: &El) -> Res<Option<String>> {
match c.value("node-interpreter").or_else(|| c.attr("node-interpreter")).map(str::trim) {
None | Some("" | "project") => Ok(None),
interpreter => self.text(interpreter),
}
}
/// Kinds without an interpreter field keep the project's and list the setting as ignored.
fn node_interpreter_ignored(&self, c: &El, ignored: &mut Vec<String>) -> Res<()> {
if self.node_interpreter(c)?.is_some() {
ignored.push("node-interpreter".into());
}
Ok(())
}
fn npm(&self, name: &str, c: &El, ignored: &mut Vec<String>) -> Res<RunConfig> {
let package_json = self.text(c.value("package-json"))?;
let dir = match &package_json {
Some(p) => normalize(&self.root.join(p)).parent().map(Path::to_path_buf).unwrap_or(self.root.to_path_buf()),
None => self.root.to_path_buf(),
};
let manager = match self.text(c.value("package-manager"))? {
Some(pm) if ["npm", "pnpm", "yarn", "bun"].contains(&command_name(&pm)) => command_name(&pm).to_owned(),
_ => {
let pkg = std::fs::read_to_string(dir.join("package.json"))
.ok()
.and_then(|t| serde_json::from_str(&t).ok())
.unwrap_or_default();
crate::detect::package_manager(&dir, &pkg).to_owned()
}
};
let command = self.text(c.value("command"))?.unwrap_or_else(|| "run".into());
let scripts: Vec<&str> = c
.child("scripts")
.into_iter()
.flat_map(|s| s.children_named("script"))
.filter_map(|s| s.attr("value"))
.collect();
let mut args = vec![command.clone()];
let mut test = command == "test";
if command == "run" {
let Some(script) = scripts.first() else { return Err(SkipReason::NothingToRun) };
test = crate::detect::is_test_script(script);
args.push((*script).to_owned());
if scripts.len() > 1 {
ignored.push("scripts".into());
}
}
let arguments = self.words(c.value("arguments"))?;
args.extend(arguments.iter().cloned());
let interpreter = self.node_interpreter(c)?;
let package_dir = working_dir(self.root, &dir.to_string_lossy()).1;
let mut rc = config(name, "npm", manager.clone(), args, Some("js-debug"));
if let Some(interpreter) = interpreter {
// The npm form's fields, so the interpreter shows there (and the rest still match).
let script = if command == "run" { scripts.first().map(|s| (*s).to_owned()) } else { None };
let mut params = BTreeMap::from([
("packageManager".to_owned(), manager),
("command".to_owned(), command),
("script".to_owned(), script.unwrap_or_default()),
("arguments".to_owned(), format_args(&arguments)),
(
"packageDir".to_owned(),
package_dir.as_ref().map(|d| d.to_string_lossy().into_owned()).unwrap_or_default(),
),
("nodeInterpreter".to_owned(), interpreter),
]);
params.retain(|_, v| !v.is_empty());
rc.extras.params = params;
}
rc.cwd = package_dir.map(Into::into);
rc.is_test = test;
self.envs(c, &mut rc);
Ok(rc)
}
fn js_tests(&self, name: &str, c: &El, vitest: bool, ignored: &mut Vec<String>) -> Res<RunConfig> {
let (dir, cwd) = self.cwd(c.value("working-dir"))?;
let mut args: Vec<String> = if vitest { vec!["vitest".into(), "run".into()] } else { vec!["jest".into()] };
if !vitest && let Some(config_file) = self.text(c.value("config-file"))? {
args.extend(["--config".to_owned(), path_from(self.root, config_file, &dir)]);
}
args.extend(self.args(c.value(if vitest { "vitest-options" } else { "jest-options" }), &dir)?);
match c.value("scope-kind").unwrap_or("ALL") {
"DIRECTORY" => args.extend(self.text(c.value("test-directory"))?.map(|d| path_from(self.root, d, &dir))),
"TEST_FILE" | "SUITE" | "TEST" => {
args.extend(self.text(c.value("test-file"))?.map(|f| path_from(self.root, f, &dir)));
let names: Vec<&str> = c
.child("test-names")
.into_iter()
.flat_map(|n| n.children_named("test-name"))
.filter_map(|n| n.attr("value"))
.collect();
if !names.is_empty() {
let pattern = names.iter().map(|n| regex_escape(n)).collect::<Vec<_>>().join(" ");
let anchored = if c.value("scope-kind") == Some("TEST") {
format!("^{pattern}$")
} else {
format!("^{pattern}")
};
args.extend(["-t".to_owned(), anchored]);
}
}
_ => {}
}
self.node_interpreter_ignored(c, ignored)?;
let mut rc = config(name, "node", "npx".into(), args, Some("js-debug"));
rc.cwd = cwd.map(Into::into);
rc.is_test = true;
self.envs(c, &mut rc);
Ok(rc)
}
fn node(&self, name: &str, c: &El, ignored: &mut Vec<String>) -> Res<RunConfig> {
let (dir, cwd) = self.cwd(c.attr("working-dir"))?;
let Some(file) = self.text(c.attr("path-to-js-file"))? else {
return Err(SkipReason::NothingToRun);
};
let mut args = self.words(c.attr("node-parameters"))?;
args.push(path_from(self.root, file, &dir));
args.extend(self.args(c.attr("application-parameters"), &dir)?);
self.node_interpreter_ignored(c, ignored)?;
let mut rc = config(name, "node", "node".into(), args, Some("js-debug"));
rc.cwd = cwd.map(Into::into);
self.envs(c, &mut rc);
rc.extras.include_system_env = c.attr("pass-parent-envs") != Some("false");
Ok(rc)
}
// ------------------------------------------------------------- Shell
/// A script file (with its interpreter) or script text, kept as the Shell Script form's fields.
fn shell(&self, name: &str, c: &El) -> Res<RunConfig> {
let (dir, cwd) = self.cwd(c.option("SCRIPT_WORKING_DIRECTORY"))?;
let mut params = BTreeMap::new();
// Both default to true when missing (`ShRunConfiguration.java:148-149`).
let rc = if c.option("EXECUTE_SCRIPT_FILE") != Some("false") {
let Some(path) = self.text(c.option("SCRIPT_PATH"))? else { return Err(SkipReason::NothingToRun) };
let path = path_from(self.root, path, &dir);
let options = self.text(c.option("SCRIPT_OPTIONS"))?.unwrap_or_default();
let interpreter =
self.text(c.option("INTERPRETER_PATH"))?.map(|i| path_from(self.root, i, &dir)).unwrap_or_default();
let interpreter_options = self.text(c.option("INTERPRETER_OPTIONS"))?.unwrap_or_default();
// Run directly, the script is the program: `./run.sh`, so it isn't looked up on the PATH.
let path = if interpreter.is_empty() { dot_slash(path) } else { path };
let (command, args) = if interpreter.is_empty() {
(path.clone(), parse_params(&options, Quotes::Double))
} else {
let mut args = parse_params(&interpreter_options, Quotes::Double);
args.push(path.clone());
args.extend(parse_params(&options, Quotes::Double));
(interpreter.clone(), args)
};
params.extend([
("mode".to_owned(), "file".to_owned()),
("path".to_owned(), path),
("options".to_owned(), options),
("interpreter".to_owned(), interpreter),
("interpreterOptions".to_owned(), interpreter_options),
]);
config(name, "shell", command, args, None)
} else {
// IDEA writes `$PROJECT_DIR$` / `$USER_HOME$` into the text too; any other `$` is the shell's.
let script = self.expand_with(c.option("SCRIPT_TEXT").unwrap_or_default().trim(), Unknown::Keep)?;
if script.is_empty() {
return Err(SkipReason::NothingToRun);
}
params.extend([("mode".to_owned(), "text".to_owned()), ("script".to_owned(), script.clone())]);
config(name, "shell", script, Vec::new(), None)
};
let mut rc = rc;
params.retain(|_, v| !v.is_empty());
rc.extras.params = params;
rc.cwd = cwd.map(Into::into);
rc.extras.emulate_terminal = c.option("EXECUTE_IN_TERMINAL") != Some("false");
self.envs(c, &mut rc);
Ok(rc)
}
}
/// Which quotes group words in a parameters field.
#[derive(Clone, Copy)]
enum Quotes {
Double,
Single,
}
/// IDEA's `ParametersListUtil.parse`: words split at whitespace, grouped by quotes (dropped);
/// a backslash escapes only a quote, so Windows paths (`C:\p\out`) keep theirs.
fn parse_params(line: &str, quotes: Quotes) -> Vec<String> {
let is_quote = |c: char| c == '"' || (matches!(quotes, Quotes::Single) && c == '\'');
let mut out = Vec::new();
let mut token = String::new();
let mut quote: Option<char> = None;
let mut escaped = false;
let mut non_empty = false;
let mut chars = line.trim().chars().peekable();
while let Some(c) = chars.next() {
if quote.map_or(is_quote(c), |q| q == c) {
if !escaped {
quote = if quote.is_some() { None } else { Some(c) };
non_empty = true;
continue;
}
escaped = false;
} else if c.is_whitespace() && quote.is_none() {
if !token.is_empty() || non_empty {
out.push(std::mem::take(&mut token));
non_empty = false;
}
continue;
} else if c == '\\'
&& let Some(&next) = chars.peek()
&& quote.map_or(is_quote(next), |q| q == next)
{
escaped = true;
continue;
}
token.push(c);
}
if !token.is_empty() || non_empty {
out.push(token);
}
out
}
/// A relative path as a program: `run.sh` → `./run.sh` (`../x` and rooted paths stay; `/opt/x`
/// has no drive on Windows but isn't relative either).
fn dot_slash(path: String) -> String {
if Path::new(&path).has_root() || path.starts_with("./") || path.starts_with("../") {
path
} else {
format!("./{path}")
}
}
/// How IDEA joins a configuration's `.env` files (`File.pathSeparator`).
const PATH_SEPARATOR: char = if cfg!(windows) { ';' } else { ':' };
/// The `ENV_FILES` paths, joined with `separator` (intellij-community
/// `AbstractPythonRunConfiguration.java:303-304`, `:337`).
fn split_env_files(value: &str, separator: char) -> impl Iterator<Item = &str> {
value.split(separator).filter(|f| !f.trim().is_empty())
}
/// Keywords without the quotes around them (`"not slow"` → `not slow`).
fn strip_quotes(s: &str) -> &str {
let s = s.trim();
for q in ['"', '\''] {
if let Some(inner) = s.strip_prefix(q).and_then(|r| r.strip_suffix(q)) {
return inner;
}
}
s
}
/// Modules in `.idea/modules.xml` (0 without the file).
fn module_count(root: &Path) -> usize {
let Some(doc) = std::fs::read_to_string(root.join(".idea/modules.xml")).ok().and_then(|t| parse(&t).ok()) else {
return 0;
};
let components = std::iter::once(&doc).chain(doc.children.iter());
components
.filter(|c| c.name == "component" && c.attr("name") == Some("ProjectModuleManager"))
.flat_map(|c| c.children_named("modules"))
.map(|m| m.children_named("module").count())
.sum()
}
/// `--settings` from PyCharm's "Settings" field: a file (`mysite/settings/dev.py`, relative to
/// the project) becomes its module relative to the working directory; a module stays as it is.
fn settings_module(value: &str, root: &Path, dir: &Path) -> String {
let Some(stem) = value.strip_suffix(".py") else { return value.to_owned() };
let file = normalize(&root.join(stem));
let rel = file.strip_prefix(dir).map(Path::to_path_buf).unwrap_or_else(|_| PathBuf::from(stem));
rel.components().map(|c| c.as_os_str().to_string_lossy().into_owned()).collect::<Vec<_>>().join(".")
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parses_elements_attributes_and_escapes() {
let doc = parse(
"<?xml version=\"1.0\"?>\n<project><component name=\"RunManager\"><configuration name=\"a &amp; b\">\
<option name=\"SCRIPT_TEXT\" value=\"echo 1&#10;echo 2\" /></configuration></component></project>",
)
.unwrap();
assert_eq!(doc.name, "project");
let c = &doc.children[0].children[0];
assert_eq!(c.attr("name"), Some("a & b"));
assert_eq!(c.option("SCRIPT_TEXT"), Some("echo 1\necho 2"));
assert!(parse("<a><b></a>").is_err());
}
#[test]
fn env_files_split_on_the_path_separator() {
assert_eq!(
split_env_files("$PROJECT_DIR$/.env:$PROJECT_DIR$/.env.local", ':').collect::<Vec<_>>(),
["$PROJECT_DIR$/.env", "$PROJECT_DIR$/.env.local"]
);
assert_eq!(
split_env_files(r"C:\p\.env;C:\p\.env.local", ';').collect::<Vec<_>>(),
[r"C:\p\.env", r"C:\p\.env.local"]
);
assert_eq!(split_env_files("", ':').count(), 0);
assert_eq!(PATH_SEPARATOR, if cfg!(windows) { ';' } else { ':' });
}
#[test]
fn parameters_split_like_idea() {
let p = |s: &str| parse_params(s, Quotes::Double);
assert_eq!(p(r"--out C:\p\out \\srv\share"), [r"--out", r"C:\p\out", r"\\srv\share"]);
assert_eq!(p(r#" a "b c" "" d\"e "f\"g" "#), ["a", "b c", "", r#"d"e"#, r#"f"g"#]);
assert_eq!(p("'a b'"), ["'a", "b'"]);
assert_eq!(parse_params("-k 'a b' \"c d\"", Quotes::Single), ["-k", "a b", "c d"]);
assert!(p(" ").is_empty());
}
#[test]
fn macros_and_settings_modules() {
let cx =
Cx { root: Path::new("/p"), home: Some("/home/me".into()), modules: 1, machine_values: Cell::new(false) };
assert_eq!(cx.expand("$PROJECT_DIR$/src $5 a$b").unwrap(), "/p/src $5 a$b");
assert_eq!(cx.expand_with("cd $PROJECT_DIR$ && $PATH$", Unknown::Keep).unwrap(), "cd /p && $PATH$");
assert!(!cx.machine_values.get());
assert_eq!(dot_slash("run.sh".into()), "./run.sh");
assert_eq!(dot_slash("../run.sh".into()), "../run.sh");
assert_eq!(dot_slash("/opt/run.sh".into()), "/opt/run.sh");
assert_eq!(cx.expand("$USER_HOME$/.venvs").unwrap(), "/home/me/.venvs");
assert!(cx.machine_values.get());
assert_eq!(
cx.expand("$MAVEN_REPOSITORY$/x"),
Err(SkipReason::PathVariable { variable: "$MAVEN_REPOSITORY$".into() })
);
assert_eq!(
settings_module("src/mysite/settings/dev.py", Path::new("/p"), Path::new("/p/src")),
"mysite.settings.dev"
);
assert_eq!(settings_module("mysite.settings.dev", Path::new("/p"), Path::new("/p")), "mysite.settings.dev");
assert_eq!(json_string("\"tests\""), "tests");
let several = Cx { modules: 2, ..cx };
assert_eq!(
several.expand("$MODULE_DIR$/x"),
Err(SkipReason::SeveralModules { variable: "$MODULE_DIR$".into() })
);
assert_eq!(several.expand("$PROJECT_DIR$/x").unwrap(), "/p/x");
assert_eq!(strip_quotes("\"not slow\""), "not slow");
assert_eq!(strip_quotes("'a or b'"), "a or b");
assert_eq!(strip_quotes("plain"), "plain");
assert_eq!(regex_escape("adds 1+1 (sum)"), r"adds 1\+1 \(sum\)");
}
}
+17 -134
View File
@@ -1,9 +1,8 @@
//! Run configurations imported on request from another IDE's project files
//! (`.vscode/launch.json`, JetBrains `.run/*.run.xml` and `.idea/workspace.xml`). Each source
//! turns its entries into [`Entry`]s; [`finish`] gives them ids, skips names that already exist
//! (so importing again adds nothing twice) and resolves compound members by name.
//! (`.vscode/launch.json`). Each source turns its entries into [`Entry`]s; [`finish`]
//! gives them ids, skips names that already exist (so importing again adds nothing
//! twice) and resolves compound members by name.
mod jetbrains;
mod vscode;
use crate::RunConfig;
@@ -13,31 +12,21 @@ use specta::Type;
use std::collections::{BTreeMap, HashMap, HashSet};
use std::path::{Component, Path, PathBuf};
/// Project files run configurations can be imported from.
/// A project file run configurations can be imported from.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum RunImportSource {
/// VS Code's `.vscode/launch.json`.
Vscode,
/// JetBrains IDEs' `.run/*.run.xml`, `.idea/runConfigurations/*.xml` and `.idea/workspace.xml`.
Jetbrains,
}
impl RunImportSource {
pub const ALL: [Self; 2] = [Self::Vscode, Self::Jetbrains];
pub const ALL: [Self; 1] = [Self::Vscode];
/// The files read, relative to the project root, as the summary names them.
/// The file read, relative to the project root.
pub fn file(self) -> &'static str {
match self {
Self::Vscode => ".vscode/launch.json",
Self::Jetbrains => ".run, .idea",
}
}
fn is_present(self, root: &Path) -> bool {
match self {
Self::Vscode => root.join(self.file()).is_file(),
Self::Jetbrains => !jetbrains::files(root).is_empty(),
}
}
}
@@ -73,7 +62,7 @@ pub struct SkippedRunConfig {
pub enum SkipReason {
/// A configuration with this name already exists.
NameTaken,
/// The configuration type has no equivalent here: a VS Code debugger (`cppdbg`), a JetBrains type (`Application`).
/// The debugger type (`type`) has no equivalent here, e.g. `cppdbg`.
UnsupportedType {
#[serde(rename = "type")]
type_: String,
@@ -86,14 +75,6 @@ pub enum SkipReason {
Variable { variable: String },
/// No program, module or command.
NothingToRun,
/// A JetBrains path variable with no value here (`$MAVEN_REPOSITORY$`, user-defined ones).
PathVariable { variable: String },
/// `$MODULE_DIR$` in a project with several modules: which module's directory is unknown.
SeveralModules { variable: String },
/// A file the configuration needs isn't in the project (Django's `manage.py`).
MissingFile { file: String },
/// A project file that can't be read, or isn't XML; `name` is the file.
InvalidFile { detail: String },
/// A compound member that is neither imported nor an existing configuration.
MissingMember { member: String },
/// Not an object with a name.
@@ -123,19 +104,22 @@ pub fn import(
env: &BTreeMap<String, String>,
) -> IdeResult<RunImport> {
let file = source.file();
if !source.is_present(root) {
return Err(IdeError::not_found(format!("{file} not found")));
}
let text = match std::fs::read_to_string(root.join(file)) {
Ok(t) => t,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
return Err(IdeError::not_found(format!("{file} not found")));
}
Err(e) => return Err(e.into()),
};
let entries = match source {
RunImportSource::Vscode => vscode::entries(&std::fs::read_to_string(root.join(file))?, root, env)?,
RunImportSource::Jetbrains => jetbrains::entries(root, env)?,
RunImportSource::Vscode => vscode::entries(&text, root, env)?,
};
Ok(finish(source, entries, existing))
}
/// The sources with files under `root`.
/// The sources whose file exists under `root`.
pub fn available(root: &Path) -> Vec<RunImportSource> {
RunImportSource::ALL.into_iter().filter(|s| s.is_present(root)).collect()
RunImportSource::ALL.into_iter().filter(|s| root.join(s.file()).is_file()).collect()
}
fn unique_id(name: &str, ids: &mut HashSet<String>) -> String {
@@ -205,107 +189,6 @@ pub(crate) fn finish(source: RunImportSource, entries: Vec<Entry>, existing: &[R
}
}
// ------------------------------------------------------------- shared by the sources
/// The variable `name` of `env`; Windows names ignore case (`${env:PATH}` finds `Path`).
pub(crate) fn env_var(env: &BTreeMap<String, String>, name: &str, windows: bool) -> Option<String> {
match env.get(name) {
Some(v) => Some(v.clone()),
None if windows => env.iter().find(|(k, _)| k.eq_ignore_ascii_case(name)).map(|(_, v)| v.clone()),
None => None,
}
}
/// The program's file name without a Windows extension (`npm.cmd` → `npm`).
pub(crate) fn command_name(cmd: &str) -> &str {
let name = cmd.rsplit(['/', '\\']).next().unwrap_or(cmd);
name.strip_suffix(".cmd").or_else(|| name.strip_suffix(".exe")).unwrap_or(name)
}
pub(crate) fn config(name: &str, kind: &str, command: String, args: Vec<String>, debug: Option<&str>) -> RunConfig {
RunConfig {
id: String::new(),
name: name.to_owned(),
kind: kind.to_owned(),
command,
args,
cwd: None,
env: BTreeMap::new(),
detected: false,
is_test: false,
debug_adapter: debug.map(str::to_owned),
allow_parallel: false,
extras: crate::RunExtras::default(),
}
}
/// Splits a command line into words: whitespace, `'…'` and `"…"` quoting, backslash escapes
/// (like `parseArgs` in `src/platform/commandLine.ts`).
pub(crate) fn split_args(line: &str) -> Vec<String> {
let mut out = Vec::new();
let mut cur = String::new();
let mut has = false;
let mut quote: Option<char> = None;
let mut chars = line.chars().peekable();
while let Some(c) = chars.next() {
match quote {
Some(q) if c == q => quote = None,
Some('"') if c == '\\' && chars.peek().is_some_and(|n| "\"\\$`".contains(*n)) => {
cur.extend(chars.next());
}
Some(_) => cur.push(c),
None if c == '\'' || c == '"' => {
quote = Some(c);
has = true;
}
None if c == '\\' && chars.peek().is_some() => {
cur.extend(chars.next());
has = true;
}
None if c.is_whitespace() => {
if has || !cur.is_empty() {
out.push(std::mem::take(&mut cur));
}
has = false;
}
None => cur.push(c),
}
}
if has || !cur.is_empty() {
out.push(cur);
}
out
}
/// An absolute path inside the project, written relative to `dir` (the run's working
/// directory, where the process resolves it); anything else as it is.
pub(crate) fn path_from(root: &Path, value: String, dir: &Path) -> String {
let path = Path::new(&value);
if !path.is_absolute() {
return value;
}
let path = normalize(path);
if path.starts_with(root) && dir.starts_with(root) { relative_to(&path, dir).unwrap_or(value) } else { value }
}
/// The project's interpreter (see [`crate::detect::project_python`]), relative to `dir`.
pub(crate) fn project_python(root: &Path, dir: &Path) -> (String, Vec<String>) {
let (exe, lead) = crate::detect::project_python(root);
let exe = if exe.contains('/') { path_from(root, root.join(&exe).to_string_lossy().into(), dir) } else { exe };
(exe, lead)
}
/// The working directory `cwd` (absolute) and the configuration's `cwd` (relative to the root, `None` for the root).
pub(crate) fn working_dir(root: &Path, cwd: &str) -> (PathBuf, Option<PathBuf>) {
let abs = normalize(&root.join(cwd));
let rel = match abs.strip_prefix(root) {
Ok(r) if r.as_os_str().is_empty() => None,
Ok(r) => Some(PathBuf::from(relative_to(&abs, root).unwrap_or_else(|| r.to_string_lossy().into()))),
Err(_) => Some(abs.clone()),
};
(abs, rel)
}
/// `path` as written relative to `base` (both absolute, without `..`), with `/` separators;
/// `None` when they don't share a prefix (another drive).
pub(crate) fn relative_to(path: &Path, base: &Path) -> Option<String> {
+99 -9
View File
@@ -6,11 +6,8 @@
//! an entry using one that depends on the moment VS Code launches it (`${file}`,
//! `${input:…}`, `${command:…}`) is skipped.
use super::{
Entry, Found, SkipReason, command_name, config, env_var, normalize, path_from, project_python, split_args,
working_dir,
};
use crate::RunConfig;
use super::{Entry, Found, SkipReason, normalize, relative_to};
use crate::{RunConfig, RunExtras};
use ide_core::{IdeError, IdeResult};
use serde_json::{Map, Value};
use std::cell::Cell;
@@ -90,6 +87,15 @@ fn name_of(v: &Value) -> String {
v.get("name").and_then(Value::as_str).unwrap_or_default().trim().to_owned()
}
/// The variable `name` of `env`; Windows names ignore case (`${env:PATH}` finds `Path`).
fn env_var(env: &BTreeMap<String, String>, name: &str, windows: bool) -> Option<String> {
match env.get(name) {
Some(v) => Some(v.clone()),
None if windows => env.iter().find(|(k, _)| k.eq_ignore_ascii_case(name)).map(|(_, v)| v.clone()),
None => None,
}
}
/// String form of a scalar (`"args": [8000]`, `"env": { "DEBUG": true }`).
fn scalar(v: &Value) -> Option<String> {
match v {
@@ -100,6 +106,29 @@ fn scalar(v: &Value) -> Option<String> {
}
}
/// The program's file name without a Windows extension (`npm.cmd` → `npm`).
fn command_name(cmd: &str) -> &str {
let name = cmd.rsplit(['/', '\\']).next().unwrap_or(cmd);
name.strip_suffix(".cmd").or_else(|| name.strip_suffix(".exe")).unwrap_or(name)
}
fn config(name: &str, kind: &str, command: String, args: Vec<String>, debug: Option<&str>) -> RunConfig {
RunConfig {
id: String::new(),
name: name.to_owned(),
kind: kind.to_owned(),
command,
args,
cwd: None,
env: BTreeMap::new(),
detected: false,
is_test: false,
debug_adapter: debug.map(str::to_owned),
allow_parallel: false,
extras: RunExtras::default(),
}
}
/// `npm run dev`-style words: an npm configuration's `(command, args)`, `run-script` read as `run`.
fn npm_line(words: &[String]) -> Option<(String, Vec<String>)> {
let (exe, rest) = words.split_first()?;
@@ -113,6 +142,44 @@ fn npm_line(words: &[String]) -> Option<(String, Vec<String>)> {
NPM_COMMANDS.contains(&args.first()?.as_str()).then(|| (exe.clone(), args))
}
/// Splits a command line into words: whitespace, `'…'` and `"…"` quoting, backslash escapes
/// (like `parseArgs` in `src/platform/commandLine.ts`).
fn split_args(line: &str) -> Vec<String> {
let mut out = Vec::new();
let mut cur = String::new();
let mut has = false;
let mut quote: Option<char> = None;
let mut chars = line.chars().peekable();
while let Some(c) = chars.next() {
match quote {
Some(q) if c == q => quote = None,
Some('"') if c == '\\' && chars.peek().is_some_and(|n| "\"\\$`".contains(*n)) => {
cur.extend(chars.next());
}
Some(_) => cur.push(c),
None if c == '\'' || c == '"' => {
quote = Some(c);
has = true;
}
None if c == '\\' && chars.peek().is_some() => {
cur.extend(chars.next());
has = true;
}
None if c.is_whitespace() => {
if has || !cur.is_empty() {
out.push(std::mem::take(&mut cur));
}
has = false;
}
None => cur.push(c),
}
}
if has || !cur.is_empty() {
out.push(cur);
}
out
}
impl Cx<'_> {
/// Replaces the variables in `s`.
fn expand(&self, s: &str) -> Res<String> {
@@ -168,7 +235,16 @@ impl Cx<'_> {
/// An absolute path inside the project, written relative to `dir` (the run's working
/// directory, where the process resolves it); anything else as it is.
fn path_from(&self, value: String, dir: &Path) -> String {
path_from(self.root, value, dir)
let path = Path::new(&value);
if !path.is_absolute() {
return value;
}
let path = normalize(path);
if path.starts_with(self.root) && dir.starts_with(self.root) {
relative_to(&path, dir).unwrap_or(value)
} else {
value
}
}
/// The working directory (absolute) and the configuration's `cwd` (relative to the root).
@@ -176,7 +252,13 @@ impl Cx<'_> {
let Some(cwd) = self.string(obj, "cwd")? else {
return Ok((self.root.to_path_buf(), None));
};
Ok(working_dir(self.root, &cwd))
let abs = normalize(&self.root.join(cwd));
let rel = match abs.strip_prefix(self.root) {
Ok(r) if r.as_os_str().is_empty() => None,
Ok(r) => Some(PathBuf::from(relative_to(&abs, self.root).unwrap_or_else(|| r.to_string_lossy().into()))),
Err(_) => Some(abs.clone()),
};
Ok((abs, rel))
}
/// `env` (a variable set to itself is inherited anyway) and `envFile`.
@@ -259,14 +341,14 @@ impl Cx<'_> {
Some((exe, opts)) => {
let opts = opts.iter().map(|o| self.expand(o)).collect::<Res<Vec<_>>>()?;
if exe.trim().is_empty() || selected(exe) {
let (exe, mut lead) = project_python(self.root, &dir);
let (exe, mut lead) = self.project_python(&dir);
lead.extend(opts);
(exe, lead)
} else {
(self.path_from(self.expand(exe)?, &dir), opts)
}
}
None => project_python(self.root, &dir),
None => self.project_python(&dir),
};
args.extend(self.words(obj, "pythonArgs")?);
let rest = self.words(obj, "args")?;
@@ -293,6 +375,14 @@ impl Cx<'_> {
Ok(c)
}
/// The project's interpreter (see [`crate::detect::project_python`]), relative to `dir`.
fn project_python(&self, dir: &Path) -> (String, Vec<String>) {
let (exe, lead) = crate::detect::project_python(self.root);
let exe =
if exe.contains('/') { self.path_from(self.root.join(&exe).to_string_lossy().into(), dir) } else { exe };
(exe, lead)
}
fn python_attach(&self, name: &str, obj: &Map<String, Value>) -> Res<RunConfig> {
if obj.contains_key("processId") || obj.contains_key("listen") {
return Err(SkipReason::UnsupportedAttach);
-136
View File
@@ -203,94 +203,6 @@ pub fn explode(war: &Path, destination: &Path, resources_only: bool) -> IdeResul
remove_stale(destination, destination, &expected, resources_only)
}
/// Updates the exploded artifact deployment from the module's own class output directories and
/// its web resources (`src/main/webapp`), file by file and only what changed, instead of
/// rebuilding and re-exploding the WAR. `module` and `class_paths` are absolute.
/// `Ok(false)`: not possible this way (not deployed from the artifact yet, the module's output
/// isn't on the classpath, or a module it depends on was compiled since the last update, whose
/// archive in `WEB-INF/lib` would be stale) — rebuild the WAR instead.
pub fn sync_outputs(base: &Path, module: &Path, class_paths: &[PathBuf], resources_only: bool) -> IdeResult<bool> {
let dest = base.join("deployment");
let manifest = base.join("synced-files.json");
let since = [base.join("artifact-path.txt"), manifest.clone()]
.iter()
.filter_map(|p| fs::metadata(p).and_then(|m| m.modified()).ok())
.max();
let Some(since) = since else { return Ok(false) };
if !dest.join("WEB-INF").is_dir() {
return Ok(false);
}
let (own, others): (Vec<&PathBuf>, Vec<&PathBuf>) =
class_paths.iter().filter(|p| p.is_dir()).partition(|p| p.starts_with(module));
if own.is_empty() || others.iter().any(|d| modified_since(d, since)) {
return Ok(false);
}
let previous: Vec<String> =
fs::read(&manifest).ok().and_then(|b| serde_json::from_slice(&b).ok()).unwrap_or_default();
let mut current = std::collections::BTreeSet::new();
let classes = dest.join("WEB-INF/classes");
for dir in own {
sync_tree(dir, dir, &classes, Path::new("WEB-INF/classes"), resources_only, &mut current)?;
}
let webapp = module.join("src/main/webapp");
if webapp.is_dir() {
sync_tree(&webapp, &webapp, &dest, Path::new(""), resources_only, &mut current)?;
}
for rel in previous.iter().filter(|p| !current.contains(*p)) {
let path = dest.join(rel);
if path.is_file() {
fs::remove_file(path)?;
}
}
fs::write(manifest, serde_json::to_vec(&current).map_err(invalid)?)?;
Ok(true)
}
fn modified_since(dir: &Path, since: std::time::SystemTime) -> bool {
let Ok(entries) = fs::read_dir(dir) else { return false };
entries.flatten().any(|e| match e.file_type() {
Ok(t) if t.is_dir() => modified_since(&e.path(), since),
Ok(t) if t.is_file() => e.metadata().and_then(|m| m.modified()).is_ok_and(|m| m > since),
_ => false,
})
}
/// Copies changed files of `from` (below `root`) to `to`; unchanged files keep their timestamps,
/// so Tomcat only reloads when a class really changed. Records each source file as `prefix/rel`.
fn sync_tree(
root: &Path,
from: &Path,
to: &Path,
prefix: &Path,
resources_only: bool,
seen: &mut std::collections::BTreeSet<String>,
) -> IdeResult<()> {
for entry in fs::read_dir(from)? {
let entry = entry?;
let path = entry.path();
let kind = entry.file_type()?;
if kind.is_dir() {
sync_tree(root, &path, to, prefix, resources_only, seen)?;
continue;
}
if !kind.is_file() {
continue;
}
let rel = path.strip_prefix(root).map_err(invalid)?;
seen.insert(prefix.join(rel).to_string_lossy().replace('\\', "/"));
if resources_only && rel.extension().is_some_and(|e| e == "class") {
continue;
}
let target = to.join(rel);
let bytes = fs::read(&path)?;
if fs::read(&target).ok().as_deref() != Some(&bytes) {
fs::create_dir_all(target.parent().unwrap_or(to))?;
fs::write(target, bytes)?;
}
}
Ok(())
}
pub fn deployment(root: &Path, base: &Path, config: &RunConfig, resources_only: bool) -> IdeResult<PathBuf> {
let p = &config.extras.params;
match p.get("deployment").map(String::as_str).unwrap_or("artifact") {
@@ -527,54 +439,6 @@ mod tests {
assert!(deployment(tmp.path(), &base, &c, false).unwrap().ends_with("WebContent"));
}
#[test]
fn updates_the_exploded_deployment_from_module_outputs() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
let (base, module) = (root.join("base"), root.join("web"));
let (dest, out, dep) =
(base.join("deployment"), module.join("target/classes"), root.join("core/target/classes"));
let webapp = module.join("src/main/webapp");
for dir in [dest.join("WEB-INF/classes/a"), out.join("a"), webapp.clone(), dep.clone()] {
fs::create_dir_all(dir).unwrap();
}
for (path, text) in [
(dest.join("WEB-INF/classes/a/Same.class"), "same"),
(dest.join("WEB-INF/classes/a/App.class"), "old"),
(dest.join("index.jsp"), "old"),
(out.join("a/Same.class"), "same"),
(out.join("a/App.class"), "new"),
(out.join("app.properties"), "new"),
(webapp.join("index.jsp"), "new"),
] {
fs::write(path, text).unwrap();
}
let paths = [out.clone(), dep.clone(), root.join("lib.jar")];
// Not deployed from the artifact yet.
assert!(!sync_outputs(&base, &module, &paths, false).unwrap());
fs::write(base.join("artifact-path.txt"), "web.war").unwrap();
let same = || fs::metadata(dest.join("WEB-INF/classes/a/Same.class")).unwrap().modified().unwrap();
let before = same();
assert!(sync_outputs(&base, &module, &paths, true).unwrap());
assert_eq!(fs::read_to_string(dest.join("WEB-INF/classes/a/App.class")).unwrap(), "old");
assert_eq!(fs::read_to_string(dest.join("WEB-INF/classes/app.properties")).unwrap(), "new");
assert_eq!(fs::read_to_string(dest.join("index.jsp")).unwrap(), "new");
assert!(sync_outputs(&base, &module, &paths, false).unwrap());
assert_eq!(fs::read_to_string(dest.join("WEB-INF/classes/a/App.class")).unwrap(), "new");
assert_eq!(same(), before);
fs::remove_file(out.join("app.properties")).unwrap();
assert!(sync_outputs(&base, &module, &paths, false).unwrap());
assert!(!dest.join("WEB-INF/classes/app.properties").exists());
assert!(dest.join("WEB-INF/classes/a/Same.class").exists());
// A dependency module compiled since: its archive in WEB-INF/lib is stale.
let later = std::time::SystemTime::now() + std::time::Duration::from_secs(60);
fs::File::create(dep.join("Core.class")).unwrap().set_modified(later).unwrap();
assert!(!sync_outputs(&base, &module, &paths, false).unwrap());
}
#[test]
fn deployment_errors_identify_the_checked_path() {
let tmp = tempfile::tempdir().unwrap();
let base = tmp.path().join("cache");
Loaded 100 of 536 files, more files were not shown because too many files have changed in this diff. Show more