Compare commits

..
Author SHA1 Message Date
l0ng-ai b321690741 test(agents): give the Claude command discovery test's waits a 10 s bound
The test waited 2 s for its Python stub agent to start and to exit. Under a
parallel test run the interpreter's start-up alone can take longer, so the test
failed now and then. Both are polls for a condition, so a longer bound costs
nothing when it passes.
2026-11-12 05:03:03 +08:00
418 changed files with 784 additions and 9633 deletions

No files matched your search

-83
View File
@@ -1,83 +0,0 @@
---
name: eval
description: Run the code-intelligence evals (`pnpm eval`), pick the worst failing scenario, diagnose it from the attached timeline, fix it, prove the fix with before / after numbers measured on this machine without regressing other scenarios, and open one PR. Use for "跑 eval", "优化 Java 代码智能", "eval 失败了", an issue labelled `eval`, or any change to Java navigation / usages / gutter markers that needs numbers.
---
# Eval loop: measure → fix → prove
`pnpm eval` drives the production workbench in Chromium against a release `openidea-server`
with real language servers (Java / jdtls for now) and judges every scenario on what the user
sees: time to feedback, to the first result, to the final one, and the results themselves vs
`evals/<language>/golden/`. One machine, before and after: numbers from different machines or
from the nightly host don't judge a fix.
## 0. Preconditions
- `jdtls` on PATH and `JAVA_HOME` pointing at a JDK of the pinned `JAVA` version (projects
compile against it, and the JDK scenarios' results are in its sources; jdtls itself runs on
a newer installed JDK when it needs one). Pinned versions: `.github/tool-versions.env`. A
mismatch prints a warning and the report records both: say so in the PR when it applies.
- First run builds `cargo build --release -p ide-host --bin openidea-server` and the perf
bundle; later iterations of a frontend-only change can use `--no-build` after a
`pnpm exec vite build --mode perf --outDir target/eval/dist`, a backend-only one after the
cargo build alone.
- Run evals one at a time on this machine, with nothing heavy (builds, other evals) beside them.
## 1. Baseline on the base commit
```sh
git switch --detach origin/main # or the fix branch's merge base
pnpm eval --save-baseline # target/eval/baseline.json, this machine only
```
Read `target/eval/report.json`. Each verdict has `status`, `failures` (`kind`: `error`,
`timeout`, `degraded`, `correctness`, `budget`, `golden`), `regressions`, `repro` (the command
that reruns it) and, for failures, `diagnostics`: every command sent to the server in the
slowest run's window (`cmd`, `start`, `ms`, `ok`, `bytes`, `items`), background tasks, server
status changes and the server's log tail.
## 2. Pick one problem
Worst first: wrong or missing results and silent degradation, then timeouts, then the
feedback budget (100 ms), then the slowest `completeMs`. One problem per PR. Say which
scenario and metric, with its value vs budget.
## 3. Diagnose from the timeline
- Commands in the window ordered by `start`: what ran, what waited on what, what's serial
that could overlap, which request returned many `items`/`bytes`, which failed.
- `feedbackMs` far above 16 ms with a fast `firstResultMs`: the UI shows nothing while it
waits (a hint or popup that only appears after a delay).
- A task still running across the window: background work competing with the request.
- For the frontend side rerun with `--trace` and open `target/eval/traces/<id>@<cond>.json`
in Chrome's Performance panel.
- Name the cause at class level (as in `issue-fix`): it usually affects sibling scenarios.
## 4. Fix, then rerun that scenario until it passes
```sh
pnpm eval --only <scenario> --condition <warm|cold> # the verdict's `repro`
```
Correctness is enforced, so a faster answer that drops, truncates or serves stale results
fails. Don't trade a budget miss for a degraded answer.
## 5. Full suite, then PR
```sh
pnpm eval # vs the baseline from step 1: no new failures or regressions
```
PR per `issue-fix` §6, plus in **How it was verified** the before / after table from
`target/eval/report.md` (both on this machine, commits named) and the tool versions line.
## Guard rails
- Goldens (`evals/*/golden/`) and budgets (`evals/budgets.json`) are human-owned. Don't edit
them, and don't run `--update-golden`, to make a scenario pass. When a golden looks wrong
(a tool bump, a deliberate behaviour change), stop and say so: a PR that touches them gets
`needs-decision`.
- Don't weaken a scenario's checks or timeouts in `evals/<language>/scenarios.ts` to pass.
Adding a scenario (for a case the sweep or a user found) is fine; record its golden with
`--update-golden --only <id>` and ask a person to review the file.
- A flaky timing (passes on rerun) is not a regression: rerun once before acting on one.
-1
View File
@@ -1 +0,0 @@
../../.agents/skills/eval
-14
View File
@@ -1,14 +0,0 @@
# Language server and tool versions for Language smoke (.github/workflows/language-smoke.yml)
# and the code-intelligence evals (pnpm eval). Pinned so a release can't fail an unrelated
# PR or move a golden result; bump them deliberately (and re-record goldens: pnpm eval --update-golden).
VTSLS=0.3.0
TYPESCRIPT=6.0.3
BASEDPYRIGHT=1.40.2
RUFF=0.17.3
GOPLS=v0.24.0
JDTLS=1.63.0-202610191322
KOTLIN_LSP=263.7084.0
GRADLE=8.14.3
INTELEPHENSE=1.18.5
# The JDK projects compile against (JAVA_HOME): library navigation opens its sources.
JAVA=21
+11 -4
View File
@@ -24,7 +24,6 @@ on:
- "package.json"
- "pnpm-lock.yaml"
- ".github/workflows/language-smoke.yml"
- ".github/tool-versions.env"
pull_request:
branches: [main]
types: [opened, synchronize, reopened]
@@ -39,6 +38,16 @@ concurrency:
env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
# Pinned so a server release can't fail an unrelated PR; bump them deliberately.
VTSLS: 0.3.0
TYPESCRIPT: 6.0.3
BASEDPYRIGHT: 1.40.2
RUFF: 0.17.3
GOPLS: v0.24.0
JDTLS: 1.63.0-202610191322
KOTLIN_LSP: 263.7084.0
GRADLE: 8.14.3
INTELEPHENSE: 1.18.5
jobs:
smoke:
@@ -52,8 +61,6 @@ jobs:
timeout-minutes: 40
steps:
- uses: actions/checkout@v4
# Tool versions, shared with the evals (pnpm eval).
- run: grep -E '^[A-Z0-9_]+=' .github/tool-versions.env >> "$GITHUB_ENV"
- uses: dtolnay/rust-toolchain@stable
with: { components: "rust-analyzer, rustfmt" }
# One build cache for every language; main saves it.
@@ -89,7 +96,7 @@ jobs:
- if: matrix.language == 'java' || matrix.language == 'kotlin'
uses: actions/setup-java@v4
with: { distribution: temurin, java-version: "${{ env.JAVA }}" }
with: { distribution: temurin, java-version: "21" }
- if: matrix.language == 'java'
run: |
mkdir -p "$RUNNER_TEMP/jdtls"
-1
View File
@@ -40,7 +40,6 @@ CLAUDE.local.md
.claude/skills/*
!.claude/skills/verify
!.claude/skills/issue-fix
!.claude/skills/eval
# Local reference screenshots
pycharm-screenshots/
-1
View File
@@ -9,7 +9,6 @@
"dist",
"target",
"src/bindings.ts",
"evals/*/golden",
"src/editor/core/cm/angular/parser*.ts",
"src/editor/core/cm/jsp/parser*.ts",
"src/editor/core/cm/astro/parser*.ts",
+3 -11
View File
@@ -24,8 +24,8 @@ Visual style: dark, low-contrast greys, one cool accent
Plugin docs: `plugins/README.md`, `plugins/themes.md`.
- `java/` — JDT LS extension bundles (`java-usages`, `kotlin-bridge`) and the patched
Lombok agent, embedded into the app by `ide-sdk` / `ide-tools`.
- `e2e/` — Playwright UI tests (`pnpm e2e`). `evals/` — code-intelligence evals (`pnpm eval`).
`fixtures/` — sample projects per language for manual and automated testing.
- `e2e/` — Playwright UI tests (`pnpm e2e`). `fixtures/` — sample projects per language
for manual and automated testing.
- `docs/` — the Mintlify documentation site (`docs.json` navigation, `en/` and `zh/` pages);
preview with `npx mint dev` in `docs/`.
@@ -59,12 +59,9 @@ Unclear product decisions → `needs-decision` and stop.
## Skills
Do not use Impeccable or its related skills in this repository, including globally installed copies.
Shared agent skills live in `.agents/skills/` (`.claude/skills/` links to them):
`verify` drives the running app through Tauri MCP and captures screenshot evidence;
`issue-fix` takes an issue from reproduction to one PR;
`eval` takes a failing code-intelligence eval to a fix with before / after numbers.
`issue-fix` takes an issue from reproduction to one PR.
## Commands
@@ -84,11 +81,6 @@ Shared agent skills live in `.agents/skills/` (`.claude/skills/` links to them):
`pnpm gen:bindings`.
- `pnpm perf` (UI interaction latency / frames / leaks), `pnpm perf:rust` (criterion
benches vs budgets). Budgets: `perf/budgets.json`.
- `pnpm eval [--only <id>] [--runs n] [--save-baseline]` — Java code intelligence (navigation,
usages, gutter markers) in the production workbench against a release `openidea-server` and
jdtls: time to feedback / first result / complete, results vs `evals/<language>/golden/`.
Report: `target/eval/report.json`. Goldens and `evals/budgets.json` are changed by people only.
Tool versions: `.github/tool-versions.env` (shared with Language smoke).
- `pnpm release:prepare <version>` — the release commit's changes: app version everywhere,
`Cargo.lock`, and the plugin API snapshot when `wit/` moved to a new version.
Generated
-7
View File
@@ -4303,7 +4303,6 @@ dependencies = [
"ide-core",
"ide-http",
"ide-ssh",
"notify",
"oracledb",
"parking_lot",
"reqwest",
@@ -4460,7 +4459,6 @@ dependencies = [
"specta-typescript",
"tempfile",
"tokio",
"tokio-tungstenite",
"toml 0.9.12+spec-1.1.0",
"tracing",
"tracing-subscriber",
@@ -9869,10 +9867,7 @@ checksum = "d25a406cddcc431a75d3d9afc6a7c0f7428d4891dd973e4d54c56b46127bf857"
dependencies = [
"futures-util",
"log",
"rustls",
"rustls-pki-types",
"tokio",
"tokio-rustls",
"tungstenite",
]
@@ -10566,8 +10561,6 @@ dependencies = [
"httparse",
"log",
"rand 0.9.5",
"rustls",
"rustls-pki-types",
"sha1 0.10.7",
"thiserror 2.0.21",
"utf-8",
-1
View File
@@ -68,7 +68,6 @@ parking_lot = "0.12"
rayon = "1"
dashmap = "6"
futures = "0.3"
tokio-tungstenite = { version = "0.28", default-features = false, features = ["connect", "__rustls-tls"] }
# errors & diagnostics
thiserror = "2"
+4 -12
View File
@@ -1368,20 +1368,12 @@ impl AgentManager {
let restamped = blocking(move || {
let changes = at.changes(&before)?;
let rev = at.rev()?;
let sources = match rev.as_deref() {
Some(rev) => worktree::rename_sources(at.cwd(), rev)?,
None => Default::default(),
};
let mut restamped = Vec::new();
for change in changes.iter().filter(|c| rel_paths.as_ref().is_none_or(|p| p.contains(&c.rel_path))) {
// A rename goes back to its old path too; the new one is taken away first, so a
// case-only rename on a case-insensitive disk ends as the old file.
for change in std::iter::once(change).chain(sources.get(&change.rel_path)) {
worktree::restore(at.cwd(), rev.as_deref(), change, &before, saved.as_deref())?;
// Back to the user's own edits: not a change of the task any more.
if before.contains_key(&change.rel_path) {
restamped.push((change.rel_path.clone(), worktree::stamp(&change.path.0)));
}
worktree::restore(at.cwd(), rev.as_deref(), change, &before, saved.as_deref())?;
// Back to the user's own edits: not a change of the task any more.
if before.contains_key(&change.rel_path) {
restamped.push((change.rel_path.clone(), worktree::stamp(&change.path.0)));
}
}
Ok(restamped)
-27
View File
@@ -3055,33 +3055,6 @@ async fn commits_made_during_a_local_task_are_kept_by_its_rollback() {
}
}
#[tokio::test(flavor = "multi_thread")]
async fn rolling_back_a_rename_brings_the_old_file_back() {
for eol in EOLS {
let repo = Checkout::new(eol);
repo.write("Old.java", "class Old {}\n");
repo.write("Case.java", "class Case {}\n");
repo.commit("init");
let (mgr, task) = repo.task().await;
repo.git(&["mv", "Old.java", "New.java"]);
repo.git(&["mv", "Case.java", "CASE.java"]);
// Renames list their new path only.
let listed = changed(&mgr, &task.id).await;
assert_eq!(listed, ["CASE.java", "New.java"], "{eol:?}");
mgr.rollback_changes(&task.id, Some(listed)).await.unwrap();
assert_eq!(repo.read("Old.java"), "class Old {}\n", "{eol:?}");
assert!(!repo.dir.join("New.java").exists(), "{eol:?}");
assert_eq!(repo.read("Case.java"), "class Case {}\n", "{eol:?}");
let names: Vec<String> = std::fs::read_dir(&repo.dir)
.unwrap()
.map(|e| e.unwrap().file_name().to_string_lossy().into_owned())
.filter(|n| n.ends_with(".java"))
.collect();
assert!(!names.contains(&"CASE.java".to_owned()), "{eol:?}: {names:?}");
}
}
#[tokio::test(flavor = "multi_thread")]
async fn a_local_task_in_a_repository_without_commits_lists_and_rolls_back_its_changes() {
for eol in EOLS {
+25 -142
View File
@@ -124,41 +124,11 @@ pub(crate) fn changes(cwd: &Path, base: &str) -> IdeResult<Vec<AgentChange>> {
Ok(out)
}
/// Files renamed since `rev`, by their new repository-relative path: the old path, as a deletion
/// to roll back along with the rename.
pub(crate) fn rename_sources(cwd: &Path, rev: &str) -> IdeResult<BTreeMap<String, AgentChange>> {
let Some(repo) = discover(cwd)? else { return Ok(BTreeMap::new()) };
let root = repo.root().to_path_buf();
Ok(repo
.diff_with_worktree(rev)?
.into_iter()
.filter_map(|f| {
let old = f.old_path.filter(|old| *old != f.path)?;
let source = AgentChange {
path: IpcPath(root.join(&old)),
rel_path: old,
kind: ChangeKind::Deleted,
added: 0,
removed: 0,
};
Some((f.path, source))
})
.collect())
}
fn added(path: IpcPath, rel_path: String) -> AgentChange {
let added = std::fs::read_to_string(&*path).map_or(0, |t| t.lines().count() as u32);
AgentChange { path, rel_path, kind: ChangeKind::Added, added, removed: 0 }
}
/// Whether two paths name one file (a case-insensitive disk resolving both spellings).
fn same_file(a: &Path, b: &Path) -> bool {
match (std::fs::canonicalize(a), std::fs::canonicalize(b)) {
(Ok(a), Ok(b)) => a == b,
_ => false,
}
}
fn crlf_to_lf(bytes: &[u8]) -> Vec<u8> {
let mut out = Vec::with_capacity(bytes.len());
for (i, &b) in bytes.iter().enumerate() {
@@ -275,70 +245,34 @@ pub(crate) fn accept(
changes: &[AgentChange],
) -> IdeResult<crate::AgentAcceptOutcome> {
let repo = repo_at(cwd)?;
// Change paths are repository-relative, and the task's project may be a folder inside it.
let local_root = discover(root)?.map_or_else(|| root.to_path_buf(), |r| r.root().to_path_buf());
let renamed_from: BTreeMap<String, String> =
repo.diff_with_worktree(base)?.into_iter().filter_map(|f| Some((f.path, f.old_path?))).collect();
// Brings one path of the checkout to the task's version, unless the checkout changed it itself.
let apply = |rel: &str| -> IdeResult<bool> {
let local = local_root.join(rel);
let in_task = repo.root().join(rel);
let base_bytes = repo.file_bytes_at(&in_task, Some(base))?;
let current = std::fs::read(&in_task).ok();
let local_bytes = std::fs::read(&local).ok();
if local_bytes == current {
return Ok(true);
}
// The checkout may hold the committed LF text with CRLF line ends (`core.autocrlf`).
let unchanged = match (&local_bytes, &base_bytes) {
(Some(now), Some(at_base)) => now == at_base || crlf_to_lf(now) == *at_base,
(now, at_base) => now == at_base,
};
if !unchanged {
return Ok(false);
}
match current {
Some(bytes) => {
if let Some(parent) = local.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(&local, bytes)?;
}
None => match std::fs::remove_file(&local) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => return Err(e.into()),
_ => {}
},
}
Ok(true)
};
let mut accepted = Vec::with_capacity(changes.len());
let mut skipped = Vec::new();
let skip = |path: &str| crate::AgentAcceptSkipped {
path: path.to_owned(),
reason: "the checkout has changes of its own to it; accept it by hand".to_owned(),
};
for change in changes {
// A case-only rename on a case-insensitive disk: both names reach the same file, so
// rename it in place before bringing it to the task's version.
if let Some(old) = renamed_from.get(&change.rel_path)
&& old != &change.rel_path
&& old.to_lowercase() == change.rel_path.to_lowercase()
&& local_root.join(old).exists()
&& same_file(&local_root.join(old), &local_root.join(&change.rel_path))
{
std::fs::rename(local_root.join(old), local_root.join(&change.rel_path))?;
}
if !apply(&change.rel_path)? {
skipped.push(skip(&change.rel_path));
continue;
}
accepted.push(change.rel_path.clone());
// A rename also takes the file away from its old path (unless only its case changed).
if let Some(old) = renamed_from.get(&change.rel_path)
&& old.to_lowercase() != change.rel_path.to_lowercase()
&& !apply(old)?
{
skipped.push(skip(old));
let local = root.join(&change.rel_path);
let base_bytes = repo.file_bytes_at(&change.path.0, Some(base))?;
let current = std::fs::read(&change.path.0).ok();
let local_bytes = std::fs::read(&local).ok();
if local_bytes == current {
accepted.push(change.rel_path.clone());
} else if local_bytes == base_bytes {
match current {
Some(bytes) => {
if let Some(parent) = local.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(&local, bytes)?;
}
None => match std::fs::remove_file(&local) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => return Err(e.into()),
_ => {}
},
}
accepted.push(change.rel_path.clone());
} else {
skipped.push(crate::AgentAcceptSkipped {
path: change.rel_path.clone(),
reason: "the checkout has changes of its own to it; accept it by hand".to_owned(),
});
}
}
Ok(crate::AgentAcceptOutcome { accepted, skipped })
@@ -646,57 +580,6 @@ mod tests {
without_untouched(changes(cwd, base).unwrap(), &BTreeMap::new())
}
#[test]
fn accept_moves_a_renamed_file_and_maps_paths_from_a_project_folder() {
let (_tmp, repo, created) = accept_repo();
std::fs::create_dir(created.path.join("app")).unwrap();
git(&created.path, &["mv", "a.txt", "app/moved.txt"]);
std::fs::write(created.path.join("app/new.txt"), "n\n").unwrap();
git(&created.path, &["add", "."]);
git(&created.path, &["-c", "user.name=t", "-c", "user.email=t@t", "commit", "-q", "-m", "move"]);
// The task's project is the repository's `app` folder.
let project = repo.join("app");
std::fs::create_dir(&project).unwrap();
let changes = listed(&created.path, &created.base_commit);
let outcome = accept(&project, &created.path, &created.base_commit, &changes).unwrap();
assert!(outcome.skipped.is_empty(), "{:?}", outcome.skipped);
assert_eq!(std::fs::read_to_string(repo.join("app/moved.txt")).unwrap(), "one\ntwo\n");
assert_eq!(std::fs::read_to_string(repo.join("app/new.txt")).unwrap(), "n\n");
assert!(!repo.join("a.txt").exists(), "the rename's old path is gone");
assert!(!repo.join("app/app").exists());
}
#[test]
fn accept_treats_a_crlf_checkout_of_the_base_as_unchanged() {
let (_tmp, repo, created) = accept_repo();
// The checkout holds the committed text with CRLF line ends, as core.autocrlf leaves it.
std::fs::write(repo.join("a.txt"), "one\r\ntwo\r\n").unwrap();
std::fs::write(created.path.join("a.txt"), "one\nTWO\n").unwrap();
git(&created.path, &["mv", "gone.txt", "kept.txt"]);
std::fs::write(repo.join("gone.txt"), "bye\r\n").unwrap();
let changes = listed(&created.path, &created.base_commit);
let outcome = accept(&repo, &created.path, &created.base_commit, &changes).unwrap();
assert!(outcome.skipped.is_empty(), "{:?}", outcome.skipped);
assert_eq!(std::fs::read_to_string(repo.join("a.txt")).unwrap(), "one\nTWO\n");
assert!(repo.join("kept.txt").exists() && !repo.join("gone.txt").exists());
}
#[test]
fn accept_applies_a_case_only_rename() {
let (_tmp, repo, created) = accept_repo();
git(&created.path, &["mv", "a.txt", "A.txt"]);
let changes = listed(&created.path, &created.base_commit);
let outcome = accept(&repo, &created.path, &created.base_commit, &changes).unwrap();
assert!(outcome.skipped.is_empty(), "{:?}", outcome.skipped);
let names: Vec<_> = std::fs::read_dir(&repo)
.unwrap()
.map(|e| e.unwrap().file_name().to_string_lossy().into_owned())
.filter(|n| n.eq_ignore_ascii_case("a.txt"))
.collect();
assert_eq!(names, ["A.txt"]);
}
#[test]
fn accept_copies_new_modified_and_deleted_files_into_the_checkout() {
let (_tmp, repo, created) = accept_repo();
+1 -5
View File
@@ -1203,8 +1203,6 @@ pub struct MavenSettings {
pub threads: String,
/// Active Maven profiles (`-P`), comma-separated; empty leaves profiles to the POM/settings.
pub active_profiles: String,
/// Delegate IDE build/run actions to Maven: launches build with Maven (`javac`) instead of jdtls (ECJ).
pub delegate_build: bool,
}
impl MavenSettings {
@@ -1709,12 +1707,10 @@ mod maven_linkage_tests {
fn maven_runner_jre_defaults_and_round_trips() {
let old: MavenSettings = serde_json::from_str(r#"{"home":"/opt/maven"}"#).unwrap();
assert!(old.runner_jre.is_empty());
assert!(!old.delegate_build);
assert!(MavenSettings::default().runner_jre.is_empty());
let settings = MavenSettings { runner_jre: "/opt/jdk-21".into(), delegate_build: true, ..old };
let settings = MavenSettings { runner_jre: "/opt/jdk-21".into(), ..old };
let encoded = serde_json::to_string(&settings).unwrap();
assert!(encoded.contains(r#""runnerJre":"/opt/jdk-21""#));
assert!(encoded.contains(r#""delegateBuild":true"#));
assert_eq!(serde_json::from_str::<MavenSettings>(&encoded).unwrap(), settings);
}
+3 -12
View File
@@ -105,12 +105,9 @@ fn builtin(adapter: &str) -> Option<DebuggerConfig> {
"stdio",
"Install netcoredbg (.NET debugger) in Settings › Languages & Frameworks › Language Servers",
),
"lldb-dap" => cfg(
"lldb-dap",
&[],
"stdio",
"Install LLVM's lldb-dap (brew install llvm), or on macOS use the one found by xcrun -f lldb-dap",
),
"lldb-dap" => {
cfg("lldb-dap", &[], "stdio", "Install LLVM's lldb-dap (brew install llvm; on macOS `xcrun -f lldb-dap`)")
}
// GDB's own DAP interpreter, stdio transport. The DAP support it needs shipped in GDB 14
// (`prepare_gdb` refuses older ones with the required version in the message).
"gdb" => cfg(
@@ -127,12 +124,6 @@ fn builtin(adapter: &str) -> Option<DebuggerConfig> {
"tcp",
"Install Microsoft java-debug (Settings › Languages & Frameworks › Language Servers)",
),
"bun-dap" => cfg(
"bun-dap",
&["${port}", "127.0.0.1"],
"tcp",
"To install bun-dap: git clone https://github.com/morf-bls/bun-dap, run bun install && bun run build, then place dist/bun-dap in your PATH",
),
_ => return None,
})
}
-1
View File
@@ -17,7 +17,6 @@ blake3.workspace = true
tokio.workspace = true
tracing.workspace = true
parking_lot.workspace = true
notify.workspace = true
futures.workspace = true
sqlx.workspace = true
tiberius.workspace = true
-42
View File
@@ -12,8 +12,6 @@ pub mod secrets;
pub mod source;
pub mod split;
pub mod sql;
mod watch;
pub use watch::SourcesWatch;
pub use conn::{Held, Pool, QueryResult, ResultColumn};
pub use history::HistoryEntry;
@@ -237,9 +235,6 @@ pub struct DbManager {
/// Manual-commit data sources, each with the connection of its open
/// transaction (taken by the first statement after a commit / rollback).
sessions: Mutex<HashMap<String, Session>>,
/// The data sources as last read from disk, for [`DbManager::reconcile_sources`]
/// (`None` until [`DbManager::watch_sources`]).
on_disk: Mutex<Option<Vec<DataSource>>>,
}
type Session = Arc<tokio::sync::Mutex<Option<Held>>>;
@@ -367,7 +362,6 @@ impl DbManager {
tunnel_open: tokio::sync::Mutex::new(()),
ssh,
session_passwords: Mutex::default(),
on_disk: Mutex::default(),
sessions: Mutex::default(),
}
}
@@ -480,45 +474,9 @@ impl DbManager {
}
}
}
// Our own write is not reported as an edit on disk.
if let Some(known) = self.on_disk.lock().as_mut() {
*known = self.sources().unwrap_or(sources);
}
Ok(())
}
/// Watches the data source files (the project's and the global one) for edits made
/// outside the IDE; `on_change` runs on a background thread once they settle, and
/// should call [`DbManager::reconcile_sources`]. Stops when the result is dropped.
pub fn watch_sources(&self, on_change: Box<dyn Fn() + Send>) -> IdeResult<SourcesWatch> {
*self.on_disk.lock() = Some(self.sources().unwrap_or_default());
SourcesWatch::start(&self.root, self.global.as_ref().map(|(dir, _)| dir.as_path()), on_change)
}
/// Re-reads the data sources after their files changed on disk: connections of
/// removed or changed sources close, as on [`DbManager::save_sources`] (saved
/// passwords stay, the source may come back). Returns whether the list the tree
/// shows changed, or the files no longer parse (the tree reports the error).
pub async fn reconcile_sources(&self) -> bool {
let now = match self.sources() {
Ok(now) => now,
Err(e) => {
tracing::warn!("data sources: {}", e.message);
return true;
}
};
let before = self.on_disk.lock().replace(now.clone()).unwrap_or_default();
if before == now {
return false;
}
for o in &before {
if now.iter().find(|s| s.id == o.id) != Some(o) {
self.disconnect_source(&o.id).await;
}
}
true
}
/// Remembers `password` for the session, and in the saved passwords when
/// the data source saves passwords (`None` forgets it).
pub async fn set_password(&self, id: &str, password: Option<String>, save: bool) -> IdeResult<()> {
+1 -1
View File
@@ -8,7 +8,7 @@ use serde::{Deserialize, Serialize};
use specta::Type;
use std::path::Path;
pub(crate) const FILE: &str = "data-sources.toml";
const FILE: &str = "data-sources.toml";
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
-151
View File
@@ -1,151 +0,0 @@
//! Data source files edited outside the Database tool window (by hand, by a coding
//! agent, by `git checkout`) are picked up while the project is open, as IDEA does
//! for `dataSources.xml`. The project file tree's watcher is no help here: it never
//! reports a modification inside a gitignored `.openidea/`.
use crate::source::FILE;
use ide_core::{IdeError, IdeResult, project_dir};
use notify::{RecommendedWatcher, RecursiveMode, Watcher};
use parking_lot::Mutex;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Weak, mpsc};
use std::time::Duration;
/// How long a burst of writes (an editor's save, an agent's rewrite) settles before
/// the files are read.
const SETTLE: Duration = Duration::from_millis(200);
/// Watches `<root>/.openidea/data-sources.toml` and `<global>/data-sources.toml`;
/// stops when dropped.
pub struct SourcesWatch {
_watcher: Arc<Mutex<RecommendedWatcher>>,
}
impl SourcesWatch {
/// Calls `on_change` once a burst of changes to either file settled, on a
/// background thread.
pub fn start(root: &Path, global: Option<&Path>, on_change: Box<dyn Fn() + Send>) -> IdeResult<Self> {
let config: PathBuf = root.join(project_dir::DIR);
let (tx, rx) = mpsc::channel::<()>();
let relevant = {
let config = config.clone();
move |p: &Path| p.file_name().is_some_and(|n| n == FILE) || p == config
};
let watcher = notify::recommended_watcher(move |res: notify::Result<notify::Event>| {
if let Ok(e) = res
&& e.paths.iter().any(|p| relevant(p))
{
let _ = tx.send(());
}
})
.map_err(|e| IdeError::internal(format!("data sources watcher: {e}")))?;
let watcher = Arc::new(Mutex::new(watcher));
{
let mut w = watcher.lock();
// The root (not recursively) sees `.openidea/` itself appear or go.
w.watch(root, RecursiveMode::NonRecursive)
.map_err(|e| IdeError::internal(format!("data sources watcher: {e}")))?;
if let Some(dir) = global.filter(|d| d.is_dir())
&& let Err(e) = w.watch(dir, RecursiveMode::NonRecursive)
{
tracing::warn!("global data sources watcher: {e}");
}
}
let mut watching = watch_config(&watcher, &config, false);
let weak: Weak<Mutex<RecommendedWatcher>> = Arc::downgrade(&watcher);
std::thread::Builder::new().name("openidea-db-sources-watch".into()).spawn(move || {
while rx.recv().is_ok() {
// Wait until the writes stop.
loop {
match rx.recv_timeout(SETTLE) {
Ok(()) => {}
Err(mpsc::RecvTimeoutError::Timeout) => break,
Err(mpsc::RecvTimeoutError::Disconnected) => return,
}
}
let Some(watcher) = weak.upgrade() else { return };
watching = watch_config(&watcher, &config, watching);
drop(watcher);
on_change();
}
})?;
Ok(Self { _watcher: watcher })
}
}
/// Watches `.openidea/` once it exists (again: a removed folder loses its watch);
/// returns whether it is watched.
fn watch_config(watcher: &Mutex<RecommendedWatcher>, config: &Path, watching: bool) -> bool {
if !config.is_dir() {
return false;
}
if watching {
return true;
}
match watcher.lock().watch(config, RecursiveMode::NonRecursive) {
Ok(()) => true,
Err(e) => {
tracing::warn!("data sources watcher {}: {e}", config.display());
false
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::Instant;
fn wait_for(count: &AtomicUsize, at_least: usize) -> bool {
let deadline = Instant::now() + Duration::from_secs(10);
while Instant::now() < deadline {
if count.load(Ordering::SeqCst) >= at_least {
return true;
}
std::thread::sleep(Duration::from_millis(20));
}
false
}
fn counting() -> (Arc<AtomicUsize>, Box<dyn Fn() + Send>) {
let count = Arc::new(AtomicUsize::new(0));
let c = count.clone();
(count, Box::new(move || _ = c.fetch_add(1, Ordering::SeqCst)))
}
#[test]
fn reports_project_and_global_files_and_a_config_folder_created_later() {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
let (project, global) = (root.join("p"), root.join("app"));
std::fs::create_dir_all(&project).unwrap();
std::fs::create_dir_all(&global).unwrap();
let (count, on_change) = counting();
let _watch = SourcesWatch::start(&project, Some(&global), on_change).unwrap();
// No `.openidea/` yet: it is created, then the file written into it.
std::fs::create_dir_all(project.join(".openidea")).unwrap();
assert!(wait_for(&count, 1));
let seen = count.load(Ordering::SeqCst);
std::fs::write(project.join(".openidea/data-sources.toml"), "").unwrap();
assert!(wait_for(&count, seen + 1), "project file");
let seen = count.load(Ordering::SeqCst);
std::fs::write(global.join("data-sources.toml"), "").unwrap();
assert!(wait_for(&count, seen + 1), "global file");
}
#[test]
fn ignores_other_files() {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
std::fs::create_dir_all(root.join(".openidea")).unwrap();
let (count, on_change) = counting();
let _watch = SourcesWatch::start(&root, None, on_change).unwrap();
std::fs::write(root.join(".openidea/settings.toml"), "").unwrap();
std::fs::write(root.join("main.py"), "").unwrap();
std::thread::sleep(SETTLE * 3);
assert_eq!(count.load(Ordering::SeqCst), 0);
}
}
-28
View File
@@ -342,31 +342,3 @@ async fn applies_driver_properties_and_the_startup_script_on_connect() -> IdeRes
assert_eq!(db.schemas("s1").await?, vec!["main"]);
Ok(())
}
#[tokio::test]
async fn data_sources_edited_on_disk_close_their_connections() -> IdeResult<()> {
let (tmp, db) = setup().await;
std::fs::File::create(tmp.path().join("other.db")).unwrap();
let (tx, rx) = std::sync::mpsc::channel();
let _watch = db.watch_sources(Box::new(move || _ = tx.send(())))?;
db.pool("s1").await?;
assert_eq!(db.connected(), vec!["s1"]);
// Our own save is no edit on disk.
let mut sources = db.sources()?;
sources[0].name = "renamed".into();
db.save_sources(sources.clone()).await?;
assert!(!db.reconcile_sources().await);
// Edited outside the IDE (by hand, by an agent): the watcher reports it, and the
// connection made with the old settings closes.
db.pool("s1").await?;
sources[0].file = Some("other.db".into());
ide_db::source::save(tmp.path(), &sources)?;
rx.recv_timeout(std::time::Duration::from_secs(10)).expect("the watcher reports the edit");
assert!(db.reconcile_sources().await);
assert!(db.connected().is_empty());
assert_eq!(db.sources()?[0].file.as_ref().map(|f| f.0.clone()), Some("other.db".into()));
assert!(!db.reconcile_sources().await);
Ok(())
}
+2 -29
View File
@@ -34,7 +34,7 @@ use std::path::{Component, Path, PathBuf};
pub const MANIFEST_FILE: &str = "plugin.toml";
/// The plugin API (engine) version this host implements.
pub const ENGINE_VERSION: &str = "1.6.0";
pub const ENGINE_VERSION: &str = "1.5.0";
/// The 0.1 API, still accepted for contribution-only packages and 0.1 components.
pub const LEGACY_API: &str = "0.1";
/// Marketplace categories, in the order they are shown.
@@ -151,8 +151,6 @@ impl ActivationEvent {
/// Workbench event kinds (`on-event:<kind>`, WIT `events.event-kind`).
/// `scheduled` comes from `on-schedule` instead.
pub const EVENT_KINDS: &[&str] = &[
"document-changed",
"websocket-ready",
"run-exited",
"terminal-exited",
"content-tab-action",
@@ -368,10 +366,6 @@ impl Manifest {
}
let mut seen = BTreeSet::new();
for event in &self.activation {
if matches!(event, ActivationEvent::Event(kind) if matches!(kind.as_str(), "document-changed" | "websocket-ready"))
{
self.needs_api(&format!("`{event}`"), 6)?;
}
if matches!(event, ActivationEvent::Event(kind) if matches!(kind.as_str(), "file-closed" | "settings-changed" | "input-mode-changed" | "run-exited" | "terminal-exited" | "content-tab-action"))
{
self.needs_api(&format!("`{event}`"), 4)?;
@@ -588,8 +582,6 @@ mod tests {
ActivationEvent::parse("on-event:files-changed"),
Ok(ActivationEvent::Event("files-changed".into()))
);
assert!(ActivationEvent::parse("on-event:document-changed").is_ok());
assert!(ActivationEvent::parse("on-event:websocket-ready").is_ok());
let err = ActivationEvent::parse("on-event:keystroke").unwrap_err();
assert!(err.contains("unknown event kind") && err.contains("action-performed"), "{err}");
assert!(ActivationEvent::parse("on-event:scheduled").is_err(), "declared with on-schedule");
@@ -868,7 +860,7 @@ mod commands_api_tests {
mod net_ask_api_tests {
use super::*;
#[test]
fn api_1_5_declarations_require_api_1_5() {
fn net_ask_requires_api_1_5() {
for declaration in ["[capabilities]\nrequired = [\"net:ask\"]", "[capabilities]\noptional = [\"net:ask\"]"] {
for engine in ["^1.4", "^1.5"] {
let manifest = format!(
@@ -883,23 +875,4 @@ mod net_ask_api_tests {
}
}
}
#[test]
fn realtime_events_require_api_1_6() {
for declaration in
["activation = [\"on-event:document-changed\"]", "activation = [\"on-event:websocket-ready\"]"]
{
for engine in ["^1.5", "^1.6"] {
let manifest = format!(
"id = 'test'\nname = 'Test'\nversion = '0.1.0'\nwasm = 'plugin.wasm'\nengines.openidea = '{engine}'\n{declaration}\n"
);
let parsed = Manifest::parse(&manifest);
if engine == "^1.6" {
parsed.unwrap();
} else {
assert!(parsed.unwrap_err().contains("new in plugin API 1.6"), "{engine}");
}
}
}
}
}
-1
View File
@@ -14,7 +14,6 @@ encoding_rs.workspace = true
ide-core.workspace = true
ide-mcp.workspace = true
futures.workspace = true
tokio-tungstenite.workspace = true
ide-text.workspace = true
ide-vfs.workspace = true
regex-syntax.workspace = true
+8 -9
View File
@@ -176,13 +176,14 @@ fn apply_custom_migration_in(path: &Path, local: &Path) {
/// Entries of a local-data tree that move to a newly chosen directory.
/// Non-relocatable tool installs (virtual environments with absolute interpreter
/// paths) stay behind and are reinstalled on demand, as in `migrate_entries`;
/// so do registry agents, whose saved commands are absolute paths into `agents/tools`.
/// paths) stay behind and are reinstalled on demand, as in `migrate_entries`.
fn custom_entries(root: &Path) -> Vec<PathBuf> {
let mut entries = local_entries(root);
let logs = PathBuf::from("logs");
if !entries.contains(&logs) {
entries.push(logs);
for extra in ["logs", "agents/tools"] {
let entry = PathBuf::from(extra);
if !entries.contains(&entry) {
entries.push(entry);
}
}
entries
}
@@ -883,13 +884,11 @@ mod tests {
}
#[test]
fn custom_entries_cover_logs_and_keep_agent_installs() {
fn custom_entries_cover_logs_and_agent_tools() {
let tmp = tempfile::tempdir().unwrap();
let entries = custom_entries(tmp.path());
for expected in ["logs", "index", "cache", "lsp", "agents/registry.json"] {
for expected in ["logs", "agents/tools", "index", "cache", "lsp", "agents/registry.json"] {
assert!(entries.contains(&PathBuf::from(expected)), "{entries:?}");
}
// Registry agent files name their executables by absolute path.
assert!(!entries.iter().any(|e| e.starts_with("agents/tools")), "{entries:?}");
}
}
+4 -83
View File
@@ -84,14 +84,6 @@ pub struct NotebookService {
registry: Arc<Mutex<Registry>>,
kernels: Arc<KernelManager>,
sink: SharedSink,
observer: parking_lot::RwLock<Option<Arc<dyn NotebookObserver>>>,
}
/// Learns each open notebook's contents after every change of them, and when it closes (the
/// language servers' copy of its code cells).
pub trait NotebookObserver: Send + Sync {
fn changed(&self, path: &Path, snapshot: &NotebookSnapshot);
fn closed(&self, path: &Path);
}
impl NotebookService {
@@ -129,23 +121,7 @@ impl NotebookService {
}
}),
);
Arc::new(Self { registry, kernels: Arc::new(kernels), sink, observer: Default::default() })
}
pub fn set_observer(&self, observer: Arc<dyn NotebookObserver>) {
*self.observer.write() = Some(observer);
}
/// Called with the document's state locked, so observers see its changes in order.
fn changed(&self, path: &Path, snapshot: &NotebookSnapshot) {
if let Some(observer) = self.observer.read().clone() {
observer.changed(path, snapshot);
}
}
/// The file of an open notebook.
pub fn path_of(&self, id: &str) -> IdeResult<PathBuf> {
Ok(self.resolve(id)?.path.clone())
Arc::new(Self { registry, kernels: Arc::new(kernels), sink })
}
pub fn open(&self, path: &Path) -> IdeResult<NotebookSnapshot> {
@@ -160,7 +136,6 @@ impl NotebookService {
}
let document = NotebookDocument::load(&path)?;
let initial = document.snapshot();
self.changed(&path, &initial);
let id = initial.id.clone();
registry.paths.insert(path.clone(), id.clone());
registry.documents.insert(
@@ -198,9 +173,7 @@ impl NotebookService {
let document = self.resolve(id)?;
let mut state = document.state.lock();
mutable(&state)?;
let snapshot = state.document.edit(version, edit)?;
self.changed(&document.path, &snapshot);
Ok(snapshot)
state.document.edit(version, edit)
}
pub fn save(&self, id: &str) -> IdeResult<NotebookSnapshot> {
@@ -217,9 +190,7 @@ impl NotebookService {
if state.active.is_some() {
return Err(IdeError::unavailable("stop notebook execution before undo or redo"));
}
let snapshot = if redo { state.document.redo() } else { state.document.undo() }?;
self.changed(&document.path, &snapshot);
Ok(snapshot)
if redo { state.document.redo() } else { state.document.undo() }
}
pub async fn interrupt(&self, id: &str) -> IdeResult<()> {
@@ -267,10 +238,7 @@ impl NotebookService {
snapshot(&document)
} else {
self.kernels.shutdown_kernel(&document.path).await?;
let mut state = document.state.lock();
let snapshot = state.document.reload()?;
self.changed(&document.path, &snapshot);
Ok(snapshot)
document.state.lock().document.reload()
}
}
.await;
@@ -316,10 +284,6 @@ impl NotebookService {
let mut registry = self.registry.lock();
registry.documents.remove(id);
registry.paths.remove(&document.path);
drop(registry);
if let Some(observer) = self.observer.read().clone() {
observer.closed(&document.path);
}
Ok(())
}
@@ -608,49 +572,6 @@ mod tests {
KernelOutput { kind: kind.into(), content, msg_id: "actual-parent-request".into() }
}
#[tokio::test]
async fn the_observer_follows_every_change_of_an_open_notebook_and_its_close() {
#[derive(Default)]
struct Seen(Mutex<Vec<String>>);
impl NotebookObserver for Seen {
fn changed(&self, path: &Path, snapshot: &NotebookSnapshot) {
let sources: Vec<&str> = snapshot.cells.iter().map(|c| c.source.as_str()).collect();
let name = path.file_name().unwrap().to_string_lossy();
self.0.lock().push(format!("{name}: {}", sources.join(" | ")));
}
fn closed(&self, path: &Path) {
self.0.lock().push(format!("closed {}", path.file_name().unwrap().to_string_lossy()));
}
}
let (dir, service, initial) = fixture();
let seen = Arc::new(Seen::default());
service.set_observer(seen.clone());
service.close(&initial.id).await.unwrap();
let opened = service.open(&dir.path().join("session.ipynb")).unwrap();
assert_eq!(
service.path_of(&opened.id).unwrap(),
dunce::canonicalize(dir.path().join("session.ipynb")).unwrap()
);
let edit = NotebookEdit::SetSource { cell: "second".into(), source: "print(x + 1)".into() };
let edited = service.edit(&opened.id, opened.version, edit).unwrap();
service.undo(&opened.id, false).unwrap();
// A second view of the open notebook changes nothing.
service.open(&dir.path().join("session.ipynb")).unwrap();
assert!(edited.version > opened.version);
service.close(&opened.id).await.unwrap();
assert_eq!(
*seen.0.lock(),
[
"closed session.ipynb",
"session.ipynb: x = 1 | print(x) | notes",
"session.ipynb: x = 1 | print(x + 1) | notes",
"session.ipynb: x = 1 | print(x) | notes",
"closed session.ipynb",
]
);
service.shutdown().await;
}
#[tokio::test]
async fn canonical_split_opens_share_incarnation_but_reopen_and_projects_do_not() {
let (dir, service, initial) = fixture();
+10 -46
View File
@@ -48,7 +48,7 @@ async fn resolve_launch(
let build = policy != BuildPolicy::NoBuild;
let full = policy == BuildPolicy::Rebuild;
let build_kind = if full { compile::BuildKind::Rebuild } else { compile::BuildKind::Build };
let (model, external, delegated, prefer_maven) = {
let (model, external, delegated) = {
let project = project.clone();
let settings = settings.clone();
let module = module.clone();
@@ -58,17 +58,15 @@ async fn resolve_launch(
let non_java = build && compile::launch_has_non_java_sources(&project.root, &model, &module, test);
let is_maven =
model.maven.as_ref().is_some_and(|m| m.projects.iter().any(|p| project.root.join(&p.dir) == module));
let prefer_maven = settings.languages.java.maven.delegate_build;
let delegated = build
&& delegates_to_external_build(
is_maven,
policy,
non_java,
cache::has_record(&project.root, &project.build_cache_dir(), &module),
prefer_maven,
);
let external = non_java || delegated;
(model, external, delegated, prefer_maven)
(model, external, delegated)
})
.await
.map_err(|e| IdeError::internal(e.to_string()))?
@@ -81,7 +79,6 @@ async fn resolve_launch(
tokio::task::spawn_blocking(move || {
let _external_build = external_build;
let built = if delegated {
let policy = delegated_policy(policy, prefer_maven, info.error_count);
let outputs = project.java_launch_snapshots.lock().build(
&project.root,
&model,
@@ -172,17 +169,6 @@ async fn resolve_launch(
.map_err(|e| IdeError::internal(e.to_string()))?
}
/// With Settings › Maven › Delegate IDE build/run actions to Maven, javac's verdict replaces
/// JDT's. Classes JDT wrote for sources with errors look up to date to Maven's incremental
/// compiler, so a launch JDT reports errors for recompiles everything.
fn delegated_policy(policy: BuildPolicy, prefer_maven: bool, jdt_errors: u32) -> BuildPolicy {
if prefer_maven && jdt_errors > 0 && matches!(policy, BuildPolicy::Automatic | BuildPolicy::Make) {
BuildPolicy::Rebuild
} else {
policy
}
}
fn apply_external_build_result(info: &mut JavaLaunchInfo, failure: Option<String>, clean: bool) {
info.build = Some(if let Some(failure) = failure {
info.hints.push(failure);
@@ -255,15 +241,8 @@ fn apply_library_classpath(
/// Whether a launch should be built by the project's build system instead of jdtls.
/// A Rebuild has to clean the output; jdtls' full build only recompiles sources, so
/// Maven modules always delegate for it, even when every source is Java.
/// `prefer_maven` is Settings › Maven › Delegate IDE build/run actions to Maven.
fn delegates_to_external_build(
is_maven: bool,
policy: BuildPolicy,
non_java: bool,
has_record: bool,
prefer_maven: bool,
) -> bool {
is_maven && (prefer_maven || policy == BuildPolicy::Rebuild || non_java || has_record)
fn delegates_to_external_build(is_maven: bool, policy: BuildPolicy, non_java: bool, has_record: bool) -> bool {
is_maven && (policy == BuildPolicy::Rebuild || non_java || has_record)
}
fn same_entry(a: &str, b: &str, windows: bool) -> bool {
@@ -342,29 +321,14 @@ pub(crate) fn register(d: &mut Dispatcher) {
mod tests {
use super::*;
#[test]
fn delegated_launches_with_jdt_errors_recompile_with_maven() {
assert_eq!(delegated_policy(BuildPolicy::Make, true, 1), BuildPolicy::Rebuild);
assert_eq!(delegated_policy(BuildPolicy::Automatic, true, 2), BuildPolicy::Rebuild);
assert_eq!(delegated_policy(BuildPolicy::Make, true, 0), BuildPolicy::Make);
assert_eq!(delegated_policy(BuildPolicy::Make, false, 1), BuildPolicy::Make);
assert_eq!(
delegated_policy(BuildPolicy::MakeAndContinueOnErrors, true, 1),
BuildPolicy::MakeAndContinueOnErrors
);
}
#[test]
fn rebuild_delegates_maven_builds_to_the_build_system() {
assert!(delegates_to_external_build(true, BuildPolicy::Rebuild, false, false, false));
assert!(delegates_to_external_build(true, BuildPolicy::Automatic, true, false, false));
assert!(delegates_to_external_build(true, BuildPolicy::Make, false, true, false));
assert!(!delegates_to_external_build(true, BuildPolicy::Automatic, false, false, false));
assert!(!delegates_to_external_build(true, BuildPolicy::Make, false, false, false));
assert!(!delegates_to_external_build(false, BuildPolicy::Rebuild, false, false, false));
assert!(delegates_to_external_build(true, BuildPolicy::Automatic, false, false, true));
assert!(delegates_to_external_build(true, BuildPolicy::Make, false, false, true));
assert!(!delegates_to_external_build(false, BuildPolicy::Automatic, false, false, true));
assert!(delegates_to_external_build(true, BuildPolicy::Rebuild, false, false));
assert!(delegates_to_external_build(true, BuildPolicy::Automatic, true, false));
assert!(delegates_to_external_build(true, BuildPolicy::Make, false, true));
assert!(!delegates_to_external_build(true, BuildPolicy::Automatic, false, false));
assert!(!delegates_to_external_build(true, BuildPolicy::Make, false, false));
assert!(!delegates_to_external_build(false, BuildPolicy::Rebuild, false, false));
}
#[test]
+4 -6
View File
@@ -314,12 +314,10 @@ pub async fn lsp_goto(host: &Host, window: &str, doc: DocId, pos: Pos, kind: Got
&& let Some(plugins) = crate::plugins::editor_plugin_host()
{
let project = host.project(window)?;
// A notebook cell is a language-server document only: no plugin navigates in it.
if let Ok((path, version)) = project.documents.identity(doc)
&& let Some(found) = plugins.navigation_at(&project.root, &path, version, pos, Some(window), |p| {
project.documents.by_path(p).and_then(|id| project.documents.version(id).ok())
})
{
let (path, version) = project.documents.identity(doc)?;
if let Some(found) = plugins.navigation_at(&project.root, &path, version, pos, Some(window), |p| {
project.documents.by_path(p).and_then(|id| project.documents.version(id).ok())
}) {
return Ok(found.targets);
}
}
-8
View File
@@ -19,13 +19,6 @@ pub fn get(host: &Host, window: &str, id: String) -> IdeResult<NotebookSnapshot>
host.project(window)?.notebooks.get(&id)
}
/// The language-server documents of the notebook's code cells (empty when no server syncs notebooks).
pub fn cell_docs(host: &Host, window: &str, id: String) -> IdeResult<Vec<ide_lsp::NotebookCellDoc>> {
let project = host.project(window)?;
let path = project.notebooks.path_of(&id)?;
Ok(project.lsp.notebook_cells(&path))
}
pub async fn edit(
host: &Host,
window: &str,
@@ -139,7 +132,6 @@ impl Drop for RequestGuard<'_> {
pub(crate) fn register(d: &mut Dispatcher) {
d.add("notebook_open", |c| async move { open(&c.host, &c.window, c.arg("path")?).await });
d.add("notebook_get", |c| async move { get(&c.host, &c.window, c.arg("id")?) });
d.add("notebook_cell_docs", |c| async move { cell_docs(&c.host, &c.window, c.arg("id")?) });
d.add("notebook_edit", |c| async move {
edit(&c.host, &c.window, c.arg("id")?, c.arg("version")?, c.arg("edit")?).await
});
+20 -86
View File
@@ -111,9 +111,8 @@ pub trait PluginWorkspace: Send + Sync {
/// Version and text of the open document at `path`.
fn document(&self, path: &Path) -> Option<(u64, String)>;
/// Applies `edits` to the open document at `path` if it is still at
/// `version`, as one change: one undo step in the editor, or none for a
/// `background` caller (a plugin event rather than a command).
fn apply_edits(&self, path: &Path, version: u64, edits: &[TextEdit], background: bool) -> Result<u64, HostError>;
/// `version`, as one change (one undo step in the editor).
fn apply_edits(&self, path: &Path, version: u64, edits: &[TextEdit]) -> Result<u64, HostError>;
/// Code from the project may run (git hooks and config on fetch).
fn trusted(&self, root: &Path) -> bool;
/// Root of the project open in `window`.
@@ -181,7 +180,7 @@ impl SecretStore for FileSecretStore {
/// Largest response body a plugin receives.
const MAX_RESPONSE: usize = 8 << 20;
/// Largest stdout / stderr kept from a process.
const MAX_OUTPUT: usize = 16 << 20;
const MAX_OUTPUT: usize = 4 << 20;
/// Most processes one plugin keeps running at once.
const MAX_PROCESSES: usize = 8;
/// Most output one stream of a handle holds until the plugin reads it.
@@ -489,7 +488,6 @@ type SettingSnapshots = BTreeMap<(Option<String>, String), BTreeMap<String, Stri
/// State shared by the host and every plugin's [`PluginApi`].
pub(super) struct Shared {
pub sink: Arc<dyn PluginEventSink>,
pub websockets: super::websocket::WebSocketHub,
pub workspace: RwLock<Option<Arc<dyn PluginWorkspace>>>,
/// `<plugins>/.data`: one directory per plugin id.
pub data_dir: PathBuf,
@@ -567,8 +565,7 @@ impl Shared {
ui,
registry,
locale: RwLock::new("en".into()),
sink: sink.clone(),
websockets: super::websocket::WebSocketHub::new(sink.clone()),
sink,
workspace: RwLock::new(None),
secrets: RwLock::new(Arc::new(FileSecretStore { dir: data_dir.clone() })),
pins,
@@ -778,7 +775,6 @@ impl Shared {
self.kill_processes(plugin);
self.connections.clear(plugin);
self.websockets.disconnect_plugin(plugin);
self.setting_snapshots.lock().retain(|(_, p), _| p != plugin);
let channels: Vec<_> = {
let mut output = self.output.lock();
@@ -2052,14 +2048,13 @@ impl HostApi for PluginApi {
fn apply_edits(
&self,
client: Option<&str>,
_client: Option<&str>,
path: &Path,
version: u64,
edits: Vec<TextEdit>,
) -> Result<u64, HostError> {
let workspace = self.shared.workspace().ok_or_else(|| HostError::NotFound("no project is open".into()))?;
// Edits from a window's command are undoable; `editor.apply-edits-at` from a background event isn't.
workspace.apply_edits(path, version, &edits, client.is_none())
workspace.apply_edits(path, version, &edits)
}
fn storage_get(&self, key: &str) -> Result<Option<String>, HostError> {
@@ -2280,49 +2275,6 @@ impl HostApi for PluginApi {
})
}
fn websocket_connect(
&self,
root: &Path,
request: ide_plugin::WebSocketRequest,
timeout: Duration,
) -> Result<u64, HostError> {
let secrets = self.shared.secrets.read().clone();
let plugin = self.id().to_owned();
let headers = request
.headers
.into_iter()
.map(|(name, value)| {
let value = fill_secrets(&value, |name| secrets.get(&plugin, name))?;
Ok((name, value))
})
.collect::<Result<Vec<_>, HostError>>()?;
net_runtime()
.block_on(self.shared.websockets.connect(
&plugin,
&to_ipc(root),
&request.url,
&headers,
&request.subprotocol,
timeout,
))
.map_err(|error| match error {
super::websocket::ConnectError::Invalid(message) => HostError::Invalid(message),
super::websocket::ConnectError::Failed(message) => HostError::Failed(message),
})
}
fn websocket_send(&self, connection: u64, bytes: Vec<u8>) -> Result<(), HostError> {
self.shared.websockets.send(self.id(), connection, bytes).map_err(HostError::Failed)
}
fn websocket_drain(&self, connection: u64, limit: usize) -> Result<ide_plugin::WebSocketBatch, HostError> {
self.shared.websockets.drain(self.id(), connection, limit).map_err(HostError::Invalid)
}
fn websocket_close(&self, connection: u64) {
let _ = self.shared.websockets.disconnect(self.id(), connection);
}
/// The `net:ask` answer: a host the user allowed with "always" is
/// remembered in the ledger (no dialog again, background included); a new
/// host is asked about in `client`'s window when there is one. Dismissal
@@ -2491,14 +2443,9 @@ impl HostApi for PluginApi {
let read = |pipe: Option<Box<dyn Read + Send>>| {
std::thread::spawn(move || {
let mut out = Vec::new();
let mut overflow = false;
if let Some(p) = pipe {
let mut p = p;
let _ = p.by_ref().take((MAX_OUTPUT + 1) as u64).read_to_end(&mut out);
if out.len() > MAX_OUTPUT {
overflow = true;
out.truncate(MAX_OUTPUT);
}
let _ = p.by_ref().take(MAX_OUTPUT as u64).read_to_end(&mut out);
let _ = std::io::copy(&mut p, &mut std::io::sink());
}
let mut text = String::from_utf8_lossy(&out).into_owned();
@@ -2507,7 +2454,7 @@ impl HostApi for PluginApi {
end -= 1;
}
text.truncate(end);
(text, overflow)
text
})
};
let stdout = read(child.stdout.take().map(|p| Box::new(p) as Box<dyn Read + Send>));
@@ -2527,18 +2474,11 @@ impl HostApi for PluginApi {
}
std::thread::sleep(Duration::from_millis(10));
};
let (stdout, stdout_overflow) = stdout.join().unwrap_or_default();
let (stderr, _) = stderr.join().unwrap_or_default();
if stdout_overflow {
tracing::warn!(
plugin = %self.id(),
program = %cmd.program,
limit_mib = MAX_OUTPUT >> 20,
"process stdout exceeded limit"
);
return Err(HostError::Failed(format!("{} stdout exceeded {} MiB limit", cmd.program, MAX_OUTPUT >> 20)));
}
Ok(ProcessOutput { status: status.code(), stdout, stderr })
Ok(ProcessOutput {
status: status.code(),
stdout: stdout.join().unwrap_or_default(),
stderr: stderr.join().unwrap_or_default(),
})
}
fn subscribe(&self, kind: EventKind) {
@@ -3268,16 +3208,11 @@ impl PluginWorkspace for HostWorkspace {
Some((u64::from(snap.version), snap.text))
}
fn apply_edits(&self, path: &Path, version: u64, edits: &[TextEdit], background: bool) -> Result<u64, HostError> {
fn apply_edits(&self, path: &Path, version: u64, edits: &[TextEdit]) -> Result<u64, HostError> {
let (project, doc) =
self.open_document(path).ok_or_else(|| HostError::NotFound(format!("{} is not open", path.display())))?;
let version = u32::try_from(version).map_err(|_| HostError::Stale)?;
let applied = if background {
project.documents.apply_background_edits_at(doc, version, edits)
} else {
project.documents.apply_edits_at(doc, version, edits)
};
match applied {
match project.documents.apply_edits_at(doc, version, edits) {
Ok(Some(v)) => Ok(u64::from(v)),
Ok(None) => Err(HostError::Stale),
Err(e) if e.kind == ide_core::error::ErrorKind::InvalidInput => Err(HostError::Invalid(e.message)),
@@ -3607,7 +3542,7 @@ mod tests {
fn document(&self, _: &Path) -> Option<(u64, String)> {
None
}
fn apply_edits(&self, _: &Path, _: u64, _: &[TextEdit], _: bool) -> Result<u64, HostError> {
fn apply_edits(&self, _: &Path, _: u64, _: &[TextEdit]) -> Result<u64, HostError> {
Err(HostError::Stale)
}
fn trusted(&self, _: &Path) -> bool {
@@ -3671,7 +3606,7 @@ mod tests {
fn document(&self, _: &Path) -> Option<(u64, String)> {
None
}
fn apply_edits(&self, _: &Path, _: u64, _: &[TextEdit], _: bool) -> Result<u64, HostError> {
fn apply_edits(&self, _: &Path, _: u64, _: &[TextEdit]) -> Result<u64, HostError> {
Err(HostError::Stale)
}
fn trusted(&self, _: &Path) -> bool {
@@ -3872,11 +3807,10 @@ mod tests {
let tmp = tempfile::tempdir().unwrap();
let api = shell_api(tmp.path());
let token = CancellationToken::new();
// stdout within the limit comes back whole; stderr beyond it is truncated (stdout beyond it fails).
let cmd = shell(&format!("head -c 5242880 /dev/zero; head -c {} /dev/zero >&2", MAX_OUTPUT + (1 << 20)));
let cmd = shell("head -c 5242880 /dev/zero; head -c 5242880 /dev/zero >&2");
let out = api.run_process(cmd.clone(), None, None, &token, Duration::from_secs(5)).unwrap();
assert_eq!(out.status, Some(0));
assert_eq!((out.stdout.len(), out.stderr.len()), (5 << 20, MAX_OUTPUT));
assert_eq!((out.stdout.len(), out.stderr.len()), (MAX_OUTPUT, MAX_OUTPUT));
let id = api.start_process(cmd, None, None).unwrap();
let out = api.wait_process(id, MAX_PENDING, Duration::from_secs(5), &token).unwrap();
assert_eq!(out.status, Some(0));
@@ -4132,7 +4066,7 @@ mod workbench_tests {
fn document(&self, _: &Path) -> Option<(u64, String)> {
self.text.clone().map(|s| (1, s))
}
fn apply_edits(&self, _: &Path, _: u64, _: &[TextEdit], _: bool) -> Result<u64, HostError> {
fn apply_edits(&self, _: &Path, _: u64, _: &[TextEdit]) -> Result<u64, HostError> {
Err(HostError::Stale)
}
fn trusted(&self, _: &Path) -> bool {
+1 -1
View File
@@ -262,7 +262,7 @@ mod tests {
fn project_of(&self, _: &str) -> Option<PathBuf> {
None
}
fn apply_edits(&self, _: &Path, _: u64, _: &[ide_core::TextEdit], _: bool) -> Result<u64, HostError> {
fn apply_edits(&self, _: &Path, _: u64, _: &[ide_core::TextEdit]) -> Result<u64, HostError> {
unreachable!()
}
fn publish_diagnostics(&self, _: &str, _: &Path, _: Vec<ide_core::Diagnostic>) -> Result<(), HostError> {
+6 -29
View File
@@ -1,7 +1,6 @@
//! Project-relative event filters, kept with runtime subscriptions across idle reclaim.
use super::{LoadedPlugin, PluginHost};
use globset::{GlobBuilder, GlobMatcher};
use ide_extension::Capability;
use ide_plugin::{EventKind, HostError, WorkbenchEvent};
use std::{
collections::{BTreeSet, HashMap},
@@ -22,9 +21,9 @@ pub(super) struct PathFilter {
impl PathFilter {
pub fn new(kind: &str, globs: Vec<String>) -> Result<Self, HostError> {
if !matches!(kind, "file-opened" | "file-saved" | "files-changed" | "document-changed") {
if !matches!(kind, "file-opened" | "file-saved" | "files-changed") {
return Err(HostError::Invalid(
"subscribe-paths supports file-opened, file-saved, files-changed and document-changed".into(),
"subscribe-paths supports file-opened, file-saved and files-changed".into(),
));
}
if globs.len() > 64 {
@@ -96,15 +95,13 @@ impl PluginHost {
events: &[WorkbenchEvent],
subs: &Subscriptions,
) -> Vec<WorkbenchEvent> {
let filters = subs.paths.get(plugin.id());
let Some(filters) = subs.paths.get(plugin.id()) else { return events.to_vec() };
let workspace = shared.workspace();
events
.iter()
.filter_map(|event| {
if let WorkbenchEvent::FilesChanged(batch) = event {
let Some(filter) = filters.and_then(|filters| filters.get(event.kind().name())) else {
return Some(event.clone());
};
let Some(filter) = filters.get(event.kind().name()) else { return Some(event.clone()) };
let root = workspace.as_ref()?.project_of(client?)?;
if plugin.dev_root.as_deref().is_some_and(|r| r != root) {
return None;
@@ -129,9 +126,9 @@ impl PluginHost {
}
let path = match event {
WorkbenchEvent::FileOpened(path) | WorkbenchEvent::FileSaved(path) => Path::new(path),
WorkbenchEvent::DocumentChanged(change) => Path::new(&change.path),
_ => return Some(event.clone()),
};
let Some(filter) = filters.get(event.kind().name()) else { return Some(event.clone()) };
if !path.is_absolute() {
return None;
}
@@ -150,26 +147,7 @@ impl PluginHost {
if plugin.dev_root.as_deref().is_some_and(|r| r != root) {
return None;
}
if let WorkbenchEvent::DocumentChanged(change) = event {
// Typed text: only for plugins that may read editors, and only for paths they named.
let filter = filters.and_then(|filters| filters.get(event.kind().name()))?;
let granted = &plugin.meta.lock().granted;
if !granted.contains(&Capability::ProjectRead)
|| !granted.contains(&Capability::EditorRead)
|| !filter.matches(&root, path)
{
return None;
}
let real = ide_plugin::resolve_real(path);
let relative = real.strip_prefix(ide_plugin::resolve_real(&root)).ok()?;
let mut change = change.clone();
change.path = relative.to_string_lossy().replace('\\', "/");
return Some(WorkbenchEvent::DocumentChanged(change));
}
filters
.and_then(|filters| filters.get(event.kind().name()))
.is_none_or(|filter| filter.matches(&root, path))
.then(|| event.clone())
filter.matches(&root, path).then(|| event.clone())
})
.collect()
}
@@ -189,7 +167,6 @@ mod tests {
assert!(PathFilter::new("file-saved", vec!["./".repeat(513)]).is_err());
assert!(PathFilter::new("file-saved", vec![]).is_ok());
assert!(PathFilter::new("files-changed", vec![]).is_ok());
assert!(PathFilter::new("document-changed", vec!["src/**/*.rs".into()]).is_ok());
}
#[test]
+1 -7
View File
@@ -507,13 +507,7 @@ mod tests {
fn document(&self, _: &Path) -> Option<(u64, String)> {
None
}
fn apply_edits(
&self,
_: &Path,
_: u64,
_: &[ide_core::TextEdit],
_: bool,
) -> Result<u64, ide_plugin::HostError> {
fn apply_edits(&self, _: &Path, _: u64, _: &[ide_core::TextEdit]) -> Result<u64, ide_plugin::HostError> {
unreachable!()
}
fn trusted(&self, _: &Path) -> bool {
-3
View File
@@ -41,7 +41,6 @@ pub mod language_features;
mod market;
mod mcp;
pub mod navigation;
mod network_inbox;
mod points;
mod providers;
pub mod remote;
@@ -55,7 +54,6 @@ mod types;
pub mod ui_points;
pub mod views;
pub mod web;
mod websocket;
pub use api::{FileSecretStore, HostWorkspace, PluginWorkspace, SecretStore};
pub(crate) use hover::merge as merge_hover;
@@ -1923,7 +1921,6 @@ impl PluginHost {
for p in plugins {
let wanted: Vec<WorkbenchEvent> = events
.iter()
.filter(|event| !matches!(event, WorkbenchEvent::WebsocketReady(ready) if ready.plugin_id != p.id()))
.filter(|e| Self::wants(&p, e.kind(), subs.kinds.get(p.id())))
.filter(|e| {
!matches!(e, WorkbenchEvent::FilesChanged(_))
@@ -1,203 +0,0 @@
//! Bounded inbox for the proposed background network API.
//!
//! Experimental transport-independent primitive. This module does not expose
//! sockets to plugins. Overflow closes the inbox: silently dropping a document
//! update would make a collaboration session appear healthy while diverging.
use std::collections::VecDeque;
use std::sync::Mutex;
#[derive(Debug, PartialEq, Eq)]
pub(super) enum Push {
/// The caller must schedule one plugin wake-up after releasing this lock.
Wake,
Queued,
/// This message was refused and the caller must schedule a wake-up.
OverflowWake,
/// This message was refused; a wake-up is already pending.
OverflowQueued,
Closed,
}
#[derive(Debug, PartialEq, Eq)]
pub(super) struct Batch {
pub messages: Vec<Vec<u8>>,
pub closed: bool,
pub more: bool,
}
#[derive(Default)]
struct State {
messages: VecDeque<Box<[u8]>>,
bytes: usize,
closed: bool,
notified: bool,
}
pub(super) struct Inbox {
state: Mutex<State>,
max_messages: usize,
max_bytes: usize,
max_frame: usize,
}
impl Inbox {
pub fn new(max_messages: usize, max_bytes: usize, max_frame: usize) -> Self {
assert!(max_messages > 0 && max_bytes > 0 && max_frame > 0);
Self { state: Mutex::new(State::default()), max_messages, max_bytes, max_frame }
}
pub fn push(&self, message: Vec<u8>) -> Push {
let mut state = self.state.lock().unwrap();
if state.closed {
return Push::Closed;
}
let overflow = message.len() > self.max_frame
|| state.messages.len() >= self.max_messages
|| message.len() > self.max_bytes.saturating_sub(state.bytes);
if overflow {
state.closed = true;
} else {
state.bytes += message.len();
state.messages.push_back(message.into_boxed_slice());
}
let wake = !state.notified;
state.notified = true;
match (overflow, wake) {
(false, true) => Push::Wake,
(false, false) => Push::Queued,
(true, true) => Push::OverflowWake,
(true, false) => Push::OverflowQueued,
}
}
/// Returns whether the caller must schedule a wake-up. Idempotent.
pub fn close(&self) -> bool {
let mut state = self.state.lock().unwrap();
if state.closed {
return false;
}
state.closed = true;
let wake = !state.notified;
state.notified = true;
wake
}
/// A partial drain retains the notification; the consumer must continue
/// draining when `more` is true. An empty drain rearms the next wake-up.
pub fn drain(&self, limit: usize) -> Batch {
assert!(limit > 0);
let mut state = self.state.lock().unwrap();
let count = limit.min(state.messages.len());
let messages: Vec<Vec<u8>> = state.messages.drain(..count).map(|message| message.into_vec()).collect();
state.bytes -= messages.iter().map(Vec::len).sum::<usize>();
let more = !state.messages.is_empty();
state.notified = more;
Batch { messages, closed: state.closed, more }
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn coalesces_and_rearms_wakeups() {
let inbox = Inbox::new(4, 16, 8);
assert_eq!(inbox.push(vec![1]), Push::Wake);
assert_eq!(inbox.push(vec![2]), Push::Queued);
assert!(inbox.drain(1).more);
assert_eq!(inbox.push(vec![3]), Push::Queued);
assert_eq!(inbox.drain(4).messages, vec![vec![2], vec![3]]);
assert_eq!(inbox.push(vec![4]), Push::Wake);
}
#[test]
fn overflow_preserves_accepted_messages_and_reports_close() {
for (messages, bytes, frame) in [(1, 8, 8), (4, 1, 8), (4, 8, 1)] {
let inbox = Inbox::new(messages, bytes, frame);
assert_eq!(inbox.push(vec![1]), Push::Wake);
assert_eq!(inbox.push(vec![2, 3]), Push::OverflowQueued);
let batch = inbox.drain(4);
assert_eq!(batch.messages, vec![vec![1]]);
assert!(batch.closed);
assert_eq!(inbox.push(vec![]), Push::Closed);
}
}
#[test]
fn accepted_frame_does_not_retain_excess_capacity() {
let inbox = Inbox::new(2, 8, 8);
let mut frame = Vec::with_capacity(1024 * 1024);
frame.push(42);
assert_eq!(inbox.push(frame), Push::Wake);
let state = inbox.state.lock().unwrap();
assert_eq!(state.bytes, 1);
assert_eq!(state.messages[0].len(), 1);
assert_eq!(std::mem::size_of_val(&*state.messages[0]), 1);
}
#[test]
fn close_during_partial_drain_remains_visible_without_duplicate_wakeup() {
let inbox = Inbox::new(3, 12, 4);
assert_eq!(inbox.push(vec![1]), Push::Wake);
assert_eq!(inbox.push(vec![2]), Push::Queued);
let first = inbox.drain(1);
assert_eq!(first.messages, vec![vec![1]]);
assert!(first.more);
assert!(!inbox.close());
let second = inbox.drain(1);
assert_eq!(second.messages, vec![vec![2]]);
assert!(second.closed);
assert!(!second.more);
assert_eq!(inbox.push(vec![3]), Push::Closed);
}
#[test]
fn draining_releases_byte_budget_for_later_messages() {
let inbox = Inbox::new(2, 4, 4);
assert_eq!(inbox.push(vec![1, 2, 3, 4]), Push::Wake);
assert_eq!(inbox.drain(1).messages, vec![vec![1, 2, 3, 4]]);
assert_eq!(inbox.push(vec![5, 6, 7, 8]), Push::Wake);
assert_eq!(inbox.drain(1).messages, vec![vec![5, 6, 7, 8]]);
}
#[test]
fn oversized_first_message_requests_wakeup_and_reports_refusal() {
let inbox = Inbox::new(2, 8, 1);
assert_eq!(inbox.push(vec![1, 2]), Push::OverflowWake);
let batch = inbox.drain(2);
assert!(batch.closed);
assert!(batch.messages.is_empty());
}
#[test]
fn close_wakes_once_even_without_messages() {
let inbox = Inbox::new(4, 16, 8);
assert!(inbox.close());
assert!(!inbox.close());
assert!(inbox.drain(4).closed);
assert_eq!(inbox.push(vec![1]), Push::Closed);
}
#[test]
fn concurrent_producers_remain_bounded_and_fifo_per_producer() {
let inbox = std::sync::Arc::new(Inbox::new(400, 800, 2));
std::thread::scope(|scope| {
for producer in 0..4u8 {
let inbox = inbox.clone();
scope.spawn(move || {
for index in 0..100u8 {
assert_ne!(inbox.push(vec![producer, index]), Push::Closed);
}
});
}
});
let batch = inbox.drain(400);
assert_eq!(batch.messages.len(), 400);
assert!(!batch.closed);
for producer in 0..4u8 {
let sequence: Vec<_> = batch.messages.iter().filter(|m| m[0] == producer).map(|m| m[1]).collect();
assert_eq!(sequence, (0..100u8).collect::<Vec<_>>());
}
}
}
@@ -155,7 +155,6 @@ pub fn forwarded(event: &PluginEvent) -> bool {
| PluginEvent::TaskProgress { .. }
| PluginEvent::TaskEnded { .. }
| PluginEvent::SubscriptionsChanged
| PluginEvent::WebsocketReady { .. }
| PluginEvent::ViewChanged { .. }
| PluginEvent::ViewMessage { .. }
| PluginEvent::OpenTerminal { .. }
-3
View File
@@ -275,9 +275,6 @@ pub enum PluginEvent {
/// The kinds of workbench events plugins want changed: re-fetch
/// `plugin_event_kinds` (the UI reports only those).
SubscriptionsChanged,
/// A background WebSocket has messages ready for its owning plugin.
#[serde(rename_all = "camelCase")]
WebsocketReady { plugin_id: String, connection: u64 },
#[serde(rename_all = "camelCase")]
ThemeChanged { theme: Option<ActiveTheme> },
/// A plugin needs capabilities the user has not granted yet (first use
-531
View File
@@ -1,531 +0,0 @@
//! Host-owned background WebSocket connections for plugins.
use super::{
PluginEvent, PluginEventSink,
network_inbox::{Inbox, Push},
};
use futures::{SinkExt, StreamExt};
use ide_plugin::WebSocketBatch;
use parking_lot::Mutex;
use std::{
collections::HashMap,
sync::{
Arc, Weak,
atomic::{AtomicU64, Ordering},
},
time::Duration,
};
use tokio::sync::{Notify, watch};
use tokio_tungstenite::{
Connector, connect_async_tls_with_config,
tungstenite::{
Message,
client::IntoClientRequest,
http::{HeaderName, HeaderValue},
protocol::WebSocketConfig,
},
};
const MAX_MESSAGES: usize = 256;
const MAX_BYTES: usize = 8 << 20;
const MAX_FRAME: usize = 1 << 20;
const DRAIN_BATCH: usize = 32;
const CLOSED_HANDLE_TTL: Duration = Duration::from_secs(60);
const MAX_CONNECTIONS_PER_PLUGIN: usize = 8;
pub type ConnectionId = u64;
#[derive(Debug, PartialEq, Eq)]
pub enum ConnectError {
Invalid(String),
Failed(String),
}
struct Connection {
plugin: String,
outgoing: Arc<Inbox>,
incoming: Arc<Inbox>,
wake: Notify,
stop: watch::Sender<bool>,
}
struct Inner {
next: AtomicU64,
connections: Mutex<HashMap<ConnectionId, Arc<Connection>>>,
pending: Mutex<HashMap<String, usize>>,
sink: Option<Arc<dyn PluginEventSink>>,
}
#[derive(Clone)]
pub struct WebSocketHub(Arc<Inner>);
impl Default for WebSocketHub {
fn default() -> Self {
Self(Arc::new(Inner {
next: AtomicU64::new(1),
connections: Mutex::new(HashMap::new()),
pending: Mutex::new(HashMap::new()),
sink: None,
}))
}
}
impl WebSocketHub {
pub fn new(sink: Arc<dyn PluginEventSink>) -> Self {
Self(Arc::new(Inner {
next: AtomicU64::new(1),
connections: Mutex::new(HashMap::new()),
pending: Mutex::new(HashMap::new()),
sink: Some(sink),
}))
}
pub async fn connect(
&self,
plugin: &str,
_project: &str,
url: &str,
headers: &[(String, String)],
subprotocol: &str,
timeout: Duration,
) -> Result<ConnectionId, ConnectError> {
validate_url(url).map_err(ConnectError::Invalid)?;
if subprotocol.len() > 256 {
return Err(ConnectError::Invalid("WebSocket subprotocol is too long".into()));
}
if timeout.is_zero() {
return Err(ConnectError::Failed("WebSocket connection timed out".into()));
}
self.reserve(plugin)?;
let result = async {
let mut request =
url.into_client_request().map_err(|_| ConnectError::Invalid("invalid WebSocket URL".into()))?;
for (name, value) in headers {
if protected_header(name) {
return Err(ConnectError::Invalid(format!(
"WebSocket header {name:?} is managed by the handshake"
)));
}
let name: HeaderName =
name.parse().map_err(|_| ConnectError::Invalid("invalid WebSocket header name".into()))?;
let value = HeaderValue::from_str(value)
.map_err(|_| ConnectError::Invalid("invalid WebSocket header value".into()))?;
request.headers_mut().append(name, value);
}
if !subprotocol.is_empty() {
request.headers_mut().insert(
"sec-websocket-protocol",
subprotocol.parse().map_err(|_| ConnectError::Invalid("invalid WebSocket subprotocol".into()))?,
);
}
let config = WebSocketConfig::default()
.read_buffer_size(16 << 10)
.write_buffer_size(16 << 10)
.max_write_buffer_size(2 << 20)
.max_message_size(Some(MAX_FRAME))
.max_frame_size(Some(MAX_FRAME));
let tls = ide_http::client::tls_config()
.map(Arc::new)
.map(Connector::Rustls)
.map_err(|error| ConnectError::Failed(error.message))?;
let (socket, response) =
tokio::time::timeout(timeout, connect_async_tls_with_config(request, Some(config), false, Some(tls)))
.await
.map_err(|_| ConnectError::Failed("WebSocket connection timed out".into()))?
.map_err(|_| ConnectError::Failed("WebSocket connection failed".into()))?;
if !subprotocol.is_empty()
&& response.headers().get("sec-websocket-protocol").and_then(|value| value.to_str().ok())
!= Some(subprotocol)
{
return Err(ConnectError::Failed("WebSocket server did not accept the required subprotocol".into()));
}
let id = self.0.next.fetch_add(1, Ordering::Relaxed);
let (stop, stopped) = watch::channel(false);
let connection = Arc::new(Connection {
plugin: plugin.to_owned(),
outgoing: Arc::new(Inbox::new(MAX_MESSAGES, MAX_BYTES, MAX_FRAME)),
incoming: Arc::new(Inbox::new(MAX_MESSAGES, MAX_BYTES, MAX_FRAME)),
wake: Notify::new(),
stop,
});
self.0.connections.lock().insert(id, connection.clone());
tokio::spawn(run_socket(id, socket, connection, Arc::downgrade(&self.0), stopped));
Ok(id)
}
.await;
self.release(plugin);
result
}
pub fn send(&self, plugin: &str, connection: ConnectionId, bytes: Vec<u8>) -> Result<(), String> {
let connection = self.connection(plugin, connection)?;
match connection.outgoing.push(bytes) {
Push::Wake => {
connection.wake.notify_one();
Ok(())
}
Push::Queued => Ok(()),
Push::OverflowWake => {
connection.wake.notify_one();
Err("WebSocket send queue is full; message was rejected".into())
}
Push::OverflowQueued => Err("WebSocket send queue is full; message was rejected".into()),
Push::Closed => Err("WebSocket connection is closed; message was rejected".into()),
}
}
pub fn drain(&self, plugin: &str, connection: ConnectionId, limit: usize) -> Result<WebSocketBatch, String> {
if limit == 0 || limit > DRAIN_BATCH {
return Err("invalid drain limit".into());
}
let connection_state = self.connection(plugin, connection)?;
let batch = connection_state.incoming.drain(limit);
if batch.more {
self.emit_ready(connection, plugin);
}
if batch.closed && !batch.more {
self.0.connections.lock().remove(&connection);
}
Ok(WebSocketBatch { messages: batch.messages, closed: batch.closed })
}
pub fn disconnect(&self, plugin: &str, id: ConnectionId) -> Result<(), String> {
let connection = self
.0
.connections
.lock()
.get(&id)
.filter(|connection| connection.plugin == plugin)
.cloned()
.ok_or_else(|| "unknown WebSocket connection".to_owned())?;
connection.outgoing.close();
let _ = connection.stop.send(true);
connection.wake.notify_one();
self.0.connections.lock().remove(&id);
Ok(())
}
pub fn disconnect_plugin(&self, plugin: &str) {
let removed: Vec<_> = {
let mut connections = self.0.connections.lock();
let ids: Vec<_> =
connections.iter().filter(|(_, connection)| connection.plugin == plugin).map(|(id, _)| *id).collect();
ids.into_iter().filter_map(|id| connections.remove(&id)).collect()
};
for connection in removed {
connection.outgoing.close();
let _ = connection.stop.send(true);
connection.wake.notify_one();
}
}
fn connection(&self, plugin: &str, id: ConnectionId) -> Result<Arc<Connection>, String> {
self.0
.connections
.lock()
.get(&id)
.filter(|connection| connection.plugin == plugin)
.cloned()
.ok_or_else(|| "unknown WebSocket connection".into())
}
fn reserve(&self, plugin: &str) -> Result<(), ConnectError> {
let open = self.0.connections.lock().values().filter(|connection| connection.plugin == plugin).count();
let mut pending = self.0.pending.lock();
if open + pending.get(plugin).copied().unwrap_or_default() >= MAX_CONNECTIONS_PER_PLUGIN {
return Err(ConnectError::Invalid("at most 8 WebSocket connections per plugin".into()));
}
*pending.entry(plugin.to_owned()).or_default() += 1;
Ok(())
}
fn release(&self, plugin: &str) {
let mut pending = self.0.pending.lock();
if let Some(count) = pending.get_mut(plugin) {
*count -= 1;
if *count == 0 {
pending.remove(plugin);
}
}
}
fn emit_ready(&self, connection: ConnectionId, plugin: &str) {
if let Some(sink) = &self.0.sink {
sink.emit(None, PluginEvent::WebsocketReady { plugin_id: plugin.to_owned(), connection });
}
}
}
fn validate_url(url: &str) -> Result<(), String> {
let parsed = reqwest::Url::parse(url).map_err(|_| "invalid WebSocket URL")?;
// `host_str` keeps an IPv6 address in brackets.
let loopback = parsed.host_str().is_some_and(|host| {
let host = host.trim_start_matches('[').trim_end_matches(']');
host.eq_ignore_ascii_case("localhost") || host.parse::<std::net::IpAddr>().is_ok_and(|ip| ip.is_loopback())
});
if parsed.scheme() != "wss" && !(parsed.scheme() == "ws" && loopback) {
return Err("WebSocket requires WSS outside loopback".into());
}
if !parsed.username().is_empty() || parsed.password().is_some() || parsed.fragment().is_some() {
return Err("WebSocket URLs may not carry credentials or a fragment".into());
}
Ok(())
}
fn protected_header(name: &str) -> bool {
name.eq_ignore_ascii_case("host")
|| name.eq_ignore_ascii_case("connection")
|| name.eq_ignore_ascii_case("upgrade")
|| name.get(.."sec-websocket-".len()).is_some_and(|prefix| prefix.eq_ignore_ascii_case("sec-websocket-"))
}
async fn run_socket<S>(
id: ConnectionId,
socket: tokio_tungstenite::WebSocketStream<S>,
connection: Arc<Connection>,
inner: Weak<Inner>,
mut stopped: watch::Receiver<bool>,
) where
S: tokio::io::AsyncRead + tokio::io::AsyncWrite + Unpin + Send + 'static,
{
let (mut writer, mut reader) = socket.split();
loop {
tokio::select! {
_ = connection.wake.notified() => {
let batch = connection.outgoing.drain(DRAIN_BATCH);
let mut failed = false;
for bytes in batch.messages {
if writer.send(Message::Binary(bytes.into())).await.is_err() {
failed = true;
break;
}
}
if failed || batch.closed { break; }
if batch.more { connection.wake.notify_one(); }
}
changed = stopped.changed() => {
if changed.is_err() || *stopped.borrow() {
let mut failed = false;
loop {
let batch = connection.outgoing.drain(DRAIN_BATCH);
for bytes in batch.messages {
if writer.send(Message::Binary(bytes.into())).await.is_err() {
failed = true;
break;
}
}
if failed || !batch.more { break; }
}
break;
}
}
message = reader.next() => match message {
Some(Ok(Message::Binary(bytes))) => {
let pushed = connection.incoming.push(bytes.to_vec());
if matches!(pushed, Push::Wake | Push::OverflowWake)
&& let Some(inner) = inner.upgrade()
&& let Some(sink) = &inner.sink
{
sink.emit(None, PluginEvent::WebsocketReady { plugin_id: connection.plugin.clone(), connection: id });
}
if matches!(pushed, Push::OverflowWake | Push::OverflowQueued | Push::Closed) { break; }
}
Some(Ok(Message::Ping(bytes))) => if writer.send(Message::Pong(bytes)).await.is_err() { break; },
Some(Ok(Message::Close(_))) | Some(Err(_)) | None => break,
_ => {}
}
}
}
let _ = writer.close().await;
connection.outgoing.close();
let wake = connection.incoming.close();
if let Some(inner) = inner.upgrade() {
if wake && let Some(sink) = &inner.sink {
sink.emit(None, PluginEvent::WebsocketReady { plugin_id: connection.plugin.clone(), connection: id });
}
let retained = connection.clone();
tokio::spawn(async move {
tokio::time::sleep(CLOSED_HANDLE_TTL).await;
let mut connections = inner.connections.lock();
if connections.get(&id).is_some_and(|current| Arc::ptr_eq(current, &retained)) {
connections.remove(&id);
}
});
}
}
#[cfg(test)]
mod tests {
use super::*;
use tokio::net::TcpListener;
use tokio_tungstenite::{
accept_hdr_async,
tungstenite::handshake::server::{ErrorResponse, Request, Response},
};
#[allow(clippy::result_large_err)] // Required by tungstenite's callback signature.
fn accept_test_protocol(request: &Request, mut response: Response) -> Result<Response, ErrorResponse> {
assert_eq!(request.headers()["authorization"], "Bearer test");
assert_eq!(request.headers()["sec-websocket-protocol"], "openidea.test.v1");
response.headers_mut().insert("sec-websocket-protocol", "openidea.test.v1".parse().unwrap());
Ok(response)
}
async fn server() -> (String, tokio::task::JoinHandle<Vec<Vec<u8>>>) {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let url = format!("ws://{}/socket", listener.local_addr().unwrap());
let task = tokio::spawn(async move {
let (stream, _) = listener.accept().await.unwrap();
let socket = accept_hdr_async(stream, accept_test_protocol).await.unwrap();
let (_, mut reader) = socket.split();
let mut messages = Vec::new();
while let Ok(Some(message)) = tokio::time::timeout(Duration::from_secs(2), reader.next()).await {
match message.unwrap() {
Message::Binary(bytes) => messages.push(bytes.to_vec()),
Message::Close(_) => break,
_ => {}
}
}
messages
});
(url, task)
}
#[tokio::test]
async fn outbound_message_reaches_the_websocket_server() {
let (url, server) = server().await;
let hub = WebSocketHub::default();
let connection = hub
.connect(
"plugin",
"/project",
&url,
&[("authorization".into(), "Bearer test".into())],
"openidea.test.v1",
Duration::from_secs(1),
)
.await
.unwrap();
hub.send("plugin", connection, b"message".to_vec()).unwrap();
hub.disconnect("plugin", connection).unwrap();
assert_eq!(server.await.unwrap(), [b"message"]);
}
#[tokio::test]
async fn inbound_messages_can_be_drained_and_disconnect_cleans_the_handle() {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let url = format!("ws://{}/socket", listener.local_addr().unwrap());
let server = tokio::spawn(async move {
let (stream, _) = listener.accept().await.unwrap();
let mut socket = accept_hdr_async(stream, accept_test_protocol).await.unwrap();
socket.send(Message::Binary(vec![1, 2, 3].into())).await.unwrap();
socket.close(None).await.unwrap();
});
let hub = WebSocketHub::default();
let connection = hub
.connect(
"plugin",
"/project",
&url,
&[("authorization".into(), "Bearer test".into())],
"openidea.test.v1",
Duration::from_secs(1),
)
.await
.unwrap();
server.await.unwrap();
tokio::time::sleep(Duration::from_millis(20)).await;
assert_eq!(
hub.drain("plugin", connection, 1).unwrap(),
WebSocketBatch { messages: vec![vec![1, 2, 3]], closed: true }
);
assert!(hub.drain("plugin", connection, 1).is_err());
}
#[test]
fn send_rejects_full_queues_without_reporting_success() {
let hub = WebSocketHub::default();
let (stop, _) = watch::channel(false);
let connection = Arc::new(Connection {
plugin: "plugin".into(),
outgoing: Arc::new(Inbox::new(1, MAX_BYTES, MAX_FRAME)),
incoming: Arc::new(Inbox::new(1, MAX_BYTES, MAX_FRAME)),
wake: Notify::new(),
stop,
});
hub.0.connections.lock().insert(1, connection.clone());
assert!(hub.send("other-plugin", 1, b"cross-plugin".to_vec()).is_err());
assert!(hub.drain("other-plugin", 1, 1).is_err());
assert!(hub.send("plugin", 1, b"first".to_vec()).is_ok());
assert!(hub.send("plugin", 1, b"second".to_vec()).unwrap_err().contains("rejected"));
assert_eq!(connection.outgoing.drain(1).messages, [b"first"]);
}
#[test]
fn websocket_urls_require_tls_and_keep_credentials_out_of_the_url() {
assert!(validate_url("wss://relay.example/socket").is_ok());
assert!(validate_url("wss://relay.example/socket?channel=main&token=opaque").is_ok());
assert!(validate_url("ws://127.0.0.1:9000/socket").is_ok());
assert!(validate_url("ws://[::1]:9000/socket").is_ok());
assert!(validate_url("ws://localhost:9000/socket").is_ok());
assert!(validate_url("ws://relay.example/socket").unwrap_err().contains("WSS"));
assert!(validate_url("wss://[email protected]/socket").is_err());
assert!(validate_url("wss://relay.example/socket#fragment").is_err());
assert!(protected_header("Host"));
assert!(protected_header("Connection"));
assert!(protected_header("Upgrade"));
assert!(protected_header("Sec-WebSocket-Key"));
assert!(!protected_header("Cookie"));
assert!(!protected_header("X-API-Key"));
}
#[tokio::test]
#[allow(clippy::result_large_err)] // Required by tungstenite's callback signature.
async fn optional_subprotocol_and_custom_headers_reach_the_handshake() {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let url = format!("ws://{}/socket?channel=main", listener.local_addr().unwrap());
let server = tokio::spawn(async move {
let (stream, _) = listener.accept().await.unwrap();
let mut socket = accept_hdr_async(stream, |request: &Request, response: Response| {
assert_eq!(request.headers()["cookie"], "session=test");
assert_eq!(request.headers()["x-api-key"], "key");
assert!(request.headers().get("sec-websocket-protocol").is_none());
Ok(response)
})
.await
.unwrap();
socket.close(None).await.unwrap();
});
let hub = WebSocketHub::default();
let connection = hub
.connect(
"plugin",
"/project",
&url,
&[("cookie".into(), "session=test".into()), ("x-api-key".into(), "key".into())],
"",
Duration::from_secs(1),
)
.await
.unwrap();
hub.disconnect("plugin", connection).unwrap();
server.await.unwrap();
}
#[test]
fn connection_limit_is_enforced_before_connecting() {
let hub = WebSocketHub::default();
for _ in 0..MAX_CONNECTIONS_PER_PLUGIN {
hub.reserve("plugin").unwrap();
}
assert_eq!(
hub.reserve("plugin"),
Err(ConnectError::Invalid("at most 8 WebSocket connections per plugin".into()))
);
for _ in 0..MAX_CONNECTIONS_PER_PLUGIN {
hub.release("plugin");
}
}
}
-47
View File
@@ -69,8 +69,6 @@ pub struct Project {
/// watchers for (a virtualenv elsewhere, a path dependency).
lsp_watchers: Mutex<Vec<(PathBuf, Watcher)>>,
pub(crate) _scratch_watcher: Option<Watcher>,
/// Picks up data sources edited on disk outside the Database tool window.
_db_sources_watch: Option<ide_db::SourcesWatch>,
/// A LightEdit window's loose files ([`Host::open_light`]): global settings only,
/// nothing indexed, no language servers, repositories or project services.
pub light: bool,
@@ -538,12 +536,10 @@ impl Host {
tracing::info!("opened {} in window {window} in {} ms {laps:?}", root.display(), started.elapsed().as_millis());
let (db, http_history, agents) = self.project_services(&root, &project_key, &settings);
let db_sources_watch = db_sources_watch(&db, &sink, &self.runtime);
let notebooks = crate::notebook::NotebookService::new(
self.local_data_dir.join("notebook-runtime").join(uuid::Uuid::new_v4().to_string()),
sink.clone(),
);
notebooks.set_observer(Arc::new(NotebookLspObserver(lsp.clone())));
ide_agent::registry::maintain(crate::ops::agent::registry_paths(self));
let plugin_diagnostics =
crate::plugins::diagnostics::Diagnostics::new(sink.clone(), self.runtime.clone(), &documents, &root);
@@ -580,7 +576,6 @@ impl Host {
content_watchers: Mutex::new(content_watchers),
lsp_watchers: Mutex::new(Vec::new()),
_scratch_watcher: scratch_watcher,
_db_sources_watch: db_sources_watch,
files_events_active,
light: false,
});
@@ -743,7 +738,6 @@ impl Host {
content_watchers: Mutex::new(Vec::new()),
lsp_watchers: Mutex::new(Vec::new()),
_scratch_watcher: None,
_db_sources_watch: None,
files_events_active: Arc::new(AtomicBool::new(false)),
light: true,
root: dir,
@@ -796,28 +790,6 @@ fn documents(sink: &SharedSink, languages: &Arc<LanguageRegistry>, root: Option<
)
}
/// Data sources edited on disk outside the Database tool window (by hand, by a coding
/// agent): connections made with the old settings close and the tree re-reads them,
/// without reopening the project.
fn db_sources_watch(
db: &Arc<ide_db::DbManager>,
sink: &SharedSink,
runtime: &tokio::runtime::Handle,
) -> Option<ide_db::SourcesWatch> {
let (weak, sink, runtime) = (Arc::downgrade(db), sink.clone(), runtime.clone());
db.watch_sources(Box::new(move || {
let Some(db) = weak.upgrade() else { return };
let sink = sink.clone();
runtime.spawn(async move {
if db.reconcile_sources().await {
sink.emit(IdeEvent::DbSourcesChanged);
}
});
}))
.inspect_err(|e| tracing::warn!("{}", e.message))
.ok()
}
fn light_watcher(
dir: &Path,
sink: &SharedSink,
@@ -1606,25 +1578,6 @@ impl DocumentObserver for LspObserver {
}
}
/// Feeds the code cells of open notebooks to the Python language servers that sync notebooks.
struct NotebookLspObserver(Arc<LspManager>);
impl crate::notebook::NotebookObserver for NotebookLspObserver {
fn changed(&self, path: &Path, snapshot: &ide_notebook::NotebookSnapshot) {
let cells: Vec<ide_lsp::NotebookCell> = snapshot
.cells
.iter()
.filter(|c| c.cell_type == "code")
.map(|c| ide_lsp::NotebookCell { id: c.id.clone(), source: c.source.clone() })
.collect();
// Notebooks run Python kernels only.
self.0.sync_notebook(path, &LanguageId::new("python"), &cells);
}
fn closed(&self, path: &Path) {
self.0.close_notebook(path);
}
}
/// Feeds document edits into the per-project syntax tree cache.
struct SyntaxObserver(Arc<SyntaxCache>);
+1 -1
View File
@@ -39,7 +39,7 @@ impl PluginWorkspace for Workspace {
fn document(&self, _: &Path) -> Option<(u64, String)> {
None
}
fn apply_edits(&self, _: &Path, _: u64, _: &[ide_core::TextEdit], _: bool) -> Result<u64, HostError> {
fn apply_edits(&self, _: &Path, _: u64, _: &[ide_core::TextEdit]) -> Result<u64, HostError> {
Err(HostError::NotFound("not open".into()))
}
fn trusted(&self, _: &Path) -> bool {
+6 -112
View File
@@ -154,20 +154,9 @@ impl PluginWorkspace for TestWorkspace {
let snap = self.documents.snapshot(self.documents.by_path(path)?).ok()?;
Some((u64::from(snap.version), snap.text))
}
fn apply_edits(
&self,
path: &Path,
version: u64,
edits: &[ide_core::TextEdit],
background: bool,
) -> Result<u64, HostError> {
fn apply_edits(&self, path: &Path, version: u64, edits: &[ide_core::TextEdit]) -> Result<u64, HostError> {
let doc = self.documents.by_path(path).ok_or_else(|| HostError::NotFound("not open".into()))?;
let applied = if background {
self.documents.apply_background_edits_at(doc, version as u32, edits)
} else {
self.documents.apply_edits_at(doc, version as u32, edits)
};
match applied {
match self.documents.apply_edits_at(doc, version as u32, edits) {
Ok(Some(v)) => Ok(u64::from(v)),
Ok(None) => Err(HostError::Stale),
Err(e) => Err(HostError::Failed(e.message)),
@@ -466,10 +455,10 @@ fn edits_are_refused_when_the_document_changed() {
range: ide_core::Range::new(ide_core::Pos::new(0, 0), ide_core::Pos::new(0, 3)),
new_text: "ONE".into(),
};
assert_eq!(f.workspace.apply_edits(&editor.path, doc.0, std::slice::from_ref(&edit), false), Err(HostError::Stale));
assert_eq!(f.workspace.apply_edits(&editor.path, doc.0, std::slice::from_ref(&edit)), Err(HostError::Stale));
assert_eq!(f.workspace.text("src/notes.txt"), "typed one two three\nfour five\n");
let now = f.workspace.document(&editor.path).unwrap().0;
assert_eq!(f.workspace.apply_edits(&editor.path, now, &[edit], false), Ok(now + 1));
assert_eq!(f.workspace.apply_edits(&editor.path, now, &[edit]), Ok(now + 1));
assert_eq!(f.workspace.text("src/notes.txt"), "ONEed one two three\nfour five\n");
let _ = (
std::any::type_name::<Runtime>(),
@@ -819,28 +808,6 @@ fn a_reporting_task_outlives_its_background_budget() {
assert_eq!((info.status, info.stats.crashes, info.stats.errors), (PluginStatus::Active, 0, 0), "{info:?}");
}
#[test]
fn background_tasks_scale_and_renew_cpu_fuel_budget() {
let f = fixture(Limits {
fuel: 3_500,
timeout: Duration::from_millis(500),
background: Duration::from_millis(5_000),
..Limits::default()
});
f.install(&misbehave_pkg());
if f.host.set_capability("misbehave", "process:git", true).is_err() {
return;
}
// Warm up activation in its own call so `patient` starts with a fresh 3_500-fuel budget.
assert_eq!(f.run("misbehave", "echo", Some("warm")).unwrap().as_deref(), Some("warm"));
let out = f.run("misbehave", "patient", None).expect("background task succeeds").expect("output");
assert_eq!(out, "still here");
let err = f.run("misbehave", "spin", None).unwrap_err();
assert!(err.contains("exceeded its CPU budget (3500 fuel per call)"), "{err}");
}
#[test]
fn private_directory_is_owned_persistent_and_removed_on_uninstall() {
let f = fixture(Limits {
@@ -4840,14 +4807,8 @@ impl PluginWorkspace for EndpointWorkspace {
fn project_of(&self, _: &str) -> Option<PathBuf> {
self.roots.lock().first().cloned()
}
fn apply_edits(
&self,
path: &Path,
version: u64,
edits: &[ide_core::TextEdit],
background: bool,
) -> Result<u64, HostError> {
self.inner.apply_edits(path, version, edits, background)
fn apply_edits(&self, path: &Path, version: u64, edits: &[ide_core::TextEdit]) -> Result<u64, HostError> {
self.inner.apply_edits(path, version, edits)
}
fn publish_diagnostics(
&self,
@@ -6687,73 +6648,6 @@ fn project_write_cannot_replace_the_git_file_of_a_linked_worktree() {
}
}
#[cfg(unix)]
#[test]
fn process_timeout_returns_guest_error_without_disabling_plugin() {
use std::os::unix::fs::PermissionsExt;
let f = fixture(Limits {
timeout: Duration::from_millis(120),
max_failures: 3,
restart_backoff: Duration::ZERO,
..Limits::default()
});
let package = f._tmp.path().join("api14-slow-proc");
copy_dir(&sample("api14"), &package);
let manifest = package.join("plugin.toml");
let text =
std::fs::read_to_string(&manifest).unwrap().replace("optional = [", "optional = [\"process:api14-tool\", ");
std::fs::write(manifest, text).unwrap();
f.install(&package);
let slow = f._tmp.path().join("slow-tool");
std::fs::write(&slow, "#!/bin/sh\nsleep 5\n").unwrap();
std::fs::set_permissions(&slow, std::fs::Permissions::from_mode(0o755)).unwrap();
let slow = slow.canonicalize().unwrap();
f.host.choose_program("api14", "process:api14-tool", Some(&f.project), Some(&slow)).unwrap();
// Even past `max_failures` (3), external subprocess timeouts unwind as guest errors
// instead of poisoning the Wasm instance and disabling the plugin for the session.
for _ in 0..4 {
let err = f.run("api14", "process-run", None).unwrap_err();
assert!(err.contains("api14-tool timed out"), "{err}");
}
let info = f.host.info_of("api14").unwrap();
assert_eq!(info.status, PluginStatus::Active);
assert_eq!(info.stats.crashes, 0);
}
#[cfg(unix)]
#[test]
fn process_stdout_accepts_large_payloads_and_reports_overflow_explicitly() {
use std::os::unix::fs::PermissionsExt;
let f = fixture(Limits { timeout: Duration::from_secs(10), ..Limits::default() });
let package = f._tmp.path().join("api14-large-proc");
copy_dir(&sample("api14"), &package);
let manifest = package.join("plugin.toml");
let text =
std::fs::read_to_string(&manifest).unwrap().replace("optional = [", "optional = [\"process:api14-tool\", ");
std::fs::write(manifest, text).unwrap();
f.install(&package);
let medium_bytes = 5 * 1024 * 1024;
let medium = f._tmp.path().join("medium-tool");
std::fs::write(&medium, "#!/bin/sh\ndd if=/dev/zero bs=1048576 count=5 2>/dev/null | tr '\\0' 'x'\n").unwrap();
std::fs::set_permissions(&medium, std::fs::Permissions::from_mode(0o755)).unwrap();
let medium = medium.canonicalize().unwrap();
f.host.choose_program("api14", "process:api14-tool", Some(&f.project), Some(&medium)).unwrap();
let out = f.run("api14", "process-run", None).unwrap().unwrap();
assert_eq!(out.len(), medium_bytes);
let huge = f._tmp.path().join("huge-tool");
std::fs::write(&huge, "#!/bin/sh\ndd if=/dev/zero bs=1048576 count=17 2>/dev/null | tr '\\0' 'x'\n").unwrap();
std::fs::set_permissions(&huge, std::fs::Permissions::from_mode(0o755)).unwrap();
let huge = huge.canonicalize().unwrap();
f.host.choose_program("api14", "process:api14-tool", Some(&f.project), Some(&huge)).unwrap();
let err = f.run("api14", "process-run", None).unwrap_err();
assert!(err.contains("stdout exceeded 16 MiB limit"), "{err}");
assert_eq!(f.host.info_of("api14").unwrap().stats.crashes, 0);
}
/// A link already in the project is written through, so it is judged by where it points.
#[cfg(unix)]
#[test]
+6 -12
View File
@@ -29,23 +29,17 @@ pub fn build(proxy: &ProxySettings) -> IdeResult<reqwest::Client> {
build_with_redirect(proxy, reqwest::redirect::Policy::default())
}
/// The backend's TLS configuration: ring with the operating system's trust store.
pub fn tls_config() -> IdeResult<rustls::ClientConfig> {
/// A client with an explicit redirect policy, for callers enforcing network grants.
pub fn build_with_redirect(proxy: &ProxySettings, redirect: reqwest::redirect::Policy) -> IdeResult<reqwest::Client> {
use rustls_platform_verifier::BuilderVerifierExt;
let fail = |e: String| IdeError::internal(format!("TLS client: {e}"));
let fail = |e: String| IdeError::internal(format!("HTTP client: {e}"));
let provider = Arc::new(rustls::crypto::ring::default_provider());
rustls::ClientConfig::builder_with_provider(provider)
let tls = rustls::ClientConfig::builder_with_provider(provider)
.with_safe_default_protocol_versions()
.map_err(|e| fail(e.to_string()))?
.with_platform_verifier()
.map_err(|e| fail(e.to_string()))
.map(|builder| builder.with_no_client_auth())
}
/// A client with an explicit redirect policy, for callers enforcing network grants.
pub fn build_with_redirect(proxy: &ProxySettings, redirect: reqwest::redirect::Policy) -> IdeResult<reqwest::Client> {
let fail = |e: String| IdeError::internal(format!("HTTP client: {e}"));
let tls = tls_config()?;
.map_err(|e| fail(e.to_string()))?
.with_no_client_auth();
// `localhost` is tried on IPv4 first: local servers (Ollama, dev servers)
// usually listen there, and on Windows a refused `::1` costs ~2 s before
// the system resolver's order falls back. Ports come from the URL.
+2 -17
View File
@@ -5,11 +5,10 @@
use std::path::{Path, PathBuf};
/// The outermost ancestor of `file` containing a root marker, within
/// `project_root` (any ancestor without one: loose files). A directory counts
/// as its own ancestor: a module folder is served like the files in it.
/// `project_root` (any ancestor without one: loose files).
pub fn marked_root(project_root: Option<&Path>, file: &Path, markers: &[String]) -> Option<PathBuf> {
let mut best = None;
for dir in file.ancestors().skip(usize::from(!file.is_dir())) {
for dir in file.ancestors().skip(1) {
if project_root.is_some_and(|root| !dir.starts_with(root)) {
break;
}
@@ -113,20 +112,6 @@ mod tests {
assert_eq!(marked_root(Some(root), &root.join("apps/api/main.ts"), &markers), None);
}
#[test]
fn a_marked_directory_is_its_own_root() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
let module = root.join("first");
std::fs::create_dir_all(module.join("src")).unwrap();
std::fs::write(module.join("pom.xml"), "").unwrap();
let markers = vec!["pom.xml".to_string()];
assert_eq!(marked_root(Some(root), &module.join("src/Main.java"), &markers), Some(module.clone()));
assert_eq!(marked_root(Some(root), &module, &markers), Some(module.clone()));
assert_eq!(marked_root(Some(root), &module.join("src"), &markers), Some(module));
assert_eq!(marked_root(Some(root), &root.join("second"), &markers), None);
}
#[test]
fn markers_above_the_project_root_only_count_without_one() {
let tmp = tempfile::tempdir().unwrap();
+1 -51
View File
@@ -163,9 +163,7 @@ pub fn client_capabilities() -> Value {
"typeHierarchy": { "dynamicRegistration": false }
},
// rust-analyzer: report when the project has loaded (`quiescent`).
"experimental": { "serverStatusNotification": true },
// Jupyter notebooks: code cells are synced with `notebookDocument/*` (see `notebook.rs`).
"notebookDocument": { "synchronization": { "dynamicRegistration": false, "executionSummarySupport": false } }
"experimental": { "serverStatusNotification": true }
})
}
@@ -428,34 +426,6 @@ impl ServerCaps {
.unwrap_or_default()
}
/// Whether `notebookDocumentSync` takes Jupyter notebooks (`file:`) with cells of `language`:
/// a selector that names no notebook, or `jupyter-notebook` / `*`, and no cell languages or
/// `language` among them.
pub fn syncs_notebooks(&self, language: &str) -> bool {
let Some(selectors) = self.raw.pointer("/notebookDocumentSync/notebookSelector").and_then(Value::as_array)
else {
return false;
};
selectors.iter().any(|s| {
let notebook = match s.get("notebook") {
None | Some(Value::Null) => true,
Some(Value::String(kind)) => kind == "jupyter-notebook" || kind == "*",
Some(filter) => {
filter
.get("notebookType")
.and_then(Value::as_str)
.is_none_or(|t| t == "jupyter-notebook" || t == "*")
&& filter.get("scheme").and_then(Value::as_str).is_none_or(|s| s == "file")
}
};
let cells = match s.get("cells").and_then(Value::as_array) {
None => true,
Some(cells) => cells.iter().any(|c| c.get("language").and_then(Value::as_str) == Some(language)),
};
notebook && cells && (s.get("notebook").is_some() || s.get("cells").is_some())
})
}
/// `workspace.fileOperations.<op>` registration options (`op`: `willRename`, `didRename`).
pub fn file_operation(&self, op: &str) -> Option<&Value> {
self.raw.pointer(&format!("/workspace/fileOperations/{op}")).filter(|v| v.is_object())
@@ -698,26 +668,6 @@ mod tests {
assert_eq!(c.code_action_kinds(), Some(vec!["refactor.inline".into()]));
}
#[test]
fn notebook_sync_selectors() {
let syncs = |selector: Value| {
ServerCaps::parse(json!({ "notebookDocumentSync": { "notebookSelector": [selector] } }))
.syncs_notebooks("python")
};
// basedpyright
assert!(syncs(json!({
"notebook": { "scheme": "file", "notebookType": "jupyter-notebook" }, "cells": [{ "language": "python" }]
})));
// Ruff: any notebook with Python cells.
assert!(syncs(json!({ "cells": [{ "language": "python" }] })));
assert!(syncs(json!({ "notebook": "*" })));
assert!(!syncs(json!({ "notebook": "interactive" })));
assert!(!syncs(json!({ "notebook": { "notebookType": "jupyter-notebook", "scheme": "untitled" } })));
assert!(!syncs(json!({ "cells": [{ "language": "r" }] })));
assert!(!syncs(json!({})));
assert!(!ServerCaps::parse(json!({ "textDocumentSync": 2 })).syncs_notebooks("python"));
}
#[test]
fn language_definitions_use_known_feature_names() {
for lang in ide_language::LanguageRegistry::load(None).unwrap().all() {
+8 -37
View File
@@ -966,12 +966,10 @@ pub fn inlay_hint(conv: &Conv, path: &Path, h: lsp::InlayHint) -> InlayHint {
}
}
/// Decodes relative semantic token data into absolute UTF-16 quintuples, sorted by position.
/// Deltas are read as signed because some servers (JDT LS) emit tokens out of order; tokens that
/// land before the document start are dropped and exact duplicates removed.
pub fn decode_semantic_tokens(data: &[i64], enc: Encoding, text: Option<&DocText>) -> Vec<u32> {
let mut out: Vec<[u32; 5]> = Vec::with_capacity(data.len() / 5);
let (mut line, mut start) = (0i64, 0i64);
/// Decodes relative semantic token data into absolute UTF-16 quintuples.
pub fn decode_semantic_tokens(data: &[u32], enc: Encoding, text: Option<&DocText>) -> Vec<u32> {
let mut out = Vec::with_capacity(data.len());
let (mut line, mut start) = (0u32, 0u32);
for &[dl, ds, len, ty, mods] in data.as_chunks::<5>().0 {
if dl == 0 {
start = start.saturating_add(ds);
@@ -979,21 +977,16 @@ pub fn decode_semantic_tokens(data: &[i64], enc: Encoding, text: Option<&DocText
line = line.saturating_add(dl);
start = ds;
}
let (Ok(line), Ok(start), Ok(len)) = (u32::try_from(line), u32::try_from(start), u32::try_from(len)) else {
continue;
};
let (col, len16) = match (enc, text) {
(Encoding::Utf16, _) | (_, None) => (start, len),
(_, Some(t)) => {
let (a, b) = convert_offsets(t.line(line as usize), start as usize, start as usize + len as usize, enc);
let (a, b) = convert_offsets(t.line(line as usize), start as usize, (start + len) as usize, enc);
(a, b - a)
}
};
out.push([line, col, len16, ty as u32, mods as u32]);
out.extend_from_slice(&[line, col, len16, ty, mods]);
}
out.sort_by_key(|t| (t[0], t[1]));
out.dedup();
out.into_flattened()
out
}
pub fn code_lens(conv: &Conv, path: &Path, l: lsp::CodeLens) -> CodeLens {
@@ -1480,7 +1473,7 @@ mod tests {
#[test]
fn semantic_token_decoding() {
let data: [i64; 15] = [1, 2, 3, 0, 1, 0, 5, 2, 1, 0, 2, 0, 4, 2, 3];
let data = [1, 2, 3, 0, 1, 0, 5, 2, 1, 0, 2, 0, 4, 2, 3];
let out = decode_semantic_tokens(&data, Encoding::Utf16, None);
assert_eq!(out, vec![1, 2, 3, 0, 1, 1, 7, 2, 1, 0, 3, 0, 4, 2, 3]);
// UTF-8 server: "😀ab" token "ab" at byte 4 len 2 → utf16 col 2 len 2.
@@ -1488,28 +1481,6 @@ mod tests {
assert_eq!(decode_semantic_tokens(&[0, 4, 2, 0, 0], Encoding::Utf8, Some(&text)), vec![0, 2, 2, 0, 0]);
}
#[test]
fn semantic_tokens_out_of_order() {
// JDT LS for `public class Repro<T extends Number> extends\n ArrayList<T> {`: the bound's
// `extends` comes after `Number`, the superclass is emitted twice and the line goes backwards.
let data: [i64; 95] = [
0, 8, 4, 0, 0, 2, 7, 4, 0, 4096, 0, 5, 4, 0, 4096, 0, 5, 9, 1, 5184, 2, 0, 6, 11, 0, 0, 7, 5, 11, 0, 0, 6,
5, 1, 1104, 0, 6, 1, 6, 16, 0, 10, 6, 1, 65, 0, -8, 7, 11, 0, 1, 8, 9, 1, 1088, 0, 10, 1, 6, 2048, -1, 37,
7, 11, 0, 1, 8, 9, 1, 1088, 0, 10, 1, 6, 2048, 1, 4, 7, 11, 0, 0, 12, 5, 8, 144, 2, 9, 1, 7, 1040, 1, 8, 5,
8, 128,
];
let out = decode_semantic_tokens(&data, Encoding::Utf16, None);
let positions: Vec<(u32, u32)> = out.chunks(5).map(|t| (t[0], t[1])).collect();
#[rustfmt::skip]
assert_eq!(positions, [
(0, 8), (2, 7), (2, 12), (2, 17),
(4, 0), (4, 7), (4, 13), (4, 19), (4, 21), (4, 29), (4, 37),
(5, 8), (5, 18), (6, 4), (6, 16), (8, 9), (9, 8),
]);
// A token that would start before the document is dropped rather than wrapped.
assert_eq!(decode_semantic_tokens(&[0, -1, 2, 0, 0, 0, 5, 1, 0, 0], Encoding::Utf16, None), [0, 4, 1, 0, 0]);
}
#[test]
fn diagnostics_mapping() {
let conv = Conv::utf16();
+1 -6
View File
@@ -10,7 +10,6 @@ use ide_core::{IdeEvent, TaskId};
use ide_jsonrpc::{Handler, codes};
use lsp_types as lsp;
use serde_json::{Value, json};
use std::path::PathBuf;
use std::sync::{Arc, Weak};
use std::time::Duration;
@@ -230,11 +229,7 @@ impl ClientHandler {
return;
}
};
let Some(path) = uri_to_path(params.uri.as_str())
.or_else(|| crate::notebook::is_cell_uri(params.uri.as_str()).then(|| PathBuf::from(params.uri.as_str())))
else {
return;
};
let Some(path) = uri_to_path(params.uri.as_str()) else { return };
if self.server.name == "jdtls"
&& let Some(failure) = crate::jdtls::build_plan_failure(&path, &self.server.root, &params.diagnostics)
{
+7 -42
View File
@@ -25,7 +25,6 @@ mod kotlin_lsp;
mod launch;
mod library_source;
mod loading;
mod notebook;
mod pull;
mod scope;
mod server;
@@ -43,7 +42,6 @@ pub use inspect::{FileInspection, InspectHost, InspectOptions};
pub use jdtls::jvm_inspect::JdtJvmInspector;
pub use kotlin_lsp::{KotlinDebugServer, KotlinDebugUnavailable};
pub use launch::{LaunchError, Launcher, ProcessLauncher};
pub use notebook::{NotebookCell, NotebookCellDoc, cell_uri};
pub use scope::{CandidateFiles, CandidateFn, CandidateQuery, MAX_CANDIDATES};
pub use uri::{is_virtual_uri, path_to_uri_string, uri_target, uri_to_path};
@@ -117,8 +115,6 @@ struct DocState {
servers: Vec<Attached>,
/// Open YAML document whose filename does not match a Compose glob and whose content is Compose.
compose_detected: bool,
/// The notebook a code cell belongs to: synced through `notebookDocument/*`, not as a file.
notebook: Option<PathBuf>,
}
impl DocState {
@@ -233,8 +229,6 @@ pub struct LspManager {
/// Contents fetched for virtual URIs.
virtual_docs: Mutex<HashMap<String, VirtualDocument>>,
injections: injections::State,
/// Open notebooks by path. Locked before `docs` when both are.
notebooks: Mutex<HashMap<PathBuf, notebook::NotebookState>>,
}
/// Fetched virtual documents kept before the cache is reset.
@@ -482,7 +476,6 @@ impl LspManager {
virtual_origins: Mutex::new(HashMap::new()),
virtual_docs: Mutex::new(HashMap::new()),
injections: injections::State::default(),
notebooks: Mutex::new(HashMap::new()),
})
}
@@ -621,13 +614,8 @@ impl LspManager {
/// enabled in settings.
fn attach_fallbacks(&self) {
let _gate = self.injections.gate.lock();
let open: Vec<(DocId, LanguageId, PathBuf)> = self
.docs
.lock()
.iter()
.filter(|(_, d)| d.notebook.is_none())
.map(|(id, d)| (*id, d.language.clone(), d.path.clone()))
.collect();
let open: Vec<(DocId, LanguageId, PathBuf)> =
self.docs.lock().iter().map(|(id, d)| (*id, d.language.clone(), d.path.clone())).collect();
let wanted: Vec<(DocId, Vec<Arc<Server>>)> =
open.into_iter().map(|(id, language, path)| (id, self.servers_for(&language, &path))).collect();
let mut start = Vec::new();
@@ -660,11 +648,6 @@ impl LspManager {
for (doc, server) in opened {
self.schedule_pull(doc, Some(&server), Duration::ZERO);
}
for server in self.attach_notebook_servers() {
if !start.iter().any(|s| s.id == server.id) {
start.push(server);
}
}
for server in &start {
self.ensure_started(server);
}
@@ -773,7 +756,7 @@ impl LspManager {
Err(LaunchError::Missing) => {
let cfg = server.effective_config();
let message =
cfg.install_hint.clone().unwrap_or_else(|| format!("{} was not found on PATH", cfg.command));
cfg.install_hint.clone().unwrap_or_else(|| format!("`{}` was not found on PATH", cfg.command));
if server.fail_start(
generation,
Conn::Missing,
@@ -884,7 +867,7 @@ impl LspManager {
false
} else {
let open_close = run.caps.read().open_close;
for (id, d) in docs.iter_mut().filter(|(_, d)| d.notebook.is_none()) {
for (id, d) in docs.iter_mut() {
let params = d.did_open_params();
for a in
d.servers.iter_mut().filter(|a| a.server.id == server.id && a.opened_gen != Some(generation))
@@ -906,7 +889,6 @@ impl LspManager {
for doc in reopened {
self.schedule_pull(doc, Some(&server), Duration::ZERO);
}
self.reopen_notebooks(&server, &run);
self.check_clang_tidy(&server).await;
self.check_vue_navigation(&server).await;
server.refresh_busy_status(&self.sink);
@@ -1101,7 +1083,6 @@ impl LspManager {
text: Arc::new(DocText::new(text)),
servers: servers.iter().map(|s| Attached { server: s.clone(), opened_gen: None }).collect(),
compose_detected,
notebook: None,
};
let compose_added = {
let mut docs = self.docs.lock();
@@ -1233,7 +1214,7 @@ impl LspManager {
return;
}
let docs = self.docs.lock();
for d in docs.values().filter(|d| d.notebook.is_none()) {
for d in docs.values() {
if !d.servers.iter().any(|a| a.server.id == server.id && a.opened_gen == Some(generation)) {
continue;
}
@@ -1307,23 +1288,13 @@ impl LspManager {
fn snapshot(&self, doc: DocId) -> IdeResult<DocSnap> {
let docs = self.docs.lock();
let d = docs.get(&doc).ok_or_else(|| IdeError::not_found(format!("{doc} is not open")))?;
// A cell is asked only of the servers its notebook is open on (those that sync notebooks).
let servers = d
.servers
.iter()
.filter(|a| {
d.notebook.is_none()
|| a.opened_gen.is_some_and(|g| a.server.running().is_some_and(|run| run.generation == g))
})
.map(|a| a.server.clone())
.collect();
Ok(DocSnap {
version: d.version,
path: d.path.clone(),
uri: d.uri.clone(),
language: d.language.clone(),
text: d.text.clone(),
servers,
servers: d.servers.iter().map(|a| a.server.clone()).collect(),
})
}
@@ -2851,7 +2822,7 @@ impl LspManager {
let params = json!({ "textDocument": snap.ident() });
let r: Option<Value> = self.call(t, "textDocument/semanticTokens/full", params, None, DEFAULT_TIMEOUT).await?;
let Some(data) = r.as_ref().and_then(|v| v.get("data")).and_then(Value::as_array) else { return Ok(None) };
let data: Vec<i64> = data.iter().map(|n| n.as_i64().unwrap_or(0)).collect();
let data: Vec<u32> = data.iter().map(|n| n.as_u64().unwrap_or(0) as u32).collect();
let (enc, legend) = {
let caps = t.caps();
(caps.encoding, caps.semantic_legend.clone().unwrap_or_default())
@@ -3519,17 +3490,11 @@ impl LspManager {
join_all(servers.iter().map(|s| self.stop_instance(s, true))).await;
return;
}
let existing: Vec<ide_core::ServerId> = self.servers.lock().iter().map(|s| s.id).collect();
self.reconfigure_jdtls_workspace().await;
self.stop_unlisted().await;
let Some(me) = self.me.upgrade() else { return };
let servers: Vec<Arc<Server>> = self.servers.lock().clone();
for s in servers {
// Created in this pass (the jdtls of a new workspace layout): it starts with the
// current configuration, though it may not have recorded it yet.
if !existing.contains(&s.id) {
continue;
}
// The settings hook may run tools (`npm root -g`), as when the server starts.
let next = match tokio::task::spawn_blocking({
let (me, s) = (me.clone(), s.clone());
-382
View File
@@ -1,382 +0,0 @@
//! Jupyter notebooks on servers that sync them with LSP 3.17 `notebookDocument/*` (basedpyright,
//! Ruff). Each code cell is a document of its own, `vscode-notebook-cell:<notebook path>#<cell id>`,
//! kept in `docs` under a [`DocId`]: completion, hover and the other requests address a cell like
//! any open file. Only opening, changing and closing go through the notebook methods; a change in
//! the cells' order closes and reopens the notebook.
use crate::server::{Running, Server};
use crate::text::DocText;
use crate::{Attached, DocState, LspManager, uri};
use ide_core::{DocId, LanguageId, ServerId};
use lsp_types as lsp;
use percent_encoding::{AsciiSet, NON_ALPHANUMERIC, utf8_percent_encode};
use serde::Serialize;
use serde_json::{Value, json};
use specta::Type;
use std::path::{Path, PathBuf};
use std::str::FromStr;
use std::sync::Arc;
const CELL_SCHEME: &str = "vscode-notebook-cell";
const NOTEBOOK_TYPE: &str = "jupyter-notebook";
/// `NotebookCellKind.Code`.
const CODE_CELL: u32 = 2;
/// Escaped in a cell id: nbformat ids are `[a-zA-Z0-9-_]`, older notebooks may have others.
const FRAGMENT: &AsciiSet = &NON_ALPHANUMERIC.remove(b'-').remove(b'_');
/// A code cell as the notebook holds it.
pub struct NotebookCell {
pub id: String,
pub source: String,
}
/// The document the language servers know a code cell by.
#[derive(Clone, Debug, PartialEq, Serialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct NotebookCellDoc {
pub cell: String,
pub doc: DocId,
/// Also the `path` of the cell's diagnostics and of locations in it.
pub uri: String,
}
pub(crate) struct NotebookState {
uri: lsp::Uri,
language: LanguageId,
version: i32,
/// Code cells in notebook order.
cells: Vec<(String, DocId)>,
servers: Vec<Arc<Server>>,
/// Servers it is open on, by instance generation.
opened: Vec<(ServerId, u64)>,
}
/// The URI of cell `cell` of the notebook at `path`: the notebook's `file:` URI with the cell
/// scheme and the cell id as fragment (as VS Code names cells; basedpyright expects the scheme).
pub fn cell_uri(path: &Path, cell: &str) -> String {
let file = uri::path_to_uri_string(path);
let rest = file.strip_prefix("file://").unwrap_or(&file);
// `file:///C:/a` → `…:/C:/a`; a share (`file://server/s/a`) keeps its authority.
let authority = if rest.starts_with('/') { "" } else { "//" };
format!("{CELL_SCHEME}:{authority}{rest}#{}", utf8_percent_encode(cell, FRAGMENT))
}
pub(crate) fn is_cell_uri(uri: &str) -> bool {
uri.strip_prefix(CELL_SCHEME).is_some_and(|rest| rest.starts_with(':'))
}
fn parse_uri(s: &str) -> Option<lsp::Uri> {
lsp::Uri::from_str(s).inspect_err(|e| tracing::debug!("not a valid URI {s}: {e}")).ok()
}
impl NotebookState {
fn opened_on(&self, server: ServerId, generation: u64) -> bool {
self.opened.contains(&(server, generation))
}
/// The servers it is open on whose instance is still the one it was opened on.
fn open_targets(&self) -> Vec<(Arc<Server>, Running)> {
self.servers
.iter()
.filter_map(|s| {
let run = s.running()?;
self.opened_on(s.id, run.generation).then(|| (s.clone(), run))
})
.collect()
}
}
impl LspManager {
/// Opens the notebook at `path` on its language's servers that sync notebooks, or brings
/// them up to date with `cells` (its code cells, in order).
pub fn sync_notebook(&self, path: &Path, language: &LanguageId, cells: &[NotebookCell]) {
let mut notebooks = self.notebooks.lock();
let mut changed = Vec::new();
let start = match notebooks.get_mut(path) {
None => {
let Some(notebook_uri) = parse_uri(&uri::path_to_uri_string(path)) else { return };
let servers = self.servers_for(language, path);
let mut state = NotebookState {
uri: notebook_uri,
language: language.clone(),
version: 1,
cells: Vec::new(),
servers: servers.clone(),
opened: Vec::new(),
};
self.set_cells(path, &mut state, cells);
self.open_notebook(&mut state, None);
notebooks.insert(path.to_owned(), state);
servers
}
Some(state) => {
let same_cells =
state.cells.len() == cells.len() && state.cells.iter().zip(cells).all(|((id, _), c)| *id == c.id);
if same_cells {
changed = self.change_cells(state, cells);
} else {
self.close_notebook_on_servers(state);
state.version += 1;
self.set_cells(path, state, cells);
self.open_notebook(state, None);
}
Vec::new()
}
};
drop(notebooks);
for server in &start {
self.ensure_started(server);
}
for doc in changed {
self.schedule_pull(doc, None, crate::pull::AFTER_EDIT);
}
}
/// Closes the notebook at `path` on its servers and forgets its cells.
pub fn close_notebook(&self, path: &Path) {
let Some(mut state) = self.notebooks.lock().remove(path) else { return };
self.close_notebook_on_servers(&mut state);
let removed: Vec<DocId> = state.cells.iter().map(|(_, doc)| *doc).collect();
self.remove_cells(&state.servers, &removed);
}
/// The documents of the notebook's code cells, in notebook order; empty when it isn't open.
pub fn notebook_cells(&self, path: &Path) -> Vec<NotebookCellDoc> {
let notebooks = self.notebooks.lock();
let Some(state) = notebooks.get(path) else { return Vec::new() };
let docs = self.docs.lock();
state
.cells
.iter()
.filter_map(|(cell, doc)| {
Some(NotebookCellDoc { cell: cell.clone(), doc: *doc, uri: docs.get(doc)?.uri.as_str().to_owned() })
})
.collect()
}
/// Opens the notebooks of `server`'s language on its instance that just became ready.
pub(crate) fn reopen_notebooks(&self, server: &Arc<Server>, run: &Running) {
let mut pull = Vec::new();
{
let mut notebooks = self.notebooks.lock();
for state in notebooks.values_mut() {
if state.servers.iter().any(|s| s.id == server.id) && !state.opened_on(server.id, run.generation) {
self.open_notebook(state, Some(server));
pull.extend(state.cells.iter().map(|(_, doc)| *doc));
}
}
}
for doc in pull {
self.schedule_pull(doc, Some(server), std::time::Duration::ZERO);
}
}
/// Adds servers of the notebooks' languages that were not wanted when they opened (a
/// fallback for a missing server, one enabled in settings) and returns them to be started.
pub(crate) fn attach_notebook_servers(&self) -> Vec<Arc<Server>> {
let mut start = Vec::new();
let mut notebooks = self.notebooks.lock();
for (path, state) in notebooks.iter_mut() {
let added: Vec<Arc<Server>> = self
.servers_for(&state.language, path)
.into_iter()
.filter(|s| !state.servers.iter().any(|known| known.id == s.id))
.collect();
if added.is_empty() {
continue;
}
{
let mut docs = self.docs.lock();
for (_, doc) in &state.cells {
if let Some(d) = docs.get_mut(doc) {
d.servers.extend(added.iter().map(|s| Attached { server: s.clone(), opened_gen: None }));
}
}
}
state.servers.extend(added.iter().cloned());
for server in &added {
self.open_notebook(state, Some(server));
}
start.extend(added);
}
start
}
/// Replaces the cell documents with `cells`: kept cells keep their [`DocId`].
fn set_cells(&self, path: &Path, state: &mut NotebookState, cells: &[NotebookCell]) {
let mut docs = self.docs.lock();
let previous = std::mem::take(&mut state.cells);
for cell in cells {
let kept = previous.iter().find(|(id, _)| *id == cell.id).map(|(_, doc)| *doc);
if let Some(doc) = kept
&& let Some(d) = docs.get_mut(&doc)
{
if d.text.as_str() != cell.source {
Arc::make_mut(&mut d.text).set(&cell.source);
}
d.version += 1;
state.cells.push((cell.id.clone(), doc));
continue;
}
let uri_string = cell_uri(path, &cell.id);
let Some(cell_uri) = parse_uri(&uri_string) else { continue };
let doc = DocId::next();
let key = PathBuf::from(&uri_string);
docs.insert(
doc,
DocState {
canonical: key.clone(),
path: key,
uri: cell_uri,
language: state.language.clone(),
version: 1,
text: Arc::new(DocText::new(&cell.source)),
servers: state.servers.iter().map(|s| Attached { server: s.clone(), opened_gen: None }).collect(),
compose_detected: false,
notebook: Some(path.to_owned()),
},
);
state.cells.push((cell.id.clone(), doc));
}
drop(docs);
let removed: Vec<DocId> = previous
.into_iter()
.filter(|(_, doc)| !state.cells.iter().any(|(_, d)| d == doc))
.map(|(_, d)| d)
.collect();
self.remove_cells(&state.servers, &removed);
}
/// Sends the cells whose text differs from `cells` as one change; returns their documents.
fn change_cells(&self, state: &mut NotebookState, cells: &[NotebookCell]) -> Vec<DocId> {
let mut docs = self.docs.lock();
let mut changed = Vec::new();
let mut content = Vec::new();
for ((_, doc), cell) in state.cells.iter().zip(cells) {
let Some(d) = docs.get_mut(doc) else { continue };
if d.text.as_str() == cell.source {
continue;
}
Arc::make_mut(&mut d.text).set(&cell.source);
d.version += 1;
content.push(json!({
"document": { "uri": d.uri, "version": d.version },
"changes": [{ "text": cell.source }],
}));
changed.push(*doc);
}
drop(docs);
if content.is_empty() {
return changed;
}
state.version += 1;
let params = json!({
"notebookDocument": { "uri": state.uri, "version": state.version },
"change": { "cells": { "textContent": content } },
});
for (server, run) in state.open_targets() {
if let Err(e) = run.peer.notify("notebookDocument/didChange", params.clone()) {
tracing::debug!("{}: notebookDocument/didChange: {e}", server.name);
}
}
changed
}
/// Opens the notebook on its running servers that sync notebooks (`only`: on that one).
fn open_notebook(&self, state: &mut NotebookState, only: Option<&Arc<Server>>) {
let mut docs = self.docs.lock();
let mut cells = Vec::new();
let mut texts = Vec::new();
for (_, doc) in &state.cells {
let Some(d) = docs.get(doc) else { continue };
cells.push(json!({ "kind": CODE_CELL, "document": d.uri }));
texts.push(json!({
"uri": d.uri,
"languageId": d.language.as_str(),
"version": d.version,
"text": d.text.as_str(),
}));
}
let params = json!({
"notebookDocument": { "uri": state.uri, "notebookType": NOTEBOOK_TYPE, "version": state.version, "cells": cells },
"cellTextDocuments": texts,
});
let servers: Vec<Arc<Server>> =
state.servers.iter().filter(|s| only.is_none_or(|o| o.id == s.id)).cloned().collect();
for server in servers {
let Some(run) = server.running() else { continue };
if state.opened_on(server.id, run.generation) || !run.caps.read().syncs_notebooks(state.language.as_str()) {
continue;
}
if let Err(e) = run.peer.notify("notebookDocument/didOpen", params.clone()) {
tracing::debug!("{}: notebookDocument/didOpen: {e}", server.name);
continue;
}
state.opened.retain(|(id, _)| *id != server.id);
state.opened.push((server.id, run.generation));
for (_, doc) in &state.cells {
if let Some(a) = docs.get_mut(doc).and_then(|d| d.servers.iter_mut().find(|a| a.server.id == server.id))
{
a.opened_gen = Some(run.generation);
}
}
}
}
fn close_notebook_on_servers(&self, state: &mut NotebookState) {
let cells: Vec<Value> = {
let docs = self.docs.lock();
state.cells.iter().filter_map(|(_, doc)| docs.get(doc)).map(|d| json!({ "uri": d.uri })).collect()
};
let params = json!({ "notebookDocument": { "uri": state.uri }, "cellTextDocuments": cells });
for (server, run) in state.open_targets() {
if let Err(e) = run.peer.notify("notebookDocument/didClose", params.clone()) {
tracing::debug!("{}: notebookDocument/didClose: {e}", server.name);
}
}
state.opened.clear();
let mut docs = self.docs.lock();
for (_, doc) in &state.cells {
if let Some(d) = docs.get_mut(doc) {
d.servers.iter_mut().for_each(|a| a.opened_gen = None);
}
}
}
/// Forgets cell documents and clears what their servers reported about them: a server need
/// not clear the diagnostics of a cell that left the notebook.
fn remove_cells(&self, servers: &[Arc<Server>], removed: &[DocId]) {
let gone: Vec<DocState> = {
let mut docs = self.docs.lock();
removed.iter().filter_map(|doc| docs.remove(doc)).collect()
};
for d in gone {
for server in servers {
let run = server.running();
self.forget_pulled(server, &d.path, run.as_ref());
let enc = run.as_ref().map(|r| r.caps.read().encoding).unwrap_or_default();
let shown = server.set_diagnostics(d.path.clone(), Vec::new());
self.emit_diagnostics(server, d.path.clone(), shown, enc);
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn cell_uris_name_the_notebook_file_and_the_cell() {
let root = if cfg!(windows) { PathBuf::from(r"C:\work\my nb") } else { PathBuf::from("/work/my nb") };
let expected = if cfg!(windows) {
"vscode-notebook-cell:/C:/work/my%20nb/a.ipynb#f3-x_1%2E"
} else {
"vscode-notebook-cell:/work/my%20nb/a.ipynb#f3-x_1%2E"
};
let uri = cell_uri(&root.join("a.ipynb"), "f3-x_1.");
assert_eq!(uri, expected);
assert!(is_cell_uri(&uri) && parse_uri(&uri).is_some());
assert!(!is_cell_uri("file:///a.ipynb") && !is_cell_uri("vscode-notebook-cellx:/a"));
}
}
+4 -146
View File
@@ -687,8 +687,6 @@ async fn java_classpath_waits_for_a_config_restart_that_is_reaping_startup() {
assert!(result.unwrap_err().message.contains("restart configuration error"));
} else {
assert!(result.is_ok(), "a configuration restart is transient: {result:?}");
// The module folder resolves to the instance serving its files, not a second one.
assert_eq!(h.mgr.servers().len(), 1);
assert_eq!(state.launches.load(Ordering::SeqCst), 2);
assert_eq!(h.mgr.servers()[0].id != old, shared);
}
@@ -1918,17 +1916,16 @@ async fn replacing_progress_keeps_cancellation_reports_and_indexing_on_the_new_t
for _ in 0..2 {
peer.notify("$/progress", json!({ "token": 7, "value": { "kind": "end" } })).unwrap();
}
h.wait_status("mock", ServerStatus::Ready).await;
let refs = tokio::time::timeout(Duration::from_secs(5), search).await.unwrap().unwrap().unwrap();
assert_eq!(refs.incomplete, None);
// A later task is a barrier for both end notifications. It starts only after the search
// finished: one that is still running would see the barrier as indexing.
// A later task is a barrier for both end notifications.
peer.notify("$/progress", json!({ "token": "barrier", "value": { "kind": "begin", "title": "Barrier" } })).unwrap();
peer.notify("$/progress", json!({ "token": "barrier", "value": { "kind": "end" } })).unwrap();
eventually(|| {
h.events.lock().iter().any(|e| matches!(e, IdeEvent::Task { title, done: true, .. } if title == "Barrier"))
})
.await;
h.wait_status("mock", ServerStatus::Ready).await;
let refs = tokio::time::timeout(Duration::from_secs(5), search).await.unwrap().unwrap().unwrap();
assert_eq!(refs.incomplete, None);
assert!(!ide_core::task::cancel(new));
let events = h.events.lock();
for id in [old, new] {
@@ -6784,142 +6781,3 @@ async fn gomod_files_are_served_by_gopls() {
assert_eq!(items[0].message, "missing requirement");
mgr.shutdown_all().await;
}
// ------------------------------------------------------------------ notebooks
fn notebook_caps() -> Value {
let mut caps = incremental_caps();
caps["notebookDocumentSync"] = json!({
"notebookSelector": [{
"notebook": { "scheme": "file", "notebookType": "jupyter-notebook" },
"cells": [{ "language": "python" }]
}]
});
caps
}
fn cells(cells: &[(&str, &str)]) -> Vec<ide_lsp::NotebookCell> {
cells.iter().map(|(id, source)| ide_lsp::NotebookCell { id: (*id).into(), source: (*source).into() }).collect()
}
fn cell_texts(open: &Value) -> Vec<(String, String)> {
open["cellTextDocuments"]
.as_array()
.unwrap()
.iter()
.map(|c| (c["uri"].as_str().unwrap().to_owned(), c["text"].as_str().unwrap().to_owned()))
.collect()
}
#[tokio::test(flavor = "multi_thread")]
async fn notebook_cells_are_documents_of_the_servers_that_sync_notebooks() {
let h = harness_for("python", "py", &[("nb", notebook_caps()), ("plain", incremental_caps())], &[]);
let path = h.root.join("analysis.ipynb");
let python = LanguageId::new("python");
h.mgr.sync_notebook(&path, &python, &cells(&[("a", "import os\n"), ("b", "os.pa")]));
h.wait_status("nb", ServerStatus::Ready).await;
h.wait_status("plain", ServerStatus::Ready).await;
let (nb, plain) = (h.launcher.state("nb"), h.launcher.state("plain"));
eventually(|| !nb.notified("notebookDocument/didOpen").is_empty()).await;
let docs = h.mgr.notebook_cells(&path);
assert_eq!(docs.iter().map(|d| d.cell.as_str()).collect::<Vec<_>>(), ["a", "b"]);
assert_eq!(docs[0].uri, ide_lsp::cell_uri(&path, "a"));
let open = &nb.notified("notebookDocument/didOpen")[0];
assert_eq!(open["notebookDocument"]["uri"], uri_of(&path));
assert_eq!(open["notebookDocument"]["notebookType"], "jupyter-notebook");
assert_eq!(open["notebookDocument"]["cells"][1], json!({ "kind": 2, "document": docs[1].uri }));
assert_eq!(
cell_texts(open),
[(docs[0].uri.clone(), "import os\n".to_owned()), (docs[1].uri.clone(), "os.pa".to_owned())]
);
assert!(plain.notified("notebookDocument/didOpen").is_empty() && plain.notified("textDocument/didOpen").is_empty());
// Requests address a cell like a file, and go only to the server it is open on.
nb.respond("textDocument/completion", json!({ "isIncomplete": false, "items": [{ "label": "path" }] }));
plain.respond("textDocument/completion", json!({ "isIncomplete": false, "items": [{ "label": "wrong" }] }));
let list = h.mgr.completion(docs[1].doc, Pos::new(0, 5), None, Default::default()).await.unwrap();
assert_eq!(list.items.iter().map(|i| i.label.as_str()).collect::<Vec<_>>(), ["path"]);
assert_eq!(nb.requested("textDocument/completion")[0]["textDocument"]["uri"], docs[1].uri);
assert!(plain.requested("textDocument/completion").is_empty());
// Its diagnostics are reported under the cell's URI.
nb.peer()
.notify(
"textDocument/publishDiagnostics",
json!({ "uri": docs[1].uri, "diagnostics": [{
"range": {"start":{"line":0,"character":3},"end":{"line":0,"character":5}}, "message": "no attribute pa"
}] }),
)
.unwrap();
let cell_path = PathBuf::from(&docs[1].uri);
eventually(|| {
h.events
.lock()
.iter()
.any(|e| matches!(e, IdeEvent::Diagnostics { path, items, .. } if path.0 == cell_path && items.len() == 1))
})
.await;
h.mgr.close_notebook(&path);
eventually(|| !nb.notified("notebookDocument/didClose").is_empty()).await;
assert_eq!(nb.notified("notebookDocument/didClose")[0]["cellTextDocuments"].as_array().unwrap().len(), 2);
assert!(h.mgr.notebook_cells(&path).is_empty());
assert!(h.mgr.completion(docs[1].doc, Pos::new(0, 5), None, Default::default()).await.is_err());
let last = h
.events
.lock()
.iter()
.filter_map(|e| match e {
IdeEvent::Diagnostics { path, items, source } if path.0 == cell_path && source == "nb" => Some(items.len()),
_ => None,
})
.next_back();
assert_eq!(last, Some(0), "a closed notebook's diagnostics are cleared");
h.mgr.shutdown_all().await;
}
#[tokio::test(flavor = "multi_thread")]
async fn notebook_edits_change_cell_text_and_reorders_reopen_it() {
let h = harness_for("python", "py", &[("nb", notebook_caps())], &[]);
let path = h.root.join("n.ipynb");
let python = LanguageId::new("python");
h.mgr.sync_notebook(&path, &python, &cells(&[("a", "x = 1\n"), ("b", "print(x)\n")]));
h.wait_status("nb", ServerStatus::Ready).await;
let nb = h.launcher.state("nb");
eventually(|| nb.notified("notebookDocument/didOpen").len() == 1).await;
let before = h.mgr.notebook_cells(&path);
// Typing in one cell sends only that cell's text.
h.mgr.sync_notebook(&path, &python, &cells(&[("a", "x = 1\n"), ("b", "print(x + 1)\n")]));
h.mgr.sync_notebook(&path, &python, &cells(&[("a", "x = 1\n"), ("b", "print(x + 1)\n")]));
eventually(|| !nb.notified("notebookDocument/didChange").is_empty()).await;
let changes = nb.notified("notebookDocument/didChange");
assert_eq!(changes.len(), 1, "an unchanged notebook sends nothing");
let content = &changes[0]["change"]["cells"]["textContent"];
assert_eq!(content.as_array().unwrap().len(), 1);
assert_eq!(content[0]["document"]["uri"], before[1].uri);
assert_eq!(content[0]["changes"], json!([{ "text": "print(x + 1)\n" }]));
assert_eq!(changes[0]["notebookDocument"]["version"], 2);
// Inserting and moving cells reopens the notebook; kept cells keep their documents.
h.mgr.sync_notebook(&path, &python, &cells(&[("c", "import math\n"), ("b", "print(x + 1)\n"), ("a", "x = 1\n")]));
eventually(|| nb.notified("notebookDocument/didOpen").len() == 2).await;
assert_eq!(nb.notified("notebookDocument/didClose").len(), 1);
let after = h.mgr.notebook_cells(&path);
assert_eq!(after.iter().map(|d| d.cell.as_str()).collect::<Vec<_>>(), ["c", "b", "a"]);
assert_eq!((after[1].doc, after[2].doc), (before[1].doc, before[0].doc));
let reopened = &nb.notified("notebookDocument/didOpen")[1];
assert_eq!(
cell_texts(reopened).into_iter().map(|(_, t)| t).collect::<Vec<_>>(),
["import math\n", "print(x + 1)\n", "x = 1\n"]
);
// A restarted server gets the notebook again.
let launches = nb.launches.load(Ordering::SeqCst);
let server = h.mgr.servers()[0].id;
h.mgr.restart(server).await.unwrap();
eventually(|| nb.launches.load(Ordering::SeqCst) > launches && nb.notified("notebookDocument/didOpen").len() == 3)
.await;
h.mgr.shutdown_all().await;
}
-96
View File
@@ -920,99 +920,3 @@ async fn angular_language_server_end_to_end() {
}
tokio::time::timeout(Duration::from_secs(10), mgr.shutdown_all()).await.unwrap();
}
/// A notebook's code cells on basedpyright and Ruff (LSP 3.17 notebook sync): names flow from
/// earlier cells, IPython magics parse, and locations and diagnostics name the cells.
#[tokio::test(flavor = "multi_thread")]
#[ignore = "requires ruff and basedpyright on PATH"]
async fn basedpyright_and_ruff_serve_notebook_cells() {
use ide_lsp::{NotebookCell, cell_uri};
let dir = tempfile::tempdir().unwrap();
let root = dir.path().canonicalize().unwrap();
let root = dunce::simplified(&root).to_path_buf();
std::fs::write(root.join("pyproject.toml"), "[tool.ruff.lint]\nselect = [\"F\"]\n").unwrap();
let path = root.join("analysis.ipynb");
std::fs::write(&path, "{}").unwrap();
let mut python = LanguageRegistry::load(None).unwrap().get(&LanguageId::new("python")).unwrap();
python.language_servers.retain(|server| matches!(server.name.as_str(), "basedpyright" | "ruff"));
let (mgr, events) = manager(&root, LanguageRegistry::from_configs(vec![python]));
let cell = |id: &str, source: &str| NotebookCell { id: id.into(), source: source.into() };
let first = "import os\nimport sys\n\ndef area(r: float) -> float:\n return 3.14 * r * r\n";
let second = "%matplotlib inline\nsize: int = area(2)\nos.pa";
mgr.sync_notebook(&path, &LanguageId::new("python"), &[cell("first", first), cell("second", second)]);
for name in ["basedpyright", "ruff"] {
retry(name, Duration::from_secs(60), || {
let status = last_status(&events, name);
if let Some((ServerStatus::Crashed | ServerStatus::Missing | ServerStatus::Misconfigured, msg)) = &status {
panic!("{name} failed: {msg:?}");
}
async move { matches!(status, Some((ServerStatus::Ready | ServerStatus::Busy, _))).then_some(()) }
})
.await;
}
let docs = mgr.notebook_cells(&path);
let b = docs[1].doc;
let (a_uri, b_uri) = (cell_uri(&path, "first"), cell_uri(&path, "second"));
assert_eq!((docs[0].uri.as_str(), docs[1].uri.as_str()), (a_uri.as_str(), b_uri.as_str()));
// `os`, imported in the first cell, completes in the second.
let labels = retry("completion in a cell", Duration::from_secs(30), || {
let mgr = mgr.clone();
async move {
let list = mgr.completion(b, Pos::new(2, 5), Some(".".into()), Default::default()).await.ok()?;
let labels: Vec<String> = list.items.into_iter().map(|i| i.label).collect();
labels.iter().any(|l| l == "path").then_some(labels)
}
})
.await;
assert!(labels.iter().any(|l| l == "path"), "{labels:?}");
let hover = mgr.hover(b, Pos::new(1, 13), Default::default()).await.unwrap().expect("hover on `area`");
assert!(
hover.contents.value.contains("area") && hover.contents.value.contains("float"),
"{}",
hover.contents.value
);
let defs = mgr.goto(b, Pos::new(1, 13), GotoKind::Definition).await.unwrap();
assert_eq!(defs.len(), 1, "{defs:?}");
assert_eq!((defs[0].path.to_path_buf(), defs[0].range.start.line), (std::path::PathBuf::from(&a_uri), 3));
let diagnostics = |source: &'static str, uri: String| {
let events = events.clone();
move || {
let items = events.lock().iter().rev().find_map(|e| match e {
IdeEvent::Diagnostics { path, source: s, items } if s == source && path.0 == Path::new(&uri) => {
Some(items.clone())
}
_ => None,
});
async move { items.filter(|items| !items.is_empty()) }
}
};
// `float` is not assignable to `int`; the magic on line 0 is no error.
let typed =
retry("basedpyright cell diagnostics", Duration::from_secs(30), diagnostics("basedpyright", b_uri)).await;
assert!(typed.iter().any(|d| d.range.start.line == 1), "{typed:?}");
assert!(typed.iter().all(|d| d.range.start.line != 0), "{typed:?}");
// F401: `sys` is unused.
let linted = retry("ruff cell diagnostics", Duration::from_secs(30), diagnostics("ruff", a_uri)).await;
assert!(linted.iter().any(|d| d.code.as_deref() == Some("F401") && d.range.start.line == 1), "{linted:?}");
// An edit reaches both servers: the error is fixed.
let fixed = "%matplotlib inline\nsize: float = area(2)\nos.path";
mgr.sync_notebook(&path, &LanguageId::new("python"), &[cell("first", first), cell("second", fixed)]);
retry("diagnostics after the fix", Duration::from_secs(30), || {
let uri = cell_uri(&path, "second");
let clear = events.lock().iter().rev().find_map(|e| match e {
IdeEvent::Diagnostics { path, source, items } if source == "basedpyright" && path.0 == Path::new(&uri) => {
Some(items.is_empty())
}
_ => None,
});
async move { (clear == Some(true)).then_some(()) }
})
.await;
mgr.close_notebook(&path);
tokio::time::timeout(Duration::from_secs(10), mgr.shutdown_all()).await.unwrap();
}
+6 -86
View File
@@ -55,7 +55,7 @@ use views_bindings::exports::openidea::plugin::view_provider as wit_view;
use views_bindings::openidea::plugin::view_types as wit_vt;
/// The API version of the world above, `major.minor` of [`host_api`].
pub const API_VERSION: &str = "1.6";
pub const API_VERSION: &str = "1.5";
const EPOCH_TICK: Duration = Duration::from_millis(10);
pub(crate) const MAX_TEXT: usize = 1 << 20;
pub(crate) const MAX_TITLE: usize = 256;
@@ -432,27 +432,6 @@ pub trait HostApi: Send + Sync {
fn publish_diagnostics(&self, path: &Path, items: Vec<ide_core::Diagnostic>) -> Result<(), HostError>;
/// Performs `request` (its host is granted), filling in secret placeholders.
fn fetch(&self, request: HttpRequest, cancel: &CancellationToken) -> Result<HttpResponse, HostError>;
fn websocket_connect(
&self,
root: &Path,
request: crate::WebSocketRequest,
timeout: Duration,
) -> Result<u64, HostError> {
let _ = (root, request, timeout);
Err(HostError::Invalid("background WebSockets are unavailable on this host".into()))
}
fn websocket_send(&self, connection: u64, bytes: Vec<u8>) -> Result<(), HostError> {
let _ = (connection, bytes);
Err(HostError::Invalid("background WebSockets are unavailable on this host".into()))
}
fn websocket_drain(&self, connection: u64, limit: usize) -> Result<crate::WebSocketBatch, HostError> {
let _ = (connection, limit);
Err(HostError::Invalid("background WebSockets are unavailable on this host".into()))
}
fn websocket_close(&self, connection: u64) {
let _ = connection;
}
/// The `net:ask` answer for a request to HTTPS host `host` of this plugin
/// (`subject`): `Some(true)` allowed and remembered (a host the user
/// already allowed connects without asking again, also in the background),
@@ -627,10 +606,6 @@ impl PluginEngine {
let mut linker = Linker::new(&engine);
wasmtime_wasi::p2::add_to_linker_sync(&mut linker).map_err(|e| e.to_string())?;
bindings::Plugin::add_to_linker::<_, HasSelf<_>>(&mut linker, |s| s).map_err(|e| e.to_string())?;
// Cranelift caches fuel in a JIT local across `new_epoch` yields and only
// reloads `VMStoreContext::fuel_consumed` across host import calls, so
// `tasks` host calls refill `StoreContextMut::set_fuel` directly on return.
bind_task_fuel_refill(&mut linker).map_err(|e| e.to_string())?;
let weak = engine.weak();
std::thread::Builder::new()
@@ -824,7 +799,7 @@ impl CallCancel {
self.token.lock().clone()
}
pub(crate) fn is_set(&self) -> bool {
fn is_set(&self) -> bool {
self.flag.load(Ordering::Acquire)
}
@@ -860,10 +835,6 @@ pub struct StoreData {
/// Tasks begun by the running call and not ended yet.
pub(crate) open_tasks: Vec<u64>,
pub(crate) last_report: Option<Instant>,
/// One-shot unwind window after a blocking host I/O call times out.
pub(crate) host_io_grace_used: bool,
/// Fuel budget active for the running call (scales when a background task begins).
pub(crate) active_fuel: u64,
}
impl WasiView for StoreData {
@@ -1016,14 +987,6 @@ const MAX_BATCH: usize = 1000;
fn event_to_wit(e: WorkbenchEvent) -> wit_events::Event {
match e {
WorkbenchEvent::DocumentChanged(value) => wit_events::Event::Other(wit_types::Extension {
kind: "document-changed".into(),
data: serde_json::to_string(&value).expect("document change serializes"),
}),
WorkbenchEvent::WebsocketReady(value) => wit_events::Event::Other(wit_types::Extension {
kind: "websocket-ready".into(),
data: serde_json::to_string(&value).expect("WebSocket ready event serializes"),
}),
WorkbenchEvent::RunExited(value) => wit_events::Event::Other(wit_types::Extension {
kind: "run-exited".into(),
data: serde_json::to_string(&value).unwrap(),
@@ -1104,8 +1067,6 @@ impl Runtime {
action_prompt_client: None,
open_tasks: Vec::new(),
last_report: None,
host_io_grace_used: false,
active_fuel: setup.limits.fuel,
};
let mut store = Store::new(&engine.engine, data);
store.limiter(|d| &mut d.store_limits);
@@ -1122,7 +1083,7 @@ impl Runtime {
});
arm(&mut store, &setup.limits, setup.limits.timeout).map_err(|e| e.to_string())?;
let api_hint = |e: wasmtime::Error| {
let msg = describe_trap(&e, &setup.limits, setup.limits.fuel);
let msg = describe_trap(&e, &setup.limits);
if msg.contains("openidea:plugin/") { link_error(&msg) } else { msg }
};
let pre = engine.linker.instantiate_pre(component).map_err(api_hint)?;
@@ -1907,11 +1868,10 @@ impl Runtime {
if !stderr.trim().is_empty() {
tracing::info!(plugin = %data.plugin_id, "stderr: {}", stderr.trim_end());
}
let active_fuel = data.active_fuel;
match result {
Ok(r) => Ok(r),
Err(e) => {
let mut msg = describe_trap(&e, &self.limits, active_fuel);
let mut msg = describe_trap(&e, &self.limits);
// Typically the panic message: "panicked at src/lib.rs:1:2:\nboom".
let detail: Vec<&str> =
stderr.lines().filter(|l| !l.trim().is_empty() && !l.starts_with("note:")).collect();
@@ -1963,55 +1923,15 @@ fn arm(store: &mut Store<StoreData>, limits: &Limits, timeout: Duration) -> wasm
data.deadline = now + timeout;
data.timeout = timeout;
data.last_report = None;
data.host_io_grace_used = false;
data.active_fuel = limits.fuel;
data.cancel.reset();
// The callback above runs every tick and decides.
store.set_epoch_deadline(1);
Ok(())
}
fn bind_task_fuel_refill(linker: &mut Linker<StoreData>) -> wasmtime::Result<()> {
use crate::wit_host::TaskRes;
use crate::wit_host::bindings::openidea::plugin::tasks::{Host, HostTask};
use wasmtime::StoreContextMut;
use wasmtime::component::Resource;
linker.allow_shadowing(true);
let tasks_iface = format!("openidea:plugin/tasks@{}", ide_extension::ENGINE_VERSION);
let mut inst = linker.instance(&tasks_iface)?;
inst.func_wrap("begin", |mut caller: StoreContextMut<'_, StoreData>, (title, cancellable): (String, bool)| {
let task = Host::begin(caller.data_mut(), title, cancellable)?;
let fuel = caller.data().active_fuel;
caller.set_fuel(fuel)?;
Ok((task,))
})?;
inst.func_wrap(
"[method]task.report",
|mut caller: StoreContextMut<'_, StoreData>,
(this, done, total, message): (Resource<TaskRes>, u32, Option<u32>, Option<String>)| {
HostTask::report(caller.data_mut(), this, done, total, message)?;
let fuel = caller.data().active_fuel;
caller.set_fuel(fuel)?;
Ok(())
},
)?;
inst.func_wrap(
"[method]task.is-cancelled",
|mut caller: StoreContextMut<'_, StoreData>, (this,): (Resource<TaskRes>,)| {
let cancelled = HostTask::is_cancelled(caller.data_mut(), this)?;
let fuel = caller.data().active_fuel;
caller.set_fuel(fuel)?;
Ok((cancelled,))
},
)?;
linker.allow_shadowing(false);
Ok(())
}
fn describe_trap(e: &wasmtime::Error, limits: &Limits, active_fuel: u64) -> String {
fn describe_trap(e: &wasmtime::Error, limits: &Limits) -> String {
match e.downcast_ref::<wasmtime::Trap>() {
Some(wasmtime::Trap::OutOfFuel) => format!("exceeded its CPU budget ({active_fuel} fuel per call)"),
Some(wasmtime::Trap::OutOfFuel) => format!("exceeded its CPU budget ({} fuel per call)", limits.fuel),
Some(wasmtime::Trap::Interrupt) => CANCELLED.to_owned(),
Some(wasmtime::Trap::UnreachableCodeReached) => "crashed (panic / unreachable)".to_owned(),
Some(trap) => format!("trapped: {trap}"),
+1 -73
View File
@@ -104,8 +104,6 @@ pub enum Prompt {
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize, Type)]
#[serde(rename_all = "kebab-case")]
pub enum EventKind {
DocumentChanged,
WebsocketReady,
FileClosed,
SettingsChanged,
InputModeChanged,
@@ -123,9 +121,7 @@ pub enum EventKind {
}
impl EventKind {
pub const ALL: [Self; 16] = [
Self::DocumentChanged,
Self::WebsocketReady,
pub const ALL: [Self; 14] = [
Self::FileClosed,
Self::SettingsChanged,
Self::InputModeChanged,
@@ -144,8 +140,6 @@ impl EventKind {
pub fn name(self) -> &'static str {
match self {
Self::DocumentChanged => "document-changed",
Self::WebsocketReady => "websocket-ready",
Self::FileClosed => "file-closed",
Self::SettingsChanged => "settings-changed",
Self::InputModeChanged => "input-mode-changed",
@@ -233,53 +227,10 @@ pub struct InputMode {
pub sub: Option<String>,
}
/// One UTF-16 replacement in a locally edited editor document.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
pub struct DocumentTextChange {
pub from: u32,
pub to: u32,
pub insert: String,
}
/// Local changes accepted by the editor's shared document model.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct DocumentChanged {
pub path: String,
pub base_version: u64,
pub version: u64,
pub changes: Vec<DocumentTextChange>,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct WebsocketReady {
pub plugin_id: String,
pub connection: u64,
}
/// A host-owned background WebSocket request.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct WebSocketRequest {
pub url: String,
pub headers: Vec<(String, String)>,
pub subprotocol: String,
}
/// Frames received from a host-owned WebSocket and its transport state.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct WebSocketBatch {
pub messages: Vec<Vec<u8>>,
pub closed: bool,
}
/// A workbench event delivered to plugins.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase", tag = "kind", content = "value")]
pub enum WorkbenchEvent {
DocumentChanged(DocumentChanged),
WebsocketReady(WebsocketReady),
RunExited(crate::RunExited),
TerminalExited(crate::TerminalExited),
ContentTabAction(crate::ContentTabAction),
@@ -300,8 +251,6 @@ pub enum WorkbenchEvent {
impl WorkbenchEvent {
pub fn kind(&self) -> EventKind {
match self {
Self::DocumentChanged(_) => EventKind::DocumentChanged,
Self::WebsocketReady(_) => EventKind::WebsocketReady,
Self::RunExited(_) => EventKind::RunExited,
Self::TerminalExited(_) => EventKind::TerminalExited,
Self::ContentTabAction(_) => EventKind::ContentTabAction,
@@ -527,18 +476,6 @@ pub fn net_capability(url: &str) -> Result<ide_extension::Capability, String> {
}
}
/// The grant a WebSocket URL requires: the same as its `https`/`http` counterpart.
pub fn websocket_capability(url: &str) -> Result<ide_extension::Capability, String> {
let mut parsed = url::Url::parse(url).map_err(|e| format!("invalid URL: {e}"))?;
let scheme = match parsed.scheme() {
"wss" => "https",
"ws" => "http",
other => return Err(format!("unsupported WebSocket scheme {other:?}")),
};
parsed.set_scheme(scheme).map_err(|()| "invalid WebSocket URL".to_owned())?;
net_capability(parsed.as_str())
}
#[cfg(test)]
mod tests {
#[test]
@@ -573,15 +510,6 @@ mod tests {
assert_eq!(net_host(url).unwrap(), host);
}
assert_ne!(net_host("https://аpi.example/").unwrap(), net_host("https://api.example/").unwrap());
assert_eq!(
websocket_capability("wss://Collab.example.com/socket?room=1").unwrap(),
ide_extension::Capability::Net("collab.example.com".into())
);
assert_eq!(
websocket_capability("ws://localhost:9000/").unwrap(),
ide_extension::Capability::Net("localhost".into())
);
assert!(websocket_capability("https://example.com/").is_err());
assert!(net_host("file:///etc/passwd").is_err());
assert!(net_host("example.com").is_err());
assert!(net_host("https://*.example.com/").is_err());
+8 -187
View File
@@ -26,7 +26,6 @@ pub(crate) mod bindings {
"openidea:plugin/tasks.task": super::TaskRes,
"openidea:plugin/process.child": super::ChildRes,
"openidea:plugin/ipc.connection": super::ConnectionRes,
"openidea:plugin/net.websocket": super::WebSocketRes,
"openidea:plugin/auth.loopback": super::LoopbackRes,
},
});
@@ -163,19 +162,6 @@ pub struct ChildRes {
host: std::sync::Arc<dyn crate::HostApi>,
}
/// A project-bound background WebSocket owned by this plugin instance.
/// Releasing the component resource closes the underlying transport.
pub struct WebSocketRes {
id: u64,
host: std::sync::Arc<dyn crate::HostApi>,
}
impl Drop for WebSocketRes {
fn drop(&mut self) {
self.host.websocket_close(self.id);
}
}
impl Drop for ChildRes {
fn drop(&mut self) {
self.host.release_process(self.id);
@@ -197,8 +183,6 @@ pub struct TaskRes {
const MAX_EDITS: usize = 10_000;
const MAX_DIAGNOSTICS: usize = 5_000;
const MAX_REQUEST_BODY: usize = 8 << 20;
/// Upper bound for a WebSocket handshake; the call's remaining time can make it shorter.
const WEBSOCKET_HANDSHAKE_TIMEOUT: Duration = Duration::from_secs(10);
const MAX_FIND: u32 = 10_000;
const MAX_STATUS_TEXT: usize = 128;
const MAX_PICK_ITEMS: usize = 5000;
@@ -838,52 +822,6 @@ impl workspace::HostProject for StoreData {
}
impl editor::Host for StoreData {
fn document_at(&mut self, project: Resource<ProjectRes>, path: String) -> Fallible<editor::Document> {
if let Err(error) = self.require(Capability::ProjectRead).and_then(|_| self.require(Capability::EditorRead)) {
return Ok(Err(error));
}
let root = self.table.get(&project)?.root.clone();
let path = match self.resolve_in_project(&root, &path) {
Ok(path) => path,
Err(error) => return Ok(Err(wit::Error::Invalid(error))),
};
Ok(match self.host.document(&path) {
Some((version, text)) => Ok(editor::Document { version, text }),
None => Err(wit::Error::NotFound(format!("{} is not open", path.display()))),
})
}
fn apply_edits_at(
&mut self,
project: Resource<ProjectRes>,
path: String,
version: u64,
edits: Vec<wit::TextEdit>,
) -> Fallible<u64> {
if let Err(error) = self.require(Capability::ProjectRead).and_then(|_| self.require(Capability::EditorWrite)) {
return Ok(Err(error));
}
if edits.len() > MAX_EDITS {
return Ok(Err(wit::Error::Invalid(format!("more than {MAX_EDITS} edits"))));
}
if edits.iter().map(|edit| edit.text.len()).sum::<usize>() > crate::runtime::MAX_TEXT * 8 {
return Ok(Err(wit::Error::Invalid("edits too large".into())));
}
let root = self.table.get(&project)?.root.clone();
let path = match self.resolve_in_project(&root, &path) {
Ok(path) => path,
Err(error) => return Ok(Err(wit::Error::Invalid(error))),
};
let edits = edits
.into_iter()
.map(|edit| ide_core::TextEdit {
range: ide_core::Range::new(pos(edit.range.start), pos(edit.range.end)),
new_text: edit.text,
})
.collect();
Ok(self.host.apply_edits(None, &path, version, edits).map_err(to_wit))
}
fn open_editors(&mut self) -> Fallible<Vec<editor::OpenEditor>> {
if let Err(e) = self.require(Capability::EditorRead).and_then(|_| self.require_window()) {
return Ok(Err(e));
@@ -919,7 +857,6 @@ impl editor::Host for StoreData {
fn core_range(r: wit::Range) -> ide_core::Range {
ide_core::Range::new(pos(r.start), pos(r.end))
}
impl StoreData {
/// A `project` resource's path: inside the resource's root, or — for a
/// project window group — inside any of its member roots (as the editor and
@@ -1259,11 +1196,6 @@ impl events::Host for StoreData {
{
return Ok(Err(to_wit(e)));
}
if kind == EventKind::DocumentChanged {
return Ok(Err(wit::Error::Invalid(
"document-changed is delivered only for paths named with subscribe-paths".into(),
)));
}
if let Some(cap) = kind.capability()
&& let Err(e) = self.require(cap)
{
@@ -1277,14 +1209,6 @@ impl events::Host for StoreData {
if let Err(e) = self.require(Capability::Events) {
return Ok(Err(e));
}
if kind == EventKind::DocumentChanged.name() {
if let Err(e) = self.host.require_api(&kind, 6) {
return Ok(Err(to_wit(e)));
}
if let Err(e) = self.require(Capability::ProjectRead).and_then(|_| self.require(Capability::EditorRead)) {
return Ok(Err(e));
}
}
Ok(self.host.subscribe_paths(&kind, globs).map_err(to_wit))
}
@@ -1343,11 +1267,6 @@ impl tasks::HostTask for StoreData {
}
}
/// Headroom reserved before `StoreData::deadline` so a timed-out host subprocess
/// or HTTP request can unwind as a recoverable `wit::Error` in guest code instead
/// of immediately tripping the Wasm epoch trap on re-entry.
const HOST_IO_UNWIND_GRACE: Duration = Duration::from_millis(250);
impl StoreData {
/// Lets the running call reach `from` plus the background budget (the
/// longer of it and what was left), so a task that keeps saying it is alive
@@ -1358,8 +1277,6 @@ impl StoreData {
self.deadline = extended;
self.timeout = self.limits.background;
}
let scale = (self.limits.background.as_millis() / self.limits.timeout.as_millis().max(1)).max(1) as u64;
self.active_fuel = self.limits.fuel.saturating_mul(scale);
}
/// What is left of the running call's wall-clock budget (host operations
@@ -1367,28 +1284,6 @@ impl StoreData {
fn remaining(&self) -> Duration {
self.deadline.saturating_duration_since(Instant::now())
}
/// Budget handed to a blocking host I/O call (`process.run`, `net.fetch`),
/// leaving room before `self.deadline` when possible so a timed-out I/O call
/// returns `Err` to the guest instead of trapping the Wasm instance.
fn io_timeout(&self) -> Duration {
let rem = self.remaining();
if !self.host_io_grace_used && rem > HOST_IO_UNWIND_GRACE * 2 { rem - HOST_IO_UNWIND_GRACE } else { rem }
}
/// Grants at most one short unwind window per call when a blocking host I/O
/// call failed at or near `self.deadline` without caller cancellation. An
/// error well before the deadline leaves the window for a later call.
fn grant_io_error_grace(&mut self) {
if self.host_io_grace_used || self.cancel.is_set() {
return;
}
let min_deadline = Instant::now() + HOST_IO_UNWIND_GRACE;
if min_deadline > self.deadline {
self.deadline = min_deadline;
self.host_io_grace_used = true;
}
}
}
impl diagnostics::Host for StoreData {
@@ -1476,15 +1371,13 @@ impl net::Host for StoreData {
url: req.url,
headers: req.headers.into_iter().map(|h| (h.name, h.value)).collect(),
body: req.body,
timeout: wanted.min(self.io_timeout()),
timeout: wanted.min(self.remaining()),
grants,
};
let token = self.cancel.token();
let outcome = self.host.fetch(request, &token);
if outcome.is_err() {
self.grant_io_error_grace();
}
Ok(outcome
Ok(self
.host
.fetch(request, &token)
.map(|r| net::Response {
status: r.status,
headers: r.headers.into_iter().map(|(name, value)| net::Header { name, value }).collect(),
@@ -1492,70 +1385,6 @@ impl net::Host for StoreData {
})
.map_err(to_wit))
}
fn websocket_connect(
&mut self,
project: Resource<ProjectRes>,
req: net::WebsocketRequest,
) -> Fallible<Resource<WebSocketRes>> {
let capability = match crate::types::websocket_capability(&req.url) {
Ok(capability) => capability,
Err(error) => return Ok(Err(wit::Error::Invalid(error))),
};
if let Err(error) = self.require(capability) {
return Ok(Err(error));
}
if req.headers.iter().any(|header| header.value.contains("{{secret:"))
&& let Err(error) = self.require(Capability::Secrets)
{
return Ok(Err(error));
}
let request = crate::WebSocketRequest {
url: req.url,
headers: req.headers.into_iter().map(|header| (header.name, header.value)).collect(),
subprotocol: req.subprotocol,
};
let root = self.table.get(&project)?.root.clone();
let timeout = WEBSOCKET_HANDSHAKE_TIMEOUT.min(self.io_timeout());
match self.host.websocket_connect(&root, request, timeout) {
Ok(id) => self.push(WebSocketRes { id, host: self.host.clone() }).map(Ok),
Err(error) => {
self.grant_io_error_grace();
Ok(Err(to_wit(error)))
}
}
}
}
impl net::HostWebsocket for StoreData {
fn id(&mut self, this: Resource<WebSocketRes>) -> wasmtime::Result<u64> {
Ok(self.table.get(&this)?.id)
}
fn send(&mut self, this: Resource<WebSocketRes>, bytes: Vec<u8>) -> Fallible<()> {
let id = self.table.get(&this)?.id;
Ok(self.host.websocket_send(id, bytes).map_err(to_wit))
}
fn drain(&mut self, this: Resource<WebSocketRes>, limit: u32) -> Fallible<net::WebsocketBatch> {
let id = self.table.get(&this)?.id;
Ok(self
.host
.websocket_drain(id, limit as usize)
.map(|batch| net::WebsocketBatch { messages: batch.messages, closed: batch.closed })
.map_err(to_wit))
}
fn close(&mut self, this: Resource<WebSocketRes>) -> Fallible<()> {
let id = self.table.get(&this)?.id;
self.host.websocket_close(id);
Ok(Ok(()))
}
fn drop(&mut self, this: Resource<WebSocketRes>) -> wasmtime::Result<()> {
self.table.delete(this)?;
Ok(())
}
}
impl secrets::Host for StoreData {
@@ -1637,12 +1466,8 @@ impl process::Host for StoreData {
};
let cwd = self.cwd_of(cwd)?;
let token = self.cancel.token();
let timeout = self.io_timeout();
let outcome = self.host.run_process(cmd, cwd.as_deref(), stdin, &token, timeout);
if outcome.is_err() {
self.grant_io_error_grace();
}
Ok(outcome.map(to_output).map_err(to_wit))
let timeout = self.remaining();
Ok(self.host.run_process(cmd, cwd.as_deref(), stdin, &token, timeout).map(to_output).map_err(to_wit))
}
fn run_command(
@@ -1657,12 +1482,8 @@ impl process::Host for StoreData {
};
let cwd = self.cwd_of(cwd)?;
let token = self.cancel.token();
let timeout = self.io_timeout();
let outcome = self.host.run_process(cmd, cwd.as_deref(), stdin, &token, timeout);
if outcome.is_err() {
self.grant_io_error_grace();
}
Ok(outcome.map(to_output).map_err(to_wit))
let timeout = self.remaining();
Ok(self.host.run_process(cmd, cwd.as_deref(), stdin, &token, timeout).map(to_output).map_err(to_wit))
}
fn start(&mut self, cmd: process::Command, cwd: Option<Resource<ProjectRes>>) -> Fallible<Resource<ChildRes>> {
+5 -5
View File
@@ -91,15 +91,15 @@ fn the_sdk_carries_the_hosts_wit() {
#[test]
fn the_version_check_runs_compatible_components_and_explains_the_rest() {
let host = v("1.6.0");
for ok in ["1.0.0", "1.3.0", "1.4.7", "1.5.0", "1.6.0", "1.0.0-rc.1"] {
let host = v("1.5.0");
for ok in ["1.0.0", "1.3.0", "1.4.7", "1.5.0", "1.0.0-rc.1"] {
assert_eq!(check_api(&v(ok), &host), Ok(()), "{ok}");
}
let newer = check_api(&v("1.7.0"), &host).unwrap_err();
assert!(newer.contains("1.7") && newer.contains("update OpenIDEA"), "{newer}");
let newer = check_api(&v("1.6.0"), &host).unwrap_err();
assert!(newer.contains("1.6") && newer.contains("update OpenIDEA"), "{newer}");
assert!(check_api(&v("2.0.0"), &host).unwrap_err().contains("update OpenIDEA"));
let older = check_api(&v("0.2.0"), &host).unwrap_err();
assert!(older.contains("rebuild it with the 1.6 SDK"), "{older}");
assert!(older.contains("rebuild it with the 1.5 SDK"), "{older}");
// 0.x: every minor is its own API.
assert_eq!(check_api(&v("0.2.0"), &v("0.2.5")), Ok(()));
assert!(check_api(&v("0.1.0"), &v("0.2.0")).is_err());
Binary file not shown.
@@ -1 +1 @@
005d9da29eb7d713a97ff872051e408f653030434d837835843298553c9e12e0
eb7d608bf1e7379620ef7d9ce5289dd9b927f137af2aafdd0861a84d60d677a2
Binary file not shown.
@@ -1 +1 @@
6359ef4271f38374889412bd926d9aa735f59a8ce047ffcb06c47976444196cf
195cb4025508e7af6e6fb63d81660c4a977327fa3d101a2a77e5f448f43518f8
Binary file not shown.
@@ -1 +1 @@
6f560f3d0d7737150d2dfa09c3b801bd3b4c3a52cef9893ed8fc045809fe33e1
49491ad2d57592cca2451b0abde5f88a3aedd3f5d36ceaf8259c3254f59c5023
Binary file not shown.
@@ -1 +1 @@
4024af2d55d7fe4e8c456efd7e0c497ade4b2c5959e9108ff218f4a669e920f4
b24383f6e05ec531ef8db368b79af710433bc3782438f5a9e3578ebb730a1ce2
Binary file not shown.
@@ -1 +1 @@
4ca977f53a18e9e8f869d34e027a15abece9e97b9331e23d4f71af721b104611
ef01f199e668063b2c444e4ccf169d5b5407e567dbe8ff583cfdfc111b2438cf
Binary file not shown.
@@ -1 +1 @@
3d12a00f81e0db1b0902c2ddebfd6454a056f15b97add80b22a0333722568dc8
a4dbe2a261e00809b64039ce9c89db681b186788db8a92b7456eca84b33c5945
Binary file not shown.
@@ -1 +1 @@
95d89e411216f50d0b34bee110c6bfc9b8fbcb2ffaae2e912c3609f7bf678b2c
3c9db6ffeaa9b782970c7a7a132878f862b0b5b18e9d11ad56649ee28317e3ca
Binary file not shown.
@@ -1 +1 @@
dd4b6c5153892a4f3facb3f23cb1d92da39a3977985b8abf4eaa9558decfa4a9
ac1dceb016b7963f44e876784ea62f776e90dd63fc6f660a5dad3dfbd7433b1c
Binary file not shown.
@@ -1 +1 @@
325bc15cd28933d9687bc3c316008f921f48a4a69d7528c6b367fa0ff22489b8
11dc1a74c0650b493ed6ec88b5d15e08421ab08c5f2320099dabd5df1bdc5096
Binary file not shown.
@@ -1 +1 @@
abe706392e04ef5a14f9feab88e0c0d6a9e96bee7de771b61deeb65e7d9e0a7c
94c391142af832a67c694968d307217aabd7c139ae29e3299930b56ce644e45b
Binary file not shown.
@@ -1 +1 @@
fb3677946021b30d226e2e3d422c352645c2065c93057c0a645246520860fc90
e2c1893a160a0f27af7eb9e689d703577335cce32f746713c2435ae70fb9753d
Binary file not shown.
@@ -1 +1 @@
a01852ead9004ac1558028da19a1ae96ccd035775484089fd39f14263082daf1
83a8f59e0bde75c5c8fc725aa0fe34954279b31b7237acc868470ccf545ca229
Binary file not shown.
@@ -1 +1 @@
bf00050b7bb9911680a94fd36aaeb83fc96d94030e8ceef51d1723c3950253f9
f9741cc8f7250a71311264613d636c5c844f1ae294362d4d331015ca73acbc57
Binary file not shown.
@@ -1 +1 @@
d00a07d9d51e7ca117cec59070d2812d4d1efb2730b60a6f171c8fd12903d055
a6f65bb0d13527d726cef88fa50a5db19d48517b1ae516db36e9756dc87e75bc
Binary file not shown.
@@ -1 +1 @@
0aa0ee56e8bacd4d218bafecdec7f4b6249381fad6e31f8b2ee89fd88b16d710
c9dbae18196b98eb4fad86fd5f36033c2b82a3087b4f51c4cba5a1362924d795
Binary file not shown.
@@ -1 +1 @@
e06955818969808e6695fba9178727239e343bc4542efbf56262dd407175fd8a
1086cdc27def1958e8ec818e197af67edc2fd252b198e0ac358b149e0f5abfe8
@@ -1 +1 @@
7af67673471d49f97aa2cc131b5deddfc78ee7fc203901eba12f4970a5b13f75
48fe38b954f91bf8adc01a4b7b41802040f663c16fde586bd5daa51be98ba51d
Binary file not shown.
@@ -1 +1 @@
e81d2454a61977ab69dc8b5c3c617627c559c4e340337a3d43309daeb9b1c12d
9ff2eaaa054ecdab56bf01220c8529da2b8d074b923cfb8b332d347767ba04f4
+3 -3
View File
@@ -5,9 +5,9 @@ wit_bindgen::generate!({
inline: "
package fixture:misbehave;
world misbehave {
include openidea:plugin/plugin-syntax@1.6.0;
export openidea:plugin/providers@1.6.0;
export openidea:plugin/file-actions@1.6.0;
include openidea:plugin/plugin-syntax@1.5.0;
export openidea:plugin/providers@1.5.0;
export openidea:plugin/file-actions@1.5.0;
}
",
path: "../../../../../wit",
Binary file not shown.
@@ -1 +1 @@
123ae68df6baa3b5983349ba05fb441ec67fddcb32cab566b0d02e443bb4800e
d72afe7db6283cdaecd7c673d1d26ff5af39f105733109ec73901ca37b00ea28
Binary file not shown.
@@ -1 +1 @@
7b26a4ec36b496f9a4f5e029ef557707961a281e31508863c989a5a5abd5e225
bccf0174932e661c40f05d779bf094e7aa4c91c359399688e68dd7a0a1d389b6
Binary file not shown.
@@ -1 +1 @@
67f278ade12d11b1445ac3d448853ec46282f2f3e2b2299c6c8fe0dfb4fc86cd
77909d57469273d0abd56d382e408ebb9a6c5dd4463b0c2cbca8f27a19bf116c
Binary file not shown.
@@ -1 +1 @@
8cf9210844a1968f485e6e747f6dff4a1fdf0a628e2945f731d8289dd01f8385
79beb3b8fdb9f412132dda4a76b88f55cfa859e3462c8cb020a0e431ad23fd72
Binary file not shown.
@@ -1 +1 @@
263e6ab4b54593f09ae2cf71e9a191acb95ed744cc16bbaf9a9263e41614e99a
f2430f9afa0c8af35d96832a0cf7b1eade8e5977fb46eed93123b65f950f4f15
Binary file not shown.
@@ -1 +1 @@
682a712b6c75ad0c0da24476f87c77a76b248c201d4d31923a79fda053b76b90
65753d7c4938560bf2cbc2d1eb2d5588f455ae40f265369aa1126c4f1fd0e5ce
Loaded 100 of 418 files, more files were not shown because too many files have changed in this diff. Show more