Compare commits

...
Author SHA1 Message Date
l0ng-ai a7708725cb feat(vcs): fill the commit message with the repository's commit template
An empty commit message field now starts with the file `commit.template`
names (any git config level; `~/` and repository-relative paths resolve
as git resolves them), without its `#` comment lines, since commits keep
the message as written. It is filled when the repository opens or
changes and when a commit clears the field; a typed message is never
replaced, and a prepared merge / cherry-pick / revert message still
takes precedence.

Fixes #1285
2026-11-05 02:04:47 +08:00
l0ng-ai c219c81bd8 fix(run): report Gradle tests in the test tree as they run (#1339)
Gradle prints only failed tests, so the live test tree only filled in
after the run from the XML reports, and stayed empty when the test task
was UP-TO-DATE and wrote no new reports.

Gradle test runs now get an init script that reports each test's start,
output, failure (with the stack trace), skip and finish as TeamCity
service messages and keeps test tasks from being UP-TO-DATE. The console
hides those lines; the JVM parser turns them into `[class, method]`
results, named as in the XML reports, and drops Gradle's own repeated
`Class > test() FAILED` lines. Tests the run already reported keep their
output instead of getting the report's copy appended. The TeamCity
message parsing moves out of the PHPUnit parser so both share it.

Fixes #1268
2026-11-05 02:02:06 +08:00
l0ng-ai 70a7c6dd2c fix(run): run Python modules with relative imports with -m (#1338)
Run or Debug on a Python file started it as a script
(`python src/tasks/cli.py`), so a module of a package that imports
relatively (`from .models import TaskList`) always failed with
"attempted relative import with no known parent package".

A file with relative imports now runs as `python -m package.module`
from the directory above its top package (the enclosing directories
with `__init__.py`, read once and re-read when one is added or
removed), with a venv-relative interpreter resolved against the project
root. Files without relative imports still run as scripts.
2026-11-05 02:01:35 +08:00
pravdomil fd69664ca7 fix(editor): follow local Markdown anchor links (#1328)
Go to Declaration and modifier-click on a fragment-only Markdown or HTML link such as [artist::order](#Artist-order) said there was nothing to go to: the text-reference fallback rejected fragment-only targets, and Markdown fragment lookup checked headings only, never HTML id / name anchors. Fragment-only href targets are now recognized and resolve in the same file (the current buffer, so unsaved anchors count), and Markdown files fall back to id / name anchors after headings.

Co-authored-by: pravdomil <[email protected]>
2026-11-05 01:48:06 +08:00
l0ng-ai 11932a87af fix(editor): Copy Reference and Paste as Plain Text in the editor's Copy / Paste Special menu (#1333)
The editor's Copy / Paste Special submenu only had Copy Path and Paste from History, so a class's fully qualified name couldn't be copied from the context menu although Copy Reference exists as an action. It now lists Copy Reference, Paste as Plain Text and Paste from History; the file path stays in the editor tab's Copy Path/Reference menu.

Fixes #1261
2026-11-05 01:27:49 +08:00
Qingandqtvbwfn 19a245fa04 fix(run): drop a removed workspace member's sessions and Services groups (#1289)
Removing a project from the workspace left its traces in Services: its run sessions, temporary configurations, remembered runs and type groups that now said there were no configurations. While the same project stays open, the run feature now watches the workspace member list and, per removed member, stops and forgets its sessions, drops its temporary configurations, reloads the configurations, removes the added types left empty and clears its remembered runs.

Co-authored-by: qtvbwfn <[email protected]>
2026-11-05 01:04:17 +08:00
Qingandqtvbwfn 62b5f11ae3 feat(build): Maven Daemon (mvnd) support with a settings switch (#1288)
Settings › Build Tools › Maven gets Use Maven Daemon (mvnd) and a Maven Daemon home path. With it on, maven_command (the one launcher point for sync, goals and run configurations) resolves mvnd from the configured home, $MVND_HOME, PATH, Homebrew or SDKMAN instead of mvn / mvnw; it falls back to mvn, with a logged warning, when mvnd is missing or the project JDK is older than the Java 17 its daemon needs. Sync passes -T1 so the daemon's parallel build doesn't interleave dependency:tree output.

Co-authored-by: qtvbwfn <[email protected]>
2026-11-05 01:03:18 +08:00
Qingandqtvbwfn 19c2d40528 feat(run): session tab icons and a Debug button in the run panel (#1283)
A finished run session's tab keeps its configuration's own icon instead of the green run arrow (a failed exit still shows the error icon), and the run panel's toolbar gets a Debug button next to Rerun that starts the same configuration under the debugger (in Services, a debug session reruns its debugger launch).

Co-authored-by: qtvbwfn <[email protected]>
2026-11-05 00:57:54 +08:00
Qingandqtvbwfn 78a3768ec1 feat(ui): syntax-highlight the read-only source previews (#1284)
The read-only previews beside Problems, Show Usages and TODO results are syntax-coloured from one whole-file parse (capped for very large files), so a line inside a block comment or a multi-line string is coloured as such; the pane renders plain first and recolours when the parse lands. SourcePreview takes the colouring as a colorize prop (the editor's previewColors), keeping src/ui free of editor imports, and the tok-* colours live once in global.css instead of a copy per view, with the theme's comment style.

Co-authored-by: qtvbwfn <[email protected]>
2026-11-05 00:55:03 +08:00
ntbang0901 f4fa2b42f6 fix(agents): style tables, headings and quotes in agent messages (#1324)
Agent replies only styled paragraphs, lists and code, so tables, headings, quotes, rules, images and task lists fell back to browser defaults: a 26px h1, a quote indented 40px, table columns squeezed to a few letters by overflow-wrap. They now get chat-sized styles from tokens; wide tables scroll sideways.

Closes #1273

Co-authored-by: ntbang0901 <[email protected]>
2026-11-05 00:50:00 +08:00
ntbang0901 d0910d3313 fix(markdown): drop raw style and form markup from rendered Markdown (#1320)
Markdown from comments, hovers, agent replies and files renders into the IDE's own document, so a raw <style> restyled the whole IDE and <form> / style= reached the page. Every Markdown sanitizer (LSP hovers and comments, agent messages, Markdown preview) now forbids <style>, <form> and the style attribute; agent messages also drop data-* written in a reply, since their click handlers trust data-copy / data-path. align, width and <details> stay.

Closes #1319

Co-authored-by: ntbang0901 <[email protected]>
2026-11-05 00:49:11 +08:00
Luis Roberto c0a74244b4 feat(i18n): Portuguese (pt) UI language (#914)
Adds Português (pt-BR) to the UI languages: the message catalog (locales/pt.json) and the source-text catalog (locales/text/pt.json, ~6,000 entries). Brazilian and European tags (pt-BR, pt_PT) resolve to it.

Co-authored-by: Psyducks2 <[email protected]>
2026-11-05 00:34:08 +08:00
l0ng-ai 8d29704d46 fix(run): merge Gradle console results into the reported test classes (#1323)
Gradle's console names a test class by its simple name and a JUnit 5
method by its display name (`method()`), while its XML reports use the
qualified class and the method name. After a run the test tree showed
each class twice, counted the failure twice, and the report's copy of
the failing test had no source location.

Console results are now moved under the reported class (when the simple
name is unambiguous), Gradle method names drop the empty `()`, and the
report's stack frame is matched for Kotlin names with spaces and frames
behind a class loader prefix (`app//`).
2026-11-05 00:33:44 +08:00
l0ng-ai c399ad2598 fix(editor): keep gutter implementation markers across tab switches (#1321)
Switching tabs remounts the editor view and the marker cache lived in its plugin; focusing an editor dropped the cache too. So every visit re-ran an implementation search per declaration in view, and on jdtls each one walks the whole workspace (#879: java at ~40% CPU while moving between classes). Answers are now cached per file and dropped only when other files change (edits, or source changes on disk; build output ignored) or a language server settles, through one window-wide subscription whose echo guard survives remounts.

Fixes #879
2026-11-05 00:31:12 +08:00
Jaw c2dd466305 fix(search): search hidden and git-ignored files outside ignored folders (#1318)
Find in Files skipped dotfiles (.eslintrc, .github/workflows/*.yml) and git-ignored files (.env, *.log) because the ide-search walkers dropped every hidden or ignored entry, files as well as folders. A shared walker now lets the ignore rules pick only the folders to enter and lists each walked folder for its files, so ignored folders (target/, logs/) are still skipped while ignored files elsewhere and hidden files are searched, as Go to File does since #1313. Find in Files, Inspect Code / Locate Duplicates (scope_files) and LSP reference candidates (files_with_word) all use it; .git, symlinks, archives, files.exclude and excluded roots still apply.

Fixes #1317

Co-authored-by: jaw52 <[email protected]>
2026-11-05 00:24:02 +08:00
tomgorandtomgor c27fea13b3 fix(ui): highlight the menu row under the pointer (#1242)
Menu rows now follow the pointer without taking focus: focusing the row on every pointer enter dropped the hover paint in the Windows webview, so menus looked frozen. Keyboard navigation continues from the hovered row, and an item's onActive fires once per enter. Focus no longer scrolls the page when a menu row takes it.

On Solarized Dark the selection and hover colours were the popup and panel colour, so menu, tree, list and tab selection was invisible there; on Solarized Light selection matched the panels. Both themes get distinct selection and hover colours, and the editor keeps Solarized's own selection colour.

Co-authored-by: tomgor <[email protected]>
2026-11-05 00:16:32 +08:00
Lee Dong Geon 94490e2b37 feat(vcs): show the history of the selected lines (#1266)
Git › Show History for Selection, as in IDEA: from Git › Current File and the editor's Git menu it opens a tab with only the commits that changed the selected lines (or the caret line), newest first, and the block before and after each commit in the diff viewer. Only Show History for the whole file existed before. git log -L tracks the lines across renames; the selection is first mapped through unsaved and uncommitted edits to HEAD's lines, and those edits come first as Local Changes. The query is its own ide-vcs module and cancellable command (vcs_block_history), read-only and allowed in Restricted Mode (--no-ext-diff --no-textconv).

Co-authored-by: ldg030201 <[email protected]>
2026-11-05 00:07:44 +08:00
Daniel AdámekandDanielADK f3afe69ca2 feat(php): PHP language level for Intelephense (#1264)
Intelephense checked PHP code against its default 8.5, whatever version the project targets. Settings › Languages & Frameworks › PHP gets a PHP language level, Auto (default: composer.json config.platform.php, else the lowest version require.php allows) or 5.6–8.5, sent as environment.phpVersion; nothing is sent without a requirement. Intelephense applies the version only to documents it parses after reading its configuration, so servers that opt in (resync-on-configuration, Intelephense only) get their open documents again right after workspace/configuration is answered (new jsonrpc Handler::after_response), and Intelephense keeps one index per level because it fixes the built-in functions when it indexes. A change to a root or workspace member composer.json reconfigures the servers.

Co-authored-by: DanielADK <[email protected]>
2026-11-05 00:04:01 +08:00
Jaw 9a9266f61b fix(index): let Go to File find git-ignored files such as .env (#1313)
The index walker used the ignore crate, which drops ignored files before we see them, and the watcher dropped their events too, so git-ignored files such as .env were missing from the index and Go to File, the Files tab and @ mentions could not reach them in any scope. As in IDEA, ignored folders stay out of the project but ignored files stay in it: each walked folder is also listed and the files the walker skipped are indexed by name only (no read, hash or symbols), the ignored flag is stored (index schema 3) so a file whose rules flip is re-parsed or dropped on the next scan, and the watcher reports ignored files in a separate list that only the index and open editors use (Local History, deployment, LSP and fsChanged see the same events as before). Watcher updates classify files by listing their folder with the scan's walker, so nested, global and info/exclude rules apply the same way.

Co-authored-by: jaw52 <[email protected]>
2026-11-04 23:55:49 +08:00
Qingandqtvbwfn 040578ba1d test(host): keep the ignore fixtures clear of a machine global excludes (#1279)
The rpc integration test wrote *.tmp fixtures; a machine whose global excludes file lists *.tmp already ignores them, so the backend never writes the .gitignore / info/exclude entry and the test failed reading it. The fixtures now use a suffix no excludes file lists.

Co-authored-by: qtvbwfn <[email protected]>
2026-11-04 23:54:26 +08:00
l0ng-ai ad2aeed54e fix(agents): don't show a branch for a task in a folder that isn't a repository (#1316)
For a folder that isn't a repository, the current-branch query answers with a repository found below it, so the New Agent Task page read "<folder> on <branch>" although the agent runs in the folder itself and the branch couldn't be changed. The branch is now shown only when the project is a repository.
2026-11-04 23:39:50 +08:00
l0ng-ai 1e4eee27c4 test(lsp): smoke-test everyday code intelligence per language with real servers (#1299)
* test(lsp): smoke-test everyday code intelligence per language with real servers

A new ide-host integration test drives completion, go to definition, Find
Usages, hover, diagnostics with a quick fix, an intention, formatting and
rename on each language's fixtures/ project, through the ops the app's
commands call and the language server the app starts by default:
TypeScript (vtsls), Python (basedpyright + Ruff), Rust (rust-analyzer),
Go (gopls), Java (jdtls) and Kotlin (kotlin-lsp). A failure names the
language and the operation.

The tests are ignored by default; a Language smoke workflow installs pinned
server versions and runs one language per job on changes to crates/,
languages/ and fixtures/. Adds fixtures/java-app (Maven, JDK 21) for Java.

* test(lsp): rename TypeScript from a usage and check Rust errors while typing

Covers the fixes the smoke test turned up: a TypeScript rename started at a
call site in another file renames the declaration (#1301), and rust-analyzer's
own diagnostics and fixes arrive without saving (#1306).
2026-11-04 23:35:53 +08:00
l0ng-ai 1789dbb98e fix(tools): run Kotlin LSP's launcher, not its deprecated wrapper script (#1310)
Current Kotlin LSP releases launch with bin/intellij-server; the
kotlin-lsp.sh / kotlin-lsp.cmd wrappers the installer picked now print a
deprecation warning and are announced to go away, which would leave a
fresh install without a server. The installer prefers the launcher and
still finds the wrappers in older releases.
2026-11-04 23:33:11 +08:00
l0ng-ai 20c439baaa fix(lsp): pull diagnostics from servers that provide them (#1307)
rust-analyzer reports its own analysis (a missing field, a wrong type) only
to clients that pull diagnostics; it pushes just `cargo check`'s, so in the
editor a Rust type error showed up only after saving. The client now
advertises `textDocument/diagnostic` and pulls an open file's diagnostics
from servers with a `diagnosticProvider`: when it opens or its server
starts, after edits (once typing pauses), on save (and other open files
for servers with inter-file dependencies) and on
`workspace/diagnostic/refresh`. Pulled and pushed diagnostics are kept
apart and shown, and given to code actions, together; a newer pull
supersedes an older reply, a server-cancelled pull is retried, and closing
a file drops what was pulled.
2026-11-04 23:33:04 +08:00
l0ng-ai a0dfb1e741 fix(lsp): rename TypeScript symbols everywhere instead of aliasing the import (#1302)
Renaming an imported function at a call site renamed only the local binding,
leaving `import { formatMoney as formatPrice }` and the declaration as it
was: both TypeScript servers default to aliases on rename. vtsls gets
`typescript.preferences.useAliasesForRenames = false` (and the JavaScript
one), typescript-language-server `providePrefixAndSuffixTextForRename = false`,
in every language that starts them.
2026-11-04 23:32:58 +08:00
l0ng-ai 4906bcb2a1 feat(vcs): Delete… for unversioned files in the Changes view (#1314)
Unversioned files in the commit view could be added or ignored but not deleted. Their context menu now offers Delete…, as IDEA's Changes view does: it asks first and moves the files to the Trash, only when every selected file is unversioned.
2026-11-04 23:32:56 +08:00
l0ng-ai f35c781f42 fix(refactor): apply server-versioned edits to files that aren't open (#1295)
gopls sends a document version for every file of a rename, including files
the editor never opened. Applying a workspace edit rejected any versioned
file without an open document as changed, so a Go rename touching a closed
file always failed with "changed; request the refactoring again". For a file
that isn't open, the preconditions captured with the edit now decide; an
edit without them is still rejected.
2026-11-04 23:32:51 +08:00
l0ng-ai 6f9819fee7 fix(vcs): Show History on a submodule folder shows the submodule's history (#1312)
Show History picked its repository with repoOf, which routes a nested repository's own root to the enclosing repository (right for the commit view's submodule pointer), so a submodule folder listed its gitlink history in the parent repository. A repository's own root now opens that repository's history, as in IDEA; other paths still use the innermost repository.
2026-11-04 23:29:44 +08:00
l0ng-ai 834b8d4e3b fix(database): keep line breaks when editing multi-line cells (#1311)
The inline cell editor is a single-line input that drops a value's line breaks on the first change, and it committed on every Enter; the Record view's fields had the same problem. Multi-line values now open in the multi-line editor, Shift+Enter opens it for any cell (from the grid or the inline editor) and Cmd/Ctrl+Enter applies it; CRLF values keep CRLF; the grid marks line breaks with a muted ↵; the Record view uses a text area for multi-line values.
2026-11-04 23:26:24 +08:00
xtr-hubandxtr-hub 7cdf690b4b fix(window): stop project tab switches from moving the group window (#1287)
Merged project windows are separate native windows that take turns being visible. Hidden members kept their own earlier frame and monitor, so the first switch after merging, and any switch after moving or resizing the window, moved or resized the window on screen; a maximized member could appear on another display. The group now copies the selected window's frame and display to the hidden members after a selection and after the visible window is moved or resized (debounced), skips copies when the frame already matches, and leaves macOS simple fullscreen to its own path, so switching projects only changes which window is visible.

Co-authored-by: xtr-hub <[email protected]>
2026-11-04 23:25:01 +08:00
Jaw 057877a39e fix(search): stop outer .gitignore rules at nested repositories in the remaining walkers (#1304)
#1309 made the Find in Files, file scope, index and walk_files walkers require git inside a repository, so an outer repository's .gitignore stops at a nested repository. External TypeScript project discovery (ide-lsp) and Lombok detection (ide-sdk) still walked with require_git(false) and could miss a nested repository's sources the same way; they now use the same check. Adds tests for walk_files (nested repository with its own .gitignore and info/exclude, a plain folder inside the outer repository, the outer repository, a folder outside any repository) and for scope_files.

Co-authored-by: jaw52 <[email protected]>
2026-11-04 23:20:38 +08:00
l0ng-ai a08707dabd fix(search): stop an outer repository's .gitignore at a nested repository (#1309)
The walkers used require_git(false), and with it the ignore crate never treats a .git directory as a boundary, so an outer repository's .gitignore hid a project that is its own repository inside an ignored folder (Find in Files found nothing). Walkers now require git when the walked folder is inside a repository, so rules stop at a nested repository's root as in git; folders outside any repository still honour their .gitignore. Covers the Find in Files engine, the file scope, the index walk and ide-vfs walk_files.
2026-11-04 23:16:47 +08:00
l0ng-ai 092a700ddc fix(vcs): keep the current branch name in New Branch even though it exists (#1305)
New Branch dropped its suggestion when validateBranchName rejected it, and the current branch always exists, so a plain New Branch from HEAD always opened empty; the test never seeded the local branches. The suggestion is kept, the dialog shows that the branch exists from the start and Create stays disabled until the name changes; other prefilled prompts (Rename Branch) keep showing errors only after editing.
2026-11-04 23:12:40 +08:00
Jaw b31edc3482 fix(editor): give each file its own find bar, query and caret (#1296)
The find bar belonged to the pane and every bar shared one query, and its effect re-ran the search and moved the caret whenever the active tab changed, so switching files dragged the previous file's search into the next one and lost its caret. Each tab now has its own find bar with its own query and options, as with IDEA's per-editor search session. The last search only pre-fills a newly opened bar (or the selection does) and drives Find Next without a bar; showing a bar again only highlights, and the caret moves only when the query or an option changes in that bar. Bars of closed tabs are dropped.

Co-authored-by: jaw52 <[email protected]>
2026-11-04 23:11:05 +08:00
ntbang0901 d032addb17 fix(run): offer Run for classes whose superclass declares main (#1204)
The JVM run-target scanner only reads the file's own text, so a class like `class App extends AtomStarter {}`, whose superclass (in the project or a library JAR) declares `static main`, got no gutter Run, no Ctrl+Shift+R and no file-level Run, although the JVM launches it. Classes with an `extends` clause and no main of their own are now candidates: the superclass chain is walked through the language server's type hierarchy, each class supertype's source (file, decompiled or source-jar document) is checked for a static main, and the answer is cached per class and `extends` clause, so the gutter, Run from context, launch and debug agree. Interfaces are skipped (a static interface main is not inherited); negative answers are asked again when the server settles. `main(java.lang.String[] a)` is accepted as well.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-04 23:11:01 +08:00
ntbang0901 8f77d7077c fix(editor): keep tooltips full width at the editor's right edge (#1203)
On WebKit, CodeMirror positions tooltips absolutely inside the editor, but the tooltip space still reported the whole window as the right bound. CodeMirror never shifted a tooltip near the editor's right edge to the left, the box was squeezed into the space left of that edge, and the next measurement read the squeezed width, so it could end as a one-character column. The tooltip space is now the editor's own horizontal extent (clamped to the window) and tooltips size to their content, which fixes hover, inline blame, debugger values, signature help and hints together.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-04 23:10:51 +08:00
ntbang0901 bdc108a61b fix(editor): show pinned tabs in their own row when Multiple rows is on (#1198)
"Show pinned tabs in a separate row" was ignored while "Multiple rows" was on, because the pinned row explicitly excluded horizontal layouts with multiple rows. The two options are independent, so the pinned row now shows whenever some but not all tabs are pinned, and each row wraps on its own. With the rows split, dropping a tab on the pinned row pins it and keeps it in the pinned group, and dropping it on the other row unpins it.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-04 23:10:28 +08:00
ppxbandppxb a363544625 fix(ui): keep the arrow cursor on scrollbars (#1191)
Scroll containers set cursor: text, and the webview applies an element's cursor to its scrollbar, so scrollbars showed the I-beam. The containers now keep the arrow and the text inside them gets the I-beam: the editor (.cm-content instead of .cm-scroller, which also covers the diff and merge editors), the source preview, the Run, Build and Debugger consoles, the Git console, the Database output and the Services log.

Co-authored-by: ppxb <[email protected]>
2026-11-04 23:10:23 +08:00
nguyenmuoi157 f942ea168d fix(editor): allow text selection and input in editable elements (#1209)
The workbench sets user-select: none on body and nothing restored it for editable elements, so a contenteditable such as CodeMirror's .cm-content inherited it. In WebKit a contenteditable with user-select: none cannot hold a selection, so typing, paste and drag-and-drop into the editor could be dropped. input, textarea and [contenteditable="true"] now get user-select: text next to the cursor: text rule they share; the selection lock still overrides it.

Co-authored-by: nguyenmuoi157 <[email protected]>
2026-11-04 23:10:08 +08:00
l0ng-ai 59f1a1e6db docs: remove the Where it's going section from the README 2026-11-04 23:06:36 +08:00
l0ng-ai a4a16d0854 fix(plugins): restore the hello fixture's source fingerprint
The recorded fingerprint did not match the fixture's sources, so
plugins/build.sh --check reported the fixture as stale.
2026-11-04 22:56:08 +08:00
l0ng-ai b440370f2d fix(workbench): focus a tool window's content once it loads (#1280)
Activating a tool window focuses its `[data-autofocus]` element or first
content field one frame later, else its body. A lazily loaded view such as
the Terminal renders its field after that, so the first ⌥F12 left focus on
the body. When no target exists yet, the tool window now watches its body
and focuses the target when it appears, unless focus has moved on.
2026-11-04 22:52:53 +08:00
l0ng-ai aa33d2eeab test(e2e): cover the core editing, code intelligence, VCS, run and debug flows (#1277)
Adds a `stubDesktop` fixture that switches the browser-mode workbench to
desktop mode over a stubbed IPC layer: tests answer the commands they need,
push Channel messages and IdeEvents, and assert on the commands invoked.
Unstubbed commands fail as unavailable; documents keep the in-memory backend.

New flows covered:
- editing: modified marker and ⌘S save, ⌘R Replace All with one undo step
- code intelligence: completion, ⌘B to another file, Find Usages, in-place
  rename, diagnostics with an ⌥⏎ quick fix
- VCS: ⌘K commit with message validation and default inclusion
- run: ⌃R output streaming and exit, ⌘F2 stop
- debug: breakpoint, stop with frames and variables, step over, resume
- search: Find in Files results opening at the match line; the Search
  Everywhere file case no longer needs fixme
- terminal: typed command reaches the shell and its output renders
- project: Close Project to Welcome, trust prompt, open a recent project
2026-11-04 22:52:48 +08:00
l0ng-ai b26318a32c fix(plugins): record the hello fixture's source fingerprint 2026-11-04 22:42:24 +08:00
l0ng-ai 8a2e370178 docs: drop the priority note from the README direction section 2026-11-04 22:41:15 +08:00
l0ng-ai 61a05e5a36 docs: add where the project is going to the README 2026-11-04 22:40:15 +08:00
l0ng-ai d9355155f3 fix(plugins): rebuild the hello fixture after its source comment changed 2026-11-04 22:39:06 +08:00
l0ng-ai ecd593eb0c ci: stop requiring a linked issue on PRs
The PR check now only asks for the How it was verified section. Linking an
issue with Closes #<n> stays recommended when a PR resolves one.
2026-11-04 22:19:31 +08:00
l0ng-ai 0716a48247 chore: v0.1.36 2026-11-04 22:16:25 +08:00
l0ng-ai 699c71c1f2 docs: introduce plugins in the README 2026-11-04 22:14:16 +08:00
l0ng-ai 014f51ae54 chore: move the JDT LS components out of plugins/ (#1292)
java-usages, kotlin-bridge and the patched Lombok agent are JDT LS bundles and a Java agent embedded into the app, not OpenIDEA plugins. Move them to java/ so plugins/ holds only plugins, and update the paths that embed or mention them.

Also fix the stale build path in the hello fixture and the contributor guide's layout (there is no templates/ directory; starter templates live in the openidea-plugin CLI).
2026-11-04 22:11:38 +08:00
l0ng-ai 30b0549196 fix(editor): don't autosave while focus is in a popup over the editor
Opening a popup (intentions, completion) moved focus out of the editor and
autosaved it. The save cleanup could change the text, such as trailing
spaces on another line, so the quick fix the popup listed no longer matched
and the first attempt to apply it failed.

While focus is in a popup the editor is not saved; it is saved when focus
moves on from the popup to anything but the editor.
2026-11-04 21:57:59 +08:00
l0ng-ai 0ecbc6b35d fix(workspace): keep a previewed edit valid when its file is saved
An edit's preconditions hashed disk contents plus unsaved buffers. Saving a
buffer moved its text from one to the other and changed the hash, so a
quick fix, rename or refactoring listed just before a save (autosave of a
freshly typed scratch, for one) failed with "changed; request the
refactoring again", and only the next attempt worked.

Unsaved buffers now stand in for their files' bytes in the hash, so saving
doesn't change it; edits, external writes and file operations still do.
2026-11-04 21:40:12 +08:00
l0ng-ai 859f63454e fix(plugins): move the repository URL to the gear menu and drop the no-repository hint (#1282)
The Available tab always showed a "There is no online plugin repository"
notice under the search field when no index was set. Release builds use
the official index by default, so the notice only appeared in development
builds, where the empty URL field above it already said the same thing.
The URL field itself sat at the top of the list although most people never
change it.

- The notice is gone, and the search placeholder reads "Search plugins"
  because the list holds marketplace plugins as well as local ones.
- With a repository configured, the empty details panel says "No plugin
  selected" instead of claiming there is no online repository.
- The marketplace index URL is edited from the gear menu's "Plugin
  Repository…" dialog; leaving it empty restores the default.
2026-11-04 21:38:59 +08:00
l0ng-ai c1ea7a9ef2 test(plugins): build the command template with the SDK's lockfile
The resolver also looks up the SDK's wasm-only dependencies, which a clean
cargo cache doesn't have, so the offline build failed in CI.
2026-11-04 21:22:40 +08:00
l0ng-ai 3100aa2c00 feat(scratch): start new scratch files from a language template
A new Java scratch was an empty file that couldn't run until a class and
main method were typed in. Languages can now define `scratch-template` in
`languages/<id>.toml`; a new scratch gets that content with the caret at
`$END$`.

- Java, Kotlin, Go, Rust, C and C++ get a runnable `main`.
- Other languages still start empty.
2026-11-04 21:09:12 +08:00
l0ng-ai 52360c72d2 fix(editor): describe a Java file outside every project accurately
jdtls still completes and navigates JDK classes in a non-project file such
as a scratch; it only skips semantic checks and project classes. The banner
said navigation and completion don't work, and in a Maven or Gradle project
it offered a build reload that can't bring a file outside the project in.

- The banner now says only syntax errors are reported and only JDK classes
  resolve.
- The Maven / Gradle reload is offered only for files inside the open
  project (or one of its workspace projects).
2026-11-04 21:09:12 +08:00
l0ng-ai 7fd330912a fix(process): wait for a killed tree's descendants to exit on Windows (#1272)
`taskkill /T /F` only starts terminating the tree, and the caller then
waits for its direct child. A grandchild could still be exiting, with its
files open, when `kill_tree` returned or `Peer::exited` resolved. The
descendants are now opened before the kill and waited for (up to 5 s)
after it. A process counts as a descendant only if it started after its
parent, so a reused parent id doesn't pull in unrelated processes.

Closes #1270
2026-11-04 21:07:08 +08:00
l0ng-ai eaca68f263 test: retry first edit-path calls and wait for effects in flaky tests (#1271)
A provider call that starts a plugin instance (or reads files) can run past
the edit-path soft deadline on a slow runner and returns nothing; the tests
now ask again, as the editor does. The AI completion test cancels the slow
request only once the server has it, and measures from the cancel. The
SourcePreview test waits for the effect that marks the selection scope.

Closes #1269
2026-11-04 21:07:03 +08:00
l0ng-ai 4b5ef7069c fix(plugins): build new plugins outside the repository
- `openidea-plugin new` makes a Rust plugin depend on the SDK at the tool's
  release tag (git) unless the plugin is inside the source tree the tool was
  built from or `--sdk` is given; the path written before only resolved on
  the machine that built the tool.
- The command template compiles again: `run_command` returns
  `openidea_plugin::Error`. A test now runs `cargo test` on the generated
  plugin against the SDK.
- plugins/README.md: installing the tool with `cargo install --git`, the SDK
  dependency, where plugins can be installed (welcome screen and trusted
  projects), and where to look when a plugin misbehaves.
2026-11-04 20:55:31 +08:00
l0ng-ai 2cd16a723d fix(lsp): let language servers register the files they watch (#1223)
* fix(lsp): resolve newly installed Python packages without a restart

Packages installed into an excluded .venv (uv add, uv pip install) never
reached the Python language servers: the watcher reports changes in excluded
folders only as excluded_dirs, and didChangeWatchedFiles forwarded nothing but
the server's own source files. basedpyright kept its failed import resolution
until restarted.

Language servers get a watched-files list of globs. Files matching it, and
excluded folders matching it, are forwarded too; the Python servers watch
their configuration and site-packages.

* fix(lsp): let servers register the files they watch

Clients now support dynamic registration of workspace/didChangeWatchedFiles,
with relative patterns. Changes on disk go to each server as its registered
watchers match them, including folders in excluded trees (a package installed
into .venv, go.mod rewritten by go get). Folders outside the project that
relative patterns name (a virtualenv elsewhere, a path dependency) get their
own watchers. Servers that register nothing keep hearing of their languages'
files.

Registrations sent right after initialized are no longer dropped while the
server is still being marked ready.

Replaces the per-server watched-files list.
2026-11-04 20:44:24 +08:00
l0ng-ai 71c0515af6 fix(ui): no focus ring on controls that were only clicked (#1213)
The webview focuses a clicked button and the focus stays there: across
project window-tab switches, and popups hand it back to their trigger. The
next keystroke switches the engine to keyboard modality, so that button got a
focus ring (window tabs, the title-bar project switcher). A pressed control is
now marked and keeps no ring until Tab or an arrow key moves focus onto it.
Focus itself is unchanged, since actions resolve their context from it.

Closes #1211
2026-11-04 20:44:19 +08:00
l0ng-ai b473876d59 feat(agents): reference folders from @ completion (#1210)
Folders now appear in the composer's `@` completion next to files, from a new
`find_dirs` index query over the folders that hold indexed files (kept
incrementally in the index, ~0.5 ms on 100k files). Enter or a click
references the folder as `@dir/` and sends it as a resource link; Tab keeps
completing inside it.

Closes #1202
2026-11-04 20:44:14 +08:00
l0ng-ai d7886bab45 test(plugins): repeat the cancel until the call is running in the cancel test (#1263)
Closes #1248
2026-11-04 20:34:33 +08:00
l0ng-ai dd370654cb feat(plugins): leave room in plugin API 1.0 for what will grow (#1245)
Closes #1237
2026-11-04 20:11:46 +08:00
l0ng-ai e3c7f362c4 fix(plugins): list the marketplace and install plugins from the welcome screen (#1256)
Closes #1255
2026-11-04 20:09:59 +08:00
l0ng-ai 02addfa2c5 test(plugins): let the first call in the deadline test start its instance (#1257)
Closes #1248
2026-11-04 20:09:44 +08:00
l0ng-ai 82fe52f595 fix(agents): wait for the agent to exit before deleting its worktree (#1240)
Closes #969

## What and why

On Windows, deleting an agent task and its worktree could fail with `os error 32` while the agent's process tree still held the directory or its files open. Deletion now waits up to 5 seconds for the agent process to be killed and reaped, then retries folder removal for transient locks. This also covers persistent ACP agents such as CodeBuddy that cannot reload their session and therefore stay alive while idle.

## How it was verified

- Windows desktop acceptance through the Tauri MCP bridge, using a deterministic ACP agent with a real PowerShell descendant holding a file open. Closed the conversation tab, then deleted the task with worktree removal enabled. A separate helper retained another file lock for 1.2 seconds: the pre-fix implementation showed `os error 32`; the fixed implementation deleted the task and directory, with both agent processes gone.
- Also verified direct deletion, archive then delete, and deletion with worktree removal unchecked. Processes exited in all cases; branches were preserved, and unchecked removal preserved the folder.
- `cargo test -p ide-jsonrpc -p ide-agent -- --test-threads=2`: 155 passed, 5 real-agent tests ignored. The test runner used Git Bash on PATH and a process-local `core.autocrlf=false`, matching CI's LF fixture setup; no global Git settings were changed.
- `cargo clippy -p ide-jsonrpc -p ide-agent --all-targets -- -D warnings`, `cargo fmt --all --check`, and `git diff --check`: passed.
- Full workspace (Windows/macOS/Linux), frontend and E2E CI: [final PR run](https://github.com/l0ng-ai/openidea/actions/runs/41819945536).
- Windows regression coverage now checks deletion while waiting for permission and while idle without `session/load`, actual sharing violations, delayed file-lock release, and `Peer::exited()` waiting for a descendant's file handle to close. Existing Unix process tests remain enabled.
- Evidence (local, not committed):
  - Before: `C:/Users/thoma/repo/pr1240-evidence-20261104/before.png`
  - After: `C:/Users/thoma/repo/pr1240-evidence-20261104/after.png`
  - Recording: `C:/Users/thoma/repo/pr1240-evidence-20261104/after.gif`
  - Runtime results: `C:/Users/thoma/repo/pr1240-evidence-20261104/runtime-cases.json`
- Real signed-in CodeBuddy and native Claude/Codex were not exercised. Native Claude/Codex use an in-process connection, so their `exited()` resolves immediately and removal relies on the bounded retry. Shared-worktree fork protection remains unchanged.
2026-11-04 19:39:40 +08:00
l0ng-ai d94a117d69 test(plugins): don't time the plugin host tests against the runner's speed (#1250)
Closes #1248
2026-11-04 18:46:52 +08:00
Marius Ciepluch b6b0ce46c3 fix(fonts): enumerate all faces and per-user fonts on Windows (#908)
Enumerate every locally installed Windows font face, including per-user font files. Correct glyph-width probing and collection-file enumeration, and exclude cloud fonts that are not installed.

Closes #1241
2026-11-04 18:46:23 +08:00
l0ng-ai 953e844541 feat(plugins): community plugins, version compatibility and categories in the marketplace (#1226)
* feat(plugins): list community plugins from a registry in the marketplace

The marketplace index can now carry plugins of other publishers, published
from their authors' repositories through the l0ng-ai/openidea-plugins
registry.

- `openidea-plugin index` with a key replaces only that publisher's plugins
  (it used to drop every other entry); without a key it indexes each package
  against the publishers the index already lists, checking the key and the
  id prefix instead of only the signature itself.
- `openidea-plugin merge` takes another index's publishers, plugins and
  revocations in place of every publisher but `--keep`; the other index can't
  list or revoke the kept publishers' plugins.
- `openidea-plugin check` accepts a `.oiplugin` file.
- Releases merge the registry's index into plugins-index.json and attach the
  Linux x64 tool; plugins-index.yml refreshes the community plugins in the
  latest release hourly or when the registry signals a change.
- plugins/README.md and CONTRIBUTING.md describe official and community
  plugins.

* feat(plugins): offer the newest plugin version this IDE can load

Index entries record the plugin API each version needs (`engine`: the
manifest's `engines.openidea` narrowed to the API its component was built
for), and an index may list several versions of a plugin. Settings › Plugins
offers the newest version this IDE can load; with none, the plugin is shown
as incompatible with the API it needs, and downloading it is refused.

`openidea-plugin index` keeps, of each plugin, the newest version per API
requirement, so users of an older IDE keep the last version built for them;
`--download-base` makes downloads absolute, and the release workflow points
them at the release that published them. Scratch directories are now unique
per call.

* feat(plugins): file marketplace plugins under categories

`plugin.toml` takes `categories` (up to 3 of languages, themes, icon-themes,
linters, vcs, ai, frameworks, tools, other). A plugin that declares none is
filed under what its contributions imply (themes, icon themes, languages),
else under other; categories an IDE doesn't know are ignored, and `check`
warns about them. Index entries carry the categories, and Settings › Plugins
› Available filters by them and shows them in a plugin's details. The
official plugins declare theirs.

* feat(plugins): let a marketplace index point to where it moved

An index with `moved` sends the IDE to the index at that URL (relative or
absolute, up to three hops; an https index can only move to https). Downloads
resolve against the index actually read, so a marketplace can change hosts
while released IDEs keep the address they were built with.

* fix(plugins): full-width category filter and aligned incompatibility note

* fix(plugins): translate the AI category in Chinese

* feat(plugins): keep a plugin's signer across updates and announce updates

- Index format: fields are only added; an IDE reading a newer `format`
  still applies its revocations and `moved`, lists no plugins and asks to
  be updated, instead of failing to read the index.
- Key continuity: an installed plugin updates silently only to a package
  signed with the same key, or a key it endorsed (`KeyEntry.endorsement`,
  `index --previous-key`, `openidea-plugin endorse`). Another signer, or an
  unsigned package replacing a signed one, is shown on the install page and
  needs the user's confirmation (`Consent.signerChange`).
- Official key: release builds carry the official publisher's public key
  (`OPENIDEA_OFFICIAL_PLUGIN_KEYS`, derived from PLUGIN_SIGNING_KEY);
  `openidea.*` plugins must be signed with it or a key it endorsed,
  whatever the index lists.
- Updates: the daily marketplace check announces new versions once, with
  Update All (also in the gear menu and as an action). An update that asks
  for nothing new installs with the grants the installed version has.
- Index entries need `engine`; `categories` beyond the first three are
  ignored instead of rejecting the manifest.
2026-11-04 18:24:43 +08:00
358 changed files with 18834 additions and 1477 deletions

No files matched your search

+1 -1
View File
@@ -1,4 +1,4 @@
Closes #
<!-- If this resolves an issue: Closes #<n> -->
## What and why
+97
View File
@@ -0,0 +1,97 @@
name: Language smoke
# Everyday code intelligence (completion, go to definition, Find Usages, hover, diagnostics
# with a quick fix, an intention, formatting, rename) per language, against the language
# server the app starts by default: crates/ide-host/tests/language_smoke.rs.
on:
push:
branches: [main]
pull_request:
branches: [main]
types: [opened, synchronize, reopened]
paths:
- "crates/**"
- "languages/**"
- "fixtures/**"
- "Cargo.toml"
- "Cargo.lock"
- ".github/workflows/language-smoke.yml"
workflow_dispatch:
concurrency:
group: language-smoke-${{ github.ref }}
cancel-in-progress: true
env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
# Pinned so a server release can't fail an unrelated PR; bump them deliberately.
VTSLS: 0.3.0
TYPESCRIPT: 6.0.3
BASEDPYRIGHT: 1.40.2
RUFF: 0.17.3
GOPLS: v0.24.0
JDTLS: 1.63.0-202610191322
KOTLIN_LSP: 263.7084.0
GRADLE: 8.14.3
jobs:
smoke:
name: Language smoke (${{ matrix.language }})
strategy:
fail-fast: false
matrix:
language: [typescript, python, rust, go, java, kotlin]
runs-on: ubuntu-22.04
timeout-minutes: 40
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with: { components: "rust-analyzer, rustfmt" }
# One build cache for every language; main saves it.
- uses: Swatinem/rust-cache@v2
with:
shared-key: language-smoke
save-if: ${{ github.ref == 'refs/heads/main' && matrix.language == 'rust' }}
- if: matrix.language == 'typescript'
uses: actions/setup-node@v4
with: { node-version: 22 }
- if: matrix.language == 'typescript'
run: npm install --global "@vtsls/language-server@$VTSLS" "typescript@$TYPESCRIPT"
- if: matrix.language == 'python'
uses: actions/setup-python@v5
with: { python-version: "3.12" }
- if: matrix.language == 'python'
run: pip install "basedpyright==$BASEDPYRIGHT" "ruff==$RUFF"
- if: matrix.language == 'go'
uses: actions/setup-go@v5
with: { go-version: "1.27", cache: false }
- if: matrix.language == 'go'
run: go install "golang.org/x/tools/gopls@$GOPLS"
- if: matrix.language == 'java' || matrix.language == 'kotlin'
uses: actions/setup-java@v4
with: { distribution: temurin, java-version: "21" }
- if: matrix.language == 'java'
run: |
mkdir -p "$RUNNER_TEMP/jdtls"
curl -fsSL "https://download.eclipse.org/jdtls/milestones/${JDTLS%-*}/jdt-language-server-$JDTLS.tar.gz" \
| tar xz -C "$RUNNER_TEMP/jdtls"
echo "$RUNNER_TEMP/jdtls/bin" >> "$GITHUB_PATH"
# The Kotlin fixture has no Gradle wrapper: the server imports it with the Gradle on PATH.
- if: matrix.language == 'kotlin'
uses: gradle/actions/setup-gradle@v4
with: { gradle-version: "${{ env.GRADLE }}" }
- if: matrix.language == 'kotlin'
run: |
mkdir -p "$RUNNER_TEMP/kotlin-lsp" "$RUNNER_TEMP/bin"
curl -fsSL "https://download.jetbrains.com/language-server/kotlin-server/$KOTLIN_LSP/kotlin-server-$KOTLIN_LSP.tar.gz" \
| tar xz -C "$RUNNER_TEMP/kotlin-lsp" --strip-components=1
ln -s "$RUNNER_TEMP/kotlin-lsp/kotlin-lsp.sh" "$RUNNER_TEMP/bin/kotlin-lsp"
echo "$RUNNER_TEMP/bin" >> "$GITHUB_PATH"
- run: cargo test --locked -p ide-host --test language_smoke --no-run
- run: cargo test --locked -p ide-host --test language_smoke -- --ignored --exact ${{ matrix.language }}
+46
View File
@@ -0,0 +1,46 @@
name: Plugin index
# Refreshes the community plugins in the latest release's plugins-index.json
# (what released apps read) from the registry, l0ng-ai/openidea-plugins. The
# official plugins, their key and revocations stay as the release wrote them.
on:
schedule:
- cron: "17 * * * *"
workflow_dispatch:
# Sent by the registry when its index changes.
repository_dispatch:
types: [community-plugins]
permissions:
contents: write
concurrency:
group: plugins-index
env:
COMMUNITY_PLUGINS_INDEX: https://github.com/l0ng-ai/openidea-plugins/releases/download/index/index.json
jobs:
merge:
runs-on: ubuntu-latest
steps:
- name: Merge the community index
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
cd "$RUNNER_TEMP"
tag="$(gh release view --repo "$GITHUB_REPOSITORY" --json tagName -q .tagName)"
tool=openidea-plugin-linux-x86_64
if ! gh release download "$tag" --repo "$GITHUB_REPOSITORY" -p plugins-index.json -p "$tool.gz"; then
echo "::notice::$tag has no plugin index or openidea-plugin tool yet"
exit 0
fi
gunzip "$tool.gz" && chmod +x "$tool"
cp plugins-index.json before.json
"./$tool" merge plugins-index.json "$COMMUNITY_PLUGINS_INDEX" --keep openidea
if cmp -s before.json plugins-index.json; then
echo "unchanged"
else
gh release upload "$tag" plugins-index.json --repo "$GITHUB_REPOSITORY" --clobber
fi
-9
View File
@@ -24,18 +24,9 @@ jobs:
const pr = context.payload.pull_request;
const LABEL = 'needs-info';
const MARKER = '<!-- pr-check -->';
// Docs, CI and pure refactors may skip the linked issue.
const NO_ISSUE_TYPES = /^(docs|ci|build|chore|refactor|style|test)(\(.+\))?!?:/;
const body = (pr.body || '').replace(/<!--[\s\S]*?-->/g, '');
const problems = [];
const linksIssue =
/\b(close[sd]?|fix(e[sd])?|resolve[sd]?)\s*:?\s+(#\d+|https:\/\/github\.com\/[\w.-]+\/[\w.-]+\/issues\/\d+)/i.test(body);
if (!linksIssue && !NO_ISSUE_TYPES.test(pr.title)) {
problems.push('Link the issue this PR resolves, e.g. `Closes #123`.');
}
const verified = body.match(/^##+\s*How it was verified\s*$([\s\S]*?)(?=^##+\s|(?![\s\S]))/im);
if (!verified || !verified[1].trim()) {
problems.push('Fill in **How it was verified**: the commands you ran and their results. UI changes need before/after screenshots.');
+32 -4
View File
@@ -18,6 +18,7 @@ env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
TAG: ${{ github.event.inputs.tag || github.ref_name }}
COMMUNITY_PLUGINS_INDEX: https://github.com/l0ng-ai/openidea-plugins/releases/download/index/index.json
jobs:
# Creates (or reuses) one draft release that every platform build uploads into.
@@ -25,6 +26,7 @@ jobs:
runs-on: ubuntu-latest
outputs:
release_id: ${{ steps.release.outputs.id }}
official_plugin_keys: ${{ steps.plugin-keys.outputs.keys }}
steps:
- uses: actions/checkout@v4
with: { ref: "${{ env.TAG }}" }
@@ -35,6 +37,20 @@ jobs:
echo "::error::tag $TAG does not match tauri.conf.json version $app"
exit 1
fi
# The official plugins' public key (from PLUGIN_SIGNING_KEY, the seed
# `openidea-plugin keygen` writes), built into the app: their packages
# must carry its signature whatever the marketplace index lists. Without
# the secret the app takes the index's word, as development builds do.
- id: plugin-keys
env:
SIGNING_KEY: ${{ secrets.PLUGIN_SIGNING_KEY }}
run: |
if [ -n "$SIGNING_KEY" ]; then
seed="$(printf %s "$SIGNING_KEY" | base64 -d | od -An -tx1 | tr -d ' \n')"
key="$(printf '302e020100300506032b657004220420%s' "$seed" | xxd -r -p |
openssl pkey -inform DER -pubout -outform DER | tail -c 32 | base64)"
echo "keys=$key" >> "$GITHUB_OUTPUT"
fi
- id: release
uses: actions/github-script@v7
with:
@@ -57,6 +73,8 @@ jobs:
build:
needs: create-release
name: Build (${{ matrix.name }})
env:
OPENIDEA_OFFICIAL_PLUGIN_KEYS: ${{ needs.create-release.outputs.official_plugin_keys }}
strategy:
fail-fast: false
matrix:
@@ -184,7 +202,11 @@ jobs:
# (releases/latest/download/plugins-index.json). They are not in the
# installers: users install them from Settings › Plugins. Without the
# signing key (secret PLUGIN_SIGNING_KEY: the file `openidea-plugin keygen`
# writes) nothing is published and the release goes on.
# writes) nothing is published and the release goes on. The index also lists
# the community plugins of the registry (COMMUNITY_PLUGINS_INDEX; refreshed
# between releases by plugins-index.yml), and the release carries the
# openidea-plugin tool for Linux x64, which that workflow and the registry's
# checks run.
plugins:
needs: create-release
name: Official plugins
@@ -237,11 +259,17 @@ jobs:
version="$(sed -n 's/^version *= *"\(.*\)".*/\1/p' "$dir/plugin.toml" | head -1)"
"$tool" pack "$dir" --key "$key" --publisher openidea --out "$out/$id-$version.oiplugin"
done
# The published index's revocations and earlier keys carry over.
# The published index's revocations, earlier keys and community plugins carry over.
curl -fsSL "https://github.com/$GITHUB_REPOSITORY/releases/latest/download/plugins-index.json" \
-o "$out/plugins-index.json" || rm -f "$out/plugins-index.json"
"$tool" index "$out" --publisher openidea --name OpenIDEA --key "$key" --out "$out/plugins-index.json"
gh release upload "$TAG" "$out"/*.oiplugin "$out/plugins-index.json" --clobber
# Downloads point at this release: older versions kept for older IDEs stay where they were published.
"$tool" index "$out" --publisher openidea --name OpenIDEA --key "$key" --out "$out/plugins-index.json" \
--download-base "https://github.com/$GITHUB_REPOSITORY/releases/download/$TAG"
"$tool" merge "$out/plugins-index.json" "$COMMUNITY_PLUGINS_INDEX" --keep openidea ||
echo "::warning::the community plugins were not refreshed"
gzip -9 -c "$tool" > "$out/openidea-plugin-linux-x86_64.gz"
gh release upload "$TAG" "$out"/*.oiplugin "$out/plugins-index.json" \
"$out/openidea-plugin-linux-x86_64.gz" --clobber
# The updater manifest (latest.json), built once from the signed bundles every build uploaded.
# Fails when a platform is missing, so a release never ships a manifest without it.
+10 -5
View File
@@ -18,9 +18,12 @@ Visual style: dark, low-contrast greys, one cool accent
- `src/ui/` — design-system primitives. `src/features/<name>/` — one feature each,
exporting `activate()` registered in `src/features/index.ts`.
- `languages/<id>.toml` — language definitions (detection, comments, LSP, DAP).
- `crates/ide-host` — the `openidea-server` binary for remote development.
- `wit/` — plugin interface definitions; `plugins/` — sample and bundled plugins;
`templates/` — plugin starter templates.
- `crates/ide-host` — the `openidea-server` binary for remote development and the
`openidea-plugin` CLI (plugin starter templates live in its `templates.rs`).
- `wit/` — plugin interface definitions; `plugins/` — sample and bundled plugins.
Plugin docs: `plugins/README.md`, `docs/plugins/themes.md`.
- `java/` — JDT LS extension bundles (`java-usages`, `kotlin-bridge`) and the patched
Lombok agent, embedded into the app by `ide-sdk` / `ide-tools`.
- `e2e/` — Playwright UI tests (`pnpm e2e`). `fixtures/` — sample projects per language
for manual and automated testing.
@@ -46,8 +49,8 @@ Visual style: dark, low-contrast greys, one cool accent
## Issues & PRs
Follow `CONTRIBUTING.md` and `.github/pull_request_template.md`: one issue per PR with
`Closes #<n>`, reproduce first, keep the diff focused, short plain description, and a
Follow `CONTRIBUTING.md` and `.github/pull_request_template.md`: one problem per PR
(`Closes #<n>` when it resolves an issue), reproduce first, keep the diff focused, short plain description, and a
**How it was verified** section with commands run, results and (for UI) before/after screenshots.
Unclear product decisions → `needs-decision` and stop.
@@ -64,6 +67,8 @@ Shared agent skills live in `.agents/skills/` (`.claude/skills/` links to them):
this worktree's running MCP bridges and actual ports.
- `pnpm build` — typecheck + bundle frontend.
- `cargo nextest run --workspace` (or `cargo test`), `cargo clippy --workspace --all-targets -- -D warnings`, `cargo fmt --all`.
- `cargo test -p ide-host --test language_smoke -- --ignored --exact <language>` — code intelligence
on `fixtures/` with that language's real server installed (CI: `Language smoke`, pinned versions).
- `pnpm test`, `pnpm e2e`, `pnpm lint`, `pnpm format:check`, `pnpm knip`, `pnpm typecheck`,
`pnpm gen:bindings`.
- `pnpm perf` (UI interaction latency / frames / leaks), `pnpm perf:rust` (criterion
+7 -4
View File
@@ -16,8 +16,7 @@ Project layout, coding rules, commands and checks are in [AGENTS.md](AGENTS.md).
## Pull requests
- One issue per PR, linked with `Closes #123`. Feature PRs need a `planned` issue.
Docs, CI and pure refactors may skip the issue.
- One problem per PR. If it resolves an issue, link it with `Closes #123`.
- Reproduce the problem first, or write a test that fails before the fix.
- Fix the cause, and check whether the same cause breaks anything nearby.
- Keep the diff focused. Unrelated refactors go in their own PR.
@@ -31,5 +30,9 @@ Project layout, coding rules, commands and checks are in [AGENTS.md](AGENTS.md).
- If a change needs a product decision you can't make, ask in the issue and stop
rather than guessing.
A check on each PR adds `needs-info` when the issue link or the verification
section is missing, and removes it once they're filled in.
Plugins: pull requests here add or change official plugins (`plugins/official/`)
and the plugin API. Community plugins live in their authors' repositories and
are listed through [l0ng-ai/openidea-plugins](https://github.com/l0ng-ai/openidea-plugins);
see [`plugins/README.md`](plugins/README.md) § 8.1.
A check on each PR adds `needs-info` when the verification section is missing, and removes it once it's filled in.
Generated
+38 -36
View File
@@ -4038,7 +4038,7 @@ checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
[[package]]
name = "ide-agent"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"dunce",
"encoding_rs",
@@ -4061,7 +4061,7 @@ dependencies = [
[[package]]
name = "ide-analysis"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ast-grep-core",
"ide-core",
@@ -4075,7 +4075,7 @@ dependencies = [
[[package]]
name = "ide-build"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"dirs 6.0.0",
"dunce",
@@ -4090,7 +4090,7 @@ dependencies = [
[[package]]
name = "ide-clipboard"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"objc2 0.6.5",
"objc2-app-kit 0.3.2",
@@ -4099,7 +4099,7 @@ dependencies = [
[[package]]
name = "ide-core"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"dunce",
"encoding_rs",
@@ -4116,7 +4116,7 @@ dependencies = [
[[package]]
name = "ide-coverage"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"ide-run",
@@ -4130,7 +4130,7 @@ dependencies = [
[[package]]
name = "ide-dap"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"ide-jsonrpc",
@@ -4148,7 +4148,7 @@ dependencies = [
[[package]]
name = "ide-db"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"bb8",
"dirs 6.0.0",
@@ -4174,7 +4174,7 @@ dependencies = [
[[package]]
name = "ide-docker"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"serde",
@@ -4187,7 +4187,7 @@ dependencies = [
[[package]]
name = "ide-extension"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"base64 0.22.1",
"blake3",
@@ -4207,7 +4207,7 @@ dependencies = [
[[package]]
name = "ide-fonts"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"objc2-core-foundation",
"objc2-core-text",
@@ -4219,7 +4219,7 @@ dependencies = [
[[package]]
name = "ide-github"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"ide-vcs",
@@ -4232,7 +4232,7 @@ dependencies = [
[[package]]
name = "ide-host"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"base64 0.22.1",
"criterion",
@@ -4276,6 +4276,7 @@ dependencies = [
"regex-syntax",
"reqwest",
"rustix",
"semver",
"serde",
"serde_json",
"specta",
@@ -4292,7 +4293,7 @@ dependencies = [
[[package]]
name = "ide-http"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"parking_lot",
@@ -4310,7 +4311,7 @@ dependencies = [
[[package]]
name = "ide-index"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"blake3",
"criterion",
@@ -4332,7 +4333,7 @@ dependencies = [
[[package]]
name = "ide-jsonrpc"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"encoding_rs",
"futures",
@@ -4340,6 +4341,7 @@ dependencies = [
"parking_lot",
"serde",
"serde_json",
"tempfile",
"thiserror 2.0.21",
"tokio",
"tokio-util",
@@ -4348,7 +4350,7 @@ dependencies = [
[[package]]
name = "ide-language"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"blake3",
"criterion",
@@ -4404,7 +4406,7 @@ dependencies = [
[[package]]
name = "ide-lsp"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"dunce",
"futures",
@@ -4430,7 +4432,7 @@ dependencies = [
[[package]]
name = "ide-notebook"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"futures",
"ide-core",
@@ -4453,7 +4455,7 @@ dependencies = [
[[package]]
name = "ide-plugin"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"blake3",
"ide-core",
@@ -4473,7 +4475,7 @@ dependencies = [
[[package]]
name = "ide-profiler"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"ide-run",
@@ -4486,7 +4488,7 @@ dependencies = [
[[package]]
name = "ide-remote"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"globset",
"ide-core",
@@ -4503,7 +4505,7 @@ dependencies = [
[[package]]
name = "ide-run"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"dunce",
"encoding_rs",
@@ -4526,7 +4528,7 @@ dependencies = [
[[package]]
name = "ide-sdk"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"dunce",
"ide-core",
@@ -4545,7 +4547,7 @@ dependencies = [
[[package]]
name = "ide-search"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"criterion",
"globset",
@@ -4565,7 +4567,7 @@ dependencies = [
[[package]]
name = "ide-spell"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"parking_lot",
@@ -4578,7 +4580,7 @@ dependencies = [
[[package]]
name = "ide-ssh"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"dirs 6.0.0",
"flate2",
@@ -4597,7 +4599,7 @@ dependencies = [
[[package]]
name = "ide-syntax"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"ide-language",
@@ -4612,7 +4614,7 @@ dependencies = [
[[package]]
name = "ide-terminal"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"libc",
@@ -4625,7 +4627,7 @@ dependencies = [
[[package]]
name = "ide-text"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"criterion",
"ide-core",
@@ -4637,7 +4639,7 @@ dependencies = [
[[package]]
name = "ide-tools"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"flate2",
"hex",
@@ -4658,7 +4660,7 @@ dependencies = [
[[package]]
name = "ide-vcs"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"dunce",
"encoding_rs",
@@ -4677,7 +4679,7 @@ dependencies = [
[[package]]
name = "ide-vfs"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"chardetng",
"dunce",
@@ -4699,7 +4701,7 @@ dependencies = [
[[package]]
name = "ide-web"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"open",
@@ -4715,7 +4717,7 @@ dependencies = [
[[package]]
name = "ide-workspace"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"blake3",
"dunce",
@@ -6187,7 +6189,7 @@ dependencies = [
[[package]]
name = "openidea"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"base64 0.22.1",
"blake3",
+1 -1
View File
@@ -5,7 +5,7 @@ members = ["crates/*", "src-tauri"]
exclude = ["sdk", "plugins"]
[workspace.package]
version = "0.1.35"
version = "0.1.36"
edition = "2024"
rust-version = "1.96"
license = "Apache-2.0"
+12
View File
@@ -14,6 +14,8 @@
&nbsp;·&nbsp;
<a href="#benchmark">Benchmark</a>
&nbsp;·&nbsp;
<a href="#plugins">Plugins</a>
&nbsp;·&nbsp;
<a href="https://discord.gg/2Zy4yn7gP">Discord</a>
</p>
@@ -45,6 +47,16 @@ IDEA analyses Python more deeply, so "ready" doesn't mean the same work on
both sides. Both are measured from the outside, the same way: see
[`perf/compare`](perf/compare) to run it on your machine and projects.
## Plugins
Plugins add actions, completion, quick fixes, tool windows, custom editors,
themes, file icons and languages. They run as sandboxed WebAssembly components;
themes, icon themes and languages need no code at all.
Install them from Settings › Plugins, or start your own with
`openidea-plugin new` — see [Writing OpenIDEA plugins](plugins/README.md) and
the [official plugins](plugins/official). The plugin API is still a preview.
## License
[Apache 2.0](LICENSE). Bundled third-party assets are listed in [`THIRD_PARTY_LICENSES.md`](THIRD_PARTY_LICENSES.md).
+3 -3
View File
@@ -2,12 +2,12 @@
## Project Lombok
`plugins/lombok/lombok-1.18.48-openidea.2.jar` contains Project Lombok v1.18.48 with the Equinox
`java/lombok/lombok-1.18.48-openidea.2.jar` contains Project Lombok v1.18.48 with the Equinox
classloader fix from [projectlombok/lombok#4059](https://github.com/projectlombok/lombok/pull/4059)
and the ECJ constructor-call fix from [projectlombok/lombok#4089](https://github.com/projectlombok/lombok/pull/4089).
Copyright (C) 2009-2021 The Project Lombok Authors. MIT license and bundled component
notices are retained in [plugins/lombok/LICENSE](plugins/lombok/LICENSE) and inside the jar.
Source tag, patch commits and build instructions: [plugins/lombok/README.md](plugins/lombok/README.md).
notices are retained in [java/lombok/LICENSE](java/lombok/LICENSE) and inside the jar.
Source tag, patch commits and build instructions: [java/lombok/README.md](java/lombok/README.md).
## JetBrains IntelliJ Platform icons
+13 -5
View File
@@ -58,6 +58,8 @@ pub(crate) const LIMITS_UPDATE: &str = "openidea_limits";
const MAX_EMBED: u64 = 256 * 1024;
/// How long a probe waits for the agent to answer `initialize`.
const PROBE_TIMEOUT: Duration = Duration::from_secs(30);
/// How long deleting a task waits for its agent to exit before removing the worktree.
const EXIT_WAIT: Duration = Duration::from_secs(5);
/// The agent tasks of one project.
pub struct AgentManager {
@@ -839,7 +841,7 @@ impl AgentManager {
/// Forgets the task; with `remove_worktree` also deletes its worktree (the branch stays).
pub async fn delete(&self, id: &str, remove_worktree: bool) -> IdeResult<()> {
let task = self.task(id)?;
task.kill();
let live = task.kill();
let (isolation, cwd, root) = {
let s = task.state.lock();
(
@@ -851,6 +853,11 @@ impl AgentManager {
// A copy of the task (a fork) may work in the same worktree.
let shared = self.tasks.read().values().any(|t| t.id != id && t.state.lock().task.cwd.0 == cwd);
if remove_worktree && !shared && isolation == Isolation::Worktree && cwd.exists() {
// On Windows the agent's processes lock the folder (their working directory,
// open files) until they have exited.
if let Some(peer) = live {
let _ = tokio::time::timeout(EXIT_WAIT, peer.exited()).await;
}
let worktrees = self.shared.paths.worktrees_dir.clone();
blocking(move || worktree::remove(&root, &worktrees, &cwd)).await?;
}
@@ -1605,11 +1612,12 @@ impl Task {
}
}
fn kill(&self) {
/// Stops the agent; returns its connection, to wait for its process to exit.
fn kill(&self) -> Option<acp::Conn> {
self.cancel_permissions();
if let Some(live) = self.live.lock().take() {
live.peer.kill();
}
let live = self.live.lock().take()?;
live.peer.kill();
Some(live.peer)
}
}
+111
View File
@@ -1109,3 +1109,114 @@ async fn idle_agents_stop_and_resume_their_session_on_the_next_message() {
assert!(live(&mgr, &task.id));
mgr.shutdown();
}
/// Deleting a task whose agent still runs in the worktree removes both and leaves no
/// process behind, whether it is waiting for permission or idle without session/load.
#[tokio::test(flavor = "multi_thread")]
async fn deleting_a_task_waits_for_its_agent_to_exit_before_removing_the_worktree() {
for no_load in [false, true] {
delete_running_agent(no_load).await;
}
}
async fn delete_running_agent(no_load: bool) {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
let git = |args: &[&str]| {
let out = std::process::Command::new("git")
.args(["-c", "user.name=t", "-c", "user.email=t@t", "-c", "commit.gpgsign=false"])
.args(args)
.current_dir(&root)
.output()
.unwrap();
assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr));
};
git(&["init", "-q", "-b", "main"]);
std::fs::write(root.join("file.txt"), "original\n").unwrap();
git(&["add", "."]);
git(&["commit", "-qm", "initial"]);
let agent = Arc::new(FakeAgent { no_load, ..FakeAgent::default() });
let pids: Arc<Mutex<Vec<u32>>> = Arc::default();
let (fake, started) = (agent.clone(), pids.clone());
// The fake agent, served on behalf of a real process that exits only when killed.
let connector: Connector = Arc::new(move |_spec, cwd, client, _stderr| {
#[cfg(unix)]
let mut cmd = {
let mut cmd = tokio::process::Command::new("sleep");
cmd.arg("30");
cmd
};
#[cfg(windows)]
let mut cmd = {
let mut cmd = tokio::process::Command::new("cmd");
cmd.args(["/D", "/C", "echo ready>agent-ready & ping -n 30 127.0.0.1 >nul"]);
cmd
};
ide_core::process::prepare(cmd.as_std_mut());
ide_core::process::own_group(cmd.as_std_mut());
cmd.current_dir(cwd).kill_on_drop(true);
let child = cmd.spawn()?;
started.lock().push(child.id().unwrap());
let (a, b) = duplex(1 << 20);
let (ar, aw) = tokio::io::split(a);
let (br, bw) = tokio::io::split(b);
// `connect_with_child` frames with Content-Length.
fake.servers.lock().push(Peer::connect(br, bw, fake.clone()));
Ok(Peer::connect_with_child(ar, aw, client, child))
});
let mgr = AgentManager::with_connector(
AgentPaths {
project_root: root.clone(),
store_dir: root.join("store"),
worktrees_dir: root.join("worktrees"),
models_file: root.join("models.json"),
custom_agents_dir: None,
home_dir: None,
},
tokio::runtime::Handle::current(),
connector,
);
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text(if no_load { "again" } else { "Fix the lexer" }),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Worktree,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, if no_load { AgentStatus::Ready } else { AgentStatus::NeedsInput }).await;
let pid = *pids.lock().last().expect("the agent was started");
#[cfg(windows)]
{
tokio::time::timeout(Duration::from_secs(5), async {
while !task.cwd.0.join("agent-ready").exists() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await
.unwrap();
assert_eq!(
std::fs::rename(&task.cwd.0, task.cwd.0.with_extension("moved")).unwrap_err().raw_os_error(),
Some(32),
"the running agent really locks its worktree"
);
}
mgr.delete(&task.id, true).await.unwrap();
assert!(!task.cwd.0.exists());
// Killed and reaped: not even a zombie is left.
#[cfg(unix)]
let alive = std::process::Command::new("kill").args(["-0", &pid.to_string()]).status().unwrap().success();
#[cfg(windows)]
let alive = {
let mut cmd = std::process::Command::new("tasklist");
ide_core::process::prepare(&mut cmd).args(["/FI", &format!("PID eq {pid}"), "/FO", "CSV", "/NH"]);
String::from_utf8_lossy(&cmd.output().unwrap().stdout).contains(&format!("\"{pid}\""))
};
assert!(!alive, "the agent (pid {pid}) is still running");
}
+70 -2
View File
@@ -5,7 +5,7 @@ use ide_core::{IdeError, IdeResult, IpcPath};
use ide_vcs::{ChangeKind, Repo};
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use std::time::UNIX_EPOCH;
use std::time::{Duration, UNIX_EPOCH};
pub(crate) struct Created {
pub path: PathBuf,
@@ -65,7 +65,27 @@ pub(crate) fn remove(project_root: &Path, worktrees_dir: &Path, path: &Path) ->
if !path.starts_with(worktrees_dir) || path == worktrees_dir {
return Err(err);
}
std::fs::remove_dir_all(path).map_err(|e| IdeError::internal(e.to_string()))
retrying(REMOVE_ATTEMPTS, REMOVE_PAUSE, || std::fs::remove_dir_all(path))
.map_err(|e| IdeError::internal(format!("{}: {e}", path.display())))
}
/// On Windows a folder can't be deleted while a process still has a file in it open
/// (os error 32): an agent's helper that outlived it, an antivirus scan, the search indexer.
/// Those let go within moments, so removing is retried for a few seconds.
const REMOVE_ATTEMPTS: u32 = 10;
const REMOVE_PAUSE: Duration = Duration::from_millis(300);
fn retrying(attempts: u32, pause: Duration, mut op: impl FnMut() -> std::io::Result<()>) -> std::io::Result<()> {
for _ in 1..attempts {
match op() {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => std::thread::sleep(pause),
_ => return Ok(()),
}
}
match op() {
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()),
r => r,
}
}
/// Files under `cwd` that differ from `base` (untracked files included), with line counts.
@@ -149,6 +169,54 @@ fn unique_slug(base: &str, taken: impl Fn(&str) -> bool) -> String {
mod tests {
use super::*;
#[test]
fn removing_is_retried_while_a_file_is_still_in_use() {
use std::io::{Error, ErrorKind};
// A sharing violation as Windows reports it (os error 32), a vanished folder, other errors.
let in_use = || Error::from_raw_os_error(32);
for (fails, outcome) in [(0, true), (3, true), (REMOVE_ATTEMPTS - 1, true), (REMOVE_ATTEMPTS, false)] {
let mut calls = 0;
let r = retrying(REMOVE_ATTEMPTS, Duration::ZERO, || {
calls += 1;
if calls <= fails { Err(in_use()) } else { Ok(()) }
});
assert_eq!(r.is_ok(), outcome, "{fails} failures");
assert_eq!(calls, (fails + 1).min(REMOVE_ATTEMPTS));
}
let mut calls = 0;
let gone = retrying(REMOVE_ATTEMPTS, Duration::ZERO, || {
calls += 1;
if calls == 1 { Err(in_use()) } else { Err(Error::from(ErrorKind::NotFound)) }
});
assert!(gone.is_ok(), "deleted meanwhile: done");
assert_eq!(calls, 2);
}
#[cfg(windows)]
#[test]
fn removing_waits_for_a_windows_file_lock_to_be_released() {
use std::os::windows::fs::OpenOptionsExt;
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
std::fs::write(repo.join("a.txt"), "one\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
let worktrees = tmp.path().join("wt");
let created = create(&repo, &worktrees, "Locked file", None).unwrap();
let file = std::fs::OpenOptions::new().read(true).share_mode(0).open(created.path.join("a.txt")).unwrap();
assert_eq!(std::fs::remove_file(created.path.join("a.txt")).unwrap_err().raw_os_error(), Some(32));
let release = std::thread::spawn(move || {
std::thread::sleep(Duration::from_millis(800));
drop(file);
});
let result = remove(&repo, &worktrees, &created.path);
release.join().unwrap();
result.unwrap();
assert!(!created.path.exists());
}
#[test]
fn slugs_are_branch_safe() {
assert_eq!(slugify("Fix lexer UTF-16 columns!"), "fix-lexer-utf-16-columns");
+25 -1
View File
@@ -315,7 +315,10 @@ pub fn sync_maven(
for (n, top) in model.projects.iter().filter(|p| !p.aggregated).enumerate() {
let effective = cache_dir.join(format!("effective-pom-{n}.xml"));
let _ = std::fs::remove_file(&effective);
let mut args: Vec<String> = vec!["-B".into(), "-fae".into(), "-Dstyle.color=never".into()];
// `-T1` keeps mvnd serial: its default parallel build interleaves the
// `dependency:tree` output of concurrent modules, which the sync parses
// as one stream. mvn is serial anyway, so this only pins the daemon.
let mut args: Vec<String> = vec!["-B".into(), "-fae".into(), "-Dstyle.color=never".into(), "-T1".into()];
args.extend(options.maven_options.iter().cloned());
if options.offline {
args.push("-o".into());
@@ -680,4 +683,25 @@ mod tests {
assert_eq!(error_summary(text), "Failed to execute goal");
assert_eq!(error_summary("one\nlast line\n\n"), "last line");
}
#[cfg(unix)]
#[test]
fn maven_sync_runs_the_daemon_command_serially() {
use std::os::unix::fs::PermissionsExt;
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path().join("p");
std::fs::create_dir_all(&root).unwrap();
std::fs::write(root.join("pom.xml"), "<project><groupId>g</groupId><artifactId>a</artifactId></project>")
.unwrap();
let launcher = tmp.path().join("fake-mvnd");
std::fs::write(&launcher, "#!/bin/sh\nprintf '%s\\n' \"$@\" > mvnd-args\n").unwrap();
std::fs::set_permissions(&launcher, std::fs::Permissions::from_mode(0o755)).unwrap();
let options =
SyncOptions { maven_command: Some(launcher.to_string_lossy().into_owned()), ..Default::default() };
sync_maven(&root, &tmp.path().join("cache"), &options, &CancellationToken::new(), &|_| {}).unwrap();
let args = std::fs::read_to_string(root.join("mvnd-args")).unwrap();
// `-T1` pins the daemon serial so the dependency trees stay parseable.
assert!(args.contains("-T1"), "args: {args}");
assert!(args.contains("help:effective-pom"), "args: {args}");
}
}
+1 -1
View File
@@ -22,4 +22,4 @@ tempfile.workspace = true
libc = "0.2"
[target.'cfg(windows)'.dependencies]
windows-sys = { version = "0.61", features = ["Win32_Globalization", "Win32_Security", "Win32_System_Registry", "Win32_Foundation"] }
windows-sys = { version = "0.61", features = ["Win32_Globalization", "Win32_Security", "Win32_System_Registry", "Win32_Foundation", "Win32_System_Diagnostics_ToolHelp", "Win32_System_Threading"] }
+23
View File
@@ -2,6 +2,18 @@
//! treats as nonexistent — hidden from the project view and navigation, never
//! indexed or searched, never offered as unversioned changes.
use std::path::Path;
/// Whether `dir` lies in a Git (or jj) working tree: it or an ancestor holds `.git` / `.jj`.
///
/// Walkers pass it to `ignore::WalkBuilder::require_git`: inside a repository, `.gitignore`
/// rules then stop at a nested repository's root as in git, so an outer repository's
/// `.gitignore` does not hide a project that is its own repository. A folder outside any
/// repository still honours its `.gitignore`.
pub fn inside_git_repo(dir: &Path) -> bool {
dir.ancestors().any(|d| d.join(".git").exists() || d.join(".jj").exists())
}
/// IDEA's default list, plus `.jj` and the Windows shell files.
pub const DEFAULT_IGNORED_NAMES: &[&str] = &[
"*.pyc",
@@ -103,4 +115,15 @@ mod tests {
assert!(m.is_ignored_path("pkg/__pycache__/mod.cpython-312.pyc"));
assert!(!m.is_ignored_path("src/main.rs"));
}
#[test]
fn finds_the_repository_at_or_above_a_directory() {
let dir = tempfile::tempdir().unwrap();
let sub = dir.path().join("a/b");
std::fs::create_dir_all(&sub).unwrap();
assert!(!inside_git_repo(&sub));
std::fs::create_dir(dir.path().join("a/.git")).unwrap();
assert!(inside_git_repo(&sub));
assert!(inside_git_repo(&dir.path().join("a")));
}
}
+101 -15
View File
@@ -43,19 +43,21 @@ pub fn own_group(cmd: &mut Command) -> &mut Command {
/// Kills the child and everything it started, then reaps it.
///
/// - Unix: `SIGKILL` to its process group, so spawn it with [`own_group`].
/// - Windows: `taskkill /T /F`, which walks the tree from the child's process id.
/// - Windows: `taskkill /T /F`, which walks the tree from the child's process id, then a
/// wait for what the child started to be gone.
pub fn kill_tree(child: &mut Child) {
#[cfg(unix)]
signal_group(child, libc::SIGKILL);
#[cfg(windows)]
taskkill_tree(child);
taskkill_tree_of(child.id());
let _ = child.kill();
let _ = child.wait();
}
/// [`kill_tree`] for a child known by its process id (a `tokio` child): `SIGKILL` to its
/// process group (spawn it with [`own_group`]) or `taskkill /T /F`. Call it before the child
/// is reaped, so the id still names it. The child itself is left to its owner to kill and reap.
/// process group (spawn it with [`own_group`]) or `taskkill /T /F` and a wait for what the
/// child started to be gone. Call it before the child is reaped, so the id still names it.
/// The child itself is left to its owner to kill and reap.
pub fn kill_tree_of(pid: u32) {
#[cfg(unix)]
if let Ok(pid) = libc::pid_t::try_from(pid) {
@@ -65,15 +67,7 @@ pub fn kill_tree_of(pid: u32) {
}
}
#[cfg(windows)]
{
let mut taskkill = Command::new("taskkill");
prepare(&mut taskkill)
.args(["/T", "/F", "/PID", &pid.to_string()])
.stdin(Stdio::null())
.stdout(Stdio::null())
.stderr(Stdio::null());
let _ = taskkill.status();
}
taskkill_tree_of(pid);
#[cfg(not(any(unix, windows)))]
let _ = pid;
}
@@ -106,15 +100,107 @@ fn signal_group(child: &Child, signal: libc::c_int) {
}
}
/// How long killing a tree waits for the processes the child started to be gone.
#[cfg(windows)]
fn taskkill_tree(child: &Child) {
const TREE_EXIT_WAIT: Duration = Duration::from_secs(5);
/// `taskkill /T /F` from `pid`, then waits (up to [`TREE_EXIT_WAIT`]) until what the child
/// started is gone: termination is asynchronous, and until a process is gone the files it
/// holds open can't be opened or deleted. The child itself is left to its owner to reap.
#[cfg(windows)]
fn taskkill_tree_of(pid: u32) {
let started = windows_tree::descendants(pid);
let mut taskkill = Command::new("taskkill");
prepare(&mut taskkill)
.args(["/T", "/F", "/PID", &child.id().to_string()])
.args(["/T", "/F", "/PID", &pid.to_string()])
.stdin(Stdio::null())
.stdout(Stdio::null())
.stderr(Stdio::null());
let _ = taskkill.status();
windows_tree::wait_all(&started, TREE_EXIT_WAIT);
}
#[cfg(windows)]
mod windows_tree {
use std::collections::HashSet;
use std::os::windows::io::{AsRawHandle, FromRawHandle, OwnedHandle};
use std::time::{Duration, Instant};
use windows_sys::Win32::Foundation::{FILETIME, INVALID_HANDLE_VALUE};
use windows_sys::Win32::System::Diagnostics::ToolHelp::{
CreateToolhelp32Snapshot, PROCESSENTRY32W, Process32FirstW, Process32NextW, TH32CS_SNAPPROCESS,
};
use windows_sys::Win32::System::Threading::{
GetProcessTimes, OpenProcess, PROCESS_QUERY_LIMITED_INFORMATION, PROCESS_SYNCHRONIZE, WaitForSingleObject,
};
/// Every process `root` started, directly or not, opened so it can be waited for. A
/// parent's id is reused once it is gone, so a process counts only if it started after
/// its parent.
pub(super) fn descendants(root: u32) -> Vec<OwnedHandle> {
let Some(root_started) = open(root).as_ref().and_then(started) else { return Vec::new() };
let processes = processes();
let mut seen = HashSet::from([root]);
let mut parents = vec![(root, root_started)];
let mut found = Vec::new();
while let Some((parent, parent_started)) = parents.pop() {
for &(pid, ppid) in &processes {
if ppid != parent || !seen.insert(pid) {
continue;
}
let Some(handle) = open(pid) else { continue };
if let Some(at) = started(&handle).filter(|&at| at >= parent_started) {
parents.push((pid, at));
found.push(handle);
}
}
}
found
}
pub(super) fn wait_all(processes: &[OwnedHandle], within: Duration) {
let until = Instant::now() + within;
for process in processes {
let left = until.saturating_duration_since(Instant::now());
// SAFETY: a live process handle opened with SYNCHRONIZE access.
unsafe { WaitForSingleObject(process.as_raw_handle(), left.as_millis().min(u32::MAX as u128 - 1) as u32) };
}
}
/// (process id, parent process id) of every running process.
fn processes() -> Vec<(u32, u32)> {
let mut out = Vec::new();
// SAFETY: a snapshot handle owned here and an entry sized as the API requires.
unsafe {
let snapshot = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
if snapshot.is_null() || snapshot == INVALID_HANDLE_VALUE {
return out;
}
let snapshot = OwnedHandle::from_raw_handle(snapshot);
let mut entry = PROCESSENTRY32W { dwSize: size_of::<PROCESSENTRY32W>() as u32, ..Default::default() };
let mut more = Process32FirstW(snapshot.as_raw_handle(), &mut entry);
while more != 0 {
out.push((entry.th32ProcessID, entry.th32ParentProcessID));
more = Process32NextW(snapshot.as_raw_handle(), &mut entry);
}
}
out
}
fn open(pid: u32) -> Option<OwnedHandle> {
// SAFETY: plain call; a non-null result is a process handle owned by the caller.
let handle = unsafe { OpenProcess(PROCESS_SYNCHRONIZE | PROCESS_QUERY_LIMITED_INFORMATION, 0, pid) };
// SAFETY: as above.
(!handle.is_null()).then(|| unsafe { OwnedHandle::from_raw_handle(handle) })
}
/// When the process started, in 100 ns ticks since 1601.
fn started(process: &OwnedHandle) -> Option<u64> {
let (mut created, mut exited, mut kernel, mut user) =
(FILETIME::default(), FILETIME::default(), FILETIME::default(), FILETIME::default());
// SAFETY: a live process handle opened with query access; four out-parameters.
let ok = unsafe { GetProcessTimes(process.as_raw_handle(), &mut created, &mut exited, &mut kernel, &mut user) };
(ok != 0).then(|| (u64::from(created.dwHighDateTime) << 32) | u64::from(created.dwLowDateTime))
}
}
/// Gives `cmd` the host's environment back when the IDE runs from an AppImage. `AppRun` and
+20
View File
@@ -713,6 +713,9 @@ impl Default for PythonSettings {
pub struct PhpSettings {
/// `php` CLI executable; empty = the one on `PATH`.
pub interpreter: String,
/// PHP version the language server checks the code against (`7.4`); empty = from
/// `composer.json` (`config.platform.php`, else the lowest version `require.php` allows).
pub language_level: String,
}
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize, Type)]
@@ -861,6 +864,12 @@ pub struct MavenSettings {
pub user_settings: String,
/// Local repository (`-Dmaven.repo.local`); empty = from `settings.xml`.
pub local_repository: String,
/// Run Maven through the Maven Daemon (`mvnd`) for faster builds. mvnd embeds
/// its own Maven, so `home` and the project's `mvnw` are bypassed when it is on.
pub daemon: bool,
/// Maven Daemon home (or its `mvnd` executable); empty = `$MVND_HOME`, else
/// `mvnd` on `PATH` / common install dirs. Only used when `daemon` is on.
pub mvnd_home: String,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
@@ -1092,6 +1101,17 @@ mod maven_linkage_tests {
assert!(encoded.contains("unlinkedProjects"));
assert_eq!(serde_json::from_str::<MavenSettings>(&encoded).unwrap(), settings);
}
#[test]
fn daemon_settings_round_trip() {
// Old settings files predate the daemon fields; `serde(default)` keeps them loadable.
let old: MavenSettings = serde_json::from_str(r#"{"home":"/opt/maven"}"#).unwrap();
assert!(!old.daemon && old.mvnd_home.is_empty());
let settings = MavenSettings { daemon: true, mvnd_home: "/opt/mvnd".into(), ..old };
let encoded = serde_json::to_string(&settings).unwrap();
assert!(encoded.contains("\"daemon\":true") && encoded.contains("mvndHome"));
assert_eq!(serde_json::from_str::<MavenSettings>(&encoded).unwrap(), settings);
}
}
#[cfg(test)]
+1
View File
@@ -923,6 +923,7 @@ fn godot_registry(port: u16, command: &str, args: &[&str]) -> Arc<LanguageRegist
ui_mode: None,
framework_ui_modes: Vec::new(),
formatter: None,
scratch_template: None,
tree_sitter: None,
language_servers: vec![],
debugger: Some(ide_language::DebuggerConfig {
+2 -2
View File
@@ -27,8 +27,8 @@ mod when;
pub use capability::{Capability, CapabilityInfo, Risk, host_matches};
pub use index::ContributionIndex;
pub use manifest::{
ActivationEvent, Capabilities, ENGINE_VERSION, EVENT_KINDS, LEGACY_API, MANIFEST_FILE, Manifest, check_relative,
engine_compatible, validate_item_id, validate_plugin_id, validate_point_id,
ActivationEvent, CATEGORIES, Capabilities, ENGINE_VERSION, EVENT_KINDS, LEGACY_API, MANIFEST_FILE, MAX_CATEGORIES,
Manifest, check_relative, engine_compatible, validate_item_id, validate_plugin_id, validate_point_id,
};
pub use point::{Commit, Disposable, ExtensionPoint, MAX_PACKAGE_FILE, Package, Placement, Prepared, decode};
pub use registry::{ContributionInfo, ExtensionRegistry, PreparedSet};
+66
View File
@@ -35,6 +35,11 @@ pub const MANIFEST_FILE: &str = "plugin.toml";
pub const ENGINE_VERSION: &str = "1.0.0";
/// The 0.1 API, still accepted for contribution-only packages and 0.1 components.
pub const LEGACY_API: &str = "0.1";
/// Marketplace categories, in the order they are shown.
pub const CATEGORIES: &[&str] =
&["languages", "themes", "icon-themes", "linters", "vcs", "ai", "frameworks", "tools", "other"];
/// Categories a plugin is listed under; later ones are ignored.
pub const MAX_CATEGORIES: usize = 3;
#[derive(Clone, Debug, PartialEq)]
pub struct Manifest {
@@ -47,6 +52,8 @@ pub struct Manifest {
pub license: Option<String>,
/// Release notes of this version (Settings › Plugins › What's New).
pub change_notes: Option<String>,
/// Marketplace categories as declared (see [`Manifest::categories`]).
pub declared_categories: Vec<String>,
/// `engines.openidea` (a semver requirement), or the legacy `openidea-api`.
pub engine: String,
/// Written in the 0.1 format.
@@ -166,6 +173,8 @@ struct Raw {
#[serde(default)]
change_notes: Option<String>,
#[serde(default)]
categories: Vec<String>,
#[serde(default)]
engines: Option<Engines>,
/// 0.1 format.
#[serde(default)]
@@ -243,6 +252,7 @@ impl Manifest {
homepage: raw.homepage,
license: raw.license,
change_notes: raw.change_notes,
declared_categories: raw.categories,
engine,
legacy,
wasm: raw.wasm,
@@ -290,6 +300,9 @@ impl Manifest {
return Err(format!("duplicate activation event {event}"));
}
}
for c in &self.declared_categories {
validate_item_id(c, "category")?;
}
for (point, decls) in &self.contributes {
validate_point_id(point)?;
if !matches!(decls, toml::Value::Array(_) | toml::Value::Table(_)) {
@@ -325,6 +338,39 @@ impl Manifest {
}
}
impl Manifest {
/// Its marketplace categories, in [`CATEGORIES`] order: the first
/// [`MAX_CATEGORIES`] declared that this IDE knows; without any, those its
/// contributions imply (themes, icon themes, languages), else `other`.
pub fn categories(&self) -> Vec<String> {
let pick = |f: &dyn Fn(&str) -> bool| CATEGORIES.iter().filter(|c| f(c)).map(|c| c.to_string()).collect();
let mut known: Vec<&str> = Vec::new();
for d in &self.declared_categories {
if CATEGORIES.contains(&d.as_str()) && !known.contains(&d.as_str()) && known.len() < MAX_CATEGORIES {
known.push(d);
}
}
let mut out: Vec<String> = pick(&|c| known.contains(&c));
if out.is_empty() {
out = pick(&|c| match c {
"themes" => self.contributes.contains_key("themes"),
"icon-themes" => self.contributes.contains_key("iconThemes"),
"languages" => self.contributes.contains_key("languages"),
_ => false,
});
}
if out.is_empty() {
out.push("other".into());
}
out
}
/// Declared categories this IDE doesn't know (ignored).
pub fn unknown_categories(&self) -> Vec<&str> {
self.declared_categories.iter().map(String::as_str).filter(|c| !CATEGORIES.contains(c)).collect()
}
}
/// Whether this host can load a plugin written for `engine`.
pub fn engine_compatible(engine: &str, legacy: bool) -> bool {
if legacy {
@@ -417,6 +463,26 @@ mod tests {
assert!(ActivationEvent::parse("on-event:scheduled").is_err(), "declared with on-schedule");
}
#[test]
fn categories_are_declared_or_implied() {
let head = "id = \"acme.x\"\nname = \"X\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"\n";
let m = |rest: &str| Manifest::parse(&format!("{head}{rest}"));
// Declared, in the catalogue's order; unknown ones (a newer IDE's) are ignored.
let declared = m("categories = [\"tools\", \"vcs\", \"robots\"]\n").unwrap();
assert_eq!(declared.categories(), ["vcs", "tools"]);
assert_eq!(declared.unknown_categories(), ["robots"]);
// None declared: implied by the contributions, else `other`.
let theme = m("[[contributes.themes]]\nid = \"t\"\n[[contributes.languages]]\nid = \"l\"\n").unwrap();
assert_eq!(theme.categories(), ["languages", "themes"]);
let declared_wins = m("categories = [\"tools\"]\n[[contributes.languages]]\nid = \"l\"\n").unwrap();
assert_eq!(declared_wins.categories(), ["tools"]);
assert_eq!(m("").unwrap().categories(), ["other"]);
// At most three: the first ones declared.
let many = m("categories = [\"vcs\", \"ai\", \"vcs\", \"tools\", \"themes\"]\n").unwrap();
assert_eq!(many.categories(), ["vcs", "ai", "tools"]);
assert!(m("categories = [\"Bad Name\"]\n").is_err());
}
#[test]
fn parses_the_02_format() {
let m = Manifest::parse(FULL).unwrap();
+513 -23
View File
@@ -9,9 +9,19 @@
//! proves who published the bytes; whether to trust that publisher is the
//! index's call.
//! - **Index** (`index.json`): the publishers and their keys (the trust root),
//! the published plugins, and revocations. A publisher owns the id prefix
//! `<publisher>.`; a revoked key invalidates everything it signed, a
//! revoked plugin version is disabled where it is installed.
//! the published plugin versions, and revocations. A publisher owns the id
//! prefix `<publisher>.`; a revoked key invalidates everything it signed, a
//! revoked plugin version is disabled where it is installed. A plugin may be
//! listed in several versions, each with the plugin API it needs: the IDE
//! offers the newest one it can load.
//! - **Key continuity**: an installed plugin updates silently only to a
//! package signed with the same key, or with a key the installed one
//! endorsed ([`Endorsement`], a publisher rotating keys); any other signer
//! is the user's call. An IDE can pin a publisher to keys it was built with
//! ([`Index::pin_publisher`]): the index then can't replace them.
//! - **Format**: fields are only ever added (IDEs ignore those they don't
//! know); `format` grows only for changes older IDEs can't read, and they
//! still apply such an index's revocations ([`Index::parse`]).
use crate::MANIFEST_FILE;
use base64::Engine as _;
@@ -40,6 +50,24 @@ fn signed_message(id: &str, version: &str, digest: &str) -> Vec<u8> {
format!("openidea-plugin/1\n{id}\n{version}\n{digest}").into_bytes()
}
fn endorsement_message(publisher: &str, key: &str) -> Vec<u8> {
format!("openidea-key/1\n{publisher}\n{key}").into_bytes()
}
/// Whether `signature` (base64) is `key`'s (base64) Ed25519 signature of
/// `message`; malformed keys and signatures are errors.
fn verify_ed25519(key: &str, message: &[u8], signature: &str) -> Result<bool, String> {
let key: [u8; 32] = B64
.decode(key)
.ok()
.and_then(|b| b.try_into().ok())
.ok_or_else(|| "signature: malformed public key".to_owned())?;
let key = VerifyingKey::from_bytes(&key).map_err(|_| "signature: invalid public key".to_owned())?;
let sig: [u8; 64] =
B64.decode(signature).ok().and_then(|b| b.try_into().ok()).ok_or_else(|| "signature: malformed".to_owned())?;
Ok(key.verify_strict(message, &Signature::from_bytes(&sig)).is_ok())
}
/// `SIGNATURE.json` of a signed package.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
@@ -74,6 +102,12 @@ impl PublisherKey {
B64.encode(self.0.verifying_key().to_bytes())
}
/// Vouches for `publisher`'s new key `key` (base64), the one that follows this one.
pub fn endorse(&self, publisher: &str, key: &str) -> Endorsement {
let sig = self.0.sign(&endorsement_message(publisher, key));
Endorsement { key: self.public_base64(), signature: B64.encode(sig.to_bytes()) }
}
pub fn sign(&self, publisher: &str, id: &str, version: &str, digest: &str) -> PackageSignature {
let sig = self.0.sign(&signed_message(id, version, digest));
PackageSignature {
@@ -88,19 +122,11 @@ impl PackageSignature {
/// Whether the signature is valid for `id`, `version` and `digest` (the
/// key's validity is checked against an [`Index`]).
pub fn check(&self, id: &str, version: &str, digest: &str) -> Result<(), String> {
let key: [u8; 32] = B64
.decode(&self.key)
.ok()
.and_then(|b| b.try_into().ok())
.ok_or_else(|| "signature: malformed public key".to_owned())?;
let key = VerifyingKey::from_bytes(&key).map_err(|_| "signature: invalid public key".to_owned())?;
let sig: [u8; 64] = B64
.decode(&self.signature)
.ok()
.and_then(|b| b.try_into().ok())
.ok_or_else(|| "signature: malformed".to_owned())?;
key.verify_strict(&signed_message(id, version, digest), &Signature::from_bytes(&sig))
.map_err(|_| "the signature does not match the package (it was changed after signing)".to_owned())
if verify_ed25519(&self.key, &signed_message(id, version, digest), &self.signature)? {
Ok(())
} else {
Err("the signature does not match the package (it was changed after signing)".to_owned())
}
}
}
@@ -242,6 +268,10 @@ pub struct Index {
pub plugins: Vec<IndexEntry>,
#[serde(default)]
pub revoked: Vec<Revocation>,
/// The marketplace moved: its index is at this URL (relative to this
/// one, or absolute), and the IDE reads that one instead.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub moved: Option<String>,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
@@ -253,7 +283,7 @@ pub struct Publisher {
pub keys: Vec<KeyEntry>,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct KeyEntry {
/// Ed25519 public key, base64.
@@ -261,6 +291,27 @@ pub struct KeyEntry {
/// Packages signed with it are no longer trusted.
#[serde(default)]
pub revoked: bool,
/// The publisher's earlier key vouching for this one.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub endorsement: Option<Endorsement>,
}
impl KeyEntry {
pub fn new(key: String) -> Self {
Self { key, ..Self::default() }
}
}
/// A publisher's earlier key vouching for a newer one: plugins installed from
/// packages signed with the earlier key update to ones signed with the newer
/// key without asking the user.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct Endorsement {
/// The endorsing key, base64.
pub key: String,
/// Its Ed25519 signature of `openidea-key/1\n<publisher>\n<endorsed key>`, base64.
pub signature: String,
}
/// A published plugin version.
@@ -281,6 +332,34 @@ pub struct IndexEntry {
pub digest: String,
#[serde(default)]
pub capabilities: Vec<String>,
/// The plugin API versions it loads in (a semver requirement): its
/// manifest's `engines.openidea` and the API its component was built for.
/// Required: an entry without one is offered to no IDE.
#[serde(default)]
pub engine: String,
/// Marketplace categories (`CATEGORIES`); an IDE ignores those it doesn't know.
#[serde(default)]
pub categories: Vec<String>,
}
impl IndexEntry {
/// Whether this IDE can load the version.
pub fn compatible(&self) -> bool {
crate::engine_compatible(&self.engine, false)
}
fn semver(&self) -> Option<semver::Version> {
semver::Version::parse(&self.version).ok()
}
}
/// `engines.openidea` of a manifest narrowed to the API its component was built
/// for (`major.minor`, if it has code): the requirement an index lists.
pub fn engine_requirement(manifest_engine: &str, built_for: Option<&semver::Version>) -> String {
match built_for {
Some(v) => format!("{manifest_engine}, ^{}.{}", v.major, v.minor),
None => manifest_engine.to_owned(),
}
}
/// A plugin (all versions when `versions` is empty) that must not run.
@@ -294,12 +373,71 @@ pub struct Revocation {
}
impl Index {
/// Parses an index. One in a newer format keeps what must reach every IDE —
/// revocations, publisher keys (revoked ones) and `moved` — and lists no
/// plugins ([`Index::is_newer`]): the IDE needs updating to install from it.
pub fn parse(text: &str) -> Result<Self, String> {
let index: Index = serde_json::from_str(text).map_err(|e| format!("index: {e}"))?;
if index.format != INDEX_FORMAT {
return Err(format!("index format {} is not supported (this IDE reads {INDEX_FORMAT})", index.format));
let value: serde_json::Value = serde_json::from_str(text).map_err(|e| format!("index: {e}"))?;
let format = value.get("format").and_then(serde_json::Value::as_u64).ok_or("index: `format` is missing")?;
if format == u64::from(INDEX_FORMAT) {
return serde_json::from_value(value).map_err(|e| format!("index: {e}"));
}
Ok(index)
if format < u64::from(INDEX_FORMAT) {
return Err(format!("index format {format} is not supported (this IDE reads {INDEX_FORMAT})"));
}
fn each<T: serde::de::DeserializeOwned>(value: &serde_json::Value, field: &str) -> Vec<T> {
let items = value.get(field).and_then(serde_json::Value::as_array);
items.into_iter().flatten().filter_map(|v| serde_json::from_value(v.clone()).ok()).collect()
}
Ok(Index {
format: u32::try_from(format).unwrap_or(u32::MAX),
publishers: each(&value, "publishers"),
plugins: Vec::new(),
revoked: each(&value, "revoked"),
moved: value.get("moved").and_then(serde_json::Value::as_str).map(str::to_owned),
})
}
/// Written in a format newer than this IDE reads: it lists no plugins.
pub fn is_newer(&self) -> bool {
self.format > INDEX_FORMAT
}
/// The version of each plugin to offer, and whether this IDE can load it:
/// the newest it can load, else the newest. Revoked versions are skipped;
/// plugins come in the order they are first listed.
pub fn offered(&self) -> Vec<(&IndexEntry, bool)> {
let mut ids: Vec<&str> = Vec::new();
for e in &self.plugins {
if !ids.contains(&e.id.as_str()) {
ids.push(&e.id);
}
}
ids.into_iter()
.filter_map(|id| {
let versions: Vec<&IndexEntry> =
self.plugins.iter().filter(|e| e.id == id && self.revocation(id, &e.version).is_none()).collect();
let newest = |loadable: bool| {
versions.iter().copied().filter(|e| !loadable || e.compatible()).max_by_key(|e| e.semver())
};
newest(true).map(|e| (e, true)).or_else(|| newest(false).map(|e| (e, false)))
})
.collect()
}
/// Drops the versions no IDE would pick: revoked ones, and of each plugin
/// all but the newest version per API requirement.
pub fn prune_versions(&mut self) {
let plugins = std::mem::take(&mut self.plugins);
let mut kept: Vec<IndexEntry> = Vec::new();
for e in plugins.into_iter().filter(|e| self.revocation(&e.id, &e.version).is_none()) {
match kept.iter_mut().find(|k| k.id == e.id && k.engine == e.engine) {
Some(k) if e.semver() > k.semver() => *k = e,
Some(_) => {}
None => kept.push(e),
}
}
self.plugins = kept;
}
/// The revocation that applies to `id` at `version`.
@@ -335,6 +473,127 @@ impl Index {
sig.check(id, version, digest)?;
Ok(publisher.id.clone())
}
/// Whether `to` is `from`, or a key of `publisher` that `from` endorsed,
/// directly or through keys it endorsed. Endorsements by revoked keys don't
/// count: whoever stole a key could vouch for their own.
pub fn succeeds(&self, publisher: &str, from: &str, to: &str) -> bool {
let keys = self.publishers.iter().find(|p| p.id == publisher).map_or(&[][..], |p| &p.keys[..]);
let mut key = to;
// Each step moves to an earlier key; more steps than keys is a loop.
for _ in 0..=keys.len() {
if key == from {
return true;
}
let Some(e) = keys.iter().find(|k| k.key == key).and_then(|k| k.endorsement.as_ref()) else {
return false;
};
let endorser_valid = keys.iter().any(|k| k.key == e.key && !k.revoked);
if !endorser_valid || verify_ed25519(&e.key, &endorsement_message(publisher, key), &e.signature) != Ok(true)
{
return false;
}
key = &e.key;
}
false
}
/// Makes `roots` (base64) the keys `publisher` signs with, whatever this
/// index says: another key it lists stays only when a root endorsed it (a
/// rotation the roots vouched for). Revocations of the keys are kept.
pub fn pin_publisher(&mut self, publisher: &str, name: &str, roots: &[String]) {
if roots.is_empty() {
return;
}
let i = match self.publishers.iter().position(|p| p.id == publisher) {
Some(i) => i,
None => {
self.publishers.push(Publisher { id: publisher.into(), name: name.into(), keys: Vec::new() });
self.publishers.len() - 1
}
};
for root in roots {
if !self.publishers[i].keys.iter().any(|k| &k.key == root) {
self.publishers[i].keys.push(KeyEntry::new(root.clone()));
}
}
let keys = self.publishers[i].keys.clone();
let trusted = keys
.into_iter()
.filter(|k| roots.iter().any(|r| self.succeeds(publisher, r, &k.key)))
.map(|k| if roots.contains(&k.key) { KeyEntry { endorsement: None, ..k } } else { k })
.collect();
self.publishers[i].keys = trusted;
}
/// Replaces what this index lists for every publisher but `keep` with what
/// `other` lists: its publishers, their plugins and their revocations
/// (`other`'s downloads should already be absolute). `other` may only
/// speak for its own publishers, and none of them may be in `keep`.
pub fn include(&mut self, other: Index, keep: &[&str]) -> Result<(), String> {
let mut theirs: Vec<&str> = Vec::new();
for p in &other.publishers {
if !publisher_id_ok(&p.id) {
return Err(format!("invalid publisher id {:?}: use a-z, 0-9, '-' and '_'", p.id));
}
if keep.contains(&p.id.as_str()) {
return Err(format!("may not list publisher {:?}", p.id));
}
if theirs.contains(&p.id.as_str()) {
return Err(format!("publisher {:?} is listed twice", p.id));
}
theirs.push(&p.id);
}
let owned_by = |id: &str| theirs.iter().any(|p| owns(p, id));
for e in &other.plugins {
if !theirs.contains(&e.publisher.as_str()) || !owns(&e.publisher, &e.id) {
return Err(format!("{}: not a plugin of publisher {:?}", e.id, e.publisher));
}
if semver::VersionReq::parse(&e.engine).is_err() {
return Err(format!("{} {}: `engine` must be the plugin API it needs", e.id, e.version));
}
}
if let Some(r) = other.revoked.iter().find(|r| !owned_by(&r.id)) {
return Err(format!("{}: revoked by an index that doesn't publish it", r.id));
}
let kept = |id: &str| keep.iter().any(|p| owns(p, id));
self.publishers.retain(|p| keep.contains(&p.id.as_str()));
self.plugins.retain(|e| keep.contains(&e.publisher.as_str()));
self.revoked.retain(|r| kept(&r.id));
self.publishers.extend(other.publishers);
self.plugins.extend(other.plugins);
self.revoked.extend(other.revoked);
Ok(())
}
}
/// Whether a package signed `new` (`None`: unsigned) may replace an installed
/// one signed `installed` without asking the user: signed with the same key,
/// or with one the installed key endorsed (per `index`). An unsigned installed
/// copy pins nothing.
pub fn signer_continues(
index: Option<&Index>,
installed: Option<&PackageSignature>,
new: Option<&PackageSignature>,
) -> bool {
match (installed, new) {
(None, _) => true,
(Some(_), None) => false,
(Some(old), Some(new)) => {
old.publisher == new.publisher
&& (old.key == new.key || index.is_some_and(|i| i.succeeds(&new.publisher, &old.key, &new.key)))
}
}
}
/// Whether the plugin `id` is in `publisher`'s namespace.
fn owns(publisher: &str, id: &str) -> bool {
id.strip_prefix(publisher).is_some_and(|rest| rest.starts_with('.'))
}
/// A publisher id is a single segment of a plugin id.
pub fn publisher_id_ok(id: &str) -> bool {
!id.contains('.') && crate::validate_plugin_id(id).is_ok()
}
#[cfg(test)]
@@ -354,7 +613,7 @@ mod tests {
publishers: vec![Publisher {
id: "acme".into(),
name: "ACME".into(),
keys: vec![KeyEntry { key: key.public_base64(), revoked: false }],
keys: vec![KeyEntry::new(key.public_base64())],
}],
plugins: Vec::new(),
revoked: vec![Revocation {
@@ -362,6 +621,7 @@ mod tests {
versions: vec!["1.0.0".into()],
reason: "malware".into(),
}],
moved: None,
}
}
@@ -399,6 +659,138 @@ mod tests {
assert_eq!(PublisherKey::from_base64(&key.seed_base64()).unwrap().public_base64(), key.public_base64());
}
fn entry(id: &str, publisher: &str) -> IndexEntry {
IndexEntry {
id: id.into(),
name: id.into(),
version: "1.0.0".into(),
description: String::new(),
authors: Vec::new(),
publisher: publisher.into(),
download: format!("https://example.com/{id}.oiplugin"),
digest: "d".into(),
capabilities: Vec::new(),
engine: "^1.0".into(),
categories: Vec::new(),
}
}
fn version(id: &str, version: &str, engine: &str) -> IndexEntry {
IndexEntry { version: version.into(), engine: engine.into(), ..entry(id, "acme") }
}
#[test]
fn the_ide_is_offered_the_newest_version_it_can_load() {
let host = crate::ENGINE_VERSION;
let newer = semver::Version::parse(host).unwrap().major + 1;
let mut index = Index {
format: INDEX_FORMAT,
plugins: vec![
version("acme.a", "1.0.0", &format!("^{host}")),
version("acme.a", "1.2.0", &format!("^{host}")),
version("acme.a", "2.0.0", &format!("^{newer}.0")),
version("acme.b", "0.1.0", &format!("^{newer}.0")),
version("acme.b", "0.2.0", &format!("^{newer}.1")),
version("acme.c", "1.0.0", ""),
version("acme.c", "1.1.0", ""),
],
revoked: vec![Revocation { id: "acme.c".into(), versions: vec!["1.1.0".into()], reason: "r".into() }],
..Index::default()
};
let offered = |index: &Index| {
index.offered().into_iter().map(|(e, ok)| (e.id.clone(), e.version.clone(), ok)).collect::<Vec<_>>()
};
let expected = [
("acme.a".to_owned(), "1.2.0".to_owned(), true),
("acme.b".to_owned(), "0.2.0".to_owned(), false),
// No `engine`: listed, but no IDE installs it.
("acme.c".to_owned(), "1.0.0".to_owned(), false),
];
assert_eq!(offered(&index), expected);
// Pruning keeps the newest version per requirement, which changes nothing offered.
index.prune_versions();
let left: Vec<_> = index.plugins.iter().map(|e| format!("{} {}", e.id, e.version)).collect();
assert_eq!(left, ["acme.a 1.2.0", "acme.a 2.0.0", "acme.b 0.1.0", "acme.b 0.2.0", "acme.c 1.0.0"]);
assert_eq!(offered(&index), expected);
let built_for = semver::Version::new(1, 3, 0);
assert_eq!(engine_requirement("^1.0", Some(&built_for)), "^1.0, ^1.3");
assert_eq!(engine_requirement("^1.0", None), "^1.0");
assert!(semver::VersionReq::parse(&engine_requirement("^1.0", Some(&built_for))).is_ok());
}
fn publisher(id: &str) -> Publisher {
Publisher { id: id.into(), name: id.into(), keys: vec![KeyEntry::new(format!("{id}-key"))] }
}
fn revocation(id: &str) -> Revocation {
Revocation { id: id.into(), versions: Vec::new(), reason: "r".into() }
}
#[test]
fn including_an_index_replaces_everything_but_the_kept_publishers() {
let mut official = Index {
format: INDEX_FORMAT,
publishers: vec![publisher("openidea"), publisher("gone")],
plugins: vec![entry("openidea.a", "openidea"), entry("gone.x", "gone")],
revoked: vec![revocation("openidea.old"), revocation("gone.x")],
moved: None,
};
let community = Index {
format: INDEX_FORMAT,
publishers: vec![publisher("alice")],
plugins: vec![entry("alice.arthas", "alice")],
revoked: vec![revocation("alice.bad")],
moved: None,
};
official.include(community.clone(), &["openidea"]).unwrap();
let ids = |v: &[IndexEntry]| v.iter().map(|e| e.id.clone()).collect::<Vec<_>>();
assert_eq!(official.publishers.iter().map(|p| p.id.as_str()).collect::<Vec<_>>(), ["openidea", "alice"]);
assert_eq!(ids(&official.plugins), ["openidea.a", "alice.arthas"]);
assert_eq!(official.revoked.iter().map(|r| r.id.as_str()).collect::<Vec<_>>(), ["openidea.old", "alice.bad"]);
// Including again is idempotent.
let once = official.clone();
official.include(community.clone(), &["openidea"]).unwrap();
assert_eq!(official, once);
}
#[test]
fn an_included_index_speaks_only_for_its_own_publishers() {
let base = Index { format: INDEX_FORMAT, publishers: vec![publisher("openidea")], ..Index::default() };
let with = |publishers: Vec<Publisher>, plugins: Vec<IndexEntry>, revoked: Vec<Revocation>| Index {
format: INDEX_FORMAT,
publishers,
plugins,
revoked,
moved: None,
};
let cases = [
(with(vec![publisher("openidea")], vec![], vec![]), "may not list"),
(with(vec![publisher("alice"), publisher("alice")], vec![], vec![]), "listed twice"),
(with(vec![publisher("openidea.x")], vec![], vec![]), "invalid publisher"),
(with(vec![publisher("alice")], vec![entry("openidea.a", "alice")], vec![]), "not a plugin of"),
(with(vec![publisher("alice")], vec![entry("alice.a", "bob")], vec![]), "not a plugin of"),
(with(vec![publisher("alice")], vec![entry("alicex.a", "alice")], vec![]), "not a plugin of"),
(with(vec![publisher("alice")], vec![], vec![revocation("openidea.a")]), "doesn't publish"),
(
with(
vec![publisher("alice")],
vec![IndexEntry { engine: String::new(), ..entry("alice.a", "alice") }],
vec![],
),
"`engine`",
),
];
for (other, error) in cases {
let mut index = base.clone();
let err = index.include(other, &["openidea"]).unwrap_err();
assert!(err.contains(error), "{err}");
assert_eq!(index, base);
}
}
#[test]
fn unpacking_refuses_paths_leaving_the_package() {
let tmp = tempfile::tempdir().unwrap();
@@ -413,7 +805,7 @@ mod tests {
let err = unpack(&out, &tmp.path().join("dest")).unwrap_err();
assert!(err.contains("invalid path"), "{err}");
assert!(!tmp.path().join("outside.txt").exists());
assert!(Index::parse(r#"{"format": 2}"#).is_err());
assert!(Index::parse(r#"{"format": 2}"#).unwrap().is_newer());
}
fn zip_of(path: &Path, entries: &[(&str, &str)]) {
@@ -462,4 +854,102 @@ mod tests {
assert!(err.contains("is not an OpenIDEA plugin"), "{name}: {err}");
}
}
#[test]
fn a_newer_index_still_revokes_and_moves() {
let newer = format!(
r#"{{"format": {}, "plugins": [{{"anything": "new"}}], "shape": "changed",
"publishers": [{{"id": "acme", "name": "ACME", "keys": [{{"key": "k1", "revoked": true}}]}}, {{"id": 5}}],
"revoked": [{{"id": "acme.bad", "reason": "malware"}}, {{"oops": 1}}],
"moved": "https://example.com/next.json"}}"#,
INDEX_FORMAT + 1
);
let index = Index::parse(&newer).unwrap();
assert!(index.is_newer());
assert!(index.plugins.is_empty() && index.offered().is_empty());
assert_eq!(index.revocation("acme.bad", "2.0.0").unwrap().reason, "malware");
assert!(index.key_revoked("k1"));
assert_eq!(index.moved.as_deref(), Some("https://example.com/next.json"));
// Fields this IDE doesn't know are ignored in its own format.
let current = format!(r#"{{"format": {INDEX_FORMAT}, "later": true, "plugins": []}}"#);
assert!(!Index::parse(&current).unwrap().is_newer());
assert!(Index::parse(r#"{"format": 0}"#).unwrap_err().contains("not supported"));
assert!(Index::parse(r#"{"plugins": []}"#).is_err());
}
fn keys<'a>(index: &'a mut Index, publisher: &str) -> &'a mut Vec<KeyEntry> {
&mut index.publishers.iter_mut().find(|p| p.id == publisher).unwrap().keys
}
#[test]
fn rotated_keys_continue_through_endorsements() {
let (k1, k2, k3) =
(PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([2; 32]), PublisherKey::from_seed([3; 32]));
let mut index = index(&k1);
let endorsed = |by: &PublisherKey, key: &PublisherKey| KeyEntry {
endorsement: Some(by.endorse("acme", &key.public_base64())),
..KeyEntry::new(key.public_base64())
};
keys(&mut index, "acme").push(endorsed(&k1, &k2));
keys(&mut index, "acme").push(endorsed(&k2, &k3));
let (p1, p2, p3) = (k1.public_base64(), k2.public_base64(), k3.public_base64());
assert!(index.succeeds("acme", &p1, &p1));
assert!(index.succeeds("acme", &p1, &p3), "through k2");
assert!(!index.succeeds("acme", &p3, &p1), "not backwards");
assert!(!index.succeeds("other", &p1, &p2));
// An endorsement for another publisher, or by a revoked key, doesn't count.
let mut foreign = index.clone();
keys(&mut foreign, "acme")[1].endorsement = Some(k1.endorse("evil", &p2));
assert!(!foreign.succeeds("acme", &p1, &p2));
let mut stolen = index.clone();
keys(&mut stolen, "acme")[0].revoked = true;
assert!(!stolen.succeeds("acme", &p1, &p2));
let sig = |key: &PublisherKey| key.sign("acme", "acme.tools", "1.0.0", "d");
assert!(signer_continues(Some(&index), Some(&sig(&k1)), Some(&sig(&k3))));
assert!(signer_continues(None, Some(&sig(&k1)), Some(&sig(&k1))));
assert!(!signer_continues(None, Some(&sig(&k1)), Some(&sig(&k2))), "no index to vouch");
let stranger = PublisherKey::from_seed([4; 32]);
assert!(!signer_continues(Some(&index), Some(&sig(&k1)), Some(&sig(&stranger))));
assert!(!signer_continues(Some(&index), Some(&sig(&k1)), None), "signed, then unsigned");
assert!(signer_continues(Some(&index), None, Some(&sig(&stranger))), "unsigned pins nothing");
}
#[test]
fn pinned_publishers_keep_their_built_in_keys() {
let (root, next, intruder) =
(PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([2; 32]), PublisherKey::from_seed([5; 32]));
let mut index = index(&intruder);
index.publishers[0].id = "openidea".into();
keys(&mut index, "openidea").push(KeyEntry {
endorsement: Some(root.endorse("openidea", &next.public_base64())),
..KeyEntry::new(next.public_base64())
});
let roots = [root.public_base64()];
let listed = |index: &Index| -> Vec<(String, bool)> {
let mut index = index.clone();
index.pin_publisher("openidea", "OpenIDEA", &roots);
keys(&mut index, "openidea").iter().map(|k| (k.key.clone(), k.revoked)).collect()
};
// The intruder's key goes, the successor the root endorsed stays, the root is added.
assert_eq!(listed(&index), [(next.public_base64(), false), (root.public_base64(), false)]);
let mut pinned = index.clone();
pinned.pin_publisher("openidea", "OpenIDEA", &roots);
let digest = "d";
let signed = |k: &PublisherKey| k.sign("openidea", "openidea.tools", "1.0.0", digest);
assert!(pinned.verify(&signed(&intruder), "openidea.tools", "1.0.0", digest).is_err());
assert!(pinned.verify(&signed(&next), "openidea.tools", "1.0.0", digest).is_ok());
// A revoked root stays revoked, and what it endorsed is no longer trusted.
keys(&mut index, "openidea").push(KeyEntry { revoked: true, ..KeyEntry::new(root.public_base64()) });
assert_eq!(listed(&index), [(root.public_base64(), true)]);
// A publisher missing from the index is added with its roots.
let mut empty = Index { format: INDEX_FORMAT, ..Index::default() };
empty.pin_publisher("openidea", "OpenIDEA", &[root.public_base64()]);
assert_eq!(empty.verify(&signed(&root), "openidea.tools", "1.0.0", digest).unwrap(), "openidea");
// No roots: the index decides.
let mut open = index.clone();
open.pin_publisher("openidea", "OpenIDEA", &[]);
assert_eq!(open, index);
}
}
+221 -21
View File
@@ -21,6 +21,18 @@ pub struct FontFamily {
pub monospaced: bool,
}
/// PANOSE bProportion value 9 means "Monospaced"; DirectWrite's
/// `IsMonospacedFont` alone misses fonts that only carry this classification.
const PANOSE_MONOSPACED: u8 = 9;
/// One face counts as monospaced when the font system flags it fixed pitch, its
/// PANOSE proportion is "Monospaced", or its glyph advances are uniform
/// (fonts with wrong/missing metadata, e.g. bitmap-style TTFs).
#[cfg_attr(not(windows), allow(dead_code))]
fn face_is_mono(is_monospaced_font: bool, panose_proportion: u8, advances_equal: bool) -> bool {
is_monospaced_font || panose_proportion == PANOSE_MONOSPACED || advances_equal
}
type Cache = Mutex<Option<Arc<[FontFamily]>>>;
/// The installed families, sorted case-insensitively, read on first use and
@@ -195,7 +207,9 @@ mod imp {
// SAFETY: CoreText descriptors carry documented CFString, CFURL and trait dictionary values.
unsafe {
let collection = CTFontCollection::from_available_fonts(None);
let Some(descriptors) = collection.matching_font_descriptors() else { return Vec::new() };
let Some(descriptors) = collection.matching_font_descriptors() else {
return Vec::new();
};
let mut seen = std::collections::BTreeSet::new();
descriptors
.cast_unchecked::<CTFontDescriptor>()
@@ -271,14 +285,16 @@ mod imp {
#[cfg(windows)]
mod imp {
use std::os::windows::ffi::OsStrExt;
use windows::Win32::Graphics::DirectWrite::IDWriteFontFace1;
use windows::Win32::Graphics::DirectWrite::{
DWRITE_FACTORY_TYPE_SHARED, DWRITE_FONT_STRETCH_NORMAL, DWRITE_FONT_STYLE_NORMAL, DWRITE_FONT_WEIGHT_NORMAL,
DWriteCreateFactory, IDWriteFactory, IDWriteFont1, IDWriteLocalizedStrings,
DWRITE_FACTORY_TYPE_SHARED, DWriteCreateFactory, IDWriteFactory, IDWriteFactory3, IDWriteFactory5, IDWriteFont,
IDWriteFont1, IDWriteFontCollection, IDWriteFontFamily, IDWriteLocalizedStrings,
};
use windows::core::{BOOL, Interface, w};
use windows::core::{BOOL, Interface, PCWSTR, w};
/// The system font collection's families, named in English when the
/// family has an English name (what CSS matches), else its first name.
/// The installed families, named in English when the family has an
/// English name (what CSS matches), else its first name.
pub(crate) fn list(refresh: bool) -> Vec<(String, bool)> {
// SAFETY: plain DirectWrite COM calls on interfaces we own.
unsafe { read(refresh) }.unwrap_or_default()
@@ -286,11 +302,85 @@ mod imp {
unsafe fn read(refresh: bool) -> windows::core::Result<Vec<(String, bool)>> {
let factory: IDWriteFactory = unsafe { DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED) }?;
let mut faces = Vec::new();
// Windows 10+: the newer collection covers registered per-user fonts.
// Exclude downloadable fonts that are not installed on this machine.
// Bitmap .FON fonts (e.g. the stock raster Fixedsys) are invisible to
// DirectWrite either way: it only reads TrueType/OpenType files.
let mut collection = None;
unsafe { factory.GetSystemFontCollection(&mut collection, refresh) }?;
let Some(collection) = collection else {
return Ok(Vec::new());
if let Ok(factory3) = factory.cast::<IDWriteFactory3>() {
let mut local = None;
if unsafe { factory3.GetSystemFontCollection(false, &mut local, refresh) }.is_ok() {
collection = local.map(Into::into);
}
}
if collection.is_none() {
unsafe { factory.GetSystemFontCollection(&mut collection, refresh) }?;
}
if let Some(collection) = collection {
faces.extend(collect(&collection));
}
// DirectWrite enumerates via the font registry, so font files copied
// into the per-user directory without registering (no HKCU entry)
// are invisible to the calls above; scan the directory directly.
faces.extend(unsafe { user_dir_faces(&factory) });
Ok(faces)
}
/// Families from `%LOCALAPPDATA%\Microsoft\Windows\Fonts` files, registered
/// or not, via a font set built from the files on disk.
unsafe fn user_dir_faces(factory: &IDWriteFactory) -> Vec<(String, bool)> {
let Some(local_app_data) = std::env::var_os("LOCALAPPDATA") else {
return Vec::new();
};
let dir = std::path::Path::new(&local_app_data).join(r"Microsoft\Windows\Fonts");
let Ok(entries) = std::fs::read_dir(&dir) else {
return Vec::new();
};
let files: Vec<_> = entries
.filter_map(|e| e.ok())
.map(|e| e.path())
.filter(|p| {
matches!(
p.extension().and_then(|e| e.to_str()).map(|e| e.to_ascii_lowercase()).as_deref(),
Some("ttf" | "otf" | "ttc" | "otc")
)
})
.collect();
unsafe { file_faces(factory, &files) }
}
unsafe fn file_faces(factory: &IDWriteFactory, files: &[std::path::PathBuf]) -> Vec<(String, bool)> {
if files.is_empty() {
return Vec::new();
}
let Ok(factory3) = factory.cast::<IDWriteFactory3>() else {
return Vec::new();
};
let Ok(factory5) = factory.cast::<IDWriteFactory5>() else {
return Vec::new();
};
let Ok(builder) = (unsafe { factory5.CreateFontSetBuilder() }) else {
return Vec::new();
};
for path in files {
let wide: Vec<u16> = path.as_os_str().encode_wide().chain(Some(0)).collect();
// SAFETY: wide is a valid NUL-terminated string for the calls.
let Ok(file) = (unsafe { factory.CreateFontFileReference(PCWSTR(wide.as_ptr()), None) }) else {
continue;
};
let _ = unsafe { builder.AddFontFile(&file) };
}
let Ok(set) = (unsafe { builder.CreateFontSet() }) else {
return Vec::new();
};
let Ok(collection) = (unsafe { factory3.CreateFontCollectionFromFontSet(&set) }) else {
return Vec::new();
};
collect(&collection)
}
fn collect(collection: &IDWriteFontCollection) -> Vec<(String, bool)> {
let mut out = Vec::new();
for i in 0..unsafe { collection.GetFontFamilyCount() } {
let Ok(family) = (unsafe { collection.GetFontFamily(i) }) else {
@@ -299,19 +389,120 @@ mod imp {
let Some(name) = unsafe { family.GetFamilyNames() }.ok().and_then(|n| unsafe { english(&n) }) else {
continue;
};
let mono = unsafe {
family.GetFirstMatchingFont(
DWRITE_FONT_WEIGHT_NORMAL,
DWRITE_FONT_STRETCH_NORMAL,
DWRITE_FONT_STYLE_NORMAL,
)
}
.ok()
.and_then(|f| f.cast::<IDWriteFont1>().ok())
.is_some_and(|f| unsafe { f.IsMonospacedFont() }.as_bool());
out.push((name, mono));
out.push((name, unsafe { family_is_mono(&family) }));
}
out
}
/// A family is monospaced when any face reports fixed pitch, PANOSE
/// proportion 9, or uniform glyph advances; probing every face matches
/// `normalize`'s any-face merge (macOS and Linux already flag every face,
/// not just the regular one).
unsafe fn family_is_mono(family: &IDWriteFontFamily) -> bool {
(0..unsafe { family.GetFontCount() }).any(|i| {
let Ok(font) = (unsafe { family.GetFont(i) }) else {
return false;
};
if let Ok(font1) = font.cast::<IDWriteFont1>() {
let is_monospaced = unsafe { font1.IsMonospacedFont() }.as_bool();
// The proportion byte is defined for text/display fonts only.
let panose = unsafe { font1.GetPanose().values };
let panose_proportion = if panose[0] == 2 { panose[3] } else { 0 };
if super::face_is_mono(is_monospaced, panose_proportion, false) {
return true;
}
}
unsafe { advances_equal(&font) }
})
}
/// Some monospaced fonts carry no fixed-pitch metadata at all (bitmap-style
/// TTFs like Fixedsys Excelsior): compare the advance widths of 'i' and
/// 'W'. Dual-width CJK monospace fonts pass too (both half-width).
unsafe fn advances_equal(font: &IDWriteFont) -> bool {
let Ok(face) = (unsafe { font.CreateFontFace() }) else {
return false;
};
let Ok(face1) = face.cast::<IDWriteFontFace1>() else {
return false;
};
let codepoints = ['i' as u32, 'W' as u32];
let mut indices = [0u16; 2];
if unsafe { face.GetGlyphIndices(codepoints.as_ptr(), 2, indices.as_mut_ptr()) }.is_err()
|| indices.contains(&0)
{
return false;
}
let mut advances = [0i32; 2];
unsafe { face1.GetDesignGlyphAdvances(2, indices.as_ptr(), advances.as_mut_ptr(), false) }.is_ok()
&& advances[0] > 0
&& advances[0] == advances[1]
}
#[test]
fn glyph_probe_distinguishes_monospace_proportional_and_symbol_fonts() {
unsafe {
let factory: IDWriteFactory = DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED).unwrap();
let mut collection = None;
factory.GetSystemFontCollection(&mut collection, false).unwrap();
let collection = collection.unwrap();
for (name, expected) in [
("Consolas", true),
("Arial", false),
("Times New Roman", false),
("Wingdings", false),
("Webdings", false),
("Segoe MDL2 Assets", false),
] {
let wide: Vec<_> = name.encode_utf16().chain(Some(0)).collect();
let mut index = 0;
let mut exists = BOOL(0);
collection.FindFamilyName(PCWSTR(wide.as_ptr()), &mut index, &mut exists).unwrap();
assert!(exists.as_bool(), "{name} ships with Windows");
let family = collection.GetFontFamily(index).unwrap();
for i in 0..family.GetFontCount() {
let font = family.GetFont(i).unwrap();
assert_eq!(advances_equal(&font), expected, "{name} face {i}");
}
}
}
}
#[test]
fn font_files_include_every_family_in_a_collection() {
let path = std::path::PathBuf::from(std::env::var_os("WINDIR").unwrap()).join(r"Fonts\cambria.ttc");
unsafe {
let factory: IDWriteFactory = DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED).unwrap();
let faces = file_faces(&factory, &[path]);
for name in ["Cambria", "Cambria Math"] {
assert!(faces.iter().any(|(family, _)| family == name), "missing {name}: {faces:?}");
}
}
}
#[test]
fn excludes_families_available_only_as_downloads() {
use windows::Win32::Graphics::DirectWrite::{DWRITE_E_REMOTEFONT, DWRITE_LOCALITY_LOCAL, IDWriteFont3};
unsafe {
let factory: IDWriteFactory = DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED).unwrap();
let factory3: IDWriteFactory3 = factory.cast().unwrap();
let set = factory3.GetSystemFontSet().unwrap();
let collection = factory3.CreateFontCollectionFromFontSet(&set).unwrap();
let listed = super::normalize(read(false).unwrap());
let user = super::normalize(user_dir_faces(&factory));
for i in 0..collection.GetFontFamilyCount() {
let family = collection.GetFontFamily(i).unwrap();
let name = english(&family.GetFamilyNames().unwrap()).unwrap();
let remote_only = (0..family.GetFontCount()).all(|j| match family.GetFont(j) {
Ok(font) => font.cast::<IDWriteFont3>().unwrap().GetLocality() != DWRITE_LOCALITY_LOCAL,
Err(error) => error.code() == DWRITE_E_REMOTEFONT,
});
if remote_only && !user.iter().any(|f| f.name == name) {
assert!(!listed.iter().any(|f| f.name == name), "{name} is not installed locally");
}
}
}
Ok(out)
}
unsafe fn english(names: &IDWriteLocalizedStrings) -> Option<String> {
@@ -476,6 +667,15 @@ mod tests {
);
}
#[test]
fn panose_monospaced_proportion_counts_as_mono() {
assert!(face_is_mono(true, 0, false));
assert!(face_is_mono(false, PANOSE_MONOSPACED, false));
assert!(face_is_mono(false, 0, true));
assert!(!face_is_mono(false, 0, false));
assert!(!face_is_mono(false, 3, false));
}
#[test]
fn fc_list_spacing_marks_monospace() {
let out = "DejaVu Sans Mono\t100\nDejaVu Sans\t\nNoto Sans Mono CJK SC\t90\nTerminus\t110\nCantarell\nFira\\-Code\tmono\n\t100\n";
+1
View File
@@ -46,6 +46,7 @@ ide-extension.workspace = true
ide-plugin.workspace = true
ide-tools = { path = "../ide-tools" }
ide-docker = { path = "../ide-docker" }
semver = "1"
serde.workspace = true
serde_json.workspace = true
toml.workspace = true
+419 -55
View File
@@ -3,13 +3,15 @@
//! ```text
//! openidea-plugin new <dir> [--template command|theme|icon-theme|language] [--id ID] [--name NAME] [--sdk PATH]
//! openidea-plugin build [dir] [--debug] # cargo → dist/plugin.wasm, then `check`
//! openidea-plugin check [dir] # validate the package as the IDE would install it
//! openidea-plugin check [dir|package] # validate the package as the IDE would install it
//! openidea-plugin link [dir] # load the folder in the IDE without installing it
//! openidea-plugin unlink [dir]
//! openidea-plugin links # list linked folders
//! openidea-plugin keygen <key-file> # a publisher's Ed25519 key
//! openidea-plugin endorse <new-public-key> --key OLD --publisher ID # vouch for a new key
//! openidea-plugin pack [dir] [--key F --publisher ID] [--out FILE] # <id>-<version>.oiplugin
//! openidea-plugin index <dir> --publisher ID --name NAME --key F # marketplace index.json
//! openidea-plugin index <dir> [--publisher ID --name NAME --key F [--previous-key F]] # marketplace index.json
//! openidea-plugin merge <index> <other-index> --keep ID # list another index's publishers
//! ```
//!
//! `link` writes `links.json` in the IDE's plugins directory
@@ -20,7 +22,7 @@
mod templates;
use ide_extension::{Manifest, validate_plugin_id};
use ide_host::plugins::PluginHost;
use ide_host::plugins::{PluginHost, read_market_url, resolve_market_href};
use ide_plugin::{Limits, PluginEngine};
use std::path::{Path, PathBuf};
use std::process::{Command, ExitCode};
@@ -31,17 +33,31 @@ usage: openidea-plugin <command> [options]
commands:
new <dir> [--template T] [--id ID] [--name NAME] [--sdk PATH]
create a plugin; T = command (default), theme, icon-theme, language
create a plugin; T = command (default), theme, icon-theme, language. A Rust
plugin depends on the SDK of this tool's release (git); --sdk PATH uses a
local sdk/rust/openidea-plugin instead
build [dir] [--debug] compile a Rust plugin to dist/plugin.wasm and check it
check [dir] validate the package as the IDE would install it
check [dir|package] validate the package (folder or .oiplugin) as the IDE would install it
link [dir] load the folder in the IDE without installing it
unlink [dir] stop loading a linked folder
links list linked folders
keygen <key-file> create a publisher signing key (keep the file secret)
endorse <new-public-key> --key F --publisher ID
print an earlier key's endorsement of a new one (JSON), for an
index the publisher doesn't write with `index --previous-key`
pack [dir] [--key F --publisher ID] [--out FILE]
write <id>-<version>.oiplugin, signed with the key
index <dir> --publisher ID --name NAME --key F [--out FILE]
write the marketplace index.json for the packages in dir
index <dir> [--publisher ID --name NAME --key F [--previous-key F]] [--out FILE]
[--download-base URL]
write the marketplace index.json for the packages in dir: with a
key, register it and list that publisher's plugins (earlier
versions stay for older IDEs); without, list only these packages,
each checked against the index's publishers. A new key replacing
an earlier one is endorsed with --previous-key, so installed
plugins update to it without asking
merge <index> <other-index> --keep ID [--out FILE]
replace every publisher but ID in index with those of other-index
(a path or URL), their plugins and revocations
environment:
OPENIDEA_PLUGINS_DIR the IDE's plugins directory (default: the app data directory)";
@@ -51,13 +67,15 @@ fn main() -> ExitCode {
let result = match args.first().map(String::as_str) {
Some("new") => cmd_new(&args[1..]),
Some("build") => cmd_build(&args[1..]),
Some("check") => dir_arg(&args[1..]).and_then(|d| check(&d)),
Some("check") => dir_arg(&args[1..]).and_then(|d| check_path(&d)),
Some("link") => dir_arg(&args[1..]).and_then(|d| link(&d, true)),
Some("unlink") => dir_arg(&args[1..]).and_then(|d| link(&d, false)),
Some("links") => list_links(),
Some("keygen") => cmd_keygen(&args[1..]),
Some("endorse") => cmd_endorse(&args[1..]),
Some("pack") => cmd_pack(&args[1..]),
Some("index") => cmd_index(&args[1..]),
Some("merge") => cmd_merge(&args[1..]),
Some("--version" | "-V") => {
println!("openidea-plugin {} (plugin API {})", env!("CARGO_PKG_VERSION"), ide_extension::ENGINE_VERSION);
Ok(())
@@ -161,7 +179,7 @@ fn cmd_new(args: &[String]) -> Result<(), String> {
std::fs::create_dir_all(&dir).map_err(|e| format!("{}: {e}", dir.display()))?;
let dir = canonical(&dir)?;
let sdk = match template {
templates::Template::Command => Some(sdk_path(a.value("sdk"), &dir)?),
templates::Template::Command => Some(sdk_dependency(a.value("sdk"), &dir)?),
_ => None,
};
let vars = templates::Vars { id: &id, name: &name, sdk: sdk.as_deref() };
@@ -216,12 +234,16 @@ fn title_case(s: &str) -> String {
.join(" ")
}
/// The SDK dependency path written into a new plugin's Cargo.toml: `--sdk`,
/// or the SDK of the source tree this tool was built from.
fn sdk_path(explicit: Option<&str>, plugin_dir: &Path) -> Result<String, String> {
/// The SDK dependency written into a new plugin's Cargo.toml: by path with `--sdk`, or
/// when the plugin is inside the source tree this tool was built from; otherwise the
/// repository at this tool's release tag.
fn sdk_dependency(explicit: Option<&str>, plugin_dir: &Path) -> Result<String, String> {
let sdk = match explicit {
Some(p) => PathBuf::from(p),
None => PathBuf::from(concat!(env!("CARGO_MANIFEST_DIR"), "/../../sdk/rust/openidea-plugin")),
None => match dunce::canonicalize(concat!(env!("CARGO_MANIFEST_DIR"), "/../..")) {
Ok(tree) if plugin_dir.starts_with(&tree) => tree.join("sdk/rust/openidea-plugin"),
_ => return Ok(format!("{{ git = \"{REPOSITORY}\", tag = \"v{}\" }}", env!("CARGO_PKG_VERSION"))),
},
};
let sdk = dunce::canonicalize(&sdk).map_err(|_| {
format!("the plugin SDK was not found at {}; pass --sdk <path to sdk/rust/openidea-plugin>", sdk.display())
@@ -230,9 +252,11 @@ fn sdk_path(explicit: Option<&str>, plugin_dir: &Path) -> Result<String, String>
return Err(format!("{} is not the openidea-plugin SDK (no Cargo.toml)", sdk.display()));
}
let path = relative(plugin_dir, &sdk).unwrap_or(sdk);
Ok(path.to_string_lossy().replace('\\', "/"))
Ok(format!("{{ path = \"{}\" }}", path.to_string_lossy().replace('\\', "/")))
}
const REPOSITORY: &str = "https://github.com/l0ng-ai/openidea";
/// `to` relative to `from` when they share a root (keeps packages in a repo movable).
fn relative(from: &Path, to: &Path) -> Option<PathBuf> {
let (f, t): (Vec<_>, Vec<_>) = (from.components().collect(), to.components().collect());
@@ -323,10 +347,21 @@ fn ensure_wasm_target() {
/// Validates `dir` exactly as Install from Disk would, and checks the
/// component imports only the plugin API.
/// [`check`] of a folder, or of a package file unpacked as installing would.
fn check_path(path: &Path) -> Result<(), String> {
if !path.is_file() {
return check(path);
}
let tmp = scratch_dir("unpack")?;
let dir = tmp.path().join("package");
package::unpack(path, &dir).and_then(|_| check(&dir))
}
fn check(dir: &Path) -> Result<(), String> {
let dir = canonical(dir)?;
let scratch = std::env::temp_dir().join(format!("openidea-plugin-check-{}", std::process::id()));
let result = (|| {
let tmp = scratch_dir("check")?;
let scratch = tmp.path().join("host");
(|| {
let host = PluginHost::new(scratch.clone(), Arc::new(|_: Option<&str>, _| {}), Limits::default())
.map_err(|e| e.to_string())?;
let preview = host.preview_install(&dir).map_err(|e| e.to_string())?;
@@ -345,6 +380,17 @@ fn check(dir: &Path) -> Result<(), String> {
} else {
println!(" code: none (contributions only)");
}
println!(" categories: {}", manifest.categories().join(", "));
for c in manifest.unknown_categories() {
println!(" warning: unknown category {c:?} (ignored); use {}", ide_extension::CATEGORIES.join(", "));
}
if manifest.declared_categories.len() > ide_extension::MAX_CATEGORIES {
println!(
" warning: only the first {} known categories are used; {} are declared",
ide_extension::MAX_CATEGORIES,
manifest.declared_categories.len()
);
}
if !manifest.activation.is_empty() {
let events: Vec<String> = manifest.activation.iter().map(ToString::to_string).collect();
println!(" activation: {}", events.join(", "));
@@ -361,9 +407,7 @@ fn check(dir: &Path) -> Result<(), String> {
}
println!("ok");
Ok(())
})();
let _ = std::fs::remove_dir_all(&scratch);
result
})()
}
fn human_size(bytes: u64) -> String {
@@ -479,14 +523,31 @@ fn cmd_keygen(args: &[String]) -> Result<(), String> {
Ok(())
}
fn cmd_endorse(args: &[String]) -> Result<(), String> {
let a = Args::parse(args, &[], &["key", "publisher"])?;
let usage = "usage: openidea-plugin endorse <new-public-key> --key <earlier-key-file> --publisher ID";
let ([new], Some(key), Some(publisher)) = (a.positional.as_slice(), a.value("key"), a.value("publisher")) else {
return Err(usage.into());
};
if !package::publisher_id_ok(publisher) {
return Err(format!("invalid publisher id {publisher:?}"));
}
let endorsement = read_key(key)?.endorse(publisher, new.trim());
println!("{}", serde_json::to_string_pretty(&endorsement).map_err(|e| e.to_string())?);
Ok(())
}
/// A fresh scratch directory, removed when dropped.
fn scratch_dir(what: &str) -> Result<tempfile::TempDir, String> {
tempfile::Builder::new().prefix(&format!("openidea-plugin-{what}-")).tempdir().map_err(|e| e.to_string())
}
/// A scratch host to list a package's files as the IDE would.
fn with_scratch_host<T>(f: impl FnOnce(&PluginHost) -> Result<T, String>) -> Result<T, String> {
let scratch = std::env::temp_dir().join(format!("openidea-plugin-pack-{}", std::process::id()));
let result = PluginHost::new(scratch.clone(), Arc::new(|_: Option<&str>, _| {}), Limits::default())
let tmp = scratch_dir("host")?;
PluginHost::new(tmp.path().join("host"), Arc::new(|_: Option<&str>, _| {}), Limits::default())
.map_err(|e| e.to_string())
.and_then(|h| f(&h));
let _ = std::fs::remove_dir_all(&scratch);
result
.and_then(|h| f(&h))
}
fn cmd_pack(args: &[String]) -> Result<(), String> {
@@ -522,29 +583,28 @@ fn cmd_pack(args: &[String]) -> Result<(), String> {
}
fn cmd_index(args: &[String]) -> Result<(), String> {
let a = Args::parse(args, &[], &["publisher", "name", "key", "out"])?;
let a = Args::parse(args, &[], &["publisher", "name", "key", "previous-key", "out", "download-base"])?;
let dir = canonical(&a.dir()?)?;
let publisher = a.value("publisher").ok_or("--publisher is required")?;
let name = a.value("name").unwrap_or(publisher);
let key = read_key(a.value("key").ok_or("--key is required")?)?;
let out = a.value("out").map(PathBuf::from).unwrap_or_else(|| dir.join("index.json"));
let base = a.value("download-base").map(|b| b.trim_end_matches('/'));
// Revocations and other publishers of an existing index are kept.
let mut index = match std::fs::read_to_string(&out) {
Ok(text) => Index::parse(&text)?,
Ok(text) => current_format(Index::parse(&text)?, &out.to_string_lossy())?,
Err(_) => Index { format: package::INDEX_FORMAT, ..Index::default() },
};
// Earlier keys of the publisher stay listed (packages they signed remain valid until revoked).
let current = key.public_base64();
let earlier: Vec<KeyEntry> = index
.publishers
.iter()
.filter(|p| p.id == publisher)
.flat_map(|p| p.keys.iter().filter(|k| k.key != current).cloned())
.collect();
index.publishers.retain(|p| p.id != publisher);
let keys: Vec<KeyEntry> = std::iter::once(KeyEntry { key: current, revoked: false }).chain(earlier).collect();
index.publishers.push(package::Publisher { id: publisher.into(), name: name.into(), keys });
index.plugins.clear();
let publisher = match (a.value("publisher"), a.value("key")) {
(Some(publisher), Some(key)) => {
let key = read_key(key)?;
let name = a.value("name").unwrap_or(publisher);
let previous = a.value("previous-key").map(read_key).transpose()?;
index.publishers =
register_key(std::mem::take(&mut index.publishers), publisher, name, &key, previous.as_ref())?;
Some(publisher)
}
(None, None) if a.has("previous-key") => return Err("--previous-key goes with --publisher and --key".into()),
(None, None) => None,
_ => return Err("--publisher and --key go together".into()),
};
let mut packages: Vec<PathBuf> = std::fs::read_dir(&dir)
.map_err(|e| e.to_string())?
.flatten()
@@ -552,35 +612,144 @@ fn cmd_index(args: &[String]) -> Result<(), String> {
.filter(|p| p.extension().is_some_and(|e| e == package::PACKAGE_EXTENSION))
.collect();
packages.sort();
let engine = PluginEngine::new()?;
let mut added: Vec<IndexEntry> = Vec::new();
for path in packages {
let scratch = std::env::temp_dir().join(format!("openidea-plugin-index-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&scratch);
let unpacked = package::unpack(&path, &scratch)?;
let tmp = scratch_dir("index")?;
let scratch = tmp.path().join("package");
let entry = (|| {
let unpacked = package::unpack(&path, &scratch)?;
let m = Manifest::load(&scratch)?;
let files = with_scratch_host(|h| h.package_files_of(&scratch).map_err(|e| e.to_string()))?;
let digest = package::package_digest(&scratch, &files).map_err(|e| e.to_string())?;
let sig = unpacked.signature.as_ref().ok_or_else(|| format!("{}: not signed", path.display()))?;
sig.check(&m.id, &m.version, &digest).map_err(|e| format!("{}: {e}", path.display()))?;
let sig = unpacked.signature.as_ref().ok_or("not signed")?;
let signer = index.verify(sig, &m.id, &m.version, &digest)?;
if publisher.is_some_and(|p| p != signer) {
return Err(format!("signed by {signer}, not {}", publisher.unwrap_or_default()));
}
if added.iter().any(|e| (&e.id, &e.version) == (&m.id, &m.version)) {
return Err(format!("another package of {} {} is in the folder", m.id, m.version));
}
if m.legacy {
return Err("plugin.toml uses the 0.1 format; see plugins/README.md".into());
}
let built_for = match &m.wasm {
Some(wasm) => engine.check_component(&engine.compile(&scratch.join(wasm))?)?,
None => None,
};
let file = path.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
Ok::<_, String>(IndexEntry {
id: m.id.clone(),
name: m.name.clone(),
version: m.version.clone(),
description: m.description.clone(),
authors: m.authors.clone(),
publisher: sig.publisher.clone(),
download: path.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default(),
publisher: signer,
download: base.map_or_else(|| file.clone(), |b| format!("{b}/{file}")),
digest,
capabilities: m.capabilities.all().map(|c| c.to_string()).collect(),
engine: package::engine_requirement(&m.engine, built_for.as_ref()),
categories: m.categories(),
})
})();
let _ = std::fs::remove_dir_all(&scratch);
let entry = entry?;
println!(" {} {}", entry.id, entry.version);
index.plugins.push(entry);
let entry = entry.map_err(|e| format!("{}: {e}", path.display()))?;
println!(" {} {} (plugin API {})", entry.id, entry.version, entry.engine);
added.push(entry);
}
let json = serde_json::to_string_pretty(&index).map_err(|e| e.to_string())?;
std::fs::write(&out, json).map_err(|e| format!("{}: {e}", out.display()))?;
// A publisher's earlier versions of these plugins stay for IDEs that can't load the new
// ones (pruned below); plugins no longer in the folder go.
index.plugins.retain(|e| match publisher {
Some(p) => {
e.publisher != p
|| (added.iter().any(|n| n.id == e.id)
&& !added.iter().any(|n| (&n.id, &n.version) == (&e.id, &e.version)))
}
None => false,
});
index.plugins.extend(added);
index.prune_versions();
write_index(&index, &out)
}
/// `publishers` with `key` as `publisher`'s current key. Earlier keys stay listed
/// (packages they signed remain valid until revoked); a key new to the
/// publisher is endorsed by `previous`, one of them, so installed plugins
/// signed with it update without asking.
fn register_key(
mut publishers: Vec<package::Publisher>,
publisher: &str,
name: &str,
key: &PublisherKey,
previous: Option<&PublisherKey>,
) -> Result<Vec<package::Publisher>, String> {
let current = key.public_base64();
let earlier: Vec<KeyEntry> = publishers.iter().filter(|p| p.id == publisher).flat_map(|p| p.keys.clone()).collect();
let entry = match earlier.iter().find(|k| k.key == current) {
Some(listed) => listed.clone(),
None => {
let endorsement = match previous {
Some(prev) => {
let prev_key = prev.public_base64();
if !earlier.iter().any(|k| k.key == prev_key && !k.revoked) {
return Err(format!("--previous-key is not one of {publisher}'s valid keys in the index"));
}
Some(prev.endorse(publisher, &current))
}
None => {
if earlier.iter().any(|k| !k.revoked) {
println!(
" warning: {publisher}'s new key is not endorsed (--previous-key): users who installed \
its plugins will be asked to confirm the change when they update"
);
}
None
}
};
KeyEntry { endorsement, ..KeyEntry::new(current.clone()) }
}
};
let keys = std::iter::once(entry).chain(earlier.into_iter().filter(|k| k.key != current)).collect();
publishers.retain(|p| p.id != publisher);
publishers.push(package::Publisher { id: publisher.into(), name: name.into(), keys });
Ok(publishers)
}
/// `index`, unless a newer `openidea-plugin` wrote it (rewriting it would drop what this one can't read).
fn current_format(index: Index, file: &str) -> Result<Index, String> {
if index.is_newer() {
return Err(format!(
"{file}: index format {} is newer than this tool writes; update openidea-plugin",
index.format
));
}
Ok(index)
}
fn cmd_merge(args: &[String]) -> Result<(), String> {
let a = Args::parse(args, &[], &["keep", "out"])?;
let [base, other] = a.positional.as_slice() else {
return Err("usage: openidea-plugin merge <index> <other-index> --keep ID [--out FILE]".into());
};
let keep = a.value("keep").ok_or("--keep is required")?;
let out = a.value("out").unwrap_or(base);
let mut index =
current_format(Index::parse(&std::fs::read_to_string(base).map_err(|e| format!("{base}: {e}"))?)?, base)?;
let bytes = read_market_url(other, 16 << 20)?;
let theirs = Index::parse(&String::from_utf8(bytes).map_err(|_| format!("{other}: not UTF-8"))?)?;
let mut theirs = current_format(theirs, other)?;
// Their downloads are relative to their index, not to ours.
for e in &mut theirs.plugins {
e.download = resolve_market_href(other, &e.download);
}
let count = theirs.plugins.len();
index.include(theirs, &[keep]).map_err(|e| format!("{other}: {e}"))?;
println!(" {count} plugins from {other}");
write_index(&index, Path::new(out))
}
fn write_index(index: &Index, out: &Path) -> Result<(), String> {
let json = serde_json::to_string_pretty(index).map_err(|e| e.to_string())?;
std::fs::write(out, json).map_err(|e| format!("{}: {e}", out.display()))?;
println!("-> {} ({} plugins)", out.display(), index.plugins.len());
Ok(())
}
@@ -589,6 +758,25 @@ fn cmd_index(args: &[String]) -> Result<(), String> {
mod tests {
use super::*;
#[test]
fn a_new_key_is_endorsed_by_the_previous_one() {
let (k1, k2, k3) =
(PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([2; 32]), PublisherKey::from_seed([3; 32]));
let first = register_key(Vec::new(), "acme", "ACME", &k1, None).unwrap();
let rotated = register_key(first.clone(), "acme", "ACME", &k2, Some(&k1)).unwrap();
let index = Index { format: package::INDEX_FORMAT, publishers: rotated.clone(), ..Index::default() };
assert!(index.succeeds("acme", &k1.public_base64(), &k2.public_base64()));
assert_eq!(rotated[0].keys.len(), 2, "the earlier key stays listed");
// Registering the current key again keeps its endorsement.
let again = register_key(rotated.clone(), "acme", "ACME", &k2, None).unwrap();
assert_eq!(again, rotated);
// Only a key the publisher has can endorse.
assert!(register_key(first.clone(), "acme", "ACME", &k2, Some(&k3)).unwrap_err().contains("--previous-key"));
// Without one the new key is listed unendorsed.
let unendorsed = register_key(first, "acme", "ACME", &k3, None).unwrap();
assert!(unendorsed[0].keys[0].endorsement.is_none());
}
#[test]
fn names() {
assert_eq!(slugify("My Cool_Plugin"), "my-cool-plugin");
@@ -601,13 +789,189 @@ mod tests {
assert_eq!(relative(Path::new("/x/y"), Path::new("/z")), None);
}
#[test]
fn new_plugins_depend_on_the_sdk_by_path_only_in_its_source_tree() {
let tree = dunce::canonicalize(concat!(env!("CARGO_MANIFEST_DIR"), "/../..")).unwrap();
assert_eq!(
sdk_dependency(None, &tree.join("plugins/official/new")).unwrap(),
"{ path = \"../../../sdk/rust/openidea-plugin\" }"
);
let outside = dunce::canonicalize(std::env::temp_dir()).unwrap().join("my-plugin");
assert_eq!(
sdk_dependency(None, &outside).unwrap(),
format!("{{ git = \"https://github.com/l0ng-ai/openidea\", tag = \"v{}\" }}", env!("CARGO_PKG_VERSION"))
);
let sdk = tree.join("sdk/rust/openidea-plugin");
assert!(sdk_dependency(Some(sdk.to_str().unwrap()), &outside).unwrap().starts_with("{ path = "));
assert!(
sdk_dependency(Some(tree.join("wit").to_str().unwrap()), &outside)
.unwrap_err()
.contains("not the openidea-plugin SDK")
);
}
/// A theme package `id` signed by `publisher` with `key`, packed into `out`.
fn signed_theme(tmp: &Path, id: &str, publisher: &str, key: &str, out: &Path) {
signed_theme_version(tmp, id, "0.1.0", "^1.0", publisher, key, out);
}
fn signed_theme_version(tmp: &Path, id: &str, version: &str, engine: &str, publisher: &str, key: &str, out: &Path) {
let src = tmp.join("src").join(format!("{id}-{version}"));
let vars = templates::Vars { id, name: id, sdk: None };
for (rel, content) in templates::render(templates::Template::Theme, &vars) {
let path = src.join(rel);
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
let content = content
.replace("version = \"0.1.0\"", &format!("version = \"{version}\""))
.replace("engines.openidea = \"^1.0\"", &format!("engines.openidea = \"{engine}\""));
std::fs::write(path, content).unwrap();
}
std::fs::create_dir_all(out).unwrap();
let args = [src.to_str().unwrap(), "--key", key, "--publisher", publisher, "--out"]
.into_iter()
.map(String::from)
.chain([out.join(format!("{id}-{version}.oiplugin")).to_string_lossy().into_owned()])
.collect::<Vec<_>>();
cmd_pack(&args).unwrap();
}
#[test]
fn a_release_keeps_the_versions_older_ides_need() {
let tmp = tempfile::tempdir().unwrap();
let t = tmp.path();
let key = t.join("acme.key");
std::fs::write(&key, PublisherKey::from_seed([5; 32]).seed_base64()).unwrap();
let key = key.to_string_lossy().into_owned();
let index_file = t.join("index.json");
let release = |n: usize, packages: &[(&str, &str, &str)]| {
let dir = t.join(format!("release-{n}"));
for (id, version, engine) in packages {
signed_theme_version(t, id, version, engine, "acme", &key, &dir);
}
let base = format!("https://example.com/download/v{n}");
let args = [dir.to_str().unwrap(), "--publisher", "acme", "--key", &key, "--download-base", &base];
cmd_index(&strings(&[&args[..], &["--out", index_file.to_str().unwrap()]].concat())).unwrap();
let index = Index::parse(&std::fs::read_to_string(&index_file).unwrap()).unwrap();
index.plugins.iter().map(|e| format!("{} {} {}", e.id, e.version, e.download)).collect::<Vec<_>>()
};
release(1, &[("acme.a", "1.0.0", "^1.0"), ("acme.b", "1.0.0", "^1.0")]);
// Same requirement: the newer version replaces it; acme.b is gone from the release.
assert_eq!(
release(2, &[("acme.a", "1.1.0", "^1.0")]),
["acme.a 1.1.0 https://example.com/download/v2/acme.a-1.1.0.oiplugin"]
);
// Another API requirement: the last version for the earlier one stays, where it was published.
assert_eq!(
release(3, &[("acme.a", "2.0.0", ">=1.0, <2")]),
[
"acme.a 1.1.0 https://example.com/download/v2/acme.a-1.1.0.oiplugin",
"acme.a 2.0.0 https://example.com/download/v3/acme.a-2.0.0.oiplugin",
]
);
let index = Index::parse(&std::fs::read_to_string(&index_file).unwrap()).unwrap();
assert_eq!(index.offered()[0].0.version, "2.0.0", "the newest this IDE can load");
}
fn strings(args: &[&str]) -> Vec<String> {
args.iter().map(|a| a.to_string()).collect()
}
#[test]
fn official_and_community_indexes_merge() {
let tmp = tempfile::tempdir().unwrap();
let t = tmp.path();
let mut keys = Vec::new();
for (name, seed) in [("openidea", 1), ("alice", 2), ("carol", 3)] {
let file = t.join(format!("{name}.key"));
let key = PublisherKey::from_seed([seed; 32]);
std::fs::write(&file, key.seed_base64()).unwrap();
keys.push((file.to_string_lossy().into_owned(), key.public_base64()));
}
let (official, community) = (t.join("official"), t.join("community"));
signed_theme(t, "openidea.night", "openidea", &keys[0].0, &official);
signed_theme(t, "alice.dawn", "alice", &keys[1].0, &community);
let official_index = official.join("index.json");
let community_index = community.join("index.json");
let release = strings(&[official.to_str().unwrap(), "--publisher", "openidea", "--key", &keys[0].0]);
cmd_index(&release).unwrap();
// The community index lists its publishers; `index` without a key checks packages against them.
let skeleton = Index {
format: package::INDEX_FORMAT,
publishers: vec![package::Publisher {
id: "alice".into(),
name: "Alice".into(),
keys: vec![KeyEntry::new(keys[1].1.clone())],
}],
..Index::default()
};
write_index(&skeleton, &community_index).unwrap();
cmd_index(&strings(&[community.to_str().unwrap()])).unwrap();
cmd_merge(&strings(&[
official_index.to_str().unwrap(),
community_index.to_str().unwrap(),
"--keep",
"openidea",
]))
.unwrap();
// A release rebuilds the official plugins and keeps the community's.
cmd_index(&release).unwrap();
let index = Index::parse(&std::fs::read_to_string(&official_index).unwrap()).unwrap();
let listed: Vec<_> = index.plugins.iter().map(|e| (e.id.as_str(), e.publisher.as_str())).collect();
assert_eq!(listed, [("alice.dawn", "alice"), ("openidea.night", "openidea")]);
assert_eq!(index.plugins[1].categories, ["themes"], "implied by the theme");
assert_eq!(index.plugins[1].download, "openidea.night-0.1.0.oiplugin");
assert!(Path::new(&index.plugins[0].download).is_absolute(), "{}", index.plugins[0].download);
// A package by a publisher the index doesn't know is refused.
signed_theme(t, "carol.noon", "carol", &keys[2].0, &community);
let err = cmd_index(&strings(&[community.to_str().unwrap()])).unwrap_err();
assert!(err.contains("unknown publisher"), "{err}");
}
/// `cargo test` of a new Rust plugin (its own tests against the mock host) with this SDK.
#[test]
fn the_command_template_builds_against_the_sdk() {
let dir = std::env::temp_dir().join(format!("openidea-plugin-template-build-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let dir = dunce::canonicalize(&dir).unwrap();
let plugin = dir.join("plugin");
let sdk = concat!(env!("CARGO_MANIFEST_DIR"), "/../../sdk/rust/openidea-plugin");
let sdk = sdk_dependency(Some(sdk), &plugin).unwrap();
let vars = templates::Vars { id: "acme.sample", name: "Sample", sdk: Some(&sdk) };
for (rel, content) in templates::render(templates::Template::Command, &vars) {
let path = plugin.join(rel);
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
std::fs::write(path, content).unwrap();
}
// The SDK's lockfile keeps the versions its own CI builds with (the resolver also
// looks up the wasm-only dependencies, which may not be downloaded yet).
let sdk_lock = concat!(env!("CARGO_MANIFEST_DIR"), "/../../sdk/rust/openidea-plugin/Cargo.lock");
std::fs::copy(sdk_lock, plugin.join("Cargo.lock")).unwrap();
let out = Command::new(std::env::var("CARGO").unwrap_or_else(|_| "cargo".into()))
.args(["test", "--manifest-path"])
.arg(plugin.join("Cargo.toml"))
.env("CARGO_TARGET_DIR", dir.join("target"))
.output()
.unwrap();
assert!(
out.status.success(),
"{}{}",
String::from_utf8_lossy(&out.stdout),
String::from_utf8_lossy(&out.stderr)
);
std::fs::remove_dir_all(&dir).unwrap();
}
#[test]
fn every_template_is_a_valid_package() {
for t in templates::Template::ALL {
let dir =
std::env::temp_dir().join(format!("openidea-plugin-template-{}-{}", t.label(), std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
let vars = templates::Vars { id: "acme.sample", name: "Sample", sdk: Some("../sdk") };
let vars = templates::Vars { id: "acme.sample", name: "Sample", sdk: Some("{ path = \"../sdk\" }") };
for (rel, content) in templates::render(t, &vars) {
let path = dir.join(rel);
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
@@ -32,6 +32,7 @@ impl Template {
pub struct Vars<'a> {
pub id: &'a str,
pub name: &'a str,
/// The SDK dependency, as a TOML value (`{ path = "…" }`).
pub sdk: Option<&'a str>,
}
@@ -82,6 +83,9 @@ name = "__NAME__"
version = "0.1.0"
description = "Adds two sample commands."
authors = []
# Marketplace categories (up to 3): languages, themes, icon-themes, linters,
# vcs, ai, frameworks, tools, other.
categories = ["tools"]
engines.openidea = "^1.0"
# Built by `openidea-plugin build`. The code is loaded the first time one of
# its commands runs; the commands below show in Find Action before that.
@@ -115,7 +119,7 @@ publish = false
crate-type = ["cdylib", "rlib"]
[dependencies]
openidea-plugin = { path = "__SDK__" }
openidea-plugin = __SDK__
[profile.release]
opt-level = "s"
@@ -127,7 +131,7 @@ panic = "abort"
const COMMAND_LIB: &str = r#"//! __NAME__: an OpenIDEA plugin.
use openidea_plugin::{Invocation, NotificationLevel, Plugin, host, register_plugin};
use openidea_plugin::{Error, Invocation, NotificationLevel, Plugin, host, register_plugin};
#[derive(Default)]
pub struct Main {
@@ -135,7 +139,7 @@ pub struct Main {
}
impl Plugin for Main {
fn run_command(&mut self, id: &str, ctx: &Invocation, _arg: Option<&str>) -> Result<Option<String>, String> {
fn run_command(&mut self, id: &str, ctx: &Invocation, _arg: Option<&str>) -> Result<Option<String>, Error> {
match id {
"say-hello" => {
// Instance state lasts until the plugin is idle for a while;
@@ -150,7 +154,7 @@ impl Plugin for Main {
let text = editor.document()?.text;
Ok(Some(format!("{} lines in {}", text.lines().count(), editor.path())))
}
_ => Err(format!("unknown command {id}")),
_ => Err(format!("unknown command {id}").into()),
}
}
}
+67 -11
View File
@@ -76,9 +76,17 @@ async fn plugin_call<T: Send + 'static>(f: impl FnOnce() -> T + Send + 'static)
tokio::task::spawn_blocking(f).await.ok()
}
/// Items from plugins carry no server (resolve and commands don't apply).
/// Items from plugins carry no server: commands don't apply, and a code
/// action is resolved by the plugin named in its `data` ([`PluginActionRef`]).
const PLUGIN_SERVER: ServerId = ServerId(0);
/// `data` of a plugin's code action whose edits it computes when picked.
#[derive(serde::Serialize, serde::Deserialize)]
struct PluginActionRef {
plugin: String,
data: String,
}
/// A plugin's `completion-item.kind` (wit/providers.wit lists them).
fn plugin_completion_kind(kind: &str) -> Option<CompletionKind> {
Some(match kind {
@@ -120,14 +128,14 @@ fn plugin_completion(item: ide_plugin::CompletionItem) -> CompletionItem {
kind,
documentation: item.documentation.map(|value| Markup { kind: "markdown".into(), value }),
sort_text: item.sort_text,
filter_text: None,
filter_text: item.filter_text,
preselect: false,
deprecated: false,
insert_text: item.insert_text,
is_snippet: false,
is_snippet: item.snippet,
replace_range: item.replace,
replace_range_tab: None,
additional_edits: Vec::new(),
additional_edits: item.additional_edits,
commit_characters: Vec::new(),
command: None,
data: None,
@@ -314,7 +322,7 @@ pub async fn lsp_code_actions(
};
let plugin_actions: Vec<CodeAction> = plugin_actions
.into_iter()
.map(|(_, a)| plugin_code_action(a))
.map(|(plugin, a)| plugin_code_action(&plugin, a))
.filter(|a| {
only.is_empty()
|| a.kind.as_ref().is_some_and(|k| only.iter().any(|o| k == o || k.starts_with(&format!("{o}."))))
@@ -330,24 +338,53 @@ pub async fn lsp_code_actions(
prepare_actions(project, doc, version, actions).await
}
fn plugin_code_action(a: ide_plugin::CodeAction) -> CodeAction {
let files = a
fn plugin_code_action(plugin: &str, a: ide_plugin::CodeAction) -> CodeAction {
// With `data` and no edits, the plugin computes them when the action is picked.
let lazy = a.edits.is_empty() && a.data.is_some();
let files: Vec<_> = a
.edits
.into_iter()
.map(|(path, edits)| ide_core::FileEdit::Edit { path: path.into(), edits, version: None })
.collect();
let data = a
.data
.filter(|_| lazy)
.and_then(|data| serde_json::to_value(PluginActionRef { plugin: plugin.to_owned(), data }).ok());
CodeAction {
title: a.title,
kind: a.kind,
is_preferred: a.preferred,
disabled_reason: None,
edit: Some(WorkspaceEdit { files, preconditions: Vec::new() }),
edit: (!lazy).then(|| WorkspaceEdit { files, preconditions: Vec::new() }),
command: None,
data: None,
data,
server: PLUGIN_SERVER,
}
}
/// A code action resolved by whoever offered it: its language server, or
/// the plugin its `data` names.
async fn resolve_action(host: &Host, window: &str, doc: DocId, action: CodeAction) -> IdeResult<CodeAction> {
if action.server != PLUGIN_SERVER {
return host.project(window)?.lsp.resolve_code_action(doc, action).await;
}
let Some(r) = action.data.clone().and_then(|d| serde_json::from_value::<PluginActionRef>(d).ok()) else {
return Ok(action);
};
let (plugins, d) = plugin_providers(host, window, doc, ProviderKind::CodeActions)
.ok_or_else(|| ide_core::IdeError::unavailable("the plugin that offered this action is gone"))?;
let pending = ide_plugin::CodeAction {
title: action.title.clone(),
kind: action.kind.clone(),
preferred: action.is_preferred,
edits: Vec::new(),
data: Some(r.data),
};
let plugin = r.plugin.clone();
let resolved = blocking(move || plugins.resolve_code_action(&plugin, &d, pending)).await?;
Ok(plugin_code_action(&r.plugin, ide_plugin::CodeAction { data: None, ..resolved }))
}
/// Cancellable code actions with the kinds the servers advertise (Refactor This, refactoring availability).
pub async fn lsp_code_action_query(
host: &Host,
@@ -381,7 +418,7 @@ pub async fn lsp_resolve_code_action(
let project = host.project(window)?;
let documents = project.documents.clone();
let version = blocking(move || documents.version(doc)).await?;
let action = project.lsp.resolve_code_action(doc, action).await?;
let action = resolve_action(host, window, doc, action).await?;
Ok(prepare_actions(project, doc, version, vec![action]).await?.remove(0))
}
@@ -391,7 +428,7 @@ pub async fn lsp_apply_code_action(host: &Host, window: &str, doc: DocId, action
let action = if action.edit.is_none() && action.command.is_none() {
let documents = project.documents.clone();
let version = blocking(move || documents.version(doc)).await?;
let action = project.lsp.resolve_code_action(doc, action).await?;
let action = resolve_action(host, window, doc, action).await?;
prepare_actions(project.clone(), doc, version, vec![action]).await?.remove(0)
} else {
action
@@ -801,6 +838,25 @@ pub(crate) fn register(d: &mut Dispatcher) {
mod tests {
use super::*;
#[test]
fn a_plugin_action_with_data_and_no_edits_is_resolved_by_its_plugin() {
let action = |edits: Vec<(std::path::PathBuf, Vec<TextEdit>)>, data: Option<&str>| ide_plugin::CodeAction {
title: "Fix".into(),
kind: None,
preferred: false,
edits,
data: data.map(str::to_owned),
};
let lazy = plugin_code_action("acme.x", action(Vec::new(), Some("ref")));
assert!(lazy.edit.is_none());
let r: PluginActionRef = serde_json::from_value(lazy.data.unwrap()).unwrap();
assert_eq!((r.plugin.as_str(), r.data.as_str()), ("acme.x", "ref"));
// With edits, `data` is the plugin's business: applied as is.
let edit = TextEdit { range: Range::default(), new_text: "x".into() };
let eager = plugin_code_action("acme.x", action(vec![("/a".into(), vec![edit])], Some("ref")));
assert!(eager.edit.is_some() && eager.data.is_none());
}
#[test]
fn every_completion_kind_the_plugin_api_documents_is_understood() {
let wit = include_str!("../../../../wit/providers.wit");
+5
View File
@@ -36,6 +36,10 @@ pub fn find_files(host: &Host, window: &str, query: String, limit: u32) -> IdeRe
Ok(host.project(window)?.index.find_files(&query, limit as usize))
}
pub fn find_dirs(host: &Host, window: &str, query: String, limit: u32) -> IdeResult<Vec<FileMatch>> {
Ok(host.project(window)?.index.find_dirs(&query, limit as usize))
}
pub fn find_symbols(
host: &Host,
window: &str,
@@ -157,6 +161,7 @@ pub async fn web_unresolved_references(host: &Host, window: &str, doc: DocId) ->
pub(crate) fn register(d: &mut Dispatcher) {
d.add("index_status", |c| async move { index_status(&c.host, &c.window) });
d.add("find_files", |c| async move { find_files(&c.host, &c.window, c.arg("query")?, c.arg("limit")?) });
d.add("find_dirs", |c| async move { find_dirs(&c.host, &c.window, c.arg("query")?, c.arg("limit")?) });
d.add("find_symbols", |c| async move {
find_symbols(&c.host, &c.window, c.arg("query")?, c.arg("filter")?, c.arg("limit")?)
});
+3
View File
@@ -155,6 +155,9 @@ pub async fn start(host: &Host, window: &str, config: RunConfig, on_output: Emit
let (root, settings) = settings_for(host, &project, &config)?;
let root = &root;
// Configurations made in the UI name bare tools (`godot`) that may live outside PATH.
// Maven re-applies too: a config listed before a settings change (daemon toggle,
// Maven home) keeps the launcher that was current then; re-resolving at launch is
// what makes the setting take effect without reopening the project.
let mut config = config;
if config.kind == "maven" {
ide_sdk::apply::prepare_maven_run(&mut config, root, &settings);
+27
View File
@@ -222,6 +222,29 @@ pub async fn vcs_blame(
.await
}
/// History for Selection: the commits that changed `lines` (0-based, inclusive) of the
/// editor's text of `path`. Cancellable.
pub async fn vcs_block_history(
host: &Host,
window: &str,
request: RequestId,
path: PathBuf,
text: Option<String>,
lines: (u32, u32),
) -> IdeResult<BlockHistory> {
let project = host.project(window)?;
let cancel = host.begin_request(request);
let result = blocking(move || {
let text = text.or_else(|| project.documents.text_of(&path));
project.with_repo_of(&path, |r| {
r.block_history(&path, text.as_deref(), lines, project.vcs_encoding(&path, None)?, &cancel)
})
})
.await;
host.end_request(request);
result
}
pub async fn vcs_log(host: &Host, window: &str, repo: Option<PathBuf>, query: LogQuery) -> IdeResult<Vec<Commit>> {
let project = host.project(window)?;
blocking(move || project.with_repo_at(repo.as_deref(), |r| r.log(&query))).await
@@ -987,6 +1010,10 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("vcs_blame", |c| async move {
vcs_blame(&c.host, &c.window, c.arg("path")?, c.arg("text")?, c.arg("revision")?).await
});
d.add("vcs_block_history", |c| async move {
let (request, path, text, lines) = (c.arg("request")?, c.arg("path")?, c.arg("text")?, c.arg("lines")?);
vcs_block_history(&c.host, &c.window, request, path, text, lines).await
});
d.add("vcs_log", |c| async move { vcs_log(&c.host, &c.window, c.arg("repo")?, c.arg("query")?).await });
d.add("vcs_commit_files", |c| async move {
vcs_commit_files(&c.host, &c.window, c.arg("repo")?, c.arg("commit")?).await
+129 -21
View File
@@ -3,11 +3,14 @@
//! plugins, and revocations. The host keeps the last index it read
//! (`market-index.json`), checks signatures against it, and disables what it
//! revokes — at start-up and on every refresh (the desktop refreshes daily).
//! Release builds pin the official publisher to the keys they were built
//! with, so no index can sign `openidea.*` plugins with another key.
use super::{PackageFile, PluginHost, PluginSource, PluginStatus};
use super::{PackageFile, PluginEvent, PluginHost, PluginSource, PluginStatus, PluginUpdate, install};
use ide_core::{IdeError, IdeResult};
use ide_extension::package::{self, Index, IndexEntry, PackageSignature, SIGNATURE_FILE};
use ide_extension::{MANIFEST_FILE, Manifest};
use semver::Version;
use serde::{Deserialize, Serialize};
use std::path::{Path, PathBuf};
@@ -15,6 +18,9 @@ use std::path::{Path, PathBuf};
/// the app, next to the signed packages of the official plugins.
pub const OFFICIAL_INDEX: &str = "https://github.com/l0ng-ai/openidea/releases/latest/download/plugins-index.json";
/// The publisher of the official plugins.
pub const OFFICIAL_PUBLISHER: &str = "openidea";
/// The last index read, in the plugins directory.
pub const INDEX_FILE: &str = "market-index.json";
const DOWNLOADS: &str = ".downloads";
@@ -22,6 +28,8 @@ const MAX_INDEX: usize = 16 << 20;
const MAX_PACKAGE: usize = 256 << 20;
const MAX_STAGED_FILES: usize = 256;
const MAX_STAGED_BYTES: usize = 16 << 20;
/// `moved` links followed before giving up (a loop, or a misconfiguration).
const MAX_MOVES: usize = 3;
/// A marketplace entry with what is installed.
#[derive(Clone, Debug, Serialize, Deserialize, specta::Type)]
@@ -31,6 +39,32 @@ pub struct MarketPlugin {
pub entry: IndexEntry,
/// Installed version, if any.
pub installed: Option<String>,
/// Whether this IDE can load `entry` (the newest version it can, else the newest).
pub compatible: bool,
}
impl MarketPlugin {
/// A version this IDE can load, newer than the one installed.
pub fn is_update(&self) -> bool {
let newer = |installed: &String| match (Version::parse(&self.entry.version), Version::parse(installed)) {
(Ok(offered), Ok(installed)) => offered > installed,
_ => false,
};
self.compatible && self.installed.as_ref().is_some_and(newer)
}
}
/// The official publisher's keys this IDE was built with
/// (`OPENIDEA_OFFICIAL_PLUGIN_KEYS`: base64, comma-separated). Development
/// builds have none: the index decides.
pub(super) fn built_in_official_keys() -> Vec<String> {
let keys = option_env!("OPENIDEA_OFFICIAL_PLUGIN_KEYS").unwrap_or_default();
keys.split(',').map(str::trim).filter(|k| !k.is_empty()).map(str::to_owned).collect()
}
fn pinned(mut index: Index, official_keys: &[String]) -> Index {
index.pin_publisher(OFFICIAL_PUBLISHER, "OpenIDEA", official_keys);
index
}
/// Reads `url`: `http(s)://…`, `file://…` or a local path.
@@ -58,6 +92,33 @@ pub fn read_url(url: &str, max: usize) -> Result<Vec<u8>, String> {
Ok(bytes)
}
/// The index at `url`, following `moved`, with downloads made absolute against
/// where it was read from.
fn read_index(url: &str) -> IdeResult<Index> {
let mut url = url.to_owned();
for _ in 0..=MAX_MOVES {
let bytes = read_url(&url, MAX_INDEX).map_err(IdeError::unavailable)?;
let text = String::from_utf8(bytes).map_err(|_| IdeError::invalid("the index is not UTF-8"))?;
let mut index = Index::parse(&text).map_err(IdeError::invalid)?;
match index.moved.take().filter(|m| !m.trim().is_empty()) {
Some(next) => {
let next = resolve(&url, &next);
if url.starts_with("https://") && !next.starts_with("https://") {
return Err(IdeError::invalid(format!("the marketplace moved to {next}, which isn't https")));
}
url = next;
}
None => {
for e in &mut index.plugins {
e.download = resolve(&url, &e.download);
}
return Ok(index);
}
}
}
Err(IdeError::unavailable(format!("the marketplace index moved more than {MAX_MOVES} times")))
}
/// `href` relative to the index at `base`: a URL, or a local path (with the
/// platform's separators, e.g. `C:\\market\\index.json`).
pub fn resolve(base: &str, href: &str) -> String {
@@ -77,9 +138,16 @@ pub fn resolve(base: &str, href: &str) -> String {
}
impl PluginHost {
pub(super) fn load_market_index(dir: &Path) -> Option<Index> {
pub(super) fn load_market_index(dir: &Path, official_keys: &[String]) -> Option<Index> {
let text = std::fs::read_to_string(dir.join(INDEX_FILE)).ok()?;
Index::parse(&text).map_err(|e| tracing::warn!("{INDEX_FILE}: {e}")).ok()
let index = Index::parse(&text).map_err(|e| tracing::warn!("{INDEX_FILE}: {e}")).ok()?;
Some(pinned(index, official_keys))
}
/// Pins the official publisher to `keys` (base64) instead of the built-in
/// ones, from the next index read.
pub fn set_official_keys(&self, keys: Vec<String>) {
*self.official_keys.write() = keys;
}
/// The marketplace index URL: the user's, else the default (none: no marketplace).
@@ -97,14 +165,16 @@ impl PluginHost {
self.save_state();
}
/// Reads the index again, keeps it, and applies its revocations.
/// Reads the index again, keeps it, and applies its revocations. Announces
/// updates of installed plugins it hasn't announced yet.
pub fn refresh_market(&self) -> IdeResult<Vec<MarketPlugin>> {
let url = self.market_url().ok_or_else(|| IdeError::unavailable("no plugin marketplace is configured"))?;
let bytes = read_url(&url, MAX_INDEX).map_err(IdeError::unavailable)?;
let text = String::from_utf8(bytes).map_err(|_| IdeError::invalid("the index is not UTF-8"))?;
let index = Index::parse(&text).map_err(IdeError::invalid)?;
let _ = std::fs::write(self.plugins_dir.join(INDEX_FILE), &text);
*self.market.write() = Some(index);
let index = read_index(&url)?;
if let Ok(json) = serde_json::to_string(&index) {
let _ = std::fs::write(self.plugins_dir.join(INDEX_FILE), json);
}
let newer = index.is_newer().then_some(index.format);
*self.market.write() = Some(pinned(index, &self.official_keys.read()));
// Trust may have changed: who signed what, what is revoked.
let installed: Vec<_> =
self.plugins.read().values().filter(|p| p.source == PluginSource::Installed).cloned().collect();
@@ -113,7 +183,41 @@ impl PluginHost {
plugin.meta.lock().publisher = publisher;
}
self.apply_revocations();
Ok(self.market_plugins())
if let Some(format) = newer {
return Err(IdeError::unavailable(format!(
"the marketplace needs a newer OpenIDEA (index format {format}); its revocations still apply"
)));
}
let plugins = self.market_plugins();
self.announce_updates(&plugins);
Ok(plugins)
}
/// Tells the windows about updates once per set of versions: a later
/// refresh offering the same ones says nothing.
fn announce_updates(&self, plugins: &[MarketPlugin]) {
let updates: Vec<PluginUpdate> = plugins
.iter()
.filter(|p| p.is_update())
.map(|p| PluginUpdate {
id: p.entry.id.clone(),
name: p.entry.name.clone(),
version: p.entry.version.clone(),
})
.collect();
let key: Vec<String> = updates.iter().map(|u| format!("{}@{}", u.id, u.version)).collect();
if std::mem::replace(&mut *self.announced_updates.lock(), key.clone()) != key && !updates.is_empty() {
self.emit(None, PluginEvent::UpdatesAvailable { updates });
}
}
/// Whether the package in `src` would replace the installed `id`'s copy
/// with one signed by another key than that copy, without that key's
/// endorsement: the user confirms the change of signer.
pub(super) fn signer_changed(&self, id: &str, src: &Path) -> bool {
let Some(current) = install::active_dir(&self.plugins_dir.join(id)) else { return false };
let market = self.market.read();
!package::signer_continues(market.as_ref(), read_signature(&current).as_ref(), read_signature(src).as_ref())
}
/// The plugins of the last index read.
@@ -121,16 +225,14 @@ impl PluginHost {
let Some(index) = self.market.read().clone() else { return Vec::new() };
let plugins = self.plugins.read();
index
.plugins
.iter()
.filter(|e| index.revocation(&e.id, &e.version).is_none())
.cloned()
.map(|entry| {
.offered()
.into_iter()
.map(|(entry, compatible)| {
let installed = plugins
.get(&entry.id)
.filter(|p| p.source == PluginSource::Installed)
.map(|p| p.manifest().version.clone());
MarketPlugin { entry, installed }
MarketPlugin { entry: entry.clone(), installed, compatible }
})
.collect()
}
@@ -141,12 +243,18 @@ impl PluginHost {
pub fn download_market_plugin(&self, id: &str) -> IdeResult<PathBuf> {
let url = self.market_url().ok_or_else(|| IdeError::unavailable("no plugin marketplace is configured"))?;
let index = self.market.read().clone().ok_or_else(|| IdeError::unavailable("refresh the marketplace first"))?;
let entry = index
.plugins
.iter()
.find(|e| e.id == id)
.cloned()
let (entry, compatible) = index
.offered()
.into_iter()
.find(|(e, _)| e.id == id)
.map(|(e, ok)| (e.clone(), ok))
.ok_or_else(|| IdeError::not_found(format!("{id} is not in the marketplace")))?;
if !compatible {
return Err(IdeError::unavailable(format!(
"{} {} needs plugin API {}: update OpenIDEA",
entry.name, entry.version, entry.engine
)));
}
let bytes = read_url(&resolve(&url, &entry.download), MAX_PACKAGE).map_err(IdeError::unavailable)?;
let downloads = self.plugins_dir.join(DOWNLOADS);
std::fs::create_dir_all(&downloads)?;
+49 -6
View File
@@ -36,7 +36,10 @@ pub mod web;
pub use api::{FileSecretStore, HostWorkspace, PluginWorkspace, SecretStore};
pub use icons::{map_asset as icon_theme_map_asset, sanitize_svg, svg_prefix as icon_theme_svg_prefix};
pub use market::{INDEX_FILE as MARKET_INDEX_FILE, MarketPlugin, OFFICIAL_INDEX as OFFICIAL_MARKET_INDEX};
pub use market::{
INDEX_FILE as MARKET_INDEX_FILE, MarketPlugin, OFFICIAL_INDEX as OFFICIAL_MARKET_INDEX, OFFICIAL_PUBLISHER,
read_url as read_market_url, resolve as resolve_market_href,
};
pub use points::{COMMANDS, CommandDecl, register_builtin_points};
pub(crate) use providers::editor_providers as editor_providers_for;
pub use providers::{ProviderKind, set_editor_providers};
@@ -242,12 +245,16 @@ pub struct Consent {
/// Optional capabilities the user accepted (`net:crates.io`, …). Required
/// ones are implied by consenting at all.
pub optional: Vec<String>,
/// The user accepted a package signed by another key than the installed
/// copy ([`InstallPreview::signer_changed`]).
#[serde(default)]
pub signer_change: bool,
}
impl Consent {
/// Accept everything the plugin asks for (tests, trusted official plugins).
pub fn all() -> Self {
Self { optional: vec!["*".into()] }
Self { optional: vec!["*".into()], signer_change: true }
}
fn accepts(&self, cap: &Capability) -> bool {
@@ -274,6 +281,14 @@ pub struct InstallPreview {
pub unsigned: bool,
/// The verified publisher.
pub publisher: Option<String>,
/// Replaces an installed copy signed with another key, which didn't
/// endorse this package's (or this one is unsigned): the user confirms it
/// ([`Consent::signer_change`]).
pub signer_changed: bool,
/// Something to confirm: a first install, an unsigned package, another
/// signer, or capabilities the installed version wasn't granted or
/// refused. Without, an update installs with the grants it has.
pub needs_consent: bool,
}
type ProjectPolicy = Arc<dyn Fn(&Path) -> bool + Send + Sync>;
@@ -320,6 +335,10 @@ pub struct PluginHost {
pins: Arc<RwLock<HashMap<(String, String), ProgramPin>>>,
/// The last marketplace index read: trust root for signatures, revocations.
market: RwLock<Option<ide_extension::package::Index>>,
/// The official publisher's keys the index is pinned to (none: as listed).
official_keys: RwLock<Vec<String>>,
/// `id@version` of the updates last announced.
announced_updates: Mutex<Vec<String>>,
/// The marketplace used while the user has set none (the app's official one).
default_market_url: RwLock<Option<String>>,
/// Updates not confirmed yet (the new version hasn't started once): what
@@ -358,9 +377,12 @@ impl PluginHost {
policy_pins.read().get(&(plugin.to_owned(), cap.to_string())).cloned()
}));
let shared = api::Shared::new(sink.clone(), plugins_dir.join(".data"), pins.clone(), ui.clone());
let market = RwLock::new(Self::load_market_index(&plugins_dir));
let official_keys = market::built_in_official_keys();
let market = RwLock::new(Self::load_market_index(&plugins_dir, &official_keys));
let host = Self {
market,
official_keys: RwLock::new(official_keys),
announced_updates: Mutex::default(),
default_market_url: RwLock::new(None),
updates: Mutex::default(),
project_policy: RwLock::new(Arc::new(|_| false)),
@@ -703,14 +725,27 @@ impl PluginHost {
}
let publisher = self.verified_publisher(src);
let m = &pkg.manifest;
let replaces = self.plugins.read().get(&m.id).map(|p| p.manifest().version.clone());
// An update shows what the installed version was granted.
let grant = replaces
.as_ref()
.and_then(|_| self.ledger.lock().get(&Subject::Installed(m.id.clone())).cloned())
.filter(|g| g.confirmed);
let granted = |c: &Capability| grant.as_ref().is_some_and(|g| g.has(c));
let mut capabilities: Vec<CapabilityInfo> =
m.capabilities.all().map(|c| CapabilityInfo::new(c, m.capabilities.is_optional(c), false)).collect();
m.capabilities.all().map(|c| CapabilityInfo::new(c, m.capabilities.is_optional(c), granted(c))).collect();
for extra in &set.capabilities {
if !m.declares(extra) {
capabilities.push(CapabilityInfo::new(extra, true, false));
capabilities.push(CapabilityInfo::new(extra, true, granted(extra)));
}
}
capabilities.sort_by(|a, b| b.risk.cmp(&a.risk).then(a.id.cmp(&b.id)));
let signer_changed = self.signer_changed(&m.id, src);
let answered = |c: &CapabilityInfo| {
c.granted || (c.optional && grant.as_ref().is_some_and(|g| g.declined.contains(&c.id)))
};
let needs_consent =
grant.is_none() || publisher.is_none() || signer_changed || !capabilities.iter().all(answered);
Ok(InstallPreview {
id: m.id.clone(),
name: m.name.clone(),
@@ -721,9 +756,11 @@ impl PluginHost {
capabilities,
contributions: counts(m),
warnings: set.warnings.clone(),
replaces: self.plugins.read().get(&m.id).map(|p| p.manifest().version.clone()),
replaces,
unsigned: publisher.is_none(),
publisher,
signer_changed,
needs_consent,
})
}
@@ -746,6 +783,11 @@ impl PluginHost {
if let Some(problem) = self.signature_problem(src) {
return Err(IdeError::invalid(format!("refusing {id}: {problem}")));
}
if !consent.signer_change && self.signer_changed(&id, src) {
return Err(IdeError::invalid(format!(
"refusing {id}: it is signed by another key than the installed version; confirm the change of signer"
)));
}
let version = pkg.manifest.version.clone();
if let Some(r) = self.market.read().as_ref().and_then(|i| i.revocation(&id, &version)) {
return Err(IdeError::invalid(format!("{id} {version} was revoked: {}", r.reason)));
@@ -2211,6 +2253,7 @@ fn placeholder_manifest(dir: &Path) -> Manifest {
homepage: None,
license: None,
change_notes: None,
declared_categories: Vec::new(),
engine: ide_extension::ENGINE_VERSION.into(),
legacy: false,
wasm: None,
+15
View File
@@ -292,6 +292,21 @@ impl PluginHost {
.collect()
}
/// The edits of an action `plugin_id` returned with `data`, now that the
/// user picked it: a call with the command budget, not the edit path's.
pub fn resolve_code_action(
&self,
plugin_id: &str,
doc: &ProviderDocument,
action: CodeAction,
) -> ide_core::IdeResult<CodeAction> {
let d = Arc::new(doc.clone());
let timeout = self.limits.timeout;
self.code_call(plugin_id, ide_plugin::Invocation::default(), "resolving the code action", move |rt, _| {
rt.resolve_code_action(&d, &action, timeout)
})
}
pub fn provide_definition(&self, doc: &ProviderDocument, at: ide_core::Pos) -> Vec<Location> {
let d = Arc::new(doc.clone());
self.edit_calls(ProviderKind::Definition, &doc.language, move |rt, timeout| {
+2 -1
View File
@@ -272,7 +272,8 @@ impl RemotePlugins {
report.failed.push(format!("{}: invalid plugin id", pkg.id));
continue;
}
let consent = Consent { optional: pkg.granted.clone() };
// The desktop asked the user, change of signer included.
let consent = Consent { optional: pkg.granted.clone(), signer_change: true };
if self.current_digest(&pkg.id).as_deref() == Some(pkg.digest.as_str()) {
if let Err(e) = self.follow_grants(&pkg, &consent) {
report.failed.push(format!("{}: {e}", pkg.id));
+12
View File
@@ -138,12 +138,24 @@ pub struct PluginsSnapshot {
pub ui: Vec<UiContribution>,
}
/// A newer version of an installed plugin in the marketplace.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PluginUpdate {
pub id: String,
pub name: String,
pub version: String,
}
/// Backend → UI notifications (Tauri event `ide://plugin`).
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(tag = "type", rename_all = "camelCase")]
pub enum PluginEvent {
/// Plugins, their status or their contributions changed: re-fetch `plugin_list`.
CommandsChanged,
/// The marketplace offers newer versions of installed plugins (announced
/// once per set of versions).
UpdatesAvailable { updates: Vec<PluginUpdate> },
#[serde(rename_all = "camelCase")]
Notification { plugin_id: String, level: NotificationLevel, title: String, body: String },
/// A plugin's status bar item was set (`item`) or removed.
+100 -8
View File
@@ -56,6 +56,9 @@ pub struct Project {
pub(crate) watcher: Watcher,
/// Watchers of the Project Structure content roots outside the project folder.
pub(crate) content_watchers: Mutex<Vec<(PathBuf, Watcher)>>,
/// Watchers of folders outside the project that language servers registered file
/// watchers for (a virtualenv elsewhere, a path dependency).
lsp_watchers: Mutex<Vec<(PathBuf, Watcher)>>,
pub(crate) _scratch_watcher: Option<Watcher>,
/// A LightEdit window's loose files ([`Host::open_light`]): global settings only,
/// nothing indexed, no language servers, repositories or project services.
@@ -218,6 +221,8 @@ impl Host {
ide_sdk::jdtls::launch_directly(cfg);
}
ide_sdk::apply::server_protocol_overrides(cfg, &effective);
// After the overrides: the PHP version they may set decides the storage.
ide_sdk::php::configure_intelephense_storage(cfg, &data_dir);
if cfg.name == "jdtls" {
let (excluded_poms, build_files) = jdtls_maven_import(root, &effective);
ide_sdk::apply::configure_maven_import(cfg, root, &excluded_poms, build_files.as_deref());
@@ -302,12 +307,16 @@ impl Host {
sink.clone(),
Box::new(move |batch| {
let changed: Vec<PathBuf> = batch.created.iter().chain(&batch.modified).cloned().collect();
index.update(&changed, &batch.removed);
let (ignored, ignored_gone) = ignored_changes(batch);
index.update(&[&changed[..], &ignored].concat(), &[&batch.removed[..], &ignored_gone].concat());
kotlin_bridge.files_changed(changed.iter().chain(&batch.removed));
lsp.project_files_changed(batch.created.iter().chain(&batch.removed));
lsp.watched_files_changed(&batch.created, &batch.modified, &batch.removed);
lsp.watched_files_changed(&batch.created, &batch.modified, &batch.removed, &batch.excluded_dirs);
if composer_changed(&root, changed.iter().chain(&batch.removed)) {
lsp.reconfigure_soon();
}
// Atomic saves by external tools surface as `created`.
documents.on_disk_changed(&changed);
documents.on_disk_changed(&[&changed[..], &ignored].concat());
// A framework's root marker (`angular.json`) created or deleted: its
// templates gain or lose their mode. Markers in `node_modules` only
// count here when the folder isn't excluded (it is by default).
@@ -411,7 +420,16 @@ impl Host {
let content_watchers = extra_roots
.iter()
.filter_map(|r| {
let w = content_root_watcher(r, &excludes, &sink, &index, &documents, &history, Some(&kotlin_bridge))?;
let w = content_root_watcher(
r,
&excludes,
&sink,
&index,
&documents,
&history,
&lsp,
Some(&kotlin_bridge),
)?;
Some((r.clone(), w))
})
.collect();
@@ -458,6 +476,7 @@ impl Host {
kotlin_bridge: Some(kotlin_bridge),
watcher,
content_watchers: Mutex::new(content_watchers),
lsp_watchers: Mutex::new(Vec::new()),
_scratch_watcher: scratch_watcher,
light: false,
});
@@ -467,6 +486,14 @@ impl Host {
weak.upgrade().map(|p| p.reference_candidates(q, cancel)).unwrap_or_default()
}));
}
{
let weak = Arc::downgrade(&project);
project.lsp.set_file_watch_roots_hook(Box::new(move |roots| {
if let Some(p) = weak.upgrade() {
p.watch_for_language_servers(roots);
}
}));
}
{
let weak = Arc::downgrade(&project);
project.lsp.set_project_excludes_hook(Box::new(move || {
@@ -582,6 +609,7 @@ impl Host {
kotlin_bridge: None,
watcher,
content_watchers: Mutex::new(Vec::new()),
lsp_watchers: Mutex::new(Vec::new()),
_scratch_watcher: None,
light: true,
root: dir,
@@ -816,7 +844,8 @@ impl Project {
/// Project files of `q.languages` under `q.root` containing `q.word` as a whole
/// word (unsaved buffers included; excluded roots, ignored names and
/// `.gitignore`d files skipped): the files a references search opens on its server.
/// `.gitignore`d folders skipped, while `.gitignore`d files elsewhere count, as
/// in IDEA's project scope): the files a references search opens on its server.
pub fn reference_candidates(
&self,
q: &ide_lsp::CandidateQuery,
@@ -894,6 +923,7 @@ impl Project {
&self.index,
&self.documents,
&self.history,
&self.lsp,
self.kotlin_bridge.as_ref(),
) {
watchers.push((r, w));
@@ -948,6 +978,11 @@ const HISTORY_BATCH_LIMIT: usize = 500;
/// Local History for changes made outside the IDE (other tools, VCS, deletions).
/// Contents equal to the last snapshot (e.g. the IDE's own saves) are skipped.
/// The batch's ignored files (`.env`), split into those still on disk and those removed.
fn ignored_changes(batch: &ide_vfs::FsBatch) -> (Vec<PathBuf>, Vec<PathBuf>) {
batch.ignored.iter().cloned().partition(|p| p.symlink_metadata().is_ok())
}
fn record_external_changes(history: &LocalHistory, changed: &[PathBuf], removed: &[PathBuf]) {
for path in changed.iter().take(HISTORY_BATCH_LIMIT) {
if let Err(e) = history.record_file(path, "External change", false) {
@@ -991,6 +1026,31 @@ fn content_roots(settings: &ide_core::settings::Settings, root: &Path, build_cac
/// Watches a content root outside the project: the index, open documents and
/// Local History follow its changes like the project folder's.
impl Project {
/// Watches `roots` (outside the project) for the language servers that asked to.
fn watch_for_language_servers(&self, roots: Vec<PathBuf>) {
let mut watchers = self.lsp_watchers.lock();
watchers.retain(|(r, _)| roots.contains(r));
for root in roots {
if watchers.iter().any(|(r, _)| *r == root) || !root.is_dir() {
continue;
}
let lsp = Arc::downgrade(&self.lsp);
let on_change = Box::new(move |batch: &ide_vfs::FsBatch| {
if let Some(lsp) = lsp.upgrade() {
lsp.watched_files_changed(&batch.created, &batch.modified, &batch.removed, &batch.excluded_dirs);
}
});
// Not project files: nothing else hears of them.
match Watcher::start(&root, Vec::new(), Arc::new(ide_core::NullSink), on_change) {
Ok(w) => watchers.push((root, w)),
Err(e) => tracing::warn!("language server watcher {}: {e}", root.display()),
}
}
}
}
#[allow(clippy::too_many_arguments)]
fn content_root_watcher(
dir: &Path,
excludes: &[String],
@@ -998,21 +1058,28 @@ fn content_root_watcher(
index: &Arc<ProjectIndex>,
documents: &Arc<Documents>,
history: &Arc<LocalHistory>,
lsp: &Arc<LspManager>,
kotlin_bridge: Option<&Arc<crate::kotlin_bridge::KotlinBridge>>,
) -> Option<Watcher> {
let kotlin_bridge = kotlin_bridge.cloned();
let (index, documents, history) = (index.clone(), documents.clone(), history.clone());
let (index, documents, history, lsp) = (index.clone(), documents.clone(), history.clone(), lsp.clone());
let root = dir.to_path_buf();
Watcher::start(
dir,
excludes.to_vec(),
sink.clone(),
Box::new(move |batch| {
let changed: Vec<PathBuf> = batch.created.iter().chain(&batch.modified).cloned().collect();
index.update(&changed, &batch.removed);
let (ignored, ignored_gone) = ignored_changes(batch);
index.update(&[&changed[..], &ignored].concat(), &[&batch.removed[..], &ignored_gone].concat());
// A workspace member's PHP language level follows its own composer.json.
if composer_changed(&root, changed.iter().chain(&batch.removed)) {
lsp.reconfigure_soon();
}
if let Some(bridge) = &kotlin_bridge {
bridge.files_changed(changed.iter().chain(&batch.removed));
}
documents.on_disk_changed(&changed);
documents.on_disk_changed(&[&changed[..], &ignored].concat());
record_external_changes(&history, &changed, &batch.removed);
}),
)
@@ -1020,6 +1087,13 @@ fn content_root_watcher(
.ok()
}
/// Whether the `composer.json` of `root` is among `paths`: the PHP language level follows it
/// (`ide_sdk::php`).
fn composer_changed<'a>(root: &Path, mut paths: impl Iterator<Item = &'a PathBuf>) -> bool {
let composer = root.join("composer.json");
paths.any(|p| *p == composer)
}
/// Where agent tasks create their worktrees: next to the project, `<parent>/<name>.worktrees`,
/// so they sit on the project's drive and outside its tree (not indexed, watched or shown in
/// its git status); `fallback` for a project at a filesystem root.
@@ -1436,3 +1510,21 @@ mod agent_worktree_tests {
assert_eq!(agent_worktrees_dir(Path::new("/"), fallback), fallback());
}
}
#[cfg(test)]
mod composer_tests {
use super::composer_changed;
use std::path::{Path, PathBuf};
#[test]
fn only_the_roots_own_composer_json_counts() {
let root = Path::new("/work/app");
let changed = |paths: &[&str]| {
let paths: Vec<PathBuf> = paths.iter().map(|p| root.join(p)).collect();
composer_changed(root, paths.iter())
};
assert!(changed(&["src/a.php", "composer.json"]));
assert!(!changed(&["composer.lock", "vendor/acme/lib/composer.json", "packages/b/composer.json"]));
assert!(!changed(&[]));
}
}
+31 -1
View File
@@ -226,6 +226,7 @@ pub fn restricted_command_allowed(cmd: &str) -> bool {
| "deploy_coverage"
| "deploy_test_connection"
| "find_files"
| "find_dirs"
| "find_symbols"
| "mark_recent"
| "remove_recent"
@@ -274,6 +275,15 @@ pub fn restricted_command_allowed(cmd: &str) -> bool {
| "plugin_set_theme_frame"
| "plugin_catalog"
| "plugin_assets"
| "plugin_market"
| "plugin_market_url"
| "plugin_set_market_url"
| "plugin_market_download"
| "plugin_unpack_package"
| "plugin_stage_package"
| "plugin_preview_install"
| "plugin_read_intellij_theme"
| "plugin_installed_intellij_themes"
| "notebook_open"
| "notebook_get"
| "notebook_edit"
@@ -295,6 +305,7 @@ pub fn restricted_command_allowed(cmd: &str) -> bool {
| "vcs_line_hunks"
| "vcs_file_at"
| "vcs_blame"
| "vcs_block_history"
| "vcs_log"
| "vcs_commit_files"
| "vcs_commits_changes"
@@ -319,12 +330,31 @@ pub fn restricted_command_allowed(cmd: &str) -> bool {
) || cmd.starts_with("plugin:")
}
/// Commands that change which plugin code runs: allowed where no project is open (the
/// welcome screen), otherwise only in a trusted project.
pub fn projectless_command_allowed(cmd: &str) -> bool {
matches!(
cmd,
"plugin_install_dir"
| "plugin_uninstall"
| "plugin_set_enabled"
| "plugin_grant"
| "plugin_set_capability"
| "plugin_roll_back"
| "plugin_reload"
)
}
impl crate::Host {
pub fn authorize_command(&self, window: &str, command: &str) -> IdeResult<()> {
if restricted_command_allowed(command) {
return Ok(());
}
self.project(window)?.require_trust()
match self.project(window) {
Ok(project) => project.require_trust(),
Err(_) if projectless_command_allowed(command) => Ok(()),
Err(error) => Err(error),
}
}
}
+236 -9
View File
@@ -131,25 +131,44 @@ fn installs_from_before_versioning_keep_working_and_move_on_update() {
/// `hello` as `acme.hello` at `version`, packed and signed with `key`.
fn signed_hello(dir: &Path, version: &str, key: &PublisherKey, host: &PluginHost) -> (PathBuf, IndexEntry) {
let src = variant(&repo().join("crates/ide-plugin/tests/fixtures/hello"), dir, &format!("src-{version}"), |t| {
t.replace("id = \"hello\"", "id = \"acme.hello\"")
.replace("version = \"0.3.0\"", &format!("version = \"{version}\""))
});
signed(dir, "acme", version, key, host, |t| t.to_owned())
}
/// `hello` as `<publisher>.hello` at `version`, its manifest changed by
/// `edit`, packed and signed with `key`.
fn signed(
dir: &Path,
publisher: &str,
version: &str,
key: &PublisherKey,
host: &PluginHost,
edit: impl Fn(&str) -> String,
) -> (PathBuf, IndexEntry) {
let id = format!("{publisher}.hello");
let src =
variant(&repo().join("crates/ide-plugin/tests/fixtures/hello"), dir, &format!("src-{id}-{version}"), |t| {
edit(
&t.replace("id = \"hello\"", &format!("id = \"{id}\""))
.replace("version = \"0.3.0\"", &format!("version = \"{version}\"")),
)
});
let files = host.package_files_of(&src).unwrap();
let digest = package::package_digest(&src, &files).unwrap();
let sig = key.sign("acme", "acme.hello", version, &digest);
let out = dir.join(format!("acme.hello-{version}.oiplugin"));
let sig = key.sign(publisher, &id, version, &digest);
let out = dir.join(format!("{id}-{version}.oiplugin"));
package::pack(&src, &files, &out, Some(&sig)).unwrap();
let entry = IndexEntry {
id: "acme.hello".into(),
id: id.clone(),
name: "Hello".into(),
version: version.into(),
description: String::new(),
authors: vec![],
publisher: "acme".into(),
publisher: publisher.into(),
download: out.file_name().unwrap().to_string_lossy().into_owned(),
digest,
capabilities: vec![],
engine: "^1.0".into(),
categories: vec!["tools".into()],
};
(out, entry)
}
@@ -166,10 +185,11 @@ fn write_index(
publishers: vec![Publisher {
id: "acme".into(),
name: "ACME".into(),
keys: vec![KeyEntry { key: key.public_base64(), revoked: key_revoked }],
keys: vec![KeyEntry { revoked: key_revoked, ..KeyEntry::new(key.public_base64()) }],
}],
plugins,
revoked,
moved: None,
};
let path = dir.join("index.json");
std::fs::write(&path, serde_json::to_string(&index).unwrap()).unwrap();
@@ -260,3 +280,210 @@ fn package_files_install_from_disk_unsigned() {
let info = f.host.install_from_dir(&dir, &Consent::all()).unwrap();
assert_eq!((info.status, info.publisher), (PluginStatus::Inactive, None));
}
#[test]
fn the_marketplace_offers_the_newest_version_this_ide_can_load() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, old) = signed_hello(&market, "1.0.0", &key, &f.host);
let (_, new) = signed_hello(&market, "2.0.0", &key, &f.host);
let next_major = ide_extension::ENGINE_VERSION.split('.').next().unwrap().parse::<u64>().unwrap() + 1;
let new = IndexEntry { engine: format!("^{next_major}.0"), ..new };
f.host.set_market_url(Some(write_index(&market, &key, vec![old.clone(), new.clone()], vec![], false)));
let listed = f.host.refresh_market().unwrap();
assert_eq!((listed.len(), listed[0].entry.version.as_str(), listed[0].compatible), (1, "1.0.0", true));
let dir = f.host.download_market_plugin("acme.hello").unwrap();
assert_eq!(f.host.preview_install(&dir).unwrap().version, "1.0.0");
// Only a version for a newer IDE: shown as such, not downloaded.
write_index(&market, &key, vec![new], vec![], false);
let listed = f.host.refresh_market().unwrap();
assert_eq!((listed[0].entry.version.as_str(), listed[0].compatible), ("2.0.0", false));
let err = f.host.download_market_plugin("acme.hello").unwrap_err();
assert!(err.message.contains("update OpenIDEA"), "{}", err.message);
}
#[test]
fn a_moved_marketplace_is_read_where_it_moved_to() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, entry) = signed_hello(&market, "1.0.0", &key, &f.host);
write_index(&market, &key, vec![entry], vec![], false);
// The old address points at the new one; downloads resolve against where the index is.
let old = f.work.join("old");
std::fs::create_dir_all(&old).unwrap();
std::fs::write(old.join("index.json"), r#"{"format": 1, "moved": "../market/index.json"}"#).unwrap();
f.host.set_market_url(Some(old.join("index.json").to_string_lossy().into_owned()));
let listed = f.host.refresh_market().unwrap();
assert_eq!(listed[0].entry.id, "acme.hello");
let dir = f.host.download_market_plugin("acme.hello").unwrap();
assert_eq!(f.host.preview_install(&dir).unwrap().version, "1.0.0");
// A loop gives up.
std::fs::write(old.join("index.json"), r#"{"format": 1, "moved": "index.json"}"#).unwrap();
let err = f.host.refresh_market().unwrap_err();
assert!(err.message.contains("moved more than"), "{}", err.message);
}
/// `dir/index.json` listing `publishers` and `plugins`; its path.
fn index_of(dir: &Path, publishers: Vec<Publisher>, plugins: Vec<IndexEntry>) -> String {
let index = Index { format: package::INDEX_FORMAT, publishers, plugins, ..Index::default() };
let path = dir.join("index.json");
std::fs::write(&path, serde_json::to_string(&index).unwrap()).unwrap();
path.to_string_lossy().into_owned()
}
fn publisher(id: &str, keys: Vec<KeyEntry>) -> Publisher {
Publisher { id: id.into(), name: id.into(), keys }
}
fn updates_announced(events: &Events) -> Vec<Vec<String>> {
events
.lock()
.iter()
.filter_map(|e| match e {
PluginEvent::UpdatesAvailable { updates } => {
Some(updates.iter().map(|u| format!("{} {}", u.id, u.version)).collect())
}
_ => None,
})
.collect()
}
/// What the install page sends for an update it doesn't need to show: the grants the installed version has.
fn as_granted(preview: &InstallPreview) -> Consent {
let optional = preview.capabilities.iter().filter(|c| c.optional && c.granted).map(|c| c.id.clone()).collect();
Consent { optional, signer_change: false }
}
#[test]
fn updates_keep_their_signer_and_need_no_consent_when_nothing_is_new() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let (k1, k2, k3) =
(PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([2; 32]), PublisherKey::from_seed([3; 32]));
let same = |t: &str| t.to_owned();
let (_, v1) = signed(&market, "acme", "1.0.0", &k1, &f.host, same);
f.host.set_market_url(Some(index_of(
&market,
vec![publisher("acme", vec![KeyEntry::new(k1.public_base64())])],
vec![v1.clone()],
)));
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let first = f.host.preview_install(&dir).unwrap();
assert!(first.needs_consent && !first.signer_changed, "a first install is confirmed");
f.host.install_from_dir(&dir, &Consent::default()).unwrap();
assert!(updates_announced(&f.events).is_empty());
// 1.1.0, same key and needs: announced once, installed with the grants it has.
let (_, v11) = signed(&market, "acme", "1.1.0", &k1, &f.host, same);
let keys1 = vec![KeyEntry::new(k1.public_base64())];
index_of(&market, vec![publisher("acme", keys1.clone())], vec![v1.clone(), v11.clone()]);
f.host.refresh_market().unwrap();
f.host.refresh_market().unwrap();
assert_eq!(updates_announced(&f.events), [["acme.hello 1.1.0"]], "once per set of versions");
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert_eq!(preview.replaces.as_deref(), Some("1.0.0"));
assert!(!preview.needs_consent && !preview.signer_changed);
assert!(preview.capabilities.iter().all(|c| c.granted), "shows what the installed version was granted");
f.host.install_from_dir(&dir, &as_granted(&preview)).unwrap();
assert!(f.host.market_plugins().iter().all(|p| !p.is_update()));
// 1.2.0 asks for something new: the user is asked.
let (_, v12) = signed(&market, "acme", "1.2.0", &k1, &f.host, |t| {
t.replace(
"required = [\"editor.read\", \"editor.write\"]",
"required = [\"editor.read\", \"editor.write\"]\noptional = [\"net:example.com\"]",
)
});
index_of(&market, vec![publisher("acme", keys1.clone())], vec![v12.clone()]);
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert!(preview.needs_consent && !preview.signer_changed);
f.host.install_from_dir(&dir, &Consent::default()).unwrap();
// 1.3.0 signed with a key the installed one didn't endorse: another signer, the user's call.
let (_, v13) = signed(&market, "acme", "1.3.0", &k2, &f.host, same);
let keys12 = vec![KeyEntry::new(k1.public_base64()), KeyEntry::new(k2.public_base64())];
index_of(&market, vec![publisher("acme", keys12)], vec![v13]);
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert!(preview.signer_changed && preview.needs_consent);
let err = f.host.install_from_dir(&dir, &as_granted(&preview)).unwrap_err();
assert!(err.message.contains("another key"), "{}", err.message);
assert_eq!(f.host.info_of("acme.hello").unwrap().version, "1.2.0", "nothing changed");
f.host.install_from_dir(&dir, &Consent { signer_change: true, ..as_granted(&preview) }).unwrap();
// 1.4.0 signed with k3, which k2 endorsed: a rotation the installed signer vouched for.
let (_, v14) = signed(&market, "acme", "1.4.0", &k3, &f.host, same);
let keys = vec![
KeyEntry { endorsement: Some(k2.endorse("acme", &k3.public_base64())), ..KeyEntry::new(k3.public_base64()) },
KeyEntry::new(k2.public_base64()),
];
index_of(&market, vec![publisher("acme", keys)], vec![v14]);
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert!(!preview.signer_changed && !preview.needs_consent);
f.host.install_from_dir(&dir, &as_granted(&preview)).unwrap();
assert_eq!(f.host.info_of("acme.hello").unwrap().version, "1.4.0");
}
#[test]
fn an_index_in_a_newer_format_still_revokes() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, entry) = signed_hello(&market, "1.0.0", &key, &f.host);
f.host.set_market_url(Some(write_index(&market, &key, vec![entry], vec![], false)));
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
f.host.install_from_dir(&dir, &Consent::all()).unwrap();
let newer = format!(
r#"{{"format": {}, "plugins": "a shape this IDE can't read",
"revoked": [{{"id": "acme.hello", "reason": "steals tokens"}}]}}"#,
package::INDEX_FORMAT + 1
);
std::fs::write(market.join("index.json"), newer).unwrap();
let err = f.host.refresh_market().unwrap_err();
assert!(err.message.contains("newer OpenIDEA"), "{}", err.message);
let info = f.host.info_of("acme.hello").unwrap();
assert_eq!((info.status, info.error.as_deref()), (PluginStatus::Failed, Some("revoked: steals tokens")));
assert!(f.host.market_plugins().is_empty());
}
#[test]
fn official_plugins_need_the_key_this_ide_was_built_with() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let (root, intruder) = (PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([6; 32]));
f.host.set_official_keys(vec![root.public_base64()]);
// The index lists the intruder's key for the official publisher.
let (_, forged) = signed(&market, OFFICIAL_PUBLISHER, "1.0.0", &intruder, &f.host, |t| t.to_owned());
let keys = vec![KeyEntry::new(intruder.public_base64())];
f.host.set_market_url(Some(index_of(&market, vec![publisher(OFFICIAL_PUBLISHER, keys.clone())], vec![forged])));
f.host.refresh_market().unwrap();
let err = f.host.download_market_plugin("openidea.hello").unwrap_err();
assert!(err.message.contains("trusted publisher"), "{}", err.message);
// Signed with the built-in key it installs, whatever keys the index lists.
let (_, genuine) = signed(&market, OFFICIAL_PUBLISHER, "1.0.1", &root, &f.host, |t| t.to_owned());
index_of(&market, vec![publisher(OFFICIAL_PUBLISHER, keys)], vec![genuine]);
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("openidea.hello").unwrap();
assert_eq!(f.host.preview_install(&dir).unwrap().publisher.as_deref(), Some(OFFICIAL_PUBLISHER));
}
+714
View File
@@ -0,0 +1,714 @@
//! Language smoke tests: the everyday code intelligence of each language — completion, go to
//! definition, Find Usages, hover, diagnostics with a quick fix, an intention, rename and
//! formatting — on its `fixtures/` project, through the `ops` the app's commands call and the
//! language server the app starts by default.
//!
//! `#[ignore]`d because they need the servers installed. CI's `Language smoke` job installs
//! pinned versions and runs one language per job:
//! `cargo test -p ide-host --test language_smoke -- --ignored --exact <language>`.
use ide_core::{ChangeSet, DocId, IdeEvent, Pos, Range, RequestId, TextChange};
use ide_host::{Host, HostConfig, ops};
use ide_lsp::types::{FormatOptions, GotoKind};
use std::collections::BTreeSet;
use std::future::Future;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
const WINDOW: &str = "main";
/// `word` on the first line of `file` that contains `line`.
#[derive(Clone, Copy, Debug)]
struct At {
file: &'static str,
line: &'static str,
word: &'static str,
}
const fn at(file: &'static str, line: &'static str, word: &'static str) -> At {
At { file, line, word }
}
/// A line typed after the first line of `file` containing `after`.
#[derive(Clone, Copy, Debug)]
struct Typed {
file: &'static str,
after: &'static str,
text: &'static str,
}
const fn typed(file: &'static str, after: &'static str, text: &'static str) -> Typed {
Typed { file, after, text }
}
/// One language's checklist. Every check names what it looks for, so a failure says which
/// operation broke.
struct Checks {
fixture: &'static str,
/// Files written into the project copy before it opens.
files: &'static [(&'static str, &'static str)],
/// How long the first answer may take (project import and indexing).
warm_up: Duration,
/// Go to definition from the first spot lands on the second.
definition: (At, At),
/// Find Usages of the symbol lists usages in these files.
references: (At, &'static [&'static str]),
/// Hover shows this text.
hover: (At, &'static str),
/// Completion at the end of the typed line offers this item.
completion: (Typed, &'static str),
/// The typed line is reported with this diagnostic; the quick fix with this title makes the
/// file contain the last text.
quick_fix: QuickFix,
/// With the word selected, this intention is offered with an edit or command.
intention: (At, &'static str),
/// Formatting turns the typed line into the expected one.
format: (Typed, &'static str),
indent: Indent,
/// Renaming the symbol to the new name edits these files.
rename: (At, &'static str, &'static [&'static str]),
}
#[derive(Clone, Copy, Debug)]
struct QuickFix {
typed: Typed,
diagnostic: &'static str,
title: &'static str,
result: &'static str,
}
/// The editor's indentation, sent with a format request.
#[derive(Clone, Copy, Debug)]
enum Indent {
Spaces(u32),
Tabs,
}
type Events = Arc<Mutex<Vec<IdeEvent>>>;
struct Smoke {
language: &'static str,
host: Arc<Host>,
root: PathBuf,
events: Events,
warm_up: Duration,
_tmp: tempfile::TempDir,
}
fn fixture_dir(name: &str) -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR")).join("../../fixtures").join(name)
}
fn copy_dir(from: &Path, to: &Path) {
std::fs::create_dir_all(to).unwrap();
for entry in std::fs::read_dir(from).unwrap() {
let entry = entry.unwrap();
let name = entry.file_name();
if matches!(name.to_str(), Some("node_modules" | "target" | "build" | ".gradle" | ".venv")) {
continue;
}
let target = to.join(&name);
if entry.file_type().unwrap().is_dir() {
copy_dir(&entry.path(), &target);
} else {
std::fs::copy(entry.path(), target).unwrap();
}
}
}
fn utf16_len(s: &str) -> u32 {
s.encode_utf16().count() as u32
}
/// The 0-based line index and text of the first line containing `needle`.
fn find_line<'a>(text: &'a str, needle: &str, what: &str) -> (u32, &'a str) {
text.lines()
.enumerate()
.find(|(_, l)| l.contains(needle))
.map(|(i, l)| (i as u32, l))
.unwrap_or_else(|| panic!("{what}: no line containing {needle:?}"))
}
/// The byte offset of `word` in `line`, not as part of a longer identifier (`parse` in `parser::parse`).
fn find_word(line: &str, word: &str) -> Option<usize> {
let ident = |c: char| c.is_alphanumeric() || c == '_';
line.match_indices(word).map(|(i, _)| i).find(|&i| {
let before = line[..i].chars().next_back();
let after = line[i + word.len()..].chars().next();
!(word.starts_with(ident) && before.is_some_and(ident)) && !(word.ends_with(ident) && after.is_some_and(ident))
})
}
impl Smoke {
async fn start(language: &'static str, checks: &Checks) -> Self {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap().join(checks.fixture);
copy_dir(&fixture_dir(checks.fixture), &root);
for (path, text) in checks.files {
let path = root.join(path);
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
std::fs::write(path, text).unwrap();
}
let events: Events = Arc::default();
let app = tmp.path().join("app");
let host = Arc::new(Host::new(HostConfig {
config_dir: app.join("config"),
data_dir: app.join("data"),
local_data_dir: app.join("data"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new({
let events = events.clone();
move |_| {
let events = events.clone();
Arc::new(move |e: IdeEvent| events.lock().unwrap().push(e))
}
}),
owner: |w| w,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
}));
host.trust.decide(&root, true, false).unwrap();
host.open_project(WINDOW, &root).unwrap();
Smoke { language, host, root, events, warm_up: checks.warm_up, _tmp: tmp }
}
fn fail(&self, check: &str, message: impl std::fmt::Display) -> ! {
let servers = ops::lsp::lsp_servers(&self.host, WINDOW).unwrap_or_default();
let servers: Vec<String> = servers.iter().map(|s| format!("{} ({:?})", s.name, s.status)).collect();
panic!("{} — {check}: {message}\nservers: {servers:?}", self.language)
}
/// Retries `f` until it succeeds (servers answer emptily while importing and indexing); on
/// timeout, fails with its last answer.
async fn retry<T, D: std::fmt::Debug, F: Future<Output = Result<T, D>>>(
&self,
check: &str,
timeout: Duration,
mut f: impl FnMut() -> F,
) -> T {
let deadline = Instant::now() + timeout;
loop {
let last = match f().await {
Ok(v) => return v,
Err(last) => last,
};
if Instant::now() > deadline {
self.fail(check, format_args!("timed out after {timeout:?}; last answer: {last:?}"));
}
tokio::time::sleep(Duration::from_millis(500)).await;
}
}
async fn doc(&self, file: &str) -> (DocId, String) {
let info = ops::doc::open(&self.host, WINDOW, self.root.join(file)).await.unwrap();
let info = ops::doc::snapshot(&self.host, WINDOW, info.id).await.unwrap();
(info.id, info.text)
}
async fn locate(&self, at: At) -> (DocId, Pos) {
let (doc, text) = self.doc(at.file).await;
let (line, content) = find_line(&text, at.line, at.file);
let col = find_word(content, at.word).unwrap_or_else(|| panic!("{at:?}: word not on the line"));
(doc, Pos::new(line, utf16_len(&content[..col])))
}
async fn word_range(&self, at: At) -> (DocId, Range) {
let (doc, start) = self.locate(at).await;
(doc, Range::new(start, Pos::new(start.line, start.col + utf16_len(at.word))))
}
/// Types a new line; returns the position at its end and the text before it.
async fn type_line(&self, t: Typed) -> (DocId, Pos, String) {
let (doc, text) = self.doc(t.file).await;
let (line, content) = find_line(&text, t.after, t.file);
let line_start: usize = text.lines().take(line as usize).map(|l| l.len() + 1).sum();
let end = utf16_len(&text[..line_start + content.len()]);
self.replace(doc, end, end, &format!("\n{}", t.text)).await;
let col = utf16_len(t.text);
(doc, Pos::new(line + 1, col), text)
}
async fn replace(&self, doc: DocId, from: u32, to: u32, insert: &str) {
let version = ops::doc::snapshot(&self.host, WINDOW, doc).await.unwrap().version;
let changes = ChangeSet { changes: vec![TextChange { from, to, insert: insert.into() }] };
ops::doc::edit(&self.host, WINDOW, doc, version, changes).unwrap();
}
async fn restore(&self, doc: DocId, text: &str) {
let current = ops::doc::snapshot(&self.host, WINDOW, doc).await.unwrap().text;
self.replace(doc, 0, utf16_len(&current), text).await;
}
fn path(&self, file: &str) -> PathBuf {
self.root.join(file)
}
fn files<P: AsRef<Path>>(&self, paths: impl IntoIterator<Item = P>) -> BTreeSet<String> {
paths
.into_iter()
.filter_map(|p| {
let p = p.as_ref().strip_prefix(&self.root).ok()?;
Some(p.to_string_lossy().replace('\\', "/"))
})
.collect()
}
async fn definition(&self, from: At, to: At) {
let (doc, pos) = self.locate(from).await;
let (_, target) = self.locate(to).await;
let want = self.path(to.file);
self.retry("go to definition", self.warm_up, || async {
let found = ops::lsp::lsp_goto(&self.host, WINDOW, doc, pos, GotoKind::Definition).await;
match &found {
Ok(locs) if locs.iter().any(|l| *l.path == *want && l.range.start.line == target.line) => Ok(()),
_ => Err(found),
}
})
.await;
}
async fn references(&self, of: At, files: &[&str]) {
let (doc, pos) = self.locate(of).await;
let want: BTreeSet<String> = files.iter().map(|f| f.to_string()).collect();
self.retry("find usages", self.warm_up, || async {
let found = ops::lsp::lsp_references(&self.host, WINDOW, RequestId(1), doc, pos, true, true).await;
match &found {
Ok(r) if want.is_subset(&self.files(r.locations.iter().map(|l| l.path.clone()))) => Ok(()),
_ => Err(found.map(|r| self.files(r.locations.into_iter().map(|l| l.path)))),
}
})
.await;
}
async fn hover(&self, over: At, shows: &str) {
let (doc, pos) = self.locate(over).await;
self.retry("hover", self.warm_up, || async {
let found = ops::lsp::lsp_hover(&self.host, WINDOW, RequestId(2), doc, pos).await;
match &found {
Ok(Some(h)) if h.contents.value.contains(shows) => Ok(()),
_ => Err(found.map(|h| h.map(|h| h.contents.value))),
}
})
.await;
}
async fn completion(&self, line: Typed, offers: &str) {
let (doc, pos, before) = self.type_line(line).await;
let trigger = line.text.ends_with('.').then(|| ".".to_string());
self.retry("completion", self.warm_up, || async {
let found = ops::lsp::lsp_completion(&self.host, WINDOW, RequestId(3), doc, pos, trigger.clone()).await;
match &found {
Ok(list)
if list.items.iter().any(|i| i.label == offers || i.filter_text.as_deref() == Some(offers)) =>
{
Ok(())
}
_ => Err(found.map(|l| l.items.into_iter().take(30).map(|i| i.label).collect::<Vec<_>>())),
}
})
.await;
self.restore(doc, &before).await;
}
async fn diagnostic(&self, file: &str, message: &str) -> Range {
let path = self.path(file);
self.retry("diagnostics", self.warm_up, || async {
let events = self.events.lock().unwrap();
let latest = events.iter().rev().filter_map(|e| match e {
IdeEvent::Diagnostics { path: p, items, source } if **p == *path => Some((source, items)),
_ => None,
});
let mut seen = Vec::new();
for (source, items) in latest {
if let Some(d) = items.iter().find(|d| d.message.contains(message)) {
return Ok(d.range);
}
seen.push(format!("{source}: {:?}", items.iter().map(|d| &d.message).collect::<Vec<_>>()));
}
Err(seen.into_iter().take(4).collect::<Vec<_>>())
})
.await
}
/// The code actions over `range` (all kinds, as the ⌥⏎ popup asks), the one titled `title`.
async fn action(&self, check: &str, doc: DocId, range: Range, title: &str) -> ide_lsp::types::CodeAction {
let action = self
.retry(check, self.warm_up, || async {
let found = ops::lsp::lsp_code_actions(&self.host, WINDOW, doc, range, vec![]).await;
match found.as_ref().ok().and_then(|a| a.iter().find(|a| a.title.contains(title))) {
Some(action) => Ok(action.clone()),
None => Err(found.map(|a| a.into_iter().map(|a| a.title).collect::<Vec<_>>())),
}
})
.await;
if action.edit.is_some() || action.command.is_some() {
return action;
}
ops::lsp::lsp_resolve_code_action(&self.host, WINDOW, doc, action)
.await
.unwrap_or_else(|e| self.fail(check, format_args!("resolving {title:?}: {e}")))
}
async fn quick_fix(&self, fix: QuickFix) {
let (doc, _, before) = self.type_line(fix.typed).await;
assert!(!before.contains(fix.result), "{fix:?}: the file already contains the result");
let range = self.diagnostic(fix.typed.file, fix.diagnostic).await;
let action = self.action("quick fix", doc, range, fix.title).await;
ops::lsp::lsp_execute_code_action(&self.host, WINDOW, RequestId(4), doc, action, vec![])
.await
.unwrap_or_else(|e| self.fail("quick fix", format_args!("applying {:?}: {e}", fix.title)));
self.retry("quick fix", Duration::from_secs(10), || async {
let text = self.doc(fix.typed.file).await.1;
if text.contains(fix.result) { Ok(()) } else { Err(text) }
})
.await;
self.restore(doc, &before).await;
}
async fn intention(&self, on: At, title: &str) {
let (doc, range) = self.word_range(on).await;
let action = self.action("intention", doc, range, title).await;
let edits = action.edit.as_ref().is_some_and(|e| !e.files.is_empty());
if !edits && action.command.is_none() {
self.fail("intention", format_args!("{title:?} has neither an edit nor a command: {action:?}"));
}
}
async fn format(&self, line: Typed, indent: Indent, formatted: &str) {
let (doc, _, before) = self.type_line(line).await;
assert!(!before.lines().any(|l| l == formatted), "{line:?}: the file already has {formatted:?}");
let (tab_size, insert_spaces) = match indent {
Indent::Spaces(n) => (n, true),
Indent::Tabs => (4, false),
};
let options = FormatOptions { tab_size, insert_spaces, on_save: false, tool: None };
let text = self
.retry("format", self.warm_up, || async {
let edits = ops::lsp::lsp_format(&self.host, WINDOW, doc, None, options.clone()).await;
let text = self.doc(line.file).await.1;
let mut document = ide_text::Document::new(&text);
match &edits {
Ok(edits) if !edits.is_empty() && document.apply_edits(edits).is_ok() => Ok(document.text()),
_ => Err(edits),
}
})
.await;
if !text.lines().any(|l| l == formatted) {
self.fail("format", format_args!("no line {formatted:?} in:\n{text}"));
}
self.restore(doc, &before).await;
}
async fn rename(&self, symbol: At, to: &str, files: &[&str]) {
let (doc, pos) = self.locate(symbol).await;
let prepared = ops::lsp::lsp_prepare_rename(&self.host, WINDOW, doc, pos).await;
if !matches!(prepared, Ok(Some(_))) {
self.fail("rename", format_args!("prepare rename: {prepared:?}"));
}
let renamed = ops::lsp::lsp_rename(&self.host, WINDOW, RequestId(5), doc, pos, to.into())
.await
.unwrap_or_else(|e| self.fail("rename", e));
let edited = self.files(renamed.edit.files.iter().filter_map(|f| match f {
ide_core::FileEdit::Edit { path, .. } => Some(path.clone()),
_ => None,
}));
let want: BTreeSet<String> = files.iter().map(|f| f.to_string()).collect();
if !want.is_subset(&edited) {
self.fail("rename", format_args!("edits {edited:?}, expected at least {want:?}: {:?}", renamed.edit));
}
let mut before = Vec::new();
for file in files {
before.push(self.doc(file).await.1.matches(to).count());
}
ops::lsp::apply_workspace_edit(&self.host, WINDOW, renamed.edit)
.await
.unwrap_or_else(|e| self.fail("rename", format_args!("applying the edit: {e}")));
for (file, before) in files.iter().zip(before) {
if self.doc(file).await.1.matches(to).count() <= before {
self.fail("rename", format_args!("{file} has no new {to:?} after the rename"));
}
}
}
async fn run(&self, c: &Checks) {
self.definition(c.definition.0, c.definition.1).await;
self.references(c.references.0, c.references.1).await;
self.hover(c.hover.0, c.hover.1).await;
self.completion(c.completion.0, c.completion.1).await;
self.quick_fix(c.quick_fix).await;
self.intention(c.intention.0, c.intention.1).await;
self.format(c.format.0, c.indent, c.format.1).await;
self.rename(c.rename.0, c.rename.1, c.rename.2).await;
let shutdown = self.host.project(WINDOW).unwrap().lsp.clone();
let _ = tokio::time::timeout(Duration::from_secs(20), shutdown.shutdown_all()).await;
}
}
async fn smoke(language: &'static str, checks: Checks) {
let smoke = Smoke::start(language, &checks).await;
smoke.run(&checks).await;
}
const TYPESCRIPT: Checks = Checks {
fixture: "ts-app",
files: &[],
warm_up: Duration::from_secs(120),
definition: (
at("src/index.ts", "Total: ${formatMoney(total)}", "formatMoney"),
at("src/money.ts", "export function formatMoney", "formatMoney"),
),
references: (at("src/money.ts", "export function formatMoney", "formatMoney"), &["src/index.ts", "src/money.ts"]),
hover: (at("src/index.ts", "price: cents(8999)", "cents"), "cents(value: number): Cents"),
completion: (typed("src/index.ts", "const total = cart.total();", " cart."), "applyDiscount"),
quick_fix: QuickFix {
typed: typed("src/index.ts", "const total = cart.total();", " console.log(addMoney(total, total));"),
diagnostic: "Cannot find name 'addMoney'",
title: "import from \"./money.ts\"",
result: "import { addMoney, cents, formatMoney } from \"./money.ts\";",
},
intention: (at("src/index.ts", "const total = cart.total();", "cart.total()"), "Extract to constant"),
format: (
typed("src/index.ts", "const total = cart.total();", " const shown={total,count:cart.count}"),
" const shown = { total, count: cart.count }",
),
indent: Indent::Spaces(2),
rename: (
// From a usage in another file: the declaration is renamed too, not aliased in the import.
at("src/index.ts", "Total: ${formatMoney(total)}", "formatMoney"),
"formatPrice",
&["src/index.ts", "src/money.ts"],
),
};
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "language smoke: needs vtsls"]
async fn typescript() {
smoke("typescript", TYPESCRIPT).await;
}
const PYTHON: Checks = Checks {
fixture: "python-app",
// Ruff (lint, quick fixes, formatting) starts in projects configured for it.
files: &[("ruff.toml", "")],
warm_up: Duration::from_secs(120),
definition: (
at("src/tasks/cli.py", "tasks = TaskList()", "TaskList"),
at("src/tasks/models.py", "class TaskList:", "TaskList"),
),
references: (
at("src/tasks/models.py", "class TaskList:", "TaskList"),
&["src/tasks/cli.py", "src/tasks/models.py"],
),
hover: (at("src/tasks/cli.py", "tasks.add(\"Write release notes\"", "add"), "def add("),
completion: (typed("src/tasks/cli.py", "tasks = build_demo()", " tasks."), "pending"),
quick_fix: QuickFix {
typed: typed("src/tasks/cli.py", "tasks = build_demo()", " print(Task)"),
diagnostic: "\"Task\" is not defined",
title: "from .models import Task",
result: "from .models import Priority, Task, TaskList",
},
// basedpyright has no refactorings; Ruff's fixes are the Python intentions.
intention: (at("src/tasks/models.py", "import os", "os"), "Remove unused import"),
format: (
typed("src/tasks/cli.py", "tasks = build_demo()", " shown2 = {'a':1,'b': 2}"),
" shown2 = {\"a\": 1, \"b\": 2}",
),
indent: Indent::Spaces(4),
rename: (
at("src/tasks/models.py", "class TaskList:", "TaskList"),
"TaskBoard",
&["src/tasks/cli.py", "src/tasks/models.py"],
),
};
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "language smoke: needs basedpyright and ruff"]
async fn python() {
smoke("python", PYTHON).await;
}
const RUST: Checks = Checks {
fixture: "rust-app",
files: &[],
warm_up: Duration::from_secs(180),
definition: (
at("src/main.rs", "match parser::parse(SAMPLE)", "parse"),
at("src/parser.rs", "pub fn parse(text: &str)", "parse"),
),
references: (at("src/parser.rs", "pub fn parse(text: &str)", "parse"), &["src/main.rs", "src/parser.rs"]),
hover: (at("src/main.rs", "inventory.take(&Sku", "take"), "pub fn take("),
completion: (typed("src/main.rs", "let mut inventory = Inventory::new();", " inventory."), "low_stock"),
quick_fix: QuickFix {
// rust-analyzer's own analysis, pulled as the file changes (no save, no `cargo check`).
typed: typed(
"src/main.rs",
"let mut inventory = Inventory::new();",
" let _blank = parser::ParseError { line: 1 };",
),
diagnostic: "missing structure fields",
title: "Fill struct fields",
result: "ParseError { line: 1, message",
},
intention: (at("src/main.rs", "match parser::parse(SAMPLE)", "parser::parse(SAMPLE)"), "Extract into variable"),
format: (
typed("src/main.rs", "let mut inventory = Inventory::new();", " let _spaced=( 1,2 );"),
" let _spaced = (1, 2);",
),
indent: Indent::Spaces(4),
rename: (at("src/parser.rs", "pub fn parse(text: &str)", "parse"), "parse_all", &["src/main.rs", "src/parser.rs"]),
};
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "language smoke: needs rust-analyzer"]
async fn rust() {
smoke("rust", RUST).await;
}
const GO: Checks = Checks {
fixture: "go-app",
files: &[],
warm_up: Duration::from_secs(180),
definition: (
at("cmd/server/main.go", "s := store.New()", "New"),
at("internal/store/store.go", "func New() *Store", "New"),
),
references: (
at("internal/store/store.go", "func New() *Store", "New"),
&["cmd/server/main.go", "internal/store/store.go"],
),
hover: (
at("cmd/server/main.go", "writeJSON(w, s.Search(", "Search"),
"func (s *store.Store) Search(q string) []store.Note",
),
completion: (typed("cmd/server/main.go", "s := store.New()", "\ts."), "Create"),
quick_fix: QuickFix {
typed: typed("cmd/server/main.go", "s := store.New()", "\tfmt.Println(s)"),
diagnostic: "undefined: fmt",
title: "\"fmt\"",
result: "\t\"fmt\"",
},
intention: (at("cmd/server/main.go", "s := store.New()", "store.New()"), "Extract"),
format: (typed("cmd/server/main.go", "s := store.New()", "\t_ = []int{1,2}"), "\t_ = []int{1, 2}"),
indent: Indent::Tabs,
rename: (
at("internal/store/store.go", "func New() *Store", "New"),
"Open",
&["cmd/server/main.go", "internal/store/store.go"],
),
};
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "language smoke: needs gopls"]
async fn go() {
smoke("go", GO).await;
}
const JAVA: Checks = Checks {
fixture: "java-app",
files: &[],
warm_up: Duration::from_secs(300),
definition: (
at("src/main/java/com/example/orders/App.java", "book.place(\"alice\", \"Mechanical", "place"),
at("src/main/java/com/example/orders/OrderBook.java", "public Order place(", "place"),
),
references: (
at("src/main/java/com/example/orders/OrderBook.java", "public Order place(", "place"),
&["src/main/java/com/example/orders/App.java", "src/test/java/com/example/orders/OrderBookTest.java"],
),
hover: (
at("src/main/java/com/example/orders/App.java", "long revenue = book.revenueCents()", "revenueCents"),
"long com.example.orders.OrderBook.revenueCents()",
),
completion: (
typed("src/main/java/com/example/orders/App.java", "long revenue = book.revenueCents();", " book."),
"largest",
),
quick_fix: QuickFix {
typed: typed(
"src/main/java/com/example/orders/App.java",
"long revenue = book.revenueCents();",
" List<Order> all = book.byCustomer(\"alice\");",
),
diagnostic: "List cannot be resolved to a type",
title: "Import 'List' (java.util)",
result: "import java.util.List;",
},
intention: (
at("src/main/java/com/example/orders/App.java", "long revenue = book.revenueCents();", "book.revenueCents()"),
"Extract to local variable",
),
format: (
typed(
"src/main/java/com/example/orders/App.java",
"long revenue = book.revenueCents();",
" long doubled=revenue*2;",
),
" long doubled = revenue * 2;",
),
indent: Indent::Spaces(4),
rename: (
at("src/main/java/com/example/orders/OrderBook.java", "public Order place(", "place"),
"submit",
&["src/main/java/com/example/orders/App.java", "src/main/java/com/example/orders/OrderBook.java"],
),
};
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "language smoke: needs jdtls and JDK 21"]
async fn java() {
smoke("java", JAVA).await;
}
const KOTLIN: Checks = Checks {
fixture: "kotlin-gradle",
files: &[],
warm_up: Duration::from_secs(420),
definition: (
at("src/main/kotlin/com/example/library/App.kt", "val library = Library()", "Library"),
at("src/main/kotlin/com/example/library/Library.kt", "class Library(", "Library"),
),
references: (
at("src/main/kotlin/com/example/library/Library.kt", "fun search(query: String)", "search"),
&["src/main/kotlin/com/example/library/App.kt", "src/main/kotlin/com/example/library/Library.kt"],
),
hover: (
at("src/main/kotlin/com/example/library/App.kt", "library.search(\"java\")", "search"),
"fun search(query: String): List<Book>",
),
completion: (
typed("src/main/kotlin/com/example/library/App.kt", "val library = Library()", " library."),
"lend",
),
quick_fix: QuickFix {
typed: typed(
"src/main/kotlin/com/example/library/App.kt",
"val library = Library()",
" val stamp = LocalDateTime.now()",
),
diagnostic: "Unresolved reference",
title: "Import",
result: "import java.time.LocalDateTime",
},
intention: (
at("src/main/kotlin/com/example/library/Library.kt", "fun giveBack(isbn: String): Boolean", "giveBack"),
"block body",
),
format: (
typed("src/main/kotlin/com/example/library/App.kt", "val library = Library()", " val pair=listOf(1,2)"),
" val pair = listOf(1, 2)",
),
indent: Indent::Spaces(4),
rename: (
at("src/main/kotlin/com/example/library/Library.kt", "fun search(query: String)", "search"),
"find",
&["src/main/kotlin/com/example/library/App.kt", "src/main/kotlin/com/example/library/Library.kt"],
),
};
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "language smoke: needs kotlin-lsp, a JDK and Gradle"]
async fn kotlin() {
smoke("kotlin", KOTLIN).await;
}
+120 -37
View File
@@ -123,6 +123,19 @@ fn misbehave_pkg() -> PathBuf {
repo().join("crates/ide-plugin/tests/fixtures/misbehave")
}
/// Whether `f` holds within 5 s. A call a slow runner keeps waiting behind
/// another (or behind a new instance starting) holds once it caught up.
fn eventually(mut f: impl FnMut() -> bool) -> bool {
let until = Instant::now() + Duration::from_secs(5);
while !f() {
if Instant::now() > until {
return false;
}
std::thread::sleep(Duration::from_millis(20));
}
true
}
/// A theme package with a light and a dark theme, plus licence files, in `dir`.
fn write_theme_pair(dir: &Path) {
std::fs::create_dir_all(dir.join("t")).unwrap();
@@ -397,26 +410,22 @@ fn crashes_restart_with_backoff_until_the_circuit_opens() {
let info = f.host.info_of("misbehave").unwrap();
assert_eq!((info.status, info.stats.activations, info.error), (PluginStatus::Active, 2, None));
// Three in a row: stopped for the session, with growing waits between.
// Three in a row: stopped for the session, backing off between them (the wait
// doubles: `Limits::backoff`, tested where it is defined; wall-clock waits on CI
// are too noisy to compare).
f.take_events();
// How long each crash kept the next call waiting, without the restart itself, whose
// instantiation time varies a lot on CI.
let mut waits = Vec::new();
for _ in 0..3 {
let started = Instant::now();
for crash in 0..3 {
let err = loop {
let attempt = Instant::now();
match f.run("misbehave", "panic", None) {
Err(e) if e.contains("restarts in") => std::thread::sleep(Duration::from_millis(20)),
other => {
waits.push(attempt - started);
break other.unwrap_err();
}
other => break other.unwrap_err(),
}
};
assert!(err.contains("crashed"), "{err}");
if crash < 2 {
assert!(f.run("misbehave", "echo", None).unwrap_err().contains("restarts in"), "backing off");
}
}
assert!(waits[1] >= backoff / 2 && waits[2] > waits[1] + backoff / 2, "the wait doubles: {waits:?}");
let info = f.host.info_of("misbehave").unwrap();
assert_eq!(info.status, PluginStatus::Failed);
assert!(info.error.as_deref().is_some_and(|e| e.contains("3 times in a row")), "{:?}", info.error);
@@ -484,12 +493,15 @@ fn running_calls_can_be_cancelled() {
let f = f.clone();
std::thread::spawn(move || f.run("misbehave", "spin", None))
};
std::thread::sleep(Duration::from_millis(100));
let started = Instant::now();
f.host.cancel("misbehave");
// A cancel before the call runs (the instance may still be starting) does nothing: repeat it.
let until = Instant::now() + Duration::from_secs(5);
while !spinning.is_finished() {
assert!(Instant::now() < until, "the running call wasn't interrupted");
f.host.cancel("misbehave");
std::thread::sleep(Duration::from_millis(20));
}
let err = spinning.join().unwrap().unwrap_err();
assert!(err.contains("was cancelled"), "{err}");
assert!(started.elapsed() < Duration::from_secs(1));
assert_eq!(f.run("misbehave", "echo", Some("x")).unwrap().as_deref(), Some("x"), "no backoff");
let stats = f.host.info_of("misbehave").unwrap().stats;
assert_eq!((stats.crashes, stats.activations), (0, 2));
@@ -531,16 +543,22 @@ fn edit_path_calls_have_soft_and_hard_deadlines() {
let provider = host.syntax_provider();
let mis = LanguageId::new("mis");
let tokens = provider.highlights(&mis, "fast").unwrap();
// The first call starts the instance, which may take longer than the soft deadline.
let mut tokens = None;
assert!(eventually(|| {
tokens = provider.highlights(&mis, "fast");
tokens.is_some()
}));
let tokens = tokens.unwrap();
assert_eq!((tokens[0].start, tokens[0].end, tokens[0].kind.as_str()), (0, 4, "keyword"));
// Slower than the soft deadline: dropped (however late this thread gets
// to look), but the instance lives on.
assert!(provider.highlights(&mis, "sleep:150").is_none());
std::thread::sleep(Duration::from_millis(150));
assert!(provider.highlights(&mis, "fast").is_some());
assert!(eventually(|| provider.highlights(&mis, "fast").is_some()));
let info = host.info_of("misbehave").unwrap();
assert_eq!((info.stats.slow_calls, info.stats.crashes, info.stats.activations), (1, 0, 1));
assert!(info.stats.slow_calls >= 1, "{:?}", info.stats);
assert_eq!((info.stats.crashes, info.stats.activations), (0, 1));
// Past the hard deadline: interrupted; the next call gets a new instance.
assert!(provider.highlights(&mis, "spin").is_none());
@@ -549,7 +567,8 @@ fn edit_path_calls_have_soft_and_hard_deadlines() {
let info = host.info_of("misbehave").unwrap();
assert_eq!(info.stats.crashes, 1, "{:?}", info.error);
assert!(info.error.as_deref().is_some_and(|e| e.contains("timed out after 400ms")), "{:?}", info.error);
assert!(provider.highlights(&mis, "fast").is_some());
// The new instance may take longer to start than the soft deadline on a slow runner.
assert!(eventually(|| provider.highlights(&mis, "fast").is_some()));
assert_eq!(host.info_of("misbehave").unwrap().stats.activations, 2);
}
@@ -558,9 +577,10 @@ fn edit_path_calls_have_soft_and_hard_deadlines() {
#[test]
fn providers_of_several_plugins_run_concurrently() {
let tmp = tempfile::tempdir().unwrap();
// A budget no runner misses, so only the overlap is measured.
let limits = Limits {
edit_soft: Duration::from_millis(300),
edit_hard: Duration::from_secs(2),
edit_soft: Duration::from_secs(5),
edit_hard: Duration::from_secs(10),
fuel: u64::MAX,
..Limits::default()
};
@@ -589,16 +609,17 @@ fn providers_of_several_plugins_run_concurrently() {
text: text.into(),
};
let at = ide_core::Pos::new(0, 0);
// Activation is not what is measured.
assert_eq!(host.provide_completion(&doc("fast"), at, None).len(), 2);
// Activation is not what is measured: a first call that starts an instance
// can run past the edit-path budget, and its answer is dropped.
assert!(eventually(|| host.provide_completion(&doc("fast"), at, None).len() == 2));
let started = Instant::now();
let items = host.provide_completion(&doc("sleep:200"), at, None);
let items = host.provide_completion(&doc("sleep:500"), at, None);
let elapsed = started.elapsed();
let mut from: Vec<&str> = items.iter().map(|(p, _)| p.as_str()).collect();
from.sort();
assert_eq!(from, ["slow-a", "slow-b"], "both within the budget");
assert!(elapsed < Duration::from_millis(350), "one after the other would take 400ms: {elapsed:?}");
assert!(elapsed < Duration::from_millis(950), "one after the other would take 1000ms: {elapsed:?}");
assert_eq!(host.info_of("slow-b").unwrap().stats.slow_calls, 0);
}
@@ -618,7 +639,7 @@ fn idle_instances_are_reclaimed_and_schedules_reactivate() {
assert_eq!(f.host.instances(), 0);
let info = f.host.info_of("misbehave").unwrap();
assert_eq!((info.status, info.stats.reclaims), (PluginStatus::Inactive, 1));
assert_eq!(f.command_titles("misbehave").len(), 8, "contributions stay");
assert_eq!(f.command_titles("misbehave").len(), 10, "contributions stay");
assert_eq!(f.run("misbehave", "echo", Some("y")).unwrap().as_deref(), Some("y"));
assert_eq!(f.host.info_of("misbehave").unwrap().stats.activations, 2);
@@ -1656,22 +1677,21 @@ fn official_ai_completion_cancels_superseded_requests() {
};
let at = ide_core::Pos::new(1, 4);
let host = Arc::new(f);
// A slow request, superseded 200 ms later by the next keystroke's.
// A slow request, superseded by the next keystroke's once the server has it
// (starting the instance comes first, and takes longer on a slow runner).
let first = ide_core::CancellationToken::new();
let slow = {
let (host, doc, first) = (host.clone(), doc.clone(), first.clone());
std::thread::spawn(move || {
let started = Instant::now();
let r = host.host.provide_inline_completion(&doc, at, &first);
(r, started.elapsed())
})
std::thread::spawn(move || (host.host.provide_inline_completion(&doc, at, &first), Instant::now()))
};
std::thread::sleep(Duration::from_millis(200));
assert!(eventually(|| seen.lock().len() == 1), "the first request reaches the server");
let cancelled = Instant::now();
first.cancel();
let started = Instant::now();
let second = host.host.provide_inline_completion(&doc, at, &ide_core::CancellationToken::new());
let took = started.elapsed();
let (old, old_took) = slow.join().unwrap();
let (old, old_done) = slow.join().unwrap();
let old_took = old_done.saturating_duration_since(cancelled);
assert!(old.is_none(), "the superseded request yields nothing");
assert!(old_took < Duration::from_secs(1), "cancelled, not waited out: {old_took:?}");
assert_eq!(second.map(|(_, s)| s.insert_text).as_deref(), Some("a + b"));
@@ -1849,7 +1869,12 @@ fn official_mybatis_navigates_between_interface_and_xml() {
assert_eq!((target.path, target.range.start), (xml_path.clone(), ide_core::Pos::new(1, 14)));
let xdoc = doc(&xml_path, "xml", xml);
let to_java = f.host.provide_definition(&xdoc, ide_core::Pos::new(1, 16));
// Reads the interface from disk: on a slow machine that too can run past the budget.
let mut to_java = Vec::new();
assert!(eventually(|| {
to_java = f.host.provide_definition(&xdoc, ide_core::Pos::new(1, 16));
!to_java.is_empty()
}));
assert_eq!(
to_java.iter().map(|l| (l.path.clone(), l.range.start)).collect::<Vec<_>>(),
[(java_path.clone(), ide_core::Pos::new(3, 9))]
@@ -2393,3 +2418,61 @@ fn uninstalling_deletes_the_plugins_secrets() {
f.host.uninstall("openidea.ai-completion").unwrap();
assert!(keychain.0.lock().is_empty(), "{:?}", keychain.0.lock().keys().collect::<Vec<_>>());
}
/// Plugin API 1.0 leaves room to grow: event kinds are names, and the editor
/// providers carry snippets, imports, lazily computed quick fixes and tagged
/// problems.
#[test]
fn completion_quick_fixes_problems_and_event_kinds_of_api_1() {
let f = fixture(Limits::default());
let pkg = f._tmp.path().join("misbehave-providers");
copy_dir(&misbehave_pkg(), &pkg);
let manifest = std::fs::read_to_string(pkg.join("plugin.toml")).unwrap();
std::fs::write(
pkg.join("plugin.toml"),
format!("{manifest}\n[[contributes.providers]]\nkinds = [\"completion\", \"code-actions\"]\nlanguages = [\"text\"]\n"),
)
.unwrap();
f.install(&pkg);
let doc = |text: &str| ide_plugin::ProviderDocument {
path: f.project.join("a.txt"),
language: "text".into(),
version: 1,
text: text.into(),
};
let at = ide_core::Pos::new(0, 0);
// The first call starts the instance, which may take longer than the edit-path budget.
let mut items = Vec::new();
assert!(eventually(|| {
items = f.host.provide_completion(&doc("rich"), at, None);
!items.is_empty()
}));
let item = &items[0].1;
assert_eq!(
(item.insert_text.as_str(), item.snippet, item.filter_text.as_deref()),
("f(${1:x})$0", true, Some("fun"))
);
assert_eq!(item.additional_edits[0].new_text, "use f;\n");
// Offered without edits; computed when picked.
let actions = f.host.provide_code_actions(&doc("lazy"), ide_core::Range::default());
let (plugin, action) = actions.into_iter().next().expect("offered");
assert!(action.edits.is_empty() && action.data.as_deref() == Some("fix"));
let resolved = f.host.resolve_code_action(&plugin, &doc("lazy"), action.clone()).unwrap();
assert_eq!(resolved.edits[0].1[0].new_text, "fixed ");
let foreign = ide_plugin::CodeAction { data: Some("other".into()), ..action };
assert!(f.host.resolve_code_action(&plugin, &doc("lazy"), foreign).is_err());
// A kind this IDE doesn't know is refused, not a trap.
let err = f.host.run_command("misbehave", "subscribe", Some("file-deleted"), f.inv()).unwrap_err();
assert!(err.message.contains("no event kind"), "{}", err.message);
let err = f.host.run_command("misbehave", "subscribe", Some("file-saved"), f.inv()).unwrap_err();
assert!(err.message.contains("events"), "known, but not granted: {}", err.message);
// Tags the IDE knows, the tool that found it.
let path = f.project.join("a.txt");
f.host.run_command("misbehave", "diagnose", Some(&path.to_string_lossy()), f.inv()).unwrap();
let d = f.workspace.diagnostics.lock().get(&("misbehave".into(), path)).cloned().unwrap();
assert_eq!((d[0].unnecessary, d[0].deprecated, d[0].source.as_deref()), (true, false, Some("lint")));
}
+6 -4
View File
@@ -228,7 +228,9 @@ async fn serves_projects_files_documents_search_and_runs() {
// Real project-host ignore writes travel through the remote command dispatcher.
assert!(Dispatcher::standard().commands().contains(&"vcs_add_to_ignore"));
let ignore_root = dunce::canonicalize(&project).unwrap();
for (name, destination) in [("shared.tmp", "gitignore"), ("private.tmp", "localExclude")] {
// Not `.tmp`: a machine's global excludes file (often `*.tmp`) already
// ignores those, so the backend never writes the entry anywhere.
for (name, destination) in [("shared.ignore-me", "gitignore"), ("private.ignore-me", "localExclude")] {
let path = ignore_root.join(name);
std::fs::write(&path, "temporary").unwrap();
invoke(&peer, "vcs_add_to_ignore", json!({ "paths": [path], "destination": destination })).await.unwrap();
@@ -237,9 +239,9 @@ async fn serves_projects_files_documents_search_and_runs() {
assert_eq!(entry["ignored"], true);
assert_eq!(entry["excluded"], false);
}
assert!(std::fs::read_to_string(project.join(".gitignore")).unwrap().contains("/shared.tmp"));
assert!(std::fs::read_to_string(project.join(".git/info/exclude")).unwrap().contains("/private.tmp"));
let outside = tmp.path().join("outside.tmp");
assert!(std::fs::read_to_string(project.join(".gitignore")).unwrap().contains("/shared.ignore-me"));
assert!(std::fs::read_to_string(project.join(".git/info/exclude")).unwrap().contains("/private.ignore-me"));
let outside = tmp.path().join("outside.ignore-me");
std::fs::write(&outside, "keep").unwrap();
assert!(
invoke(&peer, "vcs_add_to_ignore", json!({ "paths": [outside], "destination": "gitignore" })).await.is_err()
+11 -2
View File
@@ -58,6 +58,11 @@ async fn application_commands_work_in_restricted_projects_and_without_a_project(
"vcs_write_patch",
"remote_bootstrap_list",
"deploy_test_connection",
"plugin_market",
"plugin_market_url",
"plugin_set_market_url",
"plugin_market_download",
"plugin_preview_install",
] {
assert!(ide_host::trust::restricted_command_allowed(command), "{command}");
for window in ["main", "welcome"] {
@@ -72,8 +77,6 @@ async fn application_commands_work_in_restricted_projects_and_without_a_project(
"tools_list",
"tools_install",
"tools_uninstall",
"plugin_install_dir",
"plugin_reload",
"vcs_git_version",
"docker_compose_op",
"db_execute",
@@ -93,6 +96,12 @@ async fn application_commands_work_in_restricted_projects_and_without_a_project(
assert!(host.authorize_command(window, command).is_err(), "{window}: {command}");
}
}
// Plugin installs run plugin code: from the welcome screen, but not in an untrusted project.
for command in ["plugin_install_dir", "plugin_uninstall", "plugin_set_enabled", "plugin_reload"] {
assert!(!ide_host::trust::restricted_command_allowed(command), "{command}");
assert!(host.authorize_command("main", command).is_err(), "{command}");
assert!(host.authorize_command("welcome", command).is_ok(), "{command}");
}
// These requests validate the vendor before any network access, independently of project trust.
for window in ["main", "welcome"] {
let error = ide_host::ops::tools::jdk_download_releases(
+3
View File
@@ -14,6 +14,9 @@ fn bench(c: &mut Criterion) {
for q in ["ordcon", "src/user", "PaymentService", "zzzz", "a"] {
g.bench_function(format!("find_files/{q}"), |b| b.iter(|| black_box(index.find_files(q, 50).len())));
}
for q in ["util", "src/ui", "a"] {
g.bench_function(format!("find_dirs/{q}"), |b| b.iter(|| black_box(index.find_dirs(q, 50).len())));
}
for q in ["OCo", "invoiceRepo", "renderView", "q"] {
g.bench_function(format!("find_symbols/{q}"), |b| {
b.iter(|| black_box(index.find_symbols(q, SymbolFilter::All, 50).len()))
+57
View File
@@ -40,6 +40,8 @@ pub(crate) struct FileRecord {
pub size: u64,
pub hash: [u8; 32],
pub symbols: Vec<StoredSymbol>,
/// Matched by an ignore file: listed by name only (no hash, no symbols).
pub ignored: bool,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
@@ -72,6 +74,7 @@ pub(crate) struct FileEntry {
pub size: u64,
pub hash: [u8; 32],
pub symbols: Box<[Sym]>,
pub ignored: bool,
}
impl FileEntry {
@@ -100,6 +103,7 @@ impl FileEntry {
size: record.size,
hash: record.hash,
symbols,
ignored: record.ignored,
}
}
@@ -126,14 +130,23 @@ impl FileEntry {
range: s.range,
})
.collect(),
ignored: self.ignored,
}
}
/// Whether the entry is out of date for `found` on disk.
pub(crate) fn stale_for(&self, found: &crate::filter::Found) -> bool {
self.mtime != found.mtime || self.size != found.size || self.ignored != found.ignored
}
}
#[derive(Default)]
pub(crate) struct IndexData {
pub files: Vec<Option<FileEntry>>,
by_path: HashMap<PathBuf, u32>,
/// Every folder holding an indexed file at any depth, keyed by its `rel` with a
/// trailing `/`: its absolute path and the number of files below it.
pub dirs: HashMap<Box<str>, (PathBuf, u32)>,
free: Vec<u32>,
pub file_count: u32,
pub symbol_count: u32,
@@ -167,6 +180,11 @@ impl IndexData {
}
};
let path = self.files[slot as usize].as_ref().map(|e| e.path.clone()).unwrap_or_default();
if let Some(e) = &self.files[slot as usize] {
for (rel, abs) in ancestors(e) {
self.dirs.entry(rel.into()).or_insert_with(|| (abs.to_owned(), 0)).1 += 1;
}
}
self.by_path.insert(path, slot);
self.file_count += 1;
self.symbol_count += symbols;
@@ -176,6 +194,14 @@ impl IndexData {
let slot = self.by_path.remove(path)?;
let old = self.files[slot as usize].take()?;
self.free.push(slot);
for (rel, _) in ancestors(&old) {
if let Some(d) = self.dirs.get_mut(rel) {
d.1 -= 1;
if d.1 == 0 {
self.dirs.remove(rel);
}
}
}
self.file_count -= 1;
self.symbol_count -= old.symbols.len() as u32;
Some(old)
@@ -197,6 +223,7 @@ impl IndexData {
self.files.shrink_to_fit();
self.free.shrink_to_fit();
self.by_path.shrink_to_fit();
self.dirs.shrink_to_fit();
}
pub(crate) fn paths(&self) -> impl Iterator<Item = &PathBuf> {
@@ -204,6 +231,19 @@ impl IndexData {
}
}
/// The folders holding `e` (`rel` with a trailing `/`, absolute path), innermost first.
fn ancestors(e: &FileEntry) -> impl Iterator<Item = (&str, &Path)> {
let mut rel = e.dir();
let mut abs = e.path.parent();
std::iter::from_fn(move || {
let path = abs.filter(|_| !rel.is_empty())?;
let here = rel;
rel = rel[..rel.len() - 1].rfind('/').map_or("", |i| &rel[..=i]);
abs = path.parent();
Some((here, path))
})
}
#[cfg(test)]
mod tests {
use super::*;
@@ -223,6 +263,23 @@ mod tests {
FileEntry::new(PathBuf::from("/r").join(path), path.to_owned(), record)
}
#[test]
fn dirs_follow_inserts_and_removals() {
let mut d = IndexData::default();
d.insert(entry("a/b/c.rs", 0));
d.insert(entry("a/d.rs", 0));
d.insert(entry("a/d.rs", 0));
d.insert(entry("top.rs", 0));
let mut dirs: Vec<_> = d.dirs.iter().map(|(k, v)| (k.to_string(), v.1)).collect();
dirs.sort();
assert_eq!(dirs, [("a/".into(), 2), ("a/b/".into(), 1)]);
assert_eq!(d.dirs["a/b/"].0, PathBuf::from("/r/a/b"));
d.remove(Path::new("/r/a/b/c.rs"));
assert!(!d.dirs.contains_key("a/b/"));
d.remove_tree(Path::new("/r/a"));
assert!(d.dirs.is_empty());
}
#[test]
fn insert_replace_remove_keep_counts() {
let mut d = IndexData::default();
+172 -21
View File
@@ -1,10 +1,16 @@
//! Which paths belong in the index: `.gitignore`/`.ignore` rules (via the
//! `ignore` crate) plus the user's `files.exclude` globs, below the project root
//! and the extra content roots (Project Structure).
//!
//! Like IDEA, ignored directories are left out entirely, while ignored files in
//! the remaining directories (`.env`) are listed by name only, so Go to File finds
//! them but their symbols stay out of the index.
use globset::{GlobBuilder, GlobSet, GlobSetBuilder};
use ignore::gitignore::{Gitignore, GitignoreBuilder};
use ignore::{WalkBuilder, WalkState};
use std::collections::HashMap;
use std::collections::hash_map::Entry;
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc;
@@ -57,11 +63,25 @@ fn rel_string(rel: &Path) -> String {
out
}
/// Whether `path` below `root` matches the exclude globs.
fn glob_excluded(excludes: &GlobSet, root: &Path, path: &Path) -> bool {
match path.strip_prefix(root) {
Ok(rel) if !rel.as_os_str().is_empty() => {
let rel = rel.to_string_lossy();
let rel = if std::path::MAIN_SEPARATOR == '/' { rel } else { rel.replace('\\', "/").into() };
excludes.is_match(rel.as_ref())
}
_ => false,
}
}
#[derive(Clone, Debug)]
pub(crate) struct Found {
pub path: PathBuf,
pub mtime: u64,
pub size: u64,
/// Matched by an ignore file: listed by name only.
pub ignored: bool,
}
pub(crate) fn mtime_of(meta: &std::fs::Metadata) -> u64 {
@@ -177,8 +197,9 @@ impl PathFilter {
ALWAYS_SKIP.contains(&name)
}
/// Whether a path reported by the watcher should be ignored (outside the
/// root, inside an excluded/ignored directory, or ignored itself).
/// Whether a path reported by the watcher should be left out (outside the
/// root, excluded, or an ignored directory or inside one). Ignored files
/// elsewhere stay in, listed by name only.
pub(crate) fn is_excluded(&self, path: &Path, is_dir: bool) -> bool {
let Some((base, _, gitignore)) = self.base_of(path) else { return true };
let rel = path.strip_prefix(base).map(rel_string).unwrap_or_default();
@@ -198,16 +219,27 @@ impl PathFilter {
return true;
}
}
gitignore.matched_path_or_any_parents(path, is_dir).is_ignore()
let dir = if is_dir { Some(path) } else { path.parent().filter(|p| *p != base) };
dir.is_some_and(|d| gitignore.matched_path_or_any_parents(d, true).is_ignore())
}
/// All indexable files at or below `dir`, walked in parallel.
/// Stops early (returning what was found so far) once `cancel` is set.
pub(crate) fn walk(&self, dir: &Path, cancel: &AtomicBool) -> Vec<Found> {
self.walk_to(dir, None, cancel)
}
/// The indexable files directly in `dir`, classified by the same rules as [`Self::walk`].
pub(crate) fn list_dir(&self, dir: &Path) -> Vec<Found> {
self.walk_to(dir, Some(1), &AtomicBool::new(false))
}
fn walk_to(&self, dir: &Path, max_depth: Option<usize>, cancel: &AtomicBool) -> Vec<Found> {
let (tx, rx) = mpsc::channel::<Vec<Found>>();
let Some((root, _, _)) = self.base_of(dir) else { return Vec::new() };
let root = root.to_path_buf();
let excludes = self.excludes.clone();
let (walk_root, walk_excludes) = (root.clone(), excludes.clone());
let walker = WalkBuilder::new(dir)
.hidden(false)
.parents(true)
@@ -215,43 +247,65 @@ impl PathFilter {
.git_ignore(true)
.git_global(true)
.git_exclude(true)
.require_git(false)
.require_git(ide_core::ignored::inside_git_repo(dir))
.follow_links(false)
.max_depth(max_depth)
.filter_entry(move |e| {
let name = e.file_name().to_string_lossy();
if Self::skip_name(&name) {
return false;
}
match e.path().strip_prefix(&root) {
Ok(rel) if !rel.as_os_str().is_empty() => {
let rel = rel.to_string_lossy();
let rel = if std::path::MAIN_SEPARATOR == '/' { rel } else { rel.replace('\\', "/").into() };
!excludes.is_match(rel.as_ref())
}
_ => true,
}
!Self::skip_name(&name) && !glob_excluded(&walk_excludes, &walk_root, e.path())
})
.build_parallel();
walker.run(|| {
let (root, excludes) = (&root, &excludes);
let mut batch = Batch { tx: tx.clone(), items: Vec::with_capacity(256) };
Box::new(move |entry| {
if cancel.load(Ordering::Relaxed) {
return WalkState::Quit;
}
let Ok(entry) = entry else { return WalkState::Continue };
if entry.file_type().is_some_and(|t| t.is_file())
let Some(file_type) = entry.file_type() else { return WalkState::Continue };
if file_type.is_dir() && max_depth.is_none_or(|max| entry.depth() < max) {
// The walker drops ignored files unseen: list every file of a
// walked directory; those the walker doesn't report are ignored.
let Ok(children) = std::fs::read_dir(entry.path()) else { return WalkState::Continue };
for child in children.flatten() {
let path = child.path();
if child.file_type().is_ok_and(|t| t.is_file())
&& !Self::skip_name(&child.file_name().to_string_lossy())
&& !glob_excluded(excludes, root, &path)
&& let Ok(meta) = child.metadata()
{
batch.items.push(Found { path, mtime: mtime_of(&meta), size: meta.len(), ignored: true });
}
}
} else if file_type.is_file()
&& let Ok(meta) = entry.metadata()
{
batch.items.push(Found { path: entry.into_path(), mtime: mtime_of(&meta), size: meta.len() });
if batch.items.len() >= 256 && batch.tx.send(std::mem::take(&mut batch.items)).is_err() {
return WalkState::Quit;
}
batch.items.push(Found {
path: entry.into_path(),
mtime: mtime_of(&meta),
size: meta.len(),
ignored: false,
});
}
if batch.items.len() >= 256 && batch.tx.send(std::mem::take(&mut batch.items)).is_err() {
return WalkState::Quit;
}
WalkState::Continue
})
});
drop(tx);
rx.into_iter().flatten().collect()
// Each file comes once from its directory listing and, unless ignored, once from the walker.
let mut by_path: HashMap<PathBuf, Found> = HashMap::new();
for f in rx.into_iter().flatten() {
match by_path.entry(f.path.clone()) {
Entry::Occupied(mut e) => e.get_mut().ignored &= f.ignored,
Entry::Vacant(e) => {
e.insert(f);
}
}
}
by_path.into_values().collect()
}
}
@@ -311,6 +365,103 @@ mod tests {
assert!(!filter.is_excluded(&root.join("src/new.rs"), false));
}
#[test]
fn ignored_files_are_listed_but_ignored_directories_are_not() {
let dir = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(dir.path()).unwrap();
for f in [
".env",
"src/a.rs",
"src/debug.log",
"src/local/secrets.json",
"pkg/.gitignore",
"pkg/gen.rs",
"pkg/keep.rs",
"build/out.rs",
"build/nested/deep.rs",
"logs/app.log",
"node_modules/x/.env",
"src/.DS_Store",
"src/.git",
] {
touch(&root, f);
}
std::fs::write(
root.join(".gitignore"),
".env
*.log
build/
logs/
src/local/
",
)
.unwrap();
std::fs::write(
root.join("pkg/.gitignore"),
"gen.rs
",
)
.unwrap();
let filter = PathFilter::new(&root, &vec![].into(), &["**/node_modules".into(), "**/.DS_Store".into()]);
let mut found: Vec<(String, bool)> = filter
.walk(&root, &AtomicBool::new(false))
.iter()
.filter_map(|f| Some((filter.rel(&f.path)?, f.ignored)))
.collect();
found.sort();
let expected = [
(".env", true),
(".gitignore", false),
("pkg/.gitignore", false),
("pkg/gen.rs", true),
("pkg/keep.rs", false),
("src/a.rs", false),
("src/debug.log", true),
];
assert_eq!(found, expected.map(|(p, i)| (p.to_owned(), i)));
// One directory at a time (watcher updates): same classification, no descent.
let mut listed: Vec<(String, bool)> =
filter.list_dir(&root.join("pkg")).iter().filter_map(|f| Some((filter.rel(&f.path)?, f.ignored))).collect();
listed.sort();
let expected = [("pkg/.gitignore", false), ("pkg/gen.rs", true), ("pkg/keep.rs", false)];
assert_eq!(listed, expected.map(|(p, i)| (p.to_owned(), i)));
let top: Vec<String> = filter.list_dir(&root).iter().filter_map(|f| filter.rel(&f.path)).collect();
assert!(top.iter().all(|p| !p.contains('/')), "{top:?}");
for (rel, excluded) in [
(".env", false),
("src/new.log", false),
("src/new.rs", false),
("build/x.rs", true),
("logs/app.log", true),
("src/local/x.json", true),
] {
assert_eq!(filter.is_excluded(&root.join(rel), false), excluded, "{rel}");
}
assert!(filter.is_excluded(&root.join("build"), true));
assert!(!filter.is_excluded(&root.join("src"), true));
}
#[test]
fn a_nested_repository_is_not_hidden_by_the_outer_repositorys_gitignore() {
let dir = tempfile::tempdir().unwrap();
let base = dunce::canonicalize(dir.path()).unwrap();
for f in ["projects/nested/src/a.rs", "projects/nested/out/x.rs"] {
touch(&base, f);
}
std::fs::create_dir(base.join(".git")).unwrap();
std::fs::create_dir(base.join("projects/nested/.git")).unwrap();
std::fs::write(base.join("projects/.gitignore"), "*\n!.gitignore\n").unwrap();
std::fs::write(base.join("projects/nested/.gitignore"), "out/\n").unwrap();
let root = base.join("projects/nested");
let filter = PathFilter::new(&root, &vec![].into(), &[]);
let mut found: Vec<String> =
filter.walk(&root, &AtomicBool::new(false)).iter().filter_map(|f| filter.rel(&f.path)).collect();
found.sort();
assert_eq!(found, [".gitignore", "src/a.rs"]);
}
#[test]
fn extra_content_roots_are_walked_and_prefixed() {
let dir = tempfile::tempdir().unwrap();
+9
View File
@@ -167,6 +167,11 @@ impl ProjectIndex {
matching::find_files(&d, &matching::Recents { paths: &recent }, query, limit)
}
/// Fuzzy search for folders that hold indexed files; `rel_path` ends in `/`.
pub fn find_dirs(&self, query: &str, limit: usize) -> Vec<FileMatch> {
matching::find_dirs(&self.shared.data.read(), query, limit)
}
/// Fuzzy symbol search; CamelHump aware (`OCo` → `OrderController`).
pub fn find_symbols(&self, query: &str, filter: SymbolFilter, limit: usize) -> Vec<SymbolMatch> {
let recent = self.shared.recent.lock().clone();
@@ -366,6 +371,10 @@ pub mod bench_support {
matching::find_files(&self.data, &matching::Recents { paths: &self.recent }, query, limit)
}
pub fn find_dirs(&self, query: &str, limit: usize) -> Vec<FileMatch> {
matching::find_dirs(&self.data, query, limit)
}
pub fn find_symbols(&self, query: &str, filter: SymbolFilter, limit: usize) -> Vec<SymbolMatch> {
matching::find_symbols(&self.data, &matching::Recents { paths: &self.recent }, query, filter, limit)
}
+71 -1
View File
@@ -12,7 +12,7 @@ use nucleo_matcher::{Config, Matcher, Utf32Str, Utf32String};
use rayon::prelude::*;
use std::cell::RefCell;
use std::cmp::Ordering;
use std::path::PathBuf;
use std::path::{Path, PathBuf};
const TIER_NAME: u32 = 1 << 20;
const EXACT: u32 = 4000;
@@ -251,6 +251,76 @@ pub(crate) fn find_files(data: &IndexData, recents: &Recents, query: &str, limit
.collect()
}
/// Fuzzy folder search over the folders holding indexed files: folder-name matches beat
/// path matches; `/` in the query matches the whole path. `rel_path` ends in `/`.
pub(crate) fn find_dirs(data: &IndexData, query: &str, limit: usize) -> Vec<FileMatch> {
let q = normalize_file_query(query);
if limit == 0 || q.is_empty() {
return Vec::new();
}
let by_path = q.contains('/');
let needle = Needle::new(q.trim_end_matches('/'));
// (score, rel without the trailing `/`, char offset of the name when it matched by name)
let score = |m: &mut Matcher, buf: &mut Vec<char>, rel: &str| -> Option<(u32, Option<u32>)> {
let trimmed = &rel[..rel.len() - 1];
let start = trimmed.rfind('/').map_or(0, |i| i + 1);
let name = &trimmed[start..];
if !by_path
&& needle.mask & char_mask(name) == needle.mask
&& let Some(s) = needle.score(m, name, buf)
{
return Some((TIER_NAME + s * 4 + needle.name_boost(name), Some(trimmed[..start].chars().count() as u32)));
}
if needle.mask & char_mask(trimmed) != needle.mask {
return None;
}
needle.score(m, trimmed, buf).map(|s| (s, None))
};
let mut best: Vec<(u32, &str, &Path, Option<u32>)> = data
.dirs
.par_iter()
.filter_map(|(rel, (path, _))| {
let mut buf = Vec::new();
with_matcher(|m| score(m, &mut buf, rel)).map(|(s, at)| (s, &**rel, path.as_path(), at))
})
.collect();
let order = |a: &(u32, &str, &Path, Option<u32>), b: &(u32, &str, &Path, Option<u32>)| {
b.0.cmp(&a.0).then_with(|| a.1.len().cmp(&b.1.len())).then_with(|| a.1.cmp(b.1))
};
if best.len() > limit {
best.select_nth_unstable_by(limit, order);
best.truncate(limit);
}
// Match indices and the CamelHump bonus `find_files` gives, so equal matches rank alike.
let mut out: Vec<FileMatch> = with_matcher(|m| {
let (mut buf, mut idx) = (Vec::new(), Vec::new());
best.into_iter()
.map(|(score, rel, path, name_at)| {
let trimmed = &rel[..rel.len() - 1];
let (bonus, indices) = match name_at {
Some(at) => {
let name = &trimmed[trimmed.rfind('/').map_or(0, |i| i + 1)..];
needle.indices(m, name, &mut buf, &mut idx);
(hump_bonus(name, &idx), idx.iter().map(|i| i + at).collect())
}
None => {
needle.indices(m, trimmed, &mut buf, &mut idx);
(hump_bonus(trimmed, &idx), idx.clone())
}
};
FileMatch { path: path.to_path_buf().into(), rel_path: rel.to_owned(), score: score + bonus, indices }
})
.collect()
});
out.sort_by(|a, b| {
b.score
.cmp(&a.score)
.then_with(|| a.rel_path.len().cmp(&b.rel_path.len()))
.then_with(|| a.rel_path.cmp(&b.rel_path))
});
out
}
fn file(data: &IndexData, slot: u32) -> &FileEntry {
data.files[slot as usize].as_ref().expect("candidate slots are live")
}
+1 -1
View File
@@ -8,7 +8,7 @@ use std::path::Path;
const FILES: TableDefinition<&str, &[u8]> = TableDefinition::new("files");
const META: TableDefinition<&str, &[u8]> = TableDefinition::new("meta");
/// Bump when [`FileRecord`] or its encoding changes.
const SCHEMA: &str = "openidea-index-2";
const SCHEMA: &str = "openidea-index-3";
type Result<T> = std::result::Result<T, String>;
+67
View File
@@ -138,6 +138,23 @@ fn file_ranking_is_idea_like() {
assert_eq!(shown.to_lowercase(), "uibtn");
}
#[test]
fn find_dirs_ranks_folder_names() {
let f = sample();
let idx = f.open();
let dirs = |q: &str| idx.find_dirs(q, 20).into_iter().map(|m| m.rel_path).collect::<Vec<_>>();
assert_eq!(dirs("order"), ["src/order/"]);
assert_eq!(dirs("src")[0], "src/");
assert!(dirs("src").contains(&"src/ui/".to_owned()), "{:?}", dirs("src"));
assert_eq!(dirs("src/ui/"), ["src/ui/"]);
assert!(dirs("").is_empty());
assert!(dirs("main").is_empty());
let m = &idx.find_dirs("ui", 1)[0];
assert_eq!(m.path.0, f.root.join("src/ui"));
assert_eq!(m.indices, vec![4, 5]);
}
#[test]
fn shorter_paths_win_ties_and_recent_files_boost() {
let fx = Fixture::new(&[("a/b/c/util.rs", ""), ("x/util.rs", ""), ("y/util.rs", "")]);
@@ -276,6 +293,56 @@ fn binary_and_huge_files_are_listed_without_symbols() {
assert_eq!(idx.status().symbols, 0);
}
#[test]
fn go_to_file_finds_ignored_files_but_not_their_symbols() {
let fx = Fixture::new(&[
(".gitignore", ".env\n*.local.ts\ndist/\n"),
(".env", "SECRET=1\n"),
("src/app.ts", "export function app() {}\n"),
("src/config.local.ts", "export function localOnly() {}\n"),
("dist/bundle.js", "function bundled() {}\n"),
]);
let idx = fx.open();
assert_eq!(files(&idx, ".env"), vec![".env"], "the reported case: a git-ignored root file");
assert_eq!(files(&idx, "config.local"), vec!["src/config.local.ts"]);
assert!(files(&idx, "bundle").is_empty(), "ignored directories stay out");
assert!(symbols(&idx, "localOnly", SymbolFilter::All).is_empty(), "ignored files are listed by name only");
assert_eq!(symbols(&idx, "app", SymbolFilter::All), vec!["app"]);
// Watcher updates keep the distinction.
let fresh = fx.write("src/fresh.local.ts", "export function freshLocal() {}\n");
let bundled = fx.write("dist/more.js", "function more() {}\n");
idx.update(&[fresh, bundled], &[]);
assert!(idx.wait_idle(WAIT));
assert_eq!(files(&idx, "fresh.local"), vec!["src/fresh.local.ts"]);
assert!(symbols(&idx, "freshLocal", SymbolFilter::All).is_empty());
assert!(files(&idx, "more.js").is_empty());
// Nested ignore files apply to watcher updates as they do to the scan.
fx.write("pkg/.gitignore", "gen.ts\n");
let generated = fx.write("pkg/gen.ts", "export function generatedThing() {}\n");
let kept = fx.write("pkg/kept.ts", "export function keptThing() {}\n");
idx.update(&[generated, kept], &[]);
assert!(idx.wait_idle(WAIT));
assert_eq!(files(&idx, "pkg/gen"), vec!["pkg/gen.ts"]);
assert!(symbols(&idx, "generatedThing", SymbolFilter::All).is_empty());
assert_eq!(symbols(&idx, "keptThing", SymbolFilter::All), vec!["keptThing"]);
drop(idx);
// No longer ignored: the next scan parses it although its mtime is unchanged.
std::fs::write(fx.root.join(".gitignore"), ".env\ndist/\n").unwrap();
let idx = fx.open();
assert_eq!(symbols(&idx, "localOnly", SymbolFilter::All), vec!["localOnly"]);
assert_eq!(files(&idx, ".env"), vec![".env"]);
drop(idx);
// Ignored again: its symbols are dropped.
std::fs::write(fx.root.join(".gitignore"), "*.local.ts\n").unwrap();
let idx = fx.open();
assert!(symbols(&idx, "localOnly", SymbolFilter::All).is_empty());
assert_eq!(files(&idx, "bundle"), vec!["dist/bundle.js"]);
}
#[test]
fn second_window_on_same_project_works_in_memory() {
let fx = sample();
+17 -15
View File
@@ -2,13 +2,13 @@
//! updates queued by the watcher.
use crate::data::{FileEntry, FileRecord, IndexData, StoredSymbol};
use crate::filter::{Found, PathFilter, mtime_of};
use crate::filter::{Found, PathFilter};
use crate::store::Store;
use ide_core::{IdeEvent, SharedSink, TaskId};
use ide_language::{LanguageRegistry, Syntax};
use parking_lot::{Condvar, Mutex, RwLock};
use rayon::prelude::*;
use std::collections::{HashSet, VecDeque};
use std::collections::{HashMap, HashSet, VecDeque};
use std::io::Read;
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
@@ -233,10 +233,7 @@ impl Shared {
let seen: HashSet<&Path> = found.iter().map(|f| f.path.as_path()).collect();
let (todo, stale): (Vec<&Found>, Vec<PathBuf>) = {
let d = self.data.read();
let todo = found
.iter()
.filter(|f| d.get(&f.path).is_none_or(|e| e.mtime != f.mtime || e.size != f.size))
.collect();
let todo = found.iter().filter(|f| d.get(&f.path).is_none_or(|e| e.stale_for(f))).collect();
let stale = d.paths().filter(|p| !seen.contains(p.as_path())).cloned().collect();
(todo, stale)
};
@@ -311,6 +308,7 @@ impl Shared {
let mut rec = d.get(&f.path)?.record();
rec.mtime = f.mtime;
rec.size = f.size;
rec.ignored = f.ignored;
Some((rel, rec, f.path.clone()))
}
Extracted::New(rec) => Some((rel, rec, f.path.clone())),
@@ -342,8 +340,8 @@ impl Shared {
}
fn extract(&self, f: &Found, prev_hash: Option<[u8; 32]>) -> Extracted {
let mut rec = FileRecord { mtime: f.mtime, size: f.size, ..Default::default() };
if f.size > MAX_SYMBOL_FILE {
let mut rec = FileRecord { mtime: f.mtime, size: f.size, ignored: f.ignored, ..Default::default() };
if f.ignored || f.size > MAX_SYMBOL_FILE {
return Extracted::New(rec);
}
let mut bytes = Vec::with_capacity(f.size as usize);
@@ -420,6 +418,7 @@ impl Shared {
let mut files: Vec<Found> = Vec::new();
let mut seen = HashSet::new();
let mut by_dir: HashMap<PathBuf, HashSet<PathBuf>> = HashMap::new();
for path in changed {
if !seen.insert(path.clone()) {
continue;
@@ -430,16 +429,19 @@ impl Shared {
}
if meta.is_dir() {
files.extend(filter.walk(&path, &self.cancelled));
} else if meta.is_file() {
files.push(Found { mtime: mtime_of(&meta), size: meta.len(), path });
} else if meta.is_file()
&& let Some(dir) = path.parent()
{
by_dir.entry(dir.to_owned()).or_default().insert(path);
}
}
// Listing the folder applies every ignore file (nested, global) to tell ignored files apart.
for (dir, wanted) in by_dir {
files.extend(filter.list_dir(&dir).into_iter().filter(|f| wanted.contains(&f.path)));
}
let files: Vec<Found> = {
let d = self.data.read();
files
.into_iter()
.filter(|f| d.get(&f.path).is_none_or(|e| e.mtime != f.mtime || e.size != f.size))
.collect()
files.into_iter().filter(|f| d.get(&f.path).is_none_or(|e| e.stale_for(f))).collect()
};
if !files.is_empty() {
touched = true;
@@ -457,7 +459,7 @@ impl Shared {
.filter(|p| before.detect(p, None) != self.languages.detect(p, None))
.filter_map(|p| {
let e = d.get(p)?;
Some(Found { path: p.clone(), mtime: e.mtime, size: e.size })
Some(Found { path: p.clone(), mtime: e.mtime, size: e.size, ignored: e.ignored })
})
.collect()
};
+3
View File
@@ -16,3 +16,6 @@ parking_lot.workspace = true
thiserror.workspace = true
tracing.workspace = true
encoding_rs.workspace = true
[dev-dependencies]
tempfile.workspace = true
+23 -2
View File
@@ -216,6 +216,9 @@ pub trait Handler: Send + Sync + 'static {
fn on_request_with_data(&self, method: &str, params: Value) -> Result<Value, (i64, String, Option<Value>)> {
self.on_request(method, params).map_err(|(code, message)| (code, message, None))
}
/// Called once the response to a `method` request is queued: what it sends follows the response.
fn after_response(&self, _method: &str) {}
}
/// A [`Handler`] that ignores notifications and rejects requests.
@@ -243,6 +246,9 @@ struct Inner {
pid: Option<u32>,
/// Fired when the reader task has stopped (EOF, error or kill).
reader_done: CancellationToken,
/// Fired once the child process has exited and been reaped (after [`Peer::kill`],
/// also what it started); at once when there is no child.
exited: CancellationToken,
}
impl Inner {
@@ -432,6 +438,7 @@ impl Peer {
(child.wait().await.ok(), true)
}
};
inner.exited.cancel();
if !killed {
// Bounded: a grandchild may keep the pipes open.
let drained = async {
@@ -510,7 +517,11 @@ impl Peer {
kill: CancellationToken::new(),
pid,
reader_done: CancellationToken::new(),
exited: CancellationToken::new(),
});
if pid.is_none() {
inner.exited.cancel();
}
// Writer: a single task drains the queue so messages never interleave.
{
@@ -704,6 +715,13 @@ impl Peer {
self.inner.close_transport();
self.inner.kill.cancel();
}
/// Resolves once the child process is gone: exited on its own, or killed with what it
/// started and reaped. [`closed`](Self::closed) can come earlier, at EOF. Until then the
/// child may still hold files open, which on Windows can't be deleted.
pub async fn exited(&self) {
self.inner.exited.cancelled().await;
}
}
/// Removes the pending entry and tells the remote side to stop when a request
@@ -740,7 +758,8 @@ enum Incoming {
async fn handle_request(inner: Arc<Inner>, handler: Arc<dyn Handler>, id: Value, method: String, params: Value) {
let m = method.clone();
let result = tokio::task::spawn_blocking(move || handler.on_request_with_data(&m, params)).await;
let h = handler.clone();
let result = tokio::task::spawn_blocking(move || h.on_request_with_data(&m, params)).await;
let reply = match result {
Ok(Ok(result)) => json!({ "jsonrpc": "2.0", "id": id, "result": result }),
Ok(Err((code, message, data))) => {
@@ -755,7 +774,9 @@ async fn handle_request(inner: Arc<Inner>, handler: Arc<dyn Handler>, id: Value,
"error": { "code": codes::INTERNAL_ERROR, "message": format!("handler for {method} panicked") }
}),
};
let _ = inner.send_value(&reply);
if inner.send_value(&reply).is_ok() {
handler.after_response(&method);
}
}
fn dispatch(inner: &Arc<Inner>, notif_tx: &std::sync::mpsc::Sender<Incoming>, body: &[u8]) {
+87
View File
@@ -251,6 +251,34 @@ async fn server_to_client_request() {
assert!(matches!(err, RpcError::Server { code: codes::METHOD_NOT_FOUND, .. }));
}
#[tokio::test(flavor = "multi_thread")]
async fn after_response_messages_follow_the_response() {
#[derive(Default)]
struct Client(std::sync::OnceLock<Peer>);
impl Handler for Client {
fn on_notification(&self, _: &str, _: Value) {}
fn on_request(&self, _: &str, _: Value) -> Result<Value, (i64, String)> {
Ok(json!("answer"))
}
fn after_response(&self, method: &str) {
self.0.get().unwrap().notify("after", method).unwrap();
}
}
let client = Arc::new(Client::default());
let (client_io, server_io) = duplex(64 * 1024);
let (cr, cw) = tokio::io::split(client_io);
assert!(client.0.set(Peer::connect(cr, cw, client.clone())).is_ok());
let (sr, mut sw) = tokio::io::split(server_io);
let request = br#"{"jsonrpc":"2.0","id":1,"method":"workspace/configuration","params":{}}"#;
framing::write_message(&mut sw, request).await.unwrap();
let mut reader = tokio::io::BufReader::new(sr);
let mut next = async || -> Value {
serde_json::from_slice(&framing::read_message(&mut reader).await.unwrap().unwrap()).unwrap()
};
assert_eq!(next().await, json!({ "jsonrpc": "2.0", "id": 1, "result": "answer" }));
assert_eq!(next().await, json!({ "jsonrpc": "2.0", "method": "after", "params": "workspace/configuration" }));
}
#[tokio::test(flavor = "multi_thread")]
async fn requests_start_after_the_notifications_before_them() {
// `seen` answers with the notifications handled so far; handling one is slow.
@@ -413,6 +441,65 @@ async fn kill_terminates_process() {
tokio::time::timeout(Duration::from_secs(5), peer.closed()).await.unwrap();
}
#[cfg(unix)]
#[tokio::test(flavor = "multi_thread")]
async fn exited_resolves_once_the_killed_process_is_reaped() {
let peer =
Peer::spawn("sleep", &["30".into()], &std::env::temp_dir(), &[], Arc::new(NoopHandler), Box::new(|_| {}))
.unwrap();
let pid = peer.pid().unwrap().to_string();
let exited = tokio::time::timeout(Duration::from_millis(200), peer.exited()).await;
assert!(exited.is_err(), "running until killed");
peer.kill();
tokio::time::timeout(Duration::from_secs(5), peer.exited()).await.unwrap();
let alive = std::process::Command::new("kill").args(["-0", &pid]).status().unwrap().success();
assert!(!alive, "reaped, not even a zombie");
}
#[tokio::test(flavor = "multi_thread")]
async fn exited_resolves_at_once_without_a_process() {
let (client, _server) = pair(Arc::new(NoopHandler), Arc::new(Recorder::default()));
tokio::time::timeout(Duration::from_secs(1), client.exited()).await.unwrap();
}
#[cfg(windows)]
#[tokio::test(flavor = "multi_thread")]
async fn exited_waits_for_a_windows_descendant_to_release_its_file() {
use std::os::windows::fs::OpenOptionsExt;
let tmp = tempfile::tempdir().unwrap();
std::fs::write(
tmp.path().join("hold.ps1"),
"$f=[IO.File]::Open('locked.txt','Create','ReadWrite','None'); [IO.File]::WriteAllText('ready.txt','ready'); Start-Sleep 30",
)
.unwrap();
let peer = Peer::spawn(
"cmd",
&[
"/D".into(),
"/C".into(),
"powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File hold.ps1".into(),
],
tmp.path(),
&[],
Arc::new(NoopHandler),
Box::new(|_| {}),
)
.unwrap();
tokio::time::timeout(Duration::from_secs(10), async {
while !tmp.path().join("ready.txt").exists() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await
.unwrap();
let open = || std::fs::OpenOptions::new().read(true).share_mode(0).open(tmp.path().join("locked.txt"));
assert_eq!(open().unwrap_err().raw_os_error(), Some(32));
assert!(tokio::time::timeout(Duration::from_millis(200), peer.exited()).await.is_err());
peer.kill();
tokio::time::timeout(Duration::from_secs(5), peer.exited()).await.unwrap();
drop(open().expect("exited must wait for the descendant's file handle to close"));
}
/// A launcher that prints in the system code page and exits before speaking the
/// protocol: `jdtls.bat` on a Chinese Windows when its Python step fails, then
/// `pause` prints `请按任意键继续. . .` in GBK on stdout.
+13
View File
@@ -60,6 +60,9 @@ pub struct LanguageConfig {
/// External formatter used when no language server can format, or selected in settings.
#[serde(default)]
pub formatter: Option<FormatterConfig>,
/// Content of a new scratch file; `$END$` marks where the caret goes.
#[serde(default)]
pub scratch_template: Option<String>,
}
/// An external formatting tool and its language-specific parser.
@@ -152,6 +155,11 @@ pub struct LanguageServerConfig {
/// Register source files with tsserver when no project configuration exists.
#[serde(default)]
pub tsserver_project: bool,
/// Re-sends the open documents (unchanged) once `workspace/configuration` is answered:
/// the server (Intelephense) applies some settings, such as the PHP version, only when
/// it parses a document.
#[serde(default)]
pub resync_on_configuration: bool,
/// Set on plugin languages: start exactly this approved program.
#[serde(skip)]
pub pinned: Option<ProgramPin>,
@@ -242,6 +250,10 @@ pub struct LanguageInfo {
#[serde(default)]
#[specta(optional)]
pub first_line: Option<String>,
/// Content of a new scratch file; `$END$` marks where the caret goes.
#[serde(default)]
#[specta(optional)]
pub scratch_template: Option<String>,
}
impl From<&LanguageConfig> for LanguageInfo {
@@ -258,6 +270,7 @@ impl From<&LanguageConfig> for LanguageInfo {
language_servers: c.language_servers.iter().map(|s| s.name.clone()).collect(),
filenames: c.filenames.clone(),
first_line: c.first_line.clone(),
scratch_template: c.scratch_template.clone(),
}
}
}
+4
View File
@@ -772,6 +772,10 @@ mod tests {
assert!(addr.parse::<std::net::SocketAddr>().is_ok(), "{name}: bad connect address {addr}");
}
}
if let Some(t) = &cfg.scratch_template {
assert_eq!(t.matches("$END$").count(), 1, "{name}: scratch-template needs one $END$");
assert!(!cfg.extensions.is_empty(), "{name}: scratch-template without an extension");
}
for ext in &cfg.extensions {
assert_eq!(ext, &ext.to_ascii_lowercase(), "{name}: extensions must be lower case");
assert!(!ext.starts_with('.'), "{name}: extension {ext} must not start with a dot");
+3 -2
View File
@@ -37,7 +37,7 @@ pub fn client_capabilities() -> Value {
"fileOperations": { "dynamicRegistration": true, "willRename": true, "didRename": true },
"configuration": true,
"didChangeConfiguration": { "dynamicRegistration": false },
"didChangeWatchedFiles": { "dynamicRegistration": false, "relativePatternSupport": false },
"didChangeWatchedFiles": { "dynamicRegistration": true, "relativePatternSupport": true },
"symbol": {
"dynamicRegistration": false,
"symbolKind": { "valueSet": symbol_kinds },
@@ -48,7 +48,7 @@ pub fn client_capabilities() -> Value {
"semanticTokens": { "refreshSupport": true },
"codeLens": { "refreshSupport": true },
"inlayHint": { "refreshSupport": true },
"diagnostics": { "refreshSupport": false },
"diagnostics": { "refreshSupport": true },
"textDocumentContent": { "dynamicRegistration": true }
},
"textDocument": {
@@ -130,6 +130,7 @@ pub fn client_capabilities() -> Value {
"prepareSupportDefaultBehavior": 1,
"honorsChangeAnnotations": false
},
"diagnostic": { "dynamicRegistration": false, "relatedDocumentSupport": false },
"publishDiagnostics": {
"relatedInformation": true,
"tagSupport": { "valueSet": [1, 2] },
+1 -1
View File
@@ -59,7 +59,7 @@ fn source_files(root: &Path, exclude_root: &Path, excludes: Vec<String>, limit:
}
let Ok(ignores) = ignores.build() else { return Vec::new() };
let mut walk = ignore::WalkBuilder::new(root);
walk.require_git(false).follow_links(false).filter_entry(move |entry| {
walk.require_git(ide_core::ignored::inside_git_repo(root)).follow_links(false).filter_entry(move |entry| {
if entry.depth() == 0 {
return true;
}
+150
View File
@@ -0,0 +1,150 @@
//! `workspace/didChangeWatchedFiles` registrations: which changes on disk a
//! server asked to hear of (its globs, relative patterns and change kinds).
use crate::uri;
use globset::{GlobBuilder, GlobMatcher};
use serde_json::Value;
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
/// `FileChangeType`.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Change {
Created = 1,
Changed = 2,
Deleted = 3,
}
impl Change {
/// The `WatchKind` bit.
fn kind_bit(self) -> u64 {
match self {
Self::Created => 1,
Self::Changed => 2,
Self::Deleted => 4,
}
}
}
struct Watcher {
/// Relative pattern's base; `None` for a plain glob, matched below the server's root.
base: Option<PathBuf>,
/// A plain glob that is an absolute path pattern.
absolute: bool,
glob: GlobMatcher,
kind: u64,
}
/// A server's registrations, by registration id.
#[derive(Default)]
pub struct FileWatchers(BTreeMap<String, Vec<Watcher>>);
impl FileWatchers {
pub fn register(&mut self, id: &str, options: Option<&Value>) {
let watchers = options
.and_then(|o| o.get("watchers"))
.and_then(Value::as_array)
.into_iter()
.flatten()
.filter_map(parse)
.collect();
self.0.insert(id.to_owned(), watchers);
}
pub fn unregister(&mut self, id: &str) {
self.0.remove(id);
}
pub fn is_empty(&self) -> bool {
self.0.is_empty()
}
/// Whether a server rooted at `root` wants to hear of `change` to `path`.
pub fn wants(&self, root: &Path, path: &Path, change: Change) -> bool {
self.0.values().flatten().any(|w| {
if w.kind & change.kind_bit() == 0 {
return false;
}
match &w.base {
Some(base) => path.strip_prefix(base).is_ok_and(|rel| w.glob.is_match(rel)),
None if w.absolute => w.glob.is_match(path),
None => path.strip_prefix(root).is_ok_and(|rel| w.glob.is_match(rel)),
}
})
}
/// Bases of relative patterns, which may lie outside the project (a virtualenv's
/// `site-packages`, a path dependency): the client watches them for the server.
pub fn bases(&self) -> impl Iterator<Item = &Path> {
self.0.values().flatten().filter_map(|w| w.base.as_deref())
}
}
fn parse(watcher: &Value) -> Option<Watcher> {
let kind = watcher.get("kind").and_then(Value::as_u64).unwrap_or(7);
let (base, pattern) = match watcher.get("globPattern")? {
Value::String(p) => (None, p.as_str()),
relative => {
let base = relative.get("baseUri")?;
// A `WorkspaceFolder` or a URI.
let base = base.get("uri").unwrap_or(base).as_str()?;
let base = uri::uri_to_path(base)?;
// Changes are reported for canonical paths (`/tmp` is `/private/tmp` on macOS).
(Some(dunce::canonicalize(&base).unwrap_or(base)), relative.get("pattern")?.as_str()?)
}
};
let glob = GlobBuilder::new(pattern).literal_separator(true).build().ok()?.compile_matcher();
let absolute = base.is_none() && Path::new(pattern).is_absolute();
Some(Watcher { base, absolute, glob, kind })
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
fn watchers(list: Value) -> FileWatchers {
let mut w = FileWatchers::default();
w.register("1", Some(&json!({ "watchers": list })));
w
}
#[test]
fn matches_globs_relative_patterns_and_kinds() {
let root = Path::new("/p");
let venv = uri::path_to_uri(Path::new("/opt/venv/lib/site-packages"));
let w = watchers(json!([
{ "globPattern": "**/*.{go,mod,sum}" },
{ "globPattern": "**/pyrightconfig.json", "kind": 2 },
{ "globPattern": { "baseUri": venv, "pattern": "**" } },
{ "globPattern": { "baseUri": { "uri": uri::path_to_uri(Path::new("/shared")), "name": "s" }, "pattern": "*.toml" } },
]));
let cases = [
("/p/go.mod", Change::Changed, true),
("/p/sub/main.go", Change::Created, true),
("/elsewhere/main.go", Change::Created, false),
("/p/pyrightconfig.json", Change::Changed, true),
("/p/pyrightconfig.json", Change::Created, false),
("/opt/venv/lib/site-packages/idna/__init__.py", Change::Created, true),
("/opt/venv/lib/site-packages", Change::Deleted, true),
("/shared/Cargo.toml", Change::Changed, true),
("/shared/nested/Cargo.toml", Change::Changed, false),
("/p/readme.md", Change::Changed, false),
];
for (path, change, expected) in cases {
assert_eq!(w.wants(root, Path::new(path), change), expected, "{path} {change:?}");
}
let mut bases: Vec<_> = w.bases().collect();
bases.sort();
assert_eq!(bases, [Path::new("/opt/venv/lib/site-packages"), Path::new("/shared")]);
}
#[test]
fn unregistering_drops_the_watchers() {
let mut w = watchers(json!([{ "globPattern": "**" }]));
assert!(w.wants(Path::new("/p"), Path::new("/p/a"), Change::Created));
w.unregister("1");
assert!(w.is_empty());
assert!(!w.wants(Path::new("/p"), Path::new("/p/a"), Change::Created));
}
}
+35 -15
View File
@@ -130,15 +130,8 @@ impl ClientHandler {
{
return;
}
let conv = mgr.conv(self.encoding());
let items = params
.diagnostics
.iter()
.cloned()
.map(|d| convert::diagnostic(&conv, &path, d, &self.server.name))
.collect();
self.server.set_diagnostics(path.clone(), params.diagnostics);
mgr.sink().emit(IdeEvent::Diagnostics { path: path.into(), source: self.server.name.clone(), items });
let shown = self.server.set_diagnostics(path.clone(), params.diagnostics);
mgr.emit_diagnostics(&self.server, path, shown, self.encoding());
}
fn show_message(&self, mgr: &LspManager, params: &Value) {
@@ -206,19 +199,33 @@ impl ClientHandler {
Ok(reply)
}
fn registration(&self, params: &Value, register: bool) {
let Some(run) = self.server.running_gen(self.generation) else { return };
fn registration(&self, mgr: &LspManager, params: &Value, register: bool) {
let Some(run) = self.server.initialized_gen(self.generation) else { return };
let key = if register { "registrations" } else { "unregisterations" };
let list = params.get(key).or_else(|| params.get("unregistrations")).and_then(Value::as_array);
let mut caps = run.caps.write();
let mut watchers_changed = false;
for reg in list.into_iter().flatten() {
let Some(method) = reg.get("method").and_then(Value::as_str) else { continue };
if register {
if method == "workspace/didChangeWatchedFiles" {
let id = reg.get("id").and_then(Value::as_str).unwrap_or_default();
let mut watchers = run.watchers.write();
if register {
watchers.register(id, reg.get("registerOptions"));
} else {
watchers.unregister(id);
}
watchers_changed = true;
} else if register {
caps.register(method, reg.get("registerOptions"));
} else {
caps.unregister(method);
}
}
drop(caps);
if watchers_changed {
mgr.file_watch_roots_changed();
}
}
}
@@ -278,11 +285,11 @@ impl Handler for ClientHandler {
"workspace/configuration" => Ok(self.configuration(&params)),
"workspace/workspaceFolders" => Ok(mgr.workspace_folders(&self.server)),
"client/registerCapability" => {
self.registration(&params, true);
self.registration(&mgr, &params, true);
Ok(Value::Null)
}
"client/unregisterCapability" => {
self.registration(&params, false);
self.registration(&mgr, &params, false);
Ok(Value::Null)
}
"workspace/applyEdit" => self.apply_edit(&mgr, params),
@@ -295,10 +302,23 @@ impl Handler for ClientHandler {
mgr.sink().emit(IdeEvent::LspRefresh { server: self.server.id, kind });
Ok(Value::Null)
}
"workspace/diagnostic/refresh" | "workspace/inlineValue/refresh" => Ok(Value::Null),
"workspace/diagnostic/refresh" => {
mgr.refresh_pulled_diagnostics(&self.server);
Ok(Value::Null)
}
"workspace/inlineValue/refresh" => Ok(Value::Null),
other => Err((codes::METHOD_NOT_FOUND, format!("unsupported method {other}"))),
}
}
fn after_response(&self, method: &str) {
if method == "workspace/configuration"
&& self.server.config.resync_on_configuration
&& let Some(mgr) = self.mgr.upgrade()
{
mgr.resync_documents(&self.server, self.generation);
}
}
}
/// What `workspace/configuration` sections are looked up in: the server's
-2
View File
@@ -6,8 +6,6 @@
//! (`textDocument/diagnostic`). The others are waited on: the first
//! `publishDiagnostics` for the file after it was opened, then a short quiet
//! period so follow-up publishes (syntax first, semantics later) are caught.
//! The client never advertises pull support, so servers keep pushing for the
//! editor as before.
use crate::convert;
use crate::server::Server;
+1 -1
View File
@@ -1,5 +1,5 @@
//! Pull Members Up / Push Members Down through the app-owned JDT extension
//! (`plugins/java-usages`). The server checks the move and computes the edit; the UI
//! (`java/java-usages`). The server checks the move and computes the edit; the UI
//! shows its problems and applies the edit itself.
use super::COMMAND_TIMEOUT;
use crate::types::{HierarchyChoice, HierarchyMembers, HierarchyRefactored, HierarchyRefactoring, RefactoringProblem};
+117 -12
View File
@@ -12,11 +12,13 @@ mod clang_tidy;
mod convert;
mod external_project;
mod file_ops;
mod file_watch;
mod handler;
mod inspect;
pub mod jdtls;
mod launch;
mod library_source;
mod pull;
mod scope;
mod server;
mod tcp;
@@ -160,6 +162,9 @@ impl DocSnap {
}
}
/// Receives the folders outside the project that servers want watched.
pub type WatchRootsFn = dyn Fn(Vec<PathBuf>) + Send + Sync;
pub struct LspManager {
me: Weak<LspManager>,
project_root: PathBuf,
@@ -177,6 +182,7 @@ pub struct LspManager {
servers_hook: RwLock<Option<Arc<ServersFn>>>,
candidates_hook: RwLock<Option<Arc<CandidateFn>>>,
excludes_hook: RwLock<Option<Arc<external_project::ExcludesFn>>>,
watch_roots_hook: RwLock<Option<Arc<WatchRootsFn>>>,
shutting_down: AtomicBool,
execution_allowed: AtomicBool,
java_usages_warned: AtomicBool,
@@ -336,6 +342,7 @@ impl LspManager {
servers_hook: RwLock::new(None),
candidates_hook: RwLock::new(None),
excludes_hook: RwLock::new(None),
watch_roots_hook: RwLock::new(None),
shutting_down: AtomicBool::new(false),
execution_allowed: AtomicBool::new(true),
java_usages_warned: AtomicBool::new(false),
@@ -426,6 +433,7 @@ impl LspManager {
let wanted: Vec<(DocId, Vec<Arc<Server>>)> =
open.into_iter().map(|(id, language, path)| (id, self.servers_for(&language, &path))).collect();
let mut start = Vec::new();
let mut opened = Vec::new();
{
let mut docs = self.docs.lock();
for (id, servers) in wanted {
@@ -440,6 +448,7 @@ impl LspManager {
user_did_open(server, &run, &d.path, &d.uri, d.did_open_params());
}
attached.opened_gen = Some(run.generation);
opened.push((id, server.clone()));
}
d.servers.push(attached);
if !start.iter().any(|s: &Arc<Server>| s.id == server.id) {
@@ -450,6 +459,9 @@ impl LspManager {
d.servers.sort_by_key(|a| servers.iter().position(|s| s.id == a.server.id).unwrap_or(usize::MAX));
}
}
for (doc, server) in opened {
self.schedule_pull(doc, Some(&server), Duration::ZERO);
}
for server in &start {
self.ensure_started(server);
}
@@ -587,6 +599,9 @@ impl LspManager {
return;
}
};
let run =
Running { generation, peer: peer.clone(), caps: Arc::new(RwLock::new(caps)), watchers: Default::default() };
server.set_initializing(Some(run.clone()));
if let Err(e) = peer.notify("initialized", json!({})) {
tracing::warn!("{}: {e}", server.name);
}
@@ -598,29 +613,34 @@ impl LspManager {
{
tracing::debug!("{}: didChangeConfiguration: {e}", server.name);
}
let run = Running { generation, peer: peer.clone(), caps: Arc::new(RwLock::new(caps)) };
self.refresh_external_project(&server, &run).await;
// Publish readiness and (re)open attached documents atomically w.r.t. did_change.
let mut reopened = Vec::new();
{
let mut docs = self.docs.lock();
if !server.mark_ready(generation, run.clone()) {
drop(docs);
server.set_initializing(None);
peer.kill();
return;
}
let open_close = run.caps.read().open_close;
for d in docs.values_mut() {
for (id, d) in docs.iter_mut() {
let params = d.did_open_params();
for a in d.servers.iter_mut().filter(|a| a.server.id == server.id && a.opened_gen != Some(generation)) {
if open_close && let Err(e) = peer.notify("textDocument/didOpen", params.clone()) {
tracing::debug!("{}: didOpen: {e}", server.name);
}
a.opened_gen = Some(generation);
reopened.push(*id);
}
}
}
for doc in reopened {
self.schedule_pull(doc, Some(&server), Duration::ZERO);
}
self.check_clang_tidy(&server).await;
server.refresh_busy_status(&self.sink);
server.announce_ready(generation);
@@ -644,6 +664,7 @@ impl LspManager {
None => return, // superseded instance
};
self.clear_server_state(&server);
self.file_watch_roots_changed();
if intentional {
server.announce_down();
return;
@@ -843,6 +864,7 @@ impl LspManager {
}
docs.insert(doc, state);
}
self.schedule_pull(doc, None, Duration::ZERO);
for s in &servers {
self.ensure_started(s);
if s.running().is_some()
@@ -920,6 +942,30 @@ impl LspManager {
tracing::debug!("{}: didChange: {e}", server.name);
}
}
drop(docs);
self.schedule_pull(doc, None, pull::AFTER_EDIT);
}
/// Sends the documents open on instance `generation` of `server` again, as a `didChange`
/// with their unchanged text, so the server parses them with the settings it just read.
pub(crate) fn resync_documents(&self, server: &Server, generation: u64) {
let Some(run) = server.running_gen(generation) else { return };
if run.caps.read().sync == SyncKind::None {
return;
}
let docs = self.docs.lock();
for d in docs.values() {
if !d.servers.iter().any(|a| a.server.id == server.id && a.opened_gen == Some(generation)) {
continue;
}
let params = json!({
"textDocument": { "uri": d.uri, "version": d.version },
"contentChanges": [{ "text": d.text.as_str() }],
});
if let Err(e) = run.peer.notify("textDocument/didChange", params) {
tracing::debug!("{}: didChange: {e}", server.name);
}
}
}
pub fn did_save(&self, doc: DocId, text: &str) {
@@ -942,11 +988,15 @@ impl LspManager {
tracing::debug!("{}: didSave: {e}", a.server.name);
}
}
drop(docs);
self.schedule_pull(doc, None, Duration::ZERO);
self.pull_dependents(doc);
}
pub fn did_close(&self, doc: DocId) {
let Some(d) = self.docs.lock().remove(&doc) else { return };
for a in &d.servers {
self.forget_pulled(&a.server, &d.path, a.server.running().as_ref());
let Some(run) = a.server.running() else { continue };
if a.opened_gen == Some(run.generation)
&& run.caps.read().open_close
@@ -1973,29 +2023,78 @@ impl LspManager {
}
}
/// Files that changed on disk, outside the editor too (a checkout, another tool): each running
/// server hears of those of its languages under its root through `workspace/didChangeWatchedFiles`.
/// Without it jdtls's incremental build never sees them, so a launch runs stale classes.
pub fn watched_files_changed(&self, created: &[PathBuf], modified: &[PathBuf], removed: &[PathBuf]) {
/// Files that changed on disk, outside the editor too (a checkout, another tool), sent to
/// running servers through `workspace/didChangeWatchedFiles`. A server that registered file
/// watchers hears of what they match, including `excluded_dirs` (folders in excluded or
/// ignored trees whose entries came or went, reported as created): a package installed
/// into `.venv` or a `go.mod` rewritten by `go get` is picked up without a restart. Other
/// servers hear of their languages' files under their root; without it jdtls's incremental
/// build never sees them, so a launch runs stale classes.
pub fn watched_files_changed(
&self,
created: &[PathBuf],
modified: &[PathBuf],
removed: &[PathBuf],
excluded_dirs: &[PathBuf],
) {
use file_watch::Change;
let servers: Vec<Arc<Server>> = self.servers.lock().clone();
for s in servers {
let Some(run) = s.running() else { continue };
let languages = s.languages();
let changes: Vec<Value> = [(created, 1), (modified, 2), (removed, 3)]
let all = [(created, Change::Created), (modified, Change::Changed), (removed, Change::Deleted)]
.into_iter()
.flat_map(|(paths, kind)| paths.iter().map(move |p| (p, kind)))
.filter(|(p, _)| p.starts_with(&s.root) && languages.contains(&self.languages.detect(p, None)))
.map(|(p, kind)| json!({ "uri": uri::path_to_uri(p), "type": kind }))
.collect();
.flat_map(|(paths, change)| paths.iter().map(move |p| (p, change)));
let watchers = run.watchers.read();
let changes: Vec<(&PathBuf, Change)> = if watchers.is_empty() {
let languages = s.languages();
all.filter(|(p, _)| p.starts_with(&s.root) && languages.contains(&self.languages.detect(p, None)))
.collect()
} else {
all.chain(excluded_dirs.iter().map(|p| (p, Change::Created)))
.filter(|(p, change)| watchers.wants(&s.root, p, *change))
.collect()
};
drop(watchers);
if changes.is_empty() {
continue;
}
let changes: Vec<Value> =
changes.iter().map(|(p, c)| json!({ "uri": uri::path_to_uri(p), "type": *c as u8 })).collect();
if let Err(e) = run.peer.notify("workspace/didChangeWatchedFiles", json!({ "changes": changes })) {
tracing::debug!("{}: didChangeWatchedFiles: {e}", s.name);
}
}
}
/// Called with the folders outside the project that servers registered file watchers
/// for (a virtualenv elsewhere, a path dependency), whenever they change. Changes
/// below them go to [`Self::watched_files_changed`].
pub fn set_file_watch_roots_hook(&self, hook: Box<WatchRootsFn>) {
*self.watch_roots_hook.write() = Some(Arc::from(hook));
self.file_watch_roots_changed();
}
pub(crate) fn file_watch_roots_changed(&self) {
let Some(hook) = self.watch_roots_hook.read().clone() else { return };
let servers: Vec<Arc<Server>> = self.servers.lock().clone();
let mut roots: Vec<PathBuf> = Vec::new();
for run in servers.iter().filter_map(|s| s.instance()) {
roots.extend(
run.watchers.read().bases().filter(|b| !b.starts_with(&self.project_root)).map(Path::to_path_buf),
);
}
roots.sort();
roots.dedup();
// Sorted, so a folder's subfolders follow it.
let mut outer: Vec<PathBuf> = Vec::new();
for r in roots {
if !outer.last().is_some_and(|o| r.starts_with(o)) {
outer.push(r);
}
}
hook(outer);
}
/// Checks initialized capabilities, including dynamic registrations, before choosing a formatter.
pub async fn can_format(&self, doc: DocId) -> IdeResult<bool> {
let snap = self.snapshot(doc)?;
@@ -2451,6 +2550,12 @@ impl LspManager {
}
}
/// [`Self::reconfigure`] on the manager's runtime, for callers outside one (file watchers).
pub fn reconfigure_soon(self: &Arc<Self>) {
let me = self.clone();
self.runtime.spawn(async move { me.reconfigure().await });
}
pub async fn reconfigure(&self) {
if !self.execution_allowed.load(Ordering::Acquire) {
let servers = self.servers.lock().clone();
+192
View File
@@ -0,0 +1,192 @@
//! Pull diagnostics (`textDocument/diagnostic`). Servers that provide them may stop pushing
//! their own analysis to a client that can pull (rust-analyzer pushes only `cargo check`
//! then): the editor pulls an open file's diagnostics when it opens, after edits and saves,
//! and when the server asks for a refresh. Pulled diagnostics are kept apart from pushed ones
//! and shown together.
use crate::server::{Running, Server};
use crate::{LspManager, convert};
use ide_core::{DocId, IdeEvent};
use ide_jsonrpc::RpcError;
use lsp_types as lsp;
use serde_json::{Value, json};
use std::path::PathBuf;
use std::sync::Arc;
use std::time::Duration;
/// How long typing must pause before a changed file is pulled again.
pub(crate) const AFTER_EDIT: Duration = Duration::from_millis(300);
const TIMEOUT: Duration = Duration::from_secs(30);
/// Retries of a pull the server cancelled (`ServerCancelled`, e.g. while indexing).
const RETRIES: u32 = 20;
struct PullTarget {
server: Arc<Server>,
run: Running,
path: PathBuf,
uri: lsp::Uri,
}
fn provides_pull(run: &Running) -> bool {
let caps = run.caps.read();
caps.restriction.diagnostics() && caps.has("diagnosticProvider")
}
fn inter_file(run: &Running) -> bool {
run.caps.read().raw.pointer("/diagnosticProvider/interFileDependencies").and_then(Value::as_bool) == Some(true)
}
impl LspManager {
/// Pulls `doc`'s diagnostics from its servers that provide them (`only`: from that one)
/// after `delay`; a later pull of the same file supersedes it.
pub(crate) fn schedule_pull(&self, doc: DocId, only: Option<&Server>, delay: Duration) {
let targets: Vec<PullTarget> = {
let docs = self.docs.lock();
let Some(d) = docs.get(&doc) else { return };
d.servers
.iter()
.filter(|a| only.is_none_or(|s| s.id == a.server.id))
.filter_map(|a| {
let run = a.server.running()?;
(a.opened_gen == Some(run.generation) && provides_pull(&run)).then(|| PullTarget {
server: a.server.clone(),
run,
path: d.path.clone(),
uri: d.uri.clone(),
})
})
.collect()
};
for t in targets {
if t.server.suppresses_diagnostics(&t.path) {
continue;
}
let pull = t.server.next_pull(&t.path);
let me = self.me.clone();
self.runtime.spawn(async move {
tokio::time::sleep(delay).await;
if let Some(me) = me.upgrade() {
me.pull(t, pull).await;
}
});
}
}
/// Pulls every open file of `server` again (`workspace/diagnostic/refresh`).
pub(crate) fn refresh_pulled_diagnostics(&self, server: &Server) {
let docs: Vec<DocId> = self
.docs
.lock()
.iter()
.filter(|(_, d)| d.servers.iter().any(|a| a.server.id == server.id))
.map(|(id, _)| *id)
.collect();
for doc in docs {
self.schedule_pull(doc, Some(server), Duration::ZERO);
}
}
/// After `doc` was saved: servers whose diagnostics depend on other files pull those
/// open files too.
pub(crate) fn pull_dependents(&self, doc: DocId) {
let docs = self.docs.lock();
let Some(d) = docs.get(&doc) else { return };
let servers: Vec<Arc<Server>> = d
.servers
.iter()
.filter(|a| a.server.running().is_some_and(|run| provides_pull(&run) && inter_file(&run)))
.map(|a| a.server.clone())
.collect();
let others: Vec<(DocId, Vec<Arc<Server>>)> = docs
.iter()
.filter(|(id, _)| **id != doc)
.map(|(id, o)| {
(*id, servers.iter().filter(|s| o.servers.iter().any(|a| a.server.id == s.id)).cloned().collect())
})
.collect();
drop(docs);
for (other, servers) in others {
for server in servers {
self.schedule_pull(other, Some(&server), Duration::ZERO);
}
}
}
/// Drops what was pulled for a closed file and shows what remains.
pub(crate) fn forget_pulled(&self, server: &Server, path: &std::path::Path, run: Option<&Running>) {
if let Some(shown) = server.clear_pulled(path) {
let enc = run.map(|r| r.caps.read().encoding).unwrap_or_default();
self.emit_diagnostics(server, path.to_owned(), shown, enc);
}
}
async fn pull(&self, t: PullTarget, pull: u64) {
let mut attempt = 0;
loop {
if !t.server.is_latest_pull(&t.path, pull) {
return;
}
let mut params = json!({ "textDocument": { "uri": t.uri } });
if let Some(id) = t.server.pulled_result_id(&t.path) {
params["previousResultId"] = json!(id);
}
if let Some(identifier) = t.run.caps.read().raw.pointer("/diagnosticProvider/identifier") {
params["identifier"] = identifier.clone();
}
let request = t.run.peer.request::<Value, Value>("textDocument/diagnostic", params, None);
let report = match tokio::time::timeout(TIMEOUT, request).await {
Ok(Ok(report)) => report,
// This request has no cancellation of its own: the server cancelled it
// (`ServerCancelled` while busy, which asks to be asked again).
Ok(Err(RpcError::Cancelled)) if attempt < RETRIES => {
attempt += 1;
tokio::time::sleep(Duration::from_millis(250 * u64::from(attempt.min(8)))).await;
continue;
}
Ok(Err(e)) => {
tracing::debug!("{}: textDocument/diagnostic: {e}", t.server.name);
return;
}
Err(_) => {
tracing::debug!("{}: textDocument/diagnostic timed out", t.server.name);
return;
}
};
// A newer pull, a closed file or a restarted server makes this report stale.
if !t.server.is_latest_pull(&t.path, pull)
|| t.server.running().is_none_or(|r| r.generation != t.run.generation)
|| !self.docs.lock().values().any(|d| d.path == t.path)
{
return;
}
let result_id = report.get("resultId").and_then(Value::as_str).map(str::to_owned);
let items = match report.get("kind").and_then(Value::as_str) {
Some("unchanged") => None,
_ => match serde_json::from_value(report.get("items").cloned().unwrap_or(json!([]))) {
Ok(items) => Some(items),
Err(e) => {
tracing::debug!("{}: bad textDocument/diagnostic report: {e}", t.server.name);
return;
}
},
};
let shown = t.server.set_pulled(t.path.clone(), result_id, items);
let enc = t.run.caps.read().encoding;
self.emit_diagnostics(&t.server, t.path, shown, enc);
return;
}
}
/// Shows `items` as `server`'s diagnostics for `path`.
pub(crate) fn emit_diagnostics(
&self,
server: &Server,
path: PathBuf,
items: Vec<lsp::Diagnostic>,
enc: crate::text::Encoding,
) {
let conv = self.conv(enc);
let items = items.into_iter().map(|d| convert::diagnostic(&conv, &path, d, &server.name)).collect();
self.sink.emit(IdeEvent::Diagnostics { path: path.into(), source: server.name.clone(), items });
}
}
+109 -6
View File
@@ -27,6 +27,8 @@ pub struct Running {
pub generation: u64,
pub peer: Peer,
pub caps: Arc<RwLock<ServerCaps>>,
/// `workspace/didChangeWatchedFiles` registrations.
pub watchers: Arc<RwLock<crate::file_watch::FileWatchers>>,
}
pub enum Conn {
@@ -82,6 +84,9 @@ pub struct Server {
pub resolved_binary: RwLock<Option<PathBuf>>,
languages: Mutex<Vec<LanguageId>>,
state: Mutex<State>,
/// The instance being initialized, from `initialized` until ready: registrations
/// sent right after `initialized` must not be lost.
initializing: Mutex<Option<Running>>,
/// Fired when the instance being started is abandoned (stop / restart while starting).
start_cancel: Mutex<CancellationToken>,
phase: watch::Sender<Phase>,
@@ -92,7 +97,13 @@ pub struct Server {
started: Mutex<Option<Instant>>,
stderr: Mutex<VecDeque<String>>,
pub progress: Mutex<HashMap<String, Progress>>,
/// What the editor shows per path: the pushed and the pulled diagnostics together.
diagnostics: Mutex<HashMap<PathBuf, Vec<lsp::Diagnostic>>>,
/// `publishDiagnostics` per path.
pushed: Mutex<HashMap<PathBuf, Vec<lsp::Diagnostic>>>,
/// `textDocument/diagnostic` reports per path, and the latest pull asked for each.
pulled: Mutex<HashMap<PathBuf, Pulled>>,
pulls: Mutex<HashMap<PathBuf, u64>>,
/// When diagnostics were last published per path (empty lists included).
published: Mutex<HashMap<PathBuf, Instant>>,
/// Woken on every `publishDiagnostics` (Inspect Code waits on it).
@@ -118,6 +129,7 @@ impl Server {
resolved_binary: RwLock::new(None),
languages: Mutex::new(vec![language]),
state: Mutex::new(State { generation: 0, conn: Conn::Idle }),
initializing: Mutex::new(None),
start_cancel: Mutex::new(CancellationToken::new()),
phase: watch::channel(Phase::Down).0,
status: Mutex::new((ServerStatus::Stopped, None)),
@@ -126,6 +138,9 @@ impl Server {
stderr: Mutex::new(VecDeque::new()),
progress: Mutex::new(HashMap::new()),
diagnostics: Mutex::new(HashMap::new()),
pushed: Mutex::new(HashMap::new()),
pulled: Mutex::new(HashMap::new()),
pulls: Mutex::new(HashMap::new()),
published: Mutex::new(HashMap::new()),
published_notify: Notify::new(),
idle_notify: Notify::new(),
@@ -210,6 +225,8 @@ impl Server {
return false;
}
st.conn = Conn::Ready(run);
drop(st);
self.set_initializing(None);
true
}
@@ -234,6 +251,20 @@ impl Server {
}
}
/// The instance told `initialized`, ready or not.
pub fn instance(&self) -> Option<Running> {
self.running().or_else(|| self.initializing.lock().clone())
}
/// Generation `generation` once it was told `initialized`, ready or not.
pub fn initialized_gen(&self, generation: u64) -> Option<Running> {
self.instance().filter(|r| r.generation == generation)
}
pub fn set_initializing(&self, run: Option<Running>) {
*self.initializing.lock() = run;
}
/// The running instance if it is still generation `generation`.
pub fn running_gen(&self, generation: u64) -> Option<Running> {
self.running().filter(|r| r.generation == generation)
@@ -471,17 +502,71 @@ impl Server {
self.progress.lock().drain().map(|(_, p)| (p.task, p.title)).collect()
}
pub fn set_diagnostics(&self, path: PathBuf, items: Vec<lsp::Diagnostic>) {
/// Records the diagnostics the server pushed for `path`; returns what the file now shows.
pub fn set_diagnostics(&self, path: PathBuf, items: Vec<lsp::Diagnostic>) -> Vec<lsp::Diagnostic> {
set_or_remove(&mut self.pushed.lock(), path.clone(), items);
self.combine(path)
}
/// Records a pulled report for `path` (`None` items: unchanged since `result_id`'s
/// predecessor); returns what the file now shows.
pub fn set_pulled(
&self,
path: PathBuf,
result_id: Option<String>,
items: Option<Vec<lsp::Diagnostic>>,
) -> Vec<lsp::Diagnostic> {
{
let mut pulled = self.pulled.lock();
let entry = pulled.entry(path.clone()).or_default();
entry.result_id = result_id;
if let Some(items) = items {
entry.items = items;
}
}
self.combine(path)
}
/// Forgets the pulled report of a closed file; `None` when there was none.
pub fn clear_pulled(&self, path: &Path) -> Option<Vec<lsp::Diagnostic>> {
self.pulls.lock().remove(path);
self.pulled.lock().remove(path)?;
Some(self.combine(path.to_owned()))
}
/// The `resultId` of the last report pulled for `path`.
pub fn pulled_result_id(&self, path: &Path) -> Option<String> {
self.pulled.lock().get(path).and_then(|p| p.result_id.clone())
}
/// Starts a pull for `path`, superseding earlier ones; see [`Server::is_latest_pull`].
pub fn next_pull(&self, path: &Path) -> u64 {
static NEXT: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(1);
let id = NEXT.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
self.pulls.lock().insert(path.to_owned(), id);
id
}
pub fn is_latest_pull(&self, path: &Path, id: u64) -> bool {
self.pulls.lock().get(path) == Some(&id)
}
fn combine(&self, path: PathBuf) -> Vec<lsp::Diagnostic> {
let mut items = self.pushed.lock().get(&path).cloned().unwrap_or_default();
if let Some(pulled) = self.pulled.lock().get(&path) {
for d in &pulled.items {
if !items.contains(d) {
items.push(d.clone());
}
}
}
{
let mut d = self.diagnostics.lock();
self.published.lock().insert(path.clone(), Instant::now());
if items.is_empty() {
d.remove(&path);
} else {
d.insert(path, items);
}
set_or_remove(&mut d, path, items.clone());
}
self.published_notify.notify_waiters();
items
}
/// When diagnostics for `path` were last published by this server.
@@ -506,10 +591,28 @@ impl Server {
pub fn take_diagnostic_paths(&self) -> Vec<PathBuf> {
self.published.lock().clear();
self.pushed.lock().clear();
self.pulled.lock().clear();
self.pulls.lock().clear();
self.diagnostics.lock().drain().map(|(p, _)| p).collect()
}
}
/// A file's last `textDocument/diagnostic` report.
#[derive(Default)]
struct Pulled {
result_id: Option<String>,
items: Vec<lsp::Diagnostic>,
}
fn set_or_remove(map: &mut HashMap<PathBuf, Vec<lsp::Diagnostic>>, path: PathBuf, items: Vec<lsp::Diagnostic>) {
if items.is_empty() {
map.remove(&path);
} else {
map.insert(path, items);
}
}
/// The last `n` stderr lines, led by the exception headline: Node.js / Python /
/// JVM crashes print the error first and a stack after it, so a plain tail
/// shows frames without saying what went wrong.
+236 -1
View File
@@ -31,6 +31,8 @@ struct MockState {
errors: Mutex<HashMap<String, (i64, String)>>,
/// Computes a response from the request and the open documents (before `responses`).
responder: Mutex<Option<Responder>>,
/// Holds the handling of notifications of this method until the gate opens.
held: Mutex<Option<(String, Arc<Gate>)>>,
peer: Mutex<Option<Peer>>,
}
@@ -74,6 +76,12 @@ struct MockHandler(Arc<MockState>);
impl Handler for MockHandler {
fn on_notification(&self, method: &str, params: Value) {
let s = &self.0;
let held = s.held.lock().clone();
if let Some((m, gate)) = held
&& m == method
{
gate.hold();
}
// Held throughout: a test that sees the effect (a document, the peer gone) also sees the record.
let mut log = s.notifications.lock();
let uri = params["textDocument"]["uri"].as_str().unwrap_or_default().to_owned();
@@ -230,6 +238,7 @@ fn server_cfg(name: &str) -> LanguageServerConfig {
max_java: None,
launch_error: None,
tsserver_project: false,
resync_on_configuration: false,
pinned: None,
}
}
@@ -269,6 +278,7 @@ fn harness_configured(
ui_mode: None,
framework_ui_modes: Vec::new(),
formatter: None,
scratch_template: None,
tree_sitter: None,
language_servers: servers
.iter()
@@ -419,6 +429,7 @@ async fn files_changed_on_disk_reach_the_servers_of_their_language() {
&[created.clone(), other],
&[modified.clone(), outside],
std::slice::from_ref(&removed),
&[h.root.join(".venv/lib/site-packages")],
);
eventually(|| !s.notified("workspace/didChangeWatchedFiles").is_empty()).await;
assert_eq!(
@@ -430,12 +441,81 @@ async fn files_changed_on_disk_reach_the_servers_of_their_language() {
])
);
// Nothing of its languages: no notification.
h.mgr.watched_files_changed(&[h.root.join("b.txt")], &[], &[]);
h.mgr.watched_files_changed(&[h.root.join("b.txt")], &[], &[], &[]);
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
assert_eq!(s.notified("workspace/didChangeWatchedFiles").len(), 1);
h.mgr.shutdown_all().await;
}
/// A server that registers file watchers hears of what they match, inside excluded folders
/// too (`uv add` into `.venv`, `go get` rewriting `go.mod`); folders outside the project
/// that relative patterns name are reported for watching; unregistered, it falls back to
/// its languages' files.
#[tokio::test(flavor = "multi_thread")]
async fn registered_file_watchers_choose_the_changes_a_server_hears_of() {
let h = harness(&[("mock", incremental_caps())]);
let roots = Arc::new(Mutex::new(Vec::<Vec<PathBuf>>::new()));
{
let roots = roots.clone();
h.mgr.set_file_watch_roots_hook(Box::new(move |r| roots.lock().push(r)));
}
h.open("a.mk", "x");
h.wait_status("mock", ServerStatus::Ready).await;
let s = h.launcher.state("mock");
let outside = tempfile::tempdir().unwrap();
let outside = dunce::canonicalize(outside.path()).unwrap();
let external = outside.join("venv/lib/site-packages");
s.peer()
.request::<_, Value>(
"client/registerCapability",
json!({ "registrations": [{ "id": "w", "method": "workspace/didChangeWatchedFiles", "registerOptions": { "watchers": [
{ "globPattern": "**/*.{mk,mod}" },
{ "globPattern": "**/pyrightconfig.json", "kind": 2 },
{ "globPattern": "**/site-packages/**" },
{ "globPattern": { "baseUri": uri_of(&external), "pattern": "**" } },
{ "globPattern": { "baseUri": uri_of(&h.root.join("sub")), "pattern": "**" } },
] } }] }),
None,
)
.await
.unwrap();
assert_eq!(roots.lock().last(), Some(&vec![external.clone()]), "only folders outside the project");
let sp = h.root.join(".venv/lib/python3.14/site-packages");
let notified = || s.notified("workspace/didChangeWatchedFiles");
h.mgr.watched_files_changed(
&[h.root.join("go.mod"), h.root.join("notes.txt"), h.root.join("pyrightconfig.json")],
&[h.root.join("pyrightconfig.json"), external.join("idna/core.py")],
&[h.root.join("old.mk")],
&[sp.join("idna"), h.root.join("node_modules/x")],
);
eventually(|| !notified().is_empty()).await;
assert_eq!(
notified()[0]["changes"],
json!([
{ "uri": uri_of(&h.root.join("go.mod")), "type": 1 },
{ "uri": uri_of(&h.root.join("pyrightconfig.json")), "type": 2 },
{ "uri": uri_of(&external.join("idna/core.py")), "type": 2 },
{ "uri": uri_of(&h.root.join("old.mk")), "type": 3 },
{ "uri": uri_of(&sp.join("idna")), "type": 1 },
])
);
s.peer()
.request::<_, Value>(
"client/unregisterCapability",
json!({ "unregisterations": [{ "id": "w", "method": "workspace/didChangeWatchedFiles" }] }),
None,
)
.await
.unwrap();
assert_eq!(roots.lock().last(), Some(&vec![]));
h.mgr.watched_files_changed(&[h.root.join("go.mod"), h.root.join("b.mk")], &[], &[], &[sp.join("idna")]);
eventually(|| notified().len() == 2).await;
assert_eq!(notified()[1]["changes"], json!([{ "uri": uri_of(&h.root.join("b.mk")), "type": 1 }]));
h.mgr.shutdown_all().await;
}
#[tokio::test(flavor = "multi_thread")]
async fn initialize_and_incremental_sync() {
let h = harness(&[("mock", incremental_caps())]);
@@ -697,6 +777,113 @@ async fn diagnostics_become_events_and_feed_code_actions() {
assert_eq!(err.kind, ide_core::error::ErrorKind::Unavailable);
}
/// The diagnostics shown last for `path` by `source`, as their messages.
fn shown(h: &Harness, path: &Path, source: &str) -> Option<Vec<String>> {
h.events.lock().iter().rev().find_map(|e| match e {
IdeEvent::Diagnostics { path: p, source: s, items } if **p == *path && s == source => {
Some(items.iter().map(|d| d.message.clone()).collect())
}
_ => None,
})
}
/// Reports a "bad" diagnostic per line containing `bad`, as `textDocument/diagnostic`; answers
/// `unchanged` when the text is what `previousResultId` was computed from.
fn pull_responder() -> Responder {
Arc::new(|s: &MockState, method: &str, params: &Value| {
if method != "textDocument/diagnostic" {
return None;
}
let text = s.docs.lock().get(params["textDocument"]["uri"].as_str()?).cloned()?;
let id = format!("{}", text.len() * 31 + text.matches("bad").count());
if params["previousResultId"].as_str() == Some(id.as_str()) {
return Some(json!({ "kind": "unchanged", "resultId": id }));
}
let items: Vec<Value> = text
.lines()
.enumerate()
.filter(|(_, l)| l.contains("bad"))
.map(|(i, _)| {
json!({ "range": {"start":{"line":i,"character":0},"end":{"line":i,"character":3}},
"severity": 1, "message": "bad" })
})
.collect();
Some(json!({ "kind": "full", "resultId": id, "items": items }))
})
}
#[tokio::test(flavor = "multi_thread")]
async fn pulled_diagnostics_follow_edits_and_join_pushed_ones() {
let mut pull_caps = incremental_caps();
pull_caps["diagnosticProvider"] = json!({ "interFileDependencies": false, "workspaceDiagnostics": false });
let h = harness(&[("mock", pull_caps), ("push", incremental_caps())]);
let s = h.launcher.state("mock");
*s.responder.lock() = Some(pull_responder());
let (doc, path) = h.open("a.mk", "ok\nbad\n");
h.wait_status("mock", ServerStatus::Ready).await;
h.wait_status("push", ServerStatus::Ready).await;
assert!(s.requested("initialize")[0]["capabilities"]["textDocument"]["diagnostic"].is_object());
eventually(|| shown(&h, &path, "mock") == Some(vec!["bad".into()])).await;
assert!(h.launcher.state("push").requested("textDocument/diagnostic").is_empty(), "it pulls from providers only");
// What the server still pushes (rust-analyzer: `cargo check`) shows next to what was pulled.
let pushed = json!({ "range": {"start":{"line":0,"character":0},"end":{"line":0,"character":2}},
"severity": 2, "message": "pushed" });
s.peer()
.notify("textDocument/publishDiagnostics", json!({ "uri": uri_of(&path), "diagnostics": [pushed] }))
.unwrap();
eventually(|| shown(&h, &path, "mock") == Some(vec!["pushed".into(), "bad".into()])).await;
// Edits are pulled again once typing pauses.
h.mgr.did_change(doc, 2, &[change(rng(0, 0, 0, 0), "bad\n")], &|| unreachable!());
eventually(|| shown(&h, &path, "mock") == Some(vec!["pushed".into(), "bad".into(), "bad".into()])).await;
// Pulled diagnostics are code actions' context like pushed ones.
let _ = h.mgr.code_actions(doc, rng(2, 1, 2, 1), vec![]).await;
let ctx = &s.requested("textDocument/codeAction")[0]["context"]["diagnostics"];
assert_eq!(ctx.as_array().unwrap().iter().map(|d| d["message"].as_str().unwrap()).collect::<Vec<_>>(), ["bad"]);
// Saving unchanged text asks with the last result id; `unchanged` keeps what is shown.
let pulls = s.requested("textDocument/diagnostic").len();
h.mgr.did_save(doc, "bad\nok\nbad\n");
eventually(|| s.requested("textDocument/diagnostic").len() > pulls).await;
let last = s.requested("textDocument/diagnostic").last().cloned().unwrap();
assert!(last["previousResultId"].is_string());
tokio::time::sleep(Duration::from_millis(100)).await;
assert_eq!(shown(&h, &path, "mock"), Some(vec!["pushed".into(), "bad".into(), "bad".into()]));
// Closing drops what was pulled; the pushed diagnostics stay the server's to clear.
h.mgr.did_close(doc);
eventually(|| shown(&h, &path, "mock") == Some(vec!["pushed".into()])).await;
}
#[tokio::test(flavor = "multi_thread")]
async fn a_cancelled_pull_is_retried_and_a_refresh_pulls_again() {
let mut caps = incremental_caps();
caps["diagnosticProvider"] = json!({ "interFileDependencies": false, "workspaceDiagnostics": false });
let h = harness(&[("mock", caps)]);
let s = h.launcher.state("mock");
*s.responder.lock() = Some(pull_responder());
// Busy indexing: the server cancels and asks to be asked again.
s.errors.lock().insert("textDocument/diagnostic".into(), (-32802, "indexing".into()));
let (_doc, path) = h.open("a.mk", "bad\n");
h.wait_status("mock", ServerStatus::Ready).await;
eventually(|| s.requested("textDocument/diagnostic").len() >= 2).await;
assert_eq!(shown(&h, &path, "mock"), None);
s.errors.lock().clear();
eventually(|| shown(&h, &path, "mock") == Some(vec!["bad".into()])).await;
// `workspace/diagnostic/refresh` (e.g. after a project reload) pulls open files again.
*s.responder.lock() = Some(Arc::new(|_: &MockState, method: &str, _: &Value| {
(method == "textDocument/diagnostic").then(|| {
json!({ "kind": "full", "items": [
{ "range": {"start":{"line":0,"character":0},"end":{"line":0,"character":3}}, "message": "reloaded" } ] })
})
}));
let refreshed: Value = s.peer().request("workspace/diagnostic/refresh", Value::Null, None).await.unwrap();
assert_eq!(refreshed, Value::Null);
eventually(|| shown(&h, &path, "mock") == Some(vec!["reloaded".into()])).await;
}
#[tokio::test(flavor = "multi_thread")]
async fn apply_edit_hook_and_server_requests() {
let h = harness(&[("mock", incremental_caps())]);
@@ -1040,6 +1227,54 @@ async fn preparation_refreshes_config_and_survives_unchanged_settings() {
assert_eq!(h.launcher.state("mock").launches.load(Ordering::SeqCst), 1);
}
/// Intelephense applies the PHP version only to documents it parses after reading its
/// configuration: once that is answered, servers that opt in get their open documents again.
#[tokio::test(flavor = "multi_thread")]
async fn configuration_answer_resyncs_open_documents_of_opted_in_servers() {
let h = harness_configured("mock", "mk", &[("mock", incremental_caps()), ("extra", incremental_caps())], |cfg| {
cfg.resync_on_configuration = cfg.name == "mock"
});
let (doc, path) = h.open("a.mk", "enum A {}");
let uri = uri_of(&path);
let (s, extra) = (h.launcher.state("mock"), h.launcher.state("extra"));
h.wait_status("mock", ServerStatus::Ready).await;
h.wait_status("extra", ServerStatus::Ready).await;
eventually(|| s.docs.lock().contains_key(&uri) && extra.docs.lock().contains_key(&uri)).await;
h.mgr.did_change(doc, 2, &[change(rng(0, 9, 0, 9), "\n")], &|| unreachable!());
eventually(|| s.notified("textDocument/didChange").len() == 1).await;
let items = |section: &str| json!({ "items": [{ "section": section }] });
let options = json!([{ "flag": true, "nested": { "deep": 1 } }]);
let answer: Value = extra.peer().request("workspace/configuration", items("extra"), None).await.unwrap();
assert_eq!(answer, options);
// The mock handles what arrives in order, answers included: a re-sync sent before the
// answer would hold it back here.
let gate = Arc::new(Gate::default());
*s.held.lock() = Some(("textDocument/didChange".into(), gate.clone()));
let peer = s.peer();
let request = peer.request::<_, Value>("workspace/configuration", items("mock"), None);
let answer = tokio::time::timeout(Duration::from_secs(10), request).await.expect("re-sync sent before the answer");
assert_eq!(answer.unwrap(), options);
eventually(|| gate.entered()).await;
assert_eq!(s.notified("textDocument/didChange").len(), 1);
gate.open();
eventually(|| s.notified("textDocument/didChange").len() == 2).await;
assert_eq!(
s.notified("textDocument/didChange")[1],
json!({ "textDocument": { "uri": uri, "version": 2 }, "contentChanges": [{ "text": "enum A {}\n" }] })
);
assert_eq!(s.docs.lock()[&uri], "enum A {}\n");
// A notification sent after both answers reaches the servers after anything they triggered.
h.mgr.did_save(doc, "enum A {}\n");
eventually(|| extra.notified("textDocument/didSave").len() == 1 && s.notified("textDocument/didSave").len() == 1)
.await;
assert_eq!(s.notified("textDocument/didChange").len(), 2, "one re-sync per answer");
let methods: Vec<String> = extra.notifications.lock().iter().map(|(m, _)| m.clone()).collect();
assert_eq!(methods, ["initialized", "textDocument/didOpen", "textDocument/didChange", "textDocument/didSave"]);
assert_eq!(s.launches.load(Ordering::SeqCst), 1);
}
#[tokio::test(flavor = "multi_thread")]
async fn settings_changes_reach_servers_without_restarts() {
let h = harness(&[("mock", incremental_caps()), ("extra", incremental_caps())]);
+4
View File
@@ -56,6 +56,7 @@ fn lang(id: &str, ext: &str, server: LanguageServerConfig) -> LanguageConfig {
ui_mode: None,
framework_ui_modes: Vec::new(),
formatter: None,
scratch_template: None,
tree_sitter: None,
language_servers: vec![server],
debugger: None,
@@ -134,6 +135,7 @@ async fn missing_binary_is_reported_with_install_hint() {
max_java: None,
launch_error: None,
tsserver_project: false,
resync_on_configuration: false,
pinned: None,
};
let (mgr, events) = manager(dir.path(), LanguageRegistry::from_configs(vec![lang("nope", "nope", server)]));
@@ -403,6 +405,7 @@ async fn typescript_language_server_end_to_end() {
max_java: None,
launch_error: None,
tsserver_project: false,
resync_on_configuration: false,
pinned: None,
};
let (mgr, events) = manager(&root, LanguageRegistry::from_configs(vec![lang("typescript", "ts", server)]));
@@ -452,6 +455,7 @@ fn npm_server(tools: &Path, name: &str, bin: &str, init: Option<serde_json::Valu
max_java: None,
launch_error: None,
tsserver_project: false,
resync_on_configuration: false,
pinned: None,
}
}
+96 -35
View File
@@ -467,7 +467,6 @@ const MAX_SYNTAX_ITEMS: usize = 200_000;
const MAX_BATCH: usize = 1000;
fn event_to_wit(e: WorkbenchEvent) -> wit_events::Event {
use crate::types::ActionSource as S;
match e {
WorkbenchEvent::FileOpened(p) => wit_events::Event::FileOpened(p),
WorkbenchEvent::FileSaved(p) => wit_events::Event::FileSaved(p),
@@ -477,14 +476,7 @@ fn event_to_wit(e: WorkbenchEvent) -> wit_events::Event {
WorkbenchEvent::ActionPerformed(a) => wit_events::Event::ActionPerformed(wit_events::ActionPerformed {
action: a.action,
title: a.title,
source: match a.source {
S::Keyboard => wit_events::ActionSource::Keyboard,
S::Menu => wit_events::ActionSource::Menu,
S::Toolbar => wit_events::ActionSource::Toolbar,
S::ContextMenu => wit_events::ActionSource::ContextMenu,
S::Search => wit_events::ActionSource::Search,
S::Other => wit_events::ActionSource::Other,
},
source: a.source.name().to_owned(),
shortcut: a.shortcut,
}),
}
@@ -670,9 +662,17 @@ impl Runtime {
detail: i.detail.map(|d| clip(d, MAX_TITLE)),
kind: i.kind.map(|k| clip(k, 32)),
insert_text: clip(i.insert_text, 64 << 10),
snippet: i.snippet,
replace: i.replace.map(range_from_wit),
sort_text: i.sort_text.map(|t| clip(t, 64)),
filter_text: i.filter_text.map(|t| clip(t, MAX_TITLE)),
documentation: i.documentation.map(|d| clip(d, 16 << 10)),
additional_edits: i
.additional_edits
.into_iter()
.take(MAX_ADDITIONAL_EDITS)
.map(edit_from_wit)
.collect(),
})
.collect())
}
@@ -699,27 +699,20 @@ impl Runtime {
) -> Result<Vec<CodeAction>, CallError> {
let d = doc_to_wit(doc);
let actions = self.provider_call(timeout, |p, s| p.call_code_actions(s, &d, range_to_wit(selection)))?;
Ok(actions
.into_iter()
.take(MAX_PROVIDER_ITEMS)
.map(|a| CodeAction {
title: clip(a.title, MAX_TITLE),
kind: a.kind.map(|k| clip(k, 64)),
preferred: a.preferred,
edits: a
.edits
.into_iter()
.map(|f| {
let edits = f
.edits
.into_iter()
.map(|e| ide_core::TextEdit { range: range_from_wit(e.range), new_text: e.text })
.collect();
(std::path::PathBuf::from(f.path), edits)
})
.collect(),
})
.collect())
Ok(actions.into_iter().take(MAX_PROVIDER_ITEMS).map(code_action_from_wit).collect())
}
/// The edits of an action `provide_code_actions` returned with `data`.
pub fn resolve_code_action(
&mut self,
doc: &ProviderDocument,
action: &CodeAction,
timeout: Duration,
) -> Result<CodeAction, CallError> {
let d = doc_to_wit(doc);
let a = code_action_to_wit(action);
let resolved = self.provider_call(timeout, |p, s| p.call_resolve_code_action(s, &d, &a))?;
resolved.map(code_action_from_wit).map_err(|e| CallError::Guest(e.into()))
}
pub fn provide_definition(
@@ -968,6 +961,47 @@ const MAX_VIEW_ITEMS: usize = 20_000;
const MAX_CELL: usize = 64 << 10;
const MAX_HTML: usize = 4 << 20;
/// Most additional edits of a completion item.
const MAX_ADDITIONAL_EDITS: usize = 100;
fn edit_from_wit(e: wit_types::TextEdit) -> ide_core::TextEdit {
ide_core::TextEdit { range: range_from_wit(e.range), new_text: e.text }
}
fn code_action_from_wit(a: wit_prov::CodeAction) -> CodeAction {
CodeAction {
title: clip(a.title, MAX_TITLE),
kind: a.kind.map(|k| clip(k, 64)),
preferred: a.preferred,
edits: a
.edits
.into_iter()
.map(|f| (std::path::PathBuf::from(f.path), f.edits.into_iter().map(edit_from_wit).collect()))
.collect(),
data: a.data.map(|d| clip(d, 64 << 10)),
}
}
fn code_action_to_wit(a: &CodeAction) -> wit_prov::CodeAction {
wit_prov::CodeAction {
title: a.title.clone(),
kind: a.kind.clone(),
preferred: a.preferred,
edits: a
.edits
.iter()
.map(|(path, edits)| wit_prov::FileEdit {
path: path.to_string_lossy().into_owned(),
edits: edits
.iter()
.map(|e| wit_types::TextEdit { range: range_to_wit(e.range), text: e.new_text.clone() })
.collect(),
})
.collect(),
data: a.data.clone(),
}
}
fn opt_clip(s: Option<String>, max: usize) -> Option<String> {
s.map(|s| clip(s, max))
}
@@ -975,6 +1009,9 @@ fn opt_clip(s: Option<String>, max: usize) -> Option<String> {
fn view_from_wit(v: wit_vt::View) -> View {
let body = match v.body {
wit_vt::Body::Message(m) => ViewBody::Message(clip(m, 16 << 10)),
wit_vt::Body::Other(e) => {
ViewBody::Message(format!("This view needs a newer version of OpenIDEA ({}).", clip(e.kind, 64)))
}
wit_vt::Body::Tree(items) => ViewBody::Tree(
items
.into_iter()
@@ -1006,10 +1043,8 @@ fn view_from_wit(v: wit_vt::View) -> View {
fields
.into_iter()
.take(MAX_VIEW_ITEMS)
.map(|f| ViewField {
key: clip(f.key, 128),
label: clip(f.label, MAX_TITLE),
value: match f.value {
.filter_map(|f| {
let value = match f.value {
wit_vt::FieldValue::Text(t) => FieldValue::Text(clip(t, MAX_CELL)),
wit_vt::FieldValue::Multiline(t) => FieldValue::Multiline(clip(t, 1 << 20)),
wit_vt::FieldValue::Checkbox(b) => FieldValue::Checkbox(b),
@@ -1017,7 +1052,10 @@ fn view_from_wit(v: wit_vt::View) -> View {
options: options.into_iter().take(1000).map(|o| clip(o, MAX_TITLE)).collect(),
selected: clip(selected, MAX_TITLE),
},
},
// A field type of a newer API: left out.
wit_vt::FieldValue::Other(_) => return None,
};
Some(ViewField { key: clip(f.key, 128), label: clip(f.label, MAX_TITLE), value })
})
.collect(),
),
@@ -1067,3 +1105,26 @@ fn event_to_wit_view(e: ViewEvent) -> wit_vt::ViewEvent {
ViewEvent::Message { text } => W::Message(text),
}
}
#[cfg(test)]
mod tests {
use super::*;
fn ext(kind: &str) -> wit_types::Extension {
wit_types::Extension { kind: kind.into(), data: "{}".into() }
}
#[test]
fn bodies_and_fields_of_a_newer_api_degrade() {
let view = |body| wit_vt::View { toolbar: Vec::new(), body, status: None };
let shown = view_from_wit(view(wit_vt::Body::Other(ext("markdown"))));
assert_eq!(shown.body, ViewBody::Message("This view needs a newer version of OpenIDEA (markdown).".into()));
let field = |key: &str, value| wit_vt::Field { key: key.into(), label: key.into(), value };
let form = view_from_wit(view(wit_vt::Body::Form(vec![
field("color", wit_vt::FieldValue::Other(ext("color"))),
field("name", wit_vt::FieldValue::Text("n".into())),
])));
let ViewBody::Form(fields) = form.body else { panic!("a form") };
assert_eq!(fields.iter().map(|f| f.key.as_str()).collect::<Vec<_>>(), ["name"], "unknown fields are left out");
}
}
+31
View File
@@ -126,6 +126,20 @@ pub enum ActionSource {
Other,
}
impl ActionSource {
/// As plugins see it.
pub fn name(self) -> &'static str {
match self {
Self::Keyboard => "keyboard",
Self::Menu => "menu",
Self::Toolbar => "toolbar",
Self::ContextMenu => "context-menu",
Self::Search => "search",
Self::Other => "other",
}
}
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct ActionPerformed {
@@ -303,6 +317,16 @@ pub fn net_host(url: &str) -> Result<String, String> {
#[cfg(test)]
mod tests {
#[test]
fn the_restart_backoff_doubles_up_to_a_minute() {
let limits = Limits { restart_backoff: Duration::from_millis(150), ..Limits::default() };
let waits: Vec<u128> = (1..=4).map(|n| limits.backoff(n).as_millis()).collect();
assert_eq!(waits, [150, 300, 600, 1200]);
assert_eq!(limits.backoff(30), Duration::from_millis(150 * 64), "the factor stops growing");
let long = Limits { restart_backoff: Duration::from_secs(10), ..Limits::default() };
assert_eq!(long.backoff(5), Duration::from_secs(60));
}
use super::*;
#[test]
@@ -338,9 +362,14 @@ pub struct CompletionItem {
/// WIT `completion-kind` name (`method`, `keyword`, …).
pub kind: Option<String>,
pub insert_text: String,
/// `insert_text` has tab stops and placeholders.
pub snippet: bool,
pub replace: Option<ide_core::Range>,
pub sort_text: Option<String>,
pub filter_text: Option<String>,
pub documentation: Option<String>,
/// Applied with the item elsewhere in the document (an import).
pub additional_edits: Vec<ide_core::TextEdit>,
}
#[derive(Clone, Debug, PartialEq, Eq)]
@@ -355,6 +384,8 @@ pub struct CodeAction {
pub kind: Option<String>,
pub preferred: bool,
pub edits: Vec<(PathBuf, Vec<ide_core::TextEdit>)>,
/// The plugin's reference: the edits come from `resolve_code_action`.
pub data: Option<String>,
}
#[derive(Clone, Debug, PartialEq, Eq)]
+11 -18
View File
@@ -440,20 +440,11 @@ impl vcs::Host for StoreData {
}
}
fn kind_from_wit(k: events::EventKind) -> EventKind {
match k {
events::EventKind::FileOpened => EventKind::FileOpened,
events::EventKind::FileSaved => EventKind::FileSaved,
events::EventKind::EditorFocused => EventKind::EditorFocused,
events::EventKind::ActionPerformed => EventKind::ActionPerformed,
events::EventKind::VcsChanged => EventKind::VcsChanged,
events::EventKind::Scheduled => EventKind::Scheduled,
}
}
impl events::Host for StoreData {
fn subscribe(&mut self, kind: events::EventKind) -> Fallible<()> {
let kind = kind_from_wit(kind);
fn subscribe(&mut self, kind: String) -> Fallible<()> {
let Some(kind) = EventKind::parse(&kind) else {
return Ok(Err(wit::Error::Invalid(format!("no event kind {kind:?} in this OpenIDEA"))));
};
if let Some(cap) = kind.capability()
&& let Err(e) = self.require(cap)
{
@@ -463,8 +454,10 @@ impl events::Host for StoreData {
Ok(Ok(()))
}
fn unsubscribe(&mut self, kind: events::EventKind) -> wasmtime::Result<()> {
self.host.subscribe(kind_from_wit(kind), false);
fn unsubscribe(&mut self, kind: String) -> wasmtime::Result<()> {
if let Some(kind) = EventKind::parse(&kind) {
self.host.subscribe(kind, false);
}
Ok(())
}
}
@@ -542,10 +535,10 @@ impl diagnostics::Host for StoreData {
diagnostics::Severity::Hint => ide_core::Severity::Hint,
},
message: clip(d.message, 4096),
source: Some(source.clone()),
source: Some(d.source.map_or_else(|| source.clone(), |s| clip(s, 128))),
code: d.code.map(|c| clip(c, 128)),
unnecessary: false,
deprecated: false,
unnecessary: d.tags.iter().any(|t| t == "unnecessary"),
deprecated: d.tags.iter().any(|t| t == "deprecated"),
server: Some(source.clone()),
})
.collect();
Binary file not shown.
@@ -1 +1 @@
86dfe70a30ab1f7955a16e96c1b32c8be663456b871cc4d3bed7dc6052c062da
a50d1eb41ab8dd0c9c8d6bae8266906cd6eb5de8e6f6ca6f681990389472bb5b
+1 -1
View File
@@ -1,5 +1,5 @@
//! Sample OpenIDEA plugin, written with the `openidea-plugin` SDK.
//! Build with `openidea-plugin build plugins/hello` (see plugins/README.md).
//! Build with `openidea-plugin build crates/ide-plugin/tests/fixtures/hello` (see plugins/README.md).
use openidea_plugin::views::{Body, Button, TreeItem, View, ViewEvent, Viewport, Views};
use openidea_plugin::{Context, Error, Invocation, LogLevel, NotificationLevel, Plugin, host, register_plugin};
Binary file not shown.
@@ -1 +1 @@
0dacb82ac35c1f132dc66e85dbf0663b729f818279d3b32cd06403e502fbf880
2fb6aa879562733bf5b7f11dabe8fc6a415c9ef695868fe7845417a4245d2dc8
Binary file not shown.
@@ -1 +1 @@
0ad2e4c6977a576b7eb27fc1e535fd056dd9da64bdfe6cfda9ca9f57537d0384
25b0f419a68a3b1651d3c2774d834b05e40ede52bc5ed8c0eba6d27a204efe82
@@ -40,3 +40,11 @@ title = "insert"
[[contributes.commands]]
id = "cancel"
title = "cancel"
[[contributes.commands]]
id = "subscribe"
title = "subscribe"
[[contributes.commands]]
id = "diagnose"
title = "diagnose"
+52 -4
View File
@@ -14,7 +14,9 @@ wit_bindgen::generate!({
generate_all,
});
use exports::openidea::plugin::providers::{self, CodeAction, CompletionItem, Document, InlineSuggestion, LineMarker};
use exports::openidea::plugin::providers::{
self, CodeAction, CompletionItem, Document, FileEdit, InlineSuggestion, LineMarker,
};
use exports::openidea::plugin::syntax_provider::{self, Fold, Symbol, Token};
use exports::openidea::plugin::{commands, event_handler, lifecycle};
use openidea::plugin::{invocation::Invocation, types};
@@ -67,6 +69,23 @@ impl commands::Guest for Misbehave {
}
// The user gave up: not the plugin failing.
"cancel" => Err(types::Error::Cancelled),
// Event kinds are names: one this IDE doesn't know is refused.
"subscribe" => openidea::plugin::events::subscribe(&arg.unwrap_or_default()).map(|()| None),
// Tagged problems from another tool on the line 0 of `arg`.
"diagnose" => {
use openidea::plugin::diagnostics::{self, Diagnostic, Severity};
let start = types::Position { line: 0, character: 0 };
let d = Diagnostic {
range: types::Range { start, end: start },
severity: Severity::Warning,
message: "unused".into(),
code: None,
source: Some("lint".into()),
tags: vec!["unnecessary".into(), "glowing".into()],
related: Vec::new(),
};
diagnostics::publish(&arg.unwrap_or_default(), &[d]).map(|()| None)
}
other => Err(types::Error::NotFound(format!("command {other}"))),
}
}
@@ -113,14 +132,24 @@ impl syntax_provider::Guest for Misbehave {
impl providers::Guest for Misbehave {
fn completion(doc: Document, _at: types::Position, _trigger: Option<String>) -> Vec<CompletionItem> {
behave(&doc.text);
// `rich`: a snippet that also adds an import.
let rich = doc.text == "rich";
let start = types::Position { line: 0, character: 0 };
vec![CompletionItem {
label: doc.path.clone(),
detail: None,
kind: Some("keyword".into()),
insert_text: doc.path,
insert_text: if rich { "f(${1:x})$0".into() } else { doc.path },
snippet: rich,
replace: None,
sort_text: None,
filter_text: rich.then(|| "fun".into()),
documentation: None,
additional_edits: if rich {
vec![types::TextEdit { range: types::Range { start, end: start }, text: "use f;\n".into() }]
} else {
Vec::new()
},
}]
}
@@ -128,8 +157,27 @@ impl providers::Guest for Misbehave {
None
}
fn code_actions(_doc: Document, _selection: types::Range) -> Vec<CodeAction> {
Vec::new()
/// `lazy`: one action whose edits come from `resolve-code-action`.
fn code_actions(doc: Document, _selection: types::Range) -> Vec<CodeAction> {
if doc.text != "lazy" {
return Vec::new();
}
vec![CodeAction {
title: "Fix later".into(),
kind: Some("quickfix".into()),
preferred: false,
edits: Vec::new(),
data: Some("fix".into()),
}]
}
fn resolve_code_action(doc: Document, action: CodeAction) -> Result<CodeAction, types::Error> {
if action.data.as_deref() != Some("fix") {
return Err(types::Error::Invalid("not one of mine".into()));
}
let start = types::Position { line: 0, character: 0 };
let edit = types::TextEdit { range: types::Range { start, end: start }, text: "fixed ".into() };
Ok(CodeAction { edits: vec![FileEdit { path: doc.path, edits: vec![edit] }], ..action })
}
fn definition(_doc: Document, _at: types::Position) -> Vec<types::Location> {
+58 -2
View File
@@ -123,8 +123,21 @@ fn not_a_jdk(p: &Path) -> String {
/// `mvn` of the configured Maven home (`languages.java.maven.home`): the
/// executable itself, `<home>/bin/mvn` or `<bin>/mvn`; else of the Maven the project
/// pins for mise / asdf when it has no Maven wrapper.
/// pins for mise / asdf when it has no Maven wrapper. With `maven.daemon` on, the
/// Maven Daemon (`mvnd`) replaces all of that — it embeds its own Maven — unless
/// it is not installed or the project JDK is older than the Java 17 its daemon
/// needs, in which case the build stays on mvn (with a logged warning).
pub fn maven_command(root: &Path, s: &Settings) -> Option<PathBuf> {
if s.languages.java.maven.daemon {
match mvnd_command(root, s) {
// No project JDK at all lets mvnd pick its own Java, which may still be new enough.
Some(mvnd) if project_jdk(root, s).is_none_or(|j| j.major >= 17) => return Some(mvnd),
Some(_) => tracing::warn!(
"Maven daemon: the project JDK is older than the Java 17 its daemon needs; building with mvn"
),
None => tracing::warn!("Maven daemon is enabled but mvnd was not found; building with mvn"),
}
}
let name = if cfg!(windows) { "mvn.cmd" } else { "mvn" };
let Some(p) = setting_path(root, &s.languages.java.maven.home) else {
if root.join("mvnw").is_file() || root.join("mvnw.cmd").is_file() {
@@ -141,6 +154,41 @@ pub fn maven_command(root: &Path, s: &Settings) -> Option<PathBuf> {
found
}
/// The `mvnd` executable when the Maven Daemon is enabled: the configured home
/// (`languages.java.maven.mvndHome`, the executable itself or `<home>/bin/mvnd`),
/// else `$MVND_HOME`, else `mvnd` on `PATH`, else the common Homebrew / sdkman
/// locations. `None` = not found; the caller falls back to `mvn`.
pub fn mvnd_command(root: &Path, s: &Settings) -> Option<PathBuf> {
let name = if cfg!(windows) { "mvnd.cmd" } else { "mvnd" };
if let Some(p) = setting_path(root, &s.languages.java.maven.mvnd_home) {
let found = [p.clone(), p.join("bin").join(name), p.join(name)].into_iter().find(|c| c.is_file());
if found.is_none() {
tracing::warn!("Maven Daemon home {} has no {name}", p.display());
}
return found;
}
if let Some(home) = std::env::var_os("MVND_HOME").map(PathBuf::from) {
let found = [home.join("bin").join(name), home.join(name)].into_iter().find(|c| c.is_file());
if found.is_none() {
tracing::warn!("MVND_HOME {} has no {name}", home.display());
}
return found;
}
if let Some(found) = crate::exec::which(name) {
return Some(found);
}
if let Some(home) = crate::home_dir() {
let sdkman = home.join(".sdkman/candidates/mvnd/current");
let candidates = [
sdkman.join("bin").join(name),
PathBuf::from("/opt/homebrew/opt/mvnd").join("bin").join(name),
PathBuf::from("/usr/local/opt/mvnd").join("bin").join(name),
];
return candidates.into_iter().find(|c| c.is_file());
}
None
}
/// Project launcher policy for sync and tool-window tasks.
pub fn gradle_command(root: &Path, build_dir: &Path, s: &Settings) -> PathBuf {
resolve_gradle_command(root, build_dir, s, true)
@@ -301,7 +349,7 @@ pub fn tsdk(root: &Path, s: &Settings) -> Option<PathBuf> {
const PYTHON_SERVERS: [&str; 7] = ["basedpyright", "pyright", "ty", "pyrefly", "pylsp", "jedi-language-server", "ruff"];
fn merge(base: &mut Value, layer: Value) {
pub(crate) fn merge(base: &mut Value, layer: Value) {
match (base, layer) {
(Value::Object(base), Value::Object(layer)) => {
for (k, v) in layer {
@@ -562,6 +610,13 @@ pub fn configure_server(cfg: &mut LanguageServerConfig, root: &Path, server_root
configure_vue_typescript(cfg, root, server_root, s);
}
// Intelephense reads it as `intelephense.environment.phpVersion` (`workspace/configuration`).
if name == "intelephense"
&& let Some(version) = crate::php::language_level(root, s)
{
merge_init_options(cfg, json!({ "environment": { "phpVersion": version } }));
}
// Node-based servers (`#!/usr/bin/env node`) run on the selected Node.js.
if let Some(dir) = node_dir(root, s) {
path_dirs.push(dir);
@@ -628,6 +683,7 @@ fn custom_server(name: &str, ls: &LanguageServerSettings) -> LanguageServerConfi
max_java: None,
launch_error: None,
tsserver_project: false,
resync_on_configuration: false,
pinned: None,
}
}
+3 -6
View File
@@ -1128,7 +1128,7 @@ mod tests {
/// The same jar supplies the JDT command and the private Maven output redirector.
pub fn kotlin_bridge_bundle(data_dir: &Path) -> Option<PathBuf> {
let extension = data_dir.join("kotlin-bridge-1.0.0.jar");
let bytes = include_bytes!("../../../plugins/kotlin-bridge/openidea-kotlin-bridge.jar");
let bytes = include_bytes!("../../../java/kotlin-bridge/openidea-kotlin-bridge.jar");
if std::fs::read(&extension).is_ok_and(|old| old == bytes) {
return Some(extension);
}
@@ -1171,7 +1171,7 @@ mod kotlin_bridge_bundle_tests {
/// Installs the app-owned JDT LS command bundle, falling back to ordinary LSP on IO failure.
pub fn usages_bundle(data_dir: &Path) -> Option<PathBuf> {
let extension = data_dir.join("java-usages-1.3.0.jar");
let bytes = include_bytes!("../../../plugins/java-usages/openidea-java-usages.jar");
let bytes = include_bytes!("../../../java/java-usages/openidea-java-usages.jar");
if std::fs::read(&extension).is_ok_and(|old| old == bytes) {
clean_usages_bundles(data_dir, &extension);
return Some(extension);
@@ -1256,10 +1256,7 @@ mod usages_bundle_tests {
fn installs_the_exact_bundle_and_reuses_it() {
let dir = tempfile::tempdir().unwrap();
let jar = super::usages_bundle(dir.path()).unwrap();
assert_eq!(
std::fs::read(&jar).unwrap(),
include_bytes!("../../../plugins/java-usages/openidea-java-usages.jar")
);
assert_eq!(std::fs::read(&jar).unwrap(), include_bytes!("../../../java/java-usages/openidea-java-usages.jar"));
assert_eq!(super::usages_bundle(dir.path()), Some(jar));
assert_eq!(std::fs::read_dir(dir.path()).unwrap().count(), 1);
}
+1
View File
@@ -18,6 +18,7 @@ pub mod locate;
pub mod lombok;
pub mod maven_profiles;
mod npm;
pub mod php;
pub mod pinned;
pub mod python;
pub mod ruby;
+1 -1
View File
@@ -246,7 +246,7 @@ fn sources_import_lombok(root: &Path) -> bool {
// Imports precede the first type, after at most a license header.
const HEAD: u64 = 8 * 1024;
let mut walker = ignore::WalkBuilder::new(root);
walker.require_git(false).follow_links(false).filter_entry(|e| {
walker.require_git(ide_core::ignored::inside_git_repo(root)).follow_links(false).filter_entry(|e| {
e.depth() == 0
|| !e.file_type().is_some_and(|t| t.is_dir())
|| !OUTPUT_DIRS.contains(&e.file_name().to_string_lossy().as_ref())
+210
View File
@@ -0,0 +1,210 @@
//! The PHP language level: the one chosen in Settings › PHP, else the one the root
//! `composer.json` declares (`config.platform.php`, else the lowest version `require.php` allows);
//! and the Intelephense storage that goes with it.
use crate::apply::{merge, merge_init_options};
use ide_core::settings::Settings;
use ide_language::LanguageServerConfig;
use serde_json::{Value, json};
use std::path::Path;
/// The project's PHP version as `X.Y.Z`; `None` when neither the settings nor
/// `composer.json` name one.
pub fn language_level(root: &Path, s: &Settings) -> Option<String> {
let explicit = s.languages.php.language_level.trim();
if !explicit.is_empty() {
return version(explicit).map(dotted);
}
let text = std::fs::read_to_string(root.join("composer.json")).ok()?;
let composer: Value = serde_json::from_str(&text).ok()?;
let platform = composer.pointer("/config/platform/php").and_then(Value::as_str).and_then(version);
platform.or_else(|| lowest_allowed(composer.pointer("/require/php")?.as_str()?)).map(dotted)
}
/// Gives each PHP version its own Intelephense workspace storage (below `data_dir`).
/// Intelephense keeps the built-in functions of the version a workspace was first indexed
/// with in that storage, and reuses them after the version changes. Without a version, or
/// with a `storagePath` of the user's, the server keeps its own storage.
pub fn configure_intelephense_storage(cfg: &mut LanguageServerConfig, data_dir: &Path) {
if cfg.name != "intelephense" {
return;
}
let mut options =
cfg.initialization_options.as_ref().and_then(|v| serde_json::to_value(v).ok()).unwrap_or(Value::Null);
if options.get("storagePath").is_some() {
return;
}
// The version Intelephense reads: `workspace/configuration` is answered from the
// settings merged over the initialization options (`ide_lsp::handler::all_settings`).
if let Some(settings) = &cfg.settings {
merge(&mut options, settings.clone());
}
let section = options.pointer("/settings/intelephense").unwrap_or(&options);
let version = section.pointer("/environment/phpVersion").and_then(Value::as_str);
let Some(version) = version.filter(|v| !v.is_empty() && v.chars().all(|c| c.is_ascii_digit() || c == '.')) else {
return;
};
let storage = data_dir.join("lsp").join("intelephense").join(version);
merge_init_options(cfg, json!({ "storagePath": storage.to_string_lossy() }));
}
type Version = (u32, u32, u32);
fn dotted((major, minor, patch): Version) -> String {
format!("{major}.{minor}.{patch}")
}
/// `7`, `7.4`, `v7.4.3`, `8.1.*`, `7.4.0-dev`: missing parts are 0.
fn version(s: &str) -> Option<Version> {
let s = s.trim().trim_start_matches(['v', 'V']);
let mut parts = s.split('.').map(|p| {
let digits: &str = &p[..p.find(|c: char| !c.is_ascii_digit()).unwrap_or(p.len())];
digits.parse::<u32>().ok()
});
let major = parts.next()??;
let minor = parts.next().flatten().unwrap_or(0);
let patch = parts.next().flatten().unwrap_or(0);
Some((major, minor, patch))
}
/// The lowest version a Composer constraint allows: the lowest lower bound of its
/// `||` alternatives. `None` when an alternative has no lower bound (`*`, `<8.0`).
fn lowest_allowed(constraint: &str) -> Option<Version> {
let bounds: Option<Vec<Version>> =
constraint.split('|').filter(|alt| !alt.trim().is_empty()).map(lower_bound).collect();
bounds?.into_iter().min()
}
/// The lower bound of constraints that must all hold (`>=7.2 <8.4`, `^7.4, !=7.4.1`,
/// `7.2 - 8.0`): the highest of their lower bounds.
fn lower_bound(alternative: &str) -> Option<Version> {
// A hyphen range: its first version.
let alternative = alternative.split(" - ").next().unwrap_or(alternative);
let mut tokens: Vec<String> = Vec::new();
for token in alternative.split([' ', ',']).filter(|t| !t.is_empty()) {
// `>= 7.2`: the operator and its version.
match tokens.last_mut() {
Some(op) if op.chars().all(|c| "<>=!~^".contains(c)) => op.push_str(token),
_ => tokens.push(token.to_owned()),
}
}
tokens
.iter()
.filter_map(|t| {
let t = t.split('@').next().unwrap_or(t);
if t.starts_with('<') || t.starts_with("!=") {
return None;
}
version(t.trim_start_matches(['>', '=', '^', '~']))
})
.max()
}
#[cfg(test)]
mod tests {
use super::*;
fn level(composer: Option<&str>, explicit: &str) -> Option<String> {
let tmp = tempfile::tempdir().unwrap();
if let Some(composer) = composer {
std::fs::write(tmp.path().join("composer.json"), composer).unwrap();
}
let mut s = Settings::default();
s.languages.php.language_level = explicit.into();
language_level(tmp.path(), &s)
}
fn required(constraint: &str) -> Option<String> {
level(Some(&serde_json::json!({ "require": { "php": constraint } }).to_string()), "")
}
#[test]
fn lowest_version_require_php_allows() {
assert_eq!(required("^7.4 || ^8.0").as_deref(), Some("7.4.0"));
assert_eq!(required("^8.0 || ^7.4").as_deref(), Some("7.4.0"));
assert_eq!(required(">=7.2 <8.4").as_deref(), Some("7.2.0"));
assert_eq!(required(">= 7.2, < 8.4").as_deref(), Some("7.2.0"));
assert_eq!(required("8.1.*").as_deref(), Some("8.1.0"));
assert_eq!(required("~7.4.3").as_deref(), Some("7.4.3"));
assert_eq!(required("7.2 - 8.0").as_deref(), Some("7.2.0"));
assert_eq!(required("^7.3|^8.0").as_deref(), Some("7.3.0"));
assert_eq!(required(">=8.1@dev").as_deref(), Some("8.1.0"));
assert_eq!(required("8").as_deref(), Some("8.0.0"));
}
#[test]
fn no_lower_bound_gives_nothing() {
for constraint in ["*", "<8.0", "", "^7.4 || *", "dev-main"] {
assert_eq!(required(constraint), None, "{constraint}");
}
}
#[test]
fn platform_overrides_require() {
let composer = r#"{"require": {"php": "^8.2"}, "config": {"platform": {"php": "7.2.34"}}}"#;
assert_eq!(level(Some(composer), "").as_deref(), Some("7.2.34"));
}
#[test]
fn no_requirement_gives_nothing() {
assert_eq!(level(None, ""), None);
assert_eq!(level(Some(r#"{"require": {"ext-json": "*"}}"#), ""), None);
assert_eq!(level(Some("not json"), ""), None);
}
#[test]
fn each_php_version_gets_its_own_intelephense_storage() {
let data = Path::new("/data");
let server = |name: &str, options: Value| {
let mut cfg: LanguageServerConfig =
toml::from_str(&format!("name = \"{name}\"\ncommand = \"{name}\"")).unwrap();
if !options.is_null() {
merge_init_options(&mut cfg, options);
}
cfg
};
let options = |cfg: &LanguageServerConfig| serde_json::to_value(&cfg.initialization_options).unwrap();
let mut cfg = server("intelephense", json!({"environment": {"phpVersion": "7.4.0"}, "clearCache": false}));
configure_intelephense_storage(&mut cfg, data);
let storage = data.join("lsp").join("intelephense").join("7.4.0");
assert_eq!(
options(&cfg),
json!({"environment": {"phpVersion": "7.4.0"}, "clearCache": false, "storagePath": storage.to_string_lossy()})
);
// The version Intelephense reads wins: Settings > Language Servers `settings` over the options.
let storage_of = |init: Value, settings: Value| {
let mut cfg = server("intelephense", init);
cfg.settings = Some(settings);
configure_intelephense_storage(&mut cfg, data);
options(&cfg)["storagePath"].clone()
};
let path = |v: &str| json!(data.join("lsp").join("intelephense").join(v).to_string_lossy());
let init = json!({"environment": {"phpVersion": "7.4.0"}});
assert_eq!(storage_of(init.clone(), json!({"environment": {"phpVersion": "8.3.0"}})), path("8.3.0"));
assert_eq!(storage_of(init, json!({"other": 1})), path("7.4.0"));
let nested = json!({"settings": {"intelephense": {"environment": {"phpVersion": "8.0.0"}}}});
assert_eq!(storage_of(Value::Null, nested), path("8.0.0"));
// No version, the user's own storage, not a version, another server: unchanged.
for unchanged in [
server("intelephense", Value::Null),
server("intelephense", json!({"environment": {"includePaths": []}})),
server("intelephense", json!({"environment": {"phpVersion": "8.1.0"}, "storagePath": "/mine"})),
server("intelephense", json!({"environment": {"phpVersion": "../8.1"}})),
server("phpactor", json!({"environment": {"phpVersion": "8.1.0"}})),
] {
let mut cfg = unchanged.clone();
configure_intelephense_storage(&mut cfg, data);
assert_eq!(cfg, unchanged);
}
}
#[test]
fn explicit_level_wins() {
let composer = r#"{"require": {"php": "^7.4"}, "config": {"platform": {"php": "7.4.33"}}}"#;
assert_eq!(level(Some(composer), "8.1").as_deref(), Some("8.1.0"));
assert_eq!(level(None, " 5.6 ").as_deref(), Some("5.6.0"));
}
}
+32
View File
@@ -38,6 +38,7 @@ fn server(name: &str) -> LanguageServerConfig {
max_java: None,
launch_error: None,
tsserver_project: false,
resync_on_configuration: false,
pinned: None,
}
}
@@ -1293,6 +1294,37 @@ fn php_sources_and_selected_interpreter() {
assert!(configs[3].env.get("PATH").is_none_or(|p| !p.contains("php82")));
}
#[test]
fn intelephense_gets_the_php_language_level() {
use serde_json::json;
let tmp = tempfile::tempdir().unwrap();
let configured = |name: &str, s: &Settings| {
let mut cfg = server(name);
configure_server(&mut cfg, tmp.path(), tmp.path(), s);
cfg
};
let options = |cfg: LanguageServerConfig| serde_json::to_value(cfg.initialization_options).unwrap();
// Nothing to resolve: the options stay as they were (the rest depends on the machine's tools).
assert_eq!(configured("intelephense", &Settings::default()).initialization_options, None);
std::fs::write(tmp.path().join("composer.json"), r#"{"require": {"php": "^7.4 || ^8.0"}}"#).unwrap();
assert_eq!(
options(configured("intelephense", &Settings::default())),
json!({"environment": {"phpVersion": "7.4.0"}})
);
for other in ["phpactor", "basedpyright", "vtsls"] {
assert_eq!(options(configured(other, &Settings::default())).get("environment"), None, "{other}");
}
let mut s = Settings::default();
s.languages.php.language_level = "8.1".into();
let mut cfg = server("intelephense");
cfg.initialization_options = Some(toml::Value::try_from(json!({"clearCache": true})).unwrap());
configure_server(&mut cfg, tmp.path(), tmp.path(), &s);
assert_eq!(options(cfg), json!({"clearCache": true, "environment": {"phpVersion": "8.1.0"}}));
}
#[test]
fn go_constraints_change_only_protocol_settings() {
use serde_json::json;
+91 -1
View File
@@ -35,13 +35,15 @@ fn mise_and_asdf_installs_are_detected_and_pinned_versions_used() {
let tmp = dunce::canonicalize(tmp.path()).unwrap();
let mise = tmp.join("mise");
let asdf = tmp.join("asdf");
// SAFETY: the only test in this binary; nothing else reads the environment yet.
// SAFETY: the only test in this binary that writes the environment; the daemon
// tests below read it, so they run after this one's fixtures are in place.
unsafe {
std::env::set_var("HOME", tmp.join("home"));
std::env::set_var("MISE_DATA_DIR", &mise);
std::env::set_var("ASDF_DATA_DIR", &asdf);
std::env::set_var("MISE_GLOBAL_CONFIG_FILE", tmp.join("no-global.toml"));
std::env::remove_var("JAVA_HOME");
std::env::remove_var("MVND_HOME");
}
let installs = mise.join("installs");
@@ -131,11 +133,99 @@ fn mise_and_asdf_installs_are_detected_and_pinned_versions_used() {
ide_sdk::apply::apply_to_run_configs(&mut plain, &project, &s);
assert_eq!(plain[0].command, installs.join("gradle/8.7/bin/gradle").to_string_lossy());
// The Maven daemon replaces the pinned Maven when it is on and installed; the
// project JDK (21, from mise.toml) can host its JVM.
let mut daemon = s.clone();
daemon.languages.java.maven.daemon = true;
daemon.languages.java.maven.mvnd_home = tmp.join("missing/mvnd").to_string_lossy().into_owned();
assert_eq!(
ide_sdk::apply::maven_command(&project, &daemon),
Some(maven.join("bin/mvn")),
"a dead-end mvnd home falls back to the pinned Maven"
);
let mvnd = tmp.join("mvnd");
script(&mvnd.join("bin/mvnd"), "mvnd 1.0-m7");
daemon.languages.java.maven.mvnd_home = mvnd.to_string_lossy().into_owned();
assert_eq!(
ide_sdk::apply::maven_command(&project, &daemon),
Some(mvnd.join("bin/mvnd")),
"a bare home resolves to <home>/bin/mvnd"
);
// The executable itself, like `maven.home`, also works.
daemon.languages.java.maven.mvnd_home = mvnd.join("bin/mvnd").to_string_lossy().into_owned();
assert_eq!(ide_sdk::apply::maven_command(&project, &daemon), Some(mvnd.join("bin/mvnd")));
// The project's Maven wrapper keeps precedence over a pinned Maven.
script(&project.join("mvnw"), "");
assert_eq!(ide_sdk::apply::maven_command(&project, &s), None);
// ...but not over the daemon.
assert_eq!(ide_sdk::apply::maven_command(&project, &daemon), Some(mvnd.join("bin/mvnd")));
// A version the project pins but that isn't installed: nothing is made up.
write(&project.join("mise.toml"), "[tools]\njava = '25'\n");
assert_eq!(ide_sdk::apply::project_jdk(&project, &s), None);
}
#[test]
fn maven_daemon_run_configs_follow_the_setting_and_the_jdk() {
let tmp = tempfile::tempdir().unwrap();
let tmp = dunce::canonicalize(tmp.path()).unwrap();
let project = tmp.join("project");
write(&project.join("pom.xml"), "<project/>");
let mvnd = tmp.join("mvnd");
script(&mvnd.join("bin/mvnd"), "mvnd 1.0-m7");
let mvn = tmp.join("mvn");
script(&mvn.join("bin/mvn"), "Apache Maven 3.9.6");
let mvnd_path = mvnd.join("bin/mvnd").to_string_lossy().into_owned();
let mvn_path = mvn.join("bin/mvn").to_string_lossy().into_owned();
let mut s = Settings::default();
s.languages.java.maven.daemon = true;
s.languages.java.maven.mvnd_home = mvnd_path.clone();
s.languages.java.maven.home = mvn_path.clone();
let run = |command: &str, detected: bool| RunConfig {
id: "maven:x".into(),
name: "x".into(),
kind: "maven".into(),
command: command.into(),
args: vec!["package".into()],
cwd: None,
env: BTreeMap::new(),
detected,
is_test: false,
debug_adapter: None,
allow_parallel: false,
extras: Default::default(),
};
// No project JDK: the daemon may pick its own, so it stays on mvnd.
let mut configs = [run("mvn", true), run("mvn", false)];
ide_sdk::apply::apply_to_run_configs(&mut configs, &project, &s);
assert_eq!(configs[0].command, mvnd_path, "detected config runs the daemon");
assert_eq!(configs[1].command, mvnd_path, "saved `mvn` config is pointed at the daemon");
// A launcher the user wrote into a configuration is theirs: an absolute path
// or a bare `mvnd` stays as configured.
let mut configs = [run(&mvn_path, false), run("mvnd", false)];
ide_sdk::apply::apply_to_run_configs(&mut configs, &project, &s);
assert_eq!(configs[0].command, mvn_path, "a custom absolute launcher is kept");
assert_eq!(configs[1].command, "mvnd", "a user-written mvnd is kept");
// A project JDK older than the daemon's Java 17 requirement keeps mvn.
let jdk8 = tmp.join("jdk8");
jdk(&jdk8, "1.8.0_392");
s.languages.java.jdk = jdk8.to_string_lossy().into_owned();
let mut configs = [run("mvn", true), run("mvn", false)];
ide_sdk::apply::apply_to_run_configs(&mut configs, &project, &s);
assert_eq!(configs[0].command, mvn_path, "JDK 8 cannot host the daemon JVM");
assert_eq!(configs[1].command, mvn_path);
// With the daemon off, the same configurations run the configured Maven.
s.languages.java.maven.daemon = false;
s.languages.java.jdk = String::new();
let mut configs = [run("mvn", true), run("mvn", false)];
ide_sdk::apply::apply_to_run_configs(&mut configs, &project, &s);
assert_eq!(configs[0].command, mvn_path);
assert_eq!(configs[1].command, mvn_path);
}
+65 -30
View File
@@ -142,42 +142,16 @@ pub(crate) fn run(req: Request<'_>) -> IdeResult<SearchSummary> {
}
}
if let Some((first, rest)) = on_disk.split_first()
&& !ctx.ctl.should_stop()
{
let mut walk = WalkBuilder::new(first);
for path in rest {
walk.add(path);
}
walk.hidden(true)
.ignore(true)
.git_ignore(true)
.git_global(true)
.git_exclude(true)
.parents(true)
.require_git(false)
.follow_links(false)
.threads(std::thread::available_parallelism().map_or(4, |n| n.get()).min(12))
.filter_entry(|e| e.file_name() != ".git");
walk.build_parallel().run(|| {
if !ctx.ctl.should_stop() {
walk_files(&on_disk, || {
let mut searcher = searcher_builder.build();
let matcher = matcher.clone();
let ctx = &ctx;
Box::new(move |entry| {
Box::new(move |path| {
if ctx.ctl.should_stop() {
return WalkState::Quit;
}
let entry = match entry {
Ok(entry) => entry,
Err(err) => {
tracing::debug!("search walk: {err}");
return WalkState::Continue;
}
};
// Archives (jar, zip) are browsed, not searched (IDEA skips them too).
if entry.file_type().is_some_and(|t| t.is_file()) && !ide_vfs::is_archive_file(entry.path()) {
ctx.search_file(entry.path(), &matcher, &mut searcher);
}
ctx.search_file(path, &matcher, &mut searcher);
if ctx.ctl.stopped() { WalkState::Quit } else { WalkState::Continue }
})
});
@@ -193,6 +167,67 @@ pub(crate) fn run(req: Request<'_>) -> IdeResult<SearchSummary> {
})
}
/// A per-thread file visitor of [`walk_files`].
pub(crate) type VisitFile<'s> = Box<dyn FnMut(&Path) -> WalkState + Send + 's>;
/// Visits the files below `roots` (and the roots that are files) in parallel,
/// the way IDEA's project scope sees them: directories ignored by
/// `.gitignore` / `.ignore` / `info/exclude` are skipped with everything in
/// them, while ignored files elsewhere (`.env`) and hidden files and folders
/// (`.github`, `.eslintrc`) are visited. `.git`, symlinks and archives (jar,
/// zip: browsed, not searched) are never visited.
pub(crate) fn walk_files<'s>(roots: &[PathBuf], mut visitor: impl FnMut() -> VisitFile<'s> + Send) {
let Some((first, rest)) = roots.split_first() else { return };
let mut walk = WalkBuilder::new(first);
for path in rest {
walk.add(path);
}
walk.hidden(false)
.ignore(true)
.git_ignore(true)
.git_global(true)
.git_exclude(true)
.parents(true)
.require_git(ide_core::ignored::inside_git_repo(first))
.follow_links(false)
.threads(std::thread::available_parallelism().map_or(4, |n| n.get()).min(12))
.filter_entry(|e| e.file_name() != ".git");
walk.build_parallel().run(|| {
let mut visit = visitor();
Box::new(move |entry| {
let entry = match entry {
Ok(entry) => entry,
Err(err) => {
tracing::debug!("search walk: {err}");
return WalkState::Continue;
}
};
let Some(file_type) = entry.file_type() else { return WalkState::Continue };
if file_type.is_dir() {
// The walker drops ignored files unseen, so the files of every walked
// directory come from listing it; the walker only picks the directories.
let Ok(children) = std::fs::read_dir(entry.path()) else { return WalkState::Continue };
for child in children.flatten() {
let path = child.path();
if child.file_name() != ".git"
&& child.file_type().is_ok_and(|t| t.is_file())
&& !ide_vfs::is_archive_file(&path)
&& matches!(visit(&path), WalkState::Quit)
{
return WalkState::Quit;
}
}
WalkState::Continue
} else if file_type.is_file() && entry.depth() == 0 && !ide_vfs::is_archive_file(entry.path()) {
// A root that is a file (an explicit file list); files below roots come from listings.
visit(entry.path())
} else {
WalkState::Continue
}
})
});
}
/// Walk roots for a scope; relative paths are resolved against `root`.
pub(crate) fn roots(root: &Path, scope: &SearchScope) -> IdeResult<Vec<PathBuf>> {
let dir = |path: &Path| -> IdeResult<PathBuf> {
+23 -42
View File
@@ -1,10 +1,10 @@
//! The files of a search scope, for analyses that read whole files
//! (Inspect Code, Locate Duplicates, Structural Search).
use crate::engine::{rel_path, roots};
use crate::engine::{rel_path, roots, walk_files};
use crate::{FileMask, PathFilter, SearchScope};
use ide_core::{CancellationToken, IdeResult};
use ignore::{WalkBuilder, WalkState};
use ignore::WalkState;
use std::path::PathBuf;
use std::sync::Mutex;
@@ -16,7 +16,7 @@ pub struct ScopeFile {
}
/// Lists the files of `scope` under `root` the way Find in Files walks them
/// (`.gitignore` honoured, hidden files and `.git` skipped), filtered by
/// (ignored folders and `.git` skipped; ignored and hidden files kept), filtered by
/// `mask` and `filter`, sorted by relative path. Explicitly listed files are
/// kept even when they do not exist on disk (unsaved buffers). Returns what was
/// found so far when cancelled.
@@ -43,42 +43,23 @@ pub fn scope_files(
on_disk.push(path);
}
}
if let Some((first, rest)) = on_disk.split_first() {
let mut walk = WalkBuilder::new(first);
for path in rest {
walk.add(path);
}
walk.hidden(true)
.ignore(true)
.git_ignore(true)
.git_global(true)
.git_exclude(true)
.parents(true)
.require_git(false)
.follow_links(false)
.threads(std::thread::available_parallelism().map_or(4, |n| n.get()).min(12))
.filter_entry(|e| e.file_name() != ".git");
walk.build_parallel().run(|| {
let found = &found;
let keep = &keep;
Box::new(move |entry| {
if cancel.is_cancelled() {
return WalkState::Quit;
}
let Ok(entry) = entry else { return WalkState::Continue };
if entry.file_type().is_some_and(|t| t.is_file()) && !ide_vfs::is_archive_file(entry.path()) {
let rel = rel_path(root, entry.path());
if keep(&rel) {
found
.lock()
.unwrap_or_else(|e| e.into_inner())
.push(ScopeFile { path: entry.into_path(), rel_path: rel });
}
}
WalkState::Continue
})
});
}
walk_files(&on_disk, || {
let found = &found;
let keep = &keep;
Box::new(move |path| {
if cancel.is_cancelled() {
return WalkState::Quit;
}
let rel = rel_path(root, path);
if keep(&rel) {
found
.lock()
.unwrap_or_else(|e| e.into_inner())
.push(ScopeFile { path: path.to_path_buf(), rel_path: rel });
}
WalkState::Continue
})
});
let mut files = found.into_inner().unwrap_or_else(|e| e.into_inner());
files.sort_by(|a, b| a.rel_path.cmp(&b.rel_path));
files.dedup_by(|a, b| a.path == b.path);
@@ -107,12 +88,12 @@ mod tests {
}
#[test]
fn project_scope_honours_gitignore_and_masks() {
fn project_scope_skips_ignored_folders_and_honours_masks() {
let dir = tree();
let none = FileMask::new(&[], &[]).unwrap();
let cancel = CancellationToken::new();
let all = scope_files(dir.path(), &SearchScope::Project, &none, None, &cancel).unwrap();
assert_eq!(rels(&all), vec!["src/a.rs", "src/c.txt", "src/sub/b.rs"]);
assert_eq!(rels(&all), vec![".gitignore", "src/a.rs", "src/c.txt", "src/sub/b.rs"]);
let rs = FileMask::new(&["*.rs".into()], &[]).unwrap();
let only_rs = scope_files(dir.path(), &SearchScope::Project, &rs, None, &cancel).unwrap();
@@ -120,7 +101,7 @@ mod tests {
let not_sub: PathFilter<'_> = &|rel: &str| !rel.starts_with("src/sub/");
let filtered = scope_files(dir.path(), &SearchScope::Project, &none, Some(not_sub), &cancel).unwrap();
assert_eq!(rels(&filtered), vec!["src/a.rs", "src/c.txt"]);
assert_eq!(rels(&filtered), vec![".gitignore", "src/a.rs", "src/c.txt"]);
}
#[test]
+3 -2
View File
@@ -1,7 +1,8 @@
//! Find / Replace in Files (⌘⇧F / ⌘⇧R).
//!
//! Built on ripgrep's engine crates: a parallel `ignore` walk (honouring
//! `.gitignore` / `.ignore`, skipping hidden files and `.git`) feeding
//! Built on ripgrep's engine crates: a parallel `ignore` walk (skipping the
//! folders `.gitignore` / `.ignore` exclude and `.git`, but like IDEA keeping
//! ignored files elsewhere and hidden files) feeding
//! `grep-searcher` with a `grep-regex` matcher. Binary files are skipped,
//! unsaved editor buffers (the overlay) take precedence over disk content,
//! and all columns are UTF-16.
+9 -4
View File
@@ -25,8 +25,8 @@ pub struct WordFiles {
}
/// Files under `query.dir` containing `query.word` as a whole word, in walk
/// order (unsaved buffers through `overlay`, `.gitignore`d and hidden files
/// skipped like Find in Files). Stops at `query.limit` files or when cancelled.
/// order (unsaved buffers through `overlay`; ignored folders skipped, ignored
/// and hidden files searched, like Find in Files). Stops at `query.limit` files or when cancelled.
pub fn files_with_word(
root: &std::path::Path,
query: &WordQuery,
@@ -74,7 +74,9 @@ mod tests {
let root = dir.path();
fs::create_dir_all(root.join("pkg/ignored")).unwrap();
fs::create_dir_all(root.join("build")).unwrap();
fs::write(root.join(".gitignore"), "ignored/\n").unwrap();
fs::write(root.join(".gitignore"), "ignored/\nlocal.py\n").unwrap();
// An ignored file outside ignored folders is still a candidate, as in IDEA.
fs::write(root.join("pkg/local.py"), "get_value\n").unwrap();
fs::write(root.join("pkg/use.py"), "x.get_value(1)\n").unwrap();
fs::write(root.join("pkg/longer.py"), "get_values()\n").unwrap();
fs::write(root.join("pkg/case.py"), "Get_Value()\n").unwrap();
@@ -90,7 +92,10 @@ mod tests {
found.files.sort();
assert_eq!(
found,
WordFiles { files: vec![root.join("pkg/open.py"), root.join("pkg/use.py")], limit_hit: false }
WordFiles {
files: vec![root.join("pkg/local.py"), root.join("pkg/open.py"), root.join("pkg/use.py")],
limit_hit: false
}
);
}
+97 -16
View File
@@ -187,24 +187,77 @@ fn file_masks() {
assert_eq!(with(&[], &["*.kt,*.ts"]), ["notes.md", "src/Util.java"]);
}
/// The files of `scope` under `root` (relative), sorted.
fn scope_rels(root: &Path, scope: &SearchScope) -> Vec<String> {
let none = ide_search::FileMask::new(&[], &[]).unwrap();
let files = ide_search::scope_files(root, scope, &none, None, &CancellationToken::new()).unwrap();
let mut rels: Vec<_> = files.into_iter().map(|f| f.rel_path).collect();
rels.sort();
rels
}
#[test]
fn honours_ignore_files_hidden_and_git() {
let p = project(&[
(".gitignore", b"target/\n*.log\n"),
(".ignore", b"generated.rs\n"),
("src/a.rs", b"needle\n"),
("src/generated.rs", b"needle\n"),
("target/debug/out.rs", b"needle\n"),
("build.log", b"needle\n"),
(".hidden/secret.rs", b"needle\n"),
(".env", b"needle\n"),
(".git/config", b"needle\n"),
("sub/.gitignore", b"local.txt\n"),
("sub/local.txt", b"needle\n"),
("sub/kept.txt", b"needle\n"),
]);
fn searches_ignored_and_hidden_files_but_not_ignored_folders() {
// #1317: like IDEA, only ignored folders are left out; ignored files and dotfiles are searched.
// (path, content, walked): every file mentions `needle`.
let files: &[(&str, &[u8], bool)] = &[
(".gitignore", b"# needle\ntarget/\n*.log\n.env\nsrc/local/\n", true),
(".ignore", b"# needle\ngenerated.rs\nout/\n", true),
("src/a.rs", b"needle\n", true),
// Ignored files outside ignored folders.
("src/generated.rs", b"needle\n", true),
("build.log", b"needle\n", true),
(".env", b"needle\n", true),
("sub/.gitignore", b"# needle\nlocal.txt\ncache/\n", true),
("sub/local.txt", b"needle\n", true),
("sub/kept.txt", b"needle\n", true),
// Hidden files and folders.
(".eslintrc", b"needle\n", true),
(".github/workflows/ci.yml", b"needle\n", true),
(".hidden/secret.rs", b"needle\n", true),
// Ignored folders, whatever they hold.
("target/debug/out.rs", b"needle\n", false),
("target/.env", b"needle\n", false),
("src/local/notes.txt", b"needle\n", false),
("out/gen.rs", b"needle\n", false),
("sub/cache/blob.txt", b"needle\n", false),
// Never: Git's own folder.
(".git/config", b"needle\n", false),
];
let p = project(&files.iter().map(|(path, text, _)| (*path, *text)).collect::<Vec<_>>());
let mut expected: Vec<&str> = files.iter().filter(|(_, _, walked)| *walked).map(|(path, _, _)| *path).collect();
expected.sort();
let (views, _) = run(p.path(), &query("needle"));
assert_eq!(paths(&views), ["src/a.rs", "sub/kept.txt"]);
assert_eq!(paths(&views), expected, "Find in Files");
assert_eq!(scope_rels(p.path(), &SearchScope::Project), expected, "scope files");
// A directory scope and an explicit ignored file are searched the same way.
let dir = SearchScope::Directory { path: ide_core::IpcPath(p.path().join("sub")) };
let (views, _) = run(p.path(), &SearchQuery { scope: dir, ..query("needle") });
assert_eq!(paths(&views), ["sub/.gitignore", "sub/kept.txt", "sub/local.txt"]);
let one = SearchScope::Files { paths: vec![ide_core::IpcPath(p.path().join("build.log"))] };
let (views, _) = run(p.path(), &SearchQuery { scope: one, ..query("needle") });
assert_eq!(paths(&views), ["build.log"]);
}
#[test]
fn nested_repository_ignores_outer_repository_rules() {
// #1254: the opened project is a Git repository inside a folder its parent repository ignores.
let p = project(&[
(".git/HEAD", b"ref: refs/heads/main\n"),
("projects/.gitignore", b"*\n!.gitignore\n"),
("projects/nested/.git/HEAD", b"ref: refs/heads/main\n"),
("projects/nested/.gitignore", b"logs/\n"),
("projects/nested/search-test.txt", b"somesss\n"),
("projects/nested/src/main.txt", b"somesss\n"),
("projects/nested/logs/debug.txt", b"somesss\n"),
]);
let nested = p.path().join("projects/nested");
let (views, _) = run(&nested, &query("somesss"));
assert_eq!(paths(&views), ["search-test.txt", "src/main.txt"]);
assert_eq!(scope_rels(&nested, &SearchScope::Project), [".gitignore", "search-test.txt", "src/main.txt"]);
}
#[test]
@@ -643,3 +696,31 @@ fn archives_are_skipped() {
let (views, _) = run(p.path(), &query("needle"));
assert_eq!(paths(&views), ["src/a.txt"]);
}
#[test]
fn a_nested_repository_is_not_hidden_by_the_outer_repositorys_gitignore() {
// parent/.git, parent/projects/.gitignore ignores everything, parent/projects/nested is its own repository.
let p = project(&[
("projects/.gitignore", b"*\n!.gitignore\n"),
("projects/nested/search-test.txt", b"somesss\n"),
("projects/nested/src/main.txt", b"somesss\n"),
("projects/nested/.gitignore", b"logs/\n"),
("projects/nested/logs/debug.txt", b"somesss\n"),
]);
std::fs::create_dir(p.path().join(".git")).unwrap();
std::fs::create_dir(p.path().join("projects/nested/.git")).unwrap();
let (views, _) = run(&p.path().join("projects/nested"), &query("somesss"));
assert_eq!(
paths(&views),
["search-test.txt", "src/main.txt"],
"the nested repository's own .gitignore still applies"
);
}
#[test]
fn a_project_inside_a_repository_keeps_the_repositorys_gitignore() {
let p = project(&[(".gitignore", b"build/\n"), ("app/build/out.txt", b"hit\n"), ("app/src/a.txt", b"hit\n")]);
std::fs::create_dir(p.path().join(".git")).unwrap();
let (views, _) = run(&p.path().join("app"), &query("hit"));
assert_eq!(paths(&views), ["src/a.txt"]);
}
+24 -3
View File
@@ -641,11 +641,11 @@ pub static TOOLS: &[ToolSpec] = &[
// ECJ 3.46.200 keeps the constructor call in a constructor's statements, which
// @SneakyThrows / @NonNull must not wrap or precede (lombok#4089).
// Return to Maven Central once an official release includes both.
// Source commit, build instructions and checksum: plugins/lombok/README.md.
// Source commit, build instructions and checksum: java/lombok/README.md.
method: Method::Bundled {
filename: "lombok-1.18.48-openidea.2.jar",
version: "1.18.48-openidea.2",
bytes: include_bytes!("../../../plugins/lombok/lombok-1.18.48-openidea.2.jar"),
bytes: include_bytes!("../../../java/lombok/lombok-1.18.48-openidea.2.jar"),
},
fallback: None,
version: V,
@@ -680,7 +680,9 @@ pub static TOOLS: &[ToolSpec] = &[
rule(Linux, Arm64, "kotlin-lsp-*-linux-aarch64.zip"),
rule(Windows, X64, "kotlin-lsp-*-win-x64.zip"),
],
locate: &["kotlin-lsp", "kotlin-lsp.sh", "kotlin-lsp.cmd"],
// `bin/intellij-server` is the launcher; current releases still ship the
// `kotlin-lsp.sh` / `kotlin-lsp.cmd` wrappers around it, deprecated, and old ones only those.
locate: &["intellij-server", "kotlin-lsp", "kotlin-lsp.sh", "kotlin-lsp.cmd"],
},
fallback: None,
version: V,
@@ -1044,6 +1046,25 @@ mod tests {
/// The v263 kotlin-server-* release notes: the CLI distribution archives plus
/// the VS Code extension and checksums; each platform must pick its native
/// distribution and the aarch64 archive must never be picked on x64.
#[test]
fn kotlin_lsp_runs_its_launcher_rather_than_the_deprecated_wrapper() {
let Method::Release { locate, .. } = &find("kotlin-lsp").unwrap().method else { unreachable!() };
let names: Vec<String> = locate.iter().map(|n| n.to_string()).collect();
let tmp = tempfile::tempdir().unwrap();
// Current releases: `kotlin-lsp.sh` warns that it will go and execs `bin/intellij-server`.
let current = tmp.path().join("current");
let dist = current.join("kotlin-server-263.7084.0");
std::fs::create_dir_all(dist.join("bin")).unwrap();
std::fs::write(dist.join("kotlin-lsp.sh"), "").unwrap();
std::fs::write(dist.join("bin/intellij-server"), "").unwrap();
assert_eq!(crate::archive::locate(&current, &names), Some(dist.join("bin/intellij-server")));
// Older releases have only the script.
let old = tmp.path().join("old");
std::fs::create_dir_all(&old).unwrap();
std::fs::write(old.join("kotlin-lsp.sh"), "").unwrap();
assert_eq!(crate::archive::locate(&old, &names), Some(old.join("kotlin-lsp.sh")));
}
#[test]
fn kotlin_lsp_picks_the_native_archive_for_each_platform() {
use crate::platform::{Platform, select_asset};
+2 -2
View File
@@ -71,7 +71,7 @@ fn bundled_lombok_is_available_offline_and_repairs_missing_or_corrupt_files() {
let env = Env::new();
let data = env.root.parent().unwrap();
let agent = managed_bundle(data, "lombok").unwrap();
let checksum = include_str!("../../../plugins/lombok/SHA256SUMS").split_whitespace().next().unwrap();
let checksum = include_str!("../../../java/lombok/SHA256SUMS").split_whitespace().next().unwrap();
assert_eq!(sources::sha256_file(&agent).unwrap(), checksum);
for contents in [None, Some(b"incomplete jar".as_slice())] {
if let Some(contents) = contents {
@@ -93,7 +93,7 @@ fn concurrent_windows_get_the_same_complete_bundled_lombok() {
let windows: Vec<_> = (0..8).map(|_| scope.spawn(|| managed_bundle(data, "lombok").unwrap())).collect();
let paths: Vec<_> = windows.into_iter().map(|window| window.join().unwrap()).collect();
assert!(paths.iter().all(|path| path == &paths[0]));
let checksum = include_str!("../../../plugins/lombok/SHA256SUMS").split_whitespace().next().unwrap();
let checksum = include_str!("../../../java/lombok/SHA256SUMS").split_whitespace().next().unwrap();
assert_eq!(sources::sha256_file(&paths[0]).unwrap(), checksum);
});
}
+227
View File
@@ -0,0 +1,227 @@
//! History for Selection: the commits that changed a block of lines
//! (`git log -L`), each with the block as it was before and after.
use crate::text_encoding::{TextEncoding, decode};
use crate::{BlockHistory, BlockRevision, LineHunk, Repo, cli};
use ide_core::{CancellationToken, IdeResult};
use std::path::Path;
/// Separates commits in the log output; fields of the header are separated by `\x1f`.
const RECORD: u8 = 0x1e;
const FORMAT: &str = "--format=%x1e%H%x1f%h%x1f%an%x1f%ae%x1f%at%x1f%s";
pub(crate) fn block_history(
r: &Repo,
path: &Path,
current_text: Option<&str>,
lines: (u32, u32),
encoding: TextEncoding,
cancel: &CancellationToken,
) -> IdeResult<BlockHistory> {
let rel = r.rel_path(path)?;
let head = {
let repo = r.gix()?;
crate::repo::blob_at(&repo, &rel, "HEAD")?
};
let Some(head) = head else {
return Ok(BlockHistory { head_lines: None, revisions: Vec::new() });
};
let head = decode(&head, encoding);
let current = match current_text {
Some(text) => text.to_owned(),
None => decode(&std::fs::read(r.abs_path(&rel))?, encoding).into_owned(),
};
let Some((start, end)) = head_range(&crate::diff::line_hunks(&head, &current), lines) else {
return Ok(BlockHistory { head_lines: None, revisions: Vec::new() });
};
let range = format!("-L{},{}:{rel}", start + 1, end + 1);
let args = ["log", "--no-color", "--no-ext-diff", "--no-textconv", FORMAT, &range, "HEAD"];
let out = cli::run_cancellable(r.root(), &args, cancel)?;
Ok(BlockHistory { head_lines: Some((start, end)), revisions: parse_log(&out.stdout, encoding) })
}
/// The HEAD lines (0-based, inclusive) of `lines` in the current text, given the
/// hunks between HEAD and it. A selection end inside a changed region takes the
/// whole region; None when every selected line is new.
pub(crate) fn head_range(hunks: &[LineHunk], (start, end): (u32, u32)) -> Option<(u32, u32)> {
let (start, end) = (start.min(end), start.max(end));
let to_head = |line: u32, at_end: bool| -> i64 {
let mut shift = 0i64;
for h in hunks {
if line < h.start {
break;
}
if line < h.start + h.lines {
return if at_end {
i64::from(h.old_start) + i64::from(h.old_lines) - 1
} else {
i64::from(h.old_start)
};
}
shift += i64::from(h.lines) - i64::from(h.old_lines);
}
i64::from(line) - shift
};
let (from, to) = (to_head(start, false), to_head(end, true));
(from >= 0 && from <= to).then_some((from as u32, to as u32))
}
/// Commits of `git log -L` output, newest first.
pub(crate) fn parse_log(stdout: &[u8], encoding: TextEncoding) -> Vec<BlockRevision> {
stdout
.split(|b| *b == RECORD)
.filter(|r| !r.is_empty())
.filter_map(|record| parse_record(record, encoding))
.collect()
}
fn parse_record(record: &[u8], encoding: TextEncoding) -> Option<BlockRevision> {
let newline = record.iter().position(|b| *b == b'\n').unwrap_or(record.len());
let header = String::from_utf8_lossy(&record[..newline]);
let mut fields = header.split('\x1f');
let id = fields.next()?.to_owned();
let short_id = fields.next()?.to_owned();
let author = fields.next()?.to_owned();
let email = fields.next()?.to_owned();
let time_ms = fields.next()?.trim().parse::<f64>().ok()? * 1000.0;
let summary = fields.next().unwrap_or_default().to_owned();
let patch = decode(record.get(newline + 1..).unwrap_or_default(), encoding);
let mut revision = BlockRevision {
id,
short_id,
author,
email,
time_ms,
summary,
path: String::new(),
before: String::new(),
after: String::new(),
before_line: 0,
after_line: 0,
};
let mut in_hunk = false;
for line in patch.lines() {
if let Some(path) = line.strip_prefix("+++ ") {
revision.path = unquote(path).trim_start_matches("b/").to_owned();
} else if let Some(ranges) = line.strip_prefix("@@ ") {
in_hunk = true;
let mut parts = ranges.split_whitespace();
revision.before_line = hunk_start(parts.next().unwrap_or_default());
revision.after_line = hunk_start(parts.next().unwrap_or_default());
} else if in_hunk {
match line.as_bytes().first() {
Some(b' ') => {
push_line(&mut revision.before, &line[1..]);
push_line(&mut revision.after, &line[1..]);
}
Some(b'-') => push_line(&mut revision.before, &line[1..]),
Some(b'+') => push_line(&mut revision.after, &line[1..]),
_ => {}
}
}
}
Some(revision)
}
/// 0-based first line of a hunk side (`-12,3` → 11; `+0,0` for a file that did not exist → 0).
fn hunk_start(side: &str) -> u32 {
let n = side.trim_start_matches(['-', '+']).split(',').next().unwrap_or_default();
n.parse::<u32>().unwrap_or(0).saturating_sub(1)
}
fn push_line(block: &mut String, line: &str) {
block.push_str(line);
block.push('\n');
}
/// A C-quoted path (`"b/a \"b\".txt"`), as git writes unusual names.
fn unquote(path: &str) -> String {
let Some(inner) = path.strip_prefix('"').and_then(|p| p.strip_suffix('"')) else {
return path.to_owned();
};
let mut out = String::new();
let mut chars = inner.chars();
while let Some(c) = chars.next() {
if c != '\\' {
out.push(c);
continue;
}
match chars.next() {
Some('t') => out.push('\t'),
Some('n') => out.push('\n'),
Some(other) => out.push(other),
None => {}
}
}
out
}
#[cfg(test)]
mod tests {
use super::*;
use crate::HunkKind;
fn hunk(start: u32, lines: u32, old_start: u32, old_lines: u32) -> LineHunk {
LineHunk { kind: HunkKind::Modified, start, lines, old_start, old_lines, old_text: String::new() }
}
#[test]
fn maps_the_selection_to_head_lines() {
// Unchanged file.
assert_eq!(head_range(&[], (3, 5)), Some((3, 5)));
// Two lines inserted at 1: current 4..=6 were HEAD 2..=4; a reversed selection too.
let inserted = [hunk(1, 2, 1, 0)];
assert_eq!(head_range(&inserted, (4, 6)), Some((2, 4)));
assert_eq!(head_range(&inserted, (6, 4)), Some((2, 4)));
// Selecting only the inserted lines: nothing committed.
assert_eq!(head_range(&inserted, (1, 2)), None);
// From an inserted line on: the HEAD lines after the insertion.
assert_eq!(head_range(&inserted, (2, 4)), Some((1, 2)));
// Three HEAD lines (5..=7) replaced by one: a selection ending in it takes all three.
let replaced = [hunk(5, 1, 5, 3)];
assert_eq!(head_range(&replaced, (4, 5)), Some((4, 7)));
assert_eq!(head_range(&replaced, (6, 6)), Some((8, 8)));
// Two HEAD lines deleted at 2: later lines move down by two.
let deleted = [hunk(2, 0, 2, 2)];
assert_eq!(head_range(&deleted, (2, 3)), Some((4, 5)));
}
#[test]
fn parses_git_log_line_ranges() {
let out = [
"\x1eaaaa1111\x1faaaa\x1fAda\x1f[email protected]\x1f1700000000\x1fRename greet",
"",
"diff --git a/src/a.txt b/src/b.txt",
"--- a/src/a.txt",
"+++ b/src/b.txt",
"@@ -2,2 +2,2 @@",
" keep",
"-hello",
"+hi",
"\x1ebbbb2222\x1fbbbb\x1fBo\x1f[email protected]\x1f1600000000\x1fAdd",
"",
"diff --git a/src/a.txt b/src/a.txt",
"--- /dev/null",
"+++ b/src/a.txt",
"@@ -0,0 +2,2 @@",
"+keep",
"+hello",
]
.join("\n");
let revisions = parse_log(out.as_bytes(), TextEncoding::from(encoding_rs::UTF_8));
assert_eq!(revisions.len(), 2);
let [newer, older] = &revisions[..] else { unreachable!() };
assert_eq!((newer.id.as_str(), newer.short_id.as_str(), newer.author.as_str()), ("aaaa1111", "aaaa", "Ada"));
assert_eq!((newer.time_ms, newer.summary.as_str(), newer.path.as_str()), (1.7e12, "Rename greet", "src/b.txt"));
assert_eq!((newer.before.as_str(), newer.after.as_str()), ("keep\nhello\n", "keep\nhi\n"));
assert_eq!((newer.before_line, newer.after_line), (1, 1));
assert_eq!((older.before.as_str(), older.after.as_str()), ("", "keep\nhello\n"));
assert_eq!((older.before_line, older.after_line), (0, 1));
}
#[test]
fn unquotes_c_quoted_paths() {
assert_eq!(unquote(r#""b/a \"q\"\tb.txt""#), "b/a \"q\"\tb.txt");
assert_eq!(unquote("b/plain.txt"), "b/plain.txt");
}
}
+44
View File
@@ -77,6 +77,35 @@ pub struct BlameLine {
pub summary: String,
}
/// History for Selection: the commits that changed a block of lines, newest first.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct BlockHistory {
/// The selected lines in HEAD (0-based, inclusive); None when none of them is committed.
pub head_lines: Option<(u32, u32)>,
pub revisions: Vec<BlockRevision>,
}
/// A commit that changed the block, with the block in its first parent and in it.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct BlockRevision {
pub id: String,
pub short_id: String,
pub author: String,
pub email: String,
pub time_ms: f64,
pub summary: String,
/// Repository-relative path of the file in this commit (before a later rename).
pub path: String,
/// The block before the commit; empty when the commit added it.
pub before: String,
pub after: String,
/// 0-based first line of the block before and after the commit.
pub before_line: u32,
pub after_line: u32,
}
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct Commit {
@@ -459,6 +488,7 @@ pub struct ChangelistState {
}
mod blame;
mod block_history;
pub mod changelists;
mod cli;
mod compare;
@@ -685,6 +715,20 @@ impl Repo {
let _scope = crate::ExecutionScope::enter(self.execution_allowed.load(std::sync::atomic::Ordering::Acquire));
write::blame(self, path, current_text, revision, encoding)
}
/// History for Selection: commits that changed `lines` (0-based, inclusive, of the
/// current text: `current_text`, or the file on disk), mapped to HEAD's lines first.
pub fn block_history(
&self,
path: &Path,
current_text: Option<&str>,
lines: (u32, u32),
encoding: impl Into<TextEncoding>,
cancel: &CancellationToken,
) -> IdeResult<BlockHistory> {
let encoding = encoding.into();
let _scope = crate::ExecutionScope::enter(self.execution_allowed.load(std::sync::atomic::Ordering::Acquire));
block_history::block_history(self, path, current_text, lines, encoding, cancel)
}
/// Commits newest-first in topological order (children before parents).
///
/// `branch` is any revision; when absent, HEAD plus all local and remote branches are
+124
View File
@@ -0,0 +1,124 @@
//! History for Selection (`git log -L`) against throwaway repositories built with the git CLI.
use ide_core::CancellationToken;
use ide_core::error::ErrorKind;
use ide_vcs::*;
use std::path::{Path, PathBuf};
use std::process::Command;
struct Fixture {
_tmp: tempfile::TempDir,
dir: PathBuf,
}
impl Fixture {
fn new() -> Self {
let tmp = tempfile::tempdir().unwrap();
let dir = tmp.path().join("work");
std::fs::create_dir(&dir).unwrap();
let fx = Fixture { _tmp: tmp, dir };
fx.git(&["init", "-q"]);
for (k, v) in [("user.name", "Test User"), ("user.email", "[email protected]"), ("commit.gpgsign", "false")] {
fx.git(&["config", k, v]);
}
fx
}
fn git(&self, args: &[&str]) -> String {
let out = Command::new("git")
.current_dir(&self.dir)
.args(["-c", "init.defaultBranch=main", "-c", "core.autocrlf=false"])
.args(args)
.output()
.expect("git runs");
assert!(out.status.success(), "git {args:?} failed: {}", String::from_utf8_lossy(&out.stderr));
String::from_utf8_lossy(&out.stdout).trim().to_string()
}
fn write(&self, rel: &str, content: &str) {
std::fs::write(self.dir.join(rel), content).unwrap();
}
fn commit(&self, msg: &str) {
self.git(&["add", "-A"]);
self.git(&["commit", "-q", "-m", msg]);
}
fn repo(&self) -> Repo {
Repo::discover(&self.dir).unwrap().expect("repository found")
}
fn path(&self, rel: &str) -> PathBuf {
self.dir.join(rel)
}
}
fn history(repo: &Repo, path: &Path, text: Option<&str>, lines: (u32, u32)) -> BlockHistory {
repo.block_history(path, text, lines, encoding_rs::UTF_8, &CancellationToken::new()).unwrap()
}
/// a.txt: added, line 2 edited, line 5 edited, renamed to b.txt, line 3 edited.
fn renamed_history() -> Fixture {
let fx = Fixture::new();
fx.write("a.txt", "one\ntwo\nthree\nfour\nfive\n");
fx.commit("Add a");
fx.write("a.txt", "one\nTWO\nthree\nfour\nfive\n");
fx.commit("Shout two");
fx.write("a.txt", "one\nTWO\nthree\nfour\nFIVE\n");
fx.commit("Shout five");
fx.git(&["mv", "a.txt", "b.txt"]);
fx.commit("Rename a to b");
fx.write("b.txt", "one\nTWO\nTHREE\nfour\nFIVE\n");
fx.commit("Shout three");
fx
}
#[test]
fn follows_the_block_through_edits_and_a_rename() {
let fx = renamed_history();
let h = history(&fx.repo(), &fx.path("b.txt"), None, (1, 2));
assert_eq!(h.head_lines, Some((1, 2)));
let shown: Vec<_> = h.revisions.iter().map(|r| (r.summary.as_str(), r.path.as_str())).collect();
// Edits outside the block and the pure rename are left out.
assert_eq!(shown, [("Shout three", "b.txt"), ("Shout two", "a.txt"), ("Add a", "a.txt")]);
let blocks: Vec<_> = h.revisions.iter().map(|r| (r.before.as_str(), r.after.as_str())).collect();
assert_eq!(blocks, [("TWO\nthree\n", "TWO\nTHREE\n"), ("two\nthree\n", "TWO\nthree\n"), ("", "two\nthree\n")]);
let newest = &h.revisions[0];
assert_eq!((newest.before_line, newest.after_line, newest.author.as_str()), (1, 1, "Test User"));
assert!(newest.id.starts_with(&newest.short_id) && newest.time_ms > 0.0);
}
#[test]
fn maps_an_edited_selection_to_head_lines() {
let fx = renamed_history();
let repo = fx.repo();
// Two unsaved lines inserted at the top: lines 3..=4 are HEAD's 1..=2.
let edited = "new\nnew\none\nTWO\nTHREE\nfour\nFIVE\n";
let h = history(&repo, &fx.path("b.txt"), Some(edited), (3, 4));
assert_eq!(h.head_lines, Some((1, 2)));
assert_eq!(h.revisions.len(), 3);
// Only new lines selected, or a file Git does not know: nothing committed to show.
let h = history(&repo, &fx.path("b.txt"), Some(edited), (0, 1));
assert_eq!((h.head_lines, h.revisions.len()), (None, 0));
fx.write("untracked.txt", "x\n");
let h = history(&repo, &fx.path("untracked.txt"), None, (0, 0));
assert_eq!((h.head_lines, h.revisions.len()), (None, 0));
}
#[test]
fn restricted_mode_runs_no_diff_driver_of_the_repository() {
let fx = renamed_history();
let marker = fx.path("textconv-ran");
fx.write(".gitattributes", "*.txt diff=probe\n");
fx.commit("Attributes");
fx.git(&["config", "diff.probe.textconv", &format!("sh -c 'touch {}; cat \"$0\"'", marker.display())]);
let repo = fx.repo();
repo.set_execution_allowed(false);
let h = history(&repo, &fx.path("b.txt"), None, (1, 2));
assert_eq!(h.revisions.len(), 3);
assert!(!marker.exists(), "a textconv driver ran in restricted mode");
}
#[test]
fn cancelled_requests_stop() {
let fx = renamed_history();
let cancel = CancellationToken::new();
cancel.cancel();
let err = fx.repo().block_history(&fx.path("b.txt"), None, (1, 2), encoding_rs::UTF_8, &cancel).unwrap_err();
assert_eq!(err.kind, ErrorKind::Cancelled);
}
+9 -2
View File
@@ -274,13 +274,14 @@ pub fn copy(from: &Path, to: &Path) -> IdeResult<()> {
/// honouring .gitignore, .ignore and `excludes`. Shared by index and search.
///
/// Hidden files are included; `.git` is always skipped. `.gitignore` applies
/// even outside a git repository. Symlinks to files are visited, symlinked
/// even outside a git repository; inside one, a nested repository's root is a
/// boundary for the outer repository's rules. Symlinks to files are visited, symlinked
/// directories are not followed.
pub fn walk_files(root: &Path, excludes: &[String], visit: impl Fn(&Path) + Send + Sync) {
let excludes = Arc::new(ExcludeMatcher::new(excludes));
let base = root.to_path_buf();
let mut builder = ignore::WalkBuilder::new(root);
builder.hidden(false).require_git(false).follow_links(false);
builder.hidden(false).require_git(ide_core::ignored::inside_git_repo(root)).follow_links(false);
builder.filter_entry(move |entry| {
if entry.depth() > 0 && entry.file_name() == ".git" {
return false;
@@ -324,6 +325,11 @@ pub struct FsBatch {
/// Folders holding changes filtered out as excluded or ignored. Only a view already
/// showing such a folder cares, so they don't count as worktree changes.
pub excluded_dirs: Vec<PathBuf>,
/// Files changed, created or removed that an ignore rule matches while their folder
/// is not ignored (`.env`). Kept apart from the worktree changes above (no Local
/// History, deployment or `fsChanged`), but the index lists them and open editors
/// follow them, like IDEA, where only ignored folders leave the project.
pub ignored: Vec<PathBuf>,
}
impl FsBatch {
@@ -333,6 +339,7 @@ impl FsBatch {
&& self.removed.is_empty()
&& self.vcs.is_empty()
&& self.excluded_dirs.is_empty()
&& self.ignored.is_empty()
}
}
+28 -1
View File
@@ -531,6 +531,13 @@ impl EventFilter {
&& !self.gitignore.is_ignored(path, is_dir)
}
/// Whether a filtered-out file is ignored by a rule on the file itself, in a folder
/// that is neither excluded nor ignored.
fn is_ignored_file(&self, path: &Path, rel: &Path) -> bool {
!self.excludes.read().is_excluded(rel)
&& path.parent().is_some_and(|dir| dir == self.root || !self.gitignore.is_ignored(dir, true))
}
fn classify(&self, path: &Path, change: Change, out: &mut BTreeMap<PathBuf, Change>) {
let rel = match path.strip_prefix(&self.root) {
Ok(rel) if rel.as_os_str().is_empty() => return,
@@ -642,6 +649,9 @@ impl EventFilter {
if change != Change::Modified {
excluded_dirs.extend(path.parent().map(Path::to_path_buf));
}
if !is_dir && self.is_ignored_file(&path, rel) {
batch.ignored.push(path);
}
continue;
}
}
@@ -700,6 +710,7 @@ mod tests {
assert_eq!(batch.vcs, vec![root.join(".git/HEAD"), root.join(".git/refs/heads/feature")]);
assert_eq!(batch.modified, vec![root.join("kept.txt")]);
assert_eq!(batch.removed, vec![root.join("from.txt"), root.join("gone.txt"), root.join("old.txt")]);
assert_eq!(batch.ignored, vec![root.join("debug.log")], "ignored files are reported apart");
}
#[test]
@@ -751,6 +762,20 @@ mod tests {
]);
assert!(batch.created.is_empty() && batch.modified.is_empty() && batch.removed.is_empty(), "{batch:?}");
assert_eq!(batch.excluded_dirs, vec![root.clone(), root.join("dist"), root.join("target/debug")]);
assert_eq!(batch.ignored, vec![root.join("debug.log")], "only the ignored file outside ignored folders");
// An ignored file's edit and removal reach the index too (nothing else changes).
let modify = EventKind::Modify(ModifyKind::Data(DataChange::Content));
let batch = filter.batch(&[ev(modify, vec![root.join("debug.log")])]);
assert_eq!(batch.ignored, vec![root.join("debug.log")]);
assert!(!batch.is_empty() && batch.excluded_dirs.is_empty());
std::fs::remove_file(root.join("debug.log")).unwrap();
let batch = filter.batch(&[
ev(EventKind::Remove(RemoveKind::File), vec![root.join("debug.log")]),
ev(EventKind::Remove(RemoveKind::File), vec![root.join("dist/app.js")]),
]);
// A removed path can't tell file from folder: it goes out as a removal, as before.
assert_eq!(batch.removed, vec![root.join("debug.log")]);
// Sibling files in unignored directories still come through.
let batch = filter.batch(&[ev(create_file, vec![root.join("src/main.rs")])]);
@@ -801,6 +826,7 @@ mod tests {
|paths: &[&str]| -> Vec<DebouncedEvent> { paths.iter().map(|p| ev(modify, vec![root.join(p)])).collect() };
let (batch, rules) = filter.batch_detailed(&touch(&["pkg/a.tmp", "pkg/gen/x.rs", "pkg/keep.rs", "b.tmp"]));
assert_eq!(batch.modified, vec![root.join("b.tmp"), root.join("pkg/keep.rs")]);
assert_eq!(batch.ignored, vec![root.join("pkg/a.tmp")], "nested rules, but not inside ignored folders");
assert!(rules.is_empty());
assert!(!filter.should_watch_dir(&root.join("pkg/gen")));
@@ -818,7 +844,8 @@ mod tests {
std::fs::create_dir_all(root.join(".git/info")).unwrap();
std::fs::write(root.join(".git/info/exclude"), "b.tmp\n").unwrap();
let (batch, rules) = filter.batch_detailed(&touch(&[".git/info/exclude", "b.tmp"]));
assert!(batch.is_empty(), "{batch:?}");
assert!(batch.modified.is_empty() && batch.vcs.is_empty(), "{batch:?}");
assert_eq!(batch.ignored, vec![root.join("b.tmp")]);
assert_eq!(rules, vec![root.clone()]);
}
+41
View File
@@ -259,6 +259,47 @@ fn walk_honours_ignores() {
assert_eq!(found, expected);
}
#[test]
fn walk_scopes_gitignore_to_each_repository() {
let (_t, r) = root();
// An outer repository ignores everything under `projects/`, nested repository included (#1254).
write(&r.join("outer/.git/HEAD"), "ref: refs/heads/main\n");
write(&r.join("outer/top.txt"), "");
write(&r.join("outer/projects/.gitignore"), "*\n!.gitignore\n");
write(&r.join("outer/projects/nested/.git/HEAD"), "ref: refs/heads/main\n");
write(&r.join("outer/projects/nested/.git/info/exclude"), "local.txt\n");
write(&r.join("outer/projects/nested/.gitignore"), "*.log\n");
write(&r.join("outer/projects/nested/search-test.txt"), "");
write(&r.join("outer/projects/nested/src/a.rs"), "");
write(&r.join("outer/projects/nested/debug.log"), "");
write(&r.join("outer/projects/nested/local.txt"), "");
write(&r.join("outer/projects/plain/file.txt"), "");
write(&r.join("loose/.gitignore"), "*.log\n");
write(&r.join("loose/a.txt"), "");
write(&r.join("loose/b.log"), "");
let walk = |dir: &str| {
let base = r.join(dir);
let found = Mutex::new(BTreeSet::new());
walk_files(&base, &[], |p| {
found.lock().insert(p.strip_prefix(&base).unwrap().to_string_lossy().replace('\\', "/"));
});
found.into_inner().into_iter().collect::<Vec<_>>()
};
let cases: [(&str, &[&str]); 4] = [
// The nested repository's own rules apply; the outer repository's do not.
("outer/projects/nested", &[".gitignore", "search-test.txt", "src/a.rs"]),
// A plain folder inside the outer repository is still governed by it, as in Git.
("outer/projects/plain", &[]),
("outer", &["projects/.gitignore", "top.txt"]),
// Outside any repository `.gitignore` still applies.
("loose", &[".gitignore", "a.txt"]),
];
for (dir, expected) in cases {
assert_eq!(walk(dir), expected, "{dir}");
}
}
fn wait_for(events: &Mutex<Vec<FsBatch>>, pred: impl Fn(&[FsBatch]) -> bool) -> bool {
let deadline = Instant::now() + Duration::from_secs(10);
while Instant::now() < deadline {
@@ -4,7 +4,7 @@ use super::{DocState, Documents, RegEntry, disk_len, keys_for};
use ide_core::error::ErrorKind;
use ide_core::{DocId, FileEdit, IdeError, IdeEvent, IdeResult, NullSink, WorkspaceEdit, WorkspaceEditPrecondition};
use parking_lot::Mutex;
use std::collections::VecDeque;
use std::collections::{HashMap, VecDeque};
use std::fs;
use std::io::Read;
use std::path::{Path, PathBuf};
@@ -259,7 +259,9 @@ fn hash_bytes(hash: &mut blake3::Hasher, bytes: &[u8]) {
hash.update(bytes);
}
fn hash_disk(hash: &mut blake3::Hasher, path: &Path) -> IdeResult<()> {
/// Hashes `path` (recursively for a directory). A file in `unsaved` contributes the bytes its
/// buffer would save, not its disk bytes, so saving a buffer leaves the hash unchanged.
fn hash_disk(hash: &mut blake3::Hasher, path: &Path, unsaved: &HashMap<PathBuf, Vec<u8>>) -> IdeResult<()> {
let meta = match fs::symlink_metadata(path) {
Ok(meta) => meta,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
@@ -277,6 +279,10 @@ fn hash_disk(hash: &mut blake3::Hasher, path: &Path) -> IdeResult<()> {
if meta.file_type().is_symlink() {
hash.update(b"link");
hash_bytes(hash, fs::read_link(path)?.as_os_str().as_encoded_bytes());
} else if let Some(bytes) = unsaved.get(path).filter(|_| meta.is_file()) {
hash.update(b"file");
hash.update(&(bytes.len() as u64).to_le_bytes());
hash.update(bytes);
} else if meta.is_file() {
hash.update(b"file");
hash.update(&meta.len().to_le_bytes());
@@ -295,7 +301,7 @@ fn hash_disk(hash: &mut blake3::Hasher, path: &Path) -> IdeResult<()> {
entries.sort_by_key(|e| e.file_name());
for entry in entries {
hash_bytes(hash, entry.file_name().as_encoded_bytes());
hash_disk(hash, &entry.path())?;
hash_disk(hash, &entry.path(), unsaved)?;
}
} else {
return Err(IdeError::invalid(format!("{} is not a regular file or directory", path.display())));
@@ -303,14 +309,19 @@ fn hash_disk(hash: &mut blake3::Hasher, path: &Path) -> IdeResult<()> {
Ok(())
}
/// A reverted edit may leave the dirty flag set even when all saved bytes match.
fn differs_from_disk(state: &DocState) -> IdeResult<bool> {
/// The bytes saving `state` would write.
fn saved_bytes(state: &DocState) -> IdeResult<Vec<u8>> {
let text = state.doc.text_for_save();
let bytes = if state.native_to_ascii {
Ok(if state.native_to_ascii {
ide_vfs::encode_properties_ascii(&text).into_bytes()
} else {
ide_vfs::encode(&text, &state.encoding, state.has_bom)?
};
})
}
/// A reverted edit may leave the dirty flag set even when all saved bytes match.
fn differs_from_disk(state: &DocState) -> IdeResult<bool> {
let bytes = saved_bytes(state)?;
match fs::read(&state.path) {
Ok(disk) => Ok(disk != bytes),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(true),
@@ -329,35 +340,31 @@ impl Documents {
Ok(())
}
/// What each path holds: disk contents, with unsaved buffers in place of their files. Saving
/// a buffer doesn't change it; edits, external writes and file operations do.
fn fingerprints(&self, paths: &[PathBuf]) -> IdeResult<Vec<WorkspaceEditPrecondition>> {
let mut unsaved = HashMap::new();
for id in self.open_documents() {
self.with_doc(id, |state| {
let file = text_path(&state.path)?;
if state.dirty && paths.iter().any(|path| file.starts_with(path)) && differs_from_disk(state)? {
unsaved.insert(file, saved_bytes(state)?);
}
Ok(())
})?;
}
paths
.iter()
.map(|path| {
let mut hash = blake3::Hasher::new();
hash_disk(&mut hash, path)?;
let mut dirty = Vec::new();
for id in self.open_documents() {
self.with_doc(id, |state| {
let file = text_path(&state.path)?;
if state.dirty && file.starts_with(path) && differs_from_disk(state)? {
dirty.push((
file,
state.doc.text(),
state.encoding.clone(),
state.doc.line_ending(),
state.has_bom,
));
}
Ok(())
})?;
}
dirty.sort_by(|a, b| a.0.cmp(&b.0));
for (file, text, encoding, ending, bom) in dirty {
hash_disk(&mut hash, path, &unsaved)?;
// Unsaved files not on disk (yet) or outside what the walk above reached.
let mut extra: Vec<_> =
unsaved.iter().filter(|(file, _)| file.starts_with(path) && !file.exists()).collect();
extra.sort_by(|a, b| a.0.cmp(b.0));
for (file, bytes) in extra {
hash_bytes(&mut hash, file.as_os_str().as_encoded_bytes());
hash_bytes(&mut hash, text.as_bytes());
hash_bytes(&mut hash, encoding.as_bytes());
hash_bytes(&mut hash, ending.as_str().as_bytes());
hash.update(&[u8::from(bom)]);
hash_bytes(&mut hash, bytes);
}
Ok(WorkspaceEditPrecondition { path: path.clone().into(), hash: hash.finalize().to_hex().to_string() })
})
@@ -395,13 +402,18 @@ impl Documents {
return Ok(());
}
let mut edit = normalize_edit(edit)?;
let captured = !edit.preconditions.is_empty();
self.prepare_workspace_edit(&mut edit)?;
self.validate_preconditions(&edit.preconditions)?;
for file in &edit.files {
if let FileEdit::Edit { path, version: Some(version), .. } = file {
let doc = self.by_path(path).ok_or_else(|| stale(path))?;
if i64::from(self.version(doc)?) != i64::from(*version) {
return Err(stale(path));
match self.by_path(path) {
Some(doc) if i64::from(self.version(doc)?) != i64::from(*version) => return Err(stale(path)),
Some(_) => {}
// Servers version files the editor never opened (gopls renames). The
// preconditions captured with the edit already cover its disk contents.
None if captured => {}
None => return Err(stale(path)),
}
}
}
Loaded 100 of 358 files, more files were not shown because too many files have changed in this diff. Show more