Compare commits

..
Author SHA1 Message Date
l0ng-ai 2b9e5c58d4 fix(plugins): refuse components that don't link against the host in check and install
check_component only compared import names and the API version, so a component
built from unreleased WIT (a function added under @1.0.0) or missing the plugin
world's exports passed `openidea-plugin check` and installing, then failed when
the IDE started it. It now links the component against the host's linker and
reports the missing or mismatched function with what to do.

Closes #1468
2026-11-05 13:46:20 +08:00
l0ng-ai 90e336761d fix(ui): don't finish a press the engine releases itself on macOS (#1463)
The shim for WebKit's reordered taps (#1072) took a press stamped at the same time as the
release before it for a stale one, and finished it with a synthetic pointerup, mouseup, click and
dblclick a tick later, even after the engine had delivered that press's own release. A fast
double-click whose second press shared the first release's time step was activated twice: on
macOS, double-clicking a Gradle task started it twice and the second start asked to stop the
first ("is not allowed to run in parallel"), which is why the linked Gradle E2E failed on main.

Only a press stamped before the release is stale now, and a press the engine releases before the
tick is left alone.
2026-11-05 13:33:35 +08:00
l0ng-ai 73be53b6d9 fix(macos): add a project opened over a fullscreen tab group as a tab (#1467)
A project-tab group goes fullscreen through simple fullscreen, which is_fullscreen() doesn't report, so with the default "prefer tabs in full screen only" a project opened over a fullscreen group came up as a floating window. The new-project tab target now also counts simple fullscreen.

Fixes #1438
2026-11-05 13:29:49 +08:00
l0ng-ai 2760c91bde feat(welcome): group recent projects (#1465)
Welcome › Projects can put recent projects into named groups: Move to Group (other groups, New Group…, Remove from group) on a project, Rename / New / Remove Group on a group (removing keeps its projects). Groups are listed first, fold with click, Enter or ←/→, and searching lists matches without groups. Groups live in recent-groups.json beside recent.json; grouped projects are exempt from the 30-project cap and leave their group when removed from recent projects.

Fixes #1420
2026-11-05 13:24:15 +08:00
l0ng-ai a5fb380960 fix(ui): install the reordered-tap shim only in WebKit (#1462)
The shim for WebKit bug 219670 also ran in Chromium on macOS (the e2e browser), where a scripted double-click stamps its second press like the release before it; the shim took that for a reordered tap and dispatched an extra click and dblclick, activating a tree row twice and timing out e2e/linkedGradle.spec.ts on a parallel-run prompt. It is now installed only in WebKit.
2026-11-05 13:09:56 +08:00
Mariano Dupontandmarianomd 8dedf5e73b fix(themes): preserve language-specific ICLS styles (#1457)
Importing a PhpStorm .icls applied Java/Kotlin keys and PHP-only colours to every language, so PHP variables, strings and punctuation came out wrong. Java, Kotlin and PHP attributes are now kept per language (Color Scheme lists them under their own nodes, and editors scope them by data-language), explicit attribute inheritance is resolved, an explicit TextAttributes value resets omitted bold/italic, the light/dark parent follows the scheme's background, and PHP classes, method declarations, fields, brackets and separators get their own tags. Selecting a language attribute in Settings previews that language's colours.

Co-authored-by: marianomd <[email protected]>
2026-11-05 12:59:38 +08:00
Nguyen Thinh 75c4ca0be8 feat(database): search and pagination on result query (#1432)
Adds a search bar to query result grids (match case / words / regex, previous/next occurrence, Filter rows) and a pager for console results (first/previous/next/last, page size). Later pages re-run the read-only query with a skip, which every driver reads past without decoding, so only one page crosses IPC; Last counts rows with the statement wrapped so a trailing line comment can't swallow the closing parenthesis.

Closes #1431

Co-authored-by: kidp2h <[email protected]>
2026-11-05 12:54:15 +08:00
l0ng-ai 7856fabbfe fix(editor): keep selected text visible over highlighted ranges (#1454)
The selection is drawn in a layer behind the text, so any range with its own background (find
matches, warnings in light schemes, usages, semantic backgrounds) hid it: selected text there
kept its yellow or other highlight. Selected text is now also marked in the innermost span with
the selection colour, in text editors, notebook cells and the VCS viewers.
2026-11-05 12:50:25 +08:00
l0ng-ai 4a90f124c6 fix: reopen long agent conversations quickly and pack the Git log graph (#1453)
Agent conversations: an editor tab mounts only while it is active, so switching back to a
task rebuilt the whole conversation, parsed and sanitized every message again, and fetched
all entries again, which replaced them and rendered everything a second time. Loaded entries
are already kept current by the task's events, so they are no longer fetched again; rendered
Markdown is cached by text; and a conversation opened at its end renders its last 40 entries
first and the earlier ones in batches behind it (all at once when a scroll position or Find
needs them).

Git log: with branch-aware layout (#1076) a commit's lane was its branch's global layout
index, so a repository with many branches drew lanes dozens of columns apart, joined by
diagonals across the row. Each row boundary and node level is now packed side by side in
layout order; adjacent rows share their boundary, so lines stay joined. The column header row
is hidden by default and can be shown from View Options › Columns › Column Headers, where
columns are resized and reordered.
2026-11-05 12:50:20 +08:00
l0ng-ai bb4e2db47f fix(python): name the project interpreter in the status bar wherever it lives (#1461)
An interpreter that detection doesn't look for (a virtualenv in a subfolder such as backend/.venv, or one set by path) was missing from sdk_detect, so the status bar and interpreter menu showed its raw path. sdk_detect for Python now adds the project's interpreter when detection didn't find it, named by the same rules, and a project-local .venv in a subfolder is named after that folder.

Fixes #1428
2026-11-05 12:49:13 +08:00
l0ng-ai fec391b9c1 fix(agents): keep registry installs running after Settings closes (#1460)
Closing Settings while an ACP Registry agent was installing cancelled the install, and Install was offered again while a previous attempt could still hold the agent's folder. Installs now outlive the page as a background task of the project windows (stoppable from the status bar); the backend refuses a second install of the same agent and marks it installing in the list, which a reopened page follows until it ends.

Fixes #1430
2026-11-05 12:43:42 +08:00
ppxbandppxb 43af99d3ec feat(app): ask before closing with running Run/Debug processes (#1445)
Closing a window, closing or switching a project, or quitting while Run/Debug sessions are still running now asks before terminating them (Terminate / Cancel / Don't ask again, sharing run.confirmTerminate with the Run tab prompt). Run publishes its live sessions through a RunningProcessesProvider contribution, so the close guard collects them without importing the run feature; the prompt comes before the unsaved-changes step so cancelling never discards edits.

Closes #1433

Co-authored-by: ppxb <[email protected]>
2026-11-05 12:34:35 +08:00
l0ng-ai 3e88fa1f39 fix(http-client): list Spring endpoints on Windows (#1456)
Spring route sources came back from the host with native separators, so on Windows every source failed the prefix check against the forward-slash project root and Search Everywhere showed no endpoints. RouteSource.path is now an IpcPath and the index compares by path segment; the add-to-ignore result also returns IPC paths.

Fixes #1422
2026-11-05 12:30:25 +08:00
DomKingandDomKing 6b89984863 fix(terminal): deliver the IME commits xterm drops on macOS (#1441)
IMEs that commit without composition events (Doubao on macOS) inject their text asynchronously while the keyCode-229 keydown that started the commit is still pending. xterm ignores an insertText event while a keydown is pending, and its deferred textarea diff misses text that lands after its snapshot, so a fast run of digits lost characters (1234567890 arrived as 138). On macOS the terminal now delivers those commits itself (only commits during a pending IME keydown, and composition input without compositionstart); real compositions, keys xterm already sent and input without an IME keydown stay xterm's. Linux and Windows keep the fcitx5 ImeCommitGate. Keystrokes are written to the PTY as they come instead of per animation frame, and those typed while the shell is still starting are sent once it exists.

Co-authored-by: DomKing <[email protected]>
2026-11-05 12:22:42 +08:00
黄贵波 a083571537 fix(editor): let the overview lane own the editor's vertical scrollbar (#1449)
The overview lane sat over the scrollbar with pointer-events: none and is 12px wide, the native scrollbar 10px: a press on the lane's edge, or where marks covered the thumb, reached the text layer and dragged a text selection instead of scrolling. The lane now owns the vertical scrollbar, as IntelliJ's error stripe does: the native one is hidden, the lane's thumb drags (with the same geometry it is drawn with), and a press above or below the thumb pages the editor. Marks still jump to their line.

Co-authored-by: IsGuibo <[email protected]>
2026-11-05 12:19:47 +08:00
l0ng-ai d3cb7c14ca feat(maven): add POMs outside the root reactor as Maven projects (#1446)
feat(maven): add POMs outside the root reactor as Maven projects

Add Maven Projects only re-linked POMs Maven had already discovered: picking a pom.xml inside the
project that the root POM doesn't list as a module (an independent build kept out of the reactor)
closed the dialog and changed nothing. Such POMs are now kept in the project's Maven settings
(linkedProjects) and loaded as independent top-level projects, after the root reactor, by the
static model, the sync, the Java language server import and the content roots; each runs its goals
in its own directory. A picked POM that the root reactor lists stays its module. The dialog takes
several POMs at once, and unlinking a linked project drops the link instead of hiding it.
2026-11-05 12:01:03 +08:00
yoo hyeong chan 0dfdbbc323 feat(gradle): link and unlink independent Gradle builds (#1413)
Opening a folder of independent Gradle builds selected only the first build. Link Gradle Projects… (pick a build's settings or build file) and Unlink on a build's root let an ordinary project window manage several independent builds without changing their scripts. The build model holds a list of Gradle builds; each keeps its own wrapper, working directory, environment, task routing and sync cache, keyed by the owning build directory. The links persist in the project's Gradle settings (linkedProjects); unlinking leaves files and saved run configurations alone. Existing includeBuild support is unchanged, and an explicitly linked build takes precedence over a composite that also includes it. Workspace windows keep Link/Unlink disabled.

Co-authored-by: gudcks0305 <[email protected]>
2026-11-05 11:23:39 +08:00
l0ng-ai 682fee633d fix(vfs): report files deleted on macOS again (#1435)
fix(vfs): report files deleted on macOS again

Since the watcher queues raw notify events (#1412), deleting a file on macOS was never reported, so
its symbols stayed in the index and the tree kept it (the project smoke tests failed for every
language). FSEvents delivers a deleted file's coalesced history flags as create, remove, then
modify events. Create + remove merged into a transient path, the following modify turned it back
into a creation, and a creation of a path that no longer exists is dropped. A modify no longer
revives a transient path; only a create does, and the disk decides as before.
2026-11-05 11:14:38 +08:00
l0ng-ai dce57ca83a fix(tools): find Python on Windows without python3.exe (#1427)
* fix(tools): find Python on Windows without python3.exe

Installing debugpy on Windows failed with "cannot run python3.exe … (os error 3)": the tool
manager looked for `python3` first, and on Windows that only finds the App Execution Alias in
WindowsApps, which can't start without the Store's Python. Windows installs `python.exe` and the
`py` launcher instead. On Windows the interpreter is now looked up as `python`, `py`, then
`python3`, and a WindowsApps alias only counts when it runs. The debugpy adapter's default
interpreter is `python` on Windows too. macOS and Linux keep `python3` first.

* fix(tools): keep searching past broken Python aliases
2026-11-05 10:59:03 +08:00
FizzyKey8 b674d0f2e8 feat(i18n): add Spanish translations (#1426)
Spanish (es, with es-MX / es_MX system locales) joins the UI languages under Settings › Appearance & Behavior › System Settings › Language and Region, with the key catalog (locales/es.json) and the source-text catalog (locales/text/es.json, 6180 entries).

Co-authored-by: FizzyKey8 <[email protected]>
2026-11-05 10:49:26 +08:00
Lee Dong Geon 001381459e fix(themes): match IntelliJ's gutter, selected tab border and completion colours on import (#1424)
Themes imported from IntelliJ drew a lighter gutter strip and kept the built-in blue for the selected tab's border and completion matches. The converter read the classic UI's GUTTER_BACKGROUND, where the new UI paints EDITOR_GUTTER_BACKGROUND or else the editor's background; it read EditorTabs.underlineColor, where Islands borders the selected tab with underlinedBorderColor; and it didn't map CompletionPopup.matchForeground. Scheme attributes written as `<value />` (no colours, not inherited) now show as plain text instead of keeping the built-in colours.

Co-authored-by: ldg030201 <[email protected]>
2026-11-05 10:48:21 +08:00
ntbang0901 38ac62f1a4 fix(ui): deliver the click of a tap WebKit reorders on macOS (#1072)
On macOS, while an editable element has focus and a third-party input
method (reproduced with WeChat's WeType) is active, WebKit can deliver a
tap as pointerup, mouseup, pointerdown, mousedown (WebKit bug 219670).
The tap's click is lost, and because the engine still considers the
button down, the next tap starts with a bare mousedown and no
pointerdown. Taps on tree rows, tabs and tool window buttons needed a
second tap.

A press stamped no later than the pointerup just before it is a
reordered tap: the shim finishes it with a synthetic pointerup, mouseup
and click (and dblclick for a double) after the page's handlers ran, and
dispatches the skipped pointerdown when a primary mouse press arrives
with no pointer down. The next tap's own click is kept. Installed on
macOS only; untrusted events and other pointer types are ignored.

Verified in a WKWebView with WeType, alternating taps on two targets:
main 19/20 taps clicked their target, with this change 20/20, every press
ran the pointerdown handlers and setPointerCapture worked; with the ABC
layout the shim dispatches nothing.

Refs #760, #724

Co-authored-by: ntbang0901 <[email protected]>
2026-11-05 10:30:43 +08:00
ntbang0901 29c54c243a fix(titlebar): start window drags on macOS only for a press that is still held (#1074)
On macOS, with tap to click and a third-party input method active, WebKit
can deliver a tap's mousedown after the button is already up (WebKit bug
219670). Tauri's drag-region script starts the native drag on any
mousedown, so AppKit ran its drag loop with no release left to end it and
the window followed the pointer until the next click.

On macOS the title bar now uses its own drag region: a drag starts only
when the mousedown has the primary button down, and the new
window_start_drag command checks NSEvent.pressedMouseButtons before
start_dragging. Double-click zoom stays on mouseup and is cancelled by
movement. The handlers are installed once per page. Windows and Linux
keep data-tauri-drag-region.

Refs #760

Co-authored-by: ntbang0901 <[email protected]>
2026-11-05 10:29:51 +08:00
Lee Dong Geon 54502e0a17 fix(run): run single Java files whose main class is not the first top-level type (#1417)
A Java file outside a Maven / Gradle build ran as `java File.java` (JEP 330), which launches the file's first top-level type on JDK 21 and earlier. A file whose main is in a later class failed with "can't find main(String[]) method in class: Parent", and the gutter Run of a second or nested class's main launched the first class. Such files now compile with the JDK's own `javac -g -d <class dir>` and run `java -cp <class dir> <main class>`; a main in the first class, and compact source files, still use the source launcher. Debug passes the JDWP agent through JDK_JAVA_OPTIONS, which only the java launcher reads.

Co-authored-by: ldg030201 <[email protected]>
2026-11-05 10:28:21 +08:00
l0ng-ai e129840651 build: name the quick-xml version ide-host uses in Cargo.lock (#1425)
#1176 brought in quick-xml 0.41 next to ide-host's 0.42, so ide-host's dependency in Cargo.lock must name its version; without it cargo rewrites the lock on every build and --locked builds fail.
2026-11-05 10:24:43 +08:00
l0ng-ai 77f86616c0 fix(host): don't block the event loop while a closing project shuts down (#1419)
Window destruction and the close_project command detach the project at once and let its language servers, kernels and debug sessions shut down in the background; opening a project waits for pending shutdowns and exit waits for them with a bound.

Closes #1414
2026-11-05 10:06:18 +08:00
l0ng-ai 89ffde304b fix(plugins): read the marketplace index on first open (#1378)
On a fresh install, Settings › Plugins › Available showed no marketplace plugins until the user refreshed by hand: opening the page asked for the listing without a refresh, which only returned the cached index, and a first run has none. The listing now reads the index when none has been read yet; later opens keep using the index already read, and an explicit refresh reads it again.
2026-11-05 10:04:42 +08:00
Piettro 1248c3231e feat(linux): open and save dialogs through xdg-desktop-portal (#1176)
On Linux, open and save dialogs were always the in-process GTK3 file chooser, even on KDE Plasma, because tauri-plugin-dialog defaults to its gtk3 feature. It now uses the xdg-portal feature, so dialogs go through xdg-desktop-portal and show the desktop's own chooser; without a portal, rfd falls back to zenity. Only Linux and the BSDs change; macOS and Windows dialogs are the same. Message, ask and confirm boxes are drawn by the app and don't use the plugin.

Co-authored-by: pcastro93 <[email protected]>
2026-11-05 10:04:26 +08:00
qiaojianlong ff9f821621 fix(tools): reuse the gh login for GitHub API downloads (#967)
Tool downloads from GitHub releases called api.github.com anonymously unless GITHUB_TOKEN or GH_TOKEN was set, so they hit the 60-requests-an-hour limit and failed with 403. When neither variable is set, the request now uses the active github.com account from `gh auth token` (the same login the GitHub features use), asked per request with a 5 s, cancellable lookup. The token goes to curl over stdin (`--header @-`), only for https://api.github.com, with `-q` so a user curlrc can't log or forward it, HTTPS-only redirects, and redacted from error messages. Without gh or a login, downloads stay anonymous.

Co-authored-by: qjlqyf <[email protected]>
2026-11-05 09:56:38 +08:00
huanglininnandmoyining 5279443ac5 fix(debugger): send Windows netcoredbg source paths with backslashes (#1409)
Co-authored-by: moyining <[email protected]>
2026-11-05 09:55:02 +08:00
Lee Dong Geon e648c0c46f ci: build with line tables only and move actions to Node 24 (#1194)
CI Rust jobs build with line tables only (CARGO_PROFILE_DEV_DEBUG), which cut the Rust jobs from about 12 to 7.7 minutes on Ubuntu and 18 to 12.6 minutes on Windows and shrinks the cache. The workflows move to the actions releases that run on Node 24: checkout, setup-node and pnpm/action-setup v5, upload-artifact v6, github-script v8 (ci, pr-check and release).

Co-authored-by: ldg030201 <[email protected]>
2026-11-05 09:53:14 +08:00
Fabio Nunes 791c957284 chore(build): keep only line tables in dev builds (#1260)
Full debug info made up most of target/: each worktree's build output was 15–30 GB. Dev builds now keep line tables for workspace crates (file:line stays in backtraces) and no debug info for dependencies, which brings a cold `cargo build` to about 9 GB and roughly halves cold build time. A debugger can't show locals any more; re-enable with `--config profile.dev.debug=true` when needed.

Co-authored-by: LittleGuava <[email protected]>
2026-11-05 09:52:54 +08:00
l0ng-ai 59071ac426 fix(vfs): preserve removals of files discovered by scans (#1412) 2026-11-05 09:51:38 +08:00
Fabio Nunes ad27a7064a fix(security): keep saved passwords in the OS credential store (#910)
SSH passwords and key passphrases, git HTTPS credentials, the HTTP proxy
password and data source passwords were kept in plaintext JSON files in
the app data directory. They now live in the OS credential store via
keyring 4 (macOS Keychain, Windows Credential Manager, Secret Service on
Linux); each file keeps only an index of keys under a per-directory id.

Files from older versions are moved key by key on first use, with a
.migrating sidecar so a failed or interrupted move retries without
duplicating credentials. Without a credential store (headless Linux)
the old file is still read but nothing new is written to disk; the
secret lasts for the session. New data sources no longer save their
password or SSH tunnel secret unless asked.

Also keeps the index path in SecretStore for global data sources and
runs their test against the in-memory store.

Fixes #740 (credentials part)

Co-authored-by: LittleGuava <[email protected]>
2026-11-05 09:47:19 +08:00
Jaw 116b1624b3 fix(workbench): narrow the icon-only tool window stripe (#1405)
The icon-only tool window stripes were 44px wide against a 34px bottom frame, and compact mode shrank the status bar but not the stripes. Icon-only stripe buttons now follow a --stripe-icon-button token (28px, 24px in compact mode) with a 4px outer margin, so the side frames are 38px (34px compact). Dragging the stripe edge stops at the same width. The unused --h-stripe-button token is removed; the stripe with names is unchanged.

Co-authored-by: jaw52 <[email protected]>
2026-11-05 09:35:53 +08:00
l0ng-ai de0f9d6ecd feat(database): global data sources listed in every project (#1407)
Data sources can be made global (Global option in their properties): stored in data-sources.toml of the app data directory, listed in every project, with their saved passwords under an app-wide key that moves along when the scope changes. Fixes #1252
2026-11-05 09:24:23 +08:00
l0ng-ai 6d55eb4aa9 fix(plugins): hash package paths with forward slashes on every OS (#1315)
package_hash fed each package-relative path to the digest with
to_string_lossy, so a walked path hashed as `web\index.html` on Windows
and `web/index.html` elsewhere. The same package had two digests: a
market index built on Linux failed verification on Windows, and a
package signed on one OS did not verify on another.

Join path components with `/` before hashing. Digests on macOS and
Linux are unchanged byte for byte; installed plugins keep their recorded
.package-hash, and a recomputed digest is accepted as a silent update by
the grant ledger, so existing Windows installs are not re-prompted.

Closes #1258
2026-11-05 09:20:42 +08:00
l0ng-ai a247f1ce67 feat(vcs): draw lane changes in the Git log graph as straight diagonals (#1406)
Lane changes bent within half a row as S-curves. A lane change and the vertical half of the neighbouring row that continues it now meet halfway between the lanes, so edges between adjacent rows run straight from node to node; only drawn positions change. Fixes #800
2026-11-05 09:12:41 +08:00
l0ng-ai f16ec29f4a fix(dev): rewrite src/bindings.ts only when the bindings change (#1404)
Debug launches (also second instances that only hand over a path) exported the bindings and rewrote an unchanged src/bindings.ts, which made Vite reload the running app's windows. The bindings now go to a temporary file first and the source file is replaced only when it differs.
2026-11-05 09:05:42 +08:00
ntbang0901 02d30b1609 fix(editor): keep the caret and view when a file changes in distant places (#1329)
Document::diff only trimmed the common prefix and suffix, so two distant
edits on disk became one change spanning everything between them. The
caret and scroll position inside that span were mapped to its start,
which made the editor jump to the first edit after an agent, formatter
or git operation rewrote the file.

Diff the changed middle line by line (Myers via similar, 50 ms deadline
with a coarser fallback) and emit one change per edited region, trimmed
to its differing characters. The frontend DocSync path produces the same
per-region changes, so external reloads, local history restores and
formatting keep the caret and view in place.

Closes #1326

Co-authored-by: ntbang0901 <[email protected]>
2026-11-05 09:04:48 +08:00
l0ng-ai 01e09565b7 feat(run): recognize TestNG classes annotated with @Test (#1403)
TestNG's class-level @Test makes the class's public void methods tests; the scanner now recognizes them (excluding @Before*/@After*/@DataProvider/@Factory). Test results locate TestNG failures past org.testng frames and parse TestNG's comparison message. Refs #1368
2026-11-05 08:59:49 +08:00
l0ng-ai 1f70b134de feat(vcs): show the selected commit's details in the Push dialog (#1402)
Selecting one outgoing commit shows its message, revision, author and time above its files; a Group by Directory toggle switches the files between tree and flat list and is remembered. Fixes #1373
2026-11-05 08:57:19 +08:00
l0ng-ai 11fd64f399 feat(java): add the "Serializable class without 'serialVersionUID'" inspection (#1401)
Settings › Editor › Inspections › Java gets the inspection, off by default. When on, jdtls's compiler reports the missing field and offers the add default/generated ID quick fixes. Built-in inspections can now be off by default, the backend resolves the profile in effect, the compiler-defaults file is named after the variant so jdtls restarts with it, and saving inspection settings reconfigures language servers. Fixes #1199
2026-11-05 08:50:59 +08:00
l0ng-ai 8a884b6362 test(project): smoke-test opening each language's fixture (#1398)
Opens each fixtures/ project through the ops the app's commands call and checks the source and test roots from its build file, the index, Search Everywhere / Go to Class / Go to Symbol answered from it and kept current as files come and go, and the Maven / Gradle model. Ignored tests run a Maven and a Gradle sync and check virtualenv and JDK detection; the Language smoke java, kotlin and python jobs run them.
2026-11-05 08:49:08 +08:00
l0ng-ai 481d44930f fix(themes): add --syn-global to the Harbor Night scheme and the token reference (#1400)
#1393 added the --syn-global token to tokens.css but not to the Harbor Night scheme (the theme template) or the generated token reference in docs/plugins/themes.md, so the theme tests failed. The scheme gives it the --syn-variable colour, as tokens.css does.
2026-11-05 08:48:39 +08:00
l0ng-ai f5895d0336 test(debug): smoke-test debugging each language's fixture (#1357)
Debugs each language's fixture the way the editor does, against openidea-server and the debugger the IDE installs: a breakpoint on the entry point's marker line, Debug from the file's context, then the paused frame, its variables, Evaluate Expression, Step Over and Resume. Runs per language in the Language smoke workflow. The build model is loaded first, as the Build Tools feature does, so a Gradle Application is debugged through Gradle; Step Over allows a repeated stop on a line (Python 3.12 reports a conditional expression's line twice).
2026-11-05 08:32:11 +08:00
l0ng-ai c3c1d88d10 fix(vcs): let Commit and Shelf tree names use the whole row width (#1396)
Tree names in the Commit and Shelf windows were capped at 75% of the row, so long folder names were truncated with empty space left over. Names now take the whole row and the grey detail after them is truncated first. Fixes #1367
2026-11-05 08:25:51 +08:00
l0ng-ai a0ac988963 fix(debugger): debug uv / poetry run configurations in the project's environment (#1395)
debugpy ran the project manager (`uv`, `poetry`, …) as a module. Configurations run through uv, poetry, pdm, pipenv, hatch or rye now pass the manager as the debuggee's interpreter, keep the script / -m module / tool as the target, and run the adapter and launcher on a plain interpreter (debugLauncherPython, now preferred by the backend for the adapter command).
2026-11-05 08:18:18 +08:00
l0ng-ai d5c4066748 feat(editor): colour global variables (the semantic global modifier) (#1393)
lua-language-server marks globals with the `global` token modifier, but
the editor dropped it, so a global variable looked like a local unless
rainbow highlighting was on. The modifier now sets `cm-sem-global`, drawn
in a new Language Defaults › Identifiers › Global variable colour
(`--syn-global`): library globals keep the predefined colour and calls the
function-call colour. Imported IntelliJ themes and .icls schemes take it
from DEFAULT_GLOBAL_VARIABLE. Built-in schemes default it to the local
variable colour (IntelliJ's fallback for the key), so they look as before
until a scheme sets it.

Fixes #1370
2026-11-05 07:59:45 +08:00
Pedro Pessoa 8d784ac164 feat(plugins): let plugin views open a project file at a line (#1362)
Plugin tool windows that list code locations (routes, failed tests,
queries) had no way to take the user to file:line. A host import
`views.open-location(path, line, column)` opens a file inside the project
in an editor of the window the call came from, caret at the given line and
column (0-based, UTF-16). It needs `project.read` and refuses paths outside
the project; the event goes only to the calling window, as `open-terminal`
does. The Rust SDK exposes it as `host::open_location` (with the mock), the
API table lists it, and the shipped components are rebuilt for the new WIT.

Closes #1334

Co-authored-by: yzPeedro <[email protected]>
2026-11-05 07:56:02 +08:00
Pedro Pessoa 7ffaa0b5da feat(plugins): let plugin tool windows use an SVG icon from the package (#1363)
A view's `icon` could only be a built-in icon name, so a plugin's tool
window button couldn't show the plugin's own mark. Like `fileIcons`, an
`icon` ending in `.svg` is now a path in the package: the host reads and
checks it at install time (a missing file or an SVG with `<script>` refuses
the install), serves it as the `viewIcon/<id>` asset, and the tool window
registers it as an external icon, falling back to the plugin icon if it
can't be loaded. Icon names work as before.

Closes #1335

Co-authored-by: yzPeedro <[email protected]>
2026-11-05 07:48:39 +08:00
l0ng-ai ab2b53ff80 fix(debugger): ignore a failed attach to a JVM that re-listened as its debugger detached (#1380)
A JDWP agent in server mode listens again, on a new port, when its debugger detaches. When the debugger detaches as the JVM exits (kotlin-debug-adapter on VM death, or Stop on a Maven/Gradle debug run), the announcement reached the Run feature, which attached again: the new session replaced the one that had just finished, then failed with "Cannot start debugger". An attach to a run's process within 3 s of a session on it ending now keeps the finished session until it succeeds, and a failure is logged instead of reported, its leftover session removed. Sessions record when they ended.
2026-11-05 07:45:32 +08:00
Daniel AdámekandDanielADK 2ba731f0ec feat(php): switch the PHP language level from the status bar (#1371)
With a PHP file focused, the status bar shows the PHP level Intelephense
checks against (`PHP 7.4`); the tooltip says where it comes from (the
settings, config.platform.php or require.php). Clicking it opens Auto (with
the composer.json version) and 5.6 to 8.5; picking one writes the same
project setting as Settings › Languages & Frameworks › PHP, so diagnostics
follow without reopening the project. The widget is hidden for other files.
For a file of a linked workspace project, the level comes from that
project's settings and the choices are disabled. The menu is also the
action "PHP Language Level".

Closes #1091

Co-authored-by: DanielADK <[email protected]>
2026-11-05 07:45:26 +08:00
devanddevadapter 742e504719 fix(terminal): keep IME commit gating after a composition (#1231)
Typing CJK punctuation such as `;` or `,` with fcitx5 in the Linux
AppImage sent each character twice: WebKitGTK commits it as
insertFromComposition + compositionend without compositionstart, so
xterm's CompositionHelper finalized the previous composition's stale
position and emitted a suffix. ImeCommitGate was meant to drop that second
emission, but compositionStart() disabled it for the rest of the session
and cancelled its rebase. The gate now works per commit: compositionstart
stands down only for its own composition, every keydown 229 starts a fresh
commit, and the textarea is rebased after each commit.

A keydown the IME did not take (Enter, an arrow, a letter outside the IME)
ends the open commit, so a key pressed right after a directly committed
punctuation mark is not dropped by the commit's timeout.

Closes #1230

Co-authored-by: devadapter <[email protected]>
2026-11-05 07:35:24 +08:00
l0ng-ai ebe059e00f fix(run): run a Python module that imports relatively as a module of its package (#1369)
Run or Debug from a Python file's context started it as a script
(`python src/tasks/cli.py`). A module of a package that imports relatively
then fails with "attempted relative import with no known parent package",
and the debugger stops on that exception.

A file with a relative import (`from .x import y`, `from . import x`) now
runs as `python -m <package>.<module>` from the folder its top package sits
in: the last `src` of its path, else the Python project's folder. A
package's `__main__.py` runs as the package. The project's interpreter is
kept (a relative venv path is adjusted to the new working directory), and
the debugger launches it with debugpy's `module`. Files without relative
imports run as scripts as before.

Fixes #1337
2026-11-05 07:30:13 +08:00
l0ng-ai e17e59ce7d fix(run): write Gradle init scripts on the host that runs the build (#1377)
The test-events and delegated-run init scripts were written to a path built from the desktop's temporary directory, which doesn't exist on a remote host. A new `run_gradle_init_script` command writes a script under the host's own temporary directory, named by its content, and returns the path Gradle reads it from.
2026-11-05 07:25:53 +08:00
l0ng-ai 8021a8ccca fix(debugger): evaluate console input as expressions under CodeLLDB (#1375)
CodeLLDB runs debug-console input as LLDB commands unless its console mode says otherwise, so Evaluate Expression, the Debug Console and breakpoint log messages failed with "'x' is not a valid command". Launch and attach requests now set `_adapterSettings.consoleMode` to `evaluate` unless the configuration picks a mode; LLDB commands still run with a leading backtick.
2026-11-05 07:06:56 +08:00
l0ng-ai 5a6a6f7e05 test(vcs): walk a fixture through edit, commit, branch, stash and a conflicting merge (#1360)
crates/ide-host/tests/vcs_flow.rs takes rust-app through a day with Git
via the ops the app's commands call: an unsaved edit marked in the
gutter against HEAD, Local Changes and the file's diff once saved, a
commit at the head of the log, a branch with its own change, checkout
back with the open file following, a stash pushed and popped, and a
merge that conflicts. Then the open file shows the conflict, the three
versions are offered, and Accept Theirs is committed as a merge.
2026-11-05 06:16:53 +08:00
l0ng-ai a7f0f8110b feat(php): built-in PHP live templates (#1372)
PHP files had no built-in live templates. A PHP group now ships with the
others (Settings › Editor › Live Templates, editable and switchable like
them):

- fore / forek: foreach over an iterable, as value or key => value, the
  caret starting on the iterable
- pubf, prof, prif and pubsf, prosf, prisf: public, protected or private
  method, static or not
- thr (throw new), eco (echo), inc / inco / rqr / rqro (include,
  include_once, require, require_once)

PHP variables are written `$$NAME$` in the bodies, so they expand with
their `$`.

Fixes #1234
2026-11-05 06:06:39 +08:00
l0ng-ai 9669e58398 feat(php): Intelephense stubs for required extensions, Additional stubs and Include paths (#1361)
Intelephense loads only its 68 default stub sets, so code using redis,
imagick, memcached, wordpress and the like showed "Undefined type", and
there was no way to point it at a platform's classes outside the project.

- Each ext-* in require or require-dev of the root composer.json adds its
  stub set, and Settings › PHP › Additional stubs adds others (wordpress).
  `stubs` replaces Intelephense's defaults, so the defaults are sent with
  the additions; nothing is sent when nothing is added.
- Settings › PHP › Include paths: folders (absolute, ~/ or relative to the
  project) sent as environment.includePaths.

Both apply on composer.json and settings changes without reopening the
project; Settings › Language Servers options still win.

Fixes #1232
Fixes #1233
2026-11-05 03:19:54 +08:00
l0ng-ai d939dca303 test(run): smoke-test running and testing each language's fixture (#1332)
* test(run): smoke-test running and testing each language's fixture

smoke/run.test.ts drives the frontend's run store against a real
openidea-server for TypeScript, Python, Rust, Go, Java and Kotlin: the
detected configurations, a run to its output, the tests with the
fixture's failing test switched on (counts, failure name, location and
message in the test tree), and Rerun Failed Tests running only that test.

`pnpm smoke` runs it; the Language smoke workflow runs it per language
after the code intelligence smoke.

* ci(language-smoke): let main's runs finish

Merges to main came faster than a run, so each one cancelled the run
before it: main never finished the smoke nor saved the build cache PRs
restore. Only a PR's runs are superseded now.
2026-11-05 03:02:01 +08:00
l0ng-ai 23378e0cc2 test(lsp): smoke-test parameter info, structure, auto-import, Optimize Imports and file moves (#1355)
The language smoke now also checks, per language and against the
server the app starts: parameter info in a call being typed, the
Structure view's symbols, completing a name that isn't imported yet
(the import comes with it, through the item's edits or its command),
Optimize Imports on a typed import, and moving a file updating the
files that reference it (where the server supports it).
2026-11-05 03:01:07 +08:00
l0ng-ai c83264ae75 fix(debugger): coloured Spring Boot output, Soft-Wrap and Scroll to End in the debug console (#1353)
Debugging a Spring Boot configuration showed its log uncoloured: Run adds
-Dspring.output.ansi.enabled=always (Spring only colours output for a
terminal, the process runs on pipes), the java-debug and Kotlin debug
launches built their VM options without it. They now add the same console
VM options as Run, unless the configuration sets the property itself.

The debug console also gets the Run console's Soft-Wrap toggle (default
from Settings › Console › Use soft wraps) and Scroll to End, which
follows new output, turns off when scrolling up and on again at the end.

Fixes #1262
2026-11-05 02:58:59 +08:00
l0ng-ai 2fc3535300 fix(editor): no lookbehind in the color preview's pattern (#1352)
The color preview's name pattern used a lookbehind, which WebKit before
16.4 cannot parse: on older macOS the module fails to load. The
preceding character of a name is now checked in code, with the same
result (`.red`, `$white`, `off-white` are not colors).
2026-11-05 02:52:44 +08:00
Łukasz Kosickiandkosaa d96ff8e881 feat(i18n): add Polish translations (#1344)
Polish joins the UI languages (Settings › Appearance › Language), with
the key catalog and the source-text catalog; pl / pl-PL system locales
resolve to it.

Co-authored-by: kosaa <[email protected]>
2026-11-05 02:51:25 +08:00
l0ng-ai 3b02e6899d feat(markdown): show YAML front matter as a table in the preview (#1351)
A Markdown file starting with YAML front matter (agent SKILL.md files,
static site pages) showed it in the preview as a thematic break and a run
of text. The preview now shows every top-level field in a key / value
table above the body; nested lists and mappings keep their YAML layout.
Blocks after it keep their source lines for scroll sync. Text between two
thematic breaks that isn't a mapping is rendered as before.

Fixes #1239
2026-11-05 02:48:51 +08:00
fuqiuluoandfuqiuluo 948e93e1a2 fix(search): Find / Replace in Files popup layout, footer and platform shortcuts (#1227)
The Find / Replace in Files popup gets the popup surface and spacing,
the match count and empty states ("Clear option(s)"), the footer with
"Open results in new tab", a resizable popup with a remembered results /
preview split, file mask suggestions from the project's file types
(new index_file_extensions), mnemonics and inline icons. Shortcuts are
spelled for the platform: Linux showed macOS glyphs in the popup, the
shared SearchField, the find bars and Search Everywhere. Replace keeps
its Preview diff.

Closes #1219

Co-authored-by: fuqiuluo <[email protected]>
2026-11-05 02:45:36 +08:00
l0ng-ai 5d19baf06b fix(ssh): report a failed backend upload instead of waiting forever (#1350)
The IDE backend is uploaded by streaming it into `gzip -dc > file` on the
host. When that command exits early (no gzip, a folder it cannot write, a
full disk), it stops reading: once the channel window (about 2 MB) is
used up, the next write waited for window space forever, so Remote
Development stayed on "Installing the IDE backend on the host…" at a few
percent and never showed the host's error. Each write, and closing the
input, now races the command's exit; an early exit ends the upload with
the command's status and stderr.

Refs #1330
2026-11-05 02:39:20 +08:00
l0ng-ai 89309e40ba fix(project): Open in Terminal from the editor opens the edited file's folder (#1348)
Open in Terminal and Reveal in Finder / Explorer invoked without a target
(a shortcut, Find Action) always used the Project view's selection, so in
the editor they opened the folder of whatever was selected there (the
project root after reopening a project) instead of the edited file's.
Without a target they now use the Project view's selection only while it
has focus, and the active editor's file otherwise.

Refs #1327
2026-11-05 02:35:25 +08:00
Shohruh Ismoilovandshohruh-genius 9f2e6b570f feat(vcs): correct Git log graph projection and expand history views (#1076)
Git Log could lose commits when filtering by several paths, draw wrong
ancestry after filtering or collapsing rows, and open the wrong
repository in multi-root views. The graph is now projected over the full
commit DAG and laid out in Bek order, so filtered and collapsed views keep
the real ancestry; each log view keeps its own filters and requests.

Commit metadata, messages and path changes are kept in a persistent index
(by object id, in the IDE's cache directory) with seekable paging, and the
log gains parent-aware merge comparisons, a lazy revision browser,
read-only logs of repositories outside the project (opened with command
execution disabled), commit bookmarks, configurable built-in columns and
batched commit details. The adapted IntelliJ algorithms are attributed in
THIRD_PARTY_LICENSES.md.

Closes #872

Co-authored-by: shohruh-genius <[email protected]>
2026-11-05 02:31:32 +08:00
l0ng-ai 2d20e5a203 fix(lsp): update TypeScript imports when files move (#1346)
vtsls has no workspace/willRenameFiles: after didRenameFiles it updates
imports itself, behind a prompt sent as window/showMessageRequest. The
IDE shows that request as a message without choices, so moving a
TypeScript or JavaScript file never updated the imports of the files
that use it.

For a tsserver-backed server without willRename filters, the IDE now
asks tsserver for the edits up front (getEditsForFileRename through
typescript.tsserverRequest). The edits join the move's preview and undo
like any server's. The server's own update is turned off
(updateImportsOnFileMove.enabled = "never"), so it no longer prompts.
2026-11-05 02:26:28 +08:00
l0ng-ai 7c60737b69 fix(lsp): apply code actions whose follow-up command no server provides (#1343)
vtsls offers Organize Imports as an edit plus
`_typescript.didOrganizeImports`, a command VS Code's TypeScript
extension runs itself. Optimize Imports in a TypeScript file asked the
servers to resolve and run that command and failed with "no language
server provides command" before applying the edit.

A command that follows a code action's edit and that no server provides
is now skipped: the edit is applied, and resolving the command asks
nothing. An action that is only a command still reports it can't run.
2026-11-05 02:19:25 +08:00
Hacbit 7e9acdc59b feat(unity): detect projects and preserve asset metadata (#1207)
Unity projects (a folder with Assets and ProjectSettings/ProjectVersion.txt)
are detected without a generated solution: Assets / Packages become source
roots, ProjectSettings resources, and Unity's cache and build folders are
excluded before they exist. A Unity pane in the Project view shows the
editor version and hides .meta files, generated C# project files and
caches (each can be shown again).

Operations on assets under Assets keep their .meta files consistent:
rename and move carry the metadata (same GUID), delete removes both, copy
generates new GUIDs and updates references inside the copied group, and
file-move undo and workspace edits include the metadata. Destination
metadata conflicts and links inside the project are rejected before
anything changes.

Co-authored-by: hacbit <[email protected]>
2026-11-05 02:16:06 +08:00
l0ng-ai 2013ddaac2 feat(editor): color preview swatches in the gutter (#1341)
Lines with a color literal show a swatch of the (first) color in the
gutter: `#rgb` / `#rrggbb` with or without alpha, `rgb()` / `rgba()` /
`hsl()` / `hsla()`, and CSS named colors in property values of style
sheets. Outside style sheets and markup only quoted hex values and color
functions count, so `#123` issue numbers and `#add` comments get none;
prose files get none at all.

Clicking a swatch in an editable file opens the system color picker; the
picked color replaces the literal in its own format (hex case and length,
the function's name and separators, the original alpha), as one undoable
edit per pick. Settings › Editor › General › Gutter Icons › Color preview
turns the swatches off.

Fixes #1206
2026-11-05 02:10:42 +08:00
l0ng-ai ce898ea0c6 feat(vcs): fill the commit message with the repository's commit template (#1340)
An empty commit message field now starts with the file `commit.template`
names (any git config level; `~/` and repository-relative paths resolve
as git resolves them), without its `#` comment lines, since commits keep
the message as written. It is filled when the repository opens or
changes and when a commit clears the field; a typed message is never
replaced, and a prepared merge / cherry-pick / revert message still
takes precedence.

Fixes #1285
2026-11-05 02:05:07 +08:00
l0ng-ai c219c81bd8 fix(run): report Gradle tests in the test tree as they run (#1339)
Gradle prints only failed tests, so the live test tree only filled in
after the run from the XML reports, and stayed empty when the test task
was UP-TO-DATE and wrote no new reports.

Gradle test runs now get an init script that reports each test's start,
output, failure (with the stack trace), skip and finish as TeamCity
service messages and keeps test tasks from being UP-TO-DATE. The console
hides those lines; the JVM parser turns them into `[class, method]`
results, named as in the XML reports, and drops Gradle's own repeated
`Class > test() FAILED` lines. Tests the run already reported keep their
output instead of getting the report's copy appended. The TeamCity
message parsing moves out of the PHPUnit parser so both share it.

Fixes #1268
2026-11-05 02:02:06 +08:00
l0ng-ai 70a7c6dd2c fix(run): run Python modules with relative imports with -m (#1338)
Run or Debug on a Python file started it as a script
(`python src/tasks/cli.py`), so a module of a package that imports
relatively (`from .models import TaskList`) always failed with
"attempted relative import with no known parent package".

A file with relative imports now runs as `python -m package.module`
from the directory above its top package (the enclosing directories
with `__init__.py`, read once and re-read when one is added or
removed), with a venv-relative interpreter resolved against the project
root. Files without relative imports still run as scripts.
2026-11-05 02:01:35 +08:00
pravdomil fd69664ca7 fix(editor): follow local Markdown anchor links (#1328)
Go to Declaration and modifier-click on a fragment-only Markdown or HTML link such as [artist::order](#Artist-order) said there was nothing to go to: the text-reference fallback rejected fragment-only targets, and Markdown fragment lookup checked headings only, never HTML id / name anchors. Fragment-only href targets are now recognized and resolve in the same file (the current buffer, so unsaved anchors count), and Markdown files fall back to id / name anchors after headings.

Co-authored-by: pravdomil <[email protected]>
2026-11-05 01:48:06 +08:00
l0ng-ai 11932a87af fix(editor): Copy Reference and Paste as Plain Text in the editor's Copy / Paste Special menu (#1333)
The editor's Copy / Paste Special submenu only had Copy Path and Paste from History, so a class's fully qualified name couldn't be copied from the context menu although Copy Reference exists as an action. It now lists Copy Reference, Paste as Plain Text and Paste from History; the file path stays in the editor tab's Copy Path/Reference menu.

Fixes #1261
2026-11-05 01:27:49 +08:00
Qingandqtvbwfn 19a245fa04 fix(run): drop a removed workspace member's sessions and Services groups (#1289)
Removing a project from the workspace left its traces in Services: its run sessions, temporary configurations, remembered runs and type groups that now said there were no configurations. While the same project stays open, the run feature now watches the workspace member list and, per removed member, stops and forgets its sessions, drops its temporary configurations, reloads the configurations, removes the added types left empty and clears its remembered runs.

Co-authored-by: qtvbwfn <[email protected]>
2026-11-05 01:04:17 +08:00
Qingandqtvbwfn 62b5f11ae3 feat(build): Maven Daemon (mvnd) support with a settings switch (#1288)
Settings › Build Tools › Maven gets Use Maven Daemon (mvnd) and a Maven Daemon home path. With it on, maven_command (the one launcher point for sync, goals and run configurations) resolves mvnd from the configured home, $MVND_HOME, PATH, Homebrew or SDKMAN instead of mvn / mvnw; it falls back to mvn, with a logged warning, when mvnd is missing or the project JDK is older than the Java 17 its daemon needs. Sync passes -T1 so the daemon's parallel build doesn't interleave dependency:tree output.

Co-authored-by: qtvbwfn <[email protected]>
2026-11-05 01:03:18 +08:00
Qingandqtvbwfn 19c2d40528 feat(run): session tab icons and a Debug button in the run panel (#1283)
A finished run session's tab keeps its configuration's own icon instead of the green run arrow (a failed exit still shows the error icon), and the run panel's toolbar gets a Debug button next to Rerun that starts the same configuration under the debugger (in Services, a debug session reruns its debugger launch).

Co-authored-by: qtvbwfn <[email protected]>
2026-11-05 00:57:54 +08:00
Qingandqtvbwfn 78a3768ec1 feat(ui): syntax-highlight the read-only source previews (#1284)
The read-only previews beside Problems, Show Usages and TODO results are syntax-coloured from one whole-file parse (capped for very large files), so a line inside a block comment or a multi-line string is coloured as such; the pane renders plain first and recolours when the parse lands. SourcePreview takes the colouring as a colorize prop (the editor's previewColors), keeping src/ui free of editor imports, and the tok-* colours live once in global.css instead of a copy per view, with the theme's comment style.

Co-authored-by: qtvbwfn <[email protected]>
2026-11-05 00:55:03 +08:00
ntbang0901 f4fa2b42f6 fix(agents): style tables, headings and quotes in agent messages (#1324)
Agent replies only styled paragraphs, lists and code, so tables, headings, quotes, rules, images and task lists fell back to browser defaults: a 26px h1, a quote indented 40px, table columns squeezed to a few letters by overflow-wrap. They now get chat-sized styles from tokens; wide tables scroll sideways.

Closes #1273

Co-authored-by: ntbang0901 <[email protected]>
2026-11-05 00:50:00 +08:00
ntbang0901 d0910d3313 fix(markdown): drop raw style and form markup from rendered Markdown (#1320)
Markdown from comments, hovers, agent replies and files renders into the IDE's own document, so a raw <style> restyled the whole IDE and <form> / style= reached the page. Every Markdown sanitizer (LSP hovers and comments, agent messages, Markdown preview) now forbids <style>, <form> and the style attribute; agent messages also drop data-* written in a reply, since their click handlers trust data-copy / data-path. align, width and <details> stay.

Closes #1319

Co-authored-by: ntbang0901 <[email protected]>
2026-11-05 00:49:11 +08:00
Luis Roberto c0a74244b4 feat(i18n): Portuguese (pt) UI language (#914)
Adds Português (pt-BR) to the UI languages: the message catalog (locales/pt.json) and the source-text catalog (locales/text/pt.json, ~6,000 entries). Brazilian and European tags (pt-BR, pt_PT) resolve to it.

Co-authored-by: Psyducks2 <[email protected]>
2026-11-05 00:34:08 +08:00
l0ng-ai 8d29704d46 fix(run): merge Gradle console results into the reported test classes (#1323)
Gradle's console names a test class by its simple name and a JUnit 5
method by its display name (`method()`), while its XML reports use the
qualified class and the method name. After a run the test tree showed
each class twice, counted the failure twice, and the report's copy of
the failing test had no source location.

Console results are now moved under the reported class (when the simple
name is unambiguous), Gradle method names drop the empty `()`, and the
report's stack frame is matched for Kotlin names with spaces and frames
behind a class loader prefix (`app//`).
2026-11-05 00:33:44 +08:00
l0ng-ai c399ad2598 fix(editor): keep gutter implementation markers across tab switches (#1321)
Switching tabs remounts the editor view and the marker cache lived in its plugin; focusing an editor dropped the cache too. So every visit re-ran an implementation search per declaration in view, and on jdtls each one walks the whole workspace (#879: java at ~40% CPU while moving between classes). Answers are now cached per file and dropped only when other files change (edits, or source changes on disk; build output ignored) or a language server settles, through one window-wide subscription whose echo guard survives remounts.

Fixes #879
2026-11-05 00:31:12 +08:00
Jaw c2dd466305 fix(search): search hidden and git-ignored files outside ignored folders (#1318)
Find in Files skipped dotfiles (.eslintrc, .github/workflows/*.yml) and git-ignored files (.env, *.log) because the ide-search walkers dropped every hidden or ignored entry, files as well as folders. A shared walker now lets the ignore rules pick only the folders to enter and lists each walked folder for its files, so ignored folders (target/, logs/) are still skipped while ignored files elsewhere and hidden files are searched, as Go to File does since #1313. Find in Files, Inspect Code / Locate Duplicates (scope_files) and LSP reference candidates (files_with_word) all use it; .git, symlinks, archives, files.exclude and excluded roots still apply.

Fixes #1317

Co-authored-by: jaw52 <[email protected]>
2026-11-05 00:24:02 +08:00
tomgorandtomgor c27fea13b3 fix(ui): highlight the menu row under the pointer (#1242)
Menu rows now follow the pointer without taking focus: focusing the row on every pointer enter dropped the hover paint in the Windows webview, so menus looked frozen. Keyboard navigation continues from the hovered row, and an item's onActive fires once per enter. Focus no longer scrolls the page when a menu row takes it.

On Solarized Dark the selection and hover colours were the popup and panel colour, so menu, tree, list and tab selection was invisible there; on Solarized Light selection matched the panels. Both themes get distinct selection and hover colours, and the editor keeps Solarized's own selection colour.

Co-authored-by: tomgor <[email protected]>
2026-11-05 00:16:32 +08:00
Lee Dong Geon 94490e2b37 feat(vcs): show the history of the selected lines (#1266)
Git › Show History for Selection, as in IDEA: from Git › Current File and the editor's Git menu it opens a tab with only the commits that changed the selected lines (or the caret line), newest first, and the block before and after each commit in the diff viewer. Only Show History for the whole file existed before. git log -L tracks the lines across renames; the selection is first mapped through unsaved and uncommitted edits to HEAD's lines, and those edits come first as Local Changes. The query is its own ide-vcs module and cancellable command (vcs_block_history), read-only and allowed in Restricted Mode (--no-ext-diff --no-textconv).

Co-authored-by: ldg030201 <[email protected]>
2026-11-05 00:07:44 +08:00
Daniel AdámekandDanielADK f3afe69ca2 feat(php): PHP language level for Intelephense (#1264)
Intelephense checked PHP code against its default 8.5, whatever version the project targets. Settings › Languages & Frameworks › PHP gets a PHP language level, Auto (default: composer.json config.platform.php, else the lowest version require.php allows) or 5.6–8.5, sent as environment.phpVersion; nothing is sent without a requirement. Intelephense applies the version only to documents it parses after reading its configuration, so servers that opt in (resync-on-configuration, Intelephense only) get their open documents again right after workspace/configuration is answered (new jsonrpc Handler::after_response), and Intelephense keeps one index per level because it fixes the built-in functions when it indexes. A change to a root or workspace member composer.json reconfigures the servers.

Co-authored-by: DanielADK <[email protected]>
2026-11-05 00:04:01 +08:00
Jaw 9a9266f61b fix(index): let Go to File find git-ignored files such as .env (#1313)
The index walker used the ignore crate, which drops ignored files before we see them, and the watcher dropped their events too, so git-ignored files such as .env were missing from the index and Go to File, the Files tab and @ mentions could not reach them in any scope. As in IDEA, ignored folders stay out of the project but ignored files stay in it: each walked folder is also listed and the files the walker skipped are indexed by name only (no read, hash or symbols), the ignored flag is stored (index schema 3) so a file whose rules flip is re-parsed or dropped on the next scan, and the watcher reports ignored files in a separate list that only the index and open editors use (Local History, deployment, LSP and fsChanged see the same events as before). Watcher updates classify files by listing their folder with the scan's walker, so nested, global and info/exclude rules apply the same way.

Co-authored-by: jaw52 <[email protected]>
2026-11-04 23:55:49 +08:00
Qingandqtvbwfn 040578ba1d test(host): keep the ignore fixtures clear of a machine global excludes (#1279)
The rpc integration test wrote *.tmp fixtures; a machine whose global excludes file lists *.tmp already ignores them, so the backend never writes the .gitignore / info/exclude entry and the test failed reading it. The fixtures now use a suffix no excludes file lists.

Co-authored-by: qtvbwfn <[email protected]>
2026-11-04 23:54:26 +08:00
l0ng-ai ad2aeed54e fix(agents): don't show a branch for a task in a folder that isn't a repository (#1316)
For a folder that isn't a repository, the current-branch query answers with a repository found below it, so the New Agent Task page read "<folder> on <branch>" although the agent runs in the folder itself and the branch couldn't be changed. The branch is now shown only when the project is a repository.
2026-11-04 23:39:50 +08:00
l0ng-ai 1e4eee27c4 test(lsp): smoke-test everyday code intelligence per language with real servers (#1299)
* test(lsp): smoke-test everyday code intelligence per language with real servers

A new ide-host integration test drives completion, go to definition, Find
Usages, hover, diagnostics with a quick fix, an intention, formatting and
rename on each language's fixtures/ project, through the ops the app's
commands call and the language server the app starts by default:
TypeScript (vtsls), Python (basedpyright + Ruff), Rust (rust-analyzer),
Go (gopls), Java (jdtls) and Kotlin (kotlin-lsp). A failure names the
language and the operation.

The tests are ignored by default; a Language smoke workflow installs pinned
server versions and runs one language per job on changes to crates/,
languages/ and fixtures/. Adds fixtures/java-app (Maven, JDK 21) for Java.

* test(lsp): rename TypeScript from a usage and check Rust errors while typing

Covers the fixes the smoke test turned up: a TypeScript rename started at a
call site in another file renames the declaration (#1301), and rust-analyzer's
own diagnostics and fixes arrive without saving (#1306).
2026-11-04 23:35:53 +08:00
l0ng-ai 1789dbb98e fix(tools): run Kotlin LSP's launcher, not its deprecated wrapper script (#1310)
Current Kotlin LSP releases launch with bin/intellij-server; the
kotlin-lsp.sh / kotlin-lsp.cmd wrappers the installer picked now print a
deprecation warning and are announced to go away, which would leave a
fresh install without a server. The installer prefers the launcher and
still finds the wrappers in older releases.
2026-11-04 23:33:11 +08:00
l0ng-ai 20c439baaa fix(lsp): pull diagnostics from servers that provide them (#1307)
rust-analyzer reports its own analysis (a missing field, a wrong type) only
to clients that pull diagnostics; it pushes just `cargo check`'s, so in the
editor a Rust type error showed up only after saving. The client now
advertises `textDocument/diagnostic` and pulls an open file's diagnostics
from servers with a `diagnosticProvider`: when it opens or its server
starts, after edits (once typing pauses), on save (and other open files
for servers with inter-file dependencies) and on
`workspace/diagnostic/refresh`. Pulled and pushed diagnostics are kept
apart and shown, and given to code actions, together; a newer pull
supersedes an older reply, a server-cancelled pull is retried, and closing
a file drops what was pulled.
2026-11-04 23:33:04 +08:00
l0ng-ai a0dfb1e741 fix(lsp): rename TypeScript symbols everywhere instead of aliasing the import (#1302)
Renaming an imported function at a call site renamed only the local binding,
leaving `import { formatMoney as formatPrice }` and the declaration as it
was: both TypeScript servers default to aliases on rename. vtsls gets
`typescript.preferences.useAliasesForRenames = false` (and the JavaScript
one), typescript-language-server `providePrefixAndSuffixTextForRename = false`,
in every language that starts them.
2026-11-04 23:32:58 +08:00
l0ng-ai 4906bcb2a1 feat(vcs): Delete… for unversioned files in the Changes view (#1314)
Unversioned files in the commit view could be added or ignored but not deleted. Their context menu now offers Delete…, as IDEA's Changes view does: it asks first and moves the files to the Trash, only when every selected file is unversioned.
2026-11-04 23:32:56 +08:00
l0ng-ai f35c781f42 fix(refactor): apply server-versioned edits to files that aren't open (#1295)
gopls sends a document version for every file of a rename, including files
the editor never opened. Applying a workspace edit rejected any versioned
file without an open document as changed, so a Go rename touching a closed
file always failed with "changed; request the refactoring again". For a file
that isn't open, the preconditions captured with the edit now decide; an
edit without them is still rejected.
2026-11-04 23:32:51 +08:00
l0ng-ai 6f9819fee7 fix(vcs): Show History on a submodule folder shows the submodule's history (#1312)
Show History picked its repository with repoOf, which routes a nested repository's own root to the enclosing repository (right for the commit view's submodule pointer), so a submodule folder listed its gitlink history in the parent repository. A repository's own root now opens that repository's history, as in IDEA; other paths still use the innermost repository.
2026-11-04 23:29:44 +08:00
l0ng-ai 834b8d4e3b fix(database): keep line breaks when editing multi-line cells (#1311)
The inline cell editor is a single-line input that drops a value's line breaks on the first change, and it committed on every Enter; the Record view's fields had the same problem. Multi-line values now open in the multi-line editor, Shift+Enter opens it for any cell (from the grid or the inline editor) and Cmd/Ctrl+Enter applies it; CRLF values keep CRLF; the grid marks line breaks with a muted ↵; the Record view uses a text area for multi-line values.
2026-11-04 23:26:24 +08:00
xtr-hubandxtr-hub 7cdf690b4b fix(window): stop project tab switches from moving the group window (#1287)
Merged project windows are separate native windows that take turns being visible. Hidden members kept their own earlier frame and monitor, so the first switch after merging, and any switch after moving or resizing the window, moved or resized the window on screen; a maximized member could appear on another display. The group now copies the selected window's frame and display to the hidden members after a selection and after the visible window is moved or resized (debounced), skips copies when the frame already matches, and leaves macOS simple fullscreen to its own path, so switching projects only changes which window is visible.

Co-authored-by: xtr-hub <[email protected]>
2026-11-04 23:25:01 +08:00
Jaw 057877a39e fix(search): stop outer .gitignore rules at nested repositories in the remaining walkers (#1304)
#1309 made the Find in Files, file scope, index and walk_files walkers require git inside a repository, so an outer repository's .gitignore stops at a nested repository. External TypeScript project discovery (ide-lsp) and Lombok detection (ide-sdk) still walked with require_git(false) and could miss a nested repository's sources the same way; they now use the same check. Adds tests for walk_files (nested repository with its own .gitignore and info/exclude, a plain folder inside the outer repository, the outer repository, a folder outside any repository) and for scope_files.

Co-authored-by: jaw52 <[email protected]>
2026-11-04 23:20:38 +08:00
l0ng-ai a08707dabd fix(search): stop an outer repository's .gitignore at a nested repository (#1309)
The walkers used require_git(false), and with it the ignore crate never treats a .git directory as a boundary, so an outer repository's .gitignore hid a project that is its own repository inside an ignored folder (Find in Files found nothing). Walkers now require git when the walked folder is inside a repository, so rules stop at a nested repository's root as in git; folders outside any repository still honour their .gitignore. Covers the Find in Files engine, the file scope, the index walk and ide-vfs walk_files.
2026-11-04 23:16:47 +08:00
l0ng-ai 092a700ddc fix(vcs): keep the current branch name in New Branch even though it exists (#1305)
New Branch dropped its suggestion when validateBranchName rejected it, and the current branch always exists, so a plain New Branch from HEAD always opened empty; the test never seeded the local branches. The suggestion is kept, the dialog shows that the branch exists from the start and Create stays disabled until the name changes; other prefilled prompts (Rename Branch) keep showing errors only after editing.
2026-11-04 23:12:40 +08:00
Jaw b31edc3482 fix(editor): give each file its own find bar, query and caret (#1296)
The find bar belonged to the pane and every bar shared one query, and its effect re-ran the search and moved the caret whenever the active tab changed, so switching files dragged the previous file's search into the next one and lost its caret. Each tab now has its own find bar with its own query and options, as with IDEA's per-editor search session. The last search only pre-fills a newly opened bar (or the selection does) and drives Find Next without a bar; showing a bar again only highlights, and the caret moves only when the query or an option changes in that bar. Bars of closed tabs are dropped.

Co-authored-by: jaw52 <[email protected]>
2026-11-04 23:11:05 +08:00
ntbang0901 d032addb17 fix(run): offer Run for classes whose superclass declares main (#1204)
The JVM run-target scanner only reads the file's own text, so a class like `class App extends AtomStarter {}`, whose superclass (in the project or a library JAR) declares `static main`, got no gutter Run, no Ctrl+Shift+R and no file-level Run, although the JVM launches it. Classes with an `extends` clause and no main of their own are now candidates: the superclass chain is walked through the language server's type hierarchy, each class supertype's source (file, decompiled or source-jar document) is checked for a static main, and the answer is cached per class and `extends` clause, so the gutter, Run from context, launch and debug agree. Interfaces are skipped (a static interface main is not inherited); negative answers are asked again when the server settles. `main(java.lang.String[] a)` is accepted as well.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-04 23:11:01 +08:00
ntbang0901 8f77d7077c fix(editor): keep tooltips full width at the editor's right edge (#1203)
On WebKit, CodeMirror positions tooltips absolutely inside the editor, but the tooltip space still reported the whole window as the right bound. CodeMirror never shifted a tooltip near the editor's right edge to the left, the box was squeezed into the space left of that edge, and the next measurement read the squeezed width, so it could end as a one-character column. The tooltip space is now the editor's own horizontal extent (clamped to the window) and tooltips size to their content, which fixes hover, inline blame, debugger values, signature help and hints together.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-04 23:10:51 +08:00
ntbang0901 bdc108a61b fix(editor): show pinned tabs in their own row when Multiple rows is on (#1198)
"Show pinned tabs in a separate row" was ignored while "Multiple rows" was on, because the pinned row explicitly excluded horizontal layouts with multiple rows. The two options are independent, so the pinned row now shows whenever some but not all tabs are pinned, and each row wraps on its own. With the rows split, dropping a tab on the pinned row pins it and keeps it in the pinned group, and dropping it on the other row unpins it.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-04 23:10:28 +08:00
ppxbandppxb a363544625 fix(ui): keep the arrow cursor on scrollbars (#1191)
Scroll containers set cursor: text, and the webview applies an element's cursor to its scrollbar, so scrollbars showed the I-beam. The containers now keep the arrow and the text inside them gets the I-beam: the editor (.cm-content instead of .cm-scroller, which also covers the diff and merge editors), the source preview, the Run, Build and Debugger consoles, the Git console, the Database output and the Services log.

Co-authored-by: ppxb <[email protected]>
2026-11-04 23:10:23 +08:00
nguyenmuoi157 f942ea168d fix(editor): allow text selection and input in editable elements (#1209)
The workbench sets user-select: none on body and nothing restored it for editable elements, so a contenteditable such as CodeMirror's .cm-content inherited it. In WebKit a contenteditable with user-select: none cannot hold a selection, so typing, paste and drag-and-drop into the editor could be dropped. input, textarea and [contenteditable="true"] now get user-select: text next to the cursor: text rule they share; the selection lock still overrides it.

Co-authored-by: nguyenmuoi157 <[email protected]>
2026-11-04 23:10:08 +08:00
l0ng-ai 59f1a1e6db docs: remove the Where it's going section from the README 2026-11-04 23:06:36 +08:00
l0ng-ai a4a16d0854 fix(plugins): restore the hello fixture's source fingerprint
The recorded fingerprint did not match the fixture's sources, so
plugins/build.sh --check reported the fixture as stale.
2026-11-04 22:56:08 +08:00
l0ng-ai b440370f2d fix(workbench): focus a tool window's content once it loads (#1280)
Activating a tool window focuses its `[data-autofocus]` element or first
content field one frame later, else its body. A lazily loaded view such as
the Terminal renders its field after that, so the first ⌥F12 left focus on
the body. When no target exists yet, the tool window now watches its body
and focuses the target when it appears, unless focus has moved on.
2026-11-04 22:52:53 +08:00
l0ng-ai aa33d2eeab test(e2e): cover the core editing, code intelligence, VCS, run and debug flows (#1277)
Adds a `stubDesktop` fixture that switches the browser-mode workbench to
desktop mode over a stubbed IPC layer: tests answer the commands they need,
push Channel messages and IdeEvents, and assert on the commands invoked.
Unstubbed commands fail as unavailable; documents keep the in-memory backend.

New flows covered:
- editing: modified marker and ⌘S save, ⌘R Replace All with one undo step
- code intelligence: completion, ⌘B to another file, Find Usages, in-place
  rename, diagnostics with an ⌥⏎ quick fix
- VCS: ⌘K commit with message validation and default inclusion
- run: ⌃R output streaming and exit, ⌘F2 stop
- debug: breakpoint, stop with frames and variables, step over, resume
- search: Find in Files results opening at the match line; the Search
  Everywhere file case no longer needs fixme
- terminal: typed command reaches the shell and its output renders
- project: Close Project to Welcome, trust prompt, open a recent project
2026-11-04 22:52:48 +08:00
l0ng-ai b26318a32c fix(plugins): record the hello fixture's source fingerprint 2026-11-04 22:42:24 +08:00
l0ng-ai 8a2e370178 docs: drop the priority note from the README direction section 2026-11-04 22:41:15 +08:00
l0ng-ai 61a05e5a36 docs: add where the project is going to the README 2026-11-04 22:40:15 +08:00
l0ng-ai d9355155f3 fix(plugins): rebuild the hello fixture after its source comment changed 2026-11-04 22:39:06 +08:00
l0ng-ai ecd593eb0c ci: stop requiring a linked issue on PRs
The PR check now only asks for the How it was verified section. Linking an
issue with Closes #<n> stays recommended when a PR resolves one.
2026-11-04 22:19:31 +08:00
l0ng-ai 0716a48247 chore: v0.1.36 2026-11-04 22:16:25 +08:00
l0ng-ai 699c71c1f2 docs: introduce plugins in the README 2026-11-04 22:14:16 +08:00
l0ng-ai 014f51ae54 chore: move the JDT LS components out of plugins/ (#1292)
java-usages, kotlin-bridge and the patched Lombok agent are JDT LS bundles and a Java agent embedded into the app, not OpenIDEA plugins. Move them to java/ so plugins/ holds only plugins, and update the paths that embed or mention them.

Also fix the stale build path in the hello fixture and the contributor guide's layout (there is no templates/ directory; starter templates live in the openidea-plugin CLI).
2026-11-04 22:11:38 +08:00
l0ng-ai 30b0549196 fix(editor): don't autosave while focus is in a popup over the editor
Opening a popup (intentions, completion) moved focus out of the editor and
autosaved it. The save cleanup could change the text, such as trailing
spaces on another line, so the quick fix the popup listed no longer matched
and the first attempt to apply it failed.

While focus is in a popup the editor is not saved; it is saved when focus
moves on from the popup to anything but the editor.
2026-11-04 21:57:59 +08:00
l0ng-ai 0ecbc6b35d fix(workspace): keep a previewed edit valid when its file is saved
An edit's preconditions hashed disk contents plus unsaved buffers. Saving a
buffer moved its text from one to the other and changed the hash, so a
quick fix, rename or refactoring listed just before a save (autosave of a
freshly typed scratch, for one) failed with "changed; request the
refactoring again", and only the next attempt worked.

Unsaved buffers now stand in for their files' bytes in the hash, so saving
doesn't change it; edits, external writes and file operations still do.
2026-11-04 21:40:12 +08:00
l0ng-ai 859f63454e fix(plugins): move the repository URL to the gear menu and drop the no-repository hint (#1282)
The Available tab always showed a "There is no online plugin repository"
notice under the search field when no index was set. Release builds use
the official index by default, so the notice only appeared in development
builds, where the empty URL field above it already said the same thing.
The URL field itself sat at the top of the list although most people never
change it.

- The notice is gone, and the search placeholder reads "Search plugins"
  because the list holds marketplace plugins as well as local ones.
- With a repository configured, the empty details panel says "No plugin
  selected" instead of claiming there is no online repository.
- The marketplace index URL is edited from the gear menu's "Plugin
  Repository…" dialog; leaving it empty restores the default.
2026-11-04 21:38:59 +08:00
l0ng-ai c1ea7a9ef2 test(plugins): build the command template with the SDK's lockfile
The resolver also looks up the SDK's wasm-only dependencies, which a clean
cargo cache doesn't have, so the offline build failed in CI.
2026-11-04 21:22:40 +08:00
l0ng-ai 3100aa2c00 feat(scratch): start new scratch files from a language template
A new Java scratch was an empty file that couldn't run until a class and
main method were typed in. Languages can now define `scratch-template` in
`languages/<id>.toml`; a new scratch gets that content with the caret at
`$END$`.

- Java, Kotlin, Go, Rust, C and C++ get a runnable `main`.
- Other languages still start empty.
2026-11-04 21:09:12 +08:00
l0ng-ai 52360c72d2 fix(editor): describe a Java file outside every project accurately
jdtls still completes and navigates JDK classes in a non-project file such
as a scratch; it only skips semantic checks and project classes. The banner
said navigation and completion don't work, and in a Maven or Gradle project
it offered a build reload that can't bring a file outside the project in.

- The banner now says only syntax errors are reported and only JDK classes
  resolve.
- The Maven / Gradle reload is offered only for files inside the open
  project (or one of its workspace projects).
2026-11-04 21:09:12 +08:00
l0ng-ai 7fd330912a fix(process): wait for a killed tree's descendants to exit on Windows (#1272)
`taskkill /T /F` only starts terminating the tree, and the caller then
waits for its direct child. A grandchild could still be exiting, with its
files open, when `kill_tree` returned or `Peer::exited` resolved. The
descendants are now opened before the kill and waited for (up to 5 s)
after it. A process counts as a descendant only if it started after its
parent, so a reused parent id doesn't pull in unrelated processes.

Closes #1270
2026-11-04 21:07:08 +08:00
l0ng-ai eaca68f263 test: retry first edit-path calls and wait for effects in flaky tests (#1271)
A provider call that starts a plugin instance (or reads files) can run past
the edit-path soft deadline on a slow runner and returns nothing; the tests
now ask again, as the editor does. The AI completion test cancels the slow
request only once the server has it, and measures from the cancel. The
SourcePreview test waits for the effect that marks the selection scope.

Closes #1269
2026-11-04 21:07:03 +08:00
l0ng-ai 4b5ef7069c fix(plugins): build new plugins outside the repository
- `openidea-plugin new` makes a Rust plugin depend on the SDK at the tool's
  release tag (git) unless the plugin is inside the source tree the tool was
  built from or `--sdk` is given; the path written before only resolved on
  the machine that built the tool.
- The command template compiles again: `run_command` returns
  `openidea_plugin::Error`. A test now runs `cargo test` on the generated
  plugin against the SDK.
- plugins/README.md: installing the tool with `cargo install --git`, the SDK
  dependency, where plugins can be installed (welcome screen and trusted
  projects), and where to look when a plugin misbehaves.
2026-11-04 20:55:31 +08:00
l0ng-ai 2cd16a723d fix(lsp): let language servers register the files they watch (#1223)
* fix(lsp): resolve newly installed Python packages without a restart

Packages installed into an excluded .venv (uv add, uv pip install) never
reached the Python language servers: the watcher reports changes in excluded
folders only as excluded_dirs, and didChangeWatchedFiles forwarded nothing but
the server's own source files. basedpyright kept its failed import resolution
until restarted.

Language servers get a watched-files list of globs. Files matching it, and
excluded folders matching it, are forwarded too; the Python servers watch
their configuration and site-packages.

* fix(lsp): let servers register the files they watch

Clients now support dynamic registration of workspace/didChangeWatchedFiles,
with relative patterns. Changes on disk go to each server as its registered
watchers match them, including folders in excluded trees (a package installed
into .venv, go.mod rewritten by go get). Folders outside the project that
relative patterns name (a virtualenv elsewhere, a path dependency) get their
own watchers. Servers that register nothing keep hearing of their languages'
files.

Registrations sent right after initialized are no longer dropped while the
server is still being marked ready.

Replaces the per-server watched-files list.
2026-11-04 20:44:24 +08:00
l0ng-ai 71c0515af6 fix(ui): no focus ring on controls that were only clicked (#1213)
The webview focuses a clicked button and the focus stays there: across
project window-tab switches, and popups hand it back to their trigger. The
next keystroke switches the engine to keyboard modality, so that button got a
focus ring (window tabs, the title-bar project switcher). A pressed control is
now marked and keeps no ring until Tab or an arrow key moves focus onto it.
Focus itself is unchanged, since actions resolve their context from it.

Closes #1211
2026-11-04 20:44:19 +08:00
l0ng-ai b473876d59 feat(agents): reference folders from @ completion (#1210)
Folders now appear in the composer's `@` completion next to files, from a new
`find_dirs` index query over the folders that hold indexed files (kept
incrementally in the index, ~0.5 ms on 100k files). Enter or a click
references the folder as `@dir/` and sends it as a resource link; Tab keeps
completing inside it.

Closes #1202
2026-11-04 20:44:14 +08:00
l0ng-ai d7886bab45 test(plugins): repeat the cancel until the call is running in the cancel test (#1263)
Closes #1248
2026-11-04 20:34:33 +08:00
l0ng-ai dd370654cb feat(plugins): leave room in plugin API 1.0 for what will grow (#1245)
Closes #1237
2026-11-04 20:11:46 +08:00
l0ng-ai e3c7f362c4 fix(plugins): list the marketplace and install plugins from the welcome screen (#1256)
Closes #1255
2026-11-04 20:09:59 +08:00
l0ng-ai 02addfa2c5 test(plugins): let the first call in the deadline test start its instance (#1257)
Closes #1248
2026-11-04 20:09:44 +08:00
l0ng-ai 82fe52f595 fix(agents): wait for the agent to exit before deleting its worktree (#1240)
Closes #969

## What and why

On Windows, deleting an agent task and its worktree could fail with `os error 32` while the agent's process tree still held the directory or its files open. Deletion now waits up to 5 seconds for the agent process to be killed and reaped, then retries folder removal for transient locks. This also covers persistent ACP agents such as CodeBuddy that cannot reload their session and therefore stay alive while idle.

## How it was verified

- Windows desktop acceptance through the Tauri MCP bridge, using a deterministic ACP agent with a real PowerShell descendant holding a file open. Closed the conversation tab, then deleted the task with worktree removal enabled. A separate helper retained another file lock for 1.2 seconds: the pre-fix implementation showed `os error 32`; the fixed implementation deleted the task and directory, with both agent processes gone.
- Also verified direct deletion, archive then delete, and deletion with worktree removal unchecked. Processes exited in all cases; branches were preserved, and unchecked removal preserved the folder.
- `cargo test -p ide-jsonrpc -p ide-agent -- --test-threads=2`: 155 passed, 5 real-agent tests ignored. The test runner used Git Bash on PATH and a process-local `core.autocrlf=false`, matching CI's LF fixture setup; no global Git settings were changed.
- `cargo clippy -p ide-jsonrpc -p ide-agent --all-targets -- -D warnings`, `cargo fmt --all --check`, and `git diff --check`: passed.
- Full workspace (Windows/macOS/Linux), frontend and E2E CI: [final PR run](https://github.com/l0ng-ai/openidea/actions/runs/41819945536).
- Windows regression coverage now checks deletion while waiting for permission and while idle without `session/load`, actual sharing violations, delayed file-lock release, and `Peer::exited()` waiting for a descendant's file handle to close. Existing Unix process tests remain enabled.
- Evidence (local, not committed):
  - Before: `C:/Users/thoma/repo/pr1240-evidence-20261104/before.png`
  - After: `C:/Users/thoma/repo/pr1240-evidence-20261104/after.png`
  - Recording: `C:/Users/thoma/repo/pr1240-evidence-20261104/after.gif`
  - Runtime results: `C:/Users/thoma/repo/pr1240-evidence-20261104/runtime-cases.json`
- Real signed-in CodeBuddy and native Claude/Codex were not exercised. Native Claude/Codex use an in-process connection, so their `exited()` resolves immediately and removal relies on the bounded retry. Shared-worktree fork protection remains unchanged.
2026-11-04 19:39:40 +08:00
l0ng-ai d94a117d69 test(plugins): don't time the plugin host tests against the runner's speed (#1250)
Closes #1248
2026-11-04 18:46:52 +08:00
Marius Ciepluch b6b0ce46c3 fix(fonts): enumerate all faces and per-user fonts on Windows (#908)
Enumerate every locally installed Windows font face, including per-user font files. Correct glyph-width probing and collection-file enumeration, and exclude cloud fonts that are not installed.

Closes #1241
2026-11-04 18:46:23 +08:00
l0ng-ai 953e844541 feat(plugins): community plugins, version compatibility and categories in the marketplace (#1226)
* feat(plugins): list community plugins from a registry in the marketplace

The marketplace index can now carry plugins of other publishers, published
from their authors' repositories through the l0ng-ai/openidea-plugins
registry.

- `openidea-plugin index` with a key replaces only that publisher's plugins
  (it used to drop every other entry); without a key it indexes each package
  against the publishers the index already lists, checking the key and the
  id prefix instead of only the signature itself.
- `openidea-plugin merge` takes another index's publishers, plugins and
  revocations in place of every publisher but `--keep`; the other index can't
  list or revoke the kept publishers' plugins.
- `openidea-plugin check` accepts a `.oiplugin` file.
- Releases merge the registry's index into plugins-index.json and attach the
  Linux x64 tool; plugins-index.yml refreshes the community plugins in the
  latest release hourly or when the registry signals a change.
- plugins/README.md and CONTRIBUTING.md describe official and community
  plugins.

* feat(plugins): offer the newest plugin version this IDE can load

Index entries record the plugin API each version needs (`engine`: the
manifest's `engines.openidea` narrowed to the API its component was built
for), and an index may list several versions of a plugin. Settings › Plugins
offers the newest version this IDE can load; with none, the plugin is shown
as incompatible with the API it needs, and downloading it is refused.

`openidea-plugin index` keeps, of each plugin, the newest version per API
requirement, so users of an older IDE keep the last version built for them;
`--download-base` makes downloads absolute, and the release workflow points
them at the release that published them. Scratch directories are now unique
per call.

* feat(plugins): file marketplace plugins under categories

`plugin.toml` takes `categories` (up to 3 of languages, themes, icon-themes,
linters, vcs, ai, frameworks, tools, other). A plugin that declares none is
filed under what its contributions imply (themes, icon themes, languages),
else under other; categories an IDE doesn't know are ignored, and `check`
warns about them. Index entries carry the categories, and Settings › Plugins
› Available filters by them and shows them in a plugin's details. The
official plugins declare theirs.

* feat(plugins): let a marketplace index point to where it moved

An index with `moved` sends the IDE to the index at that URL (relative or
absolute, up to three hops; an https index can only move to https). Downloads
resolve against the index actually read, so a marketplace can change hosts
while released IDEs keep the address they were built with.

* fix(plugins): full-width category filter and aligned incompatibility note

* fix(plugins): translate the AI category in Chinese

* feat(plugins): keep a plugin's signer across updates and announce updates

- Index format: fields are only added; an IDE reading a newer `format`
  still applies its revocations and `moved`, lists no plugins and asks to
  be updated, instead of failing to read the index.
- Key continuity: an installed plugin updates silently only to a package
  signed with the same key, or a key it endorsed (`KeyEntry.endorsement`,
  `index --previous-key`, `openidea-plugin endorse`). Another signer, or an
  unsigned package replacing a signed one, is shown on the install page and
  needs the user's confirmation (`Consent.signerChange`).
- Official key: release builds carry the official publisher's public key
  (`OPENIDEA_OFFICIAL_PLUGIN_KEYS`, derived from PLUGIN_SIGNING_KEY);
  `openidea.*` plugins must be signed with it or a key it endorsed,
  whatever the index lists.
- Updates: the daily marketplace check announces new versions once, with
  Update All (also in the gear menu and as an action). An update that asks
  for nothing new installs with the grants the installed version has.
- Index entries need `engine`; `categories` beyond the first three are
  ignored instead of rejecting the manifest.
2026-11-04 18:24:43 +08:00
l0ng-ai 48e23c012f test(plugins): time only the restart wait in the crash backoff test
The waits included the restart that ended them, and instantiating the plugin can take a few hundred milliseconds on CI, so a slow second restart made the third wait look no longer. Measure until the call that is let through and require the third wait to exceed the second by half the base backoff.
2026-11-04 17:26:25 +08:00
l0ng-ai 0bc0396f80 test(sdk): only use a subst drive that shows the test's own directory
The two mapped-drive tests run in parallel processes and could race for the same free drive letter, then see the other test's directory. Mark the target and accept a drive only when the marker shows through it.
2026-11-04 16:57:30 +08:00
l0ng-ai d20be9c5cc fix(window): declare the detach fullscreen flag only on macOS
It is read only in macOS code, so Linux and Windows builds failed on an unused variable.
2026-11-04 16:57:28 +08:00
l0ng-ai 2d3b7ebf54 fix(window): keep a fullscreen project window group in one Space on macOS (#1218)
Native fullscreen gives every window its own Space, so switching projects in a fullscreen window group left one Space and entered another: the desktop showed in between, and quick switching could drop out of fullscreen.

A grouped window now uses simple fullscreen for every member at once; switching projects only shows one window and hides the others. The green button, the system Enter Full Screen item and Toggle Full Screen take this path for grouped windows, which keep their window buttons; ungrouped windows keep native fullscreen. Menu bar and Dock auto-hide while any window is in simple fullscreen, and detaching or closing a tab keeps the rest of the group fullscreen.
2026-11-04 16:35:43 +08:00
Lee Dong Geon 6dec1252db feat(plugins): import IntelliJ theme plugins (#1184)
Settings › Plugins (gear menu) gets Import Theme from IntelliJ IDE… (theme plugins of the JetBrains IDEs on this machine, the one each IDE uses first) and Import IntelliJ Theme from File… (a Marketplace .jar / .zip or a .theme.json). The host reads the plugin with size limits; the UI converts it with the IntelliJ mapping (moved from scripts/theme to src/features/plugins/intellijTheme.ts, still used by pnpm theme:import-intellij) into one OpenIDEA theme plugin (engines.openidea ^1.0) with all of its themes, installed through the usual install page, and applies the chosen theme.

Theme pickers separate each plugin's themes, and Quick Switch Scheme › Theme previews the theme under the pointer or keyboard after 500 ms and restores the selected one when closed without a choice.

Co-authored-by: ldg030201 <[email protected]>
2026-11-04 15:28:00 +08:00
l0ng-ai a610982655 fix(debugger): keep multi-line values on their row in the variables tree (#1196)
Multi-line value renderings (a pandas Series) were drawn with white-space: pre in a one-line row and spilled over the rows below. VariablesView rows now join them onto one line like inline values; this covers Variables, Watches, value hints and Evaluate results. Copy/Set Value keep the full text.
2026-11-04 14:59:03 +08:00
Park Jong Hyun ed145e3046 feat(refactor): Java Pull Members Up / Push Members Down (#1183)
Refactor | Pull Members Up… and Push Members Down… for Java, from the Refactor menu, the editor context menu and Refactor This. A dialog picks the destination and the members (Make abstract / Keep abstract); problems are confirmed before anything changes; the edit is applied as one change or previewed first.

JDT LS offers neither as a code action, so the app-owned JDT LS bundle (plugins/java-usages 1.3.0) adds two delegate commands that run JDT's PullUpRefactoringProcessor / PushDownRefactoringProcessor and return the problems and the edit without applying it, exposed as lsp_hierarchy_members / lsp_hierarchy_refactor (also over the remote host dispatcher). The bundled jar was rebuilt from source with JDT LS 1.63.0 and matches the recorded SHA-256.

Co-authored-by: fhrmrk <[email protected]>
2026-11-04 14:55:42 +08:00
l0ng-ai 5b46844f88 fix(process): give child processes the login shell's variables the IDE lacks (#1195)
Finder/Dock-launched apps only got PATH from the login shell; children outside run configs, LSP and DAP missed rc exports such as GOMODCACHE, so go list showed only some modules. host_env now adds the shell's variables the process lacks (call-site values win); the login-shell probe leaves them out so Reload Environment stays accurate. Covers every crate that spawns through ide_core::process::prepare/host_env.
2026-11-04 14:38:34 +08:00
Sergei Melnikov c87d2a6b55 feat(plugins): let icon themes match by pattern, replace built-in icons and describe their settings page (#1177)
Icon themes could only map files by exact name or extension and folders by name. Themes can now match by regular expression or path (patterns that could backtrack badly, lookaround, inline flags and named groups are refused at install), ship dark variants, replace built-in UI and symbol (PSI) icons, and offer packs (groups) and variants (choices). A theme describes its own options page in Settings › Plugins as sections of items naming options the IDE implements (hide file/folder/UI/PSI icons, hollow folders, accent, monochrome, saturation, opacity, icon size, row height); values are stored per theme in appearance.fileIconThemeOptions. Existing themes are unchanged.

Closes #1096

Co-authored-by: pdasilem <[email protected]>
2026-11-04 14:22:38 +08:00
l0ng-ai 4fbe41194f feat(gradle): analyze a Gradle project's dependencies per configuration (#1181)
The Dependency Analyzer lists Gradle projects next to Maven modules. For a Gradle project it runs `<project>:dependencies --configuration <name>` (compile/runtime/test classpaths) in the background (on the project host for remote projects), cancellable and cached per session, and reuses the list/tree views, search, conflicts-only filter and usage paths: an exact request Gradle replaced (1.0 -> 1.1) is a conflict showing the requested version, constraints are left out, unresolved dependencies are marked and build projects (quoted or not) are recognised. Exclude / Jump to Source stay Maven-only. Opened from the Gradle tool window toolbar, a Gradle project's context menu and a build script's menus.

Closes #1005
2026-11-04 13:51:40 +08:00
l0ng-ai 16df2ae472 fix(run): remove temporary configurations from Edit Configurations for good (#1180)
Removing a temporary configuration in Edit Configurations only changed the dialog's draft; the reload after OK read it back from the saved temporary list. OK / Apply now drops removed temporaries from that list, keeps unedited ones temporary instead of saving every non-detected temporary configuration as a user configuration, and saves edited ones as before. Removing the last temporary configuration also cancels a pending rewrite of the list that could bring the removed entry back (the chooser's Delete had the same race).

Fixes #1162
2026-11-04 13:20:41 +08:00
l0ng-ai cbc77941d4 feat(vcs): bind Git › Add to ⌥⌘A / Ctrl+Alt+A (#1159)
Git › Add (vcs.addToVcs) had no shortcut in the IntelliJ keymaps. It now uses ⌥⌘A on macOS and Ctrl+Alt+A on Windows/Linux, so it shows next to Add in the Git menus and works from the project view, editor and Commit view. On Windows the diff viewer's Apply Right Side keeps Ctrl+Alt+A while a diff is focused. The VS Code preset lists the action without a shortcut.

Closes #1158
2026-11-04 13:16:19 +08:00
Nguyen Thinh 3d7768f47f feat(window): graphql window (#1166)
Adds a GraphQL tool window: schema discovery from graphql-config files (or every GraphQL file without one), schema errors and endpoints, running the operation at the caret against a configured endpoint with ${NAME} environment variables, Query Result, and introspection into the schema file. The editor gets a GraphQL toolbar with endpoint selection, variables and run.

The tool window is listed on the stripe once used, so projects without GraphQL get no extra button.

Closes #1168

Co-authored-by: kidp2h <[email protected]>
2026-11-04 13:02:01 +08:00
ppxbandppxb 8687f6b1e3 perf(settings): open the Settings window shell before its startup chain (#1175)
The Settings window ran its whole startup chain (importing every settings page, then the project and settings IPC, one after another) before opening the dialog, so it showed an empty title bar for a while on every open. The dialog now opens at once with a loading state while the pages and the settings load in parallel, and re-bases the open dialog when the settings arrive; the search field is focused once the pages are in.

Co-authored-by: ppxb <[email protected]>
2026-11-04 12:54:53 +08:00
l0ng-ai 85c03d33af fix(editor): release a decision to leave whose project change never came (#1178)
When the project change after an authorized leave never came, the editors' close decision stayed authorized: files failed with "The project is closing" and later leaves with "An editor close decision is already pending" until restart. An authorized decision with no project opening is now released when the window opens a file or is left again.

Fixes #1139
2026-11-04 12:51:27 +08:00
l0ng-ai 65ca0390de feat(run): choose the JRE and the module of an Application configuration (#1172)
Application / Spring Boot configurations found their module only from the main class, so the same class in two modules failed with "Cannot determine the module for main class", and the JVM was always the module's. The form now has JRE (the module's JDK or an installed JDK) and Use classpath of module (found by main class, or a project module, stored as RunConfig.module). Run configuration selects can load their options when shown.

Fixes #1153
2026-11-04 12:41:34 +08:00
Mariano Dupontandmarianomd b31938db00 feat(deployment): add standalone comparison and selected-file sync (#1070)
Sync with Deployed opens in its own window, one per project window. It lists local files next to the server's copies, with filters, content and binary comparison, and an embedded side-by-side diff whose local side can be edited. The selected file can be synchronized (Enter) without closing the window.

The window is the `deployment` kind of the auxiliary windows that Settings and comparisons use. A cancelled close in it stops its project window from closing. The preview uses the vcs diff viewer through its `diff` editor kind. Also covers #811 (compare with the deployed versions).

Co-authored-by: marianomd <[email protected]>
2026-11-04 12:40:45 +08:00
l0ng-ai 60969eccb6 fix(java): preserve mapped toolchain paths and refresh Maven run settings (#1147)
* fix(sdk): keep the drive letter of a Maven home found on a mapped drive

Maven discovery canonicalized mvn on PATH and each Maven home to resolve symlinks (Homebrew's
bin/mvn). On Windows, a home on a mapped network drive or a subst drive then became
\\?\UNC\server\share\… (or another drive), the Maven home setting picked from the list stored
that path, and cmd.exe can't start mvn.cmd from it: every Maven run failed with "The system
cannot find the path specified". Discovery now keeps the path as found when canonicalizing would
change its drive letter.

Refs #822

* fix(sdk): keep the drive letter of a JDK home on a mapped drive

A JDK on a mapped network drive was canonicalized to \\?\UNC\server\share\…
too, so Maven ran with that JAVA_HOME and mvn.cmd failed with "The system
cannot find the path specified" even with a local Maven home. JDK homes
(configured, JAVA_HOME, the java on PATH) now keep their drive letter like
Maven homes do.

* fix(maven): resolve current toolchain settings for each run

* fix(sdk): distinguish mapped roots from external launcher links

* test(run): preserve explicit Maven choices across save and reload

* fix(maven): keep edited nested test launchers when refreshing defaults
2026-11-04 12:35:01 +08:00
xdpan1234andxdpan1234 0a70b0af01 fix(vcs): open comparisons in independently resizable native windows (#818)
Compare with Local, Compare Versions and the other revision comparisons open in a separate window that belongs to their project window. The window can be moved outside the main window, resized from its edges and corners, and shows the full diff. Edits made there and in the project window are mirrored both ways.

Root cause of #798: the comparison was an in-window dialog with a fixed size.

Settings and comparison windows now share one auxiliary-window mechanism (`AuxKind`, `open_auxiliary_window`). It handles labels, the title bar, placement over the parent, event forwarding, document-edit mirroring, menu routing, closing together with the parent, and remembered geometry per kind.

Co-authored-by: xdpan1234 <[email protected]>
2026-11-04 12:27:45 +08:00
l0ng-ai 268cf01a86 fix(markdown): preview headings in the preview's text colour (#1171)
Preview headings used the editor scheme's Markdown › Heading attribute (meant for the source) since #992; they now use the editor foreground like the rest of the preview.

Fixes #1151
2026-11-04 12:22:32 +08:00
l0ng-ai fd4921c58a fix(agents): answer a permission request from its line in the conversation (#1170)
A pending agent question / permission request could only be answered from the banner at the top of the session; the conversation line said "Waiting for your answer" with nothing to click. The line now offers the same answers, and a request without options gets Cancel in both places so the agent isn't left waiting.

Fixes #1148
2026-11-04 12:22:27 +08:00
kkmihaiandkkMihai e04e5dd57d feat(database): add ClickHouse data sources (#865)
Adds ClickHouse to Data Sources over its HTTP interface (HTTP/HTTPS, SSH tunnels, session settings, server-side cancellation): browse databases, tables and columns, inspect DDL, run queries and EXPLAIN plans, with ClickHouse dialect completion. Table data stays read-only and manual transactions are rejected, since ClickHouse keys are not unique and HTTP has no transactions.

Results are streamed as JSONCompactEachRowWithNamesAndTypes and parsing stops at the row limit, because result_overflow_mode=break still returns a whole block; errors show only the server's "Code: …" message, including failures raised mid-stream.

Fixes #862

Co-authored-by: kkMihai <[email protected]>
2026-11-04 12:21:03 +08:00
l0ng-ai c5e188d1e3 fix(i18n): translate the GitHub pull request operations' progress and failure titles (#1169)
The GitHub tool window's long operations passed an untagged banner title and appended " failed" for the error, so the banner stayed English and failures read half translated. withBusy now takes a tx-tagged progress title and its own failure title per operation (checkout, fetch for diff, worktree, apply/commit suggestion, resolve thread).
2026-11-04 12:02:06 +08:00
l0ng-ai d3b9a39831 fix(run): keep temporary run configurations following the toolchain settings (#1167)
Temporary run configurations were kept with what settings derived when they were made (absolute mvn, -s / -Dmaven.repo.local, JAVA_HOME), and never went through the backend again, so after a Maven home / settings file / JDK change they and gutter test runs narrowed from them still ran the old ones. They are now kept without derived values (run_without_derived) and get the current settings applied when the run list loads (run_apply_settings). Maven's settings file and local repository apply to every Maven configuration unless it sets them itself. Earlier stored entries are dropped once.

Refs #822
2026-11-04 11:56:03 +08:00
Julien Dudek 59567f1af0 feat(i18n): add French (#730)
Adds French as a UI language, with French catalogs for message keys and source text,
and the French space before a colon in settings labels.

UI languages are now registered in one place, `LOCALES` in `src/platform/i18n.ts` (name
in its own language and `<html lang>` tag); the Language and Region options, settings
validation and `<html lang>` all come from it. The catalog tests read every catalog in
`locales/` and `locales/text/`, so a new file is always checked, and literal message keys
given to `t` must exist in `en.json`. Only Chinese must cover every UI string; French and
Korean show English for text they lack.

Also makes two date tests independent of DST changes.

Co-authored-by: J-Dudek <[email protected]>
2026-11-04 11:48:45 +08:00
Park Jong Hyun 7f0c78893e feat(i18n): add Korean (#1126)
Adds Korean as a UI language: Settings › Appearance & Behavior › System Settings ›
Language and Region lists 한국어, with Korean catalogs for message keys and source text.

Only Chinese must translate every UI text. Other locales may lack entries and show
the English text for them; their catalogs are still checked for stale entries, keys
English lacks and placeholders that differ from the English text.

Co-authored-by: fhrmrk <[email protected]>
2026-11-04 11:43:30 +08:00
l0ng-ai fc6a27d4ba fix(java): give jdtls memory back when idle and stop it showing as not responding (#1137)
jdtls ran on the parallel collector, which never shrinks the heap while idle, so each server kept its import/build peak committed for as long as it lived. It now runs on G1 with a periodic collection every minute and a 10–30% free-heap band, so memory goes back to the OS once the work is done (a SkyWalking checkout: 1.9 GB → 1.25–1.3 GB idle). A collector set in the JVM options still replaces ours with its tuning; setting either heap-free ratio drops both of ours.

The Equinox launcher gets -nosplash: without it its native library loads AppKit on macOS and every jdtls shows as Not Responding in Activity Monitor.

Part of #1008.
2026-11-04 11:40:02 +08:00
l0ng-ai 5e73b1634f feat(agents): environment variables for built-in agents (#1157)
Settings › Tools › Agents › Built-in has an Environment variables field under the executable. They are saved in the agent's file like a command override, and the agent starts with them on top of the shell environment, replacing variables of the same name. Changing the command or the environment keeps the other; a file left with only the built-in's values is removed.

Closes #1133
2026-11-04 11:07:52 +08:00
l0ng-ai e88e7e6459 fix(i18n): translate Git operation titles and their notifications (#1154)
Git operations passed their title, success and failure text to runOp / runGitOperation as plain strings the text catalog check doesn't see, so notifications after a commit, merge, rebase, stash, shelve, tag, patch or rollback stayed in English, several splicing plurals or English words. Call sites now tag the text with tx, each count and variant has its own sentence, and a failure is titled "{title} failed" from the translated title.
2026-11-04 10:57:10 +08:00
l0ng-ai c86928eee7 fix(deployment): Windows absolute mapping paths, and a flaky plugin syntax test (#1130)
An absolute local mapping path outside the project falls back to the same path under the project. On Windows, joining it onto the project root kept its drive and replaced the root, so an existing outside folder became a mapping. The drive and root are now dropped first, and the mapping tests use the platform's absolute paths.

The Solidity syntax test asks again when the first call of a new instance runs past the edit-path budget and its answer is dropped.
2026-11-04 10:27:50 +08:00
l0ng-ai fa6d7276b8 fix(i18n): name each Docker prune and Compose operation in a full sentence (#1150)
Prune and Compose task titles spliced English into a sentence ("Pruning ${target}…", "Compose ${op} ${project}…"), so they stayed partly or wholly English. Each prune target and Compose operation now has its own title and failure text.
2026-11-04 10:16:15 +08:00
l0ng-ai 1bc7fd06c1 feat(gradle): navigate to version catalogs imported from published artifacts (#1149)
A catalog declared in settings as create("name") { from("group:artifact:version") } is a published .toml Gradle downloads into its cache. Build-script accessors of such a catalog now go to and describe its entries like a local catalog's: gradle_published_catalog finds the file in Gradle's cache or the local Maven repository on the project's host. A catalog not downloaded yet is looked up again the next time the script opens.

Closes #1006
2026-11-04 10:14:05 +08:00
Park Jong Hyun 280c2da48f fix(editor): keep an editor color scheme chosen together with the theme (#1138)
Since #1100 a theme that turns light or dark switches a chosen scheme of the other kind back to the theme's own. It did so also when the scheme was chosen in the same Apply as the theme, so One Light + Solarized Dark left the default light editor. A scheme set by the settings change that also changes the theme now goes with it; a later theme change on its own still switches it back.

Closes #1136

Co-authored-by: fhrmrk <[email protected]>
2026-11-04 10:07:04 +08:00
l0ng-ai 2ec2b76d7a fix(remote): keep the host in a remote window's title (#1146)
Connecting a remote window titles it `<project> [<user@host>]`, but the frontend rewrote the title from the project and active file whenever either changed (and after a reload), dropping the host. The title now includes the remote session's host.
2026-11-04 09:58:51 +08:00
l0ng-ai 425a7da003 feat(run): import the Node interpreter of JetBrains npm configurations (#1144)
An npm configuration's own Node interpreter was listed as ignored on import. The import now fills the npm form's Node interpreter field, together with the form's other fields so it shows the configuration as it runs. Jest, Vitest and Node.js configurations still list it as ignored.
2026-11-04 09:57:44 +08:00
l0ng-ai 87e9bec4f9 fix(todo): colour TODO comments in the TODO tool window's preview (#1143)
The preview is an editor view over the real document but doesn't get the editor contributions, so TODO comments showed in plain comment colour there. It now adds the same TODO colouring; the live item scan stays with the editor contribution.
2026-11-04 09:55:32 +08:00
l0ng-ai f088a4afb0 fix(i18n): translate notification titles passed through helpers (#1142)
Debugger action failures, project tree clipboard / create-file failures and Docker operation failures pass their title through a helper before notify, so the text catalog check never saw them and they showed in English. The call sites now tag the text with tx, and Docker failures name each operation in a full sentence instead of splicing an English verb or object into one.
2026-11-04 09:53:56 +08:00
l0ng-ai 2e6cda1d86 feat(run): update a Tomcat deployment from compiled classes instead of rebuilding the WAR (#1140)
"Update classes and resources" / "Update resources" on a Tomcat configuration deploying the project's WAR artifact ran a full package build and re-exploded the archive. They now compile the WAR module incrementally through jdtls and copy only the changed classes and resources into WEB-INF/classes and changed web files (src/main/webapp) into the exploded deployment; Tomcat reloads the context, or HotSwap applies the classes when debugging. Falls back to rebuilding the WAR when a dependency module was compiled since the last update (its WEB-INF/lib jar would be stale), the deployment doesn't exist yet, or no Java language server is available. Redeploy still rebuilds the WAR.

Closes #970
2026-11-04 09:43:54 +08:00
l0ng-ai 42968a39b3 style(ui): remove hover backgrounds from icon actions 2026-11-04 09:39:32 +08:00
l0ng-ai 4d086c44c9 fix(workbench): show splash-style progress when opening projects 2026-11-04 09:39:32 +08:00
l0ng-ai 770178966f fix(lsp): tell language servers about files changed outside the editor (#1135)
Language servers never got workspace/didChangeWatchedFiles, so files changed on disk outside the editor (checkout, pull, another tool) stayed unknown to them; jdtls's incremental build skipped them and Java / Spring Boot launches ran their old classes (e.g. another module of a multi-module project). The project's file watcher now forwards each batch to the running servers: created, changed and removed files of a server's languages under its root.

Refs #1015
2026-11-04 09:16:32 +08:00
Qingandqtvbwfn 2f2d3a25ae feat(todo): edit the file right in the TODO preview (#1129)
The TODO preview is an editor on the file, as in IntelliJ: a CodeMirror view attached to the real document, so edits share the buffer with editor tabs and Save All saves them; no editing switch. The caret lands on the selected item, also when another item of the same file is selected. Read-only files fall back to the read-only preview. Dirty buffers are held rather than released (releasing would auto-save or drop the edits), and previewText reads the held text.

Co-authored-by: qtvbwfn <[email protected]>
2026-11-04 08:58:53 +08:00
Qingandqtvbwfn 893cbede9c fix(todo): only count real comment markers around TODO keywords (#1128)
The comment filter accepted `#`, `--`, `;` and `%` anywhere before the keyword, so `foo(); TODO`, `if (a % 2) TODO`, `i--; TODO`, URLs and strings produced items. Inline openers (// /* <!-- # -- {- (*) now count only at the line start or after whitespace; `*` `;` `%` and docstring quotes only at the line start. A TODO anywhere inside a comment still counts, as in IntelliJ. Lines the backend truncated keep the lenient rule.

Co-authored-by: qtvbwfn <[email protected]>
2026-11-04 08:56:00 +08:00
Qingandqtvbwfn e5718d41d8 feat(ui): make preview panes selectable and copyable (#1127)
The workbench disables selection globally, so the read-only preview panes beside result trees (TODO, Usages, Problems…) could not be selected or copied. The pane opts back in (user-select: text) and uses useSelectionBoundary so drags stay inside it; the line-number gutter stays unselectable so copied text is code only.

Co-authored-by: qtvbwfn <[email protected]>
2026-11-04 08:54:25 +08:00
l0ng-ai 53b8cdeb51 feat(remote-dev): forward ports of the remote host (#1131)
Remote windows get a Ports tool window: forward localhost:<port> of the remote host to this machine (same port when free here), with Open in Browser / Copy / Stop. Forwarded ports are remembered per host and folder and forwarded again on reconnect. Tunnels reuse the window's SSH session (ide_ssh::Tunnel can prefer a local port) and close with the connection.

Closes #819
2026-11-04 08:50:25 +08:00
l0ng-ai a3bbbf90a4 feat(vcs): clone from the repositories of your GitHub account (#1125)
The Clone Repository dialog lists GitHub next to Repository URL: the repositories of the account gh is signed in to, searchable; picking one fills URL and directory, double-click or Enter clones. Clone URL follows gh's git_protocol. Clone sources are a workbench contribution (registerCloneSource); gh_account_repos needs no project.

Closes #750
2026-11-04 08:40:06 +08:00
l0ng-ai 9988ce4113 Revert "feat(i18n): add Korean" (#1124)
Reverts #1122, merged by mistake (wrong PR number) before review and under another change's commit message. Adding UI locales is still an open decision (#914), and the Korean catalog fails the catalog tests on current main.
2026-11-04 08:31:26 +08:00
l0ng-ai 35c6af2367 feat(app): open a file at a line and column from the command line (#1123)
`openidea --line 120 --column 8 <file>` opens the file at that position (1-based; without --column the line start; clamped). The position is kept by file until the window that opens it takes its launch: new instance, second instance, a project open elsewhere, an already open file. The path argument skips these flags' values.

Closes #829
2026-11-04 08:26:41 +08:00
Park Jong Hyun 3eabaf0d39 feat(app): open a file at a line and column from the command line (#1122)
`openidea --line 120 --column 8 <file>` opens the file at that position (1-based; without --column the line start; clamped). The position is kept by file until the window that opens it takes its launch: new instance, second instance, a project open elsewhere, an already open file. The path argument skips these flags' values.

Closes #829
2026-11-04 08:25:49 +08:00
l0ng-ai 192e613830 feat(run): list recently started configurations at the top of the chooser (#1121)
The Run/Debug configuration popup lists the five configurations started last under "Recent", newest first, after the pinned ones and not repeated in their group. Starts are recorded from run and debug sessions (any executor, reruns included). Kept per project.

Refs #997
2026-11-04 08:15:13 +08:00
l0ng-ai 4c5b6de9ad feat(run): choose the Node interpreter of an npm configuration (#1120)
npm configurations get a "Node interpreter" field; the script and the package manager run on that node (its directory first on the PATH). Empty keeps the project's interpreter from Settings › Languages & Frameworks › Node.js. Saving drops the PATH settings derived before the choice, so a stale project PATH isn't kept.

Closes #997
2026-11-04 08:11:43 +08:00
l0ng-ai 6bf1d25711 feat(run): pin configurations to the top of the chooser (#1119)
Hovering a configuration in the Run/Debug configuration popup shows a Pin button. Pinned configurations are listed first under "Pinned", in pin order, and not repeated in their group; a search keeps pinned matches on top. Pins are kept per project.

Refs #997
2026-11-04 08:06:23 +08:00
l0ng-ai aeb3abca34 feat(run): mark running configurations in the chooser, with Rerun and Stop (#1118)
A configuration with a running or debugged process gets a green dot on its icon in the Run/Debug configuration popup; its row offers Rerun and Stop in place of Run and Debug. Rerun closes the popup, Stop keeps it open.

Refs #997
2026-11-04 08:02:54 +08:00
l0ng-ai b9f3c07f9b fix(debugger): identifier boundary check for astral letters under V8 (#1117)
Smart Step Into and Safe Delete tested the text before a match with /[\p{L}\p{N}_$]$/u, which V8 in Node 26 fails for a letter outside the BMP (a lone property class before $), so smartStep.test.ts failed. Both checks now use an alternation with the same meaning.
2026-11-04 07:57:12 +08:00
l0ng-ai a1dd16f26d fix(i18n): translate the Delete question, prompt labels and the shortcut recorder placeholder (#1116)
The project tree's Delete question was concatenated from English fragments, and the VCS prompt / multi-choice labels and the shortcut recorder placeholder were rendered as given, so they stayed English in the Chinese UI. The question is now tx templates with zh entries; the labels and placeholder go through the catalog.
2026-11-04 07:55:44 +08:00
Mariano Dupontandmarianomd 94665528b9 fix(vcs): refresh packed refs after file replacement (#1115)
Git Log could keep old commits and report 'unknown revision HEAD' after Git replaced packed-refs without a newer modification time: gitoxide's packed-ref snapshot only reloads for a newer mtime. The cached repository handle now also watches the common packed-refs file and compares modification time plus length before reuse; covered for the main repository and a linked worktree.

Co-authored-by: marianomd <[email protected]>
2026-11-04 07:50:13 +08:00
l0ng-ai 809c6adcf1 fix(search): keep Search Everywhere's keys from the editor's actions behind it (#1114)
The capture-phase keymap ran editor actions on keys the Search Everywhere field handles, since editor actions apply while a popup is open over an editor: ⌥↵ ran Show Context Actions instead of Assign Shortcut, and ⌥↑/⌥↓ and the other navigation keys could reach the editor too. The field now claims the keys its onKeyDown handles while it has focus.
2026-11-04 07:26:56 +08:00
l0ng-ai 2bb8f3b091 feat(run): Stop lists the running processes with Stop All when several run (#1113)
Stop from the shortcut, menu or Find Action stopped the selected configuration's process whenever it was running, so with several processes there was no way to pick another or stop them all. Like IDEA's Stop it now stops the only stoppable process or lists them all with Stop All; the run widget's stop button keeps stopping the selected configuration and the Run tool window its active tab.
2026-11-04 07:23:09 +08:00
arookieofcanda-rookie-of-C-language cd77c9c6fb fix(vcs): prefill the New Branch dialog with the current branch name (#1073)
A plain New Branch (single and multi-repository) now opens with the current branch name, like JetBrains IDEs, so a related branch is an edit away; detached HEAD stays empty. New Branch from a local ref now applies its <ref>-copy suggestion, which was computed but never used.

Co-authored-by: a-rookie-of-C-language <[email protected]>
2026-11-04 07:11:19 +08:00
l0ng-ai d72f8a1506 fix(agents): show a pending question in full in the permission banner (#1111)
The permission banner kept its text on one line, so an agent's question (Claude Code's AskUserQuestion arrives as a permission request) was cut off and its answers crowded into the rest of the line. The question now wraps and scrolls past a third of the editor's height, and the answers wrap after it; permission lines in the conversation show up to three lines with the full text as tooltip.
2026-11-04 06:55:08 +08:00
l0ng-ai c5749b0c17 fix(settings): open the Settings window's content even when a startup step fails; log frontend errors (#1110)
The native Settings window showed only its title bar until its startup chain finished; a page that failed to register ended the chain silently. The steps are now guarded and openSettings runs in finally. Uncaught errors and unhandled rejections of every window (and caught errors via logError) are written to the log file through log_frontend, deduplicated and capped per window.
2026-11-04 06:51:41 +08:00
Andrea Basileandandreabasile 4d224f80c9 feat(lsp): per-server features and code-action-kinds filter (#1099)
A [[language-servers]] entry (or languageServers.<name> in settings) can list features and code-action-kinds: capabilities outside features are dropped after initialize and from later registrations, publishDiagnostics and Inspect Code only use servers with diagnostics, and code actions are narrowed to the kept kinds (advertised kinds and answers both). No built-in definition uses them yet. First stage of #901: lets a refactoring-only server sit next to the language's own one.

Co-authored-by: andreabasile <[email protected]>
2026-11-04 06:43:31 +08:00
l0ng-ai 63dd83c129 feat(build-tools): navigate Gradle version catalog accessors to their TOML entries (#1109)
In build scripts, Go to Declaration on a catalog accessor (libs.<alias>, versions./bundles./plugins.) opens the entry in the catalog file and hover shows its declaration with the referenced version. Catalogs are the build's gradle/libs.versions.toml and those declared in settings with from(files(...)), re-read when they change. In a catalog, version.ref values and bundle members go to their entries.
2026-11-04 06:14:07 +08:00
l0ng-ai afea58c1c6 feat(run): browse buttons on path fields and a resizable Run/Debug Configurations dialog (#1108)
Run configuration types mark path fields (browse: file or directory, or chosen from the other params) and the form puts a folder button next to them; Working directory and Redirect input get one too, storing project paths relative to the root. Covers Tomcat, Shell Script, Executable, Godot, npm, Gradle, Maven, Go, Ruby and Docker path fields. The dialog is resizable and its content grows with it.
2026-11-04 06:06:38 +08:00
l0ng-ai 19615e00b9 feat(vcs): select several repositories in the Git log Repository filter (#1107)
Clicking a repository in the Repository filter menu checks or unchecks it instead of replacing the selection, so the log shows any subset of the project's repositories; unchecking the last one shows all.
2026-11-04 06:00:20 +08:00
l0ng-ai c347e331ab feat(run): browse for .env files from the Environment variables field (#1106)
A folder button next to the Environment variables input opens a file dialog at the project root and appends the chosen files to the configuration's .env files (relative to the root inside the project, duplicates skipped). Hidden in remote windows, where the dialog would pick local paths.
2026-11-04 05:55:20 +08:00
Andrea Basileandandreabasile 9a6b84e287 feat(run): import JetBrains run configurations (.run/*.run.xml, .idea/workspace.xml) (#1039)
Run/Debug Configurations offers Import from JetBrains run configurations when the project has .run/*.run.xml, .idea/runConfigurations/*.xml or run configurations in .idea/workspace.xml; Find Action has Import Run Configurations from JetBrains (.run, .idea). Python, Django server/tests, pytest, npm, Jest, Vitest, Node.js, Shell Script and compound configurations map to the matching kinds; other types, unknown path variables and multi-module $MODULE_DIR$ are skipped with the reason. Shared .run files take precedence over .idea/runConfigurations and the workspace's RunManager (templates and temporary configurations left out). Path and argument helpers shared with the launch.json importer moved to import/mod.rs.

Co-authored-by: andreabasile <[email protected]>
2026-11-04 05:46:50 +08:00
Andrea Basileandandreabasile df6fbb6d21 fix(tools): offer to install only the first missing server of interchangeable ones (#1082)
Opening a Python file with none of basedpyright, Pyright, ty and Pyrefly installed raised four install notices at once: each fallback server reports missing after the one before it, and every missing server was prompted. Among interchangeable servers only the first missing one is offered now (events handled one at a time so a fallback sees whether the server before it was offered); a fallback is still offered when the servers before it can't be installed or are turned off.

Co-authored-by: andreabasile <[email protected]>
2026-11-04 05:24:23 +08:00
l0ng-ai d94499b7ac feat(vcs): the Push dialog shows the outgoing commits' changed files (#1105)
Below the outgoing commits, the Push dialog lists the files they change: all outgoing commits' combined changes by default, or the selected commit's (Show All Commits goes back). Selecting a file shows its diff beside the tree inside the modal dialog. The multi-repository Push dialog does the same for the selected repository or commit.
2026-11-04 05:20:58 +08:00
l0ng-ai f148c1ac6b fix(debugger): muted breakpoints show as grey dots, not as disabled ones (#1104)
With Mute Breakpoints on (remembered per project), every breakpoint was drawn as a hollow red circle, the disabled look. Muted breakpoints now use the grey muted icon; disabled ones keep the hollow circle. Method breakpoints and field watchpoints follow the same rule.
2026-11-04 04:56:21 +08:00
l0ng-ai 5ad5dc2a28 feat(project-tree): Undo restores files deleted in the Project view (#1103)
Ctrl+Z / Cmd+Z in the Project view (and Edit › Undo while it has focus) reverts the latest file operation there: a delete as well as a move or rename. Deleted files come back from their last Local History snapshot, folders are re-created and the restored items selected; items without a snapshot are named in a warning. Edit › Undo reaches the view through a new registerUndoHandler contribution; the Project view is found by a data attribute instead of its translated aria-label (also fixes the New popup's fallback anchor).
2026-11-04 04:53:48 +08:00
l0ng-ai fe0ee338f4 feat(debugger): completion in Evaluate, the Variables evaluate field and the Debug console (#1102)
While suspended, the debugger's expression fields complete from the debug adapter (DAP completions) in the selected frame: Evaluate, the Variables evaluate field and the Debug console share one hook. The list opens while typing an identifier or after '.', or on Ctrl+Space; Up/Down, Enter/Tab, Escape. The Evaluate dialog is resizable and its result tree grows with it. Backend: ide-dap completions (skipped without supportsCompletionsRequest), debug_completions on the host and the desktop app.
2026-11-04 04:47:12 +08:00
l0ng-ai e575bf8c24 fix(editor): the editor color scheme follows a switch between light and dark themes (#1100)
Switching the IDE theme kept an explicitly chosen editor color scheme, so a light theme could keep a dark editor. When the applied theme turns light or dark, a chosen scheme of the other kind now switches back to the theme's own; a scheme picked to go with the current theme is kept. Reacts to the applied theme, so Quick Switch, Settings, Welcome › Customize, other windows and Sync with OS are all covered.
2026-11-04 04:31:59 +08:00
l0ng-ai 1ed9275da0 fix(remote): Docker run configurations take their environment from the remote host (#1098)
The remote server didn't serve docker_run_env, so a remote window asked the local app, whose PATH then replaced the remote host's PATH in a run that starts there: docker compose failed with "docker: not found". The host now serves docker_run_env from its own PATH and docker executable; covers the Dockerfile, image and Compose run types.
2026-11-04 04:24:53 +08:00
l0ng-ai 3a98d5c5be fix(remote-dev): list this machine's SSH configurations in a remote project's window (#1097)
A remote project's window gets the remote host's settings, which don't include this machine's SSH configurations, so the Remote Development dialog showed "No SSH configurations" there. The connection is made from this machine, so the dialog reads the local global SSH configurations and reloads them when settings change. Other SSH-configuration consumers run on the host and keep the host's settings.
2026-11-04 04:16:18 +08:00
l0ng-ai 6b9ab3b323 fix(settings): Export Settings includes IDE view options (#1095)
Project view options such as "Open Files with Single Click" live in localStorage under openidea.projectTree.options, which was not on the export allowlist, so Export/Import Settings dropped them.

Other IDE-level preferences stored the same way are exported and restored too: TODO patterns, IdeaVim settings, diff/merge options, branches popup options, markdown and image viewer defaults, reopen last project. Per-project state stays out.
2026-11-04 04:01:30 +08:00
l0ng-ai 23e468252f fix(vcs): link filtered Git Log commits past the commits a filter hides (#1094)
With a User or text filter, the Git Log graph opened a lane for every commit whose parent the filter hid; the lanes never closed and filled the graph with arrows, since git keeps the real parents for these filters.

The log now links each shown commit to its nearest shown ancestors, dashed where the link passes hidden commits (the date range and No Merges count as hidden there too); a page's last commits are linked by walking on past the page. Collapse Linear Branches keeps those dashed links.

Fixes #872
2026-11-04 03:57:08 +08:00
l0ng-ai a6c5574194 feat(build): Build Project for CMake projects (#1093)
Build › Build Project and Rebuild Project reported "no supported build system" in a CMake project. They now configure the build directory when needed and run cmake --build (--clean-first for Rebuild), with the directory and options of the first CMake profile, else a configured directory or build. GCC / Clang errors and warnings are reported at their source positions.

Fixes #873
2026-11-04 03:33:59 +08:00
l0ng-ai 5d63f618ef fix(maven): include the modules of the profiles chosen in the Maven tool window (#1092)
Checking a profile in Maven › Profiles only passed -P to Maven runs and syncs: the project model was read from the POMs with the activeByDefault profiles alone, so a module declared in a chosen profile stayed a plain folder and its dependencies and properties were missing.

The POMs are now read with the chosen profiles (saved with the last sync), with Maven's rule that choosing one of a POM's profiles turns its activeByDefault ones off. A profile change refreshes the project roots at once, so the project tree and Maven tool window show the new modules even when Maven itself cannot run.

Fixes #850
2026-11-04 03:27:10 +08:00
l0ng-ai 2ba07341c2 feat(editor): mark syntax errors in files no language server checks (#1087)
Without a running language server (none configured for the language, or not installed, stopped or crashed), the editor showed no syntax errors at all, though the built-in parser already finds them. The editor now reports the parser's errors for such files, checked again after edits and when a language server starts or stops; while a server runs for the language it reports syntax errors itself. SQL is left to the database support. The check is the built-in Syntax errors inspection.

Fixes #824
2026-11-04 03:02:32 +08:00
l0ng-ai 0dffad814a fix(editor): show wavy and dotted underlines on macOS (#1086)
Error, warning, weak-warning and typo underlines never showed on macOS, nor did underline effects of syntax tokens from a colour scheme: WebKit rejects a text-decoration shorthand with a style or colour (underline wavy <colour>), so the declaration fell back to none.

The editor's diagnostic and token styles also set -webkit-text-decoration, which WebKit takes the full value through; the red wavy underlines of the Problems, Build, Project and Database views use the longhand properties.
2026-11-04 03:01:03 +08:00
1c5a92f952 feat(plugins): plugin system with extension points, consent, Wasm runtime and plugin API 1.0 (#971)
A plugin system built on WebAssembly components (wasmtime, WASI p2), with a
typed WIT API (`openidea:[email protected]`) and a Rust SDK, shipped as a preview.

- Extension points declared in `plugin.toml` (commands, languages, themes,
  icon themes, menus, key bindings, status items, settings, views, custom
  editors, file icons and templates, run detectors, editor providers),
  validated before anything takes effect and projected onto the workbench's
  own registries: plugin themes and file icon themes are chosen in
  Settings › Appearance and Quick Switch, plugin shortcuts are edited in
  Settings › Keymap, plugin problems are grouped in Settings › Inspections.
- Capabilities with risk levels and consent at install, pinned `process:`
  programs, secrets in the OS keychain.
- Runtime: one thread per active plugin, fuel/time/memory budgets, soft and
  hard deadlines on the editing path, cancellation, crash backoff, idle
  reclaim, a precompiled-component cache; remote projects run code plugins on
  `openidea-server`.
- Web views and custom editors with a sandboxed page, a document bridge
  joined to the text editor's undo history, and split or preview layouts.
- Distribution: signed `.oiplugin` packages, a static marketplace index with
  revocation, versioned installs with rollback, install from a `.zip` or a
  folder. The release workflow signs and publishes the official plugins and
  their index, which released builds use as the default marketplace.
- Official plugins: String Tools, Key Promoter, Git Toolbox, AI Completion,
  Checkstyle, MyBatis Navigator, CSV Editor, Solidity, Material Icons, Harbor
  Night, Excalidraw, Graphviz, Regex Tester, License Header.

Closes #732

Co-authored-by: Deng Liu <[email protected]>
Co-authored-by: florentvdry <[email protected]>
2026-11-04 02:54:31 +08:00
l0ng-ai cfdbd3872a feat(find): browse for the Directory scope's folder in Find in Files (#1085)
The Directory scope of Find in Files only took a typed path. A folder button next to the field now opens a folder chooser at the current directory and fills the field with the chosen folder.

Fixes #847
2026-11-04 02:53:03 +08:00
959 changed files with 93973 additions and 5361 deletions

No files matched your search

+1 -1
View File
@@ -1,4 +1,4 @@
Closes #
<!-- If this resolves an issue: Closes #<n> -->
## What and why
+17 -15
View File
@@ -15,6 +15,8 @@ concurrency:
env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
# Line tables keep test backtraces readable; full debuginfo only adds build time and cache size.
CARGO_PROFILE_DEV_DEBUG: line-tables-only
jobs:
rust:
@@ -25,10 +27,10 @@ jobs:
os: [macos-latest, ubuntu-22.04, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- uses: actions/checkout@v5
- uses: pnpm/action-setup@v5
with: { version: 11 }
- uses: actions/setup-node@v4
- uses: actions/setup-node@v5
with: { node-version: 22, cache: pnpm }
- uses: dtolnay/rust-toolchain@stable
with: { components: "clippy, rustfmt" }
@@ -62,7 +64,7 @@ jobs:
name: Remote server (linux musl)
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@stable
with: { targets: x86_64-unknown-linux-musl }
- uses: Swatinem/rust-cache@v2
@@ -79,7 +81,7 @@ jobs:
env:
CARGO_TARGET_DIR: ${{ github.workspace }}/target-plugins
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@stable
with: { targets: wasm32-wasip2, components: "clippy, rustfmt" }
- uses: Swatinem/rust-cache@v2
@@ -100,9 +102,9 @@ jobs:
cargo test --locked --manifest-path "$dir/Cargo.toml"
echo "::endgroup::"
done
- uses: pnpm/action-setup@v4
- uses: pnpm/action-setup@v5
with: { version: 11 }
- uses: actions/setup-node@v4
- uses: actions/setup-node@v5
with: { node-version: 22 }
# Plugin web pages (`web-src/`, their own pnpm projects): typecheck and test.
- name: Web pages typecheck and test
@@ -126,7 +128,7 @@ jobs:
name: Cargo deny
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@stable
- uses: taiki-e/install-action@cargo-deny
- run: cargo deny check
@@ -135,10 +137,10 @@ jobs:
name: Frontend
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- uses: actions/checkout@v5
- uses: pnpm/action-setup@v5
with: { version: 11 }
- uses: actions/setup-node@v4
- uses: actions/setup-node@v5
with: { node-version: 22, cache: pnpm }
- run: pnpm install --frozen-lockfile
- run: pnpm typecheck
@@ -154,16 +156,16 @@ jobs:
# Browsers and their system libraries are preinstalled; keep the tag in step with @playwright/test.
container: mcr.microsoft.com/playwright:v1.64.1-noble
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- uses: actions/checkout@v5
- uses: pnpm/action-setup@v5
with: { version: 11 }
- uses: actions/setup-node@v4
- uses: actions/setup-node@v5
with: { node-version: 22, cache: pnpm }
- run: pnpm install --frozen-lockfile
- run: pnpm e2e
env:
HOME: /root
- uses: actions/upload-artifact@v4
- uses: actions/upload-artifact@v6
if: failure()
with:
name: playwright-report
+124
View File
@@ -0,0 +1,124 @@
name: Language smoke
# Everyday code intelligence (completion, go to definition, Find Usages, hover, diagnostics
# with a quick fix, an intention, formatting, rename) per language, against the language
# server the app starts by default: crates/ide-host/tests/language_smoke.rs.
# Opening the fixture with its build tool and SDK (Maven / Gradle sync, the virtualenv, the
# JDK): the ignored tests of crates/ide-host/tests/project_smoke.rs.
# Then running and testing each fixture through the run store and openidea-server
# (Run, Run tests, the test tree, Rerun Failed Tests): smoke/run.test.ts; and debugging its
# entry point with the debugger the IDE installs (breakpoint, frames, variables, evaluate,
# Step Over, Resume): smoke/debug.test.ts.
on:
push:
branches: [main]
pull_request:
branches: [main]
types: [opened, synchronize, reopened]
paths:
- "crates/**"
- "languages/**"
- "fixtures/**"
- "Cargo.toml"
- "Cargo.lock"
- "src/features/run/**"
- "src/features/debugger/**"
- "src/platform/**"
- "smoke/**"
- "package.json"
- "pnpm-lock.yaml"
- ".github/workflows/language-smoke.yml"
workflow_dispatch:
concurrency:
group: language-smoke-${{ github.ref }}
# A new push supersedes a PR's run; on main every merge runs to the end (and saves the cache).
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
# Pinned so a server release can't fail an unrelated PR; bump them deliberately.
VTSLS: 0.3.0
TYPESCRIPT: 6.0.3
BASEDPYRIGHT: 1.40.2
RUFF: 0.17.3
GOPLS: v0.24.0
JDTLS: 1.63.0-202610191322
KOTLIN_LSP: 263.7084.0
GRADLE: 8.14.3
jobs:
smoke:
name: Language smoke (${{ matrix.language }})
strategy:
fail-fast: false
matrix:
language: [typescript, python, rust, go, java, kotlin]
runs-on: ubuntu-22.04
timeout-minutes: 40
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with: { components: "rust-analyzer, rustfmt" }
# One build cache for every language; main saves it.
- uses: Swatinem/rust-cache@v2
with:
shared-key: language-smoke
save-if: ${{ github.ref == 'refs/heads/main' && matrix.language == 'rust' }}
- uses: pnpm/action-setup@v4
with: { version: 11 }
- uses: actions/setup-node@v4
with: { node-version: 22, cache: pnpm }
- if: matrix.language == 'typescript'
run: npm install --global "@vtsls/language-server@$VTSLS" "typescript@$TYPESCRIPT"
- if: matrix.language == 'python'
uses: actions/setup-python@v5
with: { python-version: "3.12" }
- if: matrix.language == 'python'
run: pip install "basedpyright==$BASEDPYRIGHT" "ruff==$RUFF"
- if: matrix.language == 'go'
uses: actions/setup-go@v5
with: { go-version: "1.27", cache: false }
- if: matrix.language == 'go'
run: go install "golang.org/x/tools/gopls@$GOPLS"
- if: matrix.language == 'java' || matrix.language == 'kotlin'
uses: actions/setup-java@v4
with: { distribution: temurin, java-version: "21" }
- if: matrix.language == 'java'
run: |
mkdir -p "$RUNNER_TEMP/jdtls"
curl -fsSL "https://download.eclipse.org/jdtls/milestones/${JDTLS%-*}/jdt-language-server-$JDTLS.tar.gz" \
| tar xz -C "$RUNNER_TEMP/jdtls"
echo "$RUNNER_TEMP/jdtls/bin" >> "$GITHUB_PATH"
# The Kotlin fixture has no Gradle wrapper: the server imports it with the Gradle on PATH.
- if: matrix.language == 'kotlin'
uses: gradle/actions/setup-gradle@v4
with: { gradle-version: "${{ env.GRADLE }}" }
- if: matrix.language == 'kotlin'
run: |
mkdir -p "$RUNNER_TEMP/kotlin-lsp" "$RUNNER_TEMP/bin"
curl -fsSL "https://download.jetbrains.com/language-server/kotlin-server/$KOTLIN_LSP/kotlin-server-$KOTLIN_LSP.tar.gz" \
| tar xz -C "$RUNNER_TEMP/kotlin-lsp" --strip-components=1
ln -s "$RUNNER_TEMP/kotlin-lsp/kotlin-lsp.sh" "$RUNNER_TEMP/bin/kotlin-lsp"
echo "$RUNNER_TEMP/bin" >> "$GITHUB_PATH"
- run: cargo test --locked -p ide-host --test language_smoke --no-run
- run: cargo test --locked -p ide-host --test language_smoke -- --ignored --exact ${{ matrix.language }}
- if: matrix.language == 'java'
run: cargo test --locked -p ide-host --test project_smoke -- --ignored --exact maven_sync jdk
- if: matrix.language == 'kotlin'
run: cargo test --locked -p ide-host --test project_smoke -- --ignored --exact gradle_sync
- if: matrix.language == 'python'
run: cargo test --locked -p ide-host --test project_smoke -- --ignored --exact python_venv
- run: pnpm install --frozen-lockfile
- run: cargo build --locked -p ide-host --bin openidea-server
- run: pnpm smoke smoke/run.test.ts -t "^${{ matrix.language }}$"
- run: pnpm smoke smoke/debug.test.ts -t "^${{ matrix.language }}$"
# The IDE installs the debugger from GitHub releases: authenticated, against rate limits.
env: { GITHUB_TOKEN: "${{ github.token }}" }
+46
View File
@@ -0,0 +1,46 @@
name: Plugin index
# Refreshes the community plugins in the latest release's plugins-index.json
# (what released apps read) from the registry, l0ng-ai/openidea-plugins. The
# official plugins, their key and revocations stay as the release wrote them.
on:
schedule:
- cron: "17 * * * *"
workflow_dispatch:
# Sent by the registry when its index changes.
repository_dispatch:
types: [community-plugins]
permissions:
contents: write
concurrency:
group: plugins-index
env:
COMMUNITY_PLUGINS_INDEX: https://github.com/l0ng-ai/openidea-plugins/releases/download/index/index.json
jobs:
merge:
runs-on: ubuntu-latest
steps:
- name: Merge the community index
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
cd "$RUNNER_TEMP"
tag="$(gh release view --repo "$GITHUB_REPOSITORY" --json tagName -q .tagName)"
tool=openidea-plugin-linux-x86_64
if ! gh release download "$tag" --repo "$GITHUB_REPOSITORY" -p plugins-index.json -p "$tool.gz"; then
echo "::notice::$tag has no plugin index or openidea-plugin tool yet"
exit 0
fi
gunzip "$tool.gz" && chmod +x "$tool"
cp plugins-index.json before.json
"./$tool" merge plugins-index.json "$COMMUNITY_PLUGINS_INDEX" --keep openidea
if cmp -s before.json plugins-index.json; then
echo "unchanged"
else
gh release upload "$tag" plugins-index.json --repo "$GITHUB_REPOSITORY" --clobber
fi
+1 -10
View File
@@ -18,24 +18,15 @@ jobs:
if: github.event.pull_request.user.type != 'Bot' && !github.event.pull_request.draft
runs-on: ubuntu-latest
steps:
- uses: actions/github-script@v7
- uses: actions/github-script@v8
with:
script: |
const pr = context.payload.pull_request;
const LABEL = 'needs-info';
const MARKER = '<!-- pr-check -->';
// Docs, CI and pure refactors may skip the linked issue.
const NO_ISSUE_TYPES = /^(docs|ci|build|chore|refactor|style|test)(\(.+\))?!?:/;
const body = (pr.body || '').replace(/<!--[\s\S]*?-->/g, '');
const problems = [];
const linksIssue =
/\b(close[sd]?|fix(e[sd])?|resolve[sd]?)\s*:?\s+(#\d+|https:\/\/github\.com\/[\w.-]+\/[\w.-]+\/issues\/\d+)/i.test(body);
if (!linksIssue && !NO_ISSUE_TYPES.test(pr.title)) {
problems.push('Link the issue this PR resolves, e.g. `Closes #123`.');
}
const verified = body.match(/^##+\s*How it was verified\s*$([\s\S]*?)(?=^##+\s|(?![\s\S]))/im);
if (!verified || !verified[1].trim()) {
problems.push('Fill in **How it was verified**: the commands you ran and their results. UI changes need before/after screenshots.');
+43 -15
View File
@@ -18,6 +18,7 @@ env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
TAG: ${{ github.event.inputs.tag || github.ref_name }}
COMMUNITY_PLUGINS_INDEX: https://github.com/l0ng-ai/openidea-plugins/releases/download/index/index.json
jobs:
# Creates (or reuses) one draft release that every platform build uploads into.
@@ -25,8 +26,9 @@ jobs:
runs-on: ubuntu-latest
outputs:
release_id: ${{ steps.release.outputs.id }}
official_plugin_keys: ${{ steps.plugin-keys.outputs.keys }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5
with: { ref: "${{ env.TAG }}" }
- name: Tag matches app version
run: |
@@ -35,8 +37,22 @@ jobs:
echo "::error::tag $TAG does not match tauri.conf.json version $app"
exit 1
fi
# The official plugins' public key (from PLUGIN_SIGNING_KEY, the seed
# `openidea-plugin keygen` writes), built into the app: their packages
# must carry its signature whatever the marketplace index lists. Without
# the secret the app takes the index's word, as development builds do.
- id: plugin-keys
env:
SIGNING_KEY: ${{ secrets.PLUGIN_SIGNING_KEY }}
run: |
if [ -n "$SIGNING_KEY" ]; then
seed="$(printf %s "$SIGNING_KEY" | base64 -d | od -An -tx1 | tr -d ' \n')"
key="$(printf '302e020100300506032b657004220420%s' "$seed" | xxd -r -p |
openssl pkey -inform DER -pubout -outform DER | tail -c 32 | base64)"
echo "keys=$key" >> "$GITHUB_OUTPUT"
fi
- id: release
uses: actions/github-script@v7
uses: actions/github-script@v8
with:
script: |
const tag = process.env.TAG;
@@ -57,6 +73,8 @@ jobs:
build:
needs: create-release
name: Build (${{ matrix.name }})
env:
OPENIDEA_OFFICIAL_PLUGIN_KEYS: ${{ needs.create-release.outputs.official_plugin_keys }}
strategy:
fail-fast: false
matrix:
@@ -81,11 +99,11 @@ jobs:
target: x86_64-pc-windows-msvc
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5
with: { ref: "${{ env.TAG }}" }
- uses: pnpm/action-setup@v4
- uses: pnpm/action-setup@v5
with: { version: 11 }
- uses: actions/setup-node@v4
- uses: actions/setup-node@v5
with: { node-version: 22, cache: pnpm }
- uses: dtolnay/rust-toolchain@stable
with: { targets: "${{ matrix.target }}" }
@@ -155,7 +173,7 @@ jobs:
- { runner: macos-latest, target: x86_64-apple-darwin, os: macos, arch: x86_64 }
runs-on: ${{ matrix.runner }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5
with: { ref: "${{ env.TAG }}" }
- uses: dtolnay/rust-toolchain@stable
with: { targets: "${{ matrix.target }}" }
@@ -184,7 +202,11 @@ jobs:
# (releases/latest/download/plugins-index.json). They are not in the
# installers: users install them from Settings › Plugins. Without the
# signing key (secret PLUGIN_SIGNING_KEY: the file `openidea-plugin keygen`
# writes) nothing is published and the release goes on.
# writes) nothing is published and the release goes on. The index also lists
# the community plugins of the registry (COMMUNITY_PLUGINS_INDEX; refreshed
# between releases by plugins-index.yml), and the release carries the
# openidea-plugin tool for Linux x64, which that workflow and the registry's
# checks run.
plugins:
needs: create-release
name: Official plugins
@@ -198,7 +220,7 @@ jobs:
echo "::warning::PLUGIN_SIGNING_KEY is not set: the official plugins are not published"
echo "skip=true" >> "$GITHUB_OUTPUT"
fi
- uses: actions/checkout@v4
- uses: actions/checkout@v5
if: steps.key.outputs.skip != 'true'
with: { ref: "${{ env.TAG }}" }
- uses: dtolnay/rust-toolchain@stable
@@ -207,10 +229,10 @@ jobs:
- uses: Swatinem/rust-cache@v2
if: steps.key.outputs.skip != 'true'
with: { key: release-plugins }
- uses: pnpm/action-setup@v4
- uses: pnpm/action-setup@v5
if: steps.key.outputs.skip != 'true'
with: { version: 11 }
- uses: actions/setup-node@v4
- uses: actions/setup-node@v5
if: steps.key.outputs.skip != 'true'
with: { node-version: 22 }
# Builds the web pages; fails when a checked-in component isn't built from these sources.
@@ -237,11 +259,17 @@ jobs:
version="$(sed -n 's/^version *= *"\(.*\)".*/\1/p' "$dir/plugin.toml" | head -1)"
"$tool" pack "$dir" --key "$key" --publisher openidea --out "$out/$id-$version.oiplugin"
done
# The published index's revocations and earlier keys carry over.
# The published index's revocations, earlier keys and community plugins carry over.
curl -fsSL "https://github.com/$GITHUB_REPOSITORY/releases/latest/download/plugins-index.json" \
-o "$out/plugins-index.json" || rm -f "$out/plugins-index.json"
"$tool" index "$out" --publisher openidea --name OpenIDEA --key "$key" --out "$out/plugins-index.json"
gh release upload "$TAG" "$out"/*.oiplugin "$out/plugins-index.json" --clobber
# Downloads point at this release: older versions kept for older IDEs stay where they were published.
"$tool" index "$out" --publisher openidea --name OpenIDEA --key "$key" --out "$out/plugins-index.json" \
--download-base "https://github.com/$GITHUB_REPOSITORY/releases/download/$TAG"
"$tool" merge "$out/plugins-index.json" "$COMMUNITY_PLUGINS_INDEX" --keep openidea ||
echo "::warning::the community plugins were not refreshed"
gzip -9 -c "$tool" > "$out/openidea-plugin-linux-x86_64.gz"
gh release upload "$TAG" "$out"/*.oiplugin "$out/plugins-index.json" \
"$out/openidea-plugin-linux-x86_64.gz" --clobber
# The updater manifest (latest.json), built once from the signed bundles every build uploaded.
# Fails when a platform is missing, so a release never ships a manifest without it.
@@ -249,7 +277,7 @@ jobs:
needs: [create-release, build]
runs-on: ubuntu-latest
steps:
- uses: actions/github-script@v7
- uses: actions/github-script@v8
env:
RELEASE_ID: ${{ needs.create-release.outputs.release_id }}
with:
@@ -309,7 +337,7 @@ jobs:
needs: [create-release, build, server, plugins, updater-manifest]
runs-on: ubuntu-latest
steps:
- uses: actions/github-script@v7
- uses: actions/github-script@v8
env:
RELEASE_ID: ${{ needs.create-release.outputs.release_id }}
with:
+18 -7
View File
@@ -18,9 +18,12 @@ Visual style: dark, low-contrast greys, one cool accent
- `src/ui/` — design-system primitives. `src/features/<name>/` — one feature each,
exporting `activate()` registered in `src/features/index.ts`.
- `languages/<id>.toml` — language definitions (detection, comments, LSP, DAP).
- `crates/ide-host` — the `openidea-server` binary for remote development.
- `wit/` — plugin interface definitions; `plugins/` — sample and bundled plugins;
`templates/` — plugin starter templates.
- `crates/ide-host` — the `openidea-server` binary for remote development and the
`openidea-plugin` CLI (plugin starter templates live in its `templates.rs`).
- `wit/` — plugin interface definitions; `plugins/` — sample and bundled plugins.
Plugin docs: `plugins/README.md`, `docs/plugins/themes.md`.
- `java/` — JDT LS extension bundles (`java-usages`, `kotlin-bridge`) and the patched
Lombok agent, embedded into the app by `ide-sdk` / `ide-tools`.
- `e2e/` — Playwright UI tests (`pnpm e2e`). `fixtures/` — sample projects per language
for manual and automated testing.
@@ -39,14 +42,15 @@ Visual style: dark, low-contrast greys, one cool accent
icons from `src/ui/icons.tsx`, never emoji. Real `<button>`/`<input>`; `aria-label` on icon buttons.
- User-visible text goes through the text catalog: add every new string to both
`src/platform/locales/text/en.json` and `zh.json` (`textCatalog.test.ts` enforces it).
Other locales may lack entries; their UI shows the English text until a translator adds it.
- Keep comments sparse and purposeful; match surrounding style.
- For GUI verification, prefer Tauri MCP; use Computer Use only when Tauri MCP is
unavailable or cannot cover the required checks.
## Issues & PRs
Follow `CONTRIBUTING.md` and `.github/pull_request_template.md`: one issue per PR with
`Closes #<n>`, reproduce first, keep the diff focused, short plain description, and a
Follow `CONTRIBUTING.md` and `.github/pull_request_template.md`: one problem per PR
(`Closes #<n>` when it resolves an issue), reproduce first, keep the diff focused, short plain description, and a
**How it was verified** section with commands run, results and (for UI) before/after screenshots.
Unclear product decisions → `needs-decision` and stop.
@@ -63,6 +67,13 @@ Shared agent skills live in `.agents/skills/` (`.claude/skills/` links to them):
this worktree's running MCP bridges and actual ports.
- `pnpm build` — typecheck + bundle frontend.
- `cargo nextest run --workspace` (or `cargo test`), `cargo clippy --workspace --all-targets -- -D warnings`, `cargo fmt --all`.
- `cargo test -p ide-host --test language_smoke -- --ignored --exact <language>` — code intelligence
on `fixtures/` with that language's real server installed (CI: `Language smoke`, pinned versions).
- `pnpm smoke smoke/run.test.ts -t <language>` — run and test that language's fixture through the
run store and a real `openidea-server` (build it first: `cargo build -p ide-host --bin openidea-server`).
- `pnpm smoke smoke/debug.test.ts -t <language>` — debug that fixture's entry point the same way, with
the debugger the IDE installs (on macOS, CodeLLDB and Delve may need Developer Mode:
`sudo DevToolsSecurity -enable`).
- `pnpm test`, `pnpm e2e`, `pnpm lint`, `pnpm format:check`, `pnpm knip`, `pnpm typecheck`,
`pnpm gen:bindings`.
- `pnpm perf` (UI interaction latency / frames / leaks), `pnpm perf:rust` (criterion
@@ -110,8 +121,8 @@ Several worktrees can be developed and run side by side.
default MCP session belongs to this worktree.
- Keep each worktree's Cargo output directory separate; do not point `CARGO_TARGET_DIR` or
`target` symlinks at a shared directory.
- Each worktree's first build is cold (around 5 minutes) and its Cargo output takes
15–30 GB. Avoid running several heavy builds on one machine at the same time.
- Each worktree's first build is cold (several minutes) and `cargo build` output takes
around 9 GB; tests, clippy and incremental builds add more. Avoid running several heavy builds on one machine at the same time.
- Stop only processes started for the current worktree/session. Never use broad
process-name kills such as `pkill openidea`.
- When done: stop that worktree's app and dev session, delete its build output, then
+7 -4
View File
@@ -16,8 +16,7 @@ Project layout, coding rules, commands and checks are in [AGENTS.md](AGENTS.md).
## Pull requests
- One issue per PR, linked with `Closes #123`. Feature PRs need a `planned` issue.
Docs, CI and pure refactors may skip the issue.
- One problem per PR. If it resolves an issue, link it with `Closes #123`.
- Reproduce the problem first, or write a test that fails before the fix.
- Fix the cause, and check whether the same cause breaks anything nearby.
- Keep the diff focused. Unrelated refactors go in their own PR.
@@ -31,5 +30,9 @@ Project layout, coding rules, commands and checks are in [AGENTS.md](AGENTS.md).
- If a change needs a product decision you can't make, ask in the issue and stop
rather than guessing.
A check on each PR adds `needs-info` when the issue link or the verification
section is missing, and removes it once they're filled in.
Plugins: pull requests here add or change official plugins (`plugins/official/`)
and the plugin API. Community plugins live in their authors' repositories and
are listed through [l0ng-ai/openidea-plugins](https://github.com/l0ng-ai/openidea-plugins);
see [`plugins/README.md`](plugins/README.md) § 8.1.
A check on each PR adds `needs-info` when the verification section is missing, and removes it once it's filled in.
Generated
+375 -70
View File
@@ -123,6 +123,17 @@ version = "1.0.104"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470"
[[package]]
name = "apple-native-keyring-store"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b350bfd03649e07aa05c0a81b3e15934374e585c98204a57e20b9d49f49bb9a"
dependencies = [
"keyring-core",
"log",
"security-framework",
]
[[package]]
name = "arbitrary"
version = "1.5.0"
@@ -156,6 +167,27 @@ version = "0.7.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3fb67a6e08acf24fdeccbac2cb6ac4305825bd1f117462e0e6f2f193345ad56"
[[package]]
name = "ashpd"
version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2f3f79755c74fd155000314eb349864caa787c6592eace6c6882dad873d9c39"
dependencies = [
"enumflags2",
"futures-channel",
"futures-util",
"rand 0.9.5",
"raw-window-handle",
"serde",
"serde_repr",
"tokio",
"url",
"wayland-backend",
"wayland-client",
"wayland-protocols",
"zbus",
]
[[package]]
name = "ast-grep-core"
version = "0.40.1"
@@ -1464,7 +1496,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d045de693cb712d0b22c6a64be5b953f67b3ce00ab5ad3dd5d8b441886ab8e1a"
dependencies = [
"quote",
"syn 3.0.9",
"syn 2.0.119",
]
[[package]]
@@ -1600,6 +1632,7 @@ dependencies = [
"lock_api",
"once_cell",
"parking_lot_core",
"serde",
]
[[package]]
@@ -1795,6 +1828,15 @@ dependencies = [
"syn 3.0.9",
]
[[package]]
name = "dlib"
version = "0.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ab8ecd87370524b461f8557c119c405552c396ed91fc0a8eec68679eab26f94a"
dependencies = [
"libloading",
]
[[package]]
name = "dlopen2"
version = "0.8.2"
@@ -2163,15 +2205,6 @@ dependencies = [
"rustc_version",
]
[[package]]
name = "file-id"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e1fc6a637b6dc58414714eddd9170ff187ecb0933d4c7024d1abbd23a3cc26e9"
dependencies = [
"windows-sys 0.60.2",
]
[[package]]
name = "filedescriptor"
version = "0.8.3"
@@ -2968,7 +3001,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1c48eccd894bd15555b0346470e63491ded94aefe76b3fcb1052084b5a393238"
dependencies = [
"bstr",
"hashbrown 0.17.1",
"hashbrown 0.16.1",
]
[[package]]
@@ -4038,7 +4071,7 @@ checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
[[package]]
name = "ide-agent"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"dunce",
"encoding_rs",
@@ -4061,7 +4094,7 @@ dependencies = [
[[package]]
name = "ide-analysis"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ast-grep-core",
"ide-core",
@@ -4075,7 +4108,7 @@ dependencies = [
[[package]]
name = "ide-build"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"dirs 6.0.0",
"dunce",
@@ -4090,7 +4123,7 @@ dependencies = [
[[package]]
name = "ide-clipboard"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"objc2 0.6.5",
"objc2-app-kit 0.3.2",
@@ -4099,7 +4132,7 @@ dependencies = [
[[package]]
name = "ide-core"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"dunce",
"encoding_rs",
@@ -4116,11 +4149,11 @@ dependencies = [
[[package]]
name = "ide-coverage"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"ide-run",
"quick-xml",
"quick-xml 0.42.0",
"serde",
"serde_json",
"specta",
@@ -4130,7 +4163,7 @@ dependencies = [
[[package]]
name = "ide-dap"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"ide-jsonrpc",
@@ -4148,14 +4181,17 @@ dependencies = [
[[package]]
name = "ide-db"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"bb8",
"dirs 6.0.0",
"futures",
"ide-core",
"ide-http",
"ide-ssh",
"parking_lot",
"reqwest",
"rustls",
"serde",
"serde_json",
"specta",
@@ -4166,11 +4202,12 @@ dependencies = [
"tokio-util",
"toml 0.9.12+spec-1.1.0",
"tracing",
"uuid",
]
[[package]]
name = "ide-docker"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"serde",
@@ -4183,7 +4220,7 @@ dependencies = [
[[package]]
name = "ide-extension"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"base64 0.22.1",
"blake3",
@@ -4203,7 +4240,7 @@ dependencies = [
[[package]]
name = "ide-fonts"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"objc2-core-foundation",
"objc2-core-text",
@@ -4215,7 +4252,7 @@ dependencies = [
[[package]]
name = "ide-github"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"ide-vcs",
@@ -4228,7 +4265,7 @@ dependencies = [
[[package]]
name = "ide-host"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"base64 0.22.1",
"criterion",
@@ -4244,6 +4281,7 @@ dependencies = [
"ide-coverage",
"ide-dap",
"ide-db",
"ide-docker",
"ide-extension",
"ide-http",
"ide-index",
@@ -4267,8 +4305,11 @@ dependencies = [
"ide-workspace",
"ignore",
"parking_lot",
"quick-xml 0.42.0",
"regex-syntax",
"reqwest",
"rustix",
"semver",
"serde",
"serde_json",
"specta",
@@ -4285,7 +4326,7 @@ dependencies = [
[[package]]
name = "ide-http"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"parking_lot",
@@ -4303,7 +4344,7 @@ dependencies = [
[[package]]
name = "ide-index"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"blake3",
"criterion",
@@ -4325,7 +4366,7 @@ dependencies = [
[[package]]
name = "ide-jsonrpc"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"encoding_rs",
"futures",
@@ -4333,6 +4374,7 @@ dependencies = [
"parking_lot",
"serde",
"serde_json",
"tempfile",
"thiserror 2.0.21",
"tokio",
"tokio-util",
@@ -4341,7 +4383,7 @@ dependencies = [
[[package]]
name = "ide-language"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"blake3",
"criterion",
@@ -4397,7 +4439,7 @@ dependencies = [
[[package]]
name = "ide-lsp"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"dunce",
"futures",
@@ -4423,7 +4465,7 @@ dependencies = [
[[package]]
name = "ide-notebook"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"futures",
"ide-core",
@@ -4446,7 +4488,7 @@ dependencies = [
[[package]]
name = "ide-plugin"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"blake3",
"ide-core",
@@ -4462,11 +4504,12 @@ dependencies = [
"tracing",
"wasmtime",
"wasmtime-wasi",
"wat",
]
[[package]]
name = "ide-profiler"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"ide-run",
@@ -4479,7 +4522,7 @@ dependencies = [
[[package]]
name = "ide-remote"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"globset",
"ide-core",
@@ -4496,7 +4539,7 @@ dependencies = [
[[package]]
name = "ide-run"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"dunce",
"encoding_rs",
@@ -4505,7 +4548,7 @@ dependencies = [
"libc",
"parking_lot",
"portable-pty",
"quick-xml",
"quick-xml 0.42.0",
"serde",
"serde_json",
"specta",
@@ -4519,7 +4562,7 @@ dependencies = [
[[package]]
name = "ide-sdk"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"dunce",
"ide-core",
@@ -4538,7 +4581,7 @@ dependencies = [
[[package]]
name = "ide-search"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"criterion",
"globset",
@@ -4558,7 +4601,7 @@ dependencies = [
[[package]]
name = "ide-spell"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"parking_lot",
@@ -4571,11 +4614,13 @@ dependencies = [
[[package]]
name = "ide-ssh"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"dirs 6.0.0",
"flate2",
"ide-core",
"keyring",
"keyring-core",
"parking_lot",
"russh",
"russh-config",
@@ -4586,11 +4631,12 @@ dependencies = [
"tempfile",
"tokio",
"tracing",
"uuid",
]
[[package]]
name = "ide-syntax"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"ide-language",
@@ -4605,7 +4651,7 @@ dependencies = [
[[package]]
name = "ide-terminal"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"libc",
@@ -4618,19 +4664,20 @@ dependencies = [
[[package]]
name = "ide-text"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"criterion",
"ide-core",
"proptest",
"ropey",
"serde",
"similar",
"specta",
]
[[package]]
name = "ide-tools"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"flate2",
"hex",
@@ -4651,14 +4698,16 @@ dependencies = [
[[package]]
name = "ide-vcs"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"blake3",
"dunce",
"encoding_rs",
"gix",
"ide-core",
"ignore",
"libc",
"redb",
"regex-lite",
"serde",
"serde_json",
@@ -4670,7 +4719,7 @@ dependencies = [
[[package]]
name = "ide-vfs"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"chardetng",
"dunce",
@@ -4679,7 +4728,7 @@ dependencies = [
"globset",
"ide-core",
"ignore",
"notify-debouncer-full",
"notify",
"parking_lot",
"serde",
"serde_json",
@@ -4692,7 +4741,7 @@ dependencies = [
[[package]]
name = "ide-web"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"ide-core",
"open",
@@ -4708,7 +4757,7 @@ dependencies = [
[[package]]
name = "ide-workspace"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"blake3",
"dunce",
@@ -4726,6 +4775,7 @@ dependencies = [
"toml 0.9.12+spec-1.1.0",
"toml_edit 0.25.17+spec-1.1.0",
"tracing",
"uuid",
"zip",
]
@@ -5203,6 +5253,33 @@ dependencies = [
"serde",
]
[[package]]
name = "keyring"
version = "4.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2270074a3d26bcac93c1dc5d2845eb4c089e8d761ccf6e0ea266a16004640627"
dependencies = [
"apple-native-keyring-store",
"keyring-core",
"windows-native-keyring-store",
"zbus-secret-service-keyring-store",
]
[[package]]
name = "keyring-core"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "29fbf4b551298c67925f38cf341a1ada797a75f1837bdf8ae1fc8e2ebc6efb59"
dependencies = [
"chrono",
"dashmap",
"log",
"regex",
"ron",
"serde",
"uuid",
]
[[package]]
name = "kqueue"
version = "1.2.1"
@@ -5625,19 +5702,6 @@ dependencies = [
"windows-sys 0.60.2",
]
[[package]]
name = "notify-debouncer-full"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "375bd3a138be7bfeff3480e4a623df4cbfb55b79df617c055cd810ba466fa078"
dependencies = [
"file-id",
"log",
"notify",
"notify-types",
"walkdir",
]
[[package]]
name = "notify-types"
version = "2.1.0"
@@ -5666,6 +5730,30 @@ dependencies = [
"unicode-segmentation",
]
[[package]]
name = "num"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "35bd024e8b2ff75562e5f34e7f4905839deb4b22955ef5e73d2fea1b9813cb23"
dependencies = [
"num-bigint 0.4.8",
"num-complex",
"num-integer",
"num-iter",
"num-rational",
"num-traits",
]
[[package]]
name = "num-bigint"
version = "0.4.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c89e69e7e0f03bea5ef08013795c25018e101932225a656383bd384495ecc367"
dependencies = [
"num-integer",
"num-traits",
]
[[package]]
name = "num-bigint"
version = "0.5.1"
@@ -5678,6 +5766,15 @@ dependencies = [
"rand_core 0.10.1",
]
[[package]]
name = "num-complex"
version = "0.4.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "73f88a1307638156682bada9d7604135552957b7818057dcef22705b4d509495"
dependencies = [
"num-traits",
]
[[package]]
name = "num-conv"
version = "0.2.2"
@@ -5693,6 +5790,27 @@ dependencies = [
"num-traits",
]
[[package]]
name = "num-iter"
version = "0.1.46"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c92800bd69a1eac91786bcfe9da64a897eb72911b8dc3095decbd07429e8048b"
dependencies = [
"num-integer",
"num-traits",
]
[[package]]
name = "num-rational"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f83d14da390562dca69fc84082e73e548e1ad308d24accdedd2720017cb37824"
dependencies = [
"num-bigint 0.4.8",
"num-integer",
"num-traits",
]
[[package]]
name = "num-traits"
version = "0.2.19"
@@ -6180,7 +6298,7 @@ dependencies = [
[[package]]
name = "openidea"
version = "0.1.35"
version = "0.1.36"
dependencies = [
"base64 0.22.1",
"blake3",
@@ -6581,7 +6699,7 @@ checksum = "65dca778f41c79ebbe8b2b2139fee86f01b766b67130d108c7c2b8ffbd949f77"
dependencies = [
"base64 0.23.1",
"indexmap 2.14.2",
"quick-xml",
"quick-xml 0.42.0",
"serde",
"time",
]
@@ -6626,6 +6744,12 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "pollster"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2f3a9f18d041e6d0e102a0a46750538147e5e8992d3b4873aaafee2520b00ce3"
[[package]]
name = "poly1305"
version = "0.9.1"
@@ -6886,6 +7010,15 @@ version = "1.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a1d01941d82fa2ab50be1e79e6714289dd7cde78eba4c074bc5a4374f650dfe0"
[[package]]
name = "quick-xml"
version = "0.41.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e660451e55124f798a69a5af3f49ccfbefbd41910eefd25caf2393e1f3473ec1"
dependencies = [
"memchr",
]
[[package]]
name = "quick-xml"
version = "0.42.0"
@@ -7132,6 +7265,7 @@ dependencies = [
"rustls-platform-verifier",
"serde",
"serde_json",
"serde_urlencoded",
"sync_wrapper",
"tokio",
"tokio-rustls",
@@ -7162,18 +7296,18 @@ version = "0.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a15ad77d9e70a92437d8f74c35d99b4e4691128df018833e99f90bcd36152672"
dependencies = [
"ashpd",
"block2 0.6.2",
"dispatch2",
"glib-sys",
"gobject-sys",
"gtk-sys",
"js-sys",
"log",
"objc2 0.6.5",
"objc2-app-kit 0.3.2",
"objc2-core-foundation",
"objc2-foundation 0.3.2",
"pollster",
"raw-window-handle",
"urlencoding",
"wasm-bindgen",
"wasm-bindgen-futures",
"web-sys",
@@ -7194,6 +7328,19 @@ dependencies = [
"windows-sys 0.52.0",
]
[[package]]
name = "ron"
version = "0.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "97b3fe97c18ff24299ea56f802c5693eb34266fb0bc8317f4e101debc8a3ceaf"
dependencies = [
"bitflags 2.13.2",
"serde",
"serde_derive",
"typeid",
"unicode-ident",
]
[[package]]
name = "ropey"
version = "1.6.1"
@@ -7260,7 +7407,7 @@ dependencies = [
"md5",
"ml-kem",
"module-lattice",
"num-bigint",
"num-bigint 0.5.1",
"p256",
"p384",
"p521",
@@ -7490,6 +7637,12 @@ dependencies = [
"wait-timeout",
]
[[package]]
name = "ryu"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f"
[[package]]
name = "saa"
version = "5.6.2"
@@ -7585,6 +7738,12 @@ dependencies = [
"syn 2.0.119",
]
[[package]]
name = "scoped-tls"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e1cf6437eb19a8f4a6cc0f7dca544973b0b78843adbfeb3683d1a94a0024a294"
[[package]]
name = "scopeguard"
version = "1.2.0"
@@ -7635,6 +7794,25 @@ dependencies = [
"zeroize",
]
[[package]]
name = "secret-service"
version = "5.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5107b24b91445dd2aa449a258a1807b63240942157292354dc5bfdbeb8bc6db8"
dependencies = [
"aes",
"cbc",
"futures-util",
"getrandom 0.4.3",
"hkdf",
"hybrid-array",
"num",
"once_cell",
"serde",
"sha2 0.11.1",
"zbus",
]
[[package]]
name = "security-framework"
version = "3.7.0"
@@ -7794,6 +7972,18 @@ dependencies = [
"serde_core",
]
[[package]]
name = "serde_urlencoded"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3491c14715ca2294c4d6a88f15e84739788c1d030eed8c110436aafdaa2f3fd"
dependencies = [
"form_urlencoded",
"itoa",
"ryu",
"serde",
]
[[package]]
name = "serde_with"
version = "3.24.0"
@@ -8173,7 +8363,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0d204abcbdf8e88729306a8d0ca01a79d8a49969fe1f84696909d6dbc4321c1a"
dependencies = [
"foldhash",
"hashbrown 0.17.1",
"hashbrown 0.16.1",
]
[[package]]
@@ -9111,7 +9301,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom 0.3.4",
"getrandom 0.4.3",
"once_cell",
"rustix",
"windows-sys 0.61.2",
@@ -9299,6 +9489,7 @@ dependencies = [
"signal-hook-registry",
"socket2",
"tokio-macros",
"tracing",
"windows-sys 0.61.2",
]
@@ -10115,6 +10306,12 @@ version = "1.13.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c6f5d3c3b1bf09027a88a6bc961fc00497d651009560b5463668dc81b0fa87a8"
[[package]]
name = "unicode-width"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ca718471876992059f251e33aafc334346ada08042b7661dd1fe313a9bced557"
[[package]]
name = "universal-hash"
version = "0.6.1"
@@ -10635,6 +10832,88 @@ dependencies = [
"wasmtime",
]
[[package]]
name = "wast"
version = "258.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b912cfe5696dfaae736e344fc82082dbd63609abbaeca3926466c0b10d9fecc"
dependencies = [
"bumpalo",
"leb128fmt",
"memchr",
"unicode-width",
"wasm-encoder",
]
[[package]]
name = "wat"
version = "1.258.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3069154948f5737a8bcd5c55dcdf2c2b59ca776057732e49821167bde268ec6e"
dependencies = [
"wast",
]
[[package]]
name = "wayland-backend"
version = "0.3.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "38a91b4eaddff87b1cd1074985e3713da4af2c49742d1b356b2c01670a67a078"
dependencies = [
"cc",
"downcast-rs",
"rustix",
"scoped-tls",
"smallvec",
"wayland-sys",
]
[[package]]
name = "wayland-client"
version = "0.31.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3c36a0f861ad76d0901f2800b46321410d9f73f2ea88aac0650d86c32688073"
dependencies = [
"bitflags 2.13.2",
"rustix",
"wayland-backend",
"wayland-scanner",
]
[[package]]
name = "wayland-protocols"
version = "0.32.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "23d0c813de3daa2ed6520af85a3bd49b0e722a3078506899aa9686fea58dc4b6"
dependencies = [
"bitflags 2.13.2",
"wayland-backend",
"wayland-client",
"wayland-scanner",
]
[[package]]
name = "wayland-scanner"
version = "0.31.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "338e30461b3a2b67d70eb30a6d89f8e0c93a833e07d2ae89085cd070c4a00ac0"
dependencies = [
"proc-macro2",
"quick-xml 0.41.0",
"quote",
]
[[package]]
name = "wayland-sys"
version = "0.31.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d8eab23fefc9e41f8e841df4a9c707e8a8c4ed26e944ef69297184de2785e3be"
dependencies = [
"dlib",
"log",
"pkg-config",
]
[[package]]
name = "web-sys"
version = "0.3.106"
@@ -10920,6 +11199,19 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-native-keyring-store"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "063426e76fdec7438d56bb777f67e318a84a25c707b07e575cb8b78e10c028f8"
dependencies = [
"byteorder",
"keyring-core",
"regex",
"windows-sys 0.61.2",
"zeroize",
]
[[package]]
name = "windows-numerics"
version = "0.3.1"
@@ -11446,6 +11738,7 @@ dependencies = [
"rustix",
"serde",
"serde_repr",
"tokio",
"tracing",
"uds_windows",
"uuid",
@@ -11456,6 +11749,17 @@ dependencies = [
"zvariant",
]
[[package]]
name = "zbus-secret-service-keyring-store"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "74801d001b9e7729adb4f1825b67b398185fed424749aa3d8bacf70417137d9a"
dependencies = [
"keyring-core",
"secret-service",
"zbus",
]
[[package]]
name = "zbus_macros"
version = "5.19.0"
@@ -11661,6 +11965,7 @@ dependencies = [
"endi",
"enumflags2",
"serde",
"url",
"winnow 1.0.4",
"zcheapstr",
"zvariant_derive",
+11 -2
View File
@@ -5,7 +5,7 @@ members = ["crates/*", "src-tauri"]
exclude = ["sdk", "plugins"]
[workspace.package]
version = "0.1.35"
version = "0.1.36"
edition = "2024"
rust-version = "1.96"
license = "Apache-2.0"
@@ -76,7 +76,6 @@ ignore = "0.4"
base64 = "0.22"
globset = "0.4"
notify = "8"
notify-debouncer-full = "0.6"
trash = "5"
blake3 = "1"
dunce = "1"
@@ -86,6 +85,7 @@ zip = { version = "9", default-features = false, features = ["deflate"] }
# search, index, language
grep-searcher = "0.1"
grep-regex = "0.1"
regex-syntax = "0.8"
grep-matcher = "0.1"
nucleo-matcher = "0.3"
redb = "4"
@@ -105,6 +105,9 @@ sqlx = { version = "0.9", default-features = false, features = ["runtime-tokio",
tiberius = { version = "0.13", default-features = false, features = ["tds73", "rustls", "sql-browser-tokio"] }
bb8 = "0.9"
uuid = { version = "1", features = ["v4"] }
# OS credential stores
keyring = "4.2"
keyring-core = "1"
# testing
insta = { version = "1", features = ["yaml"] }
@@ -114,8 +117,14 @@ tempfile = "3"
[profile.dev]
# Keep debug builds snappy while giving dependencies decent speed.
opt-level = 0
# Full debug info is most of target/. Line tables keep file:line in our backtraces, but a
# debugger can't show locals; re-enable with `--config profile.dev.debug=true`.
debug = "line-tables-only"
[profile.dev.package."*"]
opt-level = 2
# Dependency frames keep function names only. Re-enable one with
# `--config 'profile.dev.package.<crate>.debug=true'`.
debug = false
[profile.release]
codegen-units = 1
+12
View File
@@ -14,6 +14,8 @@
&nbsp;·&nbsp;
<a href="#benchmark">Benchmark</a>
&nbsp;·&nbsp;
<a href="#plugins">Plugins</a>
&nbsp;·&nbsp;
<a href="https://discord.gg/2Zy4yn7gP">Discord</a>
</p>
@@ -45,6 +47,16 @@ IDEA analyses Python more deeply, so "ready" doesn't mean the same work on
both sides. Both are measured from the outside, the same way: see
[`perf/compare`](perf/compare) to run it on your machine and projects.
## Plugins
Plugins add actions, completion, quick fixes, tool windows, custom editors,
themes, file icons and languages. They run as sandboxed WebAssembly components;
themes, icon themes and languages need no code at all.
Install them from Settings › Plugins, or start your own with
`openidea-plugin new` — see [Writing OpenIDEA plugins](plugins/README.md) and
the [official plugins](plugins/official). The plugin API is still a preview.
## License
[Apache 2.0](LICENSE). Bundled third-party assets are listed in [`THIRD_PARTY_LICENSES.md`](THIRD_PARTY_LICENSES.md).
+12 -3
View File
@@ -2,12 +2,12 @@
## Project Lombok
`plugins/lombok/lombok-1.18.48-openidea.2.jar` contains Project Lombok v1.18.48 with the Equinox
`java/lombok/lombok-1.18.48-openidea.2.jar` contains Project Lombok v1.18.48 with the Equinox
classloader fix from [projectlombok/lombok#4059](https://github.com/projectlombok/lombok/pull/4059)
and the ECJ constructor-call fix from [projectlombok/lombok#4089](https://github.com/projectlombok/lombok/pull/4089).
Copyright (C) 2009-2021 The Project Lombok Authors. MIT license and bundled component
notices are retained in [plugins/lombok/LICENSE](plugins/lombok/LICENSE) and inside the jar.
Source tag, patch commits and build instructions: [plugins/lombok/README.md](plugins/lombok/README.md).
notices are retained in [java/lombok/LICENSE](java/lombok/LICENSE) and inside the jar.
Source tag, patch commits and build instructions: [java/lombok/README.md](java/lombok/README.md).
## JetBrains IntelliJ Platform icons
@@ -52,6 +52,15 @@ Their original copyright headers and Apache-2.0 licensing are retained.
At render time `src/ui/icons.tsx` may recolour monochrome icons to `currentColor`.
## JetBrains IntelliJ Platform VCS log graph algorithms
`crates/ide-vcs/src/bek.rs` and `src/features/vcs/graph.ts` adapt the Bek ordering,
graph layout and LinearBek merge linearization algorithms from
[intellij-community](https://github.com/JetBrains/intellij-community/tree/1d70d5c54302d655758edbd2f808fb8dea8b1764/platform/vcs-log/graph/src/com/intellij/vcs/log/graph)
(`impl/facade/sort/bek`, `impl/permanent/GraphLayoutBuilder.kt` and
`linearBek/LinearBekGraphBuilder.java`). Modified from the originals.
Copyright 2000-2026 JetBrains s.r.o. and contributors. Apache License 2.0 (below).
## JetBrains IntelliJ Platform diff algorithm
`src/features/vcs/comparison/**` is a TypeScript port of the text comparison of
+21 -5
View File
@@ -58,6 +58,8 @@ pub(crate) const LIMITS_UPDATE: &str = "openidea_limits";
const MAX_EMBED: u64 = 256 * 1024;
/// How long a probe waits for the agent to answer `initialize`.
const PROBE_TIMEOUT: Duration = Duration::from_secs(30);
/// How long deleting a task waits for its agent to exit before removing the worktree.
const EXIT_WAIT: Duration = Duration::from_secs(5);
/// The agent tasks of one project.
pub struct AgentManager {
@@ -345,6 +347,14 @@ impl AgentManager {
provider::set_builtin_command(&dir, id, command.as_deref())
}
/// Sets the environment variables built-in `id` starts with; empty clears them.
pub fn set_builtin_env(&self, id: &str, env: std::collections::BTreeMap<String, String>) -> IdeResult<()> {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::set_builtin_env(&dir, id, env)
}
/// Re-reads configured/installed agents, drops cached probes, and notifies subscribers.
pub fn reload_providers(&self) {
let specs = provider::load(self.shared.paths.custom_agents_dir.as_deref());
@@ -831,7 +841,7 @@ impl AgentManager {
/// Forgets the task; with `remove_worktree` also deletes its worktree (the branch stays).
pub async fn delete(&self, id: &str, remove_worktree: bool) -> IdeResult<()> {
let task = self.task(id)?;
task.kill();
let live = task.kill();
let (isolation, cwd, root) = {
let s = task.state.lock();
(
@@ -843,6 +853,11 @@ impl AgentManager {
// A copy of the task (a fork) may work in the same worktree.
let shared = self.tasks.read().values().any(|t| t.id != id && t.state.lock().task.cwd.0 == cwd);
if remove_worktree && !shared && isolation == Isolation::Worktree && cwd.exists() {
// On Windows the agent's processes lock the folder (their working directory,
// open files) until they have exited.
if let Some(peer) = live {
let _ = tokio::time::timeout(EXIT_WAIT, peer.exited()).await;
}
let worktrees = self.shared.paths.worktrees_dir.clone();
blocking(move || worktree::remove(&root, &worktrees, &cwd)).await?;
}
@@ -1597,11 +1612,12 @@ impl Task {
}
}
fn kill(&self) {
/// Stops the agent; returns its connection, to wait for its process to exit.
fn kill(&self) -> Option<acp::Conn> {
self.cancel_permissions();
if let Some(live) = self.live.lock().take() {
live.peer.kill();
}
let live = self.live.lock().take()?;
live.peer.kill();
Some(live.peer)
}
}
+3
View File
@@ -3,6 +3,7 @@
use ide_core::IpcPath;
use serde::{Deserialize, Serialize};
use specta::Type;
use std::collections::BTreeMap;
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
@@ -357,6 +358,8 @@ pub struct AgentProvider {
pub command_read_only_reason: Option<AgentCommandReadOnlyReason>,
/// Whether the command was found (on `PATH`).
pub installed: bool,
/// Environment variables it starts with (from its agent file).
pub env: BTreeMap<String, String>,
/// Settings (models, modes…) the agent offered last time it ran; empty before its first session.
pub config: Vec<AgentConfigOption>,
/// Slash commands it offered last time.
+30 -2
View File
@@ -124,6 +124,7 @@ pub(crate) fn describe(spec: &ProviderSpec) -> AgentProvider {
command_editable: reason.is_none(),
command_read_only_reason: reason,
installed: installed(&spec.command),
env: spec.env.iter().cloned().collect(),
config: vec![],
commands: vec![],
source: spec.source.clone(),
@@ -133,6 +134,16 @@ pub(crate) fn describe(spec: &ProviderSpec) -> AgentProvider {
/// Overrides built-in `id`'s command through its agent file, preserving other
/// customizations. `None` restores the built-in command and removes a redundant file.
pub(crate) fn set_builtin_command(custom_dir: &Path, id: &str, command: Option<&str>) -> IdeResult<()> {
update_builtin(custom_dir, id, |agent, builtin| agent.command = command.unwrap_or(&builtin.command).to_owned())
}
/// The environment built-in `id` starts with, through its agent file (empty clears it).
pub(crate) fn set_builtin_env(custom_dir: &Path, id: &str, env: BTreeMap<String, String>) -> IdeResult<()> {
update_builtin(custom_dir, id, |agent, _| agent.env = env)
}
/// Changes built-in `id`'s agent file; a file left with only the built-in's values is removed.
fn update_builtin(custom_dir: &Path, id: &str, change: impl FnOnce(&mut CustomAgent, &ProviderSpec)) -> IdeResult<()> {
let Some(builtin) = builtin_spec(id) else {
return Err(IdeError::invalid(format!("{id} is not a built-in agent")));
};
@@ -152,7 +163,8 @@ pub(crate) fn set_builtin_command(custom_dir: &Path, id: &str, command: Option<&
if agent.registry.is_some() {
return Err(IdeError::invalid(format!("{id} is installed from the ACP Registry")));
}
if command.is_none()
change(&mut agent, &builtin);
if agent.command == builtin.command
&& agent.name.as_deref() == Some(builtin.name.as_str())
&& agent.driver == builtin.driver
&& agent.args == builtin.args
@@ -164,7 +176,6 @@ pub(crate) fn set_builtin_command(custom_dir: &Path, id: &str, command: Option<&
}
return Ok(());
}
agent.command = command.unwrap_or(&builtin.command).to_owned();
agent.write(&file)?;
Ok(())
}
@@ -368,6 +379,23 @@ mod tests {
assert!(installed("sh"));
}
#[test]
fn set_builtin_env_writes_and_clears_the_environment() {
let dir = tempfile::tempdir().unwrap();
let env = BTreeMap::from([("CLAUDE_CODE_ENTRYPOINT".to_owned(), "openidea".to_owned())]);
set_builtin_env(dir.path(), "claude", env.clone()).unwrap();
let claude = load(Some(dir.path())).into_iter().find(|s| s.id == "claude").unwrap();
assert_eq!(claude.env, vec![("CLAUDE_CODE_ENTRYPOINT".to_owned(), "openidea".to_owned())]);
assert_eq!(describe(&claude).env, env);
// The command override and the environment are kept independently.
set_builtin_command(dir.path(), "claude", Some("/opt/claude")).unwrap();
set_builtin_command(dir.path(), "claude", None).unwrap();
assert!(dir.path().join("claude.toml").is_file());
set_builtin_env(dir.path(), "claude", BTreeMap::new()).unwrap();
assert!(!dir.path().join("claude.toml").exists());
set_builtin_env(dir.path(), "not-a-builtin", env).unwrap_err();
}
#[test]
fn set_builtin_command_overrides_and_clears_a_builtin() {
let dir = tempfile::tempdir().unwrap();
+3
View File
@@ -76,6 +76,8 @@ pub struct RegistryAgent {
pub installed: Option<String>,
/// Whether [`Self::version`] is newer than the installed one.
pub update: bool,
/// An install of it is running (it goes on after the Settings page that started it closes).
pub installing: bool,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Type)]
@@ -332,6 +334,7 @@ fn visible(
distribution,
no_checksum,
installed,
installing: false,
})
})
.collect();
+111
View File
@@ -1109,3 +1109,114 @@ async fn idle_agents_stop_and_resume_their_session_on_the_next_message() {
assert!(live(&mgr, &task.id));
mgr.shutdown();
}
/// Deleting a task whose agent still runs in the worktree removes both and leaves no
/// process behind, whether it is waiting for permission or idle without session/load.
#[tokio::test(flavor = "multi_thread")]
async fn deleting_a_task_waits_for_its_agent_to_exit_before_removing_the_worktree() {
for no_load in [false, true] {
delete_running_agent(no_load).await;
}
}
async fn delete_running_agent(no_load: bool) {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
let git = |args: &[&str]| {
let out = std::process::Command::new("git")
.args(["-c", "user.name=t", "-c", "user.email=t@t", "-c", "commit.gpgsign=false"])
.args(args)
.current_dir(&root)
.output()
.unwrap();
assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr));
};
git(&["init", "-q", "-b", "main"]);
std::fs::write(root.join("file.txt"), "original\n").unwrap();
git(&["add", "."]);
git(&["commit", "-qm", "initial"]);
let agent = Arc::new(FakeAgent { no_load, ..FakeAgent::default() });
let pids: Arc<Mutex<Vec<u32>>> = Arc::default();
let (fake, started) = (agent.clone(), pids.clone());
// The fake agent, served on behalf of a real process that exits only when killed.
let connector: Connector = Arc::new(move |_spec, cwd, client, _stderr| {
#[cfg(unix)]
let mut cmd = {
let mut cmd = tokio::process::Command::new("sleep");
cmd.arg("30");
cmd
};
#[cfg(windows)]
let mut cmd = {
let mut cmd = tokio::process::Command::new("cmd");
cmd.args(["/D", "/C", "echo ready>agent-ready & ping -n 30 127.0.0.1 >nul"]);
cmd
};
ide_core::process::prepare(cmd.as_std_mut());
ide_core::process::own_group(cmd.as_std_mut());
cmd.current_dir(cwd).kill_on_drop(true);
let child = cmd.spawn()?;
started.lock().push(child.id().unwrap());
let (a, b) = duplex(1 << 20);
let (ar, aw) = tokio::io::split(a);
let (br, bw) = tokio::io::split(b);
// `connect_with_child` frames with Content-Length.
fake.servers.lock().push(Peer::connect(br, bw, fake.clone()));
Ok(Peer::connect_with_child(ar, aw, client, child))
});
let mgr = AgentManager::with_connector(
AgentPaths {
project_root: root.clone(),
store_dir: root.join("store"),
worktrees_dir: root.join("worktrees"),
models_file: root.join("models.json"),
custom_agents_dir: None,
home_dir: None,
},
tokio::runtime::Handle::current(),
connector,
);
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text(if no_load { "again" } else { "Fix the lexer" }),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Worktree,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, if no_load { AgentStatus::Ready } else { AgentStatus::NeedsInput }).await;
let pid = *pids.lock().last().expect("the agent was started");
#[cfg(windows)]
{
tokio::time::timeout(Duration::from_secs(5), async {
while !task.cwd.0.join("agent-ready").exists() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await
.unwrap();
assert_eq!(
std::fs::rename(&task.cwd.0, task.cwd.0.with_extension("moved")).unwrap_err().raw_os_error(),
Some(32),
"the running agent really locks its worktree"
);
}
mgr.delete(&task.id, true).await.unwrap();
assert!(!task.cwd.0.exists());
// Killed and reaped: not even a zombie is left.
#[cfg(unix)]
let alive = std::process::Command::new("kill").args(["-0", &pid.to_string()]).status().unwrap().success();
#[cfg(windows)]
let alive = {
let mut cmd = std::process::Command::new("tasklist");
ide_core::process::prepare(&mut cmd).args(["/FI", &format!("PID eq {pid}"), "/FO", "CSV", "/NH"]);
String::from_utf8_lossy(&cmd.output().unwrap().stdout).contains(&format!("\"{pid}\""))
};
assert!(!alive, "the agent (pid {pid}) is still running");
}
+70 -2
View File
@@ -5,7 +5,7 @@ use ide_core::{IdeError, IdeResult, IpcPath};
use ide_vcs::{ChangeKind, Repo};
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use std::time::UNIX_EPOCH;
use std::time::{Duration, UNIX_EPOCH};
pub(crate) struct Created {
pub path: PathBuf,
@@ -65,7 +65,27 @@ pub(crate) fn remove(project_root: &Path, worktrees_dir: &Path, path: &Path) ->
if !path.starts_with(worktrees_dir) || path == worktrees_dir {
return Err(err);
}
std::fs::remove_dir_all(path).map_err(|e| IdeError::internal(e.to_string()))
retrying(REMOVE_ATTEMPTS, REMOVE_PAUSE, || std::fs::remove_dir_all(path))
.map_err(|e| IdeError::internal(format!("{}: {e}", path.display())))
}
/// On Windows a folder can't be deleted while a process still has a file in it open
/// (os error 32): an agent's helper that outlived it, an antivirus scan, the search indexer.
/// Those let go within moments, so removing is retried for a few seconds.
const REMOVE_ATTEMPTS: u32 = 10;
const REMOVE_PAUSE: Duration = Duration::from_millis(300);
fn retrying(attempts: u32, pause: Duration, mut op: impl FnMut() -> std::io::Result<()>) -> std::io::Result<()> {
for _ in 1..attempts {
match op() {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => std::thread::sleep(pause),
_ => return Ok(()),
}
}
match op() {
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()),
r => r,
}
}
/// Files under `cwd` that differ from `base` (untracked files included), with line counts.
@@ -149,6 +169,54 @@ fn unique_slug(base: &str, taken: impl Fn(&str) -> bool) -> String {
mod tests {
use super::*;
#[test]
fn removing_is_retried_while_a_file_is_still_in_use() {
use std::io::{Error, ErrorKind};
// A sharing violation as Windows reports it (os error 32), a vanished folder, other errors.
let in_use = || Error::from_raw_os_error(32);
for (fails, outcome) in [(0, true), (3, true), (REMOVE_ATTEMPTS - 1, true), (REMOVE_ATTEMPTS, false)] {
let mut calls = 0;
let r = retrying(REMOVE_ATTEMPTS, Duration::ZERO, || {
calls += 1;
if calls <= fails { Err(in_use()) } else { Ok(()) }
});
assert_eq!(r.is_ok(), outcome, "{fails} failures");
assert_eq!(calls, (fails + 1).min(REMOVE_ATTEMPTS));
}
let mut calls = 0;
let gone = retrying(REMOVE_ATTEMPTS, Duration::ZERO, || {
calls += 1;
if calls == 1 { Err(in_use()) } else { Err(Error::from(ErrorKind::NotFound)) }
});
assert!(gone.is_ok(), "deleted meanwhile: done");
assert_eq!(calls, 2);
}
#[cfg(windows)]
#[test]
fn removing_waits_for_a_windows_file_lock_to_be_released() {
use std::os::windows::fs::OpenOptionsExt;
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
std::fs::write(repo.join("a.txt"), "one\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
let worktrees = tmp.path().join("wt");
let created = create(&repo, &worktrees, "Locked file", None).unwrap();
let file = std::fs::OpenOptions::new().read(true).share_mode(0).open(created.path.join("a.txt")).unwrap();
assert_eq!(std::fs::remove_file(created.path.join("a.txt")).unwrap_err().raw_os_error(), Some(32));
let release = std::thread::spawn(move || {
std::thread::sleep(Duration::from_millis(800));
drop(file);
});
let result = remove(&repo, &worktrees, &created.path);
release.join().unwrap();
result.unwrap();
assert!(!created.path.exists());
}
#[test]
fn slugs_are_branch_safe() {
assert_eq!(slugify("Fix lexer UTF-16 columns!"), "fix-lexer-utf-16-columns");
+3 -3
View File
@@ -40,7 +40,7 @@ pub fn plan(root: &Path, model: &BuildModel, module: &Path, cache: &Path) -> Ide
kind: "maven",
});
}
if let Some(g) = &model.gradle
if let Some(g) = model.gradle_for_module(root, &module)
&& let Some(p) = g.projects.iter().find(|p| root.join(&p.dir) == module)
{
let script = cache.join("resources.init.gradle");
@@ -163,11 +163,11 @@ mod tests {
assert!(maven.args.contains(&"-Dexpression=project.build.outputDirectory".into()));
assert_eq!(maven.env["JAVA_HOME"], "/jdk");
model.maven = None;
model.gradle = Some(crate::GradleModel {
model.gradle = vec![crate::GradleModel {
command: "/tools/gradlew".into(),
projects: vec![crate::GradleProject { path: ":app".into(), dir: "app".into(), ..Default::default() }],
..Default::default()
});
}];
let gradle = plan(&root, &model, Path::new("app"), &cache).unwrap();
assert_eq!(gradle.command, "/tools/gradlew");
assert_eq!(gradle.args.last().unwrap(), ":app:openideaUpdateResources");
+201 -31
View File
@@ -10,6 +10,7 @@ pub use fingerprint::{LaunchFingerprint, launch_fingerprint};
use crate::process::{self, Stream};
use crate::{BuildModel, maven};
use ide_core::settings::CMakeProfile;
use ide_core::{CancellationToken, Diagnostic, IdeError, IdeResult, Pos, Range, Severity};
use parse::{ColUnit, Format, OutputParser, RawMessage};
use serde::{Deserialize, Serialize};
@@ -41,6 +42,8 @@ pub enum BuildSystem {
Go,
#[serde(rename = "typescript")]
TypeScript,
#[serde(rename = "cmake")]
CMake,
}
/// One command of a build.
@@ -144,7 +147,9 @@ fn detect_dir(dir: &Path, model: &BuildModel, root: &Path) -> Option<BuildSystem
let rel = rel_str(root, dir);
if model.maven.as_ref().is_some_and(|m| m.projects.iter().any(|p| p.dir == rel)) || has(dir, &["pom.xml"]) {
Some(BuildSystem::Maven)
} else if has(dir, &["build.gradle", "build.gradle.kts", "settings.gradle", "settings.gradle.kts"]) {
} else if model.gradle_for_module(root, dir).is_some()
|| has(dir, &["build.gradle", "build.gradle.kts", "settings.gradle", "settings.gradle.kts"])
{
Some(BuildSystem::Gradle)
} else if has(dir, &["Cargo.toml"]) {
Some(BuildSystem::Cargo)
@@ -152,21 +157,20 @@ fn detect_dir(dir: &Path, model: &BuildModel, root: &Path) -> Option<BuildSystem
Some(BuildSystem::Go)
} else if has(dir, &["tsconfig.json", "package.json"]) {
Some(BuildSystem::TypeScript)
} else if has(dir, &["CMakeLists.txt"]) {
Some(BuildSystem::CMake)
} else {
None
}
}
/// `root/rel`, without a trailing separator for the root itself.
/// `root` / `rel`, joined per `/`-separated component: on Windows a single join of
/// `a/b` keeps the `/`, and such a path does not compare equal to the same directory
/// as other tools (JDT LS) spell it.
/// Resolves a workspace-relative or absolute directory with native path separators.
fn subdir(root: &Path, rel: &str) -> PathBuf {
rel.split('/').filter(|part| !part.is_empty()).fold(root.to_path_buf(), |path, part| path.join(part))
root.join(rel).components().collect()
}
fn rel_str(root: &Path, dir: &Path) -> String {
dir.strip_prefix(root).map(|r| r.to_string_lossy().replace('\\', "/")).unwrap_or_default()
dir.strip_prefix(root).unwrap_or(dir).to_string_lossy().replace('\\', "/")
}
/// Directories from `file`'s folder up to `root` (inclusive).
@@ -176,11 +180,25 @@ fn ancestors_within<'a>(root: &'a Path, file: &'a Path) -> impl Iterator<Item =
}
/// The build of `root`, or of the module containing `file` (Build Module / Recompile).
pub fn plan(root: &Path, model: &BuildModel, kind: BuildKind, file: Option<&Path>) -> IdeResult<BuildPlan> {
/// `cmake`: the CMake profiles of the settings (the first one is built), if any.
pub fn plan(
root: &Path,
model: &BuildModel,
kind: BuildKind,
file: Option<&Path>,
cmake: Option<&[CMakeProfile]>,
) -> IdeResult<BuildPlan> {
let project_name = root.file_name().map_or_else(|| "project".to_owned(), |n| n.to_string_lossy().into_owned());
let module_dir = match file {
Some(f) => {
let dir = ancestors_within(root, f)
let scope = model
.gradle
.iter()
.flat_map(|g| std::iter::once(root.join(&g.dir)).chain(g.projects.iter().map(|p| root.join(&p.dir))))
.filter(|dir| f.starts_with(dir))
.max_by_key(|dir| dir.components().count())
.unwrap_or_else(|| root.to_path_buf());
let dir = ancestors_within(&scope, f)
.find(|d| detect_dir(d, model, root).is_some())
.ok_or_else(|| IdeError::not_found(format!("{} is not in a module", f.display())))?;
Some(dir.to_path_buf())
@@ -190,9 +208,11 @@ pub fn plan(root: &Path, model: &BuildModel, kind: BuildKind, file: Option<&Path
let dir = module_dir.clone().unwrap_or_else(|| root.to_path_buf());
// A folder of sibling projects has no build file itself; its Maven projects are built one by one.
let system = detect_dir(&dir, model, root)
.or_else(|| if model.gradle.is_some() { Some(BuildSystem::Gradle) } else { None })
.or((!model.gradle.is_empty()).then_some(BuildSystem::Gradle))
.or_else(|| model.maven.as_ref().filter(|m| !m.projects.is_empty()).map(|_| BuildSystem::Maven))
.ok_or_else(|| IdeError::not_found("no supported build system (Maven, Gradle, Cargo, Go, TypeScript) found"))?;
.ok_or_else(|| {
IdeError::not_found("no supported build system (Maven, Gradle, Cargo, Go, TypeScript, CMake) found")
})?;
let module = module_dir.as_deref().filter(|d| *d != root);
let title = match module {
Some(d) => format!(
@@ -209,6 +229,7 @@ pub fn plan(root: &Path, model: &BuildModel, kind: BuildKind, file: Option<&Path
// Go builds the file's package (a module holds many).
BuildSystem::Go => go_steps(file.and_then(Path::parent).unwrap_or(&dir), kind),
BuildSystem::TypeScript => ts_steps(&dir, kind),
BuildSystem::CMake => cmake_steps(&dir, kind, cmake.and_then(<[_]>::first)),
};
Ok(BuildPlan { system, title, steps })
}
@@ -255,14 +276,16 @@ fn maven_steps(root: &Path, model: &BuildModel, kind: BuildKind, module: Option<
}
fn gradle_steps(root: &Path, model: &BuildModel, kind: BuildKind, module: Option<&Path>) -> IdeResult<Vec<BuildStep>> {
let fallback;
let g = match &model.gradle {
Some(g) => g,
None => {
fallback = crate::gradle::load(root).ok_or_else(|| IdeError::not_found("no Gradle build found"))?;
&fallback
}
};
if module.is_none() && !model.gradle.is_empty() {
return Ok(model.gradle.iter().flat_map(|g| gradle_step(root, g, kind, None)).collect());
}
let g = module
.and_then(|m| model.gradle_for_module(root, m))
.ok_or_else(|| IdeError::not_found("module does not belong to a linked Gradle build"))?;
Ok(gradle_step(root, g, kind, module))
}
fn gradle_step(root: &Path, g: &crate::GradleModel, kind: BuildKind, module: Option<&Path>) -> Vec<BuildStep> {
let prefix = module
.and_then(|m| {
let rel = rel_str(root, m);
@@ -281,7 +304,7 @@ fn gradle_steps(root: &Path, model: &BuildModel, kind: BuildKind, module: Option
let mut s = BuildStep::new(&g.command, &["--console=plain"], &subdir(root, &g.dir), Format::Gradle);
s.args.extend(tasks);
s.env = g.env.clone();
Ok(vec![s])
vec![s]
}
fn cargo_steps(dir: &Path, kind: BuildKind) -> Vec<BuildStep> {
@@ -302,6 +325,50 @@ fn cargo_steps(dir: &Path, kind: BuildKind) -> Vec<BuildStep> {
}
}
/// CLion's Build Project for a CMake project: configures the build directory when it has no
/// `CMakeCache.txt` yet, then `cmake --build` (`--clean-first` to rebuild). The directory and
/// options come from the CMake profile; without one, a configured directory of the project
/// (the first by name), else `build`, as the generated run configurations use.
fn cmake_steps(dir: &Path, kind: BuildKind, profile: Option<&CMakeProfile>) -> Vec<BuildStep> {
let build_dir = match profile {
Some(p) if !p.build_dir.trim().is_empty() => p.build_dir.trim().to_owned(),
Some(p) => format!("cmake-build-{}", p.build_type.to_lowercase()),
None => {
let mut configured: Vec<String> = std::fs::read_dir(dir)
.into_iter()
.flatten()
.flatten()
.filter(|e| e.path().join("CMakeCache.txt").is_file())
.map(|e| e.file_name().to_string_lossy().into_owned())
.collect();
configured.sort();
configured.into_iter().next().unwrap_or_else(|| "build".into())
}
};
let build_type = profile.map_or("Debug", |p| p.build_type.as_str());
let env = profile.map(|p| p.env.clone()).unwrap_or_default();
let mut steps = Vec::new();
if !dir.join(&build_dir).join("CMakeCache.txt").is_file() {
let mut configure = BuildStep::new("cmake", &["-S", ".", "-B", &build_dir], dir, Format::Gcc);
if let Some(p) = profile.filter(|p| !p.generator.trim().is_empty()) {
configure.args.extend(["-G".into(), p.generator.clone()]);
}
configure.args.push(format!("-DCMAKE_BUILD_TYPE={build_type}"));
configure.args.push("-DCMAKE_EXPORT_COMPILE_COMMANDS=ON".into());
configure.args.extend(profile.map(|p| p.cmake_args.clone()).unwrap_or_default());
configure.env = env.clone();
steps.push(configure);
}
let mut build = BuildStep::new("cmake", &["--build", &build_dir, "--config", build_type], dir, Format::Gcc);
if kind == BuildKind::Rebuild {
build.args.push("--clean-first".into());
}
build.args.extend(profile.map(|p| p.build_args.clone()).unwrap_or_default());
build.env = env;
steps.push(build);
steps
}
fn go_steps(dir: &Path, kind: BuildKind) -> Vec<BuildStep> {
let module_root = dir.ancestors().find(|d| d.join("go.mod").is_file()).unwrap_or(dir);
let target = if module_root == dir { "./...".to_owned() } else { format!("./{}/...", rel_str(module_root, dir)) };
@@ -687,7 +754,7 @@ fn launch_plan(root: &Path, model: &BuildModel, module: &Path, test: bool) -> Id
} else {
BuildKind::Build
};
let mut plan = plan(root, model, kind, Some(module))?;
let mut plan = plan(root, model, kind, Some(module), None)?;
if plan.system == BuildSystem::Maven && !test {
for step in &mut plan.steps {
if let Some(goal) = step.args.last_mut() {
@@ -752,6 +819,77 @@ fn launch_class_dirs(system: BuildSystem, module: &Path, test: bool) -> Vec<Path
#[cfg(test)]
mod tests {
#[test]
fn independent_gradle_builds_keep_their_launchers_and_task_paths() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path();
for dir in ["a", "b"] {
std::fs::create_dir_all(root.join(dir).join("app/src/main/java")).unwrap();
std::fs::write(root.join(dir).join("settings.gradle"), "include 'app'").unwrap();
std::fs::write(root.join(dir).join("app/build.gradle"), "plugins { id 'java' }").unwrap();
std::fs::write(root.join(dir).join("gradlew"), "").unwrap();
}
let settings = ide_core::settings::GradleSettings {
linked_projects: Some(vec!["a".into(), "b".into()]),
..Default::default()
};
let mut model = crate::load_linked(root, &root.join("cache"), None, &Default::default(), &settings);
model.gradle[0].env.insert("JAVA_HOME".into(), "/jdk-a".into());
model.gradle[1].env.insert("JAVA_HOME".into(), "/jdk-b".into());
let all = super::plan(root, &model, super::BuildKind::Build, None, None).unwrap();
assert_eq!(all.steps.len(), 2);
assert_eq!(all.steps[0].cwd, root.join("a"));
assert_eq!(all.steps[1].cwd, root.join("b"));
let module = super::plan(root, &model, super::BuildKind::Rebuild, Some(&root.join("b/app")), None).unwrap();
assert_eq!(module.steps.len(), 1);
assert_eq!(module.steps[0].program, root.join("b/gradlew").to_string_lossy());
assert_eq!(module.steps[0].cwd, root.join("b"));
assert_eq!(module.steps[0].env["JAVA_HOME"], "/jdk-b");
assert_eq!(module.steps[0].args, ["--console=plain", ":app:clean", ":app:classes", ":app:testClasses"]);
// A directly linked build owns its module even when another composite reports it too.
let direct = model.gradle[1].projects[0].clone();
model.gradle[0].projects.push(direct);
assert_eq!(model.gradle_for_module(root, &root.join("b")).unwrap().dir, "b");
let mut nested = model.gradle[1].projects[1].clone();
nested.path = ":b:app".into();
model.gradle[0].projects.push(nested);
let nested = super::plan(root, &model, super::BuildKind::Build, Some(&root.join("b/app")), None).unwrap();
assert_eq!(nested.steps[0].program, root.join("b/gradlew").to_string_lossy());
assert_eq!(nested.steps[0].args, ["--console=plain", ":app:classes", ":app:testClasses"]);
model.gradle.clear();
assert!(super::plan(root, &model, super::BuildKind::Build, Some(&root.join("b/app")), None).is_err());
}
#[test]
fn absolute_linked_roots_build_the_project_and_external_source_module() {
let temp = tempfile::tempdir().unwrap();
let base = dunce::canonicalize(temp.path()).unwrap();
let root = base.join("workspace");
let external = base.join("external");
std::fs::create_dir(&root).unwrap();
std::fs::create_dir_all(external.join("app/src/main/java")).unwrap();
std::fs::write(external.join("settings.gradle"), "include 'app'").unwrap();
let source = external.join("app/src/main/java/App.java");
std::fs::write(&source, "class App {}").unwrap();
std::fs::write(external.join("gradlew"), "").unwrap();
let settings = ide_core::settings::GradleSettings {
linked_projects: Some(vec![external.to_string_lossy().into_owned()]),
..Default::default()
};
let mut model = crate::load_linked(&root, &base.join("cache"), None, &Default::default(), &settings);
model.gradle[0].env.insert("JAVA_HOME".into(), "/external-jdk".into());
let all = super::plan(&root, &model, super::BuildKind::Build, None, None).unwrap();
assert_eq!(all.steps[0].cwd, external);
assert_eq!(all.steps[0].program, external.join("gradlew").to_string_lossy());
let module = super::plan(&root, &model, super::BuildKind::Rebuild, Some(&source), None).unwrap();
assert_eq!(module.steps[0].cwd, external);
assert_eq!(module.steps[0].env["JAVA_HOME"], "/external-jdk");
assert_eq!(module.steps[0].args, ["--console=plain", ":app:clean", ":app:classes", ":app:testClasses"]);
let other = base.join("unlinked");
std::fs::create_dir(&other).unwrap();
std::fs::write(other.join("build.gradle"), "").unwrap();
assert!(super::plan(&root, &model, super::BuildKind::Build, Some(&other), None).is_err());
}
use super::*;
/// Launch outputs use the platform separator throughout, so they compare equal to
@@ -796,12 +934,12 @@ mod tests {
write(&root, "core/pom.xml", "<project><artifactId>core</artifactId></project>");
write(&root, "core/src/main/java/A.java", "class A {}");
let model = crate::load(&root, &tmp.path().join("cache"), None);
let p = plan(&root, &model, BuildKind::Build, None).unwrap();
let p = plan(&root, &model, BuildKind::Build, None, None).unwrap();
assert_eq!(p.system, BuildSystem::Maven);
assert_eq!(p.title, "demo");
assert_eq!(p.steps.len(), 1);
assert_eq!(p.steps[0].args, ["-B", "-Dstyle.color=never", "test-compile"]);
let p = plan(&root, &model, BuildKind::Rebuild, Some(&root.join("core/src/main/java/A.java"))).unwrap();
let p = plan(&root, &model, BuildKind::Rebuild, Some(&root.join("core/src/main/java/A.java")), None).unwrap();
assert_eq!(p.steps[0].args, ["-B", "-Dstyle.color=never", "-pl", "core", "-am", "clean", "test-compile"]);
assert_eq!(p.steps[0].cwd, root);
assert_eq!(p.title, "demo: build module 'core'");
@@ -810,24 +948,24 @@ mod tests {
write(&g, "settings.gradle", "include 'app'");
write(&g, "app/build.gradle", "apply plugin: 'java'");
let model = crate::load(&g, &tmp.path().join("cache2"), None);
let p = plan(&g, &model, BuildKind::Build, Some(&g.join("app/src/A.java"))).unwrap();
let p = plan(&g, &model, BuildKind::Build, Some(&g.join("app/src/A.java")), None).unwrap();
assert_eq!(p.system, BuildSystem::Gradle);
assert_eq!(p.steps[0].args, ["--console=plain", ":app:classes", ":app:testClasses"]);
assert_eq!(
plan(&g, &model, BuildKind::Artifacts, None).unwrap().steps[0].args,
plan(&g, &model, BuildKind::Artifacts, None, None).unwrap().steps[0].args,
["--console=plain", "assemble"]
);
let c = tmp.path().join("c");
write(&c, "Cargo.toml", "[package]\nname='c'");
let p = plan(&c, &BuildModel::default(), BuildKind::Rebuild, None).unwrap();
let p = plan(&c, &BuildModel::default(), BuildKind::Rebuild, None, None).unwrap();
assert_eq!(p.system, BuildSystem::Cargo);
assert_eq!(p.steps.len(), 2);
assert_eq!(p.steps[0].args[0], "clean");
let go = tmp.path().join("go");
write(&go, "go.mod", "module x");
let p = plan(&go, &BuildModel::default(), BuildKind::Build, Some(&go.join("pkg/a/a.go"))).unwrap();
let p = plan(&go, &BuildModel::default(), BuildKind::Build, Some(&go.join("pkg/a/a.go")), None).unwrap();
assert_eq!(p.steps[0].args, ["build", "./pkg/a/..."]);
assert_eq!(p.steps[0].cwd, go);
@@ -835,19 +973,51 @@ mod tests {
write(&ts, "package.json", r#"{"scripts":{"build":"vite build"}}"#);
write(&ts, "tsconfig.json", "{}");
write(&ts, "pnpm-lock.yaml", "");
let p = plan(&ts, &BuildModel::default(), BuildKind::Build, None).unwrap();
let p = plan(&ts, &BuildModel::default(), BuildKind::Build, None, None).unwrap();
assert_eq!(p.steps[0].args, ["--noEmit", "--pretty", "false", "-p", "."]);
let p = plan(&ts, &BuildModel::default(), BuildKind::Artifacts, None).unwrap();
let p = plan(&ts, &BuildModel::default(), BuildKind::Artifacts, None, None).unwrap();
assert_eq!(
(p.steps[0].program.as_str(), p.steps[0].args.as_slice()),
("pnpm", &["run".to_owned(), "build".to_owned()][..])
);
let cm = tmp.path().join("cm");
write(&cm, "CMakeLists.txt", "project(app)");
let args = |p: &BuildPlan| p.steps.iter().map(|s| s.args.join(" ")).collect::<Vec<_>>();
let p = plan(&cm, &BuildModel::default(), BuildKind::Build, None, None).unwrap();
assert_eq!(p.system, BuildSystem::CMake);
assert_eq!(
args(&p),
[
"-S . -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_EXPORT_COMPILE_COMMANDS=ON",
"--build build --config Debug"
],
"an unconfigured project is configured first"
);
write(&cm, "cmake-build-release/CMakeCache.txt", "");
let p = plan(&cm, &BuildModel::default(), BuildKind::Rebuild, None, None).unwrap();
assert_eq!(
args(&p),
["--build cmake-build-release --config Debug --clean-first"],
"a configured directory is reused"
);
let profile = CMakeProfile {
build_type: "Release".into(),
generator: "Ninja".into(),
build_args: vec!["-j".into(), "8".into()],
..CMakeProfile::default()
};
let p =
plan(&cm, &BuildModel::default(), BuildKind::Build, None, Some(std::slice::from_ref(&profile))).unwrap();
assert_eq!(args(&p), ["--build cmake-build-release --config Release -j 8"]);
let p = plan(&cm, &BuildModel::default(), BuildKind::Build, None, Some(&[CMakeProfile::default()])).unwrap();
assert_eq!(p.steps[0].args[..4], ["-S", ".", "-B", "cmake-build-debug"]);
let siblings = tmp.path().join("siblings");
write(&siblings, "core/pom.xml", "<project><artifactId>core</artifactId></project>");
write(&siblings, "system/pom.xml", "<project><artifactId>system</artifactId></project>");
let model = crate::load(&siblings, &tmp.path().join("cache3"), None);
let p = plan(&siblings, &model, BuildKind::Build, None).unwrap();
let p = plan(&siblings, &model, BuildKind::Build, None, None).unwrap();
assert_eq!(p.system, BuildSystem::Maven);
assert_eq!(
p.steps.iter().map(|s| s.cwd.clone()).collect::<Vec<_>>(),
@@ -856,7 +1026,7 @@ mod tests {
let none = tmp.path().join("none");
std::fs::create_dir_all(&none).unwrap();
assert!(plan(&none, &BuildModel::default(), BuildKind::Build, None).is_err());
assert!(plan(&none, &BuildModel::default(), BuildKind::Build, None, None).is_err());
}
#[test]
+32
View File
@@ -22,6 +22,8 @@ pub(crate) enum Format {
Tsc,
/// `go build`: `./a.go:12:5: msg`.
Go,
/// GCC / Clang (through `cmake --build`): `src/a.cpp:12:5: error: msg`.
Gcc,
}
/// What a printed column counts.
@@ -296,6 +298,17 @@ fn go_line(line: &str) -> Option<RawMessage> {
(!msg.is_empty()).then(|| RawMessage::new(Severity::Error, msg, ColUnit::Byte).at(path, ln, col))
}
/// A GCC / Clang diagnostic line; notes and the context lines after it are left as output.
fn gcc_line(line: &str) -> Option<RawMessage> {
let (path, ln, col, rest) = path_line_col(line)?;
let rest = rest.strip_prefix(':')?.trim_start();
let (severity, msg) =
[("fatal error:", Severity::Error), ("error:", Severity::Error), ("warning:", Severity::Warning)]
.into_iter()
.find_map(|(prefix, severity)| rest.strip_prefix(prefix).map(|m| (severity, m.trim())))?;
(!msg.is_empty()).then(|| RawMessage::new(severity, msg, ColUnit::Byte).at(path, ln, col))
}
/// rustc's JSON diagnostic (`reason: compiler-message`) → message and its rendered text.
fn cargo_json(v: &Value) -> Option<(Option<RawMessage>, String)> {
if v.get("reason")?.as_str()? != "compiler-message" {
@@ -410,6 +423,7 @@ impl OutputParser {
Format::Gradle => javac_header(&line).or_else(|| kotlin_line(&line)),
Format::Tsc => tsc_line(&line),
Format::Go => go_line(&line),
Format::Gcc => gcc_line(&line),
Format::Cargo => None,
};
if let Some(msg) = parsed {
@@ -563,6 +577,24 @@ mod tests {
assert_eq!(out[2].text, "Cannot find name 'foo'.");
}
#[test]
fn gcc_errors_and_warnings() {
let out = parse(
Format::Gcc,
"[1/2] Building CXX object CMakeFiles/app.dir/main.cpp.o\n\
/p/src/main.cpp:12:5: error: use of undeclared identifier 'foo'\n\
\x20 12 | foo();\n\
/p/src/util.h:3:10: warning: unused variable 'x' [-Wunused-variable]\n\
/p/src/main.cpp:4:10: fatal error: 'missing.h' file not found\n\
/p/src/main.cpp:2:1: note: in file included from here\n",
);
assert_eq!(out.len(), 3, "{out:?}");
assert_eq!(out[0].severity, Severity::Error);
assert_eq!(out[0].text, "use of undeclared identifier 'foo'");
assert_eq!(out[1].severity, Severity::Warning);
assert_eq!(out[2].text, "'missing.h' file not found");
}
#[test]
fn go_errors() {
let out = parse(
+94
View File
@@ -435,6 +435,15 @@ pub fn external_roots<'a>(root: &Path, dirs: impl IntoIterator<Item = &'a str>)
pub fn load(root: &Path) -> Option<GradleModel> {
let dir = build_dir(root)?;
load_build(root, dir.strip_prefix(root).unwrap_or(&dir))
}
/// Loads exactly this build, without searching its child folders.
pub fn load_build(root: &Path, dir: &Path) -> Option<GradleModel> {
let dir = root.join(dir);
if !SETTINGS_FILES.iter().chain(BUILD_FILES).any(|name| dir.join(name).is_file()) {
return None;
}
let settings = read_first(&dir, SETTINGS_FILES).map(|(_, t)| parse_settings(&t)).unwrap_or_default();
let root_name =
settings.name.unwrap_or_else(|| dir.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default());
@@ -455,6 +464,56 @@ pub fn load(root: &Path) -> Option<GradleModel> {
Some(GradleModel { dir: rel(root, &dir), projects, java_version, command: wrapper(&dir), ..GradleModel::default() })
}
/// Validates user-selected build folders and stores a stable, deduplicated identity.
pub fn normalize_linked_projects(root: &Path, dirs: &[String]) -> ide_core::IdeResult<Vec<String>> {
let root = dunce::canonicalize(root)?;
let mut out = Vec::new();
for dir in dirs {
let path = dunce::canonicalize(root.join(dir))?;
if !path.is_dir() || !SETTINGS_FILES.iter().chain(BUILD_FILES).any(|name| path.join(name).is_file()) {
return Err(ide_core::IdeError::invalid(format!("No Gradle build in {}", path.display())));
}
let dir = rel(&root, &path);
if !out.contains(&dir) {
out.push(dir);
}
}
Ok(out)
}
/// Linked build identities without reading their scripts (also used by index hot paths).
pub(crate) fn linked_dirs(root: &Path, settings: &ide_core::settings::GradleSettings) -> Vec<String> {
match &settings.linked_projects {
None => build_dir(root).map(|dir| rel(root, &dir)).into_iter().collect(),
Some(dirs) => {
let mut out = Vec::new();
for dir in dirs {
if let Ok(dirs) = normalize_linked_projects(root, std::slice::from_ref(dir)) {
for dir in dirs {
if !out.contains(&dir) {
out.push(dir);
}
}
}
}
out
}
}
}
pub fn load_linked(root: &Path, settings: &ide_core::settings::GradleSettings) -> Vec<GradleModel> {
linked_dirs(root, settings).iter().filter_map(|dir| load_build(root, Path::new(dir))).collect()
}
/// A build's task paths are local to it, so sync caches must be local to its identity too.
pub fn cache_dir(cache: &Path, build_dir: &str) -> PathBuf {
if build_dir.is_empty() {
return cache.to_path_buf();
}
let hash = build_dir.bytes().fold(0xcbf29ce484222325u64, |h, b| (h ^ u64::from(b)).wrapping_mul(0x100000001b3));
cache.join("gradle").join(format!("{hash:016x}"))
}
/// `JavaLanguageVersion.of(21)` / `sourceCompatibility = JavaVersion.VERSION_17` in a build script.
pub fn script_java_version(text: &str) -> Option<u32> {
let number = |s: &str| -> Option<u32> {
@@ -616,6 +675,41 @@ pub(crate) fn has_war_plugin(script: &str) -> bool {
#[cfg(test)]
mod tests {
#[test]
fn explicit_linkage_preserves_default_discovery_and_can_unlink_every_build() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path();
for dir in ["a", "b"] {
std::fs::create_dir(root.join(dir)).unwrap();
std::fs::write(root.join(dir).join("settings.gradle"), format!("rootProject.name = '{dir}'")).unwrap();
}
assert_eq!(super::load_linked(root, &Default::default())[0].dir, "a");
let mut settings = ide_core::settings::GradleSettings {
linked_projects: Some(vec!["b".into(), "./b".into(), "a".into()]),
..Default::default()
};
let models = super::load_linked(root, &settings);
assert_eq!(models.iter().map(|g| g.dir.as_str()).collect::<Vec<_>>(), ["b", "a"]);
settings.linked_projects = Some(vec![]);
assert!(super::load_linked(root, &settings).is_empty());
assert!(super::normalize_linked_projects(root, &["missing".into()]).is_err());
assert!(super::normalize_linked_projects(root, &["".into()]).is_err());
assert!(super::load_build(root, root).is_none(), "exact loading never picks a child");
}
#[cfg(unix)]
#[test]
fn linked_directories_deduplicate_symlinks_and_keep_external_builds() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("workspace");
let external = temp.path().join("external");
std::fs::create_dir(&root).unwrap();
std::fs::create_dir(&external).unwrap();
std::fs::write(external.join("build.gradle"), "").unwrap();
std::os::unix::fs::symlink(&external, root.join("alias")).unwrap();
let dirs = super::normalize_linked_projects(&root, &["alias".into(), "../external".into()]).unwrap();
assert_eq!(dirs, [external.canonicalize().unwrap().to_string_lossy().to_string()]);
}
#[test]
fn script_dependencies_read_string_and_map_notations() {
let script = r#"
+268
View File
@@ -0,0 +1,268 @@
//! A Gradle project's resolved dependencies for the Dependency Analyzer:
//! `gradle <project>:dependencies --configuration <name>`, parsed from its tree report.
use crate::gradle;
use crate::process;
use crate::sync::{SyncOptions, error_summary};
use ide_core::{CancellationToken, IdeError, IdeResult};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::path::Path;
/// One line of the report: an occurrence of a dependency where it is requested.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct GradleDependency {
/// `group`, empty for a project dependency.
pub group: String,
/// Artifact name, or the project path (`:core`) of a project dependency.
pub name: String,
/// The version asked for here; `None` when a platform or constraint supplies it.
pub requested: Option<String>,
/// The version Gradle selected (`-> 1.2`), when it differs from the request.
pub selected: Option<String>,
/// A project of the build (`project :core`, or a module substituted by one).
pub project: bool,
/// `(*)`: listed with its dependencies earlier in the tree.
pub repeated: bool,
/// `(c)`: a dependency constraint rather than a dependency.
pub constraint: bool,
/// `(n)` / `FAILED`: not resolved.
pub unresolved: bool,
pub children: Vec<GradleDependency>,
}
/// Parses the tree of a `dependencies --configuration` report (lines `+--- g:a:1.0 -> 1.1 (*)`).
pub fn parse_report(output: &str) -> Vec<GradleDependency> {
let mut roots: Vec<GradleDependency> = Vec::new();
// Path of child indexes from the roots to the last node of each depth.
let mut path: Vec<usize> = Vec::new();
for line in output.lines() {
let Some((depth, text)) = tree_line(line) else { continue };
let node = parse_entry(text);
path.truncate(depth);
if depth > path.len() {
continue; // A line deeper than its parent allows: not part of a well-formed tree.
}
let siblings = if depth == 0 {
&mut roots
} else {
let mut list = &mut roots;
for &i in &path[..depth - 1] {
list = &mut list[i].children;
}
let Some(parent) = list.get_mut(path[depth - 1]) else { continue };
&mut parent.children
};
siblings.push(node);
path.push(siblings.len() - 1);
}
roots
}
/// `| \--- g:a:1.0` → (1, "g:a:1.0").
fn tree_line(line: &str) -> Option<(usize, &str)> {
let mut rest = line;
let mut depth = 0;
loop {
if let Some(r) = rest.strip_prefix("+--- ").or_else(|| rest.strip_prefix("\\--- ")) {
return Some((depth, r.trim()));
}
rest = rest.strip_prefix("| ").or_else(|| rest.strip_prefix(" "))?;
depth += 1;
}
}
fn parse_entry(text: &str) -> GradleDependency {
let mut d = GradleDependency::default();
let mut text = text.trim();
loop {
if let Some(t) = text.strip_suffix(" (*)") {
d.repeated = true;
text = t;
} else if let Some(t) = text.strip_suffix(" (c)") {
d.constraint = true;
text = t;
} else if let Some(t) = text.strip_suffix(" (n)").or_else(|| text.strip_suffix(" FAILED")) {
d.unresolved = true;
text = t;
} else {
break;
}
text = text.trim_end();
}
let (left, right) = match text.split_once(" -> ") {
Some((l, r)) => (l.trim(), Some(r.trim())),
None => (text, None),
};
if let Some(project) = left.strip_prefix("project ") {
d.project = true;
d.name = project_path(project);
} else {
let mut parts = left.splitn(3, ':');
d.group = parts.next().unwrap_or_default().to_owned();
d.name = parts.next().unwrap_or_default().to_owned();
d.requested = parts.next().filter(|v| !v.is_empty()).map(str::to_owned);
}
if let Some(right) = right {
if let Some(project) = right.strip_prefix("project ") {
// A module substituted by a project of the build.
d.project = true;
d.selected = Some(project_path(project));
} else {
d.selected = Some(right.to_owned());
}
}
d
}
/// `:core`, or `':core'` as Gradle 9 quotes it.
fn project_path(text: &str) -> String {
let t = text.trim();
t.strip_prefix('\'').and_then(|t| t.strip_suffix('\'')).unwrap_or(t).to_owned()
}
/// Runs the `dependencies` report of `project` (a Gradle path, `:` for the root) for
/// `configuration` and returns its tree.
pub fn gradle_dependencies(
root: &Path,
model: &gradle::GradleModel,
options: &SyncOptions,
project: &str,
configuration: &str,
cancel: &CancellationToken,
) -> IdeResult<Vec<GradleDependency>> {
if !configuration.chars().all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '-') || configuration.is_empty() {
return Err(IdeError::invalid(format!("not a configuration name: {configuration}")));
}
if !project.starts_with(':') || project.chars().any(char::is_whitespace) {
return Err(IdeError::invalid(format!("not a Gradle project path: {project}")));
}
let task = if project == ":" { ":dependencies".to_owned() } else { format!("{project}:dependencies") };
let mut args: Vec<String> =
vec!["--console=plain".into(), "-q".into(), task, "--configuration".into(), configuration.into()];
if options.offline {
args.push("--offline".into());
}
let out = process::run(
options.gradle_command.as_deref().unwrap_or(&model.command),
&args,
&root.join(&model.dir),
&options.env,
cancel,
|_, _| {},
)?;
if !out.success {
return Err(IdeError::unavailable(error_summary(&out.text)));
}
Ok(parse_report(&out.text))
}
#[cfg(test)]
mod tests {
use super::*;
fn dep(group: &str, name: &str, requested: Option<&str>) -> GradleDependency {
GradleDependency {
group: group.into(),
name: name.into(),
requested: requested.map(Into::into),
..Default::default()
}
}
#[test]
fn parses_the_tree_with_selected_versions_and_markers() {
let report = "\
------------------------------------------------------------
Project ':app'
------------------------------------------------------------
runtimeClasspath - Runtime classpath of source set 'main'.
+--- org.springframework.boot:spring-boot-starter-web -> 3.2.0
| +--- org.springframework.boot:spring-boot-starter:3.2.0
| | \\--- org.yaml:snakeyaml:2.2
| \\--- com.fasterxml.jackson.core:jackson-databind:2.15.0 -> 2.15.3 (*)
+--- project ':core'
| \\--- com.google.guava:guava:{strictly 32.1.3-jre} -> 32.1.3-jre (c)
+--- org.example:legacy:1.0 -> project :legacy
\\--- org.missing:gone:9 FAILED
(c) - A dependency constraint, not a dependency. The dependency affected by the constraint occurs elsewhere in the tree.
(*) - Indicates repeated occurrences of a transitive dependency subtree. Gradle expands transitive dependency subtrees only once per project; repeat occurrences only display the root of the subtree, followed by this annotation.
";
let tree = parse_report(report);
assert_eq!(tree.len(), 4);
let web = &tree[0];
assert_eq!((web.group.as_str(), web.name.as_str()), ("org.springframework.boot", "spring-boot-starter-web"));
assert_eq!((web.requested.as_deref(), web.selected.as_deref()), (None, Some("3.2.0")));
assert_eq!(web.children.len(), 2);
assert_eq!(web.children[0].children, vec![dep("org.yaml", "snakeyaml", Some("2.2"))]);
let databind = &web.children[1];
assert_eq!((databind.requested.as_deref(), databind.selected.as_deref()), (Some("2.15.0"), Some("2.15.3")));
assert!(databind.repeated);
let core = &tree[1];
assert!(core.project && core.group.is_empty());
assert_eq!(core.name, ":core");
let guava = &core.children[0];
assert!(guava.constraint);
assert_eq!(guava.requested.as_deref(), Some("{strictly 32.1.3-jre}"));
let legacy = &tree[2];
assert!(legacy.project);
assert_eq!((legacy.name.as_str(), legacy.selected.as_deref()), ("legacy", Some(":legacy")));
assert!(tree[3].unresolved);
assert_eq!(tree[3].requested.as_deref(), Some("9"));
}
#[test]
fn an_empty_configuration_has_no_dependencies() {
assert!(
parse_report("compileClasspath - Compile classpath for source set 'main'.\nNo dependencies\n").is_empty()
);
}
#[test]
fn rejects_names_that_are_not_configurations_or_projects() {
let tmp = tempfile::tempdir().unwrap();
let options = SyncOptions::default();
let cancel = CancellationToken::new();
assert!(
gradle_dependencies(tmp.path(), &Default::default(), &options, ":", "x --init-script y", &cancel).is_err()
);
assert!(
gradle_dependencies(tmp.path(), &Default::default(), &options, "app", "runtimeClasspath", &cancel).is_err()
);
}
#[cfg(unix)]
#[test]
fn runs_the_report_of_the_project_and_configuration() {
use std::os::unix::fs::PermissionsExt;
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
std::fs::write(root.join("settings.gradle"), "include 'app'").unwrap();
std::fs::write(root.join("build.gradle"), "").unwrap();
let launcher = root.join("fake-gradle");
std::fs::write(&launcher, "#!/bin/sh\necho \"$@\" > args\necho '\\--- org.a:b:1.0'\n").unwrap();
std::fs::set_permissions(&launcher, std::fs::Permissions::from_mode(0o755)).unwrap();
let options = SyncOptions {
gradle_command: Some(launcher.to_string_lossy().into_owned()),
offline: true,
..Default::default()
};
let tree = gradle_dependencies(
root,
&gradle::load(root).unwrap(),
&options,
":app",
"testRuntimeClasspath",
&CancellationToken::new(),
)
.unwrap();
assert_eq!(tree, vec![dep("org.a", "b", Some("1.0"))]);
assert_eq!(
std::fs::read_to_string(root.join("args")).unwrap().trim(),
"--console=plain -q :app:dependencies --configuration testRuntimeClasspath --offline"
);
}
}
+1 -1
View File
@@ -81,7 +81,7 @@ pub fn build(
cancel: &CancellationToken,
) -> IdeResult<Outcome> {
std::fs::create_dir_all(generation)?;
let mut plan = compile::plan(root, model, BuildKind::Build, None)?;
let mut plan = compile::plan(root, model, BuildKind::Build, None, None)?;
if plan.system != compile::BuildSystem::Maven {
return Err(IdeError::invalid("Kotlin bridge requires a Maven build root"));
}
+88 -16
View File
@@ -8,6 +8,7 @@
pub mod application_update;
pub mod compile;
pub mod gradle;
mod gradle_deps;
pub mod kotlin_bridge;
pub mod maven;
pub mod pom_edit;
@@ -17,8 +18,11 @@ pub mod web;
mod xml;
pub use gradle::{GradleModel, GradleProject, GradleTask};
pub use gradle_deps::{GradleDependency, gradle_dependencies};
pub use maven::{MavenDependency, MavenModel, MavenPlugin, MavenProfile, MavenProject, Omission};
pub use sync::{SyncOptions, SyncProgress, maven_profiles, save_maven_profiles, sync_gradle, sync_maven};
pub use sync::{
SyncOptions, SyncProgress, maven_profiles, save_maven_profiles, sync_gradle, sync_gradle_model, sync_maven,
};
use serde::{Deserialize, Serialize};
use specta::Type;
@@ -36,13 +40,31 @@ pub enum BuildTool {
#[serde(rename_all = "camelCase")]
pub struct BuildModel {
pub maven: Option<MavenModel>,
pub gradle: Option<GradleModel>,
#[serde(default)]
pub gradle: Vec<GradleModel>,
}
impl BuildModel {
/// Prefer the deepest linked build owning the module over a composite including it.
pub fn gradle_for_module(&self, root: &Path, module: &Path) -> Option<&GradleModel> {
let module = root.join(module);
self.gradle.iter().filter(|g| g.projects.iter().any(|p| root.join(&p.dir) == module)).min_by_key(|g| {
let dir = root.join(&g.dir);
std::cmp::Reverse(if module.starts_with(&dir) { dir.components().count() } else { 0 })
})
}
}
/// Static model of the builds under `root`, merged with the sync results cached in `cache_dir`.
/// Parent POMs are looked up in `maven_repo` (the configured local repository), else the default one.
pub fn load(root: &Path, cache_dir: &Path, maven_repo: Option<PathBuf>) -> BuildModel {
load_linked(root, cache_dir, maven_repo, &ide_core::settings::MavenSettings::default())
load_linked(
root,
cache_dir,
maven_repo,
&ide_core::settings::MavenSettings::default(),
&ide_core::settings::GradleSettings::default(),
)
}
/// Loads only linked Maven reactors, retaining an empty model so projects can be re-added.
@@ -51,30 +73,60 @@ pub fn load_linked(
cache_dir: &Path,
maven_repo: Option<PathBuf>,
settings: &ide_core::settings::MavenSettings,
gradle_settings: &ide_core::settings::GradleSettings,
) -> BuildModel {
let maven = maven::is_maven_project(root).then(|| {
let mut m = maven::load_with_repo(root, maven_repo.or_else(maven::default_local_repo));
let maven = maven::has_projects(root, settings).then(|| {
let profiles = sync::maven_profiles(cache_dir);
let repo = maven_repo.or_else(maven::default_local_repo);
let mut m = maven::load_with_linked(root, repo, &profiles, &settings.linked_projects);
maven::UnlinkedProjects::from_model(&m, settings).filter(&mut m);
sync::apply_maven_cache(root, cache_dir, &mut m);
m
});
let gradle = gradle::load(root).map(|mut g| {
sync::apply_gradle_cache(root, cache_dir, &mut g);
g
});
let gradle = gradle::load_linked(root, gradle_settings)
.into_iter()
.map(|mut g| {
sync::apply_gradle_cache(root, &gradle::cache_dir(cache_dir, &g.dir), &mut g);
g
})
.collect();
BuildModel { maven, gradle }
}
/// Gradle project folders outside `root` as of the last sync (`includeFlat` siblings):
/// extra content roots, as IDEA lists module roots beside the project folder.
/// Reads only the sync's cached projects (no build script), so it is cheap enough for hot paths.
pub fn gradle_external_roots(root: &Path, cache_dir: &Path) -> Vec<PathBuf> {
gradle::external_roots(root, sync::reported_gradle_projects(cache_dir).iter().map(|p| p.dir.as_str()))
pub fn gradle_external_roots(
root: &Path,
cache_dir: &Path,
settings: &ide_core::settings::GradleSettings,
) -> Vec<PathBuf> {
let builds = gradle::linked_dirs(root, settings);
let dirs: Vec<String> = builds
.iter()
.flat_map(|dir| {
std::iter::once(root.join(dir).to_string_lossy().into_owned()).chain(
sync::reported_gradle_projects(&gradle::cache_dir(cache_dir, dir))
.iter()
.map(|p| p.dir.clone())
.collect::<Vec<_>>(),
)
})
.collect();
gradle::external_roots(root, dirs.iter().map(String::as_str))
}
/// Source, resource and build folders of the last Gradle sync, absolute.
pub fn gradle_source_roots(cache_dir: &Path) -> Vec<(PathBuf, gradle::ReportedRoot)> {
gradle::reported_source_roots(&sync::reported_gradle_projects(cache_dir))
pub fn gradle_source_roots(
root: &Path,
cache_dir: &Path,
settings: &ide_core::settings::GradleSettings,
) -> Vec<(PathBuf, gradle::ReportedRoot)> {
let reported = gradle::linked_dirs(root, settings)
.iter()
.flat_map(|dir| sync::reported_gradle_projects(&gradle::cache_dir(cache_dir, dir)).as_ref().clone())
.collect::<Vec<_>>();
gradle::reported_source_roots(&reported)
}
/// A library the project depends on, as `groupId:artifactId:version`.
@@ -118,7 +170,7 @@ pub fn dependency_coordinates(root: &Path, model: &BuildModel) -> Vec<Dependency
});
}
}
if let Some(g) = &model.gradle {
for g in &model.gradle {
for script in g.projects.iter().filter_map(|p| p.build_file.as_ref()) {
let Ok(text) = std::fs::read_to_string(root.join(script)) else { continue };
out.extend(gradle::script_dependencies(&text).into_iter().map(|(group_id, artifact_id, version)| {
@@ -144,6 +196,26 @@ pub fn affects_build(rel: &Path) -> bool {
mod tests {
use super::*;
#[test]
fn external_linked_build_is_indexed_before_sync_and_disappears_when_unlinked() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().join("workspace");
let external = temp.path().join("external");
std::fs::create_dir(&root).unwrap();
std::fs::create_dir(&external).unwrap();
std::fs::write(external.join("settings.gradle"), "").unwrap();
let mut settings = ide_core::settings::GradleSettings {
linked_projects: Some(vec![external.to_string_lossy().into_owned()]),
..Default::default()
};
assert_eq!(
gradle_external_roots(&root, &temp.path().join("cache"), &settings),
[external.canonicalize().unwrap()]
);
settings.linked_projects = Some(vec![]);
assert!(gradle_external_roots(&root, &temp.path().join("cache"), &settings).is_empty());
}
#[test]
fn detects_build_systems_and_build_files() {
let tmp = tempfile::tempdir().unwrap();
@@ -155,7 +227,7 @@ mod tests {
std::fs::write(root.join("build.gradle"), "").unwrap();
let m = load(&root, &cache, None);
assert_eq!(m.maven.unwrap().projects[0].pom, "svc/pom.xml");
assert_eq!(m.gradle.unwrap().projects[0].path, ":");
assert_eq!(m.gradle[0].projects[0].path, ":");
assert!(affects_build(Path::new("a/pom.xml")) && affects_build(Path::new("settings.gradle.kts")));
assert!(!affects_build(Path::new("src/Main.java")));
}
@@ -202,7 +274,7 @@ mod tests {
],
..Default::default()
};
let gradle = gradle::load(root);
let gradle = gradle::load(root).into_iter().collect();
let model = BuildModel { maven: Some(maven), gradle };
let names: Vec<String> = dependency_coordinates(root, &model)
.into_iter()
+173 -9
View File
@@ -460,6 +460,8 @@ const MAX_PARENT_DEPTH: usize = 10;
struct Resolver {
local_repo: Option<PathBuf>,
/// The chosen profiles (`-P` entries; `!id` / `-id` deactivates).
profiles: Vec<String>,
poms: HashMap<PathBuf, Option<RawPom>>,
contexts: HashMap<PathBuf, Inherited>,
}
@@ -484,8 +486,16 @@ pub fn local_repository_of(settings: &Path) -> Option<PathBuf> {
}
impl Resolver {
fn new(local_repo: Option<PathBuf>) -> Self {
Self { local_repo, poms: HashMap::new(), contexts: HashMap::new() }
fn new(local_repo: Option<PathBuf>, profiles: &[String]) -> Self {
Self { local_repo, profiles: profiles.to_vec(), poms: HashMap::new(), contexts: HashMap::new() }
}
/// The active profiles of a POM: the chosen ones, and the `activeByDefault` ones not turned
/// off while none of the POM's other profiles is chosen (Maven's rule).
fn active<'a>(&self, profiles: &'a [RawProfile]) -> impl Iterator<Item = &'a RawProfile> {
let chosen = |id: &str| profile_choice(&self.profiles, id);
let any_chosen = profiles.iter().any(|p| chosen(&p.id) == Some(true));
profiles.iter().filter(move |p| chosen(&p.id).unwrap_or(p.active_by_default && !any_chosen))
}
fn pom(&mut self, path: &Path) -> Option<RawPom> {
@@ -550,7 +560,7 @@ impl Resolver {
if pom.compiler_release.is_some() {
ctx.compiler_release = pom.compiler_release.clone();
}
for p in pom.profiles.iter().filter(|p| p.active_by_default) {
for p in self.active(&pom.profiles) {
merge_by(&mut ctx.properties, &p.properties, |(k, _)| k.clone());
merge_by(&mut ctx.deps, &p.deps, RawDep::key);
merge_plugins(&mut ctx.plugins, &p.plugins);
@@ -666,7 +676,7 @@ pub(crate) fn launch_inputs(
}
Some(())
}
let mut resolver = Resolver::new(local_repo.or_else(default_local_repo));
let mut resolver = Resolver::new(local_repo.or_else(default_local_repo), &[]);
let mut out = Vec::new();
for project in projects {
let dir = root.join(&project.dir);
@@ -1055,12 +1065,77 @@ pub fn load(root: &Path) -> MavenModel {
load_with_repo(root, default_local_repo())
}
/// [`load`] with the POMs `settings` link (Add Maven Projects).
pub fn load_linked(root: &Path, settings: &ide_core::settings::MavenSettings) -> MavenModel {
load_with_linked(root, default_local_repo(), &[], &settings.linked_projects)
}
pub(crate) fn load_with_repo(root: &Path, local_repo: Option<PathBuf>) -> MavenModel {
let mut resolver = Resolver::new(local_repo);
load_with(root, local_repo, &[])
}
/// The `-P` entries' choice for profile `id`: on, off (`!id` / `-id`), or none (the last wins).
fn profile_choice(chosen: &[String], id: &str) -> Option<bool> {
chosen.iter().rev().find_map(|c| {
let c = c.trim();
if c == id {
Some(true)
} else if c.strip_prefix('!').or_else(|| c.strip_prefix('-')) == Some(id) {
Some(false)
} else {
None
}
})
}
pub(crate) fn load_with(root: &Path, local_repo: Option<PathBuf>, profiles: &[String]) -> MavenModel {
load_with_linked(root, local_repo, profiles, &[])
}
/// The POMs of `linked` (project-relative, from Settings) that exist inside `root`.
fn linked_poms(root: &Path, linked: &[String]) -> Vec<PathBuf> {
let base = dunce::canonicalize(root).unwrap_or_else(|_| root.to_path_buf());
let mut out: Vec<PathBuf> = Vec::new();
for pom in linked {
let pom = normalized_pom(pom);
if pom.is_empty() || pom.starts_with("../") || pom == ".." {
continue;
}
let path = root.join(&pom);
let inside = dunce::canonicalize(&path).is_ok_and(|p| p.starts_with(&base));
if path.is_file() && inside && !out.contains(&path) {
out.push(path);
}
}
out
}
/// Whether `root` holds a Maven build or the settings link one inside it.
pub fn has_projects(root: &Path, settings: &ide_core::settings::MavenSettings) -> bool {
is_maven_project(root) || !linked_poms(root, &settings.linked_projects).is_empty()
}
/// The model with the chosen `profiles` (`-P` entries; `!id` deactivates: their modules,
/// dependencies and properties are part of the model, those turned off are not) and the
/// `linked` POMs (Add Maven Projects): builds that no reactor lists, shown as independent
/// top-level projects. A linked POM that turns out to be a module of the root reactor stays that module.
pub fn load_with_linked(
root: &Path,
local_repo: Option<PathBuf>,
profiles: &[String],
linked: &[String],
) -> MavenModel {
let mut resolver = Resolver::new(local_repo, profiles);
let mut model = MavenModel::default();
let mut seen: HashSet<PathBuf> = HashSet::new();
// (pom, aggregated)
let mut stack: Vec<(PathBuf, bool)> = top_level_poms(root).into_iter().rev().map(|p| (p, false)).collect();
let mut tops = top_level_poms(root);
for pom in linked_poms(root, linked) {
if !tops.contains(&pom) {
tops.push(pom);
}
}
// (pom, aggregated); the root reactor and its modules come before linked POMs.
let mut stack: Vec<(PathBuf, bool)> = tops.into_iter().rev().map(|p| (p, false)).collect();
while let Some((path, aggregated)) = stack.pop() {
if model.projects.len() >= MAX_PROJECTS {
break;
@@ -1076,7 +1151,7 @@ pub(crate) fn load_with_repo(root: &Path, local_repo: Option<PathBuf>) -> MavenM
let i = |s: &str| interpolate(s, &props);
let mut module_names: Vec<String> = pom.modules.clone();
for p in pom.profiles.iter().filter(|p| p.active_by_default) {
for p in resolver.active(&pom.profiles) {
module_names.extend(p.modules.iter().cloned());
}
let mut modules: Vec<PathBuf> = Vec::new();
@@ -1182,7 +1257,7 @@ impl UnlinkedProjects {
if settings.unlinked_projects.is_empty() {
return Self::default();
}
Self::from_model(&load(root), settings)
Self::from_model(&load_linked(root, settings), settings)
}
pub fn from_model(model: &MavenModel, settings: &ide_core::settings::MavenSettings) -> Self {
@@ -1699,6 +1774,95 @@ mod tests {
assert!(model.projects.iter().all(|p| p.command == if cfg!(windows) { "mvn.cmd" } else { "mvn" }));
}
/// Add Maven Projects: a POM inside the project that no reactor lists becomes an
/// independent top-level project; one the root reactor lists stays its module.
#[test]
fn linked_poms_outside_the_reactor_are_top_level_projects() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
write(
root,
"pom.xml",
"<project><groupId>demo</groupId><artifactId>main</artifactId><version>1.0</version><packaging>pom</packaging><modules><module>core</module></modules></project>",
);
write(
root,
"core/pom.xml",
"<project><groupId>demo</groupId><artifactId>core</artifactId><version>1.0</version></project>",
);
write(
root,
"stub/pom.xml",
"<project><groupId>demo</groupId><artifactId>stub</artifactId><version>1.0</version></project>",
);
let top = |m: &MavenModel| m.projects.iter().map(|p| (p.dir.clone(), p.aggregated)).collect::<Vec<_>>();
assert_eq!(top(&load_with_linked(root, None, &[], &[])), [("".into(), false), ("core".into(), true)]);
let linked: Vec<String> =
["stub/pom.xml", "./core/pom.xml", "../outside/pom.xml", "missing/pom.xml"].map(String::from).into();
assert_eq!(
top(&load_with_linked(root, None, &[], &linked)),
[("".into(), false), ("core".into(), true), ("stub".into(), false)]
);
// Deeper than the folder search reaches: only the link makes it a Maven project.
let deep = tempfile::tempdir().unwrap();
write(deep.path(), "a/b/c/tool/pom.xml", "<project><artifactId>tool</artifactId></project>");
let settings = ide_core::settings::MavenSettings {
linked_projects: vec!["a/b/c/tool/pom.xml".into()],
..Default::default()
};
assert!(!has_projects(deep.path(), &Default::default()));
assert!(has_projects(deep.path(), &settings));
assert_eq!(top(&load_linked(deep.path(), &settings)), [("a/b/c/tool".into(), false)]);
}
#[test]
fn chosen_profiles_bring_their_modules_and_dependencies() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
write(
root,
"pom.xml",
r#"<project><groupId>demo</groupId><artifactId>parent</artifactId><version>1.0</version><packaging>pom</packaging>
<modules><module>core</module></modules>
<profiles>
<profile><id>flink-1.17</id><activation><activeByDefault>true</activeByDefault></activation>
<modules><module>flink-1.17</module></modules></profile>
<profile><id>flink-1.20</id><modules><module>flink-1.20</module></modules>
<dependencies><dependency><groupId>org.apache.flink</groupId><artifactId>flink-core</artifactId><version>1.20.0</version></dependency></dependencies>
</profile>
</profiles>
</project>"#,
);
for m in ["core", "flink-1.17", "flink-1.20"] {
write(
root,
&format!("{m}/pom.xml"),
&format!(
"<project><parent><groupId>demo</groupId><artifactId>parent</artifactId><version>1.0</version></parent><artifactId>{m}</artifactId></project>"
),
);
}
let dirs = |m: &MavenModel| m.projects.iter().map(|p| p.dir.clone()).collect::<Vec<_>>();
let flink = |m: &MavenModel, dir: &str| {
m.projects.iter().find(|p| p.dir == dir).unwrap().dependencies.iter().any(|d| d.artifact_id == "flink-core")
};
let default = load_with(root, None, &[]);
assert_eq!(dirs(&default), ["", "core", "flink-1.17"]);
assert!(!flink(&default, "core"));
// Choosing another profile of the POM turns its activeByDefault one off (as `-P` does).
let chosen = load_with(root, None, &["flink-1.20".into()]);
assert_eq!(dirs(&chosen), ["", "core", "flink-1.20"]);
assert!(flink(&chosen, "core"), "a module inherits the chosen profile's dependencies");
let both = load_with(root, None, &["flink-1.20".into(), "flink-1.17".into()]);
assert_eq!(dirs(&both), ["", "core", "flink-1.17", "flink-1.20"]);
assert_eq!(dirs(&load_with(root, None, &["!flink-1.17".into()])), ["", "core"]);
}
#[test]
fn profiles_of_a_parent_pom_are_inherited() {
let tmp = tempfile::tempdir().unwrap();
+179 -8
View File
@@ -283,7 +283,12 @@ pub fn sync_maven(
cancel: &CancellationToken,
progress: &dyn Fn(SyncProgress),
) -> IdeResult<()> {
let mut model = maven::load(root);
let mut model = maven::load_with_linked(
root,
maven::default_local_repo(),
&options.profiles,
&options.maven_settings.linked_projects,
);
maven::UnlinkedProjects::from_model(&model, &options.maven_settings).filter(&mut model);
std::fs::create_dir_all(cache_dir)?;
let mut cache = MavenCache {
@@ -315,7 +320,10 @@ pub fn sync_maven(
for (n, top) in model.projects.iter().filter(|p| !p.aggregated).enumerate() {
let effective = cache_dir.join(format!("effective-pom-{n}.xml"));
let _ = std::fs::remove_file(&effective);
let mut args: Vec<String> = vec!["-B".into(), "-fae".into(), "-Dstyle.color=never".into()];
// `-T1` keeps mvnd serial: its default parallel build interleaves the
// `dependency:tree` output of concurrent modules, which the sync parses
// as one stream. mvn is serial anyway, so this only pins the daemon.
let mut args: Vec<String> = vec!["-B".into(), "-fae".into(), "-Dstyle.color=never".into(), "-T1".into()];
args.extend(options.maven_options.iter().cloned());
if options.offline {
args.push("-o".into());
@@ -370,8 +378,25 @@ pub fn sync_gradle(
cancel: &CancellationToken,
progress: &dyn Fn(SyncProgress),
) -> IdeResult<()> {
let mut model = gradle::load(root).ok_or_else(|| IdeError::not_found("no Gradle build in this project"))?;
std::fs::create_dir_all(cache_dir)?;
let model = gradle::load(root).ok_or_else(|| IdeError::not_found("no Gradle build in this project"))?;
sync_gradle_model(root, cache_dir, &model, options, cancel, progress)
}
/// Syncs this exact linked build; another build's task paths and cache cannot interfere.
pub fn sync_gradle_model(
root: &Path,
cache_dir: &Path,
model: &GradleModel,
options: &SyncOptions,
cancel: &CancellationToken,
progress: &dyn Fn(SyncProgress),
) -> IdeResult<()> {
if cancel.is_cancelled() {
return Err(IdeError::cancelled());
}
let mut model = model.clone();
let cache_dir = gradle::cache_dir(cache_dir, &model.dir);
std::fs::create_dir_all(&cache_dir)?;
let init_script = cache_dir.join("openidea-model.gradle");
std::fs::write(&init_script, MODEL_INIT_SCRIPT)?;
let mut args: Vec<String> = vec![
@@ -386,14 +411,24 @@ pub fn sync_gradle(
args.push("--offline".into());
}
progress(SyncProgress { message: "Loading Gradle tasks".into(), fraction: None });
let out = run(
let out = match run(
options.gradle_command.as_deref().unwrap_or(&model.command),
&args,
&root.join(&model.dir),
&options.env,
cancel,
|_| {},
)?;
) {
Ok(out) => out,
Err(_) if cancel.is_cancelled() => return Err(IdeError::cancelled()),
Err(error) if error.kind == ide_core::error::ErrorKind::Unavailable => {
Output { success: false, text: error.message }
}
Err(error) => return Err(error),
};
if cancel.is_cancelled() {
return Err(IdeError::cancelled());
}
let projects = if out.success { parse_reported_projects(&out.text) } else { Vec::new() };
// Hash the build files of the reported projects, or the model is stale right after the sync.
gradle::merge_reported_projects(root, &mut model, &projects);
@@ -404,13 +439,128 @@ pub fn sync_gradle(
tasks: if out.success { gradle::parse_tasks(&out.text) } else { Vec::new() },
projects,
};
write_cache(cache_dir, GRADLE_CACHE, &cache)
write_cache(&cache_dir, GRADLE_CACHE, &cache)
}
#[cfg(test)]
mod tests {
use super::*;
#[cfg(unix)]
#[test]
fn unavailable_gradle_launchers_are_cached_without_stopping_other_builds() {
use std::os::unix::fs::PermissionsExt;
let temp = tempfile::tempdir().unwrap();
let root = temp.path();
for dir in ["a", "b"] {
std::fs::create_dir(root.join(dir)).unwrap();
std::fs::write(root.join(dir).join("build.gradle"), "").unwrap();
}
let wrapper = root.join("b/gradlew");
std::fs::write(&wrapper, "#!/bin/sh\nprintf '%s\\n' 'Build tasks' '-----------' 'healthy - ok'\n").unwrap();
std::fs::set_permissions(&wrapper, std::fs::Permissions::from_mode(0o755)).unwrap();
let settings = ide_core::settings::GradleSettings {
linked_projects: Some(vec!["a".into(), "b".into()]),
..Default::default()
};
let mut models = gradle::load_linked(root, &settings);
models[0].command = root.join("a/missing-gradlew").to_string_lossy().into_owned();
let cache = root.join("cache");
let run = |model: &GradleModel| {
sync_gradle_model(root, &cache, model, &SyncOptions::default(), &CancellationToken::new(), &|_| {})
};
for model in &models {
run(model).unwrap();
}
let loaded = crate::load_linked(root, &cache, None, &Default::default(), &settings);
assert!(loaded.gradle[0].sync_error.as_deref().unwrap().contains("cannot start"));
assert!(loaded.gradle[1].sync_error.is_none());
assert_eq!(loaded.gradle[1].projects[0].tasks[0].name, "healthy");
let bad_wrapper = root.join("a/gradlew");
std::fs::write(&bad_wrapper, "#!/bin/sh\nexit 0\n").unwrap();
std::fs::set_permissions(&bad_wrapper, std::fs::Permissions::from_mode(0o644)).unwrap();
models[0].command = bad_wrapper.to_string_lossy().into_owned();
run(&models[0]).unwrap();
let loaded = crate::load_linked(root, &cache, None, &Default::default(), &settings);
assert!(loaded.gradle[0].sync_error.as_deref().unwrap().contains("cannot start"));
assert!(loaded.gradle[1].sync_error.is_none());
let blocked_cache = root.join("blocked-cache");
std::fs::write(&blocked_cache, "a file cannot contain sync caches").unwrap();
assert!(
sync_gradle_model(
root,
&blocked_cache,
&models[0],
&SyncOptions::default(),
&CancellationToken::new(),
&|_| {}
)
.is_err()
);
}
#[cfg(unix)]
#[test]
fn independent_syncs_do_not_share_tasks_launchers_environments_or_failures() {
use std::os::unix::fs::PermissionsExt;
let temp = tempfile::tempdir().unwrap();
let root = temp.path();
for dir in ["a", "b"] {
let build = root.join(dir);
std::fs::create_dir_all(build.join("app")).unwrap();
std::fs::write(build.join("settings.gradle"), "include 'app'").unwrap();
std::fs::write(build.join("app/build.gradle"), "").unwrap();
let report = serde_json::json!({"path": ":app", "dir": build.join("app")});
let launcher = build.join("gradlew");
std::fs::write(
&launcher,
format!("#!/bin/sh\npwd > actual-cwd\nprintf '%s' \"$JAVA_HOME\" > actual-env\nprintf '%s\\n' '@openidea-model\t{report}' 'Build tasks' '-----------' 'app:custom{dir} - custom task'\n"),
).unwrap();
std::fs::set_permissions(launcher, std::fs::Permissions::from_mode(0o755)).unwrap();
}
let settings = ide_core::settings::GradleSettings {
linked_projects: Some(vec!["a".into(), "b".into()]),
..Default::default()
};
let cache = root.join("cache");
let models = gradle::load_linked(root, &settings);
for model in &models {
let options = SyncOptions {
env: BTreeMap::from([("JAVA_HOME".into(), format!("/jdk-{}", model.dir))]),
..Default::default()
};
sync_gradle_model(root, &cache, model, &options, &CancellationToken::new(), &|_| {}).unwrap();
assert_eq!(
std::fs::read_to_string(root.join(&model.dir).join("actual-env")).unwrap(),
format!("/jdk-{}", model.dir)
);
assert_eq!(
std::fs::read_to_string(root.join(&model.dir).join("actual-cwd")).unwrap().trim(),
root.join(&model.dir).canonicalize().unwrap().to_string_lossy()
);
}
let load = || crate::load_linked(root, &cache, None, &Default::default(), &settings);
let model = load();
assert_eq!(model.gradle[0].projects[0].tasks[0].name, "customa");
assert_eq!(model.gradle[1].projects[0].tasks[0].name, "customb");
assert_ne!(gradle::cache_dir(&cache, "a"), gradle::cache_dir(&cache, "b"));
let before = std::fs::read(gradle::cache_dir(&cache, "a").join(GRADLE_CACHE)).unwrap();
std::fs::write(root.join("b/gradlew"), "#!/bin/sh\nprintf 'FAILURE: broken b\\n'\nexit 1\n").unwrap();
sync_gradle_model(root, &cache, &models[1], &SyncOptions::default(), &CancellationToken::new(), &|_| {})
.unwrap();
let model = load();
assert!(model.gradle[0].sync_error.is_none());
assert!(model.gradle[1].sync_error.is_some());
assert_eq!(std::fs::read(gradle::cache_dir(&cache, "a").join(GRADLE_CACHE)).unwrap(), before);
let cancel = CancellationToken::new();
cancel.cancel();
assert!(sync_gradle_model(root, &cache, &models[0], &SyncOptions::default(), &cancel, &|_| {}).is_err());
assert_eq!(std::fs::read(gradle::cache_dir(&cache, "a").join(GRADLE_CACHE)).unwrap(), before);
let unlinked = ide_core::settings::GradleSettings { linked_projects: Some(vec![]), ..Default::default() };
assert!(crate::load_linked(root, &cache, None, &Default::default(), &unlinked).gradle.is_empty());
assert!(crate::gradle_source_roots(root, &cache, &unlinked).is_empty());
}
#[cfg(unix)]
#[test]
fn gradle_sync_uses_resolved_command_and_environment() {
@@ -598,7 +748,7 @@ mod tests {
write(&projects);
let first = reported_gradle_projects(&cache_dir);
assert!(Arc::ptr_eq(&first, &reported_gradle_projects(&cache_dir)), "unchanged: not parsed again");
assert_eq!(crate::gradle_external_roots(&root, &cache_dir), [billing]);
assert_eq!(crate::gradle_external_roots(&root, &cache_dir, &Default::default()), [billing]);
write(&projects[..1]);
assert_eq!(reported_gradle_projects(&cache_dir).len(), 1, "a new sync is read");
std::fs::remove_file(cache_dir.join(GRADLE_CACHE)).unwrap();
@@ -680,4 +830,25 @@ mod tests {
assert_eq!(error_summary(text), "Failed to execute goal");
assert_eq!(error_summary("one\nlast line\n\n"), "last line");
}
#[cfg(unix)]
#[test]
fn maven_sync_runs_the_daemon_command_serially() {
use std::os::unix::fs::PermissionsExt;
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path().join("p");
std::fs::create_dir_all(&root).unwrap();
std::fs::write(root.join("pom.xml"), "<project><groupId>g</groupId><artifactId>a</artifactId></project>")
.unwrap();
let launcher = tmp.path().join("fake-mvnd");
std::fs::write(&launcher, "#!/bin/sh\nprintf '%s\\n' \"$@\" > mvnd-args\n").unwrap();
std::fs::set_permissions(&launcher, std::fs::Permissions::from_mode(0o755)).unwrap();
let options =
SyncOptions { maven_command: Some(launcher.to_string_lossy().into_owned()), ..Default::default() };
sync_maven(&root, &tmp.path().join("cache"), &options, &CancellationToken::new(), &|_| {}).unwrap();
let args = std::fs::read_to_string(root.join("mvnd-args")).unwrap();
// `-T1` pins the daemon serial so the dependency trees stay parseable.
assert!(args.contains("-T1"), "args: {args}");
assert!(args.contains("help:effective-pom"), "args: {args}");
}
}
+40 -5
View File
@@ -26,12 +26,12 @@ pub fn artifacts(root: &Path, model: &BuildModel) -> Vec<WebArtifact> {
});
}
}
if let Some(gradle) = &model.gradle {
for gradle in &model.gradle {
for p in &gradle.projects {
let script = p.build_file.as_ref().and_then(|f| fs::read_to_string(root.join(f)).ok()).unwrap_or_default();
if p.tasks.iter().any(|t| t.name == "war") || crate::gradle::has_war_plugin(&script) {
out.push(WebArtifact {
id: format!("gradle:{}", p.path),
id: gradle_artifact_id(gradle, &p.path),
name: p.name.clone(),
module: p.dir.clone(),
tool: "gradle".into(),
@@ -42,6 +42,14 @@ pub fn artifacts(root: &Path, model: &BuildModel) -> Vec<WebArtifact> {
out
}
fn gradle_artifact_id(build: &crate::GradleModel, path: &str) -> String {
if build.dir.is_empty() {
format!("gradle:{path}")
} else {
format!("gradle:{}", serde_json::to_string(&(&build.dir, path)).unwrap())
}
}
pub struct WebBuild {
pub command: String,
pub args: Vec<String>,
@@ -150,9 +158,14 @@ pub fn build(root: &Path, model: &BuildModel, id: &str, cache: &Path) -> IdeResu
maven_project: Some((probe, output)),
});
}
if let Some(path) = id.strip_prefix("gradle:") {
let g = model.gradle.as_ref().ok_or_else(|| IdeError::invalid("Choose a Gradle WAR module"))?;
if !artifacts(root, model).iter().any(|a| a.id == id) {
if let Some(legacy_path) = id.strip_prefix("gradle:") {
let identity = serde_json::from_str::<(String, String)>(legacy_path).ok();
let (g, path) = match &identity {
Some((dir, path)) => (model.gradle.iter().find(|g| g.dir == *dir), path.as_str()),
None => (model.gradle.first(), legacy_path),
};
let g = g.ok_or_else(|| IdeError::invalid("Choose a Gradle WAR module"))?;
if !artifacts(root, model).iter().any(|a| a.id == gradle_artifact_id(g, path)) {
return Err(IdeError::invalid("Choose a Gradle WAR module"));
}
let script = cache.join("war.init.gradle");
@@ -186,6 +199,28 @@ pub fn build(root: &Path, model: &BuildModel, id: &str, cache: &Path) -> IdeResu
#[cfg(test)]
mod tests {
#[test]
fn independent_war_modules_have_distinct_ids_and_build_in_their_own_directory() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path();
for dir in ["a", "b"] {
std::fs::create_dir(root.join(dir)).unwrap();
std::fs::write(root.join(dir).join("build.gradle"), "plugins { id 'war' }").unwrap();
}
let settings = ide_core::settings::GradleSettings {
linked_projects: Some(vec!["a".into(), "b".into()]),
..Default::default()
};
let model = crate::load_linked(root, &root.join("cache"), None, &Default::default(), &settings);
let artifacts = super::artifacts(root, &model);
assert_eq!(artifacts.len(), 2);
assert_ne!(artifacts[0].id, artifacts[1].id);
for (artifact, dir) in artifacts.iter().zip(["a", "b"]) {
let plan = super::build(root, &model, &artifact.id, &root.join(format!("cache/{dir}"))).unwrap();
assert_eq!(plan.cwd, dir);
}
assert_eq!(super::build(root, &model, "gradle::", &root.join("legacy")).unwrap().cwd, "a");
}
use super::*;
#[test]
fn maven_war_modules_use_resolved_artifact_file() {
+1 -1
View File
@@ -22,4 +22,4 @@ tempfile.workspace = true
libc = "0.2"
[target.'cfg(windows)'.dependencies]
windows-sys = { version = "0.61", features = ["Win32_Globalization", "Win32_Security", "Win32_System_Registry", "Win32_Foundation"] }
windows-sys = { version = "0.61", features = ["Win32_Globalization", "Win32_Security", "Win32_System_Registry", "Win32_Foundation", "Win32_System_Diagnostics_ToolHelp", "Win32_System_Threading"] }
+23
View File
@@ -2,6 +2,18 @@
//! treats as nonexistent — hidden from the project view and navigation, never
//! indexed or searched, never offered as unversioned changes.
use std::path::Path;
/// Whether `dir` lies in a Git (or jj) working tree: it or an ancestor holds `.git` / `.jj`.
///
/// Walkers pass it to `ignore::WalkBuilder::require_git`: inside a repository, `.gitignore`
/// rules then stop at a nested repository's root as in git, so an outer repository's
/// `.gitignore` does not hide a project that is its own repository. A folder outside any
/// repository still honours its `.gitignore`.
pub fn inside_git_repo(dir: &Path) -> bool {
dir.ancestors().any(|d| d.join(".git").exists() || d.join(".jj").exists())
}
/// IDEA's default list, plus `.jj` and the Windows shell files.
pub const DEFAULT_IGNORED_NAMES: &[&str] = &[
"*.pyc",
@@ -103,4 +115,15 @@ mod tests {
assert!(m.is_ignored_path("pkg/__pycache__/mod.cpython-312.pyc"));
assert!(!m.is_ignored_path("src/main.rs"));
}
#[test]
fn finds_the_repository_at_or_above_a_directory() {
let dir = tempfile::tempdir().unwrap();
let sub = dir.path().join("a/b");
std::fs::create_dir_all(&sub).unwrap();
assert!(!inside_git_repo(&sub));
std::fs::create_dir(dir.path().join("a/.git")).unwrap();
assert!(inside_git_repo(&sub));
assert!(inside_git_repo(&dir.path().join("a")));
}
}
@@ -36,6 +36,23 @@ impl Default for InspectionSettings {
}
}
impl InspectionSettings {
/// Overrides of the profile in effect; `Default`'s when `profile` names none that exists.
pub fn active(&self) -> &BTreeMap<String, InspectionOverride> {
let custom = |level: &str, key: &str| match level {
"ide" => self.ide_profiles.get(key),
"project" => self.project_profiles.get(key),
_ => None,
};
match self.profile.as_str() {
"project" => &self.project,
id => {
id.split_once(':').and_then(|(level, key)| custom(level, key)).map_or(&self.default, |p| &p.overrides)
}
}
}
}
/// A named profile made with Copy.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
@@ -111,4 +128,20 @@ mod tests {
let back: InspectionSettings = serde_json::from_str(&serde_json::to_string(&s).unwrap()).unwrap();
assert_eq!(back, s);
}
#[test]
fn active_profile_falls_back_to_default() {
let off = |id: &str| {
BTreeMap::from([(id.to_owned(), InspectionOverride { enabled: Some(false), ..Default::default() })])
};
let mut s = InspectionSettings { default: off("a"), project: off("b"), ..Default::default() };
s.ide_profiles.insert("strict".into(), CustomInspectionProfile { name: "Strict".into(), overrides: off("c") });
assert!(s.active().contains_key("a"));
s.profile = "project".into();
assert!(s.active().contains_key("b"));
s.profile = "ide:strict".into();
assert!(s.active().contains_key("c"));
s.profile = "project:gone".into();
assert!(s.active().contains_key("a"));
}
}
+122 -21
View File
@@ -43,19 +43,21 @@ pub fn own_group(cmd: &mut Command) -> &mut Command {
/// Kills the child and everything it started, then reaps it.
///
/// - Unix: `SIGKILL` to its process group, so spawn it with [`own_group`].
/// - Windows: `taskkill /T /F`, which walks the tree from the child's process id.
/// - Windows: `taskkill /T /F`, which walks the tree from the child's process id, then a
/// wait for what the child started to be gone.
pub fn kill_tree(child: &mut Child) {
#[cfg(unix)]
signal_group(child, libc::SIGKILL);
#[cfg(windows)]
taskkill_tree(child);
taskkill_tree_of(child.id());
let _ = child.kill();
let _ = child.wait();
}
/// [`kill_tree`] for a child known by its process id (a `tokio` child): `SIGKILL` to its
/// process group (spawn it with [`own_group`]) or `taskkill /T /F`. Call it before the child
/// is reaped, so the id still names it. The child itself is left to its owner to kill and reap.
/// process group (spawn it with [`own_group`]) or `taskkill /T /F` and a wait for what the
/// child started to be gone. Call it before the child is reaped, so the id still names it.
/// The child itself is left to its owner to kill and reap.
pub fn kill_tree_of(pid: u32) {
#[cfg(unix)]
if let Ok(pid) = libc::pid_t::try_from(pid) {
@@ -65,15 +67,7 @@ pub fn kill_tree_of(pid: u32) {
}
}
#[cfg(windows)]
{
let mut taskkill = Command::new("taskkill");
prepare(&mut taskkill)
.args(["/T", "/F", "/PID", &pid.to_string()])
.stdin(Stdio::null())
.stdout(Stdio::null())
.stderr(Stdio::null());
let _ = taskkill.status();
}
taskkill_tree_of(pid);
#[cfg(not(any(unix, windows)))]
let _ = pid;
}
@@ -106,15 +100,107 @@ fn signal_group(child: &Child, signal: libc::c_int) {
}
}
/// How long killing a tree waits for the processes the child started to be gone.
#[cfg(windows)]
fn taskkill_tree(child: &Child) {
const TREE_EXIT_WAIT: Duration = Duration::from_secs(5);
/// `taskkill /T /F` from `pid`, then waits (up to [`TREE_EXIT_WAIT`]) until what the child
/// started is gone: termination is asynchronous, and until a process is gone the files it
/// holds open can't be opened or deleted. The child itself is left to its owner to reap.
#[cfg(windows)]
fn taskkill_tree_of(pid: u32) {
let started = windows_tree::descendants(pid);
let mut taskkill = Command::new("taskkill");
prepare(&mut taskkill)
.args(["/T", "/F", "/PID", &child.id().to_string()])
.args(["/T", "/F", "/PID", &pid.to_string()])
.stdin(Stdio::null())
.stdout(Stdio::null())
.stderr(Stdio::null());
let _ = taskkill.status();
windows_tree::wait_all(&started, TREE_EXIT_WAIT);
}
#[cfg(windows)]
mod windows_tree {
use std::collections::HashSet;
use std::os::windows::io::{AsRawHandle, FromRawHandle, OwnedHandle};
use std::time::{Duration, Instant};
use windows_sys::Win32::Foundation::{FILETIME, INVALID_HANDLE_VALUE};
use windows_sys::Win32::System::Diagnostics::ToolHelp::{
CreateToolhelp32Snapshot, PROCESSENTRY32W, Process32FirstW, Process32NextW, TH32CS_SNAPPROCESS,
};
use windows_sys::Win32::System::Threading::{
GetProcessTimes, OpenProcess, PROCESS_QUERY_LIMITED_INFORMATION, PROCESS_SYNCHRONIZE, WaitForSingleObject,
};
/// Every process `root` started, directly or not, opened so it can be waited for. A
/// parent's id is reused once it is gone, so a process counts only if it started after
/// its parent.
pub(super) fn descendants(root: u32) -> Vec<OwnedHandle> {
let Some(root_started) = open(root).as_ref().and_then(started) else { return Vec::new() };
let processes = processes();
let mut seen = HashSet::from([root]);
let mut parents = vec![(root, root_started)];
let mut found = Vec::new();
while let Some((parent, parent_started)) = parents.pop() {
for &(pid, ppid) in &processes {
if ppid != parent || !seen.insert(pid) {
continue;
}
let Some(handle) = open(pid) else { continue };
if let Some(at) = started(&handle).filter(|&at| at >= parent_started) {
parents.push((pid, at));
found.push(handle);
}
}
}
found
}
pub(super) fn wait_all(processes: &[OwnedHandle], within: Duration) {
let until = Instant::now() + within;
for process in processes {
let left = until.saturating_duration_since(Instant::now());
// SAFETY: a live process handle opened with SYNCHRONIZE access.
unsafe { WaitForSingleObject(process.as_raw_handle(), left.as_millis().min(u32::MAX as u128 - 1) as u32) };
}
}
/// (process id, parent process id) of every running process.
fn processes() -> Vec<(u32, u32)> {
let mut out = Vec::new();
// SAFETY: a snapshot handle owned here and an entry sized as the API requires.
unsafe {
let snapshot = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
if snapshot.is_null() || snapshot == INVALID_HANDLE_VALUE {
return out;
}
let snapshot = OwnedHandle::from_raw_handle(snapshot);
let mut entry = PROCESSENTRY32W { dwSize: size_of::<PROCESSENTRY32W>() as u32, ..Default::default() };
let mut more = Process32FirstW(snapshot.as_raw_handle(), &mut entry);
while more != 0 {
out.push((entry.th32ProcessID, entry.th32ParentProcessID));
more = Process32NextW(snapshot.as_raw_handle(), &mut entry);
}
}
out
}
fn open(pid: u32) -> Option<OwnedHandle> {
// SAFETY: plain call; a non-null result is a process handle owned by the caller.
let handle = unsafe { OpenProcess(PROCESS_SYNCHRONIZE | PROCESS_QUERY_LIMITED_INFORMATION, 0, pid) };
// SAFETY: as above.
(!handle.is_null()).then(|| unsafe { OwnedHandle::from_raw_handle(handle) })
}
/// When the process started, in 100 ns ticks since 1601.
fn started(process: &OwnedHandle) -> Option<u64> {
let (mut created, mut exited, mut kernel, mut user) =
(FILETIME::default(), FILETIME::default(), FILETIME::default(), FILETIME::default());
// SAFETY: a live process handle opened with query access; four out-parameters.
let ok = unsafe { GetProcessTimes(process.as_raw_handle(), &mut created, &mut exited, &mut kernel, &mut user) };
(ok != 0).then(|| (u64::from(created.dwHighDateTime) << 32) | u64::from(created.dwLowDateTime))
}
}
/// Gives `cmd` the host's environment back when the IDE runs from an AppImage. `AppRun` and
@@ -123,12 +209,14 @@ fn taskkill_tree(child: &Child) {
/// inherits them loads the image's (older) libraries and plugins next to its own, e.g.
/// `curl: symbol lookup error ... nghttp2_...`. The IDE process keeps its environment
/// (WebKit and GTK still need it). Then applies the [`set_env_overlay`] variables and
/// those of a manual HTTP proxy ([`crate::proxy_settings::ProxySettings::child_env`]).
/// Variables the call site has already set are kept. A no-op outside an AppImage
/// without an overlay.
/// those of a manual HTTP proxy ([`crate::proxy_settings::ProxySettings::child_env`]),
/// and the login shell's variables this process lacks ([`shell_env`]: an app started from
/// Finder/Dock doesn't have what rc files export, such as `GOPATH`, `JAVA_HOME` or
/// `MAVEN_OPTS`). Variables the call site has already set are kept.
pub fn host_env(cmd: &mut Command) -> &mut Command {
let fixes = child_env_changes();
if fixes.is_empty() {
let shell: Vec<_> = shell_env().collect();
if fixes.is_empty() && shell.is_empty() {
return cmd;
}
let explicit: Vec<OsString> = cmd.get_envs().map(|(k, _)| k.to_owned()).collect();
@@ -138,6 +226,9 @@ pub fn host_env(cmd: &mut Command) -> &mut Command {
None => cmd.env_remove(key),
};
}
for (key, value) in shell.into_iter().filter(|(k, _)| !explicit.contains(k)) {
cmd.env(key, value);
}
cmd
}
@@ -154,12 +245,13 @@ pub fn host_vars() -> impl Iterator<Item = (OsString, OsString)> {
apply_changes(std::env::vars_os().collect(), &child_env_changes()).into_iter()
}
/// [`std::env::var_os`] as a child should see it (see [`host_env`]).
/// [`std::env::var_os`] as a child should see it (see [`host_env`]), the login shell's value
/// included when this process lacks the variable.
pub fn host_var_os(key: impl AsRef<OsStr>) -> Option<OsString> {
let key = key.as_ref();
match child_env_changes().into_iter().find(|(k, _)| k == key) {
Some((_, fixed)) => fixed,
None => std::env::var_os(key),
None => std::env::var_os(key).or_else(|| shell_env().find(|(k, _)| k == key).map(|(_, v)| v)),
}
}
@@ -572,6 +664,15 @@ mod tests {
assert_eq!(get("HOME"), Some(None));
let shell: Vec<_> = shell_env().map(|(k, _)| k).collect();
assert_eq!(shell, [OsString::from("OPENIDEA_OVERLAY_B")]);
// Every child gets what the login shell exports and the IDE process lacks (a Finder
// launch: `GOPATH`, `JAVA_HOME`...), unless the call site sets it.
assert_eq!(get("OPENIDEA_OVERLAY_B"), Some(Some("b".into())));
let mut own = Command::new("true");
own.env("OPENIDEA_OVERLAY_B", "call site");
host_env(&mut own);
let value = own.get_envs().find(|(n, _)| *n == "OPENIDEA_OVERLAY_B").and_then(|(_, v)| v);
assert_eq!(value, Some(OsStr::new("call site")));
assert_eq!(host_var_os("OPENIDEA_OVERLAY_B"), Some("b".into()));
assert_eq!(host_var_os("PATH"), Some("/reloaded".into()));
assert!(host_vars().all(|(k, _)| k != "HOME"));
assert!(host_vars().any(|(k, v)| k == "OPENIDEA_OVERLAY_A" && v == "project"));
+103
View File
@@ -157,6 +157,16 @@ pub struct LanguageServerSettings {
pub languages: Vec<String>,
/// Files marking a workspace root for a server defined here (`pyproject.toml`).
pub root_markers: Vec<String>,
/// Replaces the definition's `features`: what the server is used for
/// (`code-action`, `hover`, …); an empty list means everything.
#[serde(skip_serializing_if = "Option::is_none")]
#[specta(optional)]
pub features: Option<Vec<String>>,
/// Replaces the definition's `code-action-kinds` (`refactor`); an empty
/// list keeps every kind.
#[serde(skip_serializing_if = "Option::is_none")]
#[specta(optional)]
pub code_action_kinds: Option<Vec<String>>,
}
impl Default for LanguageServerSettings {
@@ -170,6 +180,8 @@ impl Default for LanguageServerSettings {
settings: None,
languages: Vec::new(),
root_markers: Vec::new(),
features: None,
code_action_kinds: None,
}
}
}
@@ -339,6 +351,7 @@ pub struct AppearanceSettings {
pub preferred_dark_theme: String,
/// A plugin's file icon theme (`<plugin-id>/<theme-id>`); empty = the built-in icons.
pub file_icon_theme: String,
pub file_icon_theme_options: BTreeMap<String, FileIconThemeOptions>,
/// IDE zoom in percent (50–300).
pub zoom: u32,
/// Apply `ui_font_family` / `ui_font_size` (otherwise the theme's).
@@ -374,6 +387,24 @@ pub struct AppearanceSettings {
/// Main toolbar entries per side, in order: title widget ids (`project`,
/// `vcs.branch`…) or `action:<id>`. Widgets in neither list nor `hidden`
/// (registered later) keep their default place.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct FileIconThemeOptions {
pub groups: BTreeMap<String, bool>,
pub choices: BTreeMap<String, String>,
pub hide_files: bool,
pub hide_folders: bool,
pub hollow_folders: bool,
pub hide_ui_icons: bool,
pub hide_psi_icons: bool,
pub monochrome: Option<String>,
pub saturation: Option<u8>,
pub opacity: Option<u8>,
pub icon_size: Option<u8>,
pub row_height: Option<u8>,
pub accent: Option<String>,
}
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct MainToolbarSettings {
@@ -394,6 +425,7 @@ impl Default for AppearanceSettings {
preferred_light_theme: "light".into(),
preferred_dark_theme: "dark".into(),
file_icon_theme: String::new(),
file_icon_theme_options: BTreeMap::new(),
zoom: 100,
use_custom_font: false,
ui_font_family: "Inter".into(),
@@ -681,6 +713,15 @@ impl Default for PythonSettings {
pub struct PhpSettings {
/// `php` CLI executable; empty = the one on `PATH`.
pub interpreter: String,
/// PHP version the language server checks the code against (`7.4`); empty = from
/// `composer.json` (`config.platform.php`, else the lowest version `require.php` allows).
pub language_level: String,
/// Folders outside the project whose code the language server indexes too (a platform a
/// module extends); absolute, `~/`, or relative to the project.
pub include_paths: Vec<String>,
/// Intelephense stub sets loaded on top of its defaults and the `ext-*` entries of
/// `composer.json` (`wordpress`).
pub stubs: Vec<String>,
}
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize, Type)]
@@ -787,6 +828,8 @@ pub struct JavaSettings {
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct GradleSettings {
/// Linked build directories; absent preserves automatic discovery, empty unlinks all builds.
pub linked_projects: Option<Vec<String>>,
/// Empty = Project JDK; otherwise a JDK home.
pub jvm: String,
pub distribution: GradleDistribution,
@@ -823,12 +866,20 @@ pub enum GradleDistribution {
pub struct MavenSettings {
/// Project-relative root POMs removed from the Maven project model.
pub unlinked_projects: Vec<String>,
/// Project-relative POMs outside the root reactor added as independent top-level projects.
pub linked_projects: Vec<String>,
/// Maven home (or its `mvn` executable); empty = the project's `mvnw`, else `mvn` on `PATH`.
pub home: String,
/// User `settings.xml` (`-s`); empty = `~/.m2/settings.xml`.
pub user_settings: String,
/// Local repository (`-Dmaven.repo.local`); empty = from `settings.xml`.
pub local_repository: String,
/// Run Maven through the Maven Daemon (`mvnd`) for faster builds. mvnd embeds
/// its own Maven, so `home` and the project's `mvnw` are bypassed when it is on.
pub daemon: bool,
/// Maven Daemon home (or its `mvnd` executable); empty = `$MVND_HOME`, else
/// `mvnd` on `PATH` / common install dirs. Only used when `daemon` is on.
pub mvnd_home: String,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
@@ -919,6 +970,36 @@ mod tests {
assert_eq!(serde_json::from_value::<Settings>(value).unwrap(), settings);
}
#[test]
fn file_icon_theme_options_default_empty_and_round_trip() {
let old: Settings = serde_json::from_str(r#"{"appearance":{"fileIconTheme":"p/t"}}"#).unwrap();
assert!(old.appearance.file_icon_theme_options.is_empty());
let json = serde_json::json!({"appearance":{"fileIconThemeOptions":{"p/t":{
"groups":{"tests":true},"choices":{"arrows":"plusminus"},"hideFiles":true,"hideFolders":false,"hollowFolders":true,"hideUiIcons":true,"hidePsiIcons":false,
"monochrome":"#546e7a","saturation":5,"opacity":80,"iconSize":20,"rowHeight":30,"accent":"#009688"}}}});
let settings: Settings = serde_json::from_value(json.clone()).unwrap();
let options = &settings.appearance.file_icon_theme_options["p/t"];
assert!(options.hide_files && options.hollow_folders && !options.hide_folders);
assert!(options.hide_ui_icons && !options.hide_psi_icons);
assert!(options.groups["tests"]);
assert_eq!(options.choices["arrows"], "plusminus");
assert_eq!(
(options.saturation, options.opacity, options.icon_size, options.row_height),
(Some(5), Some(80), Some(20), Some(30))
);
assert_eq!(options.monochrome.as_deref(), Some("#546e7a"));
assert_eq!(options.accent.as_deref(), Some("#009688"));
let value = serde_json::to_value(&settings).unwrap();
let restored: Settings = serde_json::from_value(value).unwrap();
assert_eq!(restored, settings);
let plain = serde_json::to_value(FileIconThemeOptions::default()).unwrap();
assert_eq!(plain["hideFiles"], false);
assert_eq!(plain["hideUiIcons"], false);
assert_eq!(plain["hidePsiIcons"], false);
assert_eq!(plain["monochrome"], serde_json::Value::Null);
assert_eq!(plain["groups"], serde_json::json!({}));
}
#[test]
fn ui_font_fallback_defaults_empty_and_round_trips() {
let old: Settings = serde_json::from_str(r#"{"appearance":{"useCustomFont":true}}"#).unwrap();
@@ -1030,12 +1111,33 @@ mod maven_linkage_tests {
assert!(encoded.contains("unlinkedProjects"));
assert_eq!(serde_json::from_str::<MavenSettings>(&encoded).unwrap(), settings);
}
#[test]
fn daemon_settings_round_trip() {
// Old settings files predate the daemon fields; `serde(default)` keeps them loadable.
let old: MavenSettings = serde_json::from_str(r#"{"home":"/opt/maven"}"#).unwrap();
assert!(!old.daemon && old.mvnd_home.is_empty());
let settings = MavenSettings { daemon: true, mvnd_home: "/opt/mvnd".into(), ..old };
let encoded = serde_json::to_string(&settings).unwrap();
assert!(encoded.contains("\"daemon\":true") && encoded.contains("mvndHome"));
assert_eq!(serde_json::from_str::<MavenSettings>(&encoded).unwrap(), settings);
}
}
#[cfg(test)]
mod gradle_settings_tests {
use super::*;
#[test]
fn absent_and_explicitly_empty_gradle_links_remain_distinct() {
let automatic: GradleSettings = serde_json::from_str("{}").unwrap();
let unlinked: GradleSettings = serde_json::from_str(r#"{"linkedProjects":[]}"#).unwrap();
assert_eq!(automatic.linked_projects, None);
assert_eq!(unlinked.linked_projects, Some(vec![]));
let restored: GradleSettings = serde_json::from_str(&serde_json::to_string(&unlinked).unwrap()).unwrap();
assert_eq!(restored.linked_projects, Some(vec![]));
}
#[test]
fn defaults_and_round_trip() {
let old: JavaSettings = serde_json::from_str(r#"{"jdk":"/jdk"}"#).unwrap();
@@ -1048,6 +1150,7 @@ mod gradle_settings_tests {
home: "/gradle".into(),
user_home: "/cache/gradle".into(),
run_using: GradleRunUsing::Ide,
linked_projects: Some(vec!["backend".into(), "../shared".into()]),
};
let restored: Settings = serde_json::from_str(&serde_json::to_string(&settings).unwrap()).unwrap();
assert_eq!(restored.languages.java.gradle, settings.languages.java.gradle);
+7 -1
View File
@@ -72,7 +72,13 @@ fn builtin(adapter: &str) -> Option<DebuggerConfig> {
"stdio",
"Install the Kotlin Debug Adapter from Settings > Language Servers",
),
"debugpy" => cfg("python3", &["-m", "debugpy.adapter"], "stdio", "pip install debugpy"),
// Windows installs `python.exe`; `python3.exe` there is at most a Store alias.
"debugpy" => cfg(
if cfg!(windows) { "python" } else { "python3" },
&["-m", "debugpy.adapter"],
"stdio",
"pip install debugpy",
),
"js-debug" => cfg(
"js-debug-adapter",
// Without a host it listens on `localhost`, which may resolve to ::1 only.
+72 -6
View File
@@ -212,6 +212,19 @@ pub struct Variable {
pub evaluate_name: Option<String>,
}
/// A completion the adapter offers for an expression (DAP `completions`).
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct DebugCompletion {
pub label: String,
/// What to insert, when it differs from `label`.
pub text: Option<String>,
/// DAP item type: `method`, `field`, `variable`, `class`, `keyword`, …
pub kind: Option<String>,
pub detail: Option<String>,
pub sort_text: Option<String>,
}
#[derive(Clone, Copy, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum StepKind {
@@ -576,11 +589,12 @@ impl DebugManager {
if let Some(command) = custom_command(&spec.command) {
spec.command = command;
}
// debugpy: run the adapter with the debuggee's interpreter (venvs carry their own debugpy).
// debugpy: run the adapter with the debuggee's interpreter (venvs carry their own debugpy),
// or with the launcher's when the debuggee runs through a manager (`uv run python`).
if config.adapter == "debugpy" {
let python = match arguments.get("python") {
Some(Value::String(p)) => Some(p.clone()),
Some(Value::Array(a)) => a.first().and_then(Value::as_str).map(str::to_owned),
let python = match (arguments.get("debugLauncherPython"), arguments.get("python")) {
(Some(Value::String(p)), _) | (_, Some(Value::String(p))) => Some(p.clone()),
(_, Some(Value::Array(a))) => a.first().and_then(Value::as_str).map(str::to_owned),
_ => None,
};
if let Some(python) = python {
@@ -638,8 +652,19 @@ impl DebugManager {
{
arguments["project"] = json!(real);
}
if config.adapter == "codelldb" && arguments.get("sourceLanguages").is_none() {
arguments["sourceLanguages"] = json!(["rust"]);
if config.adapter == "codelldb" {
if arguments.get("sourceLanguages").is_none() {
arguments["sourceLanguages"] = json!(["rust"]);
}
// The Debug Console and Evaluate Expression take expressions, as with every other
// debugger; LLDB commands still run with a leading backtick.
match arguments.get_mut("_adapterSettings") {
None => arguments["_adapterSettings"] = json!({"consoleMode": "evaluate"}),
Some(Value::Object(settings)) => {
settings.entry("consoleMode").or_insert_with(|| json!("evaluate"));
}
Some(_) => {}
}
}
let port = debug_server.map(|s| s.port);
self.start_session(id, owner, config.name, config.adapter, None, config.request, arguments, spec, port).await
@@ -1157,6 +1182,47 @@ impl DebugManager {
})
}
/// Completions for `text` at `line` / `column` (0-based, UTF-16) in the frame's scope; none
/// when the adapter doesn't offer them.
pub async fn completions(
&self,
s: DebugSessionId,
frame_id: Option<i64>,
text: String,
line: u32,
column: u32,
) -> IdeResult<Vec<DebugCompletion>> {
let session = self.session(s)?;
if !session.supports("supportsCompletionsRequest") {
return Ok(Vec::new());
}
let mut args = json!({ "text": text, "line": line + 1, "column": column + 1 });
if let Some(f) = frame_id {
args["frameId"] = f.into();
}
let body = session.conn.request_timeout("completions", args, REQUEST_TIMEOUT).await?;
let text =
|v: &Value, key: &str| v.get(key).and_then(Value::as_str).filter(|t| !t.is_empty()).map(str::to_owned);
Ok(body
.get("targets")
.and_then(Value::as_array)
.map(|targets| {
targets
.iter()
.filter_map(|t| {
Some(DebugCompletion {
label: text(t, "label")?,
text: text(t, "text"),
kind: text(t, "type"),
detail: text(t, "detail"),
sort_text: text(t, "sortText"),
})
})
.collect()
})
.unwrap_or_default())
}
pub async fn resume(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
let body = session.conn.request_timeout("continue", json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await?;
+10 -6
View File
@@ -111,8 +111,8 @@ impl Session {
if let Some(path) = self.kotlin_mirror.lock().as_ref().and_then(|m| m.adapter_path(path)) {
return path;
}
if self.adapter == "java-debug" {
let native = java_source_path(&path.to_string_lossy());
if self.adapter == "java-debug" || (cfg!(windows) && self.adapter == "netcoredbg") {
let native = windows_source_path(&path.to_string_lossy());
self.resolved_paths.lock().insert(native.clone(), path.to_path_buf());
return native;
}
@@ -133,7 +133,11 @@ impl Session {
if let Some(real) = self.kotlin_mirror.lock().as_ref().and_then(|m| m.client_path(Path::new(path))) {
return Some(real);
}
let key = if self.adapter == "java-debug" { java_source_path(path) } else { PathBuf::from(path) };
let key = if self.adapter == "java-debug" || (cfg!(windows) && self.adapter == "netcoredbg") {
windows_source_path(path)
} else {
PathBuf::from(path)
};
self.resolved_paths.lock().get(&key).cloned().or_else(|| {
(self.adapter == "java-debug" && is_windows_source_path(path))
.then(|| PathBuf::from(path.replace('\\', "/")))
@@ -265,8 +269,8 @@ impl Session {
}
}
// java-debug treats a forward-slash drive path as a URI (scheme "C").
// Keep DAP path mode, but use native Windows separators for Java source lookup.
// java-debug and Windows netcoredbg need native separators for source lookup.
// Keep DAP path mode, but normalize Windows drive and UNC paths.
fn is_windows_source_path(path: &str) -> bool {
let bytes = path.as_bytes();
(bytes.len() >= 3 && bytes[0].is_ascii_alphabetic() && bytes[1] == b':' && matches!(bytes[2], b'/' | b'\\'))
@@ -274,7 +278,7 @@ fn is_windows_source_path(path: &str) -> bool {
|| path.starts_with(r"\\")
}
fn java_source_path(path: &str) -> PathBuf {
fn windows_source_path(path: &str) -> PathBuf {
if is_windows_source_path(path) { PathBuf::from(path.replace('/', r"\")) } else { PathBuf::from(path) }
}
+66
View File
@@ -73,6 +73,7 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
msg,
json!({
"supportsConfigurationDoneRequest": true,
"supportsCompletionsRequest": true,
"supportsTerminateRequest": true,
"supportsSetVariable": true,
"supportsFunctionBreakpoints": true,
@@ -172,6 +173,14 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
]})
),
"evaluate" => ok!(msg, json!({"result": "42", "type": "int", "variablesReference": 0})),
"completions" => ok!(
msg,
json!({"targets": [
{"label": "getName()", "text": "getName", "type": "method", "detail": "String"},
{"label": "name", "type": "field"},
{"label": ""},
]})
),
"continue" => {
ok!(msg, json!({"allThreadsContinued": true}));
send!(json!({"type": "event", "event": "exited", "body": {"exitCode": 3}}));
@@ -368,6 +377,14 @@ async fn full_protocol_flow() {
let eval = h.request("evaluate");
assert_eq!(eval["arguments"]["context"], "watch");
assert_eq!(eval["arguments"]["frameId"], 10);
let items = h.manager.completions(id, Some(10), "req.na".into(), 0, 6).await.unwrap();
assert_eq!(
items.iter().map(|c| (c.label.as_str(), c.text.as_deref(), c.kind.as_deref())).collect::<Vec<_>>(),
[("getName()", Some("getName"), Some("method")), ("name", None, Some("field"))]
);
let asked = h.request("completions");
assert_eq!((asked["arguments"]["line"].clone(), asked["arguments"]["column"].clone()), (json!(1), json!(7)));
assert_eq!(asked["arguments"]["frameId"], 10);
// Live breakpoint updates reach the session.
let states = h.manager.set_breakpoints(path.clone(), vec![bp(0)]).await.unwrap();
@@ -681,6 +698,37 @@ async fn js_debug_launch_gets_port_and_workspace_folder() {
assert_eq!(launch["type"], "pwa-node");
}
/// CodeLLDB treats console input as LLDB commands unless told otherwise: Evaluate Expression
/// and the Debug Console send expressions, so its console evaluates them. A launch
/// configuration that picks a console mode keeps it.
#[tokio::test]
async fn codelldb_console_evaluates_expressions() {
let launch_with = |extra: Value| async move {
let h = harness(Script::default());
let config = LaunchConfig {
name: "demo".into(),
adapter: "codelldb".into(),
request: "launch".into(),
program: Some("/p/target/debug/demo".into()),
args: vec![],
cwd: Some("/p".into()),
env: BTreeMap::new(),
extra_json: Some(extra.to_string()),
};
h.start(config).await.unwrap();
h.request("launch")["arguments"].clone()
};
let launch = launch_with(json!({})).await;
assert_eq!(launch["_adapterSettings"]["consoleMode"], "evaluate");
assert_eq!(launch["sourceLanguages"], json!(["rust"]));
let launch =
launch_with(json!({"_adapterSettings": {"consoleMode": "commands", "showDisassembly": "never"}})).await;
assert_eq!(launch["_adapterSettings"], json!({"consoleMode": "commands", "showDisassembly": "never"}));
let launch = launch_with(json!({"_adapterSettings": {"showDisassembly": "never"}})).await;
assert_eq!(launch["_adapterSettings"], json!({"consoleMode": "evaluate", "showDisassembly": "never"}));
}
/// An adapter that is already listening (java-debug inside jdtls): the client
/// connects over TCP to the `debugServer` port instead of spawning anything.
#[tokio::test]
@@ -906,6 +954,7 @@ fn godot_registry(port: u16, command: &str, args: &[&str]) -> Arc<LanguageRegist
ui_mode: None,
framework_ui_modes: Vec::new(),
formatter: None,
scratch_template: None,
tree_sitter: None,
language_servers: vec![],
debugger: Some(ide_language::DebuggerConfig {
@@ -1092,6 +1141,23 @@ async fn kotlin_official_release_ten_no_delay_runs() {
}
}
#[cfg(windows)]
#[tokio::test]
async fn netcoredbg_windows_source_paths_round_trip() {
let client = "D:/work/LoginController.cs";
let native = r"D:\work\LoginController.cs";
let h = harness(Script { source_path: Some(native.into()), ..Script::default() });
let path = PathBuf::from(client);
h.manager.set_breakpoints(path.clone(), vec![bp(25)]).await.unwrap();
let mut cfg = config();
cfg.adapter = "netcoredbg".into();
let id = h.start(cfg).await.unwrap();
assert_eq!(h.request("setBreakpoints")["arguments"]["source"]["path"], native);
let frames = h.manager.stack_trace(id, 1).await.unwrap();
assert_eq!(frames[0].path.as_deref(), Some(path.as_path()));
h.manager.stop(id).await.unwrap();
}
#[tokio::test]
async fn java_windows_source_paths_for_launch_and_attach() {
for request in ["launch", "attach"] {
+6 -1
View File
@@ -8,7 +8,7 @@ license.workspace = true
[dependencies]
ide-core.workspace = true
serde.workspace = true
serde_json.workspace = true
serde_json = { workspace = true, features = ["raw_value"] }
specta.workspace = true
toml.workspace = true
tokio.workspace = true
@@ -21,7 +21,12 @@ bb8.workspace = true
tokio-util = { workspace = true, features = ["compat"] }
dirs.workspace = true
ide-ssh.workspace = true
ide-http.workspace = true
reqwest = { workspace = true, features = ["query"] }
rustls.workspace = true
uuid.workspace = true
[dev-dependencies]
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
tempfile.workspace = true
ide-ssh = { workspace = true, features = ["testing"] }
+889
View File
@@ -0,0 +1,889 @@
use crate::conn::{QueryResult, ResultColumn, Setup};
use crate::source::{DataSource, DbSslMode, DbTxMode};
use ide_core::{CancellationToken, IdeError, IdeResult, error::ErrorKind};
use parking_lot::RwLock;
use reqwest::{Client, RequestBuilder, StatusCode, Url};
use std::sync::Arc;
use std::time::{Duration, Instant};
use tokio::sync::Mutex;
#[derive(Clone)]
pub struct ClickhousePool(Arc<State>);
struct State {
client: RwLock<Option<Client>>,
endpoint: Url,
user: String,
password: String,
database: String,
settings: Vec<(String, String)>,
read_only: bool,
session_id: String,
session: Mutex<bool>,
closed: CancellationToken,
}
/// The response format: a names line, a types line, then one JSON array per row, so rows past
/// the grid's limit are skipped without being parsed.
const FORMAT: &str = "JSONCompactEachRowWithNamesAndTypes";
pub(crate) fn transaction_error() -> IdeError {
IdeError::invalid("ClickHouse HTTP does not support SQL transactions; use auto-commit and run writes directly")
}
impl ClickhousePool {
pub(crate) async fn connect(source: &DataSource, password: Option<&str>, setup: Setup) -> IdeResult<Self> {
if source.options.transaction == DbTxMode::Manual {
return Err(transaction_error());
}
for (name, _) in &setup.params {
if reserved_setting(name) {
return Err(IdeError::invalid(format!("ClickHouse HTTP manages the {name} setting")));
}
}
let secure = matches!(source.ssl, DbSslMode::Require | DbSslMode::VerifyCa | DbSslMode::VerifyFull);
let tls_files = [&source.ssl_ca_file, &source.ssl_cert_file, &source.ssl_key_file];
if !secure && tls_files.iter().any(|p| !p.trim().is_empty()) {
return Err(IdeError::invalid(
"ClickHouse TLS certificate files require an SSL mode of Require, Verify CA, or Verify Full",
));
}
let host = source.host.trim();
if host.is_empty() || host.contains(['/', '@', '?', '#']) {
return Err(IdeError::invalid("ClickHouse host must contain only a hostname or IP address"));
}
let host = if host.contains(':') && !host.starts_with('[') { format!("[{host}]") } else { host.to_owned() };
let scheme = if secure { "https" } else { "http" };
let port = source.port.unwrap_or(if secure { 8443 } else { 8123 });
let endpoint = Url::parse(&format!("{scheme}://{host}:{port}/"))
.map_err(|_| IdeError::invalid("invalid ClickHouse HTTP host or port"))?;
let timeout = setup.connect_timeout.unwrap_or(Duration::from_secs(30));
let _ = rustls::crypto::ring::default_provider().install_default();
let mut builder = Client::builder()
.connect_timeout(timeout)
.pool_max_idle_per_host(4)
.redirect(reqwest::redirect::Policy::none())
.user_agent(concat!("OpenIDEA/", env!("CARGO_PKG_VERSION")));
let ca = source.ssl_ca_file.trim();
if !ca.is_empty() {
let pem = read_pem(ca, "CA certificate")?;
let certificates = reqwest::Certificate::from_pem_bundle(&pem)
.map_err(|_| IdeError::invalid("invalid ClickHouse CA certificate PEM"))?;
if certificates.is_empty() {
return Err(IdeError::invalid("ClickHouse CA certificate PEM contains no certificates"));
}
builder = builder.tls_certs_only(certificates);
}
if source.ssl == DbSslMode::VerifyCa {
if ca.is_empty() {
return Err(IdeError::invalid("ClickHouse Verify CA requires a CA certificate file"));
}
builder = builder.tls_danger_accept_invalid_hostnames(true);
}
let cert = source.ssl_cert_file.trim();
let key = source.ssl_key_file.trim();
if cert.is_empty() != key.is_empty() {
return Err(IdeError::invalid("ClickHouse client TLS requires both certificate and key files"));
}
if !cert.is_empty() {
let mut pem = read_pem(cert, "client certificate")?;
pem.push(b'\n');
pem.extend(read_pem(key, "client key")?);
let identity = reqwest::Identity::from_pem(&pem)
.map_err(|_| IdeError::invalid("invalid ClickHouse client certificate or key PEM"))?;
builder = builder.identity(identity);
}
let proxy = ide_core::proxy_settings::proxy();
let client = ide_http::client::with_proxy(builder, &proxy).build().map_err(http_error)?;
let database = if source.options.default_schema.trim().is_empty() {
source.database.trim()
} else {
source.options.default_schema.trim()
};
let pool = Self(Arc::new(State {
client: RwLock::new(Some(client)),
endpoint,
user: if source.user.is_empty() { "default".into() } else { source.user.clone() },
password: password.unwrap_or_default().to_owned(),
database: if database.is_empty() { "default".into() } else { database.into() },
settings: setup.params,
read_only: source.read_only,
session_id: uuid::Uuid::new_v4().to_string(),
session: Mutex::new(false),
closed: CancellationToken::new(),
}));
let cancel = CancellationToken::new();
let initialize = async {
pool.execute("SELECT 1", 1, &cancel).await?;
for sql in setup.statements {
pool.execute(&sql, 0, &cancel).await?;
}
Ok(pool.clone())
};
tokio::pin!(initialize);
tokio::select! {
result = &mut initialize => {
if result.is_err() {
pool.close();
}
result
},
() = tokio::time::sleep(timeout) => {
cancel.cancel();
let _ = initialize.await;
pool.close();
Err(IdeError::new(ErrorKind::TimedOut, "timed out connecting to ClickHouse HTTP"))
}
}
}
pub fn close(&self) {
self.0.closed.cancel();
self.0.client.write().take();
}
pub fn is_closed(&self) -> bool {
self.0.closed.is_cancelled()
}
fn client(&self) -> IdeResult<Client> {
self.0.client.read().clone().ok_or_else(|| IdeError::new(ErrorKind::Io, "ClickHouse connection is closed"))
}
/// `max_rows`: the rows the server may send (the skipped ones included).
fn request(&self, client: &Client, query_id: &str, max_rows: usize, initialized: bool) -> RequestBuilder {
let mut request = client
.post(self.0.endpoint.clone())
.basic_auth(&self.0.user, Some(&self.0.password))
.header(reqwest::header::CONTENT_TYPE, "text/plain; charset=utf-8")
.query(&self.0.settings)
.query(&[
("session_id", self.0.session_id.clone()),
("session_timeout", "3600".into()),
("query_id", query_id.to_owned()),
("default_format", FORMAT.into()),
("output_format_json_quote_denormals", "1".into()),
("max_result_rows", max_rows.saturating_add(1).to_string()),
("result_overflow_mode", "break".into()),
]);
if initialized {
request = request.query(&[("session_check", "1")]);
} else {
request = request.query(&[("database", &self.0.database)]);
}
if self.0.read_only {
request = request.query(&[("readonly", "1")]);
}
request
}
pub async fn execute(&self, sql: &str, max_rows: usize, cancel: &CancellationToken) -> IdeResult<QueryResult> {
self.execute_from(sql, 0, max_rows, cancel).await
}
/// [`execute`](Self::execute), reading past the first `skip` rows without parsing or keeping them.
pub async fn execute_from(
&self,
sql: &str,
skip: usize,
max_rows: usize,
cancel: &CancellationToken,
) -> IdeResult<QueryResult> {
if transaction_statement(sql) {
return Err(transaction_error());
}
let mut session = tokio::select! {
biased;
() = cancel.cancelled() => return Err(IdeError::cancelled()),
() = self.0.closed.cancelled() => return Err(IdeError::new(ErrorKind::Io, "ClickHouse connection is closed")),
session = self.0.session.lock() => session,
};
let client = self.client()?;
let query_id = uuid::Uuid::new_v4().to_string();
let request = self.request(&client, &query_id, skip.saturating_add(max_rows), *session).body(sql.to_owned());
let started = Instant::now();
let fetch = async {
let mut response = request.send().await.map_err(http_error)?;
let execution_ms = started.elapsed().as_secs_f64() * 1000.0;
let status = response.status();
let header = |name: &str| response.headers().get(name).and_then(|v| v.to_str().ok()).map(str::to_owned);
let exception_code = header("X-ClickHouse-Exception-Code");
let format = header("X-ClickHouse-Format");
let affected = header("X-ClickHouse-Summary")
.and_then(|v| serde_json::from_str::<serde_json::Value>(&v).ok())
.and_then(|v| {
v.get("written_rows")
.and_then(|n| n.as_str().and_then(|n| n.parse::<u64>().ok()).or_else(|| n.as_u64()))
});
if !status.is_success() || exception_code.as_deref().is_some_and(|code| code != "0") {
let body = response.bytes().await.map_err(http_error)?;
return Err(self.server_error(status, exception_code.as_deref(), &body));
}
let (columns, rows, has_more) = if format.as_deref().is_some_and(|f| f != FORMAT) {
let body = response.bytes().await.map_err(http_error)?;
if !body.iter().all(u8::is_ascii_whitespace) {
return Err(IdeError::invalid(
"ClickHouse result grids need the default output format; remove the query's FORMAT clause",
));
}
(Vec::new(), Vec::new(), false)
} else {
let mut reader = RowReader::new(skip, max_rows);
while let Some(chunk) = response.chunk().await.map_err(http_error)? {
reader.push(&chunk)?;
}
reader.finish().map_err(|error| self.redact_error(error))?
};
let affected = if columns.is_empty() { affected.map(|n| n as f64) } else { None };
Ok(QueryResult {
statement: sql.to_owned(),
columns,
rows,
affected,
has_more,
elapsed_ms: started.elapsed().as_secs_f64() * 1000.0,
execution_ms,
})
};
let result = tokio::select! {
biased;
() = cancel.cancelled() => None,
() = self.0.closed.cancelled() => None,
result = fetch => Some(result),
};
match result {
Some(result) => {
if result.is_ok() {
*session = true;
}
result
}
None => {
let killed = self.kill(&client, &query_id).await;
match killed {
Ok(()) => Err(IdeError::cancelled()),
Err(error) => Err(IdeError::new(
ErrorKind::Cancelled,
format!("cancelled locally; ClickHouse server cancellation failed: {}", error.message),
)),
}
}
}
}
async fn kill(&self, client: &Client, query_id: &str) -> IdeResult<()> {
let response = client
.post(self.0.endpoint.clone())
.basic_auth(&self.0.user, Some(&self.0.password))
.timeout(Duration::from_secs(3))
.body(format!("KILL QUERY WHERE query_id = '{query_id}' SYNC"))
.send()
.await
.map_err(http_error)?;
let status = response.status();
let code = response
.headers()
.get("X-ClickHouse-Exception-Code")
.and_then(|value| value.to_str().ok())
.map(str::to_owned);
let body = response.bytes().await.map_err(http_error)?;
if status.is_success() && !code.as_deref().is_some_and(|code| code != "0") && !body.starts_with(b"Code:") {
Ok(())
} else {
Err(self.server_error(status, code.as_deref(), &body))
}
}
fn redact_error(&self, mut error: IdeError) -> IdeError {
if !self.0.password.is_empty() {
error.message = error.message.replace(&self.0.password, "[redacted]");
}
error
}
fn server_error(&self, status: StatusCode, code: Option<&str>, body: &[u8]) -> IdeError {
let message = String::from_utf8_lossy(body);
let message: String = exception_text(&message).chars().take(16_384).collect();
let auth = matches!(status, StatusCode::UNAUTHORIZED | StatusCode::FORBIDDEN)
|| matches!(code, Some("193" | "194" | "195" | "516" | "497"));
let detail = if message.is_empty() { status.canonical_reason().unwrap_or("request failed") } else { &message };
let detail = if detail.contains("SESSION_NOT_FOUND") || detail.contains("Session not found") {
"ClickHouse HTTP session expired; reconnect to restore session settings"
} else {
detail
};
self.redact_error(IdeError::new(
if auth { ErrorKind::PermissionDenied } else { ErrorKind::InvalidInput },
format!("ClickHouse HTTP {}: {detail}", status.as_u16()),
))
}
}
type DecodedResult = (Vec<ResultColumn>, Vec<Vec<Option<String>>>, bool);
/// Decodes a streamed [`FORMAT`] body. Reads past the first `skip` rows unparsed (a later
/// page), then keeps `max_rows` rows; later rows only set `has_more` and are read past
/// unparsed, so a grid limit bounds memory however large the block
/// `result_overflow_mode=break` lets the server send. A line that is not a JSON array
/// starts the exception ClickHouse writes when a query fails mid-stream.
struct RowReader {
skip: usize,
max_rows: usize,
pending: Vec<u8>,
names: Option<Vec<String>>,
types: Option<Vec<String>>,
rows: Vec<Vec<Option<String>>>,
more: bool,
exception: Option<String>,
}
impl RowReader {
fn new(skip: usize, max_rows: usize) -> Self {
Self {
skip,
max_rows,
pending: Vec::new(),
names: None,
types: None,
rows: Vec::new(),
more: false,
exception: None,
}
}
fn push(&mut self, chunk: &[u8]) -> IdeResult<()> {
self.pending.extend_from_slice(chunk);
let buffer = std::mem::take(&mut self.pending);
let mut start = 0;
while let Some(end) = buffer[start..].iter().position(|&b| b == b'\n') {
self.line(&buffer[start..start + end])?;
start += end + 1;
}
self.pending = buffer[start..].to_vec();
Ok(())
}
fn line(&mut self, line: &[u8]) -> IdeResult<()> {
if let Some(exception) = &mut self.exception {
if exception.len() < 16_384 {
exception.push_str(&String::from_utf8_lossy(line));
exception.push('\n');
}
return Ok(());
}
let text = line.trim_ascii();
if text.is_empty() {
return Ok(());
}
if text[0] != b'[' {
self.exception = Some(format!("{}\n", String::from_utf8_lossy(line)));
return Ok(());
}
if self.names.is_some() && self.types.is_some() {
if self.skip > 0 {
self.skip -= 1;
return Ok(());
}
if self.rows.len() >= self.max_rows {
self.more = true;
return Ok(());
}
}
let Ok(values) = serde_json::from_slice::<Vec<Box<serde_json::value::RawValue>>>(text) else {
self.exception = Some(format!("{}\n", String::from_utf8_lossy(line)));
return Ok(());
};
let protocol = |message: String| IdeError::new(ErrorKind::Protocol, message);
let header = |values: Vec<Box<serde_json::value::RawValue>>| {
values
.iter()
.map(|v| serde_json::from_str::<String>(v.get()))
.collect::<Result<Vec<_>, _>>()
.map_err(|e| protocol(format!("invalid ClickHouse result header: {e}")))
};
let Some(names) = &self.names else {
self.names = Some(header(values)?);
return Ok(());
};
if self.types.is_none() {
self.types = Some(header(values)?);
return Ok(());
}
if values.len() != names.len() {
return Err(protocol("ClickHouse result row width does not match its metadata".into()));
}
let row = values
.into_iter()
.map(|value| {
let text = value.get();
if text == "null" {
Ok(None)
} else if text.starts_with('"') {
serde_json::from_str::<String>(text).map(Some).map_err(|e| protocol(e.to_string()))
} else {
Ok(Some(text.to_owned()))
}
})
.collect::<IdeResult<_>>()?;
self.rows.push(row);
Ok(())
}
fn finish(mut self) -> IdeResult<DecodedResult> {
let rest = std::mem::take(&mut self.pending);
self.line(&rest)?;
if let Some(exception) = self.exception {
return Err(IdeError::invalid(format!("ClickHouse: {}", exception_text(&exception))));
}
let Some(names) = self.names else { return Ok((Vec::new(), Vec::new(), false)) };
let types = self.types.unwrap_or_default();
if types.len() != names.len() {
return Err(IdeError::new(ErrorKind::Protocol, "ClickHouse result types do not match its columns"));
}
let columns = names.into_iter().zip(types).map(|(name, type_name)| ResultColumn { name, type_name }).collect();
Ok((columns, self.rows, self.more))
}
}
/// The server's message in a failed response: rows already streamed and the
/// `__exception__` block markers ClickHouse wraps a mid-stream error in are dropped.
fn exception_text(body: &str) -> &str {
let Some(at) = body.find("Code:") else { return body.trim() };
let text = &body[at..];
match text.find("\n__exception__") {
// The line before the closing marker is `<length> <tag>`.
Some(end) => text[..end].trim_end().rsplit_once('\n').map_or(text[..end].trim(), |(message, _)| message.trim()),
None => text.trim(),
}
}
fn http_error(error: reqwest::Error) -> IdeError {
use std::error::Error;
let kind = if error.is_timeout() { ErrorKind::TimedOut } else { ErrorKind::Io };
let error = error.without_url();
let mut message = error.to_string();
let mut source = error.source();
while let Some(error) = source {
let detail = error.to_string();
if !message.contains(&detail) {
message.push_str(": ");
message.push_str(&detail);
}
source = error.source();
}
IdeError::new(kind, format!("ClickHouse HTTP: {message}"))
}
fn read_pem(path: &str, label: &str) -> IdeResult<Vec<u8>> {
std::fs::read(ide_ssh::expand(path))
.map_err(|error| IdeError::new(ErrorKind::Io, format!("cannot read ClickHouse {label}: {error}")))
}
fn reserved_setting(name: &str) -> bool {
let name = name.to_ascii_lowercase();
name.starts_with("output_format_json_")
|| matches!(
name.as_str(),
"user"
| "password"
| "query"
| "database"
| "session_id"
| "session_check"
| "session_timeout"
| "query_id"
| "default_format"
| "max_result_rows"
| "result_overflow_mode"
| "readonly"
)
}
fn transaction_statement(sql: &str) -> bool {
let mut sql = sql.trim_start();
loop {
if let Some(rest) = sql.strip_prefix("--").or_else(|| sql.strip_prefix('#')) {
sql = rest.split_once('\n').map_or("", |(_, rest)| rest).trim_start();
} else if let Some(rest) = sql.strip_prefix("/*") {
sql = rest.split_once("*/").map_or("", |(_, rest)| rest).trim_start();
} else {
break;
}
}
let mut words = sql.split(|c: char| !c.is_ascii_alphanumeric() && c != '_').filter(|s| !s.is_empty());
let first = words.next().unwrap_or_default();
["BEGIN", "COMMIT", "ROLLBACK", "SAVEPOINT", "RELEASE"].iter().any(|s| first.eq_ignore_ascii_case(s))
|| (first.eq_ignore_ascii_case("START") && words.next().is_some_and(|s| s.eq_ignore_ascii_case("TRANSACTION")))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::conn::Pool;
use crate::source::{DbDriver, DbOptions, DbProperty};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
const ONE: &str = "[\"1\"]\n[\"UInt8\"]\n[1]\n";
async fn read_request(stream: &mut tokio::net::TcpStream) -> String {
let mut bytes = Vec::new();
let mut buffer = [0; 4096];
loop {
let count = tokio::time::timeout(Duration::from_secs(5), stream.read(&mut buffer)).await.unwrap().unwrap();
assert!(count > 0, "request ended early");
bytes.extend_from_slice(&buffer[..count]);
if let Some(end) = bytes.windows(4).position(|s| s == b"\r\n\r\n") {
let header = String::from_utf8_lossy(&bytes[..end]);
let length: usize = header
.lines()
.find_map(|line| {
line.to_ascii_lowercase().strip_prefix("content-length:").map(|v| v.trim().parse().unwrap())
})
.unwrap_or(0);
if bytes.len() >= end + 4 + length {
return String::from_utf8(bytes).unwrap();
}
}
}
}
async fn reply(stream: &mut tokio::net::TcpStream, status: &str, headers: &str, body: &str) {
let response =
format!("HTTP/1.1 {status}\r\nContent-Length: {}\r\nConnection: close\r\n{headers}\r\n{body}", body.len());
stream.write_all(response.as_bytes()).await.unwrap();
}
async fn server(
responses: Vec<(&'static str, &'static str, &'static str)>,
) -> (u16, tokio::task::JoinHandle<Vec<String>>) {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let port = listener.local_addr().unwrap().port();
let task = tokio::spawn(async move {
let mut requests = Vec::new();
for (status, headers, body) in responses {
let (mut stream, _) =
tokio::time::timeout(Duration::from_secs(5), listener.accept()).await.unwrap().unwrap();
requests.push(read_request(&mut stream).await);
reply(&mut stream, status, headers, body).await;
}
requests
});
(port, task)
}
fn source(port: u16) -> DataSource {
DataSource { driver: DbDriver::Clickhouse, host: "127.0.0.1".into(), port: Some(port), ..Default::default() }
}
fn decode(body: &[u8], max_rows: usize, chunk: usize) -> IdeResult<DecodedResult> {
decode_from(body, 0, max_rows, chunk)
}
fn decode_from(body: &[u8], skip: usize, max_rows: usize, chunk: usize) -> IdeResult<DecodedResult> {
let mut reader = RowReader::new(skip, max_rows);
for part in body.chunks(chunk) {
reader.push(part)?;
}
reader.finish()
}
#[test]
fn rows_preserve_precision_nulls_and_strings_across_chunks() {
let body = concat!(
r#"["n", "d", "v", "v", "v"]"#,
"\n",
r#"["UInt64", "Decimal(38, 9)", "Nullable(String)", "Nullable(String)", "Nullable(String)"]"#,
"\n",
r#"[18446744073709551615, 12345678901234567890.123456789, null, "null", "ᴺᵁᴸᴸ"]"#,
);
for chunk in [1, 7, body.len()] {
let (columns, rows, more) = decode(body.as_bytes(), 1, chunk).unwrap();
assert_eq!(columns.len(), 5);
assert_eq!(columns[1].type_name, "Decimal(38, 9)");
assert_eq!(
rows[0],
vec![
Some("18446744073709551615".into()),
Some("12345678901234567890.123456789".into()),
None,
Some("null".into()),
Some("ᴺᵁᴸᴸ".into()),
]
);
assert!(!more);
}
let (columns, rows, more) = decode(b"[\"x\"]\n[\"String\"]\n", 0, 64).unwrap();
assert_eq!(columns[0].name, "x");
assert!(rows.is_empty() && !more);
assert_eq!(decode(b"", 5, 64).unwrap().0.len(), 0);
assert!(decode(b"[\"x\"]\n[\"UInt8\"]\n[1, 2]\n", 1, 64).is_err());
}
#[test]
fn a_later_page_skips_rows_without_keeping_them() {
let body = b"[\"n\"]\n[\"UInt8\"]\n[1]\n[2]\n[3]\n[4]\n[5]\n";
let page = |skip, max| {
let (_, rows, more) = decode_from(body, skip, max, 3).unwrap();
(rows.into_iter().map(|r| r[0].clone().unwrap()).collect::<Vec<_>>(), more)
};
assert_eq!(page(2, 2), (vec!["3".to_owned(), "4".to_owned()], true));
assert_eq!(page(4, 2), (vec!["5".to_owned()], false));
assert_eq!(page(9, 2), (Vec::<String>::new(), false));
}
#[test]
fn rows_past_the_limit_are_skipped_but_a_later_exception_still_fails() {
let mut body = String::from("[\"n\"]\n[\"UInt64\"]\n");
for n in 0..1000 {
body.push_str(&format!("[{n}]\n"));
}
let (_, rows, more) = decode(body.as_bytes(), 2, 4096).unwrap();
assert_eq!(rows, vec![vec![Some("0".into())], vec![Some("1".into())]]);
assert!(more);
body.push_str("\n__exception__\nabcdefgh\nCode: 395. DB::Exception: Value passed to 'throwIf' is non-zero\n");
body.push_str("62 abcdefgh\n__exception__\n");
let error = decode(body.as_bytes(), 2, 4096).unwrap_err();
assert_eq!(error.message, "ClickHouse: Code: 395. DB::Exception: Value passed to 'throwIf' is non-zero");
let multiline = "Code: 62. DB::Exception: Syntax error\nat line 2";
assert_eq!(exception_text(multiline), multiline);
}
#[tokio::test]
async fn http_requests_use_auth_settings_limits_and_a_shared_session() {
let (port, requests) = server(vec![
("200 OK", "X-ClickHouse-Format: JSONCompactEachRowWithNamesAndTypes\r\n", ONE),
("200 OK", "", ""),
(
"200 OK",
"X-ClickHouse-Format: JSONCompactEachRowWithNamesAndTypes\r\n",
"[\"n\"]\n[\"UInt64\"]\n[\"1\"]\n[\"2\"]\n[\"3\"]\n",
),
])
.await;
let source = DataSource {
user: "reader".into(),
read_only: true,
options: DbOptions { default_schema: "analytics".into(), time_zone: "UTC".into(), ..Default::default() },
properties: vec![DbProperty { name: "max_threads".into(), value: "2".into() }],
..source(port)
};
let pool = Pool::connect(&source, Some("mock-password"), None).await.unwrap();
pool.execute("USE other", 0, &CancellationToken::new()).await.unwrap();
let result = pool.execute("SELECT number FROM numbers(3)", 1, &CancellationToken::new()).await.unwrap();
assert_eq!(result.rows, vec![vec![Some("1".into())]]);
assert!(result.has_more && result.affected.is_none());
let requests = requests.await.unwrap();
let target = |request: &String| {
Url::parse(&format!(
"http://localhost{}",
request.lines().next().unwrap().split_whitespace().nth(1).unwrap()
))
.unwrap()
};
let urls: Vec<_> = requests.iter().map(target).collect();
let value = |url: &Url, key: &str| url.query_pairs().find(|(k, _)| k == key).map(|(_, v)| v.into_owned());
assert_eq!(value(&urls[0], "database").as_deref(), Some("analytics"));
assert_eq!(value(&urls[1], "database"), None);
assert_eq!(value(&urls[1], "session_check").as_deref(), Some("1"));
assert_eq!(value(&urls[0], "session_id"), value(&urls[2], "session_id"));
assert_ne!(value(&urls[0], "query_id"), value(&urls[2], "query_id"));
assert_eq!(value(&urls[2], "max_result_rows").as_deref(), Some("2"));
for (request, url) in requests.iter().zip(&urls) {
assert!(request.starts_with("POST /?"));
assert!(request.contains("Basic cmVhZGVyOm1vY2stcGFzc3dvcmQ="));
assert!(!url.as_str().contains("reader") && !url.as_str().contains("mock-password"));
assert_eq!(value(url, "readonly").as_deref(), Some("1"));
assert_eq!(value(url, "session_timezone").as_deref(), Some("UTC"));
assert_eq!(value(url, "max_threads").as_deref(), Some("2"));
}
assert!(requests[0].ends_with("SELECT 1"));
assert!(pool.transaction(&[]).await.is_err());
let cancel = CancellationToken::new();
cancel.cancel();
assert_eq!(pool.execute("SELECT 1", 1, &cancel).await.unwrap_err().kind, ErrorKind::Cancelled);
pool.close().await;
assert!(pool.is_closed());
assert!(pool.execute("SELECT 1", 1, &CancellationToken::new()).await.is_err());
}
#[tokio::test]
async fn sequential_queries_reuse_the_http_connection() {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let port = listener.local_addr().unwrap().port();
let task = tokio::spawn(async move {
let (mut stream, _) = listener.accept().await.unwrap();
for _ in 0..2 {
read_request(&mut stream).await;
let response = format!("HTTP/1.1 200 OK\r\nContent-Length: {}\r\n\r\n{ONE}", ONE.len());
stream.write_all(response.as_bytes()).await.unwrap();
}
});
let pool = Pool::connect(&source(port), None, None).await.unwrap();
assert_eq!(
pool.execute("SELECT 1", 1, &CancellationToken::new()).await.unwrap().rows,
vec![vec![Some("1".into())]]
);
task.await.unwrap();
}
#[tokio::test]
async fn server_errors_keep_details_and_hide_passwords() {
let (port, requests) = server(vec![(
"403 Forbidden",
"X-ClickHouse-Exception-Code: 516\r\n",
"Code: 516. DB::Exception: bad mock-password",
)])
.await;
let error = match Pool::connect(&source(port), Some("mock-password"), None).await {
Ok(_) => panic!("expected login failure"),
Err(error) => error,
};
assert_eq!(error.kind, ErrorKind::PermissionDenied);
assert!(error.message.contains("516") && error.message.contains("[redacted]"));
assert!(!error.message.contains("mock-password"));
requests.await.unwrap();
let (port, requests) = server(vec![("200 OK", "", "Code: 241. DB::Exception: memory limit")]).await;
let error = match Pool::connect(&source(port), None, None).await {
Ok(_) => panic!("expected late query failure"),
Err(error) => error,
};
assert!(error.message.contains("memory limit"));
requests.await.unwrap();
let (port, requests) = server(vec![(
"500 Internal Server Error",
"X-ClickHouse-Exception-Code: 395\r\n",
"[\"1\"]\n[\"UInt8\"]\n[1]\n\n__exception__\ntag\nCode: 395. DB::Exception: thrown\n32 tag\n__exception__\n",
)])
.await;
let error = match Pool::connect(&source(port), None, None).await {
Ok(_) => panic!("expected mid-stream failure"),
Err(error) => error,
};
assert_eq!(error.message, "ClickHouse HTTP 500: Code: 395. DB::Exception: thrown");
requests.await.unwrap();
}
#[tokio::test]
async fn cancellation_sends_kill_outside_the_busy_session() {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let port = listener.local_addr().unwrap().port();
let (started, received) = tokio::sync::oneshot::channel();
let server = tokio::spawn(async move {
let (mut health, _) = listener.accept().await.unwrap();
read_request(&mut health).await;
reply(&mut health, "200 OK", "", ONE).await;
drop(health);
let (mut query, _) = listener.accept().await.unwrap();
let request = read_request(&mut query).await;
started.send(()).unwrap();
let (mut kill, _) = tokio::time::timeout(Duration::from_secs(5), listener.accept()).await.unwrap().unwrap();
let kill_request = read_request(&mut kill).await;
reply(&mut kill, "200 OK", "", "").await;
(request, kill_request)
});
let pool = Pool::connect(&source(port), None, None).await.unwrap();
let cancel = CancellationToken::new();
let query = pool.execute("SELECT sleep(30)", 1, &cancel);
let cancelled = async {
received.await.unwrap();
cancel.cancel();
};
let (result, ()) = tokio::join!(query, cancelled);
assert_eq!(result.unwrap_err().kind, ErrorKind::Cancelled);
let (query, kill) = server.await.unwrap();
let target = query.lines().next().unwrap().split_whitespace().nth(1).unwrap();
let url = Url::parse(&format!("http://localhost{target}")).unwrap();
let query_id = url.query_pairs().find(|(k, _)| k == "query_id").unwrap().1.into_owned();
assert!(kill.ends_with(&format!("KILL QUERY WHERE query_id = '{query_id}' SYNC")));
assert!(!kill.contains("session_id="));
}
#[tokio::test]
async fn unsupported_settings_and_transactions_fail_before_connecting() {
for statement in ["BEGIN", "-- comment\n COMMIT", "/* comment */ ROLLBACK", "START TRANSACTION", "SAVEPOINT s"]
{
assert!(transaction_statement(statement));
}
assert!(!transaction_statement("SELECT 'BEGIN'"));
let sources = [
DataSource { options: DbOptions { transaction: DbTxMode::Manual, ..Default::default() }, ..source(1) },
DataSource {
properties: vec![DbProperty { name: "statement_cache_capacity".into(), value: "4".into() }],
..source(1)
},
DataSource { properties: vec![DbProperty { name: "password".into(), value: "x".into() }], ..source(1) },
DataSource { ssl_ca_file: "unused.pem".into(), ..source(1) },
DataSource { ssl: DbSslMode::VerifyCa, ..source(1) },
DataSource { ssl: DbSslMode::Require, ssl_cert_file: "unused.pem".into(), ..source(1) },
];
for source in sources {
let error = match Pool::connect(&source, None, None).await {
Ok(_) => panic!("unsupported settings were accepted"),
Err(error) => error,
};
assert_eq!(error.kind, ErrorKind::InvalidInput);
}
}
#[tokio::test]
#[ignore = "requires CLICKHOUSE_TEST_HOST, CLICKHOUSE_TEST_USER, and CLICKHOUSE_TEST_PASSWORD"]
async fn clickhouse_cloud_transport() {
let source = DataSource {
driver: DbDriver::Clickhouse,
host: std::env::var("CLICKHOUSE_TEST_HOST").expect("CLICKHOUSE_TEST_HOST"),
port: Some(std::env::var("CLICKHOUSE_TEST_PORT").ok().and_then(|v| v.parse().ok()).unwrap_or(8443)),
user: std::env::var("CLICKHOUSE_TEST_USER").expect("CLICKHOUSE_TEST_USER"),
ssl: DbSslMode::Require,
options: DbOptions {
time_zone: "UTC".into(),
startup_script: "SET max_threads = 2".into(),
..Default::default()
},
properties: vec![DbProperty { name: "max_block_size".into(), value: "50".into() }],
..Default::default()
};
let password = std::env::var("CLICKHOUSE_TEST_PASSWORD").expect("CLICKHOUSE_TEST_PASSWORD");
let pool = Pool::connect(&source, Some(&password), None).await.unwrap();
let cancel = CancellationToken::new();
let settings = pool
.execute("SELECT getSetting('max_threads'), getSetting('max_block_size'), timezone()", 1, &cancel)
.await
.unwrap();
assert_eq!(settings.rows, vec![vec![Some("2".into()), Some("50".into()), Some("UTC".into())]]);
let result = pool.execute("SELECT toUInt64('18446744073709551615') AS n, CAST(NULL AS Nullable(String)) AS missing, CAST('ᴺᵁᴸᴸ' AS Nullable(String)) AS text, 'a\\nb' AS lines", 1, &cancel).await.unwrap();
assert_eq!(
result.rows,
vec![vec![Some("18446744073709551615".into()), None, Some("ᴺᵁᴸᴸ".into()), Some("a\nb".into())]]
);
assert_eq!(result.columns[1].type_name, "Nullable(String)");
let limited = pool.execute("SELECT number FROM numbers(100)", 2, &cancel).await.unwrap();
assert_eq!(limited.rows.len(), 2);
assert!(limited.has_more);
let marker = format!("transport_cancel_{}", uuid::Uuid::new_v4().simple());
let sql = format!("SELECT sleepEachRow(0.05) FROM numbers(50) /* {marker} */");
let stop = CancellationToken::new();
let query = pool.execute(&sql, 1, &stop);
let cancel_query = async {
tokio::time::sleep(Duration::from_millis(500)).await;
stop.cancel();
};
let (cancelled, ()) = tokio::join!(query, cancel_query);
let cancelled = cancelled.unwrap_err();
assert_eq!(cancelled.kind, ErrorKind::Cancelled);
assert_eq!(cancelled.message, "cancelled");
let active = pool.execute(&format!("SELECT count() FROM system.processes WHERE position(query, '{marker}') > 0 AND query_id != currentQueryID()"), 1, &cancel).await.unwrap();
assert_eq!(active.rows, vec![vec![Some("0".into())]]);
let table = format!("transport_test_{}", uuid::Uuid::new_v4().simple());
pool.execute(&format!("CREATE TEMPORARY TABLE {table} (n UInt8)"), 0, &cancel).await.unwrap();
let continuity = async {
pool.execute(&format!("INSERT INTO {table} VALUES (7)"), 0, &cancel).await?;
pool.execute(&format!("SELECT n FROM {table}"), 1, &cancel).await
}
.await;
let cleanup = pool.execute(&format!("DROP TEMPORARY TABLE {table}"), 0, &cancel).await;
assert_eq!(continuity.unwrap().rows, vec![vec![Some("7".into())]]);
cleanup.unwrap();
pool.close().await;
assert!(pool.is_closed());
}
}
+84 -11
View File
@@ -106,6 +106,7 @@ pub enum Pool {
Postgres(PgPool),
Mysql(MySqlPool),
SqlServer(Bb8Pool<MssqlConnectionManager>),
Clickhouse(crate::clickhouse::ClickhousePool),
}
/// Runs `$body` with `$p` bound to the concrete pool of each driver.
@@ -115,7 +116,7 @@ macro_rules! with_pool {
Pool::Sqlite($p) => $body,
Pool::Postgres($p) => $body,
Pool::Mysql($p) => $body,
Pool::SqlServer(_) => unreachable!(),
Pool::SqlServer(_) | Pool::Clickhouse(_) => unreachable!(),
}
};
}
@@ -336,10 +337,13 @@ fn result_columns<R: Row>(row: &R) -> Vec<ResultColumn> {
.collect()
}
/// Streams the results of `$sql` on executor `$exec`, keeping at most `$max` rows
/// with their cells read as `$mode`.
/// Streams the results of `$sql` on executor `$exec`, reading past the first `$skip`
/// rows without decoding them, then keeping at most `$max` rows with their cells read as `$mode`.
macro_rules! fetch {
($exec:expr, $sql:expr, $max:expr, $mode:expr) => {{
($exec:expr, $sql:expr, $max:expr, $mode:expr) => {
fetch!($exec, $sql, 0usize, $max, $mode)
};
($exec:expr, $sql:expr, $skip:expr, $max:expr, $mode:expr) => {{
use futures::StreamExt;
let mut stream = sqlx::raw_sql(sqlx::AssertSqlSafe($sql.to_owned())).fetch_many($exec);
let mut columns: Vec<ResultColumn> = Vec::new();
@@ -348,6 +352,7 @@ macro_rules! fetch {
let mut has_more = false;
let mut failure = None;
let mut first: Option<Instant> = None;
let mut to_skip: usize = $skip;
while let Some(item) = stream.next().await {
first.get_or_insert_with(Instant::now);
match item {
@@ -356,6 +361,10 @@ macro_rules! fetch {
if columns.is_empty() {
columns = result_columns(&row);
}
if to_skip > 0 {
to_skip -= 1;
continue;
}
if rows.len() >= $max {
has_more = true;
break;
@@ -385,7 +394,8 @@ async fn mssql_command(conn: &mut MssqlClient, sql: &str) -> IdeResult<()> {
Ok(())
}
async fn mssql_fetch(conn: &mut MssqlClient, sql: &str, max: usize) -> IdeResult<Fetched> {
/// Like [`fetch!`] for SQL Server: the first `skip` rows are read past without decoding.
async fn mssql_fetch(conn: &mut MssqlClient, sql: &str, skip: usize, max: usize) -> IdeResult<Fetched> {
let mut stream = conn.simple_query(sql).await.map_err(mssql_err)?;
let first = Some(Instant::now());
let columns = stream
@@ -398,8 +408,13 @@ async fn mssql_fetch(conn: &mut MssqlClient, sql: &str, max: usize) -> IdeResult
.collect();
let mut rows = Vec::new();
let mut has_more = false;
let mut to_skip = skip;
while let Some(item) = stream.next().await {
if let tiberius::QueryItem::Row(row) = item.map_err(mssql_err)? {
if to_skip > 0 {
to_skip -= 1;
continue;
}
if rows.len() >= max {
has_more = true;
break;
@@ -503,6 +518,9 @@ impl Setup {
value.parse::<u64>().map_err(|_| IdeError::invalid(format!("{what} must be a whole number: {value}")))
};
match (source.driver, name.to_ascii_lowercase().as_str()) {
(DbDriver::Clickhouse, "statement_cache_capacity") => {
return Err(IdeError::invalid("ClickHouse HTTP does not support statement caching"));
}
(_, "statement_cache_capacity") => s.statement_cache = Some(number(&name)? as usize),
(_, "connect_timeout") => s.connect_timeout = Some(Duration::from_secs(number(&name)?)),
(DbDriver::Postgres, "application_name") => s.application_name = Some(value.to_owned()),
@@ -512,6 +530,9 @@ impl Setup {
_ => s.params.push((name, value.to_owned())),
}
}
if source.driver == DbDriver::Clickhouse && !o.time_zone.trim().is_empty() {
s.params.push(("session_timezone".into(), o.time_zone.trim().into()));
}
let script = o.startup_script.trim();
if !script.is_empty() {
s.statements.extend(crate::split::split(script, source.driver).into_iter().map(|st| st.text));
@@ -527,6 +548,7 @@ impl Pool {
Pool::Postgres(_) => DbDriver::Postgres,
Pool::Mysql(_) => DbDriver::Mysql,
Pool::SqlServer(_) => DbDriver::SqlServer,
Pool::Clickhouse(_) => DbDriver::Clickhouse,
}
}
@@ -534,6 +556,9 @@ impl Pool {
pub async fn connect(source: &DataSource, password: Option<&str>, root: Option<&Path>) -> IdeResult<Self> {
const MAX: u32 = 4;
let setup = Setup::of(source)?;
if source.driver == DbDriver::Clickhouse {
return crate::clickhouse::ClickhousePool::connect(source, password, setup).await.map(Pool::Clickhouse);
}
let timeout = setup.connect_timeout.unwrap_or(Duration::from_secs(30));
let init = Arc::new(setup.statements);
// Runs the session statements (startup script…) on every new connection.
@@ -695,12 +720,14 @@ impl Pool {
conn.query("SELECT 1", &[]).await.map_err(mssql_err)?.into_results().await.map_err(mssql_err)?;
Ok(Pool::SqlServer(pool))
}
DbDriver::Clickhouse => unreachable!(),
}
}
pub async fn close(&self) {
match self {
Pool::SqlServer(_) => {}
Pool::Clickhouse(p) => p.close(),
_ => with_pool!(self, p => p.close().await),
}
}
@@ -708,16 +735,20 @@ impl Pool {
pub fn is_closed(&self) -> bool {
match self {
Pool::SqlServer(_) => false,
Pool::Clickhouse(p) => p.is_closed(),
_ => with_pool!(self, p => p.is_closed()),
}
}
/// Runs `sql` (no row limit beyond `max`), for introspection: cells are text (see [`Cells::Text`]).
pub async fn rows(&self, sql: &str, max: usize) -> IdeResult<Vec<Vec<Option<String>>>> {
if let Pool::Clickhouse(p) = self {
return Ok(p.execute(sql, max, &CancellationToken::new()).await?.rows);
}
let fetched: IdeResult<Fetched> = match self {
Pool::SqlServer(p) => {
let mut conn = p.get_owned().await.map_err(mssql_pool_err)?;
mssql_fetch(&mut conn, sql, max).await
mssql_fetch(&mut conn, sql, 0, max).await
}
_ => with_pool!(self, p => fetch!(p, sql, max, Cells::Text)),
};
@@ -727,8 +758,23 @@ impl Pool {
/// Runs one statement and keeps at most `max_rows` rows. Cancelling stops
/// the statement on the server (PostgreSQL, MySQL) and drops the connection.
pub async fn execute(&self, sql: &str, max_rows: usize, cancel: &CancellationToken) -> IdeResult<QueryResult> {
self.execute_from(sql, 0, max_rows, cancel).await
}
/// [`execute`](Self::execute), reading past the first `skip` rows without keeping them
/// (a later page of a result: only that page is decoded and returned).
pub async fn execute_from(
&self,
sql: &str,
skip: usize,
max_rows: usize,
cancel: &CancellationToken,
) -> IdeResult<QueryResult> {
if let Pool::Clickhouse(p) = self {
return p.execute_from(sql, skip, max_rows, cancel).await;
}
let mut conn = Held::acquire(self).await?;
let r = conn.execute(self, sql, max_rows, cancel).await;
let r = conn.execute_from(self, sql, skip, max_rows, cancel).await;
if cancel.is_cancelled() {
conn.discard().await;
return Err(IdeError::cancelled());
@@ -740,6 +786,7 @@ impl Pool {
/// row when flagged (table editor submits).
pub async fn transaction(&self, statements: &[(String, bool)]) -> IdeResult<u64> {
match self {
Pool::Clickhouse(_) => Err(crate::clickhouse::transaction_error()),
Pool::SqlServer(p) => {
let mut conn = p.get_owned().await.map_err(mssql_pool_err)?;
let result = async {
@@ -790,6 +837,7 @@ pub enum Held {
Postgres(PoolConnection<Postgres>),
Mysql(PoolConnection<MySql>),
SqlServer(Box<bb8::PooledConnection<'static, MssqlConnectionManager>>),
Clickhouse(crate::clickhouse::ClickhousePool),
}
impl Held {
@@ -799,6 +847,12 @@ impl Held {
Pool::Postgres(p) => Held::Postgres(p.acquire().await.map_err(db_err)?),
Pool::Mysql(p) => Held::Mysql(p.acquire().await.map_err(db_err)?),
Pool::SqlServer(p) => Held::SqlServer(Box::new(p.get_owned().await.map_err(mssql_pool_err)?)),
Pool::Clickhouse(p) => {
if p.is_closed() {
return Err(IdeError::new(ErrorKind::Io, "ClickHouse connection is closed"));
}
Held::Clickhouse(p.clone())
}
})
}
@@ -806,13 +860,14 @@ impl Held {
pub async fn command(&mut self, sql: &str) -> IdeResult<()> {
match self {
Held::SqlServer(c) => mssql_command(c, sql).await,
Held::Clickhouse(_) => Err(crate::clickhouse::transaction_error()),
other => {
let sql = sqlx::AssertSqlSafe(sql.to_owned());
match other {
Held::Sqlite(c) => sqlx::raw_sql(sql).execute(&mut **c).await.map(drop),
Held::Postgres(c) => sqlx::raw_sql(sql).execute(&mut **c).await.map(drop),
Held::Mysql(c) => sqlx::raw_sql(sql).execute(&mut **c).await.map(drop),
Held::SqlServer(_) => unreachable!(),
Held::SqlServer(_) | Held::Clickhouse(_) => unreachable!(),
}
.map_err(db_err)
}
@@ -833,6 +888,7 @@ impl Held {
Held::SqlServer(mut c) => {
c.0.take();
}
Held::Clickhouse(_) => {}
}
}
@@ -845,6 +901,22 @@ impl Held {
max_rows: usize,
cancel: &CancellationToken,
) -> IdeResult<QueryResult> {
self.execute_from(pool, sql, 0, max_rows, cancel).await
}
/// [`execute`](Self::execute), reading past the first `skip` rows without keeping them
/// (see [`Pool::execute_from`]).
pub async fn execute_from(
&mut self,
pool: &Pool,
sql: &str,
skip: usize,
max_rows: usize,
cancel: &CancellationToken,
) -> IdeResult<QueryResult> {
if let Held::Clickhouse(c) = self {
return c.execute_from(sql, skip, max_rows, cancel).await;
}
let started = Instant::now();
// `$id`: the query naming the server process, `$kill`: how another connection stops it.
macro_rules! run {
@@ -854,7 +926,7 @@ impl Held {
None => None,
};
let r = tokio::select! {
r = async { fetch!(&mut **$conn, sql, max_rows, Cells::Display) } => r,
r = async { fetch!(&mut **$conn, sql, skip, max_rows, Cells::Display) } => r,
() = cancel.cancelled() => Err(IdeError::cancelled()),
};
if cancel.is_cancelled() {
@@ -883,10 +955,11 @@ impl Held {
Held::SqlServer(c) => {
// Tiberius has no attention/cancel API; stop reading and discard the connection.
tokio::select! {
r = mssql_fetch(c, sql, max_rows) => r?,
r = mssql_fetch(c, sql, skip, max_rows) => r?,
() = cancel.cancelled() => return Err(IdeError::cancelled()),
}
}
Held::Clickhouse(_) => unreachable!(),
};
let (columns, rows, affected, has_more, first) = fetched;
let elapsed_ms = started.elapsed().as_secs_f64() * 1000.0;
@@ -907,7 +980,7 @@ pub fn begin_statement(driver: DbDriver) -> &'static str {
match driver {
DbDriver::Mysql => "START TRANSACTION",
DbDriver::SqlServer => "BEGIN TRAN",
DbDriver::Sqlite | DbDriver::Postgres => "BEGIN",
DbDriver::Sqlite | DbDriver::Postgres | DbDriver::Clickhouse => "BEGIN",
}
}
+247 -31
View File
@@ -2,6 +2,7 @@
//! execution ([`conn`]), introspection, table data paging / editing ([`sql`])
//! and statement splitting ([`split`]), for SQLite, PostgreSQL and MySQL/MariaDB.
mod clickhouse;
pub mod conn;
pub mod history;
pub mod plan;
@@ -212,6 +213,8 @@ pub struct DbManager {
/// App-wide SSH connections and host-key trust.
ssh: Arc<SshManager>,
saved_passwords: SecretStore,
/// Directory of the global data sources and their saved passwords ([`DbManager::with_global_sources`]).
global: Option<(PathBuf, SecretStore)>,
history: Arc<HistoryStore>,
/// Records executed statements in the history off the async threads, in order.
recorder: std::sync::mpsc::Sender<(String, String, f64)>,
@@ -338,6 +341,7 @@ impl DbManager {
}
Self {
saved_passwords: SecretStore::new(passwords, &root),
global: None,
history,
recorder,
root,
@@ -350,8 +354,42 @@ impl DbManager {
}
}
/// Also lists the global data sources stored in `dir` (the app data directory).
pub fn with_global_sources(mut self, dir: PathBuf) -> Self {
let passwords = self.saved_passwords.scoped(&dir);
self.global = Some((dir, passwords));
self
}
/// The project's data sources, then the global ones (a project one wins on the same id).
pub fn sources(&self) -> IdeResult<Vec<DataSource>> {
source::load(&self.root)
let mut all = source::load(&self.root)?;
let global: Vec<_> = self.global_sources().into_iter().filter(|g| all.iter().all(|s| s.id != g.id)).collect();
all.extend(global);
Ok(all)
}
fn global_sources(&self) -> Vec<DataSource> {
let Some((dir, _)) = &self.global else { return Vec::new() };
source::load_global(dir).unwrap_or_else(|e| {
tracing::warn!("global data sources: {}", e.message);
Vec::new()
})
}
/// Where the saved passwords of data source `id` live: app-wide for a global one.
fn saved_for(&self, id: &str) -> &SecretStore {
match &self.global {
Some((_, store)) if self.global_sources().iter().any(|s| s.id == id) => store,
_ => &self.saved_passwords,
}
}
fn store_of(&self, source: &DataSource) -> &SecretStore {
match &self.global {
Some((_, store)) if source.global => store,
_ => &self.saved_passwords,
}
}
/// The connection settings of target `target` ([`DataSource::resolved`]), with
@@ -371,18 +409,43 @@ impl DbManager {
}
/// Saves the data source list; connections of removed or changed sources are closed.
/// Global ones (`global`) go to the app-wide list, the others to the project's.
pub async fn save_sources(&self, sources: Vec<DataSource>) -> IdeResult<()> {
let old = self.sources().unwrap_or_default();
source::save(&self.root, &sources)?;
let (global, project): (Vec<_>, Vec<_>) = sources.iter().cloned().partition(|s| s.global);
match &self.global {
// Rewritten only when it changes: other projects' windows edit the same file.
Some((dir, _)) => {
if global != old.iter().filter(|s| s.global).cloned().collect::<Vec<_>>() {
source::save_global(dir, &global)?;
}
}
None if !global.is_empty() => return Err(IdeError::invalid("global data sources are not available here")),
None => {}
}
source::save(&self.root, &project)?;
for o in old {
match sources.iter().find(|s| s.id == o.id) {
Some(n) if *n == o => {}
gone_or_changed => {
if gone_or_changed.is_none() {
for k in [o.id.clone(), secrets::ssh_key(&o.id)] {
self.session_passwords.lock().remove(&k);
self.saved_passwords.delete(&k);
let keys = [o.id.clone(), secrets::ssh_key(&o.id)];
match gone_or_changed {
None => {
for k in &keys {
self.session_passwords.lock().remove(k);
self.store_of(&o).delete(k);
}
}
// Made global or project-level: its saved passwords move along.
Some(n) if n.global != o.global => {
for k in &keys {
if let Some(p) = self.store_of(&o).get(k) {
self.store_of(n).set(k, &p)?;
self.store_of(&o).delete(k);
}
}
}
Some(_) => {}
}
self.disconnect_source(&o.id).await;
}
@@ -394,7 +457,7 @@ impl DbManager {
/// Remembers `password` for the session, and in the saved passwords when
/// the data source saves passwords (`None` forgets it).
pub async fn set_password(&self, id: &str, password: Option<String>, save: bool) -> IdeResult<()> {
self.remember(id.to_owned(), password, save).await
self.remember(id, id.to_owned(), password, save).await
}
/// [`DbManager::set_password`] for the SSH password / key passphrase.
@@ -406,10 +469,10 @@ impl DbManager {
self.ssh.set_secret(&c, secret, save);
return Ok(());
}
self.remember(secrets::ssh_key(id), secret, save).await
self.remember(id, secrets::ssh_key(id), secret, save).await
}
async fn remember(&self, key: String, secret: Option<String>, save: bool) -> IdeResult<()> {
async fn remember(&self, id: &str, key: String, secret: Option<String>, save: bool) -> IdeResult<()> {
match &secret {
Some(p) => {
self.session_passwords.lock().insert(key.clone(), p.clone());
@@ -418,10 +481,11 @@ impl DbManager {
self.session_passwords.lock().remove(&key);
}
}
let store = self.saved_for(id);
let result = match (secret, save) {
(Some(p), true) => self.saved_passwords.set(&key, &p),
(Some(p), true) => store.set(&key, &p),
_ => {
self.saved_passwords.delete(&key);
store.delete(&key);
Ok(())
}
};
@@ -432,18 +496,18 @@ impl DbManager {
Ok(())
}
async fn secret(&self, key: String, saved: bool) -> Option<String> {
async fn secret(&self, source: &DataSource, key: String, saved: bool) -> Option<String> {
if let Some(p) = self.session_passwords.lock().get(&key) {
return Some(p.clone());
}
if !saved {
return None;
}
self.saved_passwords.get(&key)
self.store_of(source).get(&key)
}
async fn password(&self, source: &DataSource) -> Option<String> {
match self.secret(source.id.clone(), source.driver != DbDriver::Sqlite && source.save_password).await {
match self.secret(source, source.id.clone(), source.driver != DbDriver::Sqlite && source.save_password).await {
Some(p) => Some(p),
None => source.url_password(),
}
@@ -499,7 +563,7 @@ impl DbManager {
let session = if config.is_empty() {
let secret = match ssh_secret {
Some(s) => Some(s),
None => self.secret(secrets::ssh_key(&source.id), ssh.save_secret).await,
None => self.secret(source, secrets::ssh_key(&source.id), ssh.save_secret).await,
};
let target = SshTarget {
host: ssh.host.clone(),
@@ -547,12 +611,14 @@ impl DbManager {
})?;
let version = match source.driver {
DbDriver::Sqlite => "SELECT 'SQLite ' || sqlite_version()",
DbDriver::Postgres => "SELECT version()",
DbDriver::Postgres | DbDriver::Clickhouse => "SELECT version()",
DbDriver::Mysql => "SELECT CONCAT(@@version_comment, ' ', VERSION())",
DbDriver::SqlServer => "SELECT @@VERSION",
};
let v = pool.rows(version, 1).await.ok().and_then(|r| r.into_iter().next()?.into_iter().next()?);
let result = pool.rows(version, 1).await;
pool.close().await;
let rows = if source.driver == DbDriver::Clickhouse { result? } else { result.unwrap_or_default() };
let v = rows.into_iter().next().and_then(|r| r.into_iter().next().flatten());
Ok(v.unwrap_or_else(|| "Connected".into()))
}
@@ -701,8 +767,11 @@ impl DbManager {
_ => indexes.push(IndexInfo { name, unique: flag(&unique), columns: vec![col] }),
}
}
let fk_rows =
pool.rows(&sql::table_foreign_keys(driver, schema, table), MAX_META_ROWS).await.unwrap_or_default();
let fk_rows = if driver == DbDriver::Clickhouse {
Vec::new()
} else {
pool.rows(&sql::table_foreign_keys(driver, schema, table), MAX_META_ROWS).await.unwrap_or_default()
};
let mut foreign_keys = group_foreign_keys(fk_rows, &[]);
// SQLite keys that reference the primary key implicitly: look it up.
for k in foreign_keys.iter_mut().filter(|k| k.ref_columns.is_empty()) {
@@ -715,13 +784,15 @@ impl DbManager {
.collect();
}
foreign_keys.retain(|k| k.columns.len() == k.ref_columns.len());
let triggers = pool
.rows(&sql::triggers(driver, schema, table), MAX_META_ROWS)
.await
.unwrap_or_else(|e| {
let trigger_rows = if driver == DbDriver::Clickhouse {
Vec::new()
} else {
pool.rows(&sql::triggers(driver, schema, table), MAX_META_ROWS).await.unwrap_or_else(|e| {
tracing::warn!("triggers: {}", e.message);
Vec::new()
})
};
let triggers = trigger_rows
.into_iter()
.filter_map(|r| {
let mut it = r.into_iter();
@@ -822,6 +893,11 @@ impl DbManager {
let ddl = cells(pool.rows(&q, 1).await?, 1).into_iter().next();
ddl.map(|d| format!("{d};\n")).ok_or_else(missing)
}
DbDriver::Clickhouse => {
let q = format!("SHOW CREATE TABLE {}", sql::qualified(driver, schema, name));
let ddl = cells(pool.rows(&q, 1).await?, 0).into_iter().next();
ddl.map(|d| format!("{};\n", d.trim_end().trim_end_matches(';'))).ok_or_else(missing)
}
DbDriver::SqlServer => {
let obj = format!("{}.{}", schema, name).replace('\'', "''");
let ddl = cells(pool.rows(&format!("SELECT OBJECT_DEFINITION(OBJECT_ID('{}'))", obj), 1).await?, 0)
@@ -902,10 +978,14 @@ impl DbManager {
}
}
// Foreign keys are a nicety (JOIN completion): a server that refuses the catalog query still completes.
let fk_rows = pool.rows(sql::foreign_keys(driver), MAX_META_ROWS).await.unwrap_or_else(|e| {
tracing::warn!("foreign keys: {}", e.message);
let fk_rows = if driver == DbDriver::Clickhouse {
Vec::new()
});
} else {
pool.rows(sql::foreign_keys(driver), MAX_META_ROWS).await.unwrap_or_else(|e| {
tracing::warn!("foreign keys: {}", e.message);
Vec::new()
})
};
let foreign_keys = group_foreign_keys(fk_rows, &tables);
let default_schema = match sql::current_schema(driver) {
None => "main".to_owned(),
@@ -929,7 +1009,20 @@ impl DbManager {
max_rows: usize,
cancel: &CancellationToken,
) -> IdeResult<QueryResult> {
let r = self.run(id, statement, max_rows, cancel).await;
self.execute_from(id, statement, 0, max_rows, cancel).await
}
/// [`execute`](Self::execute), reading past the first `skip` rows without keeping
/// them: a later page of a result, where only that page is decoded and returned.
pub async fn execute_from(
&self,
id: &str,
statement: &str,
skip: usize,
max_rows: usize,
cancel: &CancellationToken,
) -> IdeResult<QueryResult> {
let r = self.run(id, statement, skip, max_rows, cancel).await;
let ran = match &r {
Ok(_) => true,
Err(e) => !matches!(e.kind, ErrorKind::Cancelled | ErrorKind::PermissionDenied),
@@ -964,7 +1057,7 @@ impl DbManager {
/// The execution plan of `statement` (explained, not run).
pub async fn explain(&self, id: &str, statement: &str, cancel: &CancellationToken) -> IdeResult<Vec<PlanNode>> {
let driver = self.pool(id).await?.driver();
let result = self.run(id, &plan::explain_statement(driver, statement), MAX_META_ROWS, cancel).await?;
let result = self.run(id, &plan::explain_statement(driver, statement), 0, MAX_META_ROWS, cancel).await?;
plan::parse(driver, &result)
}
@@ -972,14 +1065,18 @@ impl DbManager {
&self,
id: &str,
statement: &str,
skip: usize,
max_rows: usize,
cancel: &CancellationToken,
) -> IdeResult<QueryResult> {
let pool = self.pool(id).await?;
let session = self.sessions.lock().get(id).cloned();
let Some(session) = session else {
return pool.execute(statement, max_rows, cancel).await;
return pool.execute_from(statement, skip, max_rows, cancel).await;
};
if pool.driver() == DbDriver::Clickhouse {
return Err(IdeError::invalid("ClickHouse manual transactions are unsupported"));
}
let mut held = session.lock().await;
if held.is_none() {
let mut conn = Held::acquire(&pool).await?;
@@ -987,7 +1084,7 @@ impl DbManager {
*held = Some(conn);
}
let conn = held.as_mut().expect("opened above");
let r = conn.execute(&pool, statement, max_rows, cancel).await;
let r = conn.execute_from(&pool, statement, skip, max_rows, cancel).await;
if cancel.is_cancelled() {
// The server rolls the transaction back with the connection.
held.take().expect("opened above").discard().await;
@@ -998,8 +1095,11 @@ impl DbManager {
/// Switches `id` between auto-commit and manual commit. Leaving manual
/// mode rolls back what was not committed.
pub async fn set_manual_commit(&self, id: &str, manual: bool) {
pub async fn set_manual_commit(&self, id: &str, manual: bool) -> IdeResult<()> {
if manual {
if self.source(id)?.driver == DbDriver::Clickhouse {
return Err(IdeError::invalid("ClickHouse manual transactions are unsupported"));
}
self.sessions.lock().entry(id.to_owned()).or_default();
} else {
let session = self.sessions.lock().remove(id);
@@ -1007,10 +1107,14 @@ impl DbManager {
end_session(&s).await;
}
}
Ok(())
}
/// Commits (or rolls back) the open transaction of a manual-commit data source.
pub async fn finish_transaction(&self, id: &str, commit: bool) -> IdeResult<()> {
if self.source(id).is_ok_and(|s| s.driver == DbDriver::Clickhouse) {
return Err(IdeError::invalid("ClickHouse manual transactions are unsupported"));
}
let session = self.sessions.lock().get(id).cloned();
let Some(session) = session else { return Ok(()) };
let mut held = session.lock().await;
@@ -1052,6 +1156,9 @@ impl DbManager {
/// Applies the table editor's pending changes in one transaction.
pub async fn apply_edits(&self, id: &str, schema: &str, table: &str, edits: &[DbRowEdit]) -> IdeResult<u64> {
if self.source(id)?.driver == DbDriver::Clickhouse {
return Err(IdeError::invalid("ClickHouse table editing is unsupported"));
}
let pool = self.pool(id).await?;
let driver = pool.driver();
let statements = edits
@@ -1064,3 +1171,112 @@ impl DbManager {
pool.transaction(&statements).await
}
}
#[cfg(test)]
mod tests {
use super::*;
use ide_ssh::secrets::testing::files_containing;
use std::path::Path;
fn manager(dir: &Path) -> DbManager {
ide_ssh::secrets::testing::use_memory_store();
let data = dir.join("data");
let ssh = Arc::new(SshManager::new(data.join("ssh-passwords.json")));
DbManager::new(dir.join("project"), data.join("db-passwords.json"), data.join("db-history.json"), ssh)
}
#[tokio::test]
async fn saved_passwords_stay_off_disk() {
let tmp = tempfile::tempdir().unwrap();
manager(tmp.path()).set_password("s1", Some("hunter2-db".into()), true).await.unwrap();
manager(tmp.path()).set_ssh_secret("s1", Some("hunter2-tunnel".into()), true).await.unwrap();
assert_eq!(files_containing(tmp.path(), "hunter2-db"), Vec::<PathBuf>::new());
assert_eq!(files_containing(tmp.path(), "hunter2-tunnel"), Vec::<PathBuf>::new());
let reopened = manager(tmp.path());
assert_eq!(reopened.secret(&DataSource::default(), "s1".into(), true).await.as_deref(), Some("hunter2-db"));
assert_eq!(
reopened.secret(&DataSource::default(), secrets::ssh_key("s1"), true).await.as_deref(),
Some("hunter2-tunnel")
);
}
#[tokio::test]
async fn moves_plaintext_passwords_of_older_versions_off_disk() {
let tmp = tempfile::tempdir().unwrap();
let project = tmp.path().join("project").to_string_lossy().into_owned();
let legacy = serde_json::json!({ project: { "s1": "old-db-pw" }, "/other": { "s2": "other-pw" } });
std::fs::create_dir_all(tmp.path().join("data")).unwrap();
std::fs::write(tmp.path().join("data/db-passwords.json"), legacy.to_string()).unwrap();
let m = manager(tmp.path());
assert_eq!(m.secret(&DataSource::default(), "s1".into(), true).await.as_deref(), Some("old-db-pw"));
assert_eq!(files_containing(tmp.path(), "old-db-pw"), Vec::<PathBuf>::new());
assert_eq!(files_containing(tmp.path(), "other-pw"), Vec::<PathBuf>::new());
let other = SecretStore::new(tmp.path().join("data/db-passwords.json"), Path::new("/other"));
assert_eq!(other.get("s2").as_deref(), Some("other-pw"));
}
#[tokio::test]
async fn global_data_sources_are_shared_by_projects_with_their_passwords() {
ide_ssh::secrets::testing::use_memory_store();
let tmp = tempfile::tempdir().unwrap();
let (app, one, two) = (tmp.path().join("app"), tmp.path().join("one"), tmp.path().join("two"));
let manager = |root: &std::path::Path| {
let ssh = Arc::new(SshManager::new(app.join("ssh-passwords.json")));
DbManager::new(root.to_path_buf(), app.join("passwords.json"), app.join("history.json"), ssh)
.with_global_sources(app.clone())
};
let (a, b) = (manager(&one), manager(&two));
let source = |id: &str, global: bool| DataSource {
id: id.into(),
name: id.into(),
driver: DbDriver::Postgres,
save_password: true,
global,
..Default::default()
};
a.save_sources(vec![source("local", false), source("shared", false)]).await.unwrap();
a.set_password("shared", Some("pw".into()), true).await.unwrap();
// Made global: listed in the other project too, with its saved password.
a.save_sources(vec![source("local", false), source("shared", true)]).await.unwrap();
let ids = |m: &DbManager| m.sources().unwrap().into_iter().map(|s| (s.id, s.global)).collect::<Vec<_>>();
assert_eq!(ids(&a), [("local".into(), false), ("shared".into(), true)]);
assert_eq!(ids(&b), [("shared".into(), true)]);
assert_eq!(b.password(&b.source("shared").unwrap()).await.as_deref(), Some("pw"));
let project_file = std::fs::read_to_string(one.join(".openidea/data-sources.toml")).unwrap();
assert!(!project_file.contains("shared") && !project_file.contains("global"), "{project_file}");
assert!(!std::fs::read_to_string(app.join("data-sources.toml")).unwrap().contains("global"));
// Made project-level again: gone from the other project, the password stays with it.
a.save_sources(vec![source("local", false), source("shared", false)]).await.unwrap();
assert!(b.sources().unwrap().is_empty());
assert_eq!(a.password(&a.source("shared").unwrap()).await.as_deref(), Some("pw"));
}
#[tokio::test]
async fn clickhouse_rejects_manual_transactions_and_table_edits_without_connecting() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path().to_path_buf();
let ssh = Arc::new(SshManager::new(root.join("ssh-passwords.json")));
let db = DbManager::new(root.clone(), root.join("passwords.json"), root.join("history.json"), ssh);
db.save_sources(vec![DataSource {
id: "clickhouse".into(),
name: "clickhouse".into(),
driver: DbDriver::Clickhouse,
host: "127.0.0.1".into(),
port: Some(1),
..Default::default()
}])
.await
.unwrap();
let error = db.set_manual_commit("clickhouse", true).await.unwrap_err();
assert!(error.message.contains("manual transactions are unsupported"));
assert!(db.sessions.lock().is_empty());
for commit in [false, true] {
let error = db.finish_transaction("clickhouse", commit).await.unwrap_err();
assert!(error.message.contains("manual transactions are unsupported"));
}
let error = db.apply_edits("clickhouse", "default", "events", &[]).await.unwrap_err();
assert!(error.message.contains("table editing is unsupported"));
assert!(!db.is_connected("clickhouse"));
db.set_manual_commit("clickhouse", false).await.unwrap();
}
}
+18 -1
View File
@@ -35,6 +35,7 @@ pub fn explain_statement(driver: DbDriver, statement: &str) -> String {
match driver {
DbDriver::Sqlite => format!("EXPLAIN QUERY PLAN {s}"),
DbDriver::Postgres => format!("EXPLAIN (FORMAT JSON) {s}"),
DbDriver::Clickhouse => format!("EXPLAIN json = 1 {s}"),
DbDriver::Mysql => format!("EXPLAIN {s}"),
DbDriver::SqlServer => format!("SET SHOWPLAN_TEXT ON; {s}"),
}
@@ -44,7 +45,7 @@ pub fn explain_statement(driver: DbDriver, statement: &str) -> String {
pub fn parse(driver: DbDriver, result: &QueryResult) -> IdeResult<Vec<PlanNode>> {
match driver {
DbDriver::Sqlite => Ok(sqlite(&result.rows)),
DbDriver::Postgres => {
DbDriver::Postgres | DbDriver::Clickhouse => {
let text: String =
result.rows.iter().filter_map(|r| r.first().cloned().flatten()).collect::<Vec<_>>().join("\n");
let json: Value = serde_json::from_str(&text).map_err(|e| IdeError::invalid(format!("plan: {e}")))?;
@@ -199,6 +200,7 @@ mod tests {
assert_eq!(explain_statement(DbDriver::Sqlite, "select 1;"), "EXPLAIN QUERY PLAN select 1");
assert_eq!(explain_statement(DbDriver::Postgres, " select 1 "), "EXPLAIN (FORMAT JSON) select 1");
assert_eq!(explain_statement(DbDriver::Mysql, "select 1"), "EXPLAIN select 1");
assert_eq!(explain_statement(DbDriver::Clickhouse, " select 1;"), "EXPLAIN json = 1 select 1");
assert_eq!(explain_statement(DbDriver::SqlServer, "select 1"), "SET SHOWPLAN_TEXT ON; select 1");
}
@@ -254,6 +256,21 @@ mod tests {
assert_eq!((plan[3].parent, plan[3].operation.as_str()), (Some(2), "Seq Scan"));
}
#[test]
fn reads_clickhouse_json_plans_and_rejects_invalid_json() {
let json = r#"[{"Plan": {"Node Type": "Expression", "Node Id": "Expression_5",
"Description": "Project names", "Plans": [
{"Node Type": "ReadFromMergeTree", "Node Id": "ReadFromMergeTree_0",
"Description": "default.events"}
]}}]"#;
let plan = parse(DbDriver::Clickhouse, &result(&["explain"], vec![vec![Some(json)]])).unwrap();
assert_eq!(plan.len(), 2);
assert_eq!((plan[0].parent, plan[0].operation.as_str()), (None, "Expression"));
assert_eq!((plan[1].parent, plan[1].operation.as_str()), (Some(0), "ReadFromMergeTree"));
assert!(plan[1].properties.iter().any(|p| p.name == "Description" && p.value == "default.events"));
assert!(parse(DbDriver::Clickhouse, &result(&["explain"], vec![vec![Some("invalid")]])).is_err());
}
#[test]
fn reads_mysql_rows() {
let r = result(
+31 -55
View File
@@ -1,20 +1,15 @@
//! Saved data source passwords, in the app's data directory rather than the OS
//! credential store (no system prompts). One JSON file for all projects, keyed
//! by project root, then credential key; owner-only on Unix.
//! Saved data source passwords, in the OS credential store like the other saved
//! passwords ([`SecretFile`]). One index file for all projects; each credential is
//! keyed by project root, then credential key.
use ide_core::{IdeError, IdeResult};
use parking_lot::Mutex;
use ide_core::IdeResult;
use ide_ssh::secrets::SecretFile;
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
/// Project root → credential key → secret.
type Contents = BTreeMap<String, BTreeMap<String, String>>;
/// Serialises read-modify-write across the projects sharing the file.
static LOCK: Mutex<()> = Mutex::new(());
pub struct SecretStore {
path: PathBuf,
file: SecretFile,
project: String,
}
@@ -23,61 +18,41 @@ pub fn ssh_key(id: &str) -> String {
format!("{id}#ssh")
}
fn file_key(project: &str, key: &str) -> String {
format!("{project}|{key}")
}
/// Older versions' file: project root → credential key → secret.
fn legacy(value: serde_json::Value) -> Option<BTreeMap<String, String>> {
let all: BTreeMap<String, BTreeMap<String, String>> = serde_json::from_value(value).ok()?;
Some(
all.into_iter()
.flat_map(|(project, keys)| keys.into_iter().map(move |(k, s)| (file_key(&project, &k), s)))
.collect(),
)
}
impl SecretStore {
pub fn new(path: PathBuf, root: &Path) -> Self {
Self { path, project: root.to_string_lossy().into_owned() }
let file = SecretFile::with_legacy(path.clone(), legacy);
Self { path, file, project: root.to_string_lossy().into_owned() }
}
/// The same file, keyed by `scope` instead (global data sources: the app data directory).
pub fn scoped(&self, scope: &Path) -> Self {
Self::new(self.path.clone(), scope)
}
pub fn get(&self, key: &str) -> Option<String> {
let _g = LOCK.lock();
self.read().remove(&self.project)?.remove(key)
self.file.get(&file_key(&self.project, key))
}
pub fn set(&self, key: &str, secret: &str) -> IdeResult<()> {
let _g = LOCK.lock();
let mut all = self.read();
all.entry(self.project.clone()).or_default().insert(key.to_owned(), secret.to_owned());
self.write(&all)
self.file.set(&file_key(&self.project, key), secret)
}
pub fn delete(&self, key: &str) {
let _g = LOCK.lock();
let mut all = self.read();
let Some(project) = all.get_mut(&self.project) else { return };
if project.remove(key).is_none() {
return;
}
if project.is_empty() {
all.remove(&self.project);
}
if let Err(e) = self.write(&all) {
tracing::warn!("saved passwords: {}", e.message);
}
}
fn read(&self) -> Contents {
match std::fs::read(&self.path) {
Ok(bytes) => serde_json::from_slice(&bytes).unwrap_or_else(|e| {
tracing::warn!("saved passwords: {}: {e}", self.path.display());
Contents::new()
}),
Err(_) => Contents::new(),
}
}
fn write(&self, all: &Contents) -> IdeResult<()> {
let err = |e: std::io::Error| IdeError::unavailable(format!("saved passwords: {e}"));
if let Some(dir) = self.path.parent() {
std::fs::create_dir_all(dir).map_err(err)?;
}
let tmp = self.path.with_extension("json.tmp");
let mut options = std::fs::OpenOptions::new();
options.write(true).create(true).truncate(true);
#[cfg(unix)]
std::os::unix::fs::OpenOptionsExt::mode(&mut options, 0o600);
let bytes = serde_json::to_vec_pretty(all).map_err(|e| IdeError::internal(e.to_string()))?;
std::io::Write::write_all(&mut options.open(&tmp).map_err(err)?, &bytes).map_err(err)?;
std::fs::rename(&tmp, &self.path).map_err(err)
self.file.delete(&file_key(&self.project, key))
}
}
@@ -87,6 +62,7 @@ mod tests {
#[test]
fn keeps_projects_apart_and_drops_empty_ones() {
ide_ssh::secrets::testing::use_memory_store();
let tmp = tempfile::tempdir().unwrap();
let file = tmp.path().join("db-passwords.json");
let a = SecretStore::new(file.clone(), Path::new("/p/a"));
+82 -11
View File
@@ -1,6 +1,7 @@
//! Data sources: per-project connection settings in `.openidea/data-sources.toml`.
//! Passwords never go there: saved ones live in [`crate::secrets`], the others
//! only in memory for the session.
//! Data sources: per-project connection settings in `.openidea/data-sources.toml`, and
//! global ones (IDEA's Make Global: listed in every project) in `data-sources.toml` of the
//! app data directory. Passwords never go there: saved ones live in [`crate::secrets`],
//! the others only in memory for the session.
use ide_core::{IdeError, IdeResult, IpcPath, project_dir};
use serde::{Deserialize, Serialize};
@@ -18,6 +19,7 @@ pub enum DbDriver {
Mysql,
/// Microsoft SQL Server.
SqlServer,
Clickhouse,
}
impl DbDriver {
@@ -27,6 +29,7 @@ impl DbDriver {
DbDriver::Postgres => Some(5432),
DbDriver::Mysql => Some(3306),
DbDriver::SqlServer => Some(1433),
DbDriver::Clickhouse => Some(8123),
}
}
}
@@ -179,6 +182,9 @@ pub struct DataSource {
pub options: DbOptions,
pub introspection: DbIntrospection,
pub properties: Vec<DbProperty>,
/// Stored app-wide and listed in every project. Implied by the file it is read from,
/// so not written to it.
pub global: bool,
}
impl Default for DataSource {
@@ -207,6 +213,7 @@ impl Default for DataSource {
options: DbOptions::default(),
introspection: DbIntrospection::default(),
properties: Vec::new(),
global: false,
}
}
}
@@ -253,6 +260,7 @@ pub fn scheme_driver(scheme: &str) -> Option<DbDriver> {
"mysql" | "mariadb" | "tidb" | "singlestore" | "memsql" | "oceanbase" => Some(DbDriver::Mysql),
"sqlserver" | "mssql" | "mssqlserver" => Some(DbDriver::SqlServer),
"sqlite" | "sqlite3" => Some(DbDriver::Sqlite),
"clickhouse" | "http" | "https" => Some(DbDriver::Clickhouse),
_ => None,
}
}
@@ -260,7 +268,7 @@ pub fn scheme_driver(scheme: &str) -> Option<DbDriver> {
/// Parses `scheme://[user[:password]@]host[:port][/database][?k=v&…]`, `jdbc:` forms
/// included, and `sqlite:path`.
pub fn parse_url(raw: &str) -> IdeResult<ParsedUrl> {
let bad = || IdeError::invalid(format!("not a database URL: {raw}"));
let bad = || IdeError::invalid("invalid database URL");
let url = raw.trim();
let url = url.strip_prefix("jdbc:").or_else(|| url.strip_prefix("JDBC:")).unwrap_or(url);
let (scheme, rest) = url.split_once(':').ok_or_else(bad)?;
@@ -367,6 +375,15 @@ impl DataSource {
}
properties.append(&mut s.properties);
s.properties = properties;
if s.driver == DbDriver::Clickhouse {
let raw_url = self.url.trim().to_ascii_lowercase();
let url = raw_url.strip_prefix("jdbc:").unwrap_or(&raw_url);
if url.starts_with("https://") {
s.ssl = DbSslMode::VerifyFull;
} else if url.starts_with("http://") {
s.ssl = DbSslMode::Disable;
}
}
Ok(s)
}
@@ -385,9 +402,32 @@ struct SourcesFile {
sources: Vec<DataSource>,
}
/// The project's data sources.
pub fn load(root: &Path) -> IdeResult<Vec<DataSource>> {
let path = project_dir::path(root, FILE);
let text = match std::fs::read_to_string(&path) {
read(&project_dir::path(root, FILE))
}
/// Saves the project's data sources.
pub fn save(root: &Path, sources: &[DataSource]) -> IdeResult<()> {
write(&project_dir::path(root, FILE), sources)
}
/// The global data sources stored in `dir` (the app data directory).
pub fn load_global(dir: &Path) -> IdeResult<Vec<DataSource>> {
let mut sources = read(&dir.join(FILE))?;
for s in &mut sources {
s.global = true;
}
Ok(sources)
}
/// Saves the global data sources in `dir`.
pub fn save_global(dir: &Path, sources: &[DataSource]) -> IdeResult<()> {
write(&dir.join(FILE), sources)
}
fn read(path: &Path) -> IdeResult<Vec<DataSource>> {
let text = match std::fs::read_to_string(path) {
Ok(t) => t,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(Vec::new()),
Err(e) => return Err(e.into()),
@@ -396,19 +436,23 @@ pub fn load(root: &Path) -> IdeResult<Vec<DataSource>> {
Ok(file.sources)
}
pub fn save(root: &Path, sources: &[DataSource]) -> IdeResult<()> {
fn write(path: &Path, sources: &[DataSource]) -> IdeResult<()> {
for s in sources {
if s.id.is_empty() || s.name.trim().is_empty() {
return Err(IdeError::invalid("every data source needs an id and a name"));
}
}
let path = project_dir::path(root, FILE);
if let Some(dir) = path.parent() {
std::fs::create_dir_all(dir)?;
}
let text = toml::to_string_pretty(&SourcesFile { sources: sources.to_vec() })
.map_err(|e| IdeError::internal(e.to_string()))?;
std::fs::write(path, text)?;
let internal = |e: &dyn std::fmt::Display| IdeError::internal(e.to_string());
let mut value = toml::Value::try_from(SourcesFile { sources: sources.to_vec() }).map_err(|e| internal(&e))?;
if let Some(list) = value.get_mut("data-source").and_then(toml::Value::as_array_mut) {
for table in list.iter_mut().filter_map(toml::Value::as_table_mut) {
table.remove("global");
}
}
std::fs::write(path, toml::to_string_pretty(&value).map_err(|e| internal(&e))?)?;
Ok(())
}
@@ -519,4 +563,31 @@ mod tests {
assert!(DataSource { url: "mysql://h/db".into(), ..s.clone() }.resolved().is_err());
assert_eq!(DataSource { url_only: false, ..s }.resolved().unwrap().host, "localhost");
}
#[test]
fn clickhouse_urls_resolve_http_https_and_credentials() {
assert_eq!(DbDriver::Clickhouse.default_port(), Some(8123));
let source = DataSource {
driver: DbDriver::Clickhouse,
url_only: true,
url: "https://reader:s%[email protected]:8443/analytics?max_execution_time=30".into(),
..Default::default()
};
let resolved = source.resolved().unwrap();
assert_eq!(resolved.host, "db.example");
assert_eq!(resolved.port, Some(8443));
assert_eq!(resolved.database, "analytics");
assert_eq!(resolved.user, "reader");
assert_eq!(resolved.ssl, DbSslMode::VerifyFull);
assert_eq!(source.url_password().as_deref(), Some("s@cret"));
assert_eq!(resolved.properties[0].name, "max_execution_time");
let http = DataSource { url: "http://localhost/default".into(), ..source.clone() }.resolved().unwrap();
assert_eq!(http.ssl, DbSslMode::Disable);
let jdbc = DataSource { url: "jdbc:clickhouse://localhost/analytics".into(), ..source.clone() };
assert_eq!(jdbc.resolved().unwrap().database, "analytics");
let secure = DataSource { url: "https://db.example/default?sslmode=disable".into(), ..source };
assert_eq!(secure.resolved().unwrap().ssl, DbSslMode::VerifyFull);
let error = parse_url("clickhouse://reader:[email protected]:invalid/default").unwrap_err();
assert!(!error.message.contains("secret"));
}
}
+15 -4
View File
@@ -28,6 +28,7 @@ enum State {
/// Every statement of `text` that contains more than whitespace and comments.
pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
let mysql = driver == DbDriver::Mysql;
let clickhouse = driver == DbDriver::Clickhouse;
let chars: Vec<(usize, char)> = text.char_indices().collect();
// UTF-16 offset of each char index (plus the end).
let mut u16_at = Vec::with_capacity(chars.len() + 1);
@@ -81,7 +82,7 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
state = State::Double;
has_code = true;
}
'`' if mysql => {
'`' if mysql || clickhouse => {
state = State::Backtick;
has_code = true;
}
@@ -117,7 +118,7 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
_ => {}
},
State::Single => {
if mysql && c == '\\' {
if (mysql || clickhouse) && c == '\\' {
i += 1;
} else if c == '\'' {
if next == Some('\'') {
@@ -128,7 +129,9 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
}
}
State::Double => {
if c == '"' {
if clickhouse && c == '\\' {
i += 1;
} else if c == '"' {
if next == Some('"') {
i += 1;
} else {
@@ -137,7 +140,9 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
}
}
State::Backtick => {
if c == '`' {
if clickhouse && (c == '\\' || (c == '`' && next == Some('`'))) {
i += 1;
} else if c == '`' {
state = State::Code;
}
}
@@ -224,6 +229,12 @@ mod tests {
assert_eq!(texts(sql, DbDriver::Mysql), vec!["select `a;b`, 'it\\'s;' from t # x;y", "select 2"]);
}
#[test]
fn clickhouse_backticks_and_escaped_quotes_keep_statements_intact() {
let query = r#"SELECT `a\`;b`, "c\";d", 'it\'s;safe'; SELECT 2"#;
assert_eq!(texts(query, DbDriver::Clickhouse), vec![r#"SELECT `a\`;b`, "c\";d", 'it\'s;safe'"#, "SELECT 2"]);
}
#[test]
fn postgres_dollar_quotes() {
let sql =
+106 -2
View File
@@ -7,6 +7,7 @@ use specta::Type;
pub fn quote_ident(driver: DbDriver, name: &str) -> String {
match driver {
DbDriver::Clickhouse => format!("`{}`", name.replace('\\', "\\\\").replace('`', "\\`")),
DbDriver::Mysql => format!("`{}`", name.replace('`', "``")),
DbDriver::SqlServer => format!("[{}]", name.replace(']', "]]")),
_ => format!("\"{}\"", name.replace('"', "\"\"")),
@@ -15,7 +16,7 @@ pub fn quote_ident(driver: DbDriver, name: &str) -> String {
pub fn quote_literal(driver: DbDriver, value: &str) -> String {
let mut s = value.replace('\'', "''");
if driver == DbDriver::Mysql {
if matches!(driver, DbDriver::Mysql | DbDriver::Clickhouse) {
s = s.replace('\\', "\\\\");
}
format!("'{s}'")
@@ -48,6 +49,10 @@ pub const INTERNAL_DATABASES: &[&str] = &["rdsadmin", "azure_maintenance", "azur
pub fn schemas(driver: DbDriver, system: bool) -> &'static str {
match driver {
DbDriver::Sqlite => "SELECT name FROM pragma_database_list WHERE name <> 'temp' ORDER BY seq",
DbDriver::Clickhouse if system => "SELECT name FROM system.databases ORDER BY name",
DbDriver::Clickhouse => {
"SELECT name FROM system.databases WHERE lower(name) NOT IN ('system', 'information_schema') ORDER BY name"
}
DbDriver::Postgres if system => {
"SELECT schema_name FROM information_schema.schemata \
WHERE schema_name NOT LIKE 'pg\\_toast%' AND schema_name NOT LIKE 'pg\\_temp%' ORDER BY 1"
@@ -73,6 +78,12 @@ pub fn schemas(driver: DbDriver, system: bool) -> &'static str {
/// Rows: `name, kind` where kind is `table`, `view` or `materialized view`.
pub fn tables(driver: DbDriver, schema: &str) -> String {
match driver {
DbDriver::Clickhouse => format!(
"SELECT name, CASE engine WHEN 'View' THEN 'view' WHEN 'MaterializedView' THEN 'materialized view' \
WHEN 'WindowView' THEN 'view' ELSE 'table' END FROM system.tables \
WHERE database = {} AND is_temporary = 0 ORDER BY name",
quote_literal(driver, schema)
),
DbDriver::Sqlite => format!(
"SELECT name, type FROM {}.sqlite_master WHERE type IN ('table', 'view') \
AND name NOT LIKE 'sqlite\\_%' ESCAPE '\\' ORDER BY name",
@@ -101,6 +112,12 @@ pub fn tables(driver: DbDriver, schema: &str) -> String {
pub fn columns(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
match driver {
DbDriver::Clickhouse => format!(
"SELECT name, type, CASE WHEN startsWith(type, 'Nullable(') \
OR startsWith(type, 'LowCardinality(Nullable(') THEN 'YES' ELSE 'NO' END, \
if(default_kind = '', NULL, default_expression), is_in_primary_key \
FROM system.columns WHERE database = {s} AND table = {t} ORDER BY position"
),
DbDriver::Sqlite => format!(
"SELECT name, type, CASE WHEN \"notnull\" = 0 THEN 'YES' ELSE 'NO' END, dflt_value, \
CASE WHEN pk > 0 THEN 1 ELSE 0 END FROM pragma_table_info({t}, {s}) ORDER BY cid"
@@ -138,6 +155,12 @@ pub fn columns(driver: DbDriver, schema: &str, table: &str) -> String {
pub fn indexes(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
match driver {
DbDriver::Clickhouse => format!(
"SELECT * FROM (SELECT 'PRIMARY KEY' AS index_name, 0 AS is_unique, primary_key AS expression \
FROM system.tables WHERE database = {s} AND name = {t} AND primary_key != '' \
UNION ALL SELECT name, 0, expr FROM system.data_skipping_indices \
WHERE database = {s} AND table = {t}) ORDER BY index_name"
),
DbDriver::Sqlite => format!(
"SELECT il.name, il.\"unique\", ii.name FROM pragma_index_list({t}, {s}) il \
JOIN pragma_index_info(il.name, {s}) ii ORDER BY il.name, ii.seqno"
@@ -169,6 +192,15 @@ pub fn indexes(driver: DbDriver, schema: &str, table: &str) -> String {
/// for SQL completion, grouped by table in column order.
pub fn completion_columns(driver: DbDriver) -> &'static str {
match driver {
DbDriver::Clickhouse => {
"SELECT c.database, c.table, CASE t.engine WHEN 'View' THEN 'view' \
WHEN 'MaterializedView' THEN 'materialized view' WHEN 'WindowView' THEN 'view' ELSE 'table' END, \
c.name, c.type, CASE WHEN startsWith(c.type, 'Nullable(') \
OR startsWith(c.type, 'LowCardinality(Nullable(') THEN 'YES' ELSE 'NO' END, c.is_in_primary_key, 1 \
FROM system.columns c JOIN system.tables t ON t.database = c.database AND t.name = c.table \
WHERE lower(c.database) NOT IN ('system', 'information_schema') AND t.is_temporary = 0 \
ORDER BY c.database, c.table, c.position LIMIT 50000"
}
DbDriver::Sqlite => {
"SELECT 'main', m.name, m.type, p.name, p.type, CASE WHEN p.\"notnull\" = 0 THEN 'YES' ELSE 'NO' END, \
CASE WHEN p.pk > 0 THEN 1 ELSE 0 END, \
@@ -221,6 +253,7 @@ pub fn completion_columns(driver: DbDriver) -> &'static str {
/// when the key references the primary key implicitly.
pub fn foreign_keys(driver: DbDriver) -> &'static str {
match driver {
DbDriver::Clickhouse => "SELECT '', '', '', '', '', '', '' WHERE 0",
DbDriver::Sqlite => {
"SELECT 'main', m.name, f.id, f.\"from\", 'main', f.\"table\", f.\"to\" \
FROM sqlite_master m JOIN pragma_foreign_key_list(m.name) f \
@@ -258,6 +291,7 @@ pub fn foreign_keys(driver: DbDriver) -> &'static str {
pub fn table_foreign_keys(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
match driver {
DbDriver::Clickhouse => foreign_keys(driver).to_owned(),
DbDriver::Sqlite => format!(
"SELECT {s}, {t}, f.id, f.\"from\", {s}, f.\"table\", f.\"to\" FROM pragma_foreign_key_list({t}, {s}) f ORDER BY f.id, f.seq"
),
@@ -276,6 +310,7 @@ pub fn table_foreign_keys(driver: DbDriver, schema: &str, table: &str) -> String
/// The schema unqualified names resolve to (`None`: always `main`).
pub fn current_schema(driver: DbDriver) -> Option<&'static str> {
match driver {
DbDriver::Clickhouse => Some("SELECT currentDatabase()"),
DbDriver::Sqlite => None,
DbDriver::Postgres => Some("SELECT current_schema()"),
DbDriver::Mysql => Some("SELECT DATABASE()"),
@@ -362,6 +397,9 @@ fn where_key(driver: DbDriver, key: &[CellValue]) -> String {
/// The DML statement for `edit`. Values are written as quoted literals, which
/// every supported database coerces to the column type.
pub fn edit_statement(driver: DbDriver, schema: &str, table: &str, edit: &DbRowEdit) -> Result<String, String> {
if driver == DbDriver::Clickhouse {
return Err("ClickHouse table editing is unsupported: primary keys do not uniquely identify rows and edits require transactions".into());
}
let target = qualified(driver, schema, table);
match edit {
DbRowEdit::Update { key, values } => {
@@ -450,6 +488,7 @@ pub fn postgres_ddl_indexes(oid: u32) -> String {
pub fn triggers(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
match driver {
DbDriver::Clickhouse => "SELECT '', '', '' WHERE 0".to_owned(),
DbDriver::Sqlite => format!(
"SELECT name, sql FROM {}.sqlite_master WHERE type = 'trigger' AND tbl_name = {t} ORDER BY name",
quote_ident(driver, schema)
@@ -492,7 +531,7 @@ pub fn sqlite_trigger_event(sql: &str) -> (String, String) {
pub fn routines(driver: DbDriver, schema: &str) -> Option<String> {
let s = quote_literal(driver, schema);
match driver {
DbDriver::Sqlite => None,
DbDriver::Sqlite | DbDriver::Clickhouse => None,
// Functions of extensions are left out, as in IDEA.
DbDriver::Postgres => Some(format!(
"SELECT p.proname, CASE WHEN p.prokind = 'p' THEN 'procedure' ELSE 'function' END, \
@@ -625,6 +664,71 @@ mod tests {
assert_eq!(quote_literal(DbDriver::Mysql, "it's \\"), "'it''s \\\\'");
}
#[test]
fn clickhouse_escapes_backslashes_and_delimiters() {
let driver = DbDriver::Clickhouse;
assert_eq!(quote_literal(driver, r"path\O'Brien"), r"'path\\O''Brien'");
assert_eq!(quote_literal(driver, r"\' OR 1 = 1 --"), r"'\\'' OR 1 = 1 --'");
assert_eq!(quote_ident(driver, r"a\`b"), r"`a\\\`b`");
assert_eq!(qualified(driver, "data base", "event`log"), r"`data base`.`event\`log`");
}
#[test]
fn clickhouse_metadata_preserves_types_defaults_and_nonunique_keys() {
let driver = DbDriver::Clickhouse;
assert_eq!(schemas(driver, true), "SELECT name FROM system.databases ORDER BY name");
assert!(schemas(driver, false).contains("lower(name) NOT IN ('system', 'information_schema')"));
assert_eq!(current_schema(driver), Some("SELECT currentDatabase()"));
let relations = tables(driver, "event's\\db");
assert!(relations.contains("WHEN 'MaterializedView' THEN 'materialized view'"));
assert!(relations.contains(r"WHERE database = 'event''s\\db' AND is_temporary = 0"));
let fields = columns(driver, "default", "events");
assert!(fields.starts_with("SELECT name, type, CASE WHEN startsWith(type, 'Nullable(')"));
assert!(fields.contains("startsWith(type, 'LowCardinality(Nullable(')"));
assert!(fields.contains("if(default_kind = '', NULL, default_expression), is_in_primary_key"));
assert!(fields.ends_with("WHERE database = 'default' AND table = 'events' ORDER BY position"));
let keys = indexes(driver, "default", "events");
assert!(keys.contains("'PRIMARY KEY' AS index_name, 0 AS is_unique, primary_key AS expression"));
assert!(keys.contains("SELECT name, 0, expr FROM system.data_skipping_indices"));
let completion = completion_columns(driver);
assert!(completion.contains("c.is_in_primary_key, 1"));
assert!(completion.contains("ORDER BY c.database, c.table, c.position LIMIT 50000"));
}
#[test]
fn clickhouse_has_no_foreign_key_trigger_or_schema_routine_catalogs() {
let driver = DbDriver::Clickhouse;
assert_eq!(foreign_keys(driver), "SELECT '', '', '', '', '', '', '' WHERE 0");
assert_eq!(table_foreign_keys(driver, "default", "events"), foreign_keys(driver));
assert_eq!(triggers(driver, "default", "events"), "SELECT '', '', '' WHERE 0");
assert!(routines(driver, "default").is_none());
assert!(sequences(driver, "default").is_none());
assert!(object_ddl(driver, "default", crate::DbObjectKind::Trigger, "t").is_none());
}
#[test]
fn clickhouse_pages_quote_keys_and_fetch_one_extra_row() {
let sort = [SortKey { column: "event`id".into(), descending: true }];
assert_eq!(
table_page(DbDriver::Clickhouse, "default", "events", Some("WHERE id > 3"), &sort, 50, 25),
r"SELECT * FROM `default`.`events` WHERE id > 3 ORDER BY `event\`id` DESC LIMIT 26 OFFSET 50"
);
}
#[test]
fn clickhouse_keys_never_enable_transactional_table_edits() {
let key = vec![cv("id", Some("1"))];
for edit in [
DbRowEdit::Update { key: key.clone(), values: vec![cv("name", Some("new"))] },
DbRowEdit::Delete { key },
DbRowEdit::Insert { values: vec![cv("name", Some("new"))] },
DbRowEdit::Insert { values: vec![] },
] {
let error = edit_statement(DbDriver::Clickhouse, "default", "events", &edit).unwrap_err();
assert!(error.contains("primary keys do not uniquely identify rows"));
}
}
#[test]
fn pages() {
let sort = [SortKey { column: "id".into(), descending: true }];
+136
View File
@@ -0,0 +1,136 @@
use ide_core::{CancellationToken, IdeError, IdeResult};
use ide_db::{CellValue, DataSource, DbDriver, DbManager, DbRowEdit, DbSslMode, SortKey, sql};
use std::sync::Arc;
use std::time::{SystemTime, UNIX_EPOCH};
fn check(condition: bool, message: &str) -> IdeResult<()> {
if condition { Ok(()) } else { Err(IdeError::invalid(message)) }
}
#[tokio::test]
#[ignore = "requires CLICKHOUSE_TEST_HOST, CLICKHOUSE_TEST_USER, and CLICKHOUSE_TEST_PASSWORD"]
async fn clickhouse_cloud_manager_catalogs_and_session_table() -> IdeResult<()> {
let tmp = tempfile::tempdir()?;
let root = tmp.path().to_path_buf();
let ssh = Arc::new(ide_ssh::SshManager::new(root.join("ssh-passwords.json")));
let db = DbManager::new(root.clone(), root.join("passwords.json"), root.join("history.json"), ssh);
let mut source = DataSource {
id: "clickhouse".into(),
name: "clickhouse".into(),
driver: DbDriver::Clickhouse,
host: std::env::var("CLICKHOUSE_TEST_HOST").expect("set CLICKHOUSE_TEST_HOST"),
port: Some(
std::env::var("CLICKHOUSE_TEST_PORT")
.map(|value| value.parse().expect("numeric CLICKHOUSE_TEST_PORT"))
.unwrap_or(8443),
),
user: std::env::var("CLICKHOUSE_TEST_USER").expect("set CLICKHOUSE_TEST_USER"),
database: "default".into(),
ssl: DbSslMode::VerifyFull,
..Default::default()
};
source.introspection.show_system = true;
let password = std::env::var("CLICKHOUSE_TEST_PASSWORD").expect("set CLICKHOUSE_TEST_PASSWORD");
check(!db.test(&source, Some(password.clone()), None).await?.is_empty(), "missing server version")?;
db.save_sources(vec![source]).await?;
db.set_password("clickhouse", Some(password), false).await?;
let cancel = CancellationToken::new();
let suffix = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_nanos();
let name = format!("manager_test_{}_{suffix}", std::process::id());
let table = sql::quote_ident(DbDriver::Clickhouse, &name);
db.execute(
"clickhouse",
&format!(
"CREATE TEMPORARY TABLE {table} (id UInt64, label Nullable(String) DEFAULT NULL, \
category LowCardinality(Nullable(String)) DEFAULT NULL, items Array(Nullable(UInt8)), \
label_size UInt64 ALIAS length(ifNull(label, ''))) ENGINE = Memory"
),
0,
&cancel,
)
.await?;
let checked: IdeResult<()> = async {
db.execute(
"clickhouse",
&format!("INSERT INTO {table} (id, label, items) VALUES (1, 'first', [1, NULL]), (1, NULL, [])"),
0,
&cancel,
)
.await?;
let schemas = db.schemas("clickhouse").await?;
check(schemas.iter().any(|schema| schema == "default"), "default database missing")?;
check(schemas.iter().any(|schema| schema == "system"), "system database missing")?;
let tables = db.tables("clickhouse", "system").await?;
check(tables.iter().any(|table| table.name == "one"), "system table listing missing one")?;
let details = db.table_details("clickhouse", "", &name).await?;
check(details.columns.len() == 5, "temporary table columns missing")?;
for (name, data_type, nullable) in [
("id", "UInt64", false),
("label", "Nullable(String)", true),
("category", "LowCardinality(Nullable(String))", true),
("items", "Array(Nullable(UInt8))", false),
] {
check(
details
.columns
.iter()
.any(|column| column.name == name && column.data_type == data_type && column.nullable == nullable),
"column type or nullability mismatch",
)?;
}
check(
details.columns.iter().any(|column| column.name == "label" && column.default.as_deref() == Some("NULL")),
"explicit default expression missing",
)?;
check(details.indexes.iter().all(|index| !index.unique), "ClickHouse index reported unique")?;
check(details.foreign_keys.is_empty() && details.triggers.is_empty(), "unsupported table objects returned")?;
let sort = [SortKey { column: "id".into(), descending: false }];
let first = db.table_page("clickhouse", "", &name, None, &sort, 0, 1, &cancel).await?;
check(first.result.rows.len() == 1 && first.result.has_more, "first page limit failed")?;
let second = db.table_page("clickhouse", "", &name, None, &sort, 1, 1, &cancel).await?;
check(second.result.rows.len() == 1 && !second.result.has_more, "page offset failed")?;
let ddl = db.ddl("clickhouse", "", &name).await?;
check(ddl.contains(&name) && ddl.contains("Memory"), "temporary table DDL missing")?;
let completion = db.completion("clickhouse").await?;
check(completion.default_schema == "default", "completion default database mismatch")?;
check(completion.foreign_keys.is_empty(), "completion returned unsupported foreign keys")?;
if let Some(table) = completion.tables.first() {
let details = db.table_details("clickhouse", &table.schema, &table.name).await?;
check(
table.columns.iter().all(|column| {
details.columns.iter().any(|field| field.name == column.name && field.data_type == column.data_type)
}),
"completion columns differ from catalog",
)?;
}
let plan = db.explain("clickhouse", &format!("SELECT id FROM {table}"), &cancel).await?;
check(!plan.is_empty(), "query plan missing")?;
let value = r"path\O'Brien";
let result = db
.execute("clickhouse", &format!("SELECT {}", sql::quote_literal(DbDriver::Clickhouse, value)), 1, &cancel)
.await?;
check(result.rows == vec![vec![Some(value.to_owned())]], "quoted literal changed value")?;
let edits = [DbRowEdit::Update {
key: vec![CellValue { column: "id".into(), value: Some("1".into()) }],
values: vec![CellValue { column: "label".into(), value: Some("changed".into()) }],
}];
check(
db.apply_edits("clickhouse", "", &name, &edits)
.await
.err()
.is_some_and(|error| error.message.contains("table editing is unsupported")),
"supplied row key authorized table editing",
)?;
check(db.set_manual_commit("clickhouse", true).await.is_err(), "manual transactions were accepted")?;
for commit in [false, true] {
check(db.finish_transaction("clickhouse", commit).await.is_err(), "transaction control was accepted")?;
}
Ok(())
}
.await;
let cleanup = db.execute("clickhouse", &format!("DROP TEMPORARY TABLE {table}"), 0, &cancel).await;
db.disconnect("clickhouse").await;
checked?;
cleanup?;
Ok(())
}
+28 -4
View File
@@ -1,7 +1,9 @@
//! Introspection, queries and table edits against a real SQLite file.
use ide_core::{CancellationToken, IdeResult};
use ide_db::{CellValue, DataSource, DbDriver, DbManager, DbOptions, DbProperty, DbRowEdit, DbTableKind, SortKey};
use ide_db::{
CellValue, DataSource, DbDriver, DbManager, DbOptions, DbProperty, DbRowEdit, DbTableKind, QueryResult, SortKey,
};
async fn setup() -> (tempfile::TempDir, DbManager) {
let tmp = tempfile::tempdir().unwrap();
@@ -123,6 +125,28 @@ async fn queries_with_limits_nulls_and_blobs() -> IdeResult<()> {
Ok(())
}
#[tokio::test]
async fn a_later_page_returns_only_its_rows() -> IdeResult<()> {
let (_tmp, db) = setup().await;
let c = CancellationToken::new();
let sql = "SELECT id FROM users ORDER BY id";
let ids = |r: &QueryResult| r.rows.iter().map(|row| row[0].clone().unwrap()).collect::<Vec<_>>();
let second = db.execute_from("s1", sql, 1, 1, &c).await?;
assert_eq!(ids(&second), vec!["2"]);
assert!(second.has_more);
assert_eq!(second.columns.len(), 1);
let last = db.execute_from("s1", sql, 2, 5, &c).await?;
assert_eq!(ids(&last), vec!["3"]);
assert!(!last.has_more);
let past = db.execute_from("s1", sql, 9, 5, &c).await?;
assert!(past.rows.is_empty() && !past.has_more);
assert_eq!(past.columns.len(), 1, "columns of a page past the end come from the prepared statement");
Ok(())
}
#[tokio::test]
async fn pages_sorts_filters_and_edits_in_a_transaction() -> IdeResult<()> {
let (_tmp, db) = setup().await;
@@ -197,7 +221,7 @@ async fn count(db: &DbManager) -> String {
async fn manual_commit_keeps_a_transaction_until_commit_or_rollback() -> IdeResult<()> {
let (_tmp, db) = setup().await;
let c = CancellationToken::new();
db.set_manual_commit("s1", true).await;
db.set_manual_commit("s1", true).await.unwrap();
db.execute("s1", "INSERT INTO users (name) VALUES ('dee')", 1, &c).await?;
// The session sees its own change.
assert_eq!(count(&db).await, "4");
@@ -208,10 +232,10 @@ async fn manual_commit_keeps_a_transaction_until_commit_or_rollback() -> IdeResu
db.finish_transaction("s1", true).await?;
// Leaving manual mode rolls back what is still open.
db.execute("s1", "DELETE FROM users", 1, &c).await?;
db.set_manual_commit("s1", false).await;
db.set_manual_commit("s1", false).await.unwrap();
assert_eq!(count(&db).await, "4");
// Disconnecting with an open transaction does not hang.
db.set_manual_commit("s1", true).await;
db.set_manual_commit("s1", true).await.unwrap();
db.execute("s1", "DELETE FROM users", 1, &c).await?;
db.disconnect("s1").await;
assert_eq!(count(&db).await, "4");
+23 -1
View File
@@ -364,7 +364,7 @@ pub fn package_hash(dir: &Path, files: &[PathBuf]) -> std::io::Result<String> {
sorted.dedup();
let mut hasher = blake3::Hasher::new();
for rel in sorted {
hasher.update(rel.to_string_lossy().as_bytes());
hasher.update(hashed_path(rel).as_bytes());
hasher.update(&[0]);
hasher.update_reader(std::fs::File::open(dir.join(rel))?)?;
hasher.update(&[0]);
@@ -372,6 +372,12 @@ pub fn package_hash(dir: &Path, files: &[PathBuf]) -> std::io::Result<String> {
Ok(hasher.finalize().to_hex().to_string())
}
/// A package-relative path as hashed: `/`-separated on every OS, so a package has one digest
/// (a walked `web\index.html` on Windows is `web/index.html` everywhere else).
fn hashed_path(rel: &Path) -> String {
rel.components().map(|c| c.as_os_str().to_string_lossy()).collect::<Vec<_>>().join("/")
}
#[cfg(test)]
mod tests {
use super::*;
@@ -480,4 +486,20 @@ mod tests {
let groups = by_risk(&caps(&["storage", "process:git", "net:x.io"]));
assert_eq!(groups.iter().map(|g| g.0).collect::<Vec<_>>(), [Risk::High, Risk::Medium, Risk::Low]);
}
#[test]
fn hashes_paths_with_forward_slashes_on_every_os() {
assert_eq!(hashed_path(Path::new("web/index.html")), "web/index.html");
assert_eq!(hashed_path(Path::new("plugin.wasm")), "plugin.wasm");
#[cfg(windows)]
assert_eq!(hashed_path(Path::new(r"web\index.html")), "web/index.html");
let tmp = tempfile::tempdir().unwrap();
std::fs::create_dir(tmp.path().join("web")).unwrap();
std::fs::write(tmp.path().join("web").join("index.html"), "<p>").unwrap();
// A walked path (native separators) hashes like the manifest's `/` spelling.
assert_eq!(
package_hash(tmp.path(), &[Path::new("web").join("index.html")]).unwrap(),
package_hash(tmp.path(), &["web/index.html".into()]).unwrap()
);
}
}
+2 -2
View File
@@ -27,8 +27,8 @@ mod when;
pub use capability::{Capability, CapabilityInfo, Risk, host_matches};
pub use index::ContributionIndex;
pub use manifest::{
ActivationEvent, Capabilities, ENGINE_VERSION, EVENT_KINDS, LEGACY_API, MANIFEST_FILE, Manifest, check_relative,
engine_compatible, validate_item_id, validate_plugin_id, validate_point_id,
ActivationEvent, CATEGORIES, Capabilities, ENGINE_VERSION, EVENT_KINDS, LEGACY_API, MANIFEST_FILE, MAX_CATEGORIES,
Manifest, check_relative, engine_compatible, validate_item_id, validate_plugin_id, validate_point_id,
};
pub use point::{Commit, Disposable, ExtensionPoint, MAX_PACKAGE_FILE, Package, Placement, Prepared, decode};
pub use registry::{ContributionInfo, ExtensionRegistry, PreparedSet};
+66
View File
@@ -35,6 +35,11 @@ pub const MANIFEST_FILE: &str = "plugin.toml";
pub const ENGINE_VERSION: &str = "1.0.0";
/// The 0.1 API, still accepted for contribution-only packages and 0.1 components.
pub const LEGACY_API: &str = "0.1";
/// Marketplace categories, in the order they are shown.
pub const CATEGORIES: &[&str] =
&["languages", "themes", "icon-themes", "linters", "vcs", "ai", "frameworks", "tools", "other"];
/// Categories a plugin is listed under; later ones are ignored.
pub const MAX_CATEGORIES: usize = 3;
#[derive(Clone, Debug, PartialEq)]
pub struct Manifest {
@@ -47,6 +52,8 @@ pub struct Manifest {
pub license: Option<String>,
/// Release notes of this version (Settings › Plugins › What's New).
pub change_notes: Option<String>,
/// Marketplace categories as declared (see [`Manifest::categories`]).
pub declared_categories: Vec<String>,
/// `engines.openidea` (a semver requirement), or the legacy `openidea-api`.
pub engine: String,
/// Written in the 0.1 format.
@@ -166,6 +173,8 @@ struct Raw {
#[serde(default)]
change_notes: Option<String>,
#[serde(default)]
categories: Vec<String>,
#[serde(default)]
engines: Option<Engines>,
/// 0.1 format.
#[serde(default)]
@@ -243,6 +252,7 @@ impl Manifest {
homepage: raw.homepage,
license: raw.license,
change_notes: raw.change_notes,
declared_categories: raw.categories,
engine,
legacy,
wasm: raw.wasm,
@@ -290,6 +300,9 @@ impl Manifest {
return Err(format!("duplicate activation event {event}"));
}
}
for c in &self.declared_categories {
validate_item_id(c, "category")?;
}
for (point, decls) in &self.contributes {
validate_point_id(point)?;
if !matches!(decls, toml::Value::Array(_) | toml::Value::Table(_)) {
@@ -325,6 +338,39 @@ impl Manifest {
}
}
impl Manifest {
/// Its marketplace categories, in [`CATEGORIES`] order: the first
/// [`MAX_CATEGORIES`] declared that this IDE knows; without any, those its
/// contributions imply (themes, icon themes, languages), else `other`.
pub fn categories(&self) -> Vec<String> {
let pick = |f: &dyn Fn(&str) -> bool| CATEGORIES.iter().filter(|c| f(c)).map(|c| c.to_string()).collect();
let mut known: Vec<&str> = Vec::new();
for d in &self.declared_categories {
if CATEGORIES.contains(&d.as_str()) && !known.contains(&d.as_str()) && known.len() < MAX_CATEGORIES {
known.push(d);
}
}
let mut out: Vec<String> = pick(&|c| known.contains(&c));
if out.is_empty() {
out = pick(&|c| match c {
"themes" => self.contributes.contains_key("themes"),
"icon-themes" => self.contributes.contains_key("iconThemes"),
"languages" => self.contributes.contains_key("languages"),
_ => false,
});
}
if out.is_empty() {
out.push("other".into());
}
out
}
/// Declared categories this IDE doesn't know (ignored).
pub fn unknown_categories(&self) -> Vec<&str> {
self.declared_categories.iter().map(String::as_str).filter(|c| !CATEGORIES.contains(c)).collect()
}
}
/// Whether this host can load a plugin written for `engine`.
pub fn engine_compatible(engine: &str, legacy: bool) -> bool {
if legacy {
@@ -417,6 +463,26 @@ mod tests {
assert!(ActivationEvent::parse("on-event:scheduled").is_err(), "declared with on-schedule");
}
#[test]
fn categories_are_declared_or_implied() {
let head = "id = \"acme.x\"\nname = \"X\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"\n";
let m = |rest: &str| Manifest::parse(&format!("{head}{rest}"));
// Declared, in the catalogue's order; unknown ones (a newer IDE's) are ignored.
let declared = m("categories = [\"tools\", \"vcs\", \"robots\"]\n").unwrap();
assert_eq!(declared.categories(), ["vcs", "tools"]);
assert_eq!(declared.unknown_categories(), ["robots"]);
// None declared: implied by the contributions, else `other`.
let theme = m("[[contributes.themes]]\nid = \"t\"\n[[contributes.languages]]\nid = \"l\"\n").unwrap();
assert_eq!(theme.categories(), ["languages", "themes"]);
let declared_wins = m("categories = [\"tools\"]\n[[contributes.languages]]\nid = \"l\"\n").unwrap();
assert_eq!(declared_wins.categories(), ["tools"]);
assert_eq!(m("").unwrap().categories(), ["other"]);
// At most three: the first ones declared.
let many = m("categories = [\"vcs\", \"ai\", \"vcs\", \"tools\", \"themes\"]\n").unwrap();
assert_eq!(many.categories(), ["vcs", "ai", "tools"]);
assert!(m("categories = [\"Bad Name\"]\n").is_err());
}
#[test]
fn parses_the_02_format() {
let m = Manifest::parse(FULL).unwrap();
+513 -23
View File
@@ -9,9 +9,19 @@
//! proves who published the bytes; whether to trust that publisher is the
//! index's call.
//! - **Index** (`index.json`): the publishers and their keys (the trust root),
//! the published plugins, and revocations. A publisher owns the id prefix
//! `<publisher>.`; a revoked key invalidates everything it signed, a
//! revoked plugin version is disabled where it is installed.
//! the published plugin versions, and revocations. A publisher owns the id
//! prefix `<publisher>.`; a revoked key invalidates everything it signed, a
//! revoked plugin version is disabled where it is installed. A plugin may be
//! listed in several versions, each with the plugin API it needs: the IDE
//! offers the newest one it can load.
//! - **Key continuity**: an installed plugin updates silently only to a
//! package signed with the same key, or with a key the installed one
//! endorsed ([`Endorsement`], a publisher rotating keys); any other signer
//! is the user's call. An IDE can pin a publisher to keys it was built with
//! ([`Index::pin_publisher`]): the index then can't replace them.
//! - **Format**: fields are only ever added (IDEs ignore those they don't
//! know); `format` grows only for changes older IDEs can't read, and they
//! still apply such an index's revocations ([`Index::parse`]).
use crate::MANIFEST_FILE;
use base64::Engine as _;
@@ -40,6 +50,24 @@ fn signed_message(id: &str, version: &str, digest: &str) -> Vec<u8> {
format!("openidea-plugin/1\n{id}\n{version}\n{digest}").into_bytes()
}
fn endorsement_message(publisher: &str, key: &str) -> Vec<u8> {
format!("openidea-key/1\n{publisher}\n{key}").into_bytes()
}
/// Whether `signature` (base64) is `key`'s (base64) Ed25519 signature of
/// `message`; malformed keys and signatures are errors.
fn verify_ed25519(key: &str, message: &[u8], signature: &str) -> Result<bool, String> {
let key: [u8; 32] = B64
.decode(key)
.ok()
.and_then(|b| b.try_into().ok())
.ok_or_else(|| "signature: malformed public key".to_owned())?;
let key = VerifyingKey::from_bytes(&key).map_err(|_| "signature: invalid public key".to_owned())?;
let sig: [u8; 64] =
B64.decode(signature).ok().and_then(|b| b.try_into().ok()).ok_or_else(|| "signature: malformed".to_owned())?;
Ok(key.verify_strict(message, &Signature::from_bytes(&sig)).is_ok())
}
/// `SIGNATURE.json` of a signed package.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
@@ -74,6 +102,12 @@ impl PublisherKey {
B64.encode(self.0.verifying_key().to_bytes())
}
/// Vouches for `publisher`'s new key `key` (base64), the one that follows this one.
pub fn endorse(&self, publisher: &str, key: &str) -> Endorsement {
let sig = self.0.sign(&endorsement_message(publisher, key));
Endorsement { key: self.public_base64(), signature: B64.encode(sig.to_bytes()) }
}
pub fn sign(&self, publisher: &str, id: &str, version: &str, digest: &str) -> PackageSignature {
let sig = self.0.sign(&signed_message(id, version, digest));
PackageSignature {
@@ -88,19 +122,11 @@ impl PackageSignature {
/// Whether the signature is valid for `id`, `version` and `digest` (the
/// key's validity is checked against an [`Index`]).
pub fn check(&self, id: &str, version: &str, digest: &str) -> Result<(), String> {
let key: [u8; 32] = B64
.decode(&self.key)
.ok()
.and_then(|b| b.try_into().ok())
.ok_or_else(|| "signature: malformed public key".to_owned())?;
let key = VerifyingKey::from_bytes(&key).map_err(|_| "signature: invalid public key".to_owned())?;
let sig: [u8; 64] = B64
.decode(&self.signature)
.ok()
.and_then(|b| b.try_into().ok())
.ok_or_else(|| "signature: malformed".to_owned())?;
key.verify_strict(&signed_message(id, version, digest), &Signature::from_bytes(&sig))
.map_err(|_| "the signature does not match the package (it was changed after signing)".to_owned())
if verify_ed25519(&self.key, &signed_message(id, version, digest), &self.signature)? {
Ok(())
} else {
Err("the signature does not match the package (it was changed after signing)".to_owned())
}
}
}
@@ -242,6 +268,10 @@ pub struct Index {
pub plugins: Vec<IndexEntry>,
#[serde(default)]
pub revoked: Vec<Revocation>,
/// The marketplace moved: its index is at this URL (relative to this
/// one, or absolute), and the IDE reads that one instead.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub moved: Option<String>,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
@@ -253,7 +283,7 @@ pub struct Publisher {
pub keys: Vec<KeyEntry>,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct KeyEntry {
/// Ed25519 public key, base64.
@@ -261,6 +291,27 @@ pub struct KeyEntry {
/// Packages signed with it are no longer trusted.
#[serde(default)]
pub revoked: bool,
/// The publisher's earlier key vouching for this one.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub endorsement: Option<Endorsement>,
}
impl KeyEntry {
pub fn new(key: String) -> Self {
Self { key, ..Self::default() }
}
}
/// A publisher's earlier key vouching for a newer one: plugins installed from
/// packages signed with the earlier key update to ones signed with the newer
/// key without asking the user.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct Endorsement {
/// The endorsing key, base64.
pub key: String,
/// Its Ed25519 signature of `openidea-key/1\n<publisher>\n<endorsed key>`, base64.
pub signature: String,
}
/// A published plugin version.
@@ -281,6 +332,34 @@ pub struct IndexEntry {
pub digest: String,
#[serde(default)]
pub capabilities: Vec<String>,
/// The plugin API versions it loads in (a semver requirement): its
/// manifest's `engines.openidea` and the API its component was built for.
/// Required: an entry without one is offered to no IDE.
#[serde(default)]
pub engine: String,
/// Marketplace categories (`CATEGORIES`); an IDE ignores those it doesn't know.
#[serde(default)]
pub categories: Vec<String>,
}
impl IndexEntry {
/// Whether this IDE can load the version.
pub fn compatible(&self) -> bool {
crate::engine_compatible(&self.engine, false)
}
fn semver(&self) -> Option<semver::Version> {
semver::Version::parse(&self.version).ok()
}
}
/// `engines.openidea` of a manifest narrowed to the API its component was built
/// for (`major.minor`, if it has code): the requirement an index lists.
pub fn engine_requirement(manifest_engine: &str, built_for: Option<&semver::Version>) -> String {
match built_for {
Some(v) => format!("{manifest_engine}, ^{}.{}", v.major, v.minor),
None => manifest_engine.to_owned(),
}
}
/// A plugin (all versions when `versions` is empty) that must not run.
@@ -294,12 +373,71 @@ pub struct Revocation {
}
impl Index {
/// Parses an index. One in a newer format keeps what must reach every IDE —
/// revocations, publisher keys (revoked ones) and `moved` — and lists no
/// plugins ([`Index::is_newer`]): the IDE needs updating to install from it.
pub fn parse(text: &str) -> Result<Self, String> {
let index: Index = serde_json::from_str(text).map_err(|e| format!("index: {e}"))?;
if index.format != INDEX_FORMAT {
return Err(format!("index format {} is not supported (this IDE reads {INDEX_FORMAT})", index.format));
let value: serde_json::Value = serde_json::from_str(text).map_err(|e| format!("index: {e}"))?;
let format = value.get("format").and_then(serde_json::Value::as_u64).ok_or("index: `format` is missing")?;
if format == u64::from(INDEX_FORMAT) {
return serde_json::from_value(value).map_err(|e| format!("index: {e}"));
}
Ok(index)
if format < u64::from(INDEX_FORMAT) {
return Err(format!("index format {format} is not supported (this IDE reads {INDEX_FORMAT})"));
}
fn each<T: serde::de::DeserializeOwned>(value: &serde_json::Value, field: &str) -> Vec<T> {
let items = value.get(field).and_then(serde_json::Value::as_array);
items.into_iter().flatten().filter_map(|v| serde_json::from_value(v.clone()).ok()).collect()
}
Ok(Index {
format: u32::try_from(format).unwrap_or(u32::MAX),
publishers: each(&value, "publishers"),
plugins: Vec::new(),
revoked: each(&value, "revoked"),
moved: value.get("moved").and_then(serde_json::Value::as_str).map(str::to_owned),
})
}
/// Written in a format newer than this IDE reads: it lists no plugins.
pub fn is_newer(&self) -> bool {
self.format > INDEX_FORMAT
}
/// The version of each plugin to offer, and whether this IDE can load it:
/// the newest it can load, else the newest. Revoked versions are skipped;
/// plugins come in the order they are first listed.
pub fn offered(&self) -> Vec<(&IndexEntry, bool)> {
let mut ids: Vec<&str> = Vec::new();
for e in &self.plugins {
if !ids.contains(&e.id.as_str()) {
ids.push(&e.id);
}
}
ids.into_iter()
.filter_map(|id| {
let versions: Vec<&IndexEntry> =
self.plugins.iter().filter(|e| e.id == id && self.revocation(id, &e.version).is_none()).collect();
let newest = |loadable: bool| {
versions.iter().copied().filter(|e| !loadable || e.compatible()).max_by_key(|e| e.semver())
};
newest(true).map(|e| (e, true)).or_else(|| newest(false).map(|e| (e, false)))
})
.collect()
}
/// Drops the versions no IDE would pick: revoked ones, and of each plugin
/// all but the newest version per API requirement.
pub fn prune_versions(&mut self) {
let plugins = std::mem::take(&mut self.plugins);
let mut kept: Vec<IndexEntry> = Vec::new();
for e in plugins.into_iter().filter(|e| self.revocation(&e.id, &e.version).is_none()) {
match kept.iter_mut().find(|k| k.id == e.id && k.engine == e.engine) {
Some(k) if e.semver() > k.semver() => *k = e,
Some(_) => {}
None => kept.push(e),
}
}
self.plugins = kept;
}
/// The revocation that applies to `id` at `version`.
@@ -335,6 +473,127 @@ impl Index {
sig.check(id, version, digest)?;
Ok(publisher.id.clone())
}
/// Whether `to` is `from`, or a key of `publisher` that `from` endorsed,
/// directly or through keys it endorsed. Endorsements by revoked keys don't
/// count: whoever stole a key could vouch for their own.
pub fn succeeds(&self, publisher: &str, from: &str, to: &str) -> bool {
let keys = self.publishers.iter().find(|p| p.id == publisher).map_or(&[][..], |p| &p.keys[..]);
let mut key = to;
// Each step moves to an earlier key; more steps than keys is a loop.
for _ in 0..=keys.len() {
if key == from {
return true;
}
let Some(e) = keys.iter().find(|k| k.key == key).and_then(|k| k.endorsement.as_ref()) else {
return false;
};
let endorser_valid = keys.iter().any(|k| k.key == e.key && !k.revoked);
if !endorser_valid || verify_ed25519(&e.key, &endorsement_message(publisher, key), &e.signature) != Ok(true)
{
return false;
}
key = &e.key;
}
false
}
/// Makes `roots` (base64) the keys `publisher` signs with, whatever this
/// index says: another key it lists stays only when a root endorsed it (a
/// rotation the roots vouched for). Revocations of the keys are kept.
pub fn pin_publisher(&mut self, publisher: &str, name: &str, roots: &[String]) {
if roots.is_empty() {
return;
}
let i = match self.publishers.iter().position(|p| p.id == publisher) {
Some(i) => i,
None => {
self.publishers.push(Publisher { id: publisher.into(), name: name.into(), keys: Vec::new() });
self.publishers.len() - 1
}
};
for root in roots {
if !self.publishers[i].keys.iter().any(|k| &k.key == root) {
self.publishers[i].keys.push(KeyEntry::new(root.clone()));
}
}
let keys = self.publishers[i].keys.clone();
let trusted = keys
.into_iter()
.filter(|k| roots.iter().any(|r| self.succeeds(publisher, r, &k.key)))
.map(|k| if roots.contains(&k.key) { KeyEntry { endorsement: None, ..k } } else { k })
.collect();
self.publishers[i].keys = trusted;
}
/// Replaces what this index lists for every publisher but `keep` with what
/// `other` lists: its publishers, their plugins and their revocations
/// (`other`'s downloads should already be absolute). `other` may only
/// speak for its own publishers, and none of them may be in `keep`.
pub fn include(&mut self, other: Index, keep: &[&str]) -> Result<(), String> {
let mut theirs: Vec<&str> = Vec::new();
for p in &other.publishers {
if !publisher_id_ok(&p.id) {
return Err(format!("invalid publisher id {:?}: use a-z, 0-9, '-' and '_'", p.id));
}
if keep.contains(&p.id.as_str()) {
return Err(format!("may not list publisher {:?}", p.id));
}
if theirs.contains(&p.id.as_str()) {
return Err(format!("publisher {:?} is listed twice", p.id));
}
theirs.push(&p.id);
}
let owned_by = |id: &str| theirs.iter().any(|p| owns(p, id));
for e in &other.plugins {
if !theirs.contains(&e.publisher.as_str()) || !owns(&e.publisher, &e.id) {
return Err(format!("{}: not a plugin of publisher {:?}", e.id, e.publisher));
}
if semver::VersionReq::parse(&e.engine).is_err() {
return Err(format!("{} {}: `engine` must be the plugin API it needs", e.id, e.version));
}
}
if let Some(r) = other.revoked.iter().find(|r| !owned_by(&r.id)) {
return Err(format!("{}: revoked by an index that doesn't publish it", r.id));
}
let kept = |id: &str| keep.iter().any(|p| owns(p, id));
self.publishers.retain(|p| keep.contains(&p.id.as_str()));
self.plugins.retain(|e| keep.contains(&e.publisher.as_str()));
self.revoked.retain(|r| kept(&r.id));
self.publishers.extend(other.publishers);
self.plugins.extend(other.plugins);
self.revoked.extend(other.revoked);
Ok(())
}
}
/// Whether a package signed `new` (`None`: unsigned) may replace an installed
/// one signed `installed` without asking the user: signed with the same key,
/// or with one the installed key endorsed (per `index`). An unsigned installed
/// copy pins nothing.
pub fn signer_continues(
index: Option<&Index>,
installed: Option<&PackageSignature>,
new: Option<&PackageSignature>,
) -> bool {
match (installed, new) {
(None, _) => true,
(Some(_), None) => false,
(Some(old), Some(new)) => {
old.publisher == new.publisher
&& (old.key == new.key || index.is_some_and(|i| i.succeeds(&new.publisher, &old.key, &new.key)))
}
}
}
/// Whether the plugin `id` is in `publisher`'s namespace.
fn owns(publisher: &str, id: &str) -> bool {
id.strip_prefix(publisher).is_some_and(|rest| rest.starts_with('.'))
}
/// A publisher id is a single segment of a plugin id.
pub fn publisher_id_ok(id: &str) -> bool {
!id.contains('.') && crate::validate_plugin_id(id).is_ok()
}
#[cfg(test)]
@@ -354,7 +613,7 @@ mod tests {
publishers: vec![Publisher {
id: "acme".into(),
name: "ACME".into(),
keys: vec![KeyEntry { key: key.public_base64(), revoked: false }],
keys: vec![KeyEntry::new(key.public_base64())],
}],
plugins: Vec::new(),
revoked: vec![Revocation {
@@ -362,6 +621,7 @@ mod tests {
versions: vec!["1.0.0".into()],
reason: "malware".into(),
}],
moved: None,
}
}
@@ -399,6 +659,138 @@ mod tests {
assert_eq!(PublisherKey::from_base64(&key.seed_base64()).unwrap().public_base64(), key.public_base64());
}
fn entry(id: &str, publisher: &str) -> IndexEntry {
IndexEntry {
id: id.into(),
name: id.into(),
version: "1.0.0".into(),
description: String::new(),
authors: Vec::new(),
publisher: publisher.into(),
download: format!("https://example.com/{id}.oiplugin"),
digest: "d".into(),
capabilities: Vec::new(),
engine: "^1.0".into(),
categories: Vec::new(),
}
}
fn version(id: &str, version: &str, engine: &str) -> IndexEntry {
IndexEntry { version: version.into(), engine: engine.into(), ..entry(id, "acme") }
}
#[test]
fn the_ide_is_offered_the_newest_version_it_can_load() {
let host = crate::ENGINE_VERSION;
let newer = semver::Version::parse(host).unwrap().major + 1;
let mut index = Index {
format: INDEX_FORMAT,
plugins: vec![
version("acme.a", "1.0.0", &format!("^{host}")),
version("acme.a", "1.2.0", &format!("^{host}")),
version("acme.a", "2.0.0", &format!("^{newer}.0")),
version("acme.b", "0.1.0", &format!("^{newer}.0")),
version("acme.b", "0.2.0", &format!("^{newer}.1")),
version("acme.c", "1.0.0", ""),
version("acme.c", "1.1.0", ""),
],
revoked: vec![Revocation { id: "acme.c".into(), versions: vec!["1.1.0".into()], reason: "r".into() }],
..Index::default()
};
let offered = |index: &Index| {
index.offered().into_iter().map(|(e, ok)| (e.id.clone(), e.version.clone(), ok)).collect::<Vec<_>>()
};
let expected = [
("acme.a".to_owned(), "1.2.0".to_owned(), true),
("acme.b".to_owned(), "0.2.0".to_owned(), false),
// No `engine`: listed, but no IDE installs it.
("acme.c".to_owned(), "1.0.0".to_owned(), false),
];
assert_eq!(offered(&index), expected);
// Pruning keeps the newest version per requirement, which changes nothing offered.
index.prune_versions();
let left: Vec<_> = index.plugins.iter().map(|e| format!("{} {}", e.id, e.version)).collect();
assert_eq!(left, ["acme.a 1.2.0", "acme.a 2.0.0", "acme.b 0.1.0", "acme.b 0.2.0", "acme.c 1.0.0"]);
assert_eq!(offered(&index), expected);
let built_for = semver::Version::new(1, 3, 0);
assert_eq!(engine_requirement("^1.0", Some(&built_for)), "^1.0, ^1.3");
assert_eq!(engine_requirement("^1.0", None), "^1.0");
assert!(semver::VersionReq::parse(&engine_requirement("^1.0", Some(&built_for))).is_ok());
}
fn publisher(id: &str) -> Publisher {
Publisher { id: id.into(), name: id.into(), keys: vec![KeyEntry::new(format!("{id}-key"))] }
}
fn revocation(id: &str) -> Revocation {
Revocation { id: id.into(), versions: Vec::new(), reason: "r".into() }
}
#[test]
fn including_an_index_replaces_everything_but_the_kept_publishers() {
let mut official = Index {
format: INDEX_FORMAT,
publishers: vec![publisher("openidea"), publisher("gone")],
plugins: vec![entry("openidea.a", "openidea"), entry("gone.x", "gone")],
revoked: vec![revocation("openidea.old"), revocation("gone.x")],
moved: None,
};
let community = Index {
format: INDEX_FORMAT,
publishers: vec![publisher("alice")],
plugins: vec![entry("alice.arthas", "alice")],
revoked: vec![revocation("alice.bad")],
moved: None,
};
official.include(community.clone(), &["openidea"]).unwrap();
let ids = |v: &[IndexEntry]| v.iter().map(|e| e.id.clone()).collect::<Vec<_>>();
assert_eq!(official.publishers.iter().map(|p| p.id.as_str()).collect::<Vec<_>>(), ["openidea", "alice"]);
assert_eq!(ids(&official.plugins), ["openidea.a", "alice.arthas"]);
assert_eq!(official.revoked.iter().map(|r| r.id.as_str()).collect::<Vec<_>>(), ["openidea.old", "alice.bad"]);
// Including again is idempotent.
let once = official.clone();
official.include(community.clone(), &["openidea"]).unwrap();
assert_eq!(official, once);
}
#[test]
fn an_included_index_speaks_only_for_its_own_publishers() {
let base = Index { format: INDEX_FORMAT, publishers: vec![publisher("openidea")], ..Index::default() };
let with = |publishers: Vec<Publisher>, plugins: Vec<IndexEntry>, revoked: Vec<Revocation>| Index {
format: INDEX_FORMAT,
publishers,
plugins,
revoked,
moved: None,
};
let cases = [
(with(vec![publisher("openidea")], vec![], vec![]), "may not list"),
(with(vec![publisher("alice"), publisher("alice")], vec![], vec![]), "listed twice"),
(with(vec![publisher("openidea.x")], vec![], vec![]), "invalid publisher"),
(with(vec![publisher("alice")], vec![entry("openidea.a", "alice")], vec![]), "not a plugin of"),
(with(vec![publisher("alice")], vec![entry("alice.a", "bob")], vec![]), "not a plugin of"),
(with(vec![publisher("alice")], vec![entry("alicex.a", "alice")], vec![]), "not a plugin of"),
(with(vec![publisher("alice")], vec![], vec![revocation("openidea.a")]), "doesn't publish"),
(
with(
vec![publisher("alice")],
vec![IndexEntry { engine: String::new(), ..entry("alice.a", "alice") }],
vec![],
),
"`engine`",
),
];
for (other, error) in cases {
let mut index = base.clone();
let err = index.include(other, &["openidea"]).unwrap_err();
assert!(err.contains(error), "{err}");
assert_eq!(index, base);
}
}
#[test]
fn unpacking_refuses_paths_leaving_the_package() {
let tmp = tempfile::tempdir().unwrap();
@@ -413,7 +805,7 @@ mod tests {
let err = unpack(&out, &tmp.path().join("dest")).unwrap_err();
assert!(err.contains("invalid path"), "{err}");
assert!(!tmp.path().join("outside.txt").exists());
assert!(Index::parse(r#"{"format": 2}"#).is_err());
assert!(Index::parse(r#"{"format": 2}"#).unwrap().is_newer());
}
fn zip_of(path: &Path, entries: &[(&str, &str)]) {
@@ -462,4 +854,102 @@ mod tests {
assert!(err.contains("is not an OpenIDEA plugin"), "{name}: {err}");
}
}
#[test]
fn a_newer_index_still_revokes_and_moves() {
let newer = format!(
r#"{{"format": {}, "plugins": [{{"anything": "new"}}], "shape": "changed",
"publishers": [{{"id": "acme", "name": "ACME", "keys": [{{"key": "k1", "revoked": true}}]}}, {{"id": 5}}],
"revoked": [{{"id": "acme.bad", "reason": "malware"}}, {{"oops": 1}}],
"moved": "https://example.com/next.json"}}"#,
INDEX_FORMAT + 1
);
let index = Index::parse(&newer).unwrap();
assert!(index.is_newer());
assert!(index.plugins.is_empty() && index.offered().is_empty());
assert_eq!(index.revocation("acme.bad", "2.0.0").unwrap().reason, "malware");
assert!(index.key_revoked("k1"));
assert_eq!(index.moved.as_deref(), Some("https://example.com/next.json"));
// Fields this IDE doesn't know are ignored in its own format.
let current = format!(r#"{{"format": {INDEX_FORMAT}, "later": true, "plugins": []}}"#);
assert!(!Index::parse(&current).unwrap().is_newer());
assert!(Index::parse(r#"{"format": 0}"#).unwrap_err().contains("not supported"));
assert!(Index::parse(r#"{"plugins": []}"#).is_err());
}
fn keys<'a>(index: &'a mut Index, publisher: &str) -> &'a mut Vec<KeyEntry> {
&mut index.publishers.iter_mut().find(|p| p.id == publisher).unwrap().keys
}
#[test]
fn rotated_keys_continue_through_endorsements() {
let (k1, k2, k3) =
(PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([2; 32]), PublisherKey::from_seed([3; 32]));
let mut index = index(&k1);
let endorsed = |by: &PublisherKey, key: &PublisherKey| KeyEntry {
endorsement: Some(by.endorse("acme", &key.public_base64())),
..KeyEntry::new(key.public_base64())
};
keys(&mut index, "acme").push(endorsed(&k1, &k2));
keys(&mut index, "acme").push(endorsed(&k2, &k3));
let (p1, p2, p3) = (k1.public_base64(), k2.public_base64(), k3.public_base64());
assert!(index.succeeds("acme", &p1, &p1));
assert!(index.succeeds("acme", &p1, &p3), "through k2");
assert!(!index.succeeds("acme", &p3, &p1), "not backwards");
assert!(!index.succeeds("other", &p1, &p2));
// An endorsement for another publisher, or by a revoked key, doesn't count.
let mut foreign = index.clone();
keys(&mut foreign, "acme")[1].endorsement = Some(k1.endorse("evil", &p2));
assert!(!foreign.succeeds("acme", &p1, &p2));
let mut stolen = index.clone();
keys(&mut stolen, "acme")[0].revoked = true;
assert!(!stolen.succeeds("acme", &p1, &p2));
let sig = |key: &PublisherKey| key.sign("acme", "acme.tools", "1.0.0", "d");
assert!(signer_continues(Some(&index), Some(&sig(&k1)), Some(&sig(&k3))));
assert!(signer_continues(None, Some(&sig(&k1)), Some(&sig(&k1))));
assert!(!signer_continues(None, Some(&sig(&k1)), Some(&sig(&k2))), "no index to vouch");
let stranger = PublisherKey::from_seed([4; 32]);
assert!(!signer_continues(Some(&index), Some(&sig(&k1)), Some(&sig(&stranger))));
assert!(!signer_continues(Some(&index), Some(&sig(&k1)), None), "signed, then unsigned");
assert!(signer_continues(Some(&index), None, Some(&sig(&stranger))), "unsigned pins nothing");
}
#[test]
fn pinned_publishers_keep_their_built_in_keys() {
let (root, next, intruder) =
(PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([2; 32]), PublisherKey::from_seed([5; 32]));
let mut index = index(&intruder);
index.publishers[0].id = "openidea".into();
keys(&mut index, "openidea").push(KeyEntry {
endorsement: Some(root.endorse("openidea", &next.public_base64())),
..KeyEntry::new(next.public_base64())
});
let roots = [root.public_base64()];
let listed = |index: &Index| -> Vec<(String, bool)> {
let mut index = index.clone();
index.pin_publisher("openidea", "OpenIDEA", &roots);
keys(&mut index, "openidea").iter().map(|k| (k.key.clone(), k.revoked)).collect()
};
// The intruder's key goes, the successor the root endorsed stays, the root is added.
assert_eq!(listed(&index), [(next.public_base64(), false), (root.public_base64(), false)]);
let mut pinned = index.clone();
pinned.pin_publisher("openidea", "OpenIDEA", &roots);
let digest = "d";
let signed = |k: &PublisherKey| k.sign("openidea", "openidea.tools", "1.0.0", digest);
assert!(pinned.verify(&signed(&intruder), "openidea.tools", "1.0.0", digest).is_err());
assert!(pinned.verify(&signed(&next), "openidea.tools", "1.0.0", digest).is_ok());
// A revoked root stays revoked, and what it endorsed is no longer trusted.
keys(&mut index, "openidea").push(KeyEntry { revoked: true, ..KeyEntry::new(root.public_base64()) });
assert_eq!(listed(&index), [(root.public_base64(), true)]);
// A publisher missing from the index is added with its roots.
let mut empty = Index { format: INDEX_FORMAT, ..Index::default() };
empty.pin_publisher("openidea", "OpenIDEA", &[root.public_base64()]);
assert_eq!(empty.verify(&signed(&root), "openidea.tools", "1.0.0", digest).unwrap(), "openidea");
// No roots: the index decides.
let mut open = index.clone();
open.pin_publisher("openidea", "OpenIDEA", &[]);
assert_eq!(open, index);
}
}
+221 -21
View File
@@ -21,6 +21,18 @@ pub struct FontFamily {
pub monospaced: bool,
}
/// PANOSE bProportion value 9 means "Monospaced"; DirectWrite's
/// `IsMonospacedFont` alone misses fonts that only carry this classification.
const PANOSE_MONOSPACED: u8 = 9;
/// One face counts as monospaced when the font system flags it fixed pitch, its
/// PANOSE proportion is "Monospaced", or its glyph advances are uniform
/// (fonts with wrong/missing metadata, e.g. bitmap-style TTFs).
#[cfg_attr(not(windows), allow(dead_code))]
fn face_is_mono(is_monospaced_font: bool, panose_proportion: u8, advances_equal: bool) -> bool {
is_monospaced_font || panose_proportion == PANOSE_MONOSPACED || advances_equal
}
type Cache = Mutex<Option<Arc<[FontFamily]>>>;
/// The installed families, sorted case-insensitively, read on first use and
@@ -195,7 +207,9 @@ mod imp {
// SAFETY: CoreText descriptors carry documented CFString, CFURL and trait dictionary values.
unsafe {
let collection = CTFontCollection::from_available_fonts(None);
let Some(descriptors) = collection.matching_font_descriptors() else { return Vec::new() };
let Some(descriptors) = collection.matching_font_descriptors() else {
return Vec::new();
};
let mut seen = std::collections::BTreeSet::new();
descriptors
.cast_unchecked::<CTFontDescriptor>()
@@ -271,14 +285,16 @@ mod imp {
#[cfg(windows)]
mod imp {
use std::os::windows::ffi::OsStrExt;
use windows::Win32::Graphics::DirectWrite::IDWriteFontFace1;
use windows::Win32::Graphics::DirectWrite::{
DWRITE_FACTORY_TYPE_SHARED, DWRITE_FONT_STRETCH_NORMAL, DWRITE_FONT_STYLE_NORMAL, DWRITE_FONT_WEIGHT_NORMAL,
DWriteCreateFactory, IDWriteFactory, IDWriteFont1, IDWriteLocalizedStrings,
DWRITE_FACTORY_TYPE_SHARED, DWriteCreateFactory, IDWriteFactory, IDWriteFactory3, IDWriteFactory5, IDWriteFont,
IDWriteFont1, IDWriteFontCollection, IDWriteFontFamily, IDWriteLocalizedStrings,
};
use windows::core::{BOOL, Interface, w};
use windows::core::{BOOL, Interface, PCWSTR, w};
/// The system font collection's families, named in English when the
/// family has an English name (what CSS matches), else its first name.
/// The installed families, named in English when the family has an
/// English name (what CSS matches), else its first name.
pub(crate) fn list(refresh: bool) -> Vec<(String, bool)> {
// SAFETY: plain DirectWrite COM calls on interfaces we own.
unsafe { read(refresh) }.unwrap_or_default()
@@ -286,11 +302,85 @@ mod imp {
unsafe fn read(refresh: bool) -> windows::core::Result<Vec<(String, bool)>> {
let factory: IDWriteFactory = unsafe { DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED) }?;
let mut faces = Vec::new();
// Windows 10+: the newer collection covers registered per-user fonts.
// Exclude downloadable fonts that are not installed on this machine.
// Bitmap .FON fonts (e.g. the stock raster Fixedsys) are invisible to
// DirectWrite either way: it only reads TrueType/OpenType files.
let mut collection = None;
unsafe { factory.GetSystemFontCollection(&mut collection, refresh) }?;
let Some(collection) = collection else {
return Ok(Vec::new());
if let Ok(factory3) = factory.cast::<IDWriteFactory3>() {
let mut local = None;
if unsafe { factory3.GetSystemFontCollection(false, &mut local, refresh) }.is_ok() {
collection = local.map(Into::into);
}
}
if collection.is_none() {
unsafe { factory.GetSystemFontCollection(&mut collection, refresh) }?;
}
if let Some(collection) = collection {
faces.extend(collect(&collection));
}
// DirectWrite enumerates via the font registry, so font files copied
// into the per-user directory without registering (no HKCU entry)
// are invisible to the calls above; scan the directory directly.
faces.extend(unsafe { user_dir_faces(&factory) });
Ok(faces)
}
/// Families from `%LOCALAPPDATA%\Microsoft\Windows\Fonts` files, registered
/// or not, via a font set built from the files on disk.
unsafe fn user_dir_faces(factory: &IDWriteFactory) -> Vec<(String, bool)> {
let Some(local_app_data) = std::env::var_os("LOCALAPPDATA") else {
return Vec::new();
};
let dir = std::path::Path::new(&local_app_data).join(r"Microsoft\Windows\Fonts");
let Ok(entries) = std::fs::read_dir(&dir) else {
return Vec::new();
};
let files: Vec<_> = entries
.filter_map(|e| e.ok())
.map(|e| e.path())
.filter(|p| {
matches!(
p.extension().and_then(|e| e.to_str()).map(|e| e.to_ascii_lowercase()).as_deref(),
Some("ttf" | "otf" | "ttc" | "otc")
)
})
.collect();
unsafe { file_faces(factory, &files) }
}
unsafe fn file_faces(factory: &IDWriteFactory, files: &[std::path::PathBuf]) -> Vec<(String, bool)> {
if files.is_empty() {
return Vec::new();
}
let Ok(factory3) = factory.cast::<IDWriteFactory3>() else {
return Vec::new();
};
let Ok(factory5) = factory.cast::<IDWriteFactory5>() else {
return Vec::new();
};
let Ok(builder) = (unsafe { factory5.CreateFontSetBuilder() }) else {
return Vec::new();
};
for path in files {
let wide: Vec<u16> = path.as_os_str().encode_wide().chain(Some(0)).collect();
// SAFETY: wide is a valid NUL-terminated string for the calls.
let Ok(file) = (unsafe { factory.CreateFontFileReference(PCWSTR(wide.as_ptr()), None) }) else {
continue;
};
let _ = unsafe { builder.AddFontFile(&file) };
}
let Ok(set) = (unsafe { builder.CreateFontSet() }) else {
return Vec::new();
};
let Ok(collection) = (unsafe { factory3.CreateFontCollectionFromFontSet(&set) }) else {
return Vec::new();
};
collect(&collection)
}
fn collect(collection: &IDWriteFontCollection) -> Vec<(String, bool)> {
let mut out = Vec::new();
for i in 0..unsafe { collection.GetFontFamilyCount() } {
let Ok(family) = (unsafe { collection.GetFontFamily(i) }) else {
@@ -299,19 +389,120 @@ mod imp {
let Some(name) = unsafe { family.GetFamilyNames() }.ok().and_then(|n| unsafe { english(&n) }) else {
continue;
};
let mono = unsafe {
family.GetFirstMatchingFont(
DWRITE_FONT_WEIGHT_NORMAL,
DWRITE_FONT_STRETCH_NORMAL,
DWRITE_FONT_STYLE_NORMAL,
)
}
.ok()
.and_then(|f| f.cast::<IDWriteFont1>().ok())
.is_some_and(|f| unsafe { f.IsMonospacedFont() }.as_bool());
out.push((name, mono));
out.push((name, unsafe { family_is_mono(&family) }));
}
out
}
/// A family is monospaced when any face reports fixed pitch, PANOSE
/// proportion 9, or uniform glyph advances; probing every face matches
/// `normalize`'s any-face merge (macOS and Linux already flag every face,
/// not just the regular one).
unsafe fn family_is_mono(family: &IDWriteFontFamily) -> bool {
(0..unsafe { family.GetFontCount() }).any(|i| {
let Ok(font) = (unsafe { family.GetFont(i) }) else {
return false;
};
if let Ok(font1) = font.cast::<IDWriteFont1>() {
let is_monospaced = unsafe { font1.IsMonospacedFont() }.as_bool();
// The proportion byte is defined for text/display fonts only.
let panose = unsafe { font1.GetPanose().values };
let panose_proportion = if panose[0] == 2 { panose[3] } else { 0 };
if super::face_is_mono(is_monospaced, panose_proportion, false) {
return true;
}
}
unsafe { advances_equal(&font) }
})
}
/// Some monospaced fonts carry no fixed-pitch metadata at all (bitmap-style
/// TTFs like Fixedsys Excelsior): compare the advance widths of 'i' and
/// 'W'. Dual-width CJK monospace fonts pass too (both half-width).
unsafe fn advances_equal(font: &IDWriteFont) -> bool {
let Ok(face) = (unsafe { font.CreateFontFace() }) else {
return false;
};
let Ok(face1) = face.cast::<IDWriteFontFace1>() else {
return false;
};
let codepoints = ['i' as u32, 'W' as u32];
let mut indices = [0u16; 2];
if unsafe { face.GetGlyphIndices(codepoints.as_ptr(), 2, indices.as_mut_ptr()) }.is_err()
|| indices.contains(&0)
{
return false;
}
let mut advances = [0i32; 2];
unsafe { face1.GetDesignGlyphAdvances(2, indices.as_ptr(), advances.as_mut_ptr(), false) }.is_ok()
&& advances[0] > 0
&& advances[0] == advances[1]
}
#[test]
fn glyph_probe_distinguishes_monospace_proportional_and_symbol_fonts() {
unsafe {
let factory: IDWriteFactory = DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED).unwrap();
let mut collection = None;
factory.GetSystemFontCollection(&mut collection, false).unwrap();
let collection = collection.unwrap();
for (name, expected) in [
("Consolas", true),
("Arial", false),
("Times New Roman", false),
("Wingdings", false),
("Webdings", false),
("Segoe MDL2 Assets", false),
] {
let wide: Vec<_> = name.encode_utf16().chain(Some(0)).collect();
let mut index = 0;
let mut exists = BOOL(0);
collection.FindFamilyName(PCWSTR(wide.as_ptr()), &mut index, &mut exists).unwrap();
assert!(exists.as_bool(), "{name} ships with Windows");
let family = collection.GetFontFamily(index).unwrap();
for i in 0..family.GetFontCount() {
let font = family.GetFont(i).unwrap();
assert_eq!(advances_equal(&font), expected, "{name} face {i}");
}
}
}
}
#[test]
fn font_files_include_every_family_in_a_collection() {
let path = std::path::PathBuf::from(std::env::var_os("WINDIR").unwrap()).join(r"Fonts\cambria.ttc");
unsafe {
let factory: IDWriteFactory = DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED).unwrap();
let faces = file_faces(&factory, &[path]);
for name in ["Cambria", "Cambria Math"] {
assert!(faces.iter().any(|(family, _)| family == name), "missing {name}: {faces:?}");
}
}
}
#[test]
fn excludes_families_available_only_as_downloads() {
use windows::Win32::Graphics::DirectWrite::{DWRITE_E_REMOTEFONT, DWRITE_LOCALITY_LOCAL, IDWriteFont3};
unsafe {
let factory: IDWriteFactory = DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED).unwrap();
let factory3: IDWriteFactory3 = factory.cast().unwrap();
let set = factory3.GetSystemFontSet().unwrap();
let collection = factory3.CreateFontCollectionFromFontSet(&set).unwrap();
let listed = super::normalize(read(false).unwrap());
let user = super::normalize(user_dir_faces(&factory));
for i in 0..collection.GetFontFamilyCount() {
let family = collection.GetFontFamily(i).unwrap();
let name = english(&family.GetFamilyNames().unwrap()).unwrap();
let remote_only = (0..family.GetFontCount()).all(|j| match family.GetFont(j) {
Ok(font) => font.cast::<IDWriteFont3>().unwrap().GetLocality() != DWRITE_LOCALITY_LOCAL,
Err(error) => error.code() == DWRITE_E_REMOTEFONT,
});
if remote_only && !user.iter().any(|f| f.name == name) {
assert!(!listed.iter().any(|f| f.name == name), "{name} is not installed locally");
}
}
}
Ok(out)
}
unsafe fn english(names: &IDWriteLocalizedStrings) -> Option<String> {
@@ -476,6 +667,15 @@ mod tests {
);
}
#[test]
fn panose_monospaced_proportion_counts_as_mono() {
assert!(face_is_mono(true, 0, false));
assert!(face_is_mono(false, PANOSE_MONOSPACED, false));
assert!(face_is_mono(false, 0, true));
assert!(!face_is_mono(false, 0, false));
assert!(!face_is_mono(false, 3, false));
}
#[test]
fn fc_list_spacing_marks_monospace() {
let out = "DejaVu Sans Mono\t100\nDejaVu Sans\t\nNoto Sans Mono CJK SC\t90\nTerminus\t110\nCantarell\nFira\\-Code\tmono\n\t100\n";
+145
View File
@@ -0,0 +1,145 @@
//! The signed-in GitHub account, independent of any project: its repositories for
//! "Get from Version Control" › GitHub.
use crate::{GhAvailability, QUICK, classify_failure, command, gh_executable, process};
use serde::{Deserialize, Serialize};
use specta::Type;
/// A repository of the signed-in account (owned, collaborator, or organization member).
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct GhAccountRepo {
/// `owner/name`.
pub name_with_owner: String,
pub description: Option<String>,
pub private: bool,
pub fork: bool,
pub archived: bool,
/// What `git clone` takes: HTTPS, or SSH when gh's `git_protocol` is `ssh`.
pub clone_url: String,
}
/// The account's repositories, or why they can't be listed.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase", tag = "kind")]
pub enum GhAccountRepos {
Ready {
login: String,
repos: Vec<GhAccountRepo>,
},
/// No `gh` executable was found.
NotInstalled,
/// `gh auth login` has not been run (or the token expired).
NotLoggedIn {
message: String,
},
Error {
message: String,
},
}
#[derive(Deserialize)]
struct RestRepo {
full_name: String,
description: Option<String>,
#[serde(default)]
private: bool,
#[serde(default)]
fork: bool,
#[serde(default)]
archived: bool,
clone_url: String,
ssh_url: String,
}
/// `gh api --paginate --slurp user/repos` output: one array per page.
pub fn parse_account_repos(slurped: &str, ssh: bool) -> Result<Vec<GhAccountRepo>, serde_json::Error> {
let pages: Vec<Vec<RestRepo>> = serde_json::from_str(slurped)?;
Ok(pages
.into_iter()
.flatten()
.map(|r| GhAccountRepo {
name_with_owner: r.full_name,
description: r.description.filter(|d| !d.trim().is_empty()),
private: r.private,
fork: r.fork,
archived: r.archived,
clone_url: if ssh { r.ssh_url } else { r.clone_url },
})
.collect())
}
/// Lists the repositories of the account gh is signed in to, most recently pushed first.
pub fn account_repositories() -> GhAccountRepos {
if gh_executable().is_none() {
return GhAccountRepos::NotInstalled;
}
let dir = std::env::temp_dir();
let gh = |args: &[&str]| -> Result<String, GhAccountRepos> {
let out = command(&dir, args)
.and_then(|cmd| process::run(cmd, QUICK, None))
.map_err(|e| GhAccountRepos::Error { message: e.message })?;
if out.success {
return Ok(out.stdout);
}
let message = if out.stderr.trim().is_empty() { out.stdout } else { out.stderr };
Err(match classify_failure(&message) {
GhAvailability::NotLoggedIn { message } => GhAccountRepos::NotLoggedIn { message },
_ => GhAccountRepos::Error { message: message.trim().to_owned() },
})
};
let result = (|| {
let login = gh(&["api", "user", "--jq", ".login"])?.trim().to_owned();
let ssh = gh(&["config", "get", "git_protocol", "--host", "github.com"]).is_ok_and(|p| p.trim() == "ssh");
let pages = gh(&[
"api",
"user/repos?per_page=100&sort=pushed&affiliation=owner,collaborator,organization_member",
"--paginate",
"--slurp",
])?;
let repos = parse_account_repos(&pages, ssh)
.map_err(|e| GhAccountRepos::Error { message: format!("unexpected gh output: {e}") })?;
Ok(GhAccountRepos::Ready { login, repos })
})();
result.unwrap_or_else(|e| e)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn reads_every_page_and_picks_the_configured_protocol() {
let pages = r#"[
[{"full_name":"me/app","description":"An app","private":true,"fork":false,"archived":false,
"clone_url":"https://github.com/me/app.git","ssh_url":"[email protected]:me/app.git"}],
[{"full_name":"org/lib","description":"","fork":true,
"clone_url":"https://github.com/org/lib.git","ssh_url":"[email protected]:org/lib.git"}]
]"#;
let https = parse_account_repos(pages, false).unwrap();
assert_eq!(
https,
vec![
GhAccountRepo {
name_with_owner: "me/app".into(),
description: Some("An app".into()),
private: true,
fork: false,
archived: false,
clone_url: "https://github.com/me/app.git".into(),
},
GhAccountRepo {
name_with_owner: "org/lib".into(),
description: None,
private: false,
fork: true,
archived: false,
clone_url: "https://github.com/org/lib.git".into(),
},
]
);
let ssh = parse_account_repos(pages, true).unwrap();
assert_eq!(ssh[1].clone_url, "[email protected]:org/lib.git");
assert!(parse_account_repos("[]", false).unwrap().is_empty());
}
}
+2
View File
@@ -3,9 +3,11 @@
//!
//! Every function blocks; callers run them off the UI / command threads.
mod account;
mod model;
mod process;
pub use account::{GhAccountRepo, GhAccountRepos, account_repositories, parse_account_repos};
pub use model::*;
use ide_core::{CancellationToken, IdeError, IdeResult};
+5 -1
View File
@@ -14,6 +14,7 @@ encoding_rs.workspace = true
ide-core.workspace = true
ide-text.workspace = true
ide-vfs.workspace = true
regex-syntax.workspace = true
ide-workspace.workspace = true
ide-language.workspace = true
ide-index.workspace = true
@@ -44,6 +45,8 @@ ide-notebook.workspace = true
ide-extension.workspace = true
ide-plugin.workspace = true
ide-tools = { path = "../ide-tools" }
ide-docker = { path = "../ide-docker" }
semver = "1"
serde.workspace = true
serde_json.workspace = true
toml.workspace = true
@@ -57,12 +60,13 @@ dunce.workspace = true
dirs.workspace = true
uuid.workspace = true
tempfile.workspace = true
zip.workspace = true
quick-xml = "0.42"
[dev-dependencies]
ide-ssh = { workspace = true, features = ["testing"] }
tokio = { workspace = true, features = ["rt-multi-thread", "macros", "io-util", "test-util"] }
criterion = { version = "0.5", default-features = false }
zip.workspace = true
[target.'cfg(unix)'.dependencies]
rustix = { version = "1.1", features = ["fs", "process"] }
+419 -55
View File
@@ -3,13 +3,15 @@
//! ```text
//! openidea-plugin new <dir> [--template command|theme|icon-theme|language] [--id ID] [--name NAME] [--sdk PATH]
//! openidea-plugin build [dir] [--debug] # cargo → dist/plugin.wasm, then `check`
//! openidea-plugin check [dir] # validate the package as the IDE would install it
//! openidea-plugin check [dir|package] # validate the package as the IDE would install it
//! openidea-plugin link [dir] # load the folder in the IDE without installing it
//! openidea-plugin unlink [dir]
//! openidea-plugin links # list linked folders
//! openidea-plugin keygen <key-file> # a publisher's Ed25519 key
//! openidea-plugin endorse <new-public-key> --key OLD --publisher ID # vouch for a new key
//! openidea-plugin pack [dir] [--key F --publisher ID] [--out FILE] # <id>-<version>.oiplugin
//! openidea-plugin index <dir> --publisher ID --name NAME --key F # marketplace index.json
//! openidea-plugin index <dir> [--publisher ID --name NAME --key F [--previous-key F]] # marketplace index.json
//! openidea-plugin merge <index> <other-index> --keep ID # list another index's publishers
//! ```
//!
//! `link` writes `links.json` in the IDE's plugins directory
@@ -20,7 +22,7 @@
mod templates;
use ide_extension::{Manifest, validate_plugin_id};
use ide_host::plugins::PluginHost;
use ide_host::plugins::{PluginHost, read_market_url, resolve_market_href};
use ide_plugin::{Limits, PluginEngine};
use std::path::{Path, PathBuf};
use std::process::{Command, ExitCode};
@@ -31,17 +33,31 @@ usage: openidea-plugin <command> [options]
commands:
new <dir> [--template T] [--id ID] [--name NAME] [--sdk PATH]
create a plugin; T = command (default), theme, icon-theme, language
create a plugin; T = command (default), theme, icon-theme, language. A Rust
plugin depends on the SDK of this tool's release (git); --sdk PATH uses a
local sdk/rust/openidea-plugin instead
build [dir] [--debug] compile a Rust plugin to dist/plugin.wasm and check it
check [dir] validate the package as the IDE would install it
check [dir|package] validate the package (folder or .oiplugin) as the IDE would install it
link [dir] load the folder in the IDE without installing it
unlink [dir] stop loading a linked folder
links list linked folders
keygen <key-file> create a publisher signing key (keep the file secret)
endorse <new-public-key> --key F --publisher ID
print an earlier key's endorsement of a new one (JSON), for an
index the publisher doesn't write with `index --previous-key`
pack [dir] [--key F --publisher ID] [--out FILE]
write <id>-<version>.oiplugin, signed with the key
index <dir> --publisher ID --name NAME --key F [--out FILE]
write the marketplace index.json for the packages in dir
index <dir> [--publisher ID --name NAME --key F [--previous-key F]] [--out FILE]
[--download-base URL]
write the marketplace index.json for the packages in dir: with a
key, register it and list that publisher's plugins (earlier
versions stay for older IDEs); without, list only these packages,
each checked against the index's publishers. A new key replacing
an earlier one is endorsed with --previous-key, so installed
plugins update to it without asking
merge <index> <other-index> --keep ID [--out FILE]
replace every publisher but ID in index with those of other-index
(a path or URL), their plugins and revocations
environment:
OPENIDEA_PLUGINS_DIR the IDE's plugins directory (default: the app data directory)";
@@ -51,13 +67,15 @@ fn main() -> ExitCode {
let result = match args.first().map(String::as_str) {
Some("new") => cmd_new(&args[1..]),
Some("build") => cmd_build(&args[1..]),
Some("check") => dir_arg(&args[1..]).and_then(|d| check(&d)),
Some("check") => dir_arg(&args[1..]).and_then(|d| check_path(&d)),
Some("link") => dir_arg(&args[1..]).and_then(|d| link(&d, true)),
Some("unlink") => dir_arg(&args[1..]).and_then(|d| link(&d, false)),
Some("links") => list_links(),
Some("keygen") => cmd_keygen(&args[1..]),
Some("endorse") => cmd_endorse(&args[1..]),
Some("pack") => cmd_pack(&args[1..]),
Some("index") => cmd_index(&args[1..]),
Some("merge") => cmd_merge(&args[1..]),
Some("--version" | "-V") => {
println!("openidea-plugin {} (plugin API {})", env!("CARGO_PKG_VERSION"), ide_extension::ENGINE_VERSION);
Ok(())
@@ -161,7 +179,7 @@ fn cmd_new(args: &[String]) -> Result<(), String> {
std::fs::create_dir_all(&dir).map_err(|e| format!("{}: {e}", dir.display()))?;
let dir = canonical(&dir)?;
let sdk = match template {
templates::Template::Command => Some(sdk_path(a.value("sdk"), &dir)?),
templates::Template::Command => Some(sdk_dependency(a.value("sdk"), &dir)?),
_ => None,
};
let vars = templates::Vars { id: &id, name: &name, sdk: sdk.as_deref() };
@@ -216,12 +234,16 @@ fn title_case(s: &str) -> String {
.join(" ")
}
/// The SDK dependency path written into a new plugin's Cargo.toml: `--sdk`,
/// or the SDK of the source tree this tool was built from.
fn sdk_path(explicit: Option<&str>, plugin_dir: &Path) -> Result<String, String> {
/// The SDK dependency written into a new plugin's Cargo.toml: by path with `--sdk`, or
/// when the plugin is inside the source tree this tool was built from; otherwise the
/// repository at this tool's release tag.
fn sdk_dependency(explicit: Option<&str>, plugin_dir: &Path) -> Result<String, String> {
let sdk = match explicit {
Some(p) => PathBuf::from(p),
None => PathBuf::from(concat!(env!("CARGO_MANIFEST_DIR"), "/../../sdk/rust/openidea-plugin")),
None => match dunce::canonicalize(concat!(env!("CARGO_MANIFEST_DIR"), "/../..")) {
Ok(tree) if plugin_dir.starts_with(&tree) => tree.join("sdk/rust/openidea-plugin"),
_ => return Ok(format!("{{ git = \"{REPOSITORY}\", tag = \"v{}\" }}", env!("CARGO_PKG_VERSION"))),
},
};
let sdk = dunce::canonicalize(&sdk).map_err(|_| {
format!("the plugin SDK was not found at {}; pass --sdk <path to sdk/rust/openidea-plugin>", sdk.display())
@@ -230,9 +252,11 @@ fn sdk_path(explicit: Option<&str>, plugin_dir: &Path) -> Result<String, String>
return Err(format!("{} is not the openidea-plugin SDK (no Cargo.toml)", sdk.display()));
}
let path = relative(plugin_dir, &sdk).unwrap_or(sdk);
Ok(path.to_string_lossy().replace('\\', "/"))
Ok(format!("{{ path = \"{}\" }}", path.to_string_lossy().replace('\\', "/")))
}
const REPOSITORY: &str = "https://github.com/l0ng-ai/openidea";
/// `to` relative to `from` when they share a root (keeps packages in a repo movable).
fn relative(from: &Path, to: &Path) -> Option<PathBuf> {
let (f, t): (Vec<_>, Vec<_>) = (from.components().collect(), to.components().collect());
@@ -323,10 +347,21 @@ fn ensure_wasm_target() {
/// Validates `dir` exactly as Install from Disk would, and checks the
/// component imports only the plugin API.
/// [`check`] of a folder, or of a package file unpacked as installing would.
fn check_path(path: &Path) -> Result<(), String> {
if !path.is_file() {
return check(path);
}
let tmp = scratch_dir("unpack")?;
let dir = tmp.path().join("package");
package::unpack(path, &dir).and_then(|_| check(&dir))
}
fn check(dir: &Path) -> Result<(), String> {
let dir = canonical(dir)?;
let scratch = std::env::temp_dir().join(format!("openidea-plugin-check-{}", std::process::id()));
let result = (|| {
let tmp = scratch_dir("check")?;
let scratch = tmp.path().join("host");
(|| {
let host = PluginHost::new(scratch.clone(), Arc::new(|_: Option<&str>, _| {}), Limits::default())
.map_err(|e| e.to_string())?;
let preview = host.preview_install(&dir).map_err(|e| e.to_string())?;
@@ -345,6 +380,17 @@ fn check(dir: &Path) -> Result<(), String> {
} else {
println!(" code: none (contributions only)");
}
println!(" categories: {}", manifest.categories().join(", "));
for c in manifest.unknown_categories() {
println!(" warning: unknown category {c:?} (ignored); use {}", ide_extension::CATEGORIES.join(", "));
}
if manifest.declared_categories.len() > ide_extension::MAX_CATEGORIES {
println!(
" warning: only the first {} known categories are used; {} are declared",
ide_extension::MAX_CATEGORIES,
manifest.declared_categories.len()
);
}
if !manifest.activation.is_empty() {
let events: Vec<String> = manifest.activation.iter().map(ToString::to_string).collect();
println!(" activation: {}", events.join(", "));
@@ -361,9 +407,7 @@ fn check(dir: &Path) -> Result<(), String> {
}
println!("ok");
Ok(())
})();
let _ = std::fs::remove_dir_all(&scratch);
result
})()
}
fn human_size(bytes: u64) -> String {
@@ -479,14 +523,31 @@ fn cmd_keygen(args: &[String]) -> Result<(), String> {
Ok(())
}
fn cmd_endorse(args: &[String]) -> Result<(), String> {
let a = Args::parse(args, &[], &["key", "publisher"])?;
let usage = "usage: openidea-plugin endorse <new-public-key> --key <earlier-key-file> --publisher ID";
let ([new], Some(key), Some(publisher)) = (a.positional.as_slice(), a.value("key"), a.value("publisher")) else {
return Err(usage.into());
};
if !package::publisher_id_ok(publisher) {
return Err(format!("invalid publisher id {publisher:?}"));
}
let endorsement = read_key(key)?.endorse(publisher, new.trim());
println!("{}", serde_json::to_string_pretty(&endorsement).map_err(|e| e.to_string())?);
Ok(())
}
/// A fresh scratch directory, removed when dropped.
fn scratch_dir(what: &str) -> Result<tempfile::TempDir, String> {
tempfile::Builder::new().prefix(&format!("openidea-plugin-{what}-")).tempdir().map_err(|e| e.to_string())
}
/// A scratch host to list a package's files as the IDE would.
fn with_scratch_host<T>(f: impl FnOnce(&PluginHost) -> Result<T, String>) -> Result<T, String> {
let scratch = std::env::temp_dir().join(format!("openidea-plugin-pack-{}", std::process::id()));
let result = PluginHost::new(scratch.clone(), Arc::new(|_: Option<&str>, _| {}), Limits::default())
let tmp = scratch_dir("host")?;
PluginHost::new(tmp.path().join("host"), Arc::new(|_: Option<&str>, _| {}), Limits::default())
.map_err(|e| e.to_string())
.and_then(|h| f(&h));
let _ = std::fs::remove_dir_all(&scratch);
result
.and_then(|h| f(&h))
}
fn cmd_pack(args: &[String]) -> Result<(), String> {
@@ -522,29 +583,28 @@ fn cmd_pack(args: &[String]) -> Result<(), String> {
}
fn cmd_index(args: &[String]) -> Result<(), String> {
let a = Args::parse(args, &[], &["publisher", "name", "key", "out"])?;
let a = Args::parse(args, &[], &["publisher", "name", "key", "previous-key", "out", "download-base"])?;
let dir = canonical(&a.dir()?)?;
let publisher = a.value("publisher").ok_or("--publisher is required")?;
let name = a.value("name").unwrap_or(publisher);
let key = read_key(a.value("key").ok_or("--key is required")?)?;
let out = a.value("out").map(PathBuf::from).unwrap_or_else(|| dir.join("index.json"));
let base = a.value("download-base").map(|b| b.trim_end_matches('/'));
// Revocations and other publishers of an existing index are kept.
let mut index = match std::fs::read_to_string(&out) {
Ok(text) => Index::parse(&text)?,
Ok(text) => current_format(Index::parse(&text)?, &out.to_string_lossy())?,
Err(_) => Index { format: package::INDEX_FORMAT, ..Index::default() },
};
// Earlier keys of the publisher stay listed (packages they signed remain valid until revoked).
let current = key.public_base64();
let earlier: Vec<KeyEntry> = index
.publishers
.iter()
.filter(|p| p.id == publisher)
.flat_map(|p| p.keys.iter().filter(|k| k.key != current).cloned())
.collect();
index.publishers.retain(|p| p.id != publisher);
let keys: Vec<KeyEntry> = std::iter::once(KeyEntry { key: current, revoked: false }).chain(earlier).collect();
index.publishers.push(package::Publisher { id: publisher.into(), name: name.into(), keys });
index.plugins.clear();
let publisher = match (a.value("publisher"), a.value("key")) {
(Some(publisher), Some(key)) => {
let key = read_key(key)?;
let name = a.value("name").unwrap_or(publisher);
let previous = a.value("previous-key").map(read_key).transpose()?;
index.publishers =
register_key(std::mem::take(&mut index.publishers), publisher, name, &key, previous.as_ref())?;
Some(publisher)
}
(None, None) if a.has("previous-key") => return Err("--previous-key goes with --publisher and --key".into()),
(None, None) => None,
_ => return Err("--publisher and --key go together".into()),
};
let mut packages: Vec<PathBuf> = std::fs::read_dir(&dir)
.map_err(|e| e.to_string())?
.flatten()
@@ -552,35 +612,144 @@ fn cmd_index(args: &[String]) -> Result<(), String> {
.filter(|p| p.extension().is_some_and(|e| e == package::PACKAGE_EXTENSION))
.collect();
packages.sort();
let engine = PluginEngine::new()?;
let mut added: Vec<IndexEntry> = Vec::new();
for path in packages {
let scratch = std::env::temp_dir().join(format!("openidea-plugin-index-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&scratch);
let unpacked = package::unpack(&path, &scratch)?;
let tmp = scratch_dir("index")?;
let scratch = tmp.path().join("package");
let entry = (|| {
let unpacked = package::unpack(&path, &scratch)?;
let m = Manifest::load(&scratch)?;
let files = with_scratch_host(|h| h.package_files_of(&scratch).map_err(|e| e.to_string()))?;
let digest = package::package_digest(&scratch, &files).map_err(|e| e.to_string())?;
let sig = unpacked.signature.as_ref().ok_or_else(|| format!("{}: not signed", path.display()))?;
sig.check(&m.id, &m.version, &digest).map_err(|e| format!("{}: {e}", path.display()))?;
let sig = unpacked.signature.as_ref().ok_or("not signed")?;
let signer = index.verify(sig, &m.id, &m.version, &digest)?;
if publisher.is_some_and(|p| p != signer) {
return Err(format!("signed by {signer}, not {}", publisher.unwrap_or_default()));
}
if added.iter().any(|e| (&e.id, &e.version) == (&m.id, &m.version)) {
return Err(format!("another package of {} {} is in the folder", m.id, m.version));
}
if m.legacy {
return Err("plugin.toml uses the 0.1 format; see plugins/README.md".into());
}
let built_for = match &m.wasm {
Some(wasm) => engine.check_component(&engine.compile(&scratch.join(wasm))?)?,
None => None,
};
let file = path.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
Ok::<_, String>(IndexEntry {
id: m.id.clone(),
name: m.name.clone(),
version: m.version.clone(),
description: m.description.clone(),
authors: m.authors.clone(),
publisher: sig.publisher.clone(),
download: path.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default(),
publisher: signer,
download: base.map_or_else(|| file.clone(), |b| format!("{b}/{file}")),
digest,
capabilities: m.capabilities.all().map(|c| c.to_string()).collect(),
engine: package::engine_requirement(&m.engine, built_for.as_ref()),
categories: m.categories(),
})
})();
let _ = std::fs::remove_dir_all(&scratch);
let entry = entry?;
println!(" {} {}", entry.id, entry.version);
index.plugins.push(entry);
let entry = entry.map_err(|e| format!("{}: {e}", path.display()))?;
println!(" {} {} (plugin API {})", entry.id, entry.version, entry.engine);
added.push(entry);
}
let json = serde_json::to_string_pretty(&index).map_err(|e| e.to_string())?;
std::fs::write(&out, json).map_err(|e| format!("{}: {e}", out.display()))?;
// A publisher's earlier versions of these plugins stay for IDEs that can't load the new
// ones (pruned below); plugins no longer in the folder go.
index.plugins.retain(|e| match publisher {
Some(p) => {
e.publisher != p
|| (added.iter().any(|n| n.id == e.id)
&& !added.iter().any(|n| (&n.id, &n.version) == (&e.id, &e.version)))
}
None => false,
});
index.plugins.extend(added);
index.prune_versions();
write_index(&index, &out)
}
/// `publishers` with `key` as `publisher`'s current key. Earlier keys stay listed
/// (packages they signed remain valid until revoked); a key new to the
/// publisher is endorsed by `previous`, one of them, so installed plugins
/// signed with it update without asking.
fn register_key(
mut publishers: Vec<package::Publisher>,
publisher: &str,
name: &str,
key: &PublisherKey,
previous: Option<&PublisherKey>,
) -> Result<Vec<package::Publisher>, String> {
let current = key.public_base64();
let earlier: Vec<KeyEntry> = publishers.iter().filter(|p| p.id == publisher).flat_map(|p| p.keys.clone()).collect();
let entry = match earlier.iter().find(|k| k.key == current) {
Some(listed) => listed.clone(),
None => {
let endorsement = match previous {
Some(prev) => {
let prev_key = prev.public_base64();
if !earlier.iter().any(|k| k.key == prev_key && !k.revoked) {
return Err(format!("--previous-key is not one of {publisher}'s valid keys in the index"));
}
Some(prev.endorse(publisher, &current))
}
None => {
if earlier.iter().any(|k| !k.revoked) {
println!(
" warning: {publisher}'s new key is not endorsed (--previous-key): users who installed \
its plugins will be asked to confirm the change when they update"
);
}
None
}
};
KeyEntry { endorsement, ..KeyEntry::new(current.clone()) }
}
};
let keys = std::iter::once(entry).chain(earlier.into_iter().filter(|k| k.key != current)).collect();
publishers.retain(|p| p.id != publisher);
publishers.push(package::Publisher { id: publisher.into(), name: name.into(), keys });
Ok(publishers)
}
/// `index`, unless a newer `openidea-plugin` wrote it (rewriting it would drop what this one can't read).
fn current_format(index: Index, file: &str) -> Result<Index, String> {
if index.is_newer() {
return Err(format!(
"{file}: index format {} is newer than this tool writes; update openidea-plugin",
index.format
));
}
Ok(index)
}
fn cmd_merge(args: &[String]) -> Result<(), String> {
let a = Args::parse(args, &[], &["keep", "out"])?;
let [base, other] = a.positional.as_slice() else {
return Err("usage: openidea-plugin merge <index> <other-index> --keep ID [--out FILE]".into());
};
let keep = a.value("keep").ok_or("--keep is required")?;
let out = a.value("out").unwrap_or(base);
let mut index =
current_format(Index::parse(&std::fs::read_to_string(base).map_err(|e| format!("{base}: {e}"))?)?, base)?;
let bytes = read_market_url(other, 16 << 20)?;
let theirs = Index::parse(&String::from_utf8(bytes).map_err(|_| format!("{other}: not UTF-8"))?)?;
let mut theirs = current_format(theirs, other)?;
// Their downloads are relative to their index, not to ours.
for e in &mut theirs.plugins {
e.download = resolve_market_href(other, &e.download);
}
let count = theirs.plugins.len();
index.include(theirs, &[keep]).map_err(|e| format!("{other}: {e}"))?;
println!(" {count} plugins from {other}");
write_index(&index, Path::new(out))
}
fn write_index(index: &Index, out: &Path) -> Result<(), String> {
let json = serde_json::to_string_pretty(index).map_err(|e| e.to_string())?;
std::fs::write(out, json).map_err(|e| format!("{}: {e}", out.display()))?;
println!("-> {} ({} plugins)", out.display(), index.plugins.len());
Ok(())
}
@@ -589,6 +758,25 @@ fn cmd_index(args: &[String]) -> Result<(), String> {
mod tests {
use super::*;
#[test]
fn a_new_key_is_endorsed_by_the_previous_one() {
let (k1, k2, k3) =
(PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([2; 32]), PublisherKey::from_seed([3; 32]));
let first = register_key(Vec::new(), "acme", "ACME", &k1, None).unwrap();
let rotated = register_key(first.clone(), "acme", "ACME", &k2, Some(&k1)).unwrap();
let index = Index { format: package::INDEX_FORMAT, publishers: rotated.clone(), ..Index::default() };
assert!(index.succeeds("acme", &k1.public_base64(), &k2.public_base64()));
assert_eq!(rotated[0].keys.len(), 2, "the earlier key stays listed");
// Registering the current key again keeps its endorsement.
let again = register_key(rotated.clone(), "acme", "ACME", &k2, None).unwrap();
assert_eq!(again, rotated);
// Only a key the publisher has can endorse.
assert!(register_key(first.clone(), "acme", "ACME", &k2, Some(&k3)).unwrap_err().contains("--previous-key"));
// Without one the new key is listed unendorsed.
let unendorsed = register_key(first, "acme", "ACME", &k3, None).unwrap();
assert!(unendorsed[0].keys[0].endorsement.is_none());
}
#[test]
fn names() {
assert_eq!(slugify("My Cool_Plugin"), "my-cool-plugin");
@@ -601,13 +789,189 @@ mod tests {
assert_eq!(relative(Path::new("/x/y"), Path::new("/z")), None);
}
#[test]
fn new_plugins_depend_on_the_sdk_by_path_only_in_its_source_tree() {
let tree = dunce::canonicalize(concat!(env!("CARGO_MANIFEST_DIR"), "/../..")).unwrap();
assert_eq!(
sdk_dependency(None, &tree.join("plugins/official/new")).unwrap(),
"{ path = \"../../../sdk/rust/openidea-plugin\" }"
);
let outside = dunce::canonicalize(std::env::temp_dir()).unwrap().join("my-plugin");
assert_eq!(
sdk_dependency(None, &outside).unwrap(),
format!("{{ git = \"https://github.com/l0ng-ai/openidea\", tag = \"v{}\" }}", env!("CARGO_PKG_VERSION"))
);
let sdk = tree.join("sdk/rust/openidea-plugin");
assert!(sdk_dependency(Some(sdk.to_str().unwrap()), &outside).unwrap().starts_with("{ path = "));
assert!(
sdk_dependency(Some(tree.join("wit").to_str().unwrap()), &outside)
.unwrap_err()
.contains("not the openidea-plugin SDK")
);
}
/// A theme package `id` signed by `publisher` with `key`, packed into `out`.
fn signed_theme(tmp: &Path, id: &str, publisher: &str, key: &str, out: &Path) {
signed_theme_version(tmp, id, "0.1.0", "^1.0", publisher, key, out);
}
fn signed_theme_version(tmp: &Path, id: &str, version: &str, engine: &str, publisher: &str, key: &str, out: &Path) {
let src = tmp.join("src").join(format!("{id}-{version}"));
let vars = templates::Vars { id, name: id, sdk: None };
for (rel, content) in templates::render(templates::Template::Theme, &vars) {
let path = src.join(rel);
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
let content = content
.replace("version = \"0.1.0\"", &format!("version = \"{version}\""))
.replace("engines.openidea = \"^1.0\"", &format!("engines.openidea = \"{engine}\""));
std::fs::write(path, content).unwrap();
}
std::fs::create_dir_all(out).unwrap();
let args = [src.to_str().unwrap(), "--key", key, "--publisher", publisher, "--out"]
.into_iter()
.map(String::from)
.chain([out.join(format!("{id}-{version}.oiplugin")).to_string_lossy().into_owned()])
.collect::<Vec<_>>();
cmd_pack(&args).unwrap();
}
#[test]
fn a_release_keeps_the_versions_older_ides_need() {
let tmp = tempfile::tempdir().unwrap();
let t = tmp.path();
let key = t.join("acme.key");
std::fs::write(&key, PublisherKey::from_seed([5; 32]).seed_base64()).unwrap();
let key = key.to_string_lossy().into_owned();
let index_file = t.join("index.json");
let release = |n: usize, packages: &[(&str, &str, &str)]| {
let dir = t.join(format!("release-{n}"));
for (id, version, engine) in packages {
signed_theme_version(t, id, version, engine, "acme", &key, &dir);
}
let base = format!("https://example.com/download/v{n}");
let args = [dir.to_str().unwrap(), "--publisher", "acme", "--key", &key, "--download-base", &base];
cmd_index(&strings(&[&args[..], &["--out", index_file.to_str().unwrap()]].concat())).unwrap();
let index = Index::parse(&std::fs::read_to_string(&index_file).unwrap()).unwrap();
index.plugins.iter().map(|e| format!("{} {} {}", e.id, e.version, e.download)).collect::<Vec<_>>()
};
release(1, &[("acme.a", "1.0.0", "^1.0"), ("acme.b", "1.0.0", "^1.0")]);
// Same requirement: the newer version replaces it; acme.b is gone from the release.
assert_eq!(
release(2, &[("acme.a", "1.1.0", "^1.0")]),
["acme.a 1.1.0 https://example.com/download/v2/acme.a-1.1.0.oiplugin"]
);
// Another API requirement: the last version for the earlier one stays, where it was published.
assert_eq!(
release(3, &[("acme.a", "2.0.0", ">=1.0, <2")]),
[
"acme.a 1.1.0 https://example.com/download/v2/acme.a-1.1.0.oiplugin",
"acme.a 2.0.0 https://example.com/download/v3/acme.a-2.0.0.oiplugin",
]
);
let index = Index::parse(&std::fs::read_to_string(&index_file).unwrap()).unwrap();
assert_eq!(index.offered()[0].0.version, "2.0.0", "the newest this IDE can load");
}
fn strings(args: &[&str]) -> Vec<String> {
args.iter().map(|a| a.to_string()).collect()
}
#[test]
fn official_and_community_indexes_merge() {
let tmp = tempfile::tempdir().unwrap();
let t = tmp.path();
let mut keys = Vec::new();
for (name, seed) in [("openidea", 1), ("alice", 2), ("carol", 3)] {
let file = t.join(format!("{name}.key"));
let key = PublisherKey::from_seed([seed; 32]);
std::fs::write(&file, key.seed_base64()).unwrap();
keys.push((file.to_string_lossy().into_owned(), key.public_base64()));
}
let (official, community) = (t.join("official"), t.join("community"));
signed_theme(t, "openidea.night", "openidea", &keys[0].0, &official);
signed_theme(t, "alice.dawn", "alice", &keys[1].0, &community);
let official_index = official.join("index.json");
let community_index = community.join("index.json");
let release = strings(&[official.to_str().unwrap(), "--publisher", "openidea", "--key", &keys[0].0]);
cmd_index(&release).unwrap();
// The community index lists its publishers; `index` without a key checks packages against them.
let skeleton = Index {
format: package::INDEX_FORMAT,
publishers: vec![package::Publisher {
id: "alice".into(),
name: "Alice".into(),
keys: vec![KeyEntry::new(keys[1].1.clone())],
}],
..Index::default()
};
write_index(&skeleton, &community_index).unwrap();
cmd_index(&strings(&[community.to_str().unwrap()])).unwrap();
cmd_merge(&strings(&[
official_index.to_str().unwrap(),
community_index.to_str().unwrap(),
"--keep",
"openidea",
]))
.unwrap();
// A release rebuilds the official plugins and keeps the community's.
cmd_index(&release).unwrap();
let index = Index::parse(&std::fs::read_to_string(&official_index).unwrap()).unwrap();
let listed: Vec<_> = index.plugins.iter().map(|e| (e.id.as_str(), e.publisher.as_str())).collect();
assert_eq!(listed, [("alice.dawn", "alice"), ("openidea.night", "openidea")]);
assert_eq!(index.plugins[1].categories, ["themes"], "implied by the theme");
assert_eq!(index.plugins[1].download, "openidea.night-0.1.0.oiplugin");
assert!(Path::new(&index.plugins[0].download).is_absolute(), "{}", index.plugins[0].download);
// A package by a publisher the index doesn't know is refused.
signed_theme(t, "carol.noon", "carol", &keys[2].0, &community);
let err = cmd_index(&strings(&[community.to_str().unwrap()])).unwrap_err();
assert!(err.contains("unknown publisher"), "{err}");
}
/// `cargo test` of a new Rust plugin (its own tests against the mock host) with this SDK.
#[test]
fn the_command_template_builds_against_the_sdk() {
let dir = std::env::temp_dir().join(format!("openidea-plugin-template-build-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let dir = dunce::canonicalize(&dir).unwrap();
let plugin = dir.join("plugin");
let sdk = concat!(env!("CARGO_MANIFEST_DIR"), "/../../sdk/rust/openidea-plugin");
let sdk = sdk_dependency(Some(sdk), &plugin).unwrap();
let vars = templates::Vars { id: "acme.sample", name: "Sample", sdk: Some(&sdk) };
for (rel, content) in templates::render(templates::Template::Command, &vars) {
let path = plugin.join(rel);
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
std::fs::write(path, content).unwrap();
}
// The SDK's lockfile keeps the versions its own CI builds with (the resolver also
// looks up the wasm-only dependencies, which may not be downloaded yet).
let sdk_lock = concat!(env!("CARGO_MANIFEST_DIR"), "/../../sdk/rust/openidea-plugin/Cargo.lock");
std::fs::copy(sdk_lock, plugin.join("Cargo.lock")).unwrap();
let out = Command::new(std::env::var("CARGO").unwrap_or_else(|_| "cargo".into()))
.args(["test", "--manifest-path"])
.arg(plugin.join("Cargo.toml"))
.env("CARGO_TARGET_DIR", dir.join("target"))
.output()
.unwrap();
assert!(
out.status.success(),
"{}{}",
String::from_utf8_lossy(&out.stdout),
String::from_utf8_lossy(&out.stderr)
);
std::fs::remove_dir_all(&dir).unwrap();
}
#[test]
fn every_template_is_a_valid_package() {
for t in templates::Template::ALL {
let dir =
std::env::temp_dir().join(format!("openidea-plugin-template-{}-{}", t.label(), std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
let vars = templates::Vars { id: "acme.sample", name: "Sample", sdk: Some("../sdk") };
let vars = templates::Vars { id: "acme.sample", name: "Sample", sdk: Some("{ path = \"../sdk\" }") };
for (rel, content) in templates::render(t, &vars) {
let path = dir.join(rel);
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
@@ -32,6 +32,7 @@ impl Template {
pub struct Vars<'a> {
pub id: &'a str,
pub name: &'a str,
/// The SDK dependency, as a TOML value (`{ path = "…" }`).
pub sdk: Option<&'a str>,
}
@@ -82,6 +83,9 @@ name = "__NAME__"
version = "0.1.0"
description = "Adds two sample commands."
authors = []
# Marketplace categories (up to 3): languages, themes, icon-themes, linters,
# vcs, ai, frameworks, tools, other.
categories = ["tools"]
engines.openidea = "^1.0"
# Built by `openidea-plugin build`. The code is loaded the first time one of
# its commands runs; the commands below show in Find Action before that.
@@ -115,7 +119,7 @@ publish = false
crate-type = ["cdylib", "rlib"]
[dependencies]
openidea-plugin = { path = "__SDK__" }
openidea-plugin = __SDK__
[profile.release]
opt-level = "s"
@@ -127,7 +131,7 @@ panic = "abort"
const COMMAND_LIB: &str = r#"//! __NAME__: an OpenIDEA plugin.
use openidea_plugin::{Invocation, NotificationLevel, Plugin, host, register_plugin};
use openidea_plugin::{Error, Invocation, NotificationLevel, Plugin, host, register_plugin};
#[derive(Default)]
pub struct Main {
@@ -135,7 +139,7 @@ pub struct Main {
}
impl Plugin for Main {
fn run_command(&mut self, id: &str, ctx: &Invocation, _arg: Option<&str>) -> Result<Option<String>, String> {
fn run_command(&mut self, id: &str, ctx: &Invocation, _arg: Option<&str>) -> Result<Option<String>, Error> {
match id {
"say-hello" => {
// Instance state lasts until the plugin is idle for a while;
@@ -150,7 +154,7 @@ impl Plugin for Main {
let text = editor.document()?.text;
Ok(Some(format!("{} lines in {}", text.lines().count(), editor.path())))
}
_ => Err(format!("unknown command {id}")),
_ => Err(format!("unknown command {id}").into()),
}
}
}
+124 -54
View File
@@ -25,7 +25,7 @@ use ide_language::{LanguageRegistry, Syntax};
use ide_run::ProcessManager;
use ide_terminal::TerminalManager;
use ide_workspace::SettingsStore;
use parking_lot::RwLock;
use parking_lot::{Mutex, RwLock};
use std::collections::HashMap;
use std::path::PathBuf;
use std::sync::{Arc, OnceLock};
@@ -74,15 +74,23 @@ pub struct Host {
pub environment: environment::Environment,
/// The HTTP proxy's saved password (see `proxy.rs`).
proxy_password: ide_ssh::secrets::SecretFile,
/// The proxy password when it could not be saved (no credential store).
proxy_session_password: Mutex<Option<String>>,
pub(crate) sink_for: SinkFactory,
owner: fn(&str) -> &str,
pub runtime: tokio::runtime::Handle,
/// One project per window, keyed by window label.
projects: RwLock<HashMap<String, Arc<Project>>>,
pub(crate) external_logs: RwLock<HashMap<String, Vec<Arc<ide_vcs::Repo>>>>,
requests: requests::Requests,
project_requests: RwLock<HashMap<String, Vec<RequestId>>>,
/// Shutdowns of projects closed with [`Host::close_project_detached`].
closing: parking_lot::Mutex<Vec<std::thread::JoinHandle<()>>>,
}
/// How long exit waits for project shutdown before giving up.
const EXIT_SHUTDOWN_WAIT: std::time::Duration = std::time::Duration::from_secs(10);
impl Host {
pub fn new(config: HostConfig) -> Self {
let HostConfig {
@@ -102,6 +110,7 @@ impl Host {
let host = Self {
trust: trust::TrustStore::load(&config_dir, &data_dir),
proxy_password: ide_ssh::secrets::SecretFile::new(data_dir.join("proxy-password.json")),
proxy_session_password: Mutex::default(),
remote: Arc::new(ide_remote::RemoteService::new(ssh.clone(), data_dir.join("remote"))),
ssh,
environment: environment::Environment::new(project_environment, local_data_dir.clone()),
@@ -122,8 +131,10 @@ impl Host {
owner,
runtime,
projects: RwLock::default(),
external_logs: RwLock::default(),
requests: Default::default(),
project_requests: RwLock::default(),
closing: Default::default(),
};
host.apply_proxy();
host
@@ -177,45 +188,69 @@ impl Host {
}
}
/// Closes `window`'s project and waits until its language servers, kernels and
/// debug sessions have shut down. Never call this on the UI event-loop thread:
/// the shutdown runs on the async runtime, whose workers may themselves be
/// waiting for the event loop (IPC responses, window queries). Use
/// [`Self::close_project_detached`] there.
pub fn close_project(&self, window: &str) {
let project = self.projects.write().remove(window);
if let Some(project) = project {
if !project.light {
self.environment.project_closed();
}
if let Some(bridge) = &project.kotlin_bridge {
bridge.stop();
}
project.documents.clear_workspace_edit_history();
project.agents.shutdown();
project.notebooks.stop_admission();
// What the closing window started goes with it — the builds and searches it
// asked for, its runs, its shells and its debug sessions — and the language
// servers finish shutting down before the window is gone, as IDEA closes a
// project's processes.
for request in self.project_requests.write().remove(self.owner_window(window)).unwrap_or_default() {
self.cancel_request(request);
}
self.processes.kill_project(&project.root);
self.terminals.kill_key(self.owner_window(window));
self.shutdown_project_workers_blocking(
vec![project.notebooks.clone()],
project.kotlin_bridge.iter().cloned().collect(),
Some(project.lsp.clone()),
Some((self.debugger.clone(), window.to_owned())),
);
if let Some(shutdown) = self.begin_close(window) {
self.wait_blocking(shutdown);
}
}
/// Synchronous project/window destruction must finish owned kernel cleanup,
/// not enqueue work which can be lost as the desktop runtime exits.
fn shutdown_project_workers_blocking(
/// Like [`Self::close_project`], but returns once the project is detached from
/// the window; its workers finish shutting down in the background, and
/// [`Self::shutdown_notebooks_on_exit`] waits for them before the process exits.
pub fn close_project_detached(&self, window: &str) {
if let Some(shutdown) = self.begin_close(window) {
let mut closing = self.closing.lock();
closing.retain(|handle| !handle.is_finished());
closing.push(shutdown);
}
}
/// Detaches the project from `window` and stops what it started, synchronously,
/// so the window can open another project at once. Returns the thread that
/// finishes shutting down its workers.
fn begin_close(&self, window: &str) -> Option<std::thread::JoinHandle<()>> {
self.external_logs.write().remove(self.owner_window(window));
for request in self.project_requests.write().remove(self.owner_window(window)).unwrap_or_default() {
self.cancel_request(request);
}
let project = self.projects.write().remove(window)?;
if !project.light {
self.environment.project_closed();
}
if let Some(bridge) = &project.kotlin_bridge {
bridge.stop();
}
project.documents.clear_workspace_edit_history();
project.agents.shutdown();
project.notebooks.stop_admission();
// What the closing window started goes with it — the builds and searches it
// asked for, its runs, its shells and its debug sessions — and its language
// servers shut down.
self.processes.kill_project(&project.root);
self.terminals.kill_key(self.owner_window(window));
Some(self.spawn_project_shutdown(
vec![project.notebooks.clone()],
project.kotlin_bridge.iter().cloned().collect(),
Some(project.lsp.clone()),
Some((self.debugger.clone(), window.to_owned())),
))
}
/// Project/window destruction must finish owned kernel cleanup, not enqueue
/// work which can be lost as the desktop runtime exits: the returned thread is
/// joined by the closer or, for a detached close, before exit.
fn spawn_project_shutdown(
&self,
notebooks: Vec<Arc<notebook::NotebookService>>,
bridges: Vec<Arc<kotlin_bridge::KotlinBridge>>,
lsp: Option<Arc<ide_lsp::LspManager>>,
debug: Option<(Arc<ide_dap::DebugManager>, String)>,
) {
) -> std::thread::JoinHandle<()> {
for bridge in &bridges {
bridge.stop();
}
@@ -223,25 +258,37 @@ impl Host {
service.stop_admission();
}
let runtime = self.runtime.clone();
let wait = move || {
let worker = std::thread::spawn(move || {
runtime.block_on(async move {
for bridge in bridges {
bridge.shutdown().await;
}
for service in notebooks {
service.shutdown().await;
}
if let Some(lsp) = lsp {
lsp.shutdown_all().await;
}
if let Some((debug, key)) = debug {
debug.stop_key(&key).await;
}
});
std::thread::spawn(move || {
runtime.block_on(async move {
for bridge in bridges {
bridge.shutdown().await;
}
for service in notebooks {
service.shutdown().await;
}
if let Some(lsp) = lsp {
lsp.shutdown_all().await;
}
if let Some((debug, key)) = debug {
debug.stop_key(&key).await;
}
});
if let Err(error) = worker.join() {
tracing::error!("notebook shutdown thread failed: {error:?}");
})
}
/// Waits for detached closes, so a project opened next never races the
/// language servers of one still shutting down (e.g. over a shared workspace).
pub(crate) fn wait_detached_closes(&self) {
let pending: Vec<_> = self.closing.lock().drain(..).collect();
for shutdown in pending {
self.wait_blocking(shutdown);
}
}
fn wait_blocking(&self, shutdown: std::thread::JoinHandle<()>) {
let wait = move || {
if let Err(error) = shutdown.join() {
tracing::error!("project shutdown thread failed: {error:?}");
}
};
if tokio::runtime::Handle::try_current()
@@ -253,15 +300,32 @@ impl Host {
}
}
/// Called before the desktop runtime exits, including update/restart exits.
/// Called before the desktop runtime exits, including update/restart exits, on
/// the event-loop thread: waits for detached closes and the open projects'
/// kernels, but no longer than [`EXIT_SHUTDOWN_WAIT`], so a runtime worker
/// stuck on the event loop cannot keep the process from exiting.
pub fn shutdown_notebooks_on_exit(&self) {
let projects = self.projects();
self.shutdown_project_workers_blocking(
let mut pending: Vec<_> = self.closing.lock().drain(..).collect();
pending.push(self.spawn_project_shutdown(
projects.iter().map(|project| project.notebooks.clone()).collect(),
projects.iter().filter_map(|project| project.kotlin_bridge.clone()).collect(),
None,
None,
);
));
let deadline = std::time::Instant::now() + EXIT_SHUTDOWN_WAIT;
while pending.iter().any(|handle| !handle.is_finished()) {
if std::time::Instant::now() >= deadline {
tracing::warn!("project shutdown did not finish within {EXIT_SHUTDOWN_WAIT:?}; exiting anyway");
return;
}
std::thread::sleep(std::time::Duration::from_millis(20));
}
for handle in pending {
if let Err(error) = handle.join() {
tracing::error!("project shutdown thread failed: {error:?}");
}
}
}
/// Called before desktop exit, which may bypass document destructors.
@@ -311,14 +375,20 @@ impl Host {
}
pub fn begin_project_request(&self, window: &str, id: RequestId) -> CancellationToken {
let token = self.begin_request(id);
self.project_requests.write().entry(self.owner_window(window).to_owned()).or_default().push(id);
let token = self.begin_project_read_request(window, id);
if self.project(window).is_ok_and(|p| !p.is_trusted()) {
token.cancel();
}
token
}
/// Read-only work remains cancellable/disposed with its window in restricted projects.
pub fn begin_project_read_request(&self, window: &str, id: RequestId) -> CancellationToken {
let token = self.begin_request(id);
self.project_requests.write().entry(self.owner_window(window).to_owned()).or_default().push(id);
token
}
pub fn end_request(&self, id: RequestId) {
for ids in self.project_requests.write().values_mut() {
ids.retain(|entry| *entry != id);
+97 -4
View File
@@ -8,8 +8,11 @@ use ide_agent::{
AgentChange, AgentCommand, AgentEntry, AgentEvent, AgentProbe, AgentPrompt, AgentProvider, AgentQueued, AgentTask,
NewAgentTask,
};
use ide_core::{IdeError, IdeResult, IpcPath, RequestId};
use ide_core::error::ErrorKind;
use ide_core::{IdeError, IdeEvent, IdeResult, IpcPath, RequestId, SharedSink, TaskId};
use ide_tools::InstallEvent;
use std::collections::HashSet;
use std::sync::{LazyLock, Mutex};
/// Streams the project's agent events to this window; returns every task.
pub fn subscribe(host: &Host, window: &str, on_event: Emit<AgentEvent>) -> IdeResult<Vec<AgentTask>> {
@@ -30,6 +33,17 @@ pub fn set_command(host: &Host, window: &str, id: String, command: Option<String
host.project(window)?.agents.set_builtin_command(&id, command)
}
/// Environment variables built-in agent `id` starts with; empty clears them.
pub fn set_env(
host: &Host,
window: &str,
id: String,
env: std::collections::BTreeMap<String, String>,
) -> IdeResult<()> {
host.require_trust(window)?;
host.project(window)?.agents.set_builtin_env(&id, env)
}
/// Where ACP Registry agents are saved (`<config>/agents`, like the user's own) and installed.
pub(crate) fn registry_paths(host: &Host) -> RegistryPaths {
RegistryPaths::new(&host.config_dir, &host.local_data_dir)
@@ -47,7 +61,39 @@ pub async fn registry_list(host: &Host, _window: &str, request: RequestId) -> Id
let result =
tokio::task::spawn_blocking(move || ide_agent::registry::list(&paths, &cancel)).await.map_err(join_err);
host.end_request(request);
result?
let mut registry = result??;
let installing = installing();
for agent in &mut registry.agents {
agent.installing = installing.contains(&agent.id);
}
Ok(registry)
}
/// Registry installs in progress, by agent id. An install outlives the Settings page
/// that started it, so a second one of the same agent is refused rather than racing it
/// for the same folder.
static INSTALLING: LazyLock<Mutex<HashSet<String>>> = LazyLock::new(Mutex::default);
fn installing() -> std::sync::MutexGuard<'static, HashSet<String>> {
INSTALLING.lock().unwrap_or_else(|e| e.into_inner())
}
/// Marks agent `id` as installing until dropped.
struct Installing(String);
impl Installing {
fn begin(id: &str) -> IdeResult<Self> {
if !installing().insert(id.to_owned()) {
return Err(IdeError::new(ErrorKind::AlreadyExists, format!("{id} is already being installed")));
}
Ok(Self(id.to_owned()))
}
}
impl Drop for Installing {
fn drop(&mut self) {
installing().remove(&self.0);
}
}
/// Installs the registry's current version of agent `id` (also to update it), streaming
@@ -61,11 +107,43 @@ pub async fn registry_install(
on_event: Channel<InstallEvent>,
) -> IdeResult<()> {
let paths = registry_paths(host);
let installing = Installing::begin(&id)?;
let cancel = host.begin_request(request);
// Also a background task of every project window, so it stays visible and can be
// stopped there once the Settings page is closed.
let task = TaskId::next();
let title = format!("Installing {id}");
let sinks: Vec<SharedSink> = host.projects().iter().map(|p| p.sink.clone()).collect();
let report = move |message: Option<String>, fraction: Option<f32>, done: bool| {
for sink in &sinks {
sink.emit(IdeEvent::Task {
id: task,
title: title.clone(),
message: message.clone(),
fraction,
done,
cancellable: !done,
});
}
};
let result = tokio::task::spawn_blocking(move || {
ide_agent::registry::install(&paths, &id, &cancel, &|e| {
let _installing = installing;
let _cancel = ide_core::task::on_cancel(task, {
let cancel = cancel.clone();
move || cancel.cancel()
});
report(None, None, false);
let result = ide_agent::registry::install(&paths, &id, &cancel, &|e| {
match &e {
InstallEvent::Line { text } => report(Some(text.clone()), None, false),
InstallEvent::Progress { fraction, message } => {
report(Some(message.clone()), fraction.map(|f| f as f32), false)
}
}
let _ = on_event.send(e);
})
});
report(None, Some(1.0), true);
result
})
.await
.map_err(join_err);
@@ -195,6 +273,7 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("agent_subscribe", |c| async move { subscribe(&c.host, &c.window, c.channel("onEvent")?) });
d.add("agent_providers", |c| async move { providers(&c.host, &c.window) });
d.add("agent_set_command", |c| async move { set_command(&c.host, &c.window, c.arg("id")?, c.arg("command")?) });
d.add("agent_set_env", |c| async move { set_env(&c.host, &c.window, c.arg("id")?, c.arg("env")?) });
d.add("agent_registry_list", |c| async move { registry_list(&c.host, &c.window, c.arg("request")?).await });
d.add("agent_registry_install", |c| async move {
registry_install(&c.host, &c.window, c.arg("id")?, c.arg("request")?, c.sender("onEvent")?).await
@@ -251,6 +330,20 @@ fn validate_project_root(root: &std::path::Path, requested: Option<&std::path::P
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn an_agent_installs_once_at_a_time() {
let first = Installing::begin("test-agent").unwrap();
assert!(installing().contains("test-agent"));
let again = Installing::begin("test-agent").err().unwrap();
assert_eq!(again.kind, ErrorKind::AlreadyExists);
let other = Installing::begin("test-other-agent").unwrap();
drop(first);
assert!(!installing().contains("test-agent"));
drop(Installing::begin("test-agent").unwrap());
drop(other);
}
#[test]
fn agent_roots_are_exact_window_or_workspace_members() {
let tmp = tempfile::tempdir().unwrap();
+252 -13
View File
@@ -32,6 +32,7 @@ pub fn load(root: &Path, cache: &Path, settings: &Settings) -> BuildModel {
cache,
ide_sdk::apply::maven_local_repository(root, settings),
&settings.languages.java.maven,
&settings.languages.java.gradle,
);
let jdk = ide_sdk::apply::project_jdk(root, settings);
if let Some(m) = &mut model.maven {
@@ -45,7 +46,7 @@ pub fn load(root: &Path, cache: &Path, settings: &Settings) -> BuildModel {
p.options = options.clone();
}
}
if let Some(g) = &mut model.gradle {
for g in &mut model.gradle {
g.env = ide_sdk::apply::gradle_env(root, settings, java_env(jdk.as_ref(), g.java_version));
g.command = ide_sdk::apply::gradle_command(root, &root.join(&g.dir), settings).to_string_lossy().into_owned();
}
@@ -65,11 +66,12 @@ pub fn load(root: &Path, cache: &Path, settings: &Settings) -> BuildModel {
let hint = m.projects.iter().find_map(|p| missing(p.java_version));
explain(&mut m.sync_error, hint);
}
if let Some(g) = &mut model.gradle
&& g.sync_error.is_some()
{
let hint = if settings.languages.java.gradle.jvm.trim().is_empty() { missing(g.java_version) } else { None };
explain(&mut g.sync_error, hint);
for g in &mut model.gradle {
if g.sync_error.is_some() {
let hint =
if settings.languages.java.gradle.jvm.trim().is_empty() { missing(g.java_version) } else { None };
explain(&mut g.sync_error, hint);
}
}
model
}
@@ -94,9 +96,88 @@ pub async fn build_model(host: &Host, window: &str) -> IdeResult<BuildModel> {
tokio::task::spawn_blocking(move || load(&root, &cache, &settings)).await.map_err(join_err)
}
fn linked_gradle_paths(
root: &Path,
settings: &ide_core::settings::GradleSettings,
paths: &[String],
link: bool,
) -> IdeResult<Vec<String>> {
let existing = settings
.linked_projects
.clone()
.unwrap_or_else(|| ide_build::gradle::load_linked(root, settings).into_iter().map(|g| g.dir).collect());
let mut linked = Vec::new();
for path in existing {
let normalized = ide_build::gradle::normalize_linked_projects(root, std::slice::from_ref(&path))
.unwrap_or_else(|_| vec![path]);
for path in normalized {
if !linked.contains(&path) {
linked.push(path);
}
}
}
if link {
let directories: Vec<String> = paths
.iter()
.map(|path| {
let path = root.join(path);
let is_script = path.file_name().and_then(|n| n.to_str()).is_some_and(|name| {
ide_build::gradle::SETTINGS_FILES.contains(&name) || ide_build::gradle::BUILD_FILES.contains(&name)
});
if is_script && path.is_file() {
path.parent().unwrap_or(root).to_string_lossy().into_owned()
} else {
path.to_string_lossy().into_owned()
}
})
.collect();
for path in ide_build::gradle::normalize_linked_projects(root, &directories)? {
if !linked.contains(&path) {
linked.push(path);
}
}
} else {
linked.retain(|path| !paths.contains(path));
}
Ok(linked)
}
async fn set_gradle_projects(host: &Host, window: &str, paths: Vec<String>, link: bool) -> IdeResult<BuildModel> {
host.require_trust(window)?;
let project = host.project(window)?;
tokio::task::spawn_blocking(move || {
if ide_workspace::workspace::Workspace::load(&project.root)?.is_some() {
return Err(IdeError::invalid("Open a workspace member as a project to link Gradle builds"));
}
let linked =
linked_gradle_paths(&project.root, &project.settings.effective().languages.java.gradle, &paths, link)?;
let key = "languages.java.gradle.linkedProjects";
let settings =
project.settings.set(ide_workspace::settings::SettingsScope::Project, key, serde_json::json!(linked))?;
project.refresh_index_scope();
project.lsp.reconfigure_soon();
project.sink.emit(IdeEvent::SettingsChanged { key: key.into() });
project.sink.emit(IdeEvent::ProjectRootsChanged { root: project.root.clone().into() });
Ok::<BuildModel, IdeError>(load(&project.root, &project.build_cache_dir(), &settings))
})
.await
.map_err(join_err)?
}
/// Links independent builds without changing their Gradle scripts.
pub async fn gradle_link_projects(host: &Host, window: &str, paths: Vec<String>) -> IdeResult<BuildModel> {
set_gradle_projects(host, window, paths, true).await
}
/// Stops importing the selected independent builds; files and saved run configurations stay intact.
pub async fn gradle_unlink_projects(host: &Host, window: &str, paths: Vec<String>) -> IdeResult<BuildModel> {
set_gradle_projects(host, window, paths, false).await
}
/// Runs Maven (`help:effective-pom dependency:tree`) or Gradle (`tasks --all`) in the
/// background, reporting progress on `on_progress`; cancellable with `cancel_request(request)`.
/// Returns the refreshed model; a failed build is reported in its `syncError`.
#[allow(clippy::too_many_arguments)]
pub async fn build_sync(
host: &Host,
window: &str,
@@ -104,6 +185,7 @@ pub async fn build_sync(
request: RequestId,
offline: bool,
profiles: Vec<String>,
build_dir: Option<String>,
on_progress: Channel<SyncProgress>,
) -> IdeResult<BuildModel> {
host.require_trust(window)?;
@@ -116,7 +198,14 @@ pub async fn build_sync(
tracing::warn!("Maven profiles not saved: {e}");
false
});
if profiles_changed {
// The modules of the chosen profiles are read from the POMs with them from now on (the
// project tree's module folders too), whether or not Maven itself can run.
project.sink.emit(ide_core::IdeEvent::ProjectRootsChanged { root: project.root.clone().into() });
}
let token = host.begin_project_request(window, request);
let completion_token = token.clone();
let original_links = settings.languages.java.gradle.linked_projects.clone();
let result = tokio::task::spawn_blocking(move || {
let current = load(&root, &cache, &settings);
let env = match tool {
@@ -126,7 +215,7 @@ pub async fn build_sync(
.and_then(|m| m.projects.iter().find(|p| !p.env.is_empty()))
.map(|p| p.env.clone())
.unwrap_or_default(),
BuildTool::Gradle => current.gradle.as_ref().map(|g| g.env.clone()).unwrap_or_default(),
BuildTool::Gradle => BTreeMap::new(),
};
let options = SyncOptions {
offline,
@@ -135,14 +224,31 @@ pub async fn build_sync(
maven_command: ide_sdk::apply::maven_command(&root, &settings).map(|p| p.to_string_lossy().into_owned()),
maven_options: ide_sdk::apply::maven_options(&root, &settings),
maven_settings: settings.languages.java.maven.clone(),
gradle_command: current.gradle.as_ref().map(|g| g.command.clone()),
gradle_command: None,
};
let progress = |p: SyncProgress| {
let _ = on_progress.send(p);
};
match tool {
BuildTool::Maven => ide_build::sync_maven(&root, &cache, &options, &token, &progress)?,
BuildTool::Gradle => ide_build::sync_gradle(&root, &cache, &options, &token, &progress)?,
BuildTool::Gradle => {
if let Some(dir) = &build_dir
&& !current.gradle.iter().any(|g| &g.dir == dir)
{
return Err(IdeError::not_found("Gradle project is no longer linked"));
}
for g in current.gradle.iter().filter(|g| build_dir.as_ref().is_none_or(|dir| dir == &g.dir)) {
if token.is_cancelled() {
return Err(IdeError::cancelled());
}
let options =
SyncOptions { env: g.env.clone(), gradle_command: Some(g.command.clone()), ..options.clone() };
ide_build::sync_gradle_model(&root, &cache, g, &options, &token, &progress)?;
}
if token.is_cancelled() {
return Err(IdeError::cancelled());
}
}
}
Ok::<BuildModel, IdeError>(load(&root, &cache, &settings))
})
@@ -150,6 +256,18 @@ pub async fn build_sync(
.map_err(join_err);
host.end_request(request);
let model = result??;
if completion_token.is_cancelled() {
return Err(IdeError::cancelled());
}
if tool == BuildTool::Gradle && original_links != project.settings.effective().languages.java.gradle.linked_projects
{
// A late sync must not re-import a build that was unlinked while it ran.
return tokio::task::spawn_blocking(move || {
load(&project.root, &project.build_cache_dir(), &project.settings.effective())
})
.await
.map_err(join_err);
}
if tool == BuildTool::Gradle {
// The sync may report project folders outside the project (`includeFlat`): index and list them.
project.refresh_index_scope();
@@ -217,6 +335,58 @@ fn declaring_pom(
)))
}
/// The `.toml` of a version catalog the build imports as `group:artifact:version`, once a
/// build has downloaded it (`ide_sdk::libraries::published_version_catalog`).
pub async fn gradle_published_catalog(
host: &Host,
window: &str,
coordinates: String,
) -> IdeResult<Option<ide_core::IpcPath>> {
let project = host.project(window)?;
tokio::task::spawn_blocking(move || {
ide_sdk::libraries::published_version_catalog(&project.root, &project.settings.effective(), &coordinates)
.map(Into::into)
})
.await
.map_err(join_err)
}
/// The resolved dependency tree of the Gradle project `gradle_path` (`:`, `:app`) for
/// `configuration`, from Gradle's `dependencies` report; cancellable with `cancel_request(request)`.
pub async fn gradle_dependencies(
host: &Host,
window: &str,
gradle_path: String,
configuration: String,
offline: bool,
request: RequestId,
build_dir: Option<String>,
) -> IdeResult<Vec<ide_build::GradleDependency>> {
host.require_trust(window)?;
let project = host.project(window)?;
let (root, cache, settings) = (project.root.clone(), project.build_cache_dir(), project.settings.effective());
let token = host.begin_project_request(window, request);
let result = tokio::task::spawn_blocking(move || {
let model = load(&root, &cache, &settings);
let gradle = model
.gradle
.iter()
.find(|g| build_dir.as_ref().is_none_or(|dir| &g.dir == dir))
.ok_or_else(|| IdeError::not_found("Gradle project is no longer linked"))?;
let options = SyncOptions {
offline,
env: gradle.env.clone(),
gradle_command: Some(gradle.command.clone()),
..Default::default()
};
ide_build::gradle_dependencies(&root, gradle, &options, &gradle_path, &configuration, &token)
})
.await
.map_err(join_err);
host.end_request(request);
result?
}
/// Where the direct dependency `dependency` of the Maven module `pom` is declared.
pub async fn maven_dependency_declaration(
host: &Host,
@@ -296,7 +466,7 @@ pub async fn build_run(
let settings = ide_workspace::SettingsStore::load(&config_dir, Some(member)).effective();
let member_cache = cache.join(crate::project::project_key(member));
let model = with_profiles(load(member, &member_cache, &settings), &member_cache);
let plan = compile::plan(member, &model, kind, file.as_deref())?;
let plan = compile::plan(member, &model, kind, file.as_deref(), settings.cmake.profiles.as_deref())?;
match &mut combined {
Some(combined) => combined.steps.extend(plan.steps),
None => combined = Some(plan),
@@ -306,7 +476,13 @@ pub async fn build_run(
plan.title = workspace.name;
plan
} else {
compile::plan(&root, &with_profiles(load(&root, &cache, &settings), &cache), kind, file.as_deref())?
compile::plan(
&root,
&with_profiles(load(&root, &cache, &settings), &cache),
kind,
file.as_deref(),
settings.cmake.profiles.as_deref(),
)?
};
let result = compile::run(&root, &plan, &token, &|e| {
let _ = on_event.send(e);
@@ -325,6 +501,11 @@ pub async fn build_run(
/// Commands of this module the remote server answers.
pub(crate) fn register(d: &mut Dispatcher) {
d.add("build_model", |c| async move { build_model(&c.host, &c.window).await });
d.add("gradle_link_projects", |c| async move { gradle_link_projects(&c.host, &c.window, c.arg("paths")?).await });
d.add(
"gradle_unlink_projects",
|c| async move { gradle_unlink_projects(&c.host, &c.window, c.arg("paths")?).await },
);
d.add("build_sync", |c| async move {
build_sync(
&c.host,
@@ -333,10 +514,26 @@ pub(crate) fn register(d: &mut Dispatcher) {
c.arg("request")?,
c.arg("offline")?,
c.arg("profiles")?,
c.arg("buildDir")?,
c.sender("onProgress")?,
)
.await
});
d.add("gradle_published_catalog", |c| async move {
gradle_published_catalog(&c.host, &c.window, c.arg("coordinates")?).await
});
d.add("gradle_dependencies", |c| async move {
gradle_dependencies(
&c.host,
&c.window,
c.arg("gradlePath")?,
c.arg("configuration")?,
c.arg("offline")?,
c.arg("request")?,
c.arg("buildDir")?,
)
.await
});
d.add("maven_dependency_declaration", |c| async move {
maven_dependency_declaration(&c.host, &c.window, c.arg("pom")?, c.arg("dependency")?).await
});
@@ -361,6 +558,48 @@ pub(crate) fn register(d: &mut Dispatcher) {
mod gradle_settings_tests {
use super::*;
#[test]
fn linking_retains_the_implicit_build_and_unlinking_all_stays_empty() {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
for name in ["a", "b"] {
std::fs::create_dir(root.join(name)).unwrap();
std::fs::write(root.join(name).join("settings.gradle"), format!("rootProject.name = '{name}'")).unwrap();
}
let mut settings = ide_core::settings::GradleSettings::default();
let linked = linked_gradle_paths(&root, &settings, &["b/settings.gradle".into()], true).unwrap();
assert_eq!(linked, ["a", "b"]);
settings.linked_projects = Some(linked);
assert_eq!(linked_gradle_paths(&root, &settings, &["b".into()], true).unwrap(), ["a", "b"]);
assert!(linked_gradle_paths(&root, &settings, &["missing".into()], true).is_err());
settings.linked_projects = Some(vec!["./a".into(), root.join("b").to_string_lossy().into_owned()]);
settings.linked_projects = Some(linked_gradle_paths(&root, &settings, &["a".into()], false).unwrap());
assert_eq!(settings.linked_projects.as_ref().unwrap(), &["b"]);
settings.linked_projects = Some(linked_gradle_paths(&root, &settings, &["b".into()], false).unwrap());
assert!(ide_build::gradle::load_linked(&root, &settings).is_empty());
assert!(root.join("a/settings.gradle").is_file());
assert!(root.join("b/settings.gradle").is_file());
}
#[test]
fn settings_resolve_each_linked_builds_own_wrapper() {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
let mut settings = Settings::default();
settings.languages.java.gradle.linked_projects = Some(vec!["a".into(), "b".into()]);
let executable = if cfg!(windows) { "gradlew.bat" } else { "gradlew" };
for dir in ["a", "b"] {
std::fs::create_dir(root.join(dir)).unwrap();
std::fs::write(root.join(dir).join("build.gradle"), "plugins { id 'java' }").unwrap();
std::fs::write(root.join(dir).join(executable), "").unwrap();
}
let model = load(&root, &root.join("cache"), &settings);
assert_eq!(model.gradle.len(), 2);
for g in model.gradle {
assert_eq!(g.command, root.join(&g.dir).join(executable).to_string_lossy());
}
}
#[test]
fn model_for_sync_tasks_and_compile_uses_gradle_settings() {
let tmp = tempfile::tempdir().unwrap();
@@ -369,13 +608,13 @@ mod gradle_settings_tests {
let wrapper = root.join(if cfg!(windows) { "gradlew.bat" } else { "gradlew" });
std::fs::write(&wrapper, "").unwrap();
let mut settings = Settings::default();
let default = load(root, &root.join("cache"), &settings).gradle.unwrap();
let default = load(root, &root.join("cache"), &settings).gradle.remove(0);
assert_eq!(default.command, wrapper.to_string_lossy());
settings.languages.java.gradle.jvm = "jdk21".into();
settings.languages.java.gradle.user_home = "user-home".into();
settings.languages.java.gradle.distribution = ide_core::settings::GradleDistribution::Local;
settings.languages.java.gradle.home = "gradle-install".into();
let model = load(root, &root.join("cache"), &settings).gradle.unwrap();
let model = load(root, &root.join("cache"), &settings).gradle.remove(0);
assert_eq!(
model.command,
root.join("gradle-install")
+8 -6
View File
@@ -123,13 +123,14 @@ pub(crate) fn register(d: &mut Dispatcher) {
statements(&c.host, &c.window, &id, &text, from, to)
});
d.add("db_execute", |c| async move {
let (request, id, statements, max_rows) = (
let (request, id, statements, skip, max_rows) = (
c.arg::<RequestId>("request")?,
c.arg::<String>("id")?,
c.arg::<Vec<SqlStatement>>("statements")?,
c.arg::<Option<u32>>("skip")?.unwrap_or(0),
c.arg::<u32>("maxRows")?,
);
execute(&c.host, &c.window, request, &id, statements, max_rows).await
execute(&c.host, &c.window, request, &id, statements, skip, max_rows).await
});
d.add("db_table_page", |c| async move {
let (request, id, schema, table) = (
@@ -216,8 +217,7 @@ pub async fn connect(host: &Host, window: &str, id: &str) -> IdeResult<()> {
/// manual mode rolls back what was not committed).
pub async fn set_manual_commit(host: &Host, window: &str, id: &str, manual: bool) -> IdeResult<()> {
let db = host.project(window)?.db.clone();
db.set_manual_commit(id, manual).await;
Ok(())
db.set_manual_commit(id, manual).await
}
/// Commits or rolls back the open transaction of a manual-commit data source.
@@ -326,20 +326,22 @@ pub fn statements(host: &Host, window: &str, id: &str, text: &str, from: u32, to
}
/// Runs `statements` in order on data source `id`, stopping at the first
/// error. Cancel with `cancel_request(request)`.
/// error. Each result reads past its first `skip` rows (a later page) and keeps
/// `max_rows`. Cancel with `cancel_request(request)`.
pub async fn execute(
host: &Host,
window: &str,
request: RequestId,
id: &str,
statements: Vec<SqlStatement>,
skip: u32,
max_rows: u32,
) -> IdeResult<Vec<StatementOutcome>> {
let db = host.project(window)?.db.clone();
let cancel = host.begin_request(request);
let mut out = Vec::new();
for statement in statements {
match db.execute(id, &statement.text, max_rows.max(1) as usize, &cancel).await {
match db.execute_from(id, &statement.text, skip as usize, max_rows.max(1) as usize, &cancel).await {
Ok(result) => out.push(StatementOutcome { statement, result: Some(result), error: None }),
Err(e) if e.kind == ide_core::error::ErrorKind::Cancelled => {
host.end_request(request);
+25
View File
@@ -188,6 +188,19 @@ pub async fn debug_stop(host: &Host, _window: &str, session: DebugSessionId) ->
host.debugger.stop(session).await
}
/// The adapter's completions for an expression being typed in the frame `frame_id`.
pub async fn debug_completions(
host: &Host,
_window: &str,
session: DebugSessionId,
frame_id: Option<i64>,
text: String,
line: u32,
column: u32,
) -> IdeResult<Vec<ide_dap::DebugCompletion>> {
host.debugger.completions(session, frame_id, text, line, column).await
}
/// Evaluate with an explicit DAP context: `repl`, `watch`, `hover` or `clipboard`.
pub async fn debug_evaluate_in(
host: &Host,
@@ -376,6 +389,18 @@ pub(crate) fn register(d: &mut Dispatcher) {
.await
});
d.add("debug_stop", |c| async move { debug_stop(&c.host, &c.window, c.arg("session")?).await });
d.add("debug_completions", |c| async move {
debug_completions(
&c.host,
&c.window,
c.arg("session")?,
c.arg("frameId")?,
c.arg("text")?,
c.arg("line")?,
c.arg("column")?,
)
.await
});
d.add("debug_evaluate_in", |c| async move {
debug_evaluate_in(
&c.host,
+1 -1
View File
@@ -18,7 +18,7 @@ pub async fn blocking<T: Send + 'static>(f: impl FnOnce() -> IdeResult<T> + Send
/// editor buffer, so syntax features and language-server requests work in it.
pub async fn doc_open_virtual(host: &Host, window: &str, uri: String) -> IdeResult<DocumentInfo> {
let project = host.project(window)?;
let vd = project.lsp.virtual_document(&uri).await?;
let vd = crate::ops::lsp::lsp_virtual_document(host, window, uri).await?;
blocking(move || project.documents.open_virtual(&vd.uri, vd.language_id, &vd.text)).await
}
+17
View File
@@ -0,0 +1,17 @@
//! Docker run configurations: the environment their `docker` command line runs with.
use crate::Host;
use crate::rpc::Dispatcher;
use ide_core::IdeResult;
use ide_docker::DockerEndpoint;
use std::collections::BTreeMap;
/// The endpoint's variables plus a `PATH` that finds this host's `docker`. A remote project
/// gets them from the server, since its run configurations start on the remote host.
pub async fn run_env(host: &Host, endpoint: DockerEndpoint) -> IdeResult<BTreeMap<String, String>> {
endpoint.validate()?;
host.blocking(move || Ok(ide_docker::run_env(&endpoint))).await
}
pub(crate) fn register(d: &mut Dispatcher) {
d.add("docker_run_env", |c| async move { run_env(&c.host, c.arg("endpoint")?).await });
}
+10 -3
View File
@@ -137,16 +137,23 @@ pub fn set_writable(host: &Host, window: &str, path: PathBuf, writable: bool) ->
pub async fn delete(host: &Host, window: &str, path: PathBuf) -> IdeResult<()> {
let history = host.project(window).ok().map(|p| p.history.clone());
host.blocking(move || {
let paths = ide_workspace::file_ops::deletion_paths(&path)?;
if let Some(h) = history {
h.record_deletion(&path, DELETE_SNAPSHOT_LIMIT);
for p in paths {
h.record_deletion(&p, DELETE_SNAPSHOT_LIMIT);
}
}
ide_vfs::delete(&path)
ide_workspace::file_ops::delete(&path)
})
.await
}
pub async fn copy(host: &Host, from: PathBuf, to: PathBuf) -> IdeResult<()> {
host.blocking(move || ide_vfs::copy(&from, &to)).await
host.blocking(move || ide_workspace::file_ops::copy(&from, &to)).await
}
pub async fn rename(host: &Host, from: PathBuf, to: PathBuf) -> IdeResult<()> {
host.blocking(move || ide_workspace::file_ops::rename(&from, &to)).await
}
/// Reads a text file without opening it as a document; prefers the unsaved buffer.
+159 -12
View File
@@ -76,9 +76,17 @@ async fn plugin_call<T: Send + 'static>(f: impl FnOnce() -> T + Send + 'static)
tokio::task::spawn_blocking(f).await.ok()
}
/// Items from plugins carry no server (resolve and commands don't apply).
/// Items from plugins carry no server: commands don't apply, and a code
/// action is resolved by the plugin named in its `data` ([`PluginActionRef`]).
const PLUGIN_SERVER: ServerId = ServerId(0);
/// `data` of a plugin's code action whose edits it computes when picked.
#[derive(serde::Serialize, serde::Deserialize)]
struct PluginActionRef {
plugin: String,
data: String,
}
/// A plugin's `completion-item.kind` (wit/providers.wit lists them).
fn plugin_completion_kind(kind: &str) -> Option<CompletionKind> {
Some(match kind {
@@ -120,14 +128,14 @@ fn plugin_completion(item: ide_plugin::CompletionItem) -> CompletionItem {
kind,
documentation: item.documentation.map(|value| Markup { kind: "markdown".into(), value }),
sort_text: item.sort_text,
filter_text: None,
filter_text: item.filter_text,
preselect: false,
deprecated: false,
insert_text: item.insert_text,
is_snippet: false,
is_snippet: item.snippet,
replace_range: item.replace,
replace_range_tab: None,
additional_edits: Vec::new(),
additional_edits: item.additional_edits,
commit_characters: Vec::new(),
command: None,
data: None,
@@ -252,6 +260,9 @@ pub async fn lsp_implementations(
/// Contents of a read-only language-server document, including JDT library
/// classes and disk class files. Cached per project.
pub async fn lsp_virtual_document(host: &Host, window: &str, uri: String) -> IdeResult<VirtualDocument> {
if uri.starts_with("git-revision:") {
return crate::ops::vcs::revision_virtual_document(host, window, &uri).await;
}
host.project(window)?.lsp.virtual_document(&uri).await
}
@@ -314,7 +325,7 @@ pub async fn lsp_code_actions(
};
let plugin_actions: Vec<CodeAction> = plugin_actions
.into_iter()
.map(|(_, a)| plugin_code_action(a))
.map(|(plugin, a)| plugin_code_action(&plugin, a))
.filter(|a| {
only.is_empty()
|| a.kind.as_ref().is_some_and(|k| only.iter().any(|o| k == o || k.starts_with(&format!("{o}."))))
@@ -330,24 +341,53 @@ pub async fn lsp_code_actions(
prepare_actions(project, doc, version, actions).await
}
fn plugin_code_action(a: ide_plugin::CodeAction) -> CodeAction {
let files = a
fn plugin_code_action(plugin: &str, a: ide_plugin::CodeAction) -> CodeAction {
// With `data` and no edits, the plugin computes them when the action is picked.
let lazy = a.edits.is_empty() && a.data.is_some();
let files: Vec<_> = a
.edits
.into_iter()
.map(|(path, edits)| ide_core::FileEdit::Edit { path: path.into(), edits, version: None })
.collect();
let data = a
.data
.filter(|_| lazy)
.and_then(|data| serde_json::to_value(PluginActionRef { plugin: plugin.to_owned(), data }).ok());
CodeAction {
title: a.title,
kind: a.kind,
is_preferred: a.preferred,
disabled_reason: None,
edit: Some(WorkspaceEdit { files, preconditions: Vec::new() }),
edit: (!lazy).then(|| WorkspaceEdit { files, preconditions: Vec::new() }),
command: None,
data: None,
data,
server: PLUGIN_SERVER,
}
}
/// A code action resolved by whoever offered it: its language server, or
/// the plugin its `data` names.
async fn resolve_action(host: &Host, window: &str, doc: DocId, action: CodeAction) -> IdeResult<CodeAction> {
if action.server != PLUGIN_SERVER {
return host.project(window)?.lsp.resolve_code_action(doc, action).await;
}
let Some(r) = action.data.clone().and_then(|d| serde_json::from_value::<PluginActionRef>(d).ok()) else {
return Ok(action);
};
let (plugins, d) = plugin_providers(host, window, doc, ProviderKind::CodeActions)
.ok_or_else(|| ide_core::IdeError::unavailable("the plugin that offered this action is gone"))?;
let pending = ide_plugin::CodeAction {
title: action.title.clone(),
kind: action.kind.clone(),
preferred: action.is_preferred,
edits: Vec::new(),
data: Some(r.data),
};
let plugin = r.plugin.clone();
let resolved = blocking(move || plugins.resolve_code_action(&plugin, &d, pending)).await?;
Ok(plugin_code_action(&r.plugin, ide_plugin::CodeAction { data: None, ..resolved }))
}
/// Cancellable code actions with the kinds the servers advertise (Refactor This, refactoring availability).
pub async fn lsp_code_action_query(
host: &Host,
@@ -381,7 +421,7 @@ pub async fn lsp_resolve_code_action(
let project = host.project(window)?;
let documents = project.documents.clone();
let version = blocking(move || documents.version(doc)).await?;
let action = project.lsp.resolve_code_action(doc, action).await?;
let action = resolve_action(host, window, doc, action).await?;
Ok(prepare_actions(project, doc, version, vec![action]).await?.remove(0))
}
@@ -391,7 +431,7 @@ pub async fn lsp_apply_code_action(host: &Host, window: &str, doc: DocId, action
let action = if action.edit.is_none() && action.command.is_none() {
let documents = project.documents.clone();
let version = blocking(move || documents.version(doc)).await?;
let action = project.lsp.resolve_code_action(doc, action).await?;
let action = resolve_action(host, window, doc, action).await?;
prepare_actions(project.clone(), doc, version, vec![action]).await?.remove(0)
} else {
action
@@ -401,17 +441,85 @@ pub async fn lsp_apply_code_action(host: &Host, window: &str, doc: DocId, action
let edit = edit.clone();
blocking(move || apply_edit(&documents, &edit)).await?;
}
if let Some(command) = action.command {
if let Some(command) = action.command
&& !(action.edit.is_some() && is_client_follow_up(&project, doc, None, &command).await)
{
project.lsp.execute_command(doc, command).await?;
}
Ok(())
}
/// A command after a code action's edit that no server provides: what other editors'
/// extensions do once the edit is applied (vtsls' `_typescript.didOrganizeImports`). The edit
/// is the action; there is nothing to run.
async fn is_client_follow_up(
project: &crate::project::Project,
doc: DocId,
server: Option<ide_core::ServerId>,
command: &Command,
) -> bool {
let provided = project.lsp.provides_command(doc, server, &command.command).await;
if !provided {
tracing::debug!("skipping `{}` after the code action's edit: no server provides it", command.command);
}
!provided
}
pub async fn lsp_prepare_rename(host: &Host, window: &str, doc: DocId, pos: Pos) -> IdeResult<Option<Range>> {
host.project(window)?.lsp.prepare_rename(doc, pos).await
}
/// Computes the rename edit without applying it, so the UI can preview conflicts.
/// The class at `pos` and the members a class-hierarchy refactoring can move.
pub async fn lsp_hierarchy_members(
host: &Host,
window: &str,
request: RequestId,
doc: DocId,
pos: Pos,
kind: HierarchyRefactoring,
) -> IdeResult<HierarchyMembers> {
let project = host.project(window)?;
let cancel = host.begin_request(request);
let result = project.lsp.hierarchy_members(doc, pos, kind, cancel).await;
host.end_request(request);
result
}
/// Checks a class-hierarchy refactoring and computes its edit without applying it.
#[allow(clippy::too_many_arguments)]
pub async fn lsp_hierarchy_refactor(
host: &Host,
window: &str,
request: RequestId,
doc: DocId,
pos: Pos,
kind: HierarchyRefactoring,
target: Option<String>,
choices: Vec<HierarchyChoice>,
) -> IdeResult<HierarchyRefactored> {
let project = host.project(window)?;
let documents = project.documents.clone();
let version = blocking(move || documents.version(doc)).await?;
let cancel = host.begin_request(request);
let result = project.lsp.hierarchy_refactor(doc, pos, kind, target, choices, cancel).await;
host.end_request(request);
let mut refactored = result?;
blocking(move || {
if project.documents.version(doc)? != version {
return Err(ide_core::IdeError::new(
ide_core::error::ErrorKind::OutOfSync,
"document changed while computing the refactoring; request it again",
));
}
if let Some(edit) = &mut refactored.edit {
project.documents.prepare_workspace_edit(edit)?;
}
Ok(refactored)
})
.await
}
pub async fn lsp_rename(
host: &Host,
window: &str,
@@ -589,6 +697,7 @@ pub async fn lsp_execute_code_action(
if cancel.is_cancelled() {
return Err(ide_core::IdeError::cancelled());
}
let edited = action.edit.is_some();
if let Some(edit) = action.edit {
let documents = project.documents.clone();
tokio::task::spawn_blocking(move || apply_edit(&documents, &edit))
@@ -596,6 +705,9 @@ pub async fn lsp_execute_code_action(
.map_err(|e| ide_core::IdeError::internal(e.to_string()))??;
}
match action.command {
Some(command) if edited && is_client_follow_up(&project, doc, Some(action.server), &command).await => {
Ok(Vec::new())
}
Some(command) => project
.lsp
.execute_action_command(doc, action.server, command, answers, cancel)
@@ -676,6 +788,22 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("lsp_rename", |c| async move {
lsp_rename(&c.host, &c.window, c.arg("request")?, c.arg("doc")?, c.arg("pos")?, c.arg("newName")?).await
});
d.add("lsp_hierarchy_members", |c| async move {
lsp_hierarchy_members(&c.host, &c.window, c.arg("request")?, c.arg("doc")?, c.arg("pos")?, c.arg("kind")?).await
});
d.add("lsp_hierarchy_refactor", |c| async move {
lsp_hierarchy_refactor(
&c.host,
&c.window,
c.arg("request")?,
c.arg("doc")?,
c.arg("pos")?,
c.arg("kind")?,
c.arg("target")?,
c.arg("choices")?,
)
.await
});
d.add("lsp_format", |c| async move {
lsp_format(&c.host, &c.window, c.arg("doc")?, c.arg("range")?, c.arg("options")?).await
});
@@ -735,6 +863,25 @@ pub(crate) fn register(d: &mut Dispatcher) {
mod tests {
use super::*;
#[test]
fn a_plugin_action_with_data_and_no_edits_is_resolved_by_its_plugin() {
let action = |edits: Vec<(std::path::PathBuf, Vec<TextEdit>)>, data: Option<&str>| ide_plugin::CodeAction {
title: "Fix".into(),
kind: None,
preferred: false,
edits,
data: data.map(str::to_owned),
};
let lazy = plugin_code_action("acme.x", action(Vec::new(), Some("ref")));
assert!(lazy.edit.is_none());
let r: PluginActionRef = serde_json::from_value(lazy.data.unwrap()).unwrap();
assert_eq!((r.plugin.as_str(), r.data.as_str()), ("acme.x", "ref"));
// With edits, `data` is the plugin's business: applied as is.
let edit = TextEdit { range: Range::default(), new_text: "x".into() };
let eager = plugin_code_action("acme.x", action(vec![("/a".into(), vec![edit])], Some("ref")));
assert!(eager.edit.is_some() && eager.data.is_none());
}
#[test]
fn every_completion_kind_the_plugin_api_documents_is_understood() {
let wit = include_str!("../../../../wit/providers.wit");
+2
View File
@@ -12,6 +12,7 @@ pub mod debug;
pub mod dependencies;
pub mod deploy;
pub mod doc;
pub mod docker;
pub mod env;
pub mod fs;
pub mod history;
@@ -87,6 +88,7 @@ pub fn register_all(d: &mut crate::rpc::Dispatcher) {
run::register(d);
env::register(d);
deploy::register(d);
docker::register(d);
db::register(d);
}
+12 -1
View File
@@ -3,7 +3,7 @@ use crate::Host;
use crate::Project;
use crate::rpc::Dispatcher;
use ide_core::{DocId, IdeResult, IpcPath, Location, Pos};
use ide_index::{FileMatch, IndexStatus, SymbolFilter, SymbolMatch};
use ide_index::{ExtensionCount, FileMatch, IndexStatus, SymbolFilter, SymbolMatch};
use ide_language::{FileReference, WebReference};
use ide_workspace::file_refs::{self, PathCompletion};
use serde::{Deserialize, Serialize};
@@ -32,10 +32,19 @@ pub fn index_status(host: &Host, window: &str) -> IdeResult<IndexStatus> {
Ok(host.project(window)?.index.status())
}
/// The project's file extensions, most common first.
pub fn index_file_extensions(host: &Host, window: &str) -> IdeResult<Vec<ExtensionCount>> {
Ok(host.project(window)?.index.file_extensions())
}
pub fn find_files(host: &Host, window: &str, query: String, limit: u32) -> IdeResult<Vec<FileMatch>> {
Ok(host.project(window)?.index.find_files(&query, limit as usize))
}
pub fn find_dirs(host: &Host, window: &str, query: String, limit: u32) -> IdeResult<Vec<FileMatch>> {
Ok(host.project(window)?.index.find_dirs(&query, limit as usize))
}
pub fn find_symbols(
host: &Host,
window: &str,
@@ -156,7 +165,9 @@ pub async fn web_unresolved_references(host: &Host, window: &str, doc: DocId) ->
/// Commands of this module the remote server answers.
pub(crate) fn register(d: &mut Dispatcher) {
d.add("index_status", |c| async move { index_status(&c.host, &c.window) });
d.add("index_file_extensions", |c| async move { index_file_extensions(&c.host, &c.window) });
d.add("find_files", |c| async move { find_files(&c.host, &c.window, c.arg("query")?, c.arg("limit")?) });
d.add("find_dirs", |c| async move { find_dirs(&c.host, &c.window, c.arg("query")?, c.arg("limit")?) });
d.add("find_symbols", |c| async move {
find_symbols(&c.host, &c.window, c.arg("query")?, c.arg("filter")?, c.arg("limit")?)
});
+12 -2
View File
@@ -17,6 +17,8 @@ pub struct ProjectInfo {
pub settings: Settings,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub workspace: Option<ide_workspace::workspace::Workspace>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub unity: Option<ide_workspace::unity::UnityProjectInfo>,
}
pub fn info(project: &Project) -> ProjectInfo {
@@ -33,6 +35,7 @@ pub fn info(project: &Project) -> ProjectInfo {
settings,
root: project.root.clone().into(),
workspace: ide_workspace::workspace::Workspace::load(&project.root).ok().flatten(),
unity: ide_workspace::unity::detect(&project.root),
}
}
@@ -172,7 +175,12 @@ pub fn set_settings(
host.apply_proxy();
}
// A new interpreter / toolchain / server executable restarts the language servers it affects.
let code_style = key == "editor" || key == "editor.codeStyle" || key.starts_with("editor.codeStyle.");
// So do inspections that are compiler options (`ide-sdk` `configure_java_compiler`).
let code_style = key == "editor"
|| key == "editor.codeStyle"
|| key.starts_with("editor.codeStyle.")
|| key == "inspections"
|| key.starts_with("inspections.");
let toolchain = ["languages", "tools", "languageServers", "projectStructure"]
.iter()
.any(|k| key == *k || key.starts_with(&format!("{k}.")));
@@ -188,7 +196,9 @@ pub fn set_settings(
}
let roots_changed = key == "projectStructure"
|| key.starts_with("projectStructure.")
|| "languages.java.maven.unlinkedProjects".starts_with(key);
|| "languages.java.maven.unlinkedProjects".starts_with(key)
|| "languages.java.maven.linkedProjects".starts_with(key)
|| "languages.java.gradle.linkedProjects".starts_with(key);
// Content roots and excludes (files.exclude, ignored names, excluded folders) reach the
// index and the watchers without reopening the project (the rescan runs in the background).
if roots_changed || key == "files" || key.starts_with("files.") {
+4 -4
View File
@@ -20,7 +20,7 @@ const MAX_FILE_BYTES: u64 = 4 << 20;
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct RouteSource {
pub path: String,
pub path: ide_core::IpcPath,
pub text: String,
}
@@ -51,7 +51,7 @@ pub(crate) fn sources_in(root: &Path, excludes: &[String]) -> Vec<RouteSource> {
return;
}
bytes.fetch_add(text.len(), Ordering::Relaxed);
found.push(RouteSource { path: path.to_string_lossy().into_owned(), text });
found.push(RouteSource { path: path.to_path_buf().into(), text });
});
let mut files = found.into_inner().unwrap_or_default();
files.sort_by(|a, b| a.path.cmp(&b.path));
@@ -116,7 +116,7 @@ mod tests {
std::fs::write(tmp.path().join("src/Notes.txt"), "@GetMapping(\"/a\")").unwrap();
let files = sources_in(tmp.path(), &[]);
assert_eq!(files.len(), 1, "{:?}", files.iter().map(|f| &f.path).collect::<Vec<_>>());
assert!(Path::new(&files[0].path).ends_with("src/Ctl.java"), "{}", files[0].path);
assert!(files[0].path.ends_with("src/Ctl.java"), "{:?}", files[0].path);
assert!(files[0].text.contains("@GetMapping"));
}
@@ -129,6 +129,6 @@ mod tests {
std::fs::write(tmp.path().join("build/B.java"), "@RequestMapping(\"/b\")").unwrap();
let files = sources_in(tmp.path(), &["build/**".to_owned()]);
assert_eq!(files.len(), 1, "{:?}", files.iter().map(|f| &f.path).collect::<Vec<_>>());
assert!(Path::new(&files[0].path).ends_with("module/src/A.java"), "{}", files[0].path);
assert!(files[0].path.ends_with("module/src/A.java"), "{:?}", files[0].path);
}
}
+64 -14
View File
@@ -41,7 +41,7 @@ pub fn configs(host: &Host, window: &str) -> IdeResult<Vec<RunConfig>> {
}
/// A project's detected configurations (minus removed ones and unlinked Maven projects), then
/// its saved ones; toolchains from Languages & Frameworks applied. Gradle choices stay as saved:
/// its saved ones; toolchains from Languages & Frameworks applied. Build-tool choices stay as saved:
/// derived defaults must not be written back as explicit environment.
fn detected_and_saved(
root: &std::path::Path,
@@ -56,7 +56,7 @@ fn detected_and_saved(
.filter(|d| !suppressed.contains(&d.id) && !saved.iter().any(|u| u.id == d.id))
.collect();
all.extend(saved.iter().cloned());
for config in all.iter_mut().filter(|c| c.kind != "gradle") {
for config in all.iter_mut().filter(|c| !matches!(c.kind.as_str(), "gradle" | "maven")) {
ide_sdk::apply::apply_to_run_configs(std::slice::from_mut(config), root, settings);
}
all
@@ -75,13 +75,65 @@ pub fn save_configs(
let configs: Vec<RunConfig> = configs
.iter()
.map(|c| match c.kind.as_str() {
"gradle" => c.clone(),
"gradle" | "maven" => c.clone(),
_ => ide_sdk::apply::without_derived(c, &project.root, &settings),
})
.collect();
ide_run::save_user_configs_with_removals(&project.root, &configs, removed_detected_ids)
}
/// The project `config` runs in (a workspace member, by its working directory) and its settings.
fn settings_for(
host: &Host,
project: &crate::project::Project,
config: &RunConfig,
) -> IdeResult<(std::path::PathBuf, ide_core::settings::Settings)> {
let member = ide_workspace::workspace::Workspace::load(&project.root)?.and_then(|workspace| {
let cwd = config.cwd.as_deref()?;
workspace.projects.into_iter().filter(|root| cwd.starts_with(root)).max_by_key(|root| root.components().count())
});
Ok(match member {
Some(root) => {
let settings = ide_workspace::SettingsStore::load(&host.config_dir, Some(&root)).effective();
(root, settings)
}
None => (project.root.clone(), project.settings.effective()),
})
}
/// Temporary configurations as they are kept between sessions: without what settings derived
/// (toolchain commands, Maven options, `JAVA_HOME`…), so they follow later settings changes.
pub fn without_derived(host: &Host, window: &str, configs: Vec<RunConfig>) -> IdeResult<Vec<RunConfig>> {
host.require_trust(window)?;
let project = host.project(window)?;
configs
.into_iter()
.map(|c| {
if c.kind == "gradle" {
return Ok(c);
}
let (root, settings) = settings_for(host, &project, &c)?;
Ok(ide_sdk::apply::without_derived(&c, &root, &settings))
})
.collect()
}
/// Kept temporary configurations with today's settings applied, as [`configs`] lists the others.
pub fn apply_settings(host: &Host, window: &str, configs: Vec<RunConfig>) -> IdeResult<Vec<RunConfig>> {
host.require_trust(window)?;
let project = host.project(window)?;
configs
.into_iter()
.map(|mut c| {
if c.kind != "gradle" {
let (root, settings) = settings_for(host, &project, &c)?;
ide_sdk::apply::apply_to_run_configs(std::slice::from_mut(&mut c), &root, &settings);
}
Ok(c)
})
.collect()
}
pub async fn start(host: &Host, window: &str, config: RunConfig, on_output: Emit<OutputChunk>) -> IdeResult<RunInfo> {
host.require_trust(window)?;
let project = host.project(window)?;
@@ -100,18 +152,16 @@ pub async fn start(host: &Host, window: &str, config: RunConfig, on_output: Emit
.run(&settings, &project.root, &config, project.sink.clone(), Box::new(move |c| out(c)))
.await;
}
let member = ide_workspace::workspace::Workspace::load(&project.root)?.and_then(|workspace| {
let cwd = config.cwd.as_deref()?;
workspace.projects.into_iter().filter(|root| cwd.starts_with(root)).max_by_key(|root| root.components().count())
});
let root = member.as_ref().unwrap_or(&project.root);
let settings = match &member {
Some(root) => ide_workspace::SettingsStore::load(&host.config_dir, Some(root)).effective(),
None => project.settings.effective(),
};
let (root, settings) = settings_for(host, &project, &config)?;
let root = &root;
// Configurations made in the UI name bare tools (`godot`) that may live outside PATH.
// Maven re-applies too: a config listed before a settings change (daemon toggle,
// Maven home) keeps the launcher that was current then; re-resolving at launch is
// what makes the setting take effect without reopening the project.
let mut config = config;
if matches!(config.kind.as_str(), "gradle" | "tomcat") {
if config.kind == "maven" {
ide_sdk::apply::prepare_maven_run(&mut config, root, &settings);
} else if matches!(config.kind.as_str(), "gradle" | "tomcat") {
ide_sdk::apply::apply_to_run_configs(std::slice::from_mut(&mut config), root, &settings);
} else {
ide_sdk::apply::resolve_run_command(&mut config, root, &settings);
@@ -147,7 +197,7 @@ pub async fn start(host: &Host, window: &str, config: RunConfig, on_output: Emit
let encoding = settings.console.encoding;
let options = StartOptions {
encoding: Some(encoding).filter(|e| !e.trim().is_empty()),
python_paths: if member.is_some() { vec![root.clone()] } else { super::python::paths(&project) },
python_paths: if *root != project.root { vec![root.clone()] } else { super::python::paths(&project) },
};
project.require_trust()?;
let mut sink = project.sink.clone();
+58
View File
@@ -142,6 +142,21 @@ pub async fn run_gradle_jvm_args_script(_host: &Host, _window: &str, jvm_args: V
.map_err(|e| IdeError::internal(e.to_string()))?
}
/// Writes one of the IDE's Gradle init scripts (`text`; the file is named `<name>-<hash>`) on
/// the machine that runs the build, for `--init-script`, and returns its path.
pub async fn run_gradle_init_script(_host: &Host, _window: &str, name: String, text: String) -> IdeResult<String> {
if name.is_empty() || !name.bytes().all(|b| b.is_ascii_alphanumeric() || b == b'-') {
return Err(IdeError::invalid(format!("invalid init script name {name:?}")));
}
tokio::task::spawn_blocking(move || {
ide_run::gradle_jvm::write_script(&ide_run::gradle_jvm::default_dir(), &name, &text)
.map(|p| p.to_string_lossy().into_owned())
.map_err(|e| IdeError::internal(format!("cannot write the Gradle init script: {e}")))
})
.await
.map_err(|e| IdeError::internal(e.to_string()))?
}
/// Results from the JUnit XML reports (Surefire, Failsafe, Gradle) of the module at
/// `module` (relative to the project root) written since `since_ms` (Unix epoch, ms).
pub async fn run_test_reports(
@@ -241,6 +256,35 @@ pub async fn run_tomcat_update(host: &Host, window: &str, config: RunConfig, mod
.map_err(|e| IdeError::internal(e.to_string()))?
}
/// Updates a running Tomcat artifact deployment from the module's compiled output and web
/// resources (`ide_run::tomcat::sync_outputs`). `false`: rebuild the WAR instead.
pub async fn run_tomcat_sync(
host: &Host,
window: &str,
config: RunConfig,
module: std::path::PathBuf,
class_paths: Vec<String>,
resources_only: bool,
) -> IdeResult<bool> {
host.require_trust(window)?;
let project = host.project(window)?;
if !host.processes.list().iter().any(|r| r.running && r.config_id == config.id) {
return Err(IdeError::invalid("Start this Tomcat configuration before updating it"));
}
if config.extras.params.get("deployment").is_some_and(|v| v != "artifact") {
return Ok(false);
}
tokio::task::spawn_blocking(move || {
let module = dunce::canonicalize(project.root.join(module))?;
let class_paths: Vec<_> =
class_paths.iter().filter_map(|p| dunce::canonicalize(project.root.join(p)).ok()).collect();
let base = ide_run::tomcat::base_dir(&project.cache_dir, &config.id);
ide_run::tomcat::sync_outputs(&base, &module, &class_paths, resources_only)
})
.await
.map_err(|e| IdeError::internal(e.to_string()))?
}
/// Commands of this module the remote server answers.
pub(crate) fn register(d: &mut Dispatcher) {
d.add("run_dotnet_resolve", |c| async move {
@@ -255,6 +299,17 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("run_tomcat_home", |c| async move { run_tomcat_home(&c.host, &c.window, c.arg("home")?).await });
d.add("run_tomcat_artifacts", |c| async move { run_tomcat_artifacts(&c.host, &c.window).await });
d.add("run_tomcat_build", |c| async move { run_tomcat_build(&c.host, &c.window, c.arg("config")?).await });
d.add("run_tomcat_sync", |c| async move {
run_tomcat_sync(
&c.host,
&c.window,
c.arg("config")?,
c.arg("module")?,
c.arg("classPaths")?,
c.arg("resourcesOnly")?,
)
.await
});
d.add("run_tomcat_update", |c| async move {
run_tomcat_update(&c.host, &c.window, c.arg("config")?, c.arg("mode")?).await
});
@@ -268,6 +323,9 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("run_gradle_jvm_args_script", |c| async move {
run_gradle_jvm_args_script(&c.host, &c.window, c.arg("jvmArgs")?).await
});
d.add("run_gradle_init_script", |c| async move {
run_gradle_init_script(&c.host, &c.window, c.arg("name")?, c.arg("text")?).await
});
d.add("run_test_reports", |c| async move {
run_test_reports(&c.host, &c.window, c.arg("module")?, c.arg("sinceMs")?).await
});
+60 -2
View File
@@ -23,8 +23,26 @@ pub struct LocatedTools {
/// Installed interpreters / toolchains of `kind`; project-local ones first.
pub async fn sdk_detect(host: &Host, window: &str, kind: SdkKind) -> IdeResult<Vec<Sdk>> {
host.require_trust(window)?;
let root = host.project(window).ok().map(|p| p.root.clone());
tokio::task::spawn_blocking(move || ide_sdk::detect(kind, root.as_deref())).await.map_err(join_err)
let project = host.project(window).ok();
let root = project.as_ref().map(|p| p.root.clone());
// The interpreter the project uses is a choice even where detection doesn't look (a
// virtualenv in a subfolder, one picked by path), so it shows by name and version.
let current = match (&project, kind) {
(Some(p), SdkKind::Python) => ide_sdk::apply::python_interpreter(&p.root, &p.settings.effective()),
_ => None,
};
tokio::task::spawn_blocking(move || {
let mut sdks = ide_sdk::detect(kind, root.as_deref());
if let Some(path) = current
&& !sdks.iter().any(|s| *s.path == *path)
&& let Some(sdk) = ide_sdk::python_interpreter_sdk(&path, root.as_deref())
{
sdks.push(sdk);
}
sdks
})
.await
.map_err(join_err)
}
/// The interpreter the project uses: the configured one, else its virtualenv.
@@ -34,6 +52,42 @@ pub fn sdk_python_effective(host: &Host, window: &str) -> IdeResult<Option<ide_c
Ok(ide_sdk::apply::python_interpreter(&project.root, &project.settings.effective()).map(Into::into))
}
/// The PHP language level of the project `path` belongs to (a workspace member, by path, as
/// the language server configuration picks it).
#[derive(Clone, Debug, serde::Serialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct PhpLanguageLevels {
/// The one Intelephense checks against; `None`: its default.
pub effective: Option<ide_sdk::php::PhpLanguageLevel>,
/// The one Auto picks, from `composer.json`.
pub auto: Option<ide_sdk::php::PhpLanguageLevel>,
/// The workspace member `path` belongs to: its own settings choose the level, not the
/// workspace's. `None`: the opened project.
pub member: Option<ide_core::IpcPath>,
}
pub async fn sdk_php_language_level(host: &Host, window: &str, path: PathBuf) -> IdeResult<PhpLanguageLevels> {
let project = host.project(window)?;
let (project_root, project_settings) = (project.root.clone(), project.settings.effective());
let config_dir = host.config_dir.clone();
tokio::task::spawn_blocking(move || {
let member = ide_workspace::workspace::Workspace::load(&project_root).ok().flatten().and_then(|w| {
w.projects.into_iter().filter(|p| path.starts_with(p)).max_by_key(|p| p.components().count())
});
let (root, settings) = match &member {
Some(member) => (member, ide_workspace::SettingsStore::load(&config_dir, Some(member)).effective()),
None => (&project_root, project_settings),
};
PhpLanguageLevels {
effective: ide_sdk::php::language_level(root, &settings),
auto: ide_sdk::php::composer_level(root),
member: member.map(ide_core::IpcPath),
}
})
.await
.map_err(join_err)
}
/// SDK / library source folders shown under "External Libraries" in the Project view,
/// with the Maven / Gradle dependency jars of the build model.
pub async fn sdk_project_libraries(host: &Host, window: &str) -> IdeResult<Vec<ide_sdk::libraries::LibraryGroup>> {
@@ -144,6 +198,10 @@ pub async fn sdk_python_pip(
pub(crate) fn register(d: &mut Dispatcher) {
d.add("sdk_detect", |c| async move { sdk_detect(&c.host, &c.window, c.arg("kind")?).await });
d.add("sdk_python_effective", |c| async move { sdk_python_effective(&c.host, &c.window) });
d.add(
"sdk_php_language_level",
|c| async move { sdk_php_language_level(&c.host, &c.window, c.arg("path")?).await },
);
d.add("sdk_project_libraries", |c| async move { sdk_project_libraries(&c.host, &c.window).await });
d.add("sdk_locate_tools", |c| async move {
sdk_locate_tools(&c.host, &c.window, c.arg("names")?, c.arg("near")?).await
+268 -24
View File
@@ -12,6 +12,76 @@ use std::path::{Path, PathBuf};
use crate::apply_git_executable;
/// Read-only repository access also works in a standalone log window.
struct LogAccess {
project: Option<std::sync::Arc<Project>>,
external: Option<std::sync::Arc<Repo>>,
cache: PathBuf,
}
/// Immutable Git objects exposed through the editor's ordinary read-only VFS.
pub async fn revision_virtual_document(
host: &Host,
window: &str,
uri: &str,
) -> IdeResult<ide_lsp::types::VirtualDocument> {
#[derive(Deserialize)]
struct Target {
repo: PathBuf,
revision: String,
path: PathBuf,
}
let encoded = uri
.strip_prefix("git-revision:")
.and_then(|s| s.split('/').next())
.ok_or_else(|| ide_core::IdeError::invalid("invalid Git revision URI"))?;
if encoded.len() > 32768 || encoded.len() % 2 != 0 || !encoded.bytes().all(|b| b.is_ascii_hexdigit()) {
return Err(ide_core::IdeError::invalid("invalid Git revision URI"));
}
let bytes = (0..encoded.len())
.step_by(2)
.map(|i| {
u8::from_str_radix(&encoded[i..i + 2], 16)
.map_err(|_| ide_core::IdeError::invalid("invalid Git revision URI"))
})
.collect::<IdeResult<Vec<_>>>()?;
let target: Target = serde_json::from_slice(&bytes).map_err(|e| ide_core::IdeError::invalid(e.to_string()))?;
let path = target.repo.join(&target.path);
let text = vcs_file_at(host, window, path, Some(target.revision.clone()))
.await?
.ok_or_else(|| ide_core::IdeError::not_found("file is absent at this revision"))?;
Ok(ide_lsp::types::VirtualDocument {
uri: uri.into(),
language_id: host.languages.detect(&target.path, text.lines().next()),
text,
title: target.path.file_name().unwrap_or_default().to_string_lossy().into_owned(),
description: Some(format!("{} · {}", target.repo.display(), target.revision)),
decompiled: false,
})
}
impl LogAccess {
fn new(host: &Host, window: &str, at: Option<&Path>) -> Self {
let project = host.project(window).ok();
let external = host
.external_logs
.read()
.get(host.owner_window(window))
.and_then(|rs| rs.iter().find(|r| Some(r.root()) == at).cloned())
.filter(|repo| project.as_ref().is_none_or(|p| p.repos().at_root(repo.root()).is_none()));
Self { project, external, cache: host.local_data_dir.join("cache/git-log") }
}
fn with_log_repo_at<T>(&self, at: Option<&Path>, f: impl FnOnce(&Repo) -> IdeResult<T>) -> IdeResult<T> {
if let Some(repo) = &self.external {
repo.enable_log_index(&self.cache);
return f(repo);
}
self.project
.as_ref()
.ok_or_else(|| ide_core::IdeError::unavailable("external Git log is not open"))?
.with_log_repo_at(at, f)
}
}
/// A repository of the project (IDEA's VCS root).
#[derive(Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
@@ -182,8 +252,8 @@ pub fn vcs_roots(host: &Host, window: &str) -> IdeResult<Vec<VcsRoot>> {
}
pub async fn vcs_branch_info(host: &Host, window: &str, repo: Option<PathBuf>) -> IdeResult<BranchInfo> {
let project = host.project(window)?;
blocking(move || project.with_repo_at(repo.as_deref(), |r| r.branch_info())).await
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.branch_info())).await
}
/// Gutter change markers for a file, computed against its current (unsaved) text.
@@ -197,9 +267,21 @@ pub async fn vcs_line_hunks(host: &Host, window: &str, path: PathBuf) -> IdeResu
}
pub async fn vcs_file_at(host: &Host, window: &str, path: PathBuf, rev: Option<String>) -> IdeResult<Option<String>> {
let project = host.project(window)?;
let project = host.project(window).ok();
let external = host.external_logs.read().get(host.owner_window(window)).and_then(|rs| {
rs.iter().filter(|r| path.starts_with(r.root())).max_by_key(|r| r.root().components().count()).cloned()
});
blocking(move || {
project.with_repo_of(&path, |r| r.file_at(&path, rev.as_deref(), project.vcs_encoding(&path, rev.as_deref())?))
if let Some(project) = project {
return project.with_log_repo_of(&path, |r| {
r.file_at(&path, rev.as_deref(), project.vcs_encoding(&path, rev.as_deref())?)
});
}
external.ok_or_else(|| ide_core::IdeError::unavailable("external Git log is not open"))?.file_at(
&path,
rev.as_deref(),
encoding_rs::UTF_8,
)
})
.await
}
@@ -222,9 +304,148 @@ pub async fn vcs_blame(
.await
}
pub async fn vcs_log(host: &Host, window: &str, repo: Option<PathBuf>, query: LogQuery) -> IdeResult<Vec<Commit>> {
/// History for Selection: the commits that changed `lines` (0-based, inclusive) of the
/// editor's text of `path`. Cancellable.
pub async fn vcs_block_history(
host: &Host,
window: &str,
request: RequestId,
path: PathBuf,
text: Option<String>,
lines: (u32, u32),
) -> IdeResult<BlockHistory> {
let project = host.project(window)?;
blocking(move || project.with_repo_at(repo.as_deref(), |r| r.log(&query))).await
let cancel = host.begin_request(request);
let result = blocking(move || {
let text = text.or_else(|| project.documents.text_of(&path));
project.with_repo_of(&path, |r| {
r.block_history(&path, text.as_deref(), lines, project.vcs_encoding(&path, None)?, &cancel)
})
})
.await;
host.end_request(request);
result
}
pub async fn vcs_log(host: &Host, window: &str, repo: Option<PathBuf>, query: LogQuery) -> IdeResult<Vec<Commit>> {
let project = LogAccess::new(host, window, repo.as_deref());
let request = query.request;
let cancel =
request.map_or_else(ide_core::CancellationToken::new, |id| host.begin_project_read_request(window, id));
let result = blocking(move || {
project.with_log_repo_at(repo.as_deref(), |r| {
r.enable_log_index(&project.cache);
r.log_cancellable(&query, &cancel)
})
})
.await;
if let Some(id) = request {
host.end_request(id);
}
result
}
pub async fn vcs_log_refs(
host: &Host,
window: &str,
repo: Option<PathBuf>,
) -> IdeResult<std::collections::BTreeMap<String, String>> {
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.log_refs())).await
}
pub async fn vcs_log_users(host: &Host, window: &str, repo: Option<PathBuf>) -> IdeResult<Vec<String>> {
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.log_users())).await
}
pub async fn vcs_open_external_log(host: &Host, window: &str, path: PathBuf) -> IdeResult<ide_core::IpcPath> {
let path = dunce::canonicalize(path)?;
let repo = Repo::discover(&path)?.ok_or_else(|| ide_core::IdeError::unavailable("folder is not a Git worktree"))?;
let root = repo.root().to_path_buf();
repo.set_execution_allowed(false);
repo.enable_log_index(&host.local_data_dir.join("cache/git-log"));
if let Ok(project) = host.project(window) {
project.open_external_log(&root)?;
}
let mut logs = host.external_logs.write();
let roots = logs.entry(host.owner_window(window).to_string()).or_default();
if !roots.iter().any(|r| r.root() == root) {
roots.push(std::sync::Arc::new(repo));
}
Ok(root.into())
}
pub async fn vcs_log_details(
host: &Host,
window: &str,
repo: Option<PathBuf>,
revision: String,
) -> IdeResult<LogDetails> {
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.log_details(&revision))).await
}
pub async fn vcs_log_details_batch(
host: &Host,
window: &str,
repo: Option<PathBuf>,
revisions: Vec<String>,
request: RequestId,
) -> IdeResult<Vec<LogDetails>> {
let project = LogAccess::new(host, window, repo.as_deref());
let cancel = host.begin_project_read_request(window, request);
let result = blocking(move || {
project.with_log_repo_at(repo.as_deref(), |r| {
revisions.iter().map(|id| r.log_details_cancellable(id, &cancel)).collect()
})
})
.await;
host.end_request(request);
result
}
pub async fn vcs_log_index(
host: &Host,
window: &str,
repo: Option<PathBuf>,
request: Option<RequestId>,
) -> IdeResult<LogIndexProgress> {
let project = LogAccess::new(host, window, repo.as_deref());
let cancel =
request.map_or_else(ide_core::CancellationToken::new, |id| host.begin_project_read_request(window, id));
let result = blocking(move || {
project.with_log_repo_at(repo.as_deref(), |r| {
r.enable_log_index(&project.cache);
if request.is_some() { r.index_log(&cancel) } else { Ok(r.log_index_progress()) }
})
})
.await;
if let Some(request) = request {
host.end_request(request);
}
result
}
pub async fn vcs_revision_children(
host: &Host,
window: &str,
repo: Option<PathBuf>,
revision: String,
directory: String,
) -> IdeResult<Vec<String>> {
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.revision_children(&revision, &directory))).await
}
pub async fn vcs_revision_files(
host: &Host,
window: &str,
repo: Option<PathBuf>,
revision: String,
) -> IdeResult<Vec<String>> {
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.revision_files(&revision))).await
}
pub async fn vcs_commit_files(
@@ -233,8 +454,8 @@ pub async fn vcs_commit_files(
repo: Option<PathBuf>,
commit: String,
) -> IdeResult<Vec<ChangedFile>> {
let project = host.project(window)?;
blocking(move || project.with_repo_at(repo.as_deref(), |r| r.commit_files(&commit))).await
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.commit_files(&commit))).await
}
/// Combined changes of several commits, `commits` oldest first.
@@ -244,13 +465,13 @@ pub async fn vcs_commits_changes(
repo: Option<PathBuf>,
commits: Vec<String>,
) -> IdeResult<Vec<RevisionChange>> {
let project = host.project(window)?;
blocking(move || project.with_repo_at(repo.as_deref(), |r| r.commits_changes(&commits))).await
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.commits_changes(&commits))).await
}
pub async fn vcs_branches(host: &Host, window: &str, repo: Option<PathBuf>) -> IdeResult<Vec<Branch>> {
let project = host.project(window)?;
blocking(move || project.with_repo_at(repo.as_deref(), |r| r.branches())).await
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.branches())).await
}
pub async fn vcs_stashes(host: &Host, window: &str, repo: Option<PathBuf>) -> IdeResult<Vec<Stash>> {
@@ -271,7 +492,7 @@ pub async fn vcs_add_to_ignore(
window: &str,
paths: Vec<PathBuf>,
destination: IgnoreDestination,
) -> IdeResult<Vec<String>> {
) -> IdeResult<Vec<ide_core::IpcPath>> {
host.require_trust(window)?;
let project = host.project(window)?;
blocking(move || {
@@ -288,7 +509,7 @@ pub async fn vcs_add_to_ignore(
watcher.reload_ignore_rules();
}
project.sink.emit(ide_core::IdeEvent::VcsChanged { root: project.root.clone().into() });
result.map(|files| files.into_iter().map(|p| p.to_string_lossy().into_owned()).collect())
result.map(|files| files.into_iter().map(Into::into).collect())
})
.await
}
@@ -605,8 +826,8 @@ pub async fn vcs_config_get(
repo: Option<PathBuf>,
key: String,
) -> IdeResult<Option<String>> {
let project = host.project(window)?;
blocking(move || project.with_repo_at(repo.as_deref(), |r| r.config_get(&key))).await
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.config_get(&key))).await
}
/// Local history operation that may take a while (hooks, large checkouts); git's
@@ -667,8 +888,8 @@ pub async fn vcs_create_tag(
}
pub async fn vcs_tags(host: &Host, window: &str, repo: Option<PathBuf>) -> IdeResult<Vec<String>> {
let project = host.project(window)?;
blocking(move || project.with_repo_at(repo.as_deref(), |r| r.tags())).await
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.tags())).await
}
/// Merge dialog: merges `branches` into the current branch.
@@ -704,8 +925,8 @@ pub async fn vcs_rebase_with(
}
pub async fn vcs_remotes(host: &Host, window: &str, repo: Option<PathBuf>) -> IdeResult<Vec<Remote>> {
let project = host.project(window)?;
blocking(move || project.with_repo_at(repo.as_deref(), |r| r.remotes())).await
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.remotes())).await
}
pub async fn vcs_edit_remote(host: &Host, window: &str, repo: Option<PathBuf>, edit: RemoteEdit) -> IdeResult<()> {
@@ -792,8 +1013,8 @@ pub async fn vcs_diff_with_worktree(
repo: Option<PathBuf>,
rev: String,
) -> IdeResult<Vec<ChangedFile>> {
let project = host.project(window)?;
blocking(move || project.with_repo_at(repo.as_deref(), |r| r.diff_with_worktree(&rev))).await
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.diff_with_worktree(&rev))).await
}
/// Files that differ between two revisions (the "N files updated" of Update Project).
@@ -804,8 +1025,8 @@ pub async fn vcs_diff_revisions(
from: String,
to: String,
) -> IdeResult<Vec<ChangedFile>> {
let project = host.project(window)?;
blocking(move || project.with_repo_at(repo.as_deref(), |r| r.diff_revisions(&from, &to))).await
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.diff_revisions(&from, &to))).await
}
/// The best common ancestor of `a` and `b` (`git merge-base`), if any: the New Pull
@@ -987,7 +1208,30 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("vcs_blame", |c| async move {
vcs_blame(&c.host, &c.window, c.arg("path")?, c.arg("text")?, c.arg("revision")?).await
});
d.add("vcs_block_history", |c| async move {
let (request, path, text, lines) = (c.arg("request")?, c.arg("path")?, c.arg("text")?, c.arg("lines")?);
vcs_block_history(&c.host, &c.window, request, path, text, lines).await
});
d.add("vcs_log", |c| async move { vcs_log(&c.host, &c.window, c.arg("repo")?, c.arg("query")?).await });
d.add("vcs_log_refs", |c| async move { vcs_log_refs(&c.host, &c.window, c.arg("repo")?).await });
d.add("vcs_log_users", |c| async move { vcs_log_users(&c.host, &c.window, c.arg("repo")?).await });
d.add("vcs_open_external_log", |c| async move { vcs_open_external_log(&c.host, &c.window, c.arg("path")?).await });
d.add("vcs_log_details", |c| async move {
vcs_log_details(&c.host, &c.window, c.arg("repo")?, c.arg("revision")?).await
});
d.add("vcs_log_details_batch", |c| async move {
vcs_log_details_batch(&c.host, &c.window, c.arg("repo")?, c.arg("revisions")?, c.arg("request")?).await
});
d.add(
"vcs_log_index",
|c| async move { vcs_log_index(&c.host, &c.window, c.arg("repo")?, c.arg("request")?).await },
);
d.add("vcs_revision_children", |c| async move {
vcs_revision_children(&c.host, &c.window, c.arg("repo")?, c.arg("revision")?, c.arg("directory")?).await
});
d.add("vcs_revision_files", |c| async move {
vcs_revision_files(&c.host, &c.window, c.arg("repo")?, c.arg("revision")?).await
});
d.add("vcs_commit_files", |c| async move {
vcs_commit_files(&c.host, &c.window, c.arg("repo")?, c.arg("commit")?).await
});
+11
View File
@@ -667,6 +667,17 @@ impl HostApi for PluginApi {
Ok(())
}
fn open_location(&self, client: Option<&str>, path: &Path, line: u32, column: u32) -> Result<(), HostError> {
let event = PluginEvent::OpenLocation {
plugin_id: self.id().to_owned(),
path: path.to_path_buf().into(),
line,
column,
};
self.shared.sink.emit(client, event);
Ok(())
}
fn setting(&self, root: Option<&Path>, key: &str) -> Option<String> {
let schema: Vec<super::ui_points::SettingSchema> = self
.shared
@@ -0,0 +1,207 @@
use super::icon_patterns::validate_pattern;
use ide_extension::validate_item_id;
use serde::{Deserialize, Serialize};
use std::collections::{BTreeMap, BTreeSet};
use std::path::PathBuf;
const MAX_GROUPS: usize = 64;
const MAX_CHOICES: usize = 16;
const MAX_CHOICE_OPTIONS: usize = 16;
const MAX_NAME_LEN: usize = 255;
pub(super) const DARK_SUFFIX: &str = "_dark";
#[derive(Deserialize)]
#[serde(untagged)]
pub(super) enum IconDefinition {
Path(PathBuf),
Variants(IconVariants),
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
pub(super) struct IconVariants {
light: PathBuf,
#[serde(default)]
dark: Option<PathBuf>,
}
impl IconDefinition {
pub(super) fn light(&self) -> &PathBuf {
match self {
IconDefinition::Path(p) => p,
IconDefinition::Variants(v) => &v.light,
}
}
pub(super) fn dark(&self) -> Option<&PathBuf> {
match self {
IconDefinition::Path(_) => None,
IconDefinition::Variants(v) => v.dark.as_ref(),
}
}
}
#[derive(Clone, Copy, Default, PartialEq, Eq, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub(super) enum Target {
#[default]
File,
Folder,
}
#[derive(Deserialize, Serialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub(super) struct Pattern {
pattern: String,
icon: String,
#[serde(default)]
target: Target,
#[serde(default)]
path: bool,
#[serde(default)]
ignore_case: bool,
#[serde(default)]
group: Option<String>,
#[serde(default)]
open_icon: Option<String>,
}
fn enabled_by_default() -> bool {
true
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub(super) struct Group {
pub(super) id: String,
name: String,
#[serde(default = "enabled_by_default")]
enabled: bool,
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub(super) struct ChoiceOption {
pub(super) id: String,
pub(super) name: String,
#[serde(default)]
overrides: BTreeMap<String, String>,
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub(super) struct Choice {
pub(super) id: String,
pub(super) name: String,
pub(super) default: String,
pub(super) options: Vec<ChoiceOption>,
}
#[derive(Default, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub(super) struct Extensions {
#[serde(default)]
pub(super) patterns: Vec<Pattern>,
#[serde(default)]
pub(super) overrides: BTreeMap<String, String>,
#[serde(default)]
pub(super) groups: Vec<Group>,
#[serde(default)]
pub(super) choices: Vec<Choice>,
#[serde(default)]
pub(super) accent: bool,
}
pub(super) fn check_icon_id(id: &str) -> Result<(), String> {
validate_item_id(id, "icon id")?;
if id.ends_with(DARK_SUFFIX) {
return Err(format!("icon id {id:?}: the {DARK_SUFFIX:?} suffix is reserved for dark variants"));
}
Ok(())
}
fn check_named(id: &str, name: &str, what: &str) -> Result<(), String> {
validate_item_id(id, what)?;
if name.trim().is_empty() || name.len() > MAX_NAME_LEN {
return Err(format!("{what} {id:?}: name must have 1 to {MAX_NAME_LEN} characters"));
}
Ok(())
}
fn check_unique<'a>(ids: impl Iterator<Item = &'a String>, what: &str) -> Result<(), String> {
let mut seen = BTreeSet::new();
match ids.into_iter().find(|id| !seen.insert(*id)) {
Some(id) => Err(format!("duplicate {what} {id:?}")),
None => Ok(()),
}
}
impl Extensions {
pub(super) fn mappings(&self) -> usize {
let chosen: usize = self.choices.iter().flat_map(|c| &c.options).map(|o| o.overrides.len()).sum();
self.patterns.len() + self.overrides.len() + chosen
}
pub(super) fn validate(&self, check_ref: &dyn Fn(&str, &str) -> Result<(), String>) -> Result<(), String> {
self.validate_groups()?;
for p in &self.patterns {
validate_pattern(&p.pattern)?;
check_ref(&p.icon, "patterns")?;
if let Some(group) = &p.group
&& !self.groups.iter().any(|g| &g.id == group)
{
return Err(format!("pattern {:?}: unknown group {group:?}", p.pattern));
}
if let Some(open) = &p.open_icon {
if p.target != Target::Folder {
return Err(format!("pattern {:?}: openIcon is only for folder patterns", p.pattern));
}
check_ref(open, "patterns")?;
}
}
check_overrides(&self.overrides, check_ref, "overrides")?;
for choice in &self.choices {
for option in &choice.options {
check_overrides(&option.overrides, check_ref, "choices")?;
}
}
Ok(())
}
fn validate_groups(&self) -> Result<(), String> {
if self.groups.len() > MAX_GROUPS || self.choices.len() > MAX_CHOICES {
return Err(format!("too many groups or choices (max {MAX_GROUPS} and {MAX_CHOICES})"));
}
for g in &self.groups {
check_named(&g.id, &g.name, "group")?;
}
check_unique(self.groups.iter().map(|g| &g.id), "group")?;
check_unique(self.choices.iter().map(|c| &c.id), "choice")?;
for choice in &self.choices {
check_named(&choice.id, &choice.name, "choice")?;
if choice.options.is_empty() || choice.options.len() > MAX_CHOICE_OPTIONS {
return Err(format!("choice {:?}: expected 1 to {MAX_CHOICE_OPTIONS} options", choice.id));
}
for option in &choice.options {
check_named(&option.id, &option.name, "choice option")?;
}
check_unique(choice.options.iter().map(|o| &o.id), "choice option")?;
if !choice.options.iter().any(|o| o.id == choice.default) {
return Err(format!("choice {:?}: default {:?} is not one of its options", choice.id, choice.default));
}
}
Ok(())
}
}
fn check_overrides(
overrides: &BTreeMap<String, String>,
check_ref: &dyn Fn(&str, &str) -> Result<(), String>,
what: &str,
) -> Result<(), String> {
for (name, icon) in overrides {
validate_item_id(name, "overridden icon name")?;
check_ref(icon, what)?;
}
Ok(())
}
@@ -0,0 +1,105 @@
use regex_syntax::hir::{Hir, HirKind};
pub(crate) const MAX_PATTERN_LEN: usize = 512;
const NAME_LEN: u64 = 256;
const MAX_COST: u64 = 4 * NAME_LEN * NAME_LEN;
const UNSUPPORTED: &[(&str, &str)] = &[
("\\A", "\\A"),
("\\z", "\\z"),
("\\Z", "\\Z"),
("\\p", "\\p"),
("\\P", "\\P"),
("[[:", "a POSIX class"),
("&&", "class intersection"),
("~~", "class symmetric difference"),
];
pub(crate) fn validate_pattern(pattern: &str) -> Result<(), String> {
if pattern.len() > MAX_PATTERN_LEN {
return Err(format!("pattern {pattern:?} is too long (max {MAX_PATTERN_LEN})"));
}
if let Some(construct) = unsupported_construct(pattern) {
return Err(format!("pattern {pattern:?}: {construct} is not supported"));
}
let hir = regex_syntax::Parser::new().parse(pattern).map_err(|e| format!("invalid pattern {pattern:?}: {e}"))?;
match cost(&hir) {
Err(why) => Err(format!("pattern {pattern:?}: {why} is not supported")),
Ok(c) if c > MAX_COST => Err(format!("pattern {pattern:?} is too complex: too many ways to match one name")),
Ok(_) => Ok(()),
}
}
fn unsupported_construct(pattern: &str) -> Option<&'static str> {
if pattern.match_indices("(?").any(|(at, _)| !pattern[at + 2..].starts_with(':')) {
return Some("an inline flag, lookaround or named group");
}
UNSUPPORTED.iter().find(|(needle, _)| pattern.contains(needle)).map(|(_, what)| *what)
}
fn cost(hir: &Hir) -> Result<u64, &'static str> {
match hir.kind() {
HirKind::Empty | HirKind::Literal(_) | HirKind::Class(_) | HirKind::Look(_) => Ok(1),
HirKind::Capture(c) => cost(&c.sub),
HirKind::Concat(items) => items.iter().try_fold(1u64, |acc, h| Ok(acc.saturating_mul(cost(h)?))),
HirKind::Alternation(items) => items.iter().try_fold(0u64, |acc, h| Ok(acc.saturating_add(cost(h)?))),
HirKind::Repetition(r) => match r.max {
None if matches!(r.sub.kind(), HirKind::Literal(_) | HirKind::Class(_)) => Ok(NAME_LEN),
None => Err("a repeated group (`(…)*`, `(…)+`)"),
Some(max) => Ok(cost(&r.sub)?.saturating_add(1).saturating_pow(max)),
},
}
}
#[cfg(test)]
mod tests {
use super::*;
fn err(pattern: &str) -> String {
validate_pattern(pattern).unwrap_err()
}
#[test]
fn accepts_typical_file_name_patterns() {
for pattern in [
r"^(architecture|ARCHITECTURE)(\.(md|rst|txt))?$",
r"(main|workflow|ci|release|build|config)\.ya?ml$",
r"^[a-z]+\.d\.ts$",
r"^\.?eslintrc(\.(js|cjs|json|ya?ml))?$",
r"^.*\.config\.[cm]?[jt]s$",
r"^.*-(test|spec)\..*$",
r"^[0-9]{4}-[0-9]{2}\.log$",
] {
assert!(validate_pattern(pattern).is_ok(), "{pattern}: {:?}", validate_pattern(pattern));
}
}
#[test]
fn rejects_patterns_that_backtrack_catastrophically() {
assert!(err(r"(a+)+$").contains("repeated group"));
assert!(err(r"(?:a*)*").contains("repeated group"));
assert!(err(r"^(\w|\w\w)*!$").contains("repeated group"));
assert!(err(r"^(a|aa)+$").contains("repeated group"));
assert!(err(r"^.*.*.*x$").contains("too complex"));
assert!(err(r"^(a?){30}a{30}$").contains("too complex"));
assert!(err(&format!("^{}b$", "(a|aa)".repeat(20))).contains("too complex"));
assert!(err(r"x{0,40}").contains("too complex"));
}
#[test]
fn rejects_constructs_javascript_reads_differently() {
assert!(err(r"(?i)readme").contains("not supported"));
assert!(err(r"(?P<n>a)").contains("not supported"));
assert!(err(r"\Areadme").contains("not supported"));
assert!(err(r"readme\z").contains("not supported"));
assert!(err(r"\p{L}").contains("not supported"));
assert!(err(r"[[:alpha:]]").contains("not supported"));
assert!(err(r"[a-z&&[^x]]").contains("not supported"));
}
#[test]
fn rejects_invalid_and_oversized_patterns() {
assert!(err("(").contains("invalid pattern"));
assert!(err(&"a".repeat(MAX_PATTERN_LEN + 1)).contains("too long"));
}
}
@@ -0,0 +1,185 @@
use super::icon_extensions::Extensions;
use serde::{Deserialize, Serialize};
const MAX_SECTIONS: usize = 16;
const MAX_ITEMS: usize = 32;
const MAX_TEXT_LEN: usize = 255;
const MAX_HINT_LEN: usize = 512;
const MAX_COLUMNS: u8 = 4;
#[derive(Clone, Copy, PartialEq, Eq, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub(super) enum SettingOption {
HideFiles,
HideFolders,
HideUiIcons,
HidePsiIcons,
HollowFolders,
Groups,
Choice,
Accent,
Monochrome,
Saturation,
Opacity,
IconSize,
RowHeight,
}
#[derive(Deserialize, Serialize)]
#[serde(untagged)]
pub(super) enum SettingDefault {
Number(u8),
Color(String),
}
#[derive(Deserialize, Serialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub(super) struct SettingItem {
option: SettingOption,
#[serde(default)]
label: Option<String>,
#[serde(default)]
hint: Option<String>,
#[serde(default)]
invert: bool,
#[serde(default)]
choice: Option<String>,
#[serde(default)]
columns: Option<u8>,
#[serde(default)]
default: Option<SettingDefault>,
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub(super) struct SettingsSection {
title: String,
items: Vec<SettingItem>,
}
#[derive(Serialize)]
struct ChoiceSummary<'a> {
id: &'a str,
name: &'a str,
default: &'a str,
options: Vec<OptionSummary<'a>>,
}
#[derive(Serialize)]
struct OptionSummary<'a> {
id: &'a str,
name: &'a str,
}
fn check_text(text: &str, max: usize, what: &str) -> Result<(), String> {
if text.trim().is_empty() || text.len() > max {
return Err(format!("settings: {what} must have 1 to {max} characters"));
}
Ok(())
}
fn range(option: SettingOption) -> Option<(u8, u8)> {
match option {
SettingOption::Saturation => Some((0, 20)),
SettingOption::Opacity => Some((10, 100)),
SettingOption::IconSize => Some((12, 32)),
SettingOption::RowHeight => Some((16, 48)),
_ => None,
}
}
fn is_color(text: &str) -> bool {
text.len() == 7 && text.starts_with('#') && text[1..].chars().all(|c| c.is_ascii_hexdigit())
}
fn is_flag(option: SettingOption) -> bool {
matches!(
option,
SettingOption::HideFiles
| SettingOption::HideFolders
| SettingOption::HideUiIcons
| SettingOption::HidePsiIcons
| SettingOption::HollowFolders
)
}
impl SettingItem {
fn validate(&self, ext: &Extensions) -> Result<(), String> {
let named = !matches!(self.option, SettingOption::Groups | SettingOption::Choice);
match &self.label {
Some(label) => check_text(label, MAX_TEXT_LEN, "label")?,
None if named => return Err("settings: label is required".into()),
None => {}
}
if let Some(hint) = &self.hint {
check_text(hint, MAX_HINT_LEN, "hint")?;
}
if self.invert && !is_flag(self.option) {
return Err("settings: invert is only for on/off options".into());
}
if self.columns.is_some_and(|c| c == 0 || c > MAX_COLUMNS)
|| (self.columns.is_some() && self.option != SettingOption::Groups)
{
return Err(format!("settings: columns are 1 to {MAX_COLUMNS}, for groups only"));
}
match (self.option, &self.choice) {
(SettingOption::Choice, Some(id)) if !ext.choices.iter().any(|c| &c.id == id) => {
return Err(format!("settings: unknown choice {id:?}"));
}
(SettingOption::Choice, None) => return Err("settings: choice items name their choice".into()),
(SettingOption::Choice, Some(_)) | (_, None) => {}
(_, Some(_)) => return Err("settings: only choice items name a choice".into()),
}
if self.option == SettingOption::Accent && !ext.accent {
return Err("settings: the accent option needs \"accent\": true".into());
}
self.validate_default()
}
fn validate_default(&self) -> Result<(), String> {
let colored = matches!(self.option, SettingOption::Accent | SettingOption::Monochrome);
match (range(self.option), &self.default) {
(Some((min, max)), Some(SettingDefault::Number(n))) if (min..=max).contains(n) => Ok(()),
(Some((min, max)), _) => Err(format!("settings: a default from {min} to {max} is required")),
(None, Some(SettingDefault::Color(c))) if colored && is_color(c) => Ok(()),
(None, _) if colored => Err("settings: a #rrggbb default is required".into()),
(None, None) => Ok(()),
(None, Some(_)) => Err("settings: this option takes no default".into()),
}
}
}
pub(super) fn validate(sections: &[SettingsSection], ext: &Extensions) -> Result<(), String> {
if sections.len() > MAX_SECTIONS {
return Err(format!("settings: more than {MAX_SECTIONS} sections"));
}
for section in sections {
check_text(&section.title, MAX_TEXT_LEN, "title")?;
if section.items.len() > MAX_ITEMS {
return Err(format!("settings: more than {MAX_ITEMS} items in {:?}", section.title));
}
for item in &section.items {
item.validate(ext)?;
}
}
Ok(())
}
pub(super) fn summary(sections: &[SettingsSection], ext: &Extensions) -> serde_json::Value {
let choices: Vec<ChoiceSummary> = ext
.choices
.iter()
.map(|c| ChoiceSummary {
id: &c.id,
name: &c.name,
default: &c.default,
options: c.options.iter().map(|o| OptionSummary { id: &o.id, name: &o.name }).collect(),
})
.collect();
serde_json::json!({
"groups": ext.groups,
"choices": choices,
"accent": ext.accent,
"settings": sections,
})
}
+238 -9
View File
@@ -24,6 +24,8 @@
//! asset; the client fetches the mapping and the icons only when the theme is
//! selected, so installed icon themes cost nothing at start-up.
use super::icon_extensions::{Choice, DARK_SUFFIX, Extensions, Group, IconDefinition, Pattern, check_icon_id};
use super::icon_settings::{self, SettingsSection};
use ide_extension::{Package, UiDecl, check_relative, decode, validate_item_id};
use serde::Deserialize;
use std::collections::{BTreeMap, BTreeSet};
@@ -45,7 +47,7 @@ struct Decl {
#[derive(Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
struct ThemeFile {
icon_definitions: BTreeMap<String, PathBuf>,
icon_definitions: BTreeMap<String, IconDefinition>,
#[serde(default)]
file: Option<String>,
#[serde(default)]
@@ -53,11 +55,25 @@ struct ThemeFile {
#[serde(default)]
folder_expanded: Option<String>,
#[serde(default)]
folder_open: Option<String>,
#[serde(default)]
file_extensions: BTreeMap<String, String>,
#[serde(default)]
file_names: BTreeMap<String, String>,
#[serde(default)]
folder_names: BTreeMap<String, String>,
#[serde(default)]
patterns: Vec<Pattern>,
#[serde(default)]
overrides: BTreeMap<String, String>,
#[serde(default)]
groups: Vec<Group>,
#[serde(default)]
choices: Vec<Choice>,
#[serde(default)]
accent: bool,
#[serde(default)]
settings: Vec<SettingsSection>,
}
/// Asset id of a theme's mapping (JSON).
@@ -65,6 +81,10 @@ pub fn map_asset(theme: &str) -> String {
format!("iconTheme/{theme}/map")
}
pub fn settings_asset(theme: &str) -> String {
format!("iconTheme/{theme}/settings")
}
/// Asset id prefix of a theme's SVGs (`…/svg/<icon id>`).
pub fn svg_prefix(theme: &str) -> String {
format!("iconTheme/{theme}/svg/")
@@ -87,12 +107,20 @@ pub(super) fn icon_theme(pkg: &Package, decl: toml::Value) -> Result<UiDecl, Str
return Err(format!("icon theme {:?} has an empty label", d.id));
}
let src = pkg.read_text(&d.path)?;
let theme: ThemeFile = serde_json::from_str(&src).map_err(|e| format!("{}: {e}", d.path.display()))?;
let mut theme: ThemeFile = serde_json::from_str(&src).map_err(|e| format!("{}: {e}", d.path.display()))?;
let extensions = Extensions {
patterns: std::mem::take(&mut theme.patterns),
overrides: std::mem::take(&mut theme.overrides),
groups: std::mem::take(&mut theme.groups),
choices: std::mem::take(&mut theme.choices),
accent: theme.accent,
};
let at = |what: &str| format!("{} ({what})", d.path.display());
if theme.icon_definitions.len() > MAX_ICONS {
return Err(format!("{}: more than {MAX_ICONS} icons", d.path.display()));
}
let mappings = theme.file_extensions.len() + theme.file_names.len() + theme.folder_names.len();
let mappings =
theme.file_extensions.len() + theme.file_names.len() + theme.folder_names.len() + extensions.mappings();
if mappings > MAX_MAPPINGS {
return Err(format!("{}: more than {MAX_MAPPINGS} mappings", d.path.display()));
}
@@ -100,7 +128,17 @@ pub(super) fn icon_theme(pkg: &Package, decl: toml::Value) -> Result<UiDecl, Str
let check_ref = |icon: &str, what: &str| -> Result<(), String> {
if defined.contains(icon) { Ok(()) } else { Err(format!("{}: unknown icon {icon:?}", at(what))) }
};
for (what, icon) in [("file", &theme.file), ("folder", &theme.folder), ("folderExpanded", &theme.folder_expanded)] {
for icon in theme.icon_definitions.keys() {
check_icon_id(icon)?;
}
extensions.validate(&check_ref)?;
icon_settings::validate(&theme.settings, &extensions).map_err(|e| format!("{}: {e}", d.path.display()))?;
for (what, icon) in [
("file", &theme.file),
("folder", &theme.folder),
("folderExpanded", &theme.folder_expanded),
("folderOpen", &theme.folder_open),
] {
if let Some(icon) = icon {
check_ref(icon, what)?;
}
@@ -125,20 +163,31 @@ pub(super) fn icon_theme(pkg: &Package, decl: toml::Value) -> Result<UiDecl, Str
let mut assets = Vec::with_capacity(theme.icon_definitions.len() + 1);
let mut files = vec![d.path.clone()];
for (icon, path) in &theme.icon_definitions {
validate_item_id(icon, "icon id")?;
let clean = read_icon(pkg, path)?;
assets.push((format!("{}{icon}", svg_prefix(&d.id)), clean));
files.push(path.clone());
for (icon, def) in &theme.icon_definitions {
let dark = def.dark().map(|path| (path, DARK_SUFFIX));
for (path, suffix) in std::iter::once((def.light(), "")).chain(dark) {
let clean = read_icon(pkg, path)?;
assets.push((format!("{}{icon}{suffix}", svg_prefix(&d.id)), clean));
files.push(path.clone());
}
}
let map = serde_json::json!({
"file": theme.file,
"folder": theme.folder,
"folderExpanded": theme.folder_expanded,
"folderOpen": theme.folder_open,
"fileExtensions": file_extensions,
"fileNames": file_names,
"folderNames": folder_names,
"patterns": extensions.patterns,
"overrides": extensions.overrides,
"groups": extensions.groups,
"choices": extensions.choices,
"accent": extensions.accent,
});
if !theme.settings.is_empty() {
assets.push((settings_asset(&d.id), icon_settings::summary(&theme.settings, &extensions).to_string()));
}
assets.push((map_asset(&d.id), map.to_string()));
let json = serde_json::json!({ "id": d.id, "label": d.label, "icons": theme.icon_definitions.len() });
let mut out = UiDecl::new(d.id.clone(), d.label, json);
@@ -612,4 +661,184 @@ mod tests {
assert!(icon_theme(&pkg, decl("evil.json")).unwrap_err().contains("evil.svg"));
assert!(icon_theme(&pkg, decl("../x.json")).is_err());
}
const SVG: &str = r#"<svg viewBox="0 0 16 16"><path d="M0 0h16v16z"/></svg>"#;
fn theme_package(theme: &str) -> (tempfile::TempDir, Package) {
let tmp = tempfile::tempdir().unwrap();
let dir = tmp.path();
std::fs::create_dir_all(dir.join("icons")).unwrap();
for icon in ["file", "folder", "folder-src", "folder-src-open", "rust", "rust-dark", "tests", "arrow"] {
std::fs::write(dir.join(format!("icons/{icon}.svg")), SVG).unwrap();
}
std::fs::write(dir.join("theme.json"), theme).unwrap();
let manifest = ide_extension::Manifest::parse(
"id = \"p\"\nname = \"P\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.0\"",
)
.unwrap();
let pkg = Package::new(manifest, dir.to_path_buf());
(tmp, pkg)
}
fn load(theme: &str) -> Result<UiDecl, String> {
let (_tmp, pkg) = theme_package(theme);
let mut t = toml::Table::new();
t.insert("id".into(), "m".into());
t.insert("label".into(), "M".into());
t.insert("path".into(), "theme.json".into());
icon_theme(&pkg, toml::Value::Table(t))
}
fn with(extra: &str) -> String {
format!(
r#"{{"iconDefinitions":{{"file":"icons/file.svg","folder":"icons/folder.svg",
"folder-src":"icons/folder-src.svg","folder-src-open":"icons/folder-src-open.svg",
"rust":{{"light":"icons/rust.svg","dark":"icons/rust-dark.svg"}},
"tests":"icons/tests.svg","arrow":"icons/arrow.svg"}},
"file":"file","folder":"folder"{extra}}}"#
)
}
const EXTENSIONS: &str = r#","patterns":[
{"pattern":"^.*\\.rs$","icon":"rust","ignoreCase":true},
{"pattern":"/tests?/","icon":"tests","path":true,"group":"tests"},
{"pattern":"^src$","icon":"folder-src","target":"folder","openIcon":"folder-src-open"}],
"overrides":{"chevron-right":"arrow","symbol-class":"rust"},
"groups":[{"id":"tests","name":"Tests","enabled":false}],
"choices":[{"id":"arrows","name":"Arrows","default":"plain","options":[
{"id":"plain","name":"Default"},{"id":"material","name":"Material","overrides":{"chevron-down":"arrow"}}]}],
"accent":true,"folderOpen":"folder-src-open""#;
fn map_of(d: &UiDecl) -> serde_json::Value {
serde_json::from_str(&d.assets.iter().find(|(k, _)| *k == map_asset("m")).unwrap().1).unwrap()
}
#[test]
fn theme_extensions_reach_the_mapping() {
let d = load(&with(EXTENSIONS)).unwrap();
let map = map_of(&d);
assert_eq!(
map["patterns"],
serde_json::json!([
{"pattern":"^.*\\.rs$","icon":"rust","target":"file","path":false,"ignoreCase":true,"group":null,"openIcon":null},
{"pattern":"/tests?/","icon":"tests","target":"file","path":true,"ignoreCase":false,"group":"tests","openIcon":null},
{"pattern":"^src$","icon":"folder-src","target":"folder","path":false,"ignoreCase":false,"group":null,"openIcon":"folder-src-open"}
])
);
assert_eq!(map["overrides"], serde_json::json!({"chevron-right":"arrow","symbol-class":"rust"}));
assert_eq!(map["groups"], serde_json::json!([{"id":"tests","name":"Tests","enabled":false}]));
assert_eq!(map["choices"][0]["default"], "plain");
assert_eq!(map["choices"][0]["options"][1]["overrides"], serde_json::json!({"chevron-down":"arrow"}));
assert_eq!(map["choices"][0]["options"][0]["overrides"], serde_json::json!({}));
assert_eq!(map["accent"], true);
assert_eq!(map["folderOpen"], "folder-src-open");
assert_eq!(d.json["icons"], 7);
assert!(d.assets.iter().any(|(k, _)| *k == format!("{}rust", svg_prefix("m"))));
assert!(d.assets.iter().any(|(k, _)| *k == format!("{}rust_dark", svg_prefix("m"))));
assert!(d.files.iter().any(|f| f.ends_with("icons/rust-dark.svg")));
}
#[test]
fn themes_without_extensions_keep_their_mapping() {
let map = map_of(&load(&with("")).unwrap());
assert_eq!(map["patterns"], serde_json::json!([]));
assert_eq!(map["overrides"], serde_json::json!({}));
assert_eq!(map["groups"], serde_json::json!([]));
assert_eq!(map["choices"], serde_json::json!([]));
assert_eq!(map["accent"], false);
}
#[test]
fn theme_extensions_are_validated() {
let cases = [
(r#","patterns":[{"pattern":"x","icon":"nope"}]"#, "unknown icon"),
(r#","patterns":[{"pattern":"(a+)+$","icon":"rust"}]"#, "repeated group"),
(r#","patterns":[{"pattern":"(","icon":"rust"}]"#, "invalid pattern"),
(r#","patterns":[{"pattern":"x","icon":"rust","group":"nope"}]"#, "unknown group"),
(r#","patterns":[{"pattern":"x","icon":"rust","openIcon":"tests"}]"#, "openIcon"),
(r#","patterns":[{"pattern":"x","icon":"folder","target":"folder","openIcon":"nope"}]"#, "unknown icon"),
(r#","patterns":[{"pattern":"x","icon":"rust","priority":1}]"#, "unknown field"),
(r#","overrides":{"chevron-right":"nope"}"#, "unknown icon"),
(r#","folderOpen":"nope""#, "unknown icon"),
(r#","overrides":{"Chevron":"rust"}"#, "invalid overridden icon"),
(r#","groups":[{"id":"a","name":" "}]"#, "name"),
(r#","groups":[{"id":"a","name":"A"},{"id":"a","name":"B"}]"#, "duplicate"),
(r#","choices":[{"id":"c","name":"C","default":"x","options":[{"id":"y","name":"Y"}]}]"#, "default"),
(r#","choices":[{"id":"c","name":"C","default":"y","options":[]}]"#, "options"),
(
r#","choices":[{"id":"c","name":"C","default":"y","options":[{"id":"y","name":"Y","overrides":{"a":"nope"}}]}]"#,
"unknown icon",
),
];
for (extra, needle) in cases {
let err = load(&with(extra)).err().unwrap_or_else(|| panic!("accepted: {extra}"));
assert!(err.contains(needle), "{extra}: {err}");
}
let dark_id = r#"{"iconDefinitions":{"x_dark":"icons/rust.svg"}}"#;
assert!(load(dark_id).unwrap_err().contains("_dark"));
}
const SETTINGS: &str = r##","settings":[
{"title":"Toggle","items":[
{"option":"hideFiles","label":"Enable file icons","hint":"Files","invert":true},
{"option":"hollowFolders","label":"Hollow"}]},
{"title":"Packs","items":[{"option":"groups","columns":2}]},
{"title":"Other","items":[
{"option":"choice","choice":"arrows"},
{"option":"accent","label":"Accent","default":"#009688"},
{"option":"saturation","label":"Saturation","default":10}]}]"##;
fn settings_of(d: &UiDecl) -> Option<serde_json::Value> {
d.assets.iter().find(|(k, _)| *k == settings_asset("m")).map(|(_, v)| serde_json::from_str(v).unwrap())
}
#[test]
fn theme_settings_get_their_own_asset() {
let d = load(&with(&format!("{EXTENSIONS}{SETTINGS}"))).unwrap();
let s = settings_of(&d).unwrap();
assert_eq!(s["accent"], true);
assert_eq!(s["groups"], serde_json::json!([{"id":"tests","name":"Tests","enabled":false}]));
assert_eq!(
s["choices"],
serde_json::json!([{"id":"arrows","name":"Arrows","default":"plain","options":[
{"id":"plain","name":"Default"},{"id":"material","name":"Material"}]}])
);
assert_eq!(s["settings"][0]["title"], "Toggle");
assert_eq!(
s["settings"][0]["items"][0],
serde_json::json!({"option":"hideFiles","label":"Enable file icons","hint":"Files","invert":true,
"choice":null,"columns":null,"default":null})
);
assert_eq!(s["settings"][2]["items"][1]["default"], "#009688");
assert_eq!(s["settings"][2]["items"][2]["default"], 10);
assert!(settings_of(&load(&with(EXTENSIONS)).unwrap()).is_none());
}
#[test]
fn theme_settings_are_validated() {
let section = |items: &str| with(&format!(r#"{EXTENSIONS},"settings":[{{"title":"T","items":[{items}]}}]"#));
let cases = [
(section(r#"{"option":"nope","label":"X"}"#), "unknown variant"),
(section(r#"{"option":"hideFiles"}"#), "label"),
(section(r#"{"option":"hideFiles","label":" "}"#), "label"),
(section(r#"{"option":"choice","choice":"nope"}"#), "unknown choice"),
(section(r#"{"option":"choice"}"#), "choice"),
(section(r#"{"option":"groups","columns":9}"#), "columns"),
(section(r#"{"option":"saturation","label":"S"}"#), "default"),
(section(r#"{"option":"saturation","label":"S","default":50}"#), "default"),
(section(r##"{"option":"opacity","label":"O","default":"#000000"}"##), "default"),
(section(r#"{"option":"monochrome","label":"M","default":"red"}"#), "default"),
(section(r#"{"option":"saturation","label":"S","default":5,"invert":true}"#), "invert"),
(section(r#"{"option":"hideFiles","label":"F","extra":1}"#), "unknown field"),
(with(&format!(r#"{EXTENSIONS},"settings":[{{"title":"","items":[]}}]"#)), "title"),
(
with(r##","settings":[{"title":"T","items":[{"option":"accent","label":"A","default":"#000000"}]}]"##),
"accent",
),
];
for (theme, needle) in cases {
let err = load(&theme).err().unwrap_or_else(|| panic!("accepted: {theme}"));
assert!(err.contains(needle), "{theme}: {err}");
}
}
}
+874
View File
@@ -0,0 +1,874 @@
//! IntelliJ theme plugins, read for import as OpenIDEA theme plugins: a
//! Marketplace download (a `.jar`, or a `.zip` holding the plugin's
//! `lib/*.jar`), an installed plugin's folder or a loose `.theme.json`; and the
//! theme plugins of the JetBrains IDEs on this machine, with the theme each IDE
//! is set to (`options/laf.xml`).
//!
//! Nothing is converted here: the UI maps IntelliJ keys onto design tokens
//! (`src/features/plugins/intellijTheme.ts`, shared with `pnpm
//! theme:import-intellij`) and [`super::PluginHost::stage_package`] writes the
//! result for the install page, which validates it like any package.
use ide_core::{IdeError, IdeResult, IpcPath};
use quick_xml::{Reader, XmlVersion, events::Event};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::ffi::OsString;
use std::io::{Cursor, Read, Seek};
use std::path::{Path, PathBuf};
/// A descriptor, theme or editor scheme larger than this is not one.
const MAX_FILE: u64 = 4 << 20;
/// A plugin jar read into memory from a Marketplace zip.
const MAX_JAR: u64 = 64 << 20;
/// Jars of one plugin looked into.
const MAX_JARS: usize = 64;
/// All the jars read into memory from one Marketplace zip.
const MAX_NESTED: u64 = 256 << 20;
/// Descriptors read for one plugin: `plugin.xml` and those it includes.
const MAX_DESCRIPTORS: usize = 64;
const PLUGIN_XML: &str = "META-INF/plugin.xml";
/// A theme plugin's themes, as read from its files.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct IntellijThemeSource {
/// The plugin's `<name>`, or the theme's for a loose file.
pub name: String,
pub plugin_id: Option<String>,
pub version: Option<String>,
pub vendor: Option<String>,
pub themes: Vec<IntellijThemeFile>,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct IntellijThemeFile {
/// The `themeProvider` id (what `laf.xml` names), or the file name without `.theme.json`.
pub id: String,
/// The `.theme.json`, comments removed.
pub theme: String,
/// The editor scheme (`.icls` XML) its `editorScheme` names, when the plugin has it.
pub editor_scheme: Option<String>,
}
/// A theme of a plugin installed in a JetBrains IDE.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct InstalledIntellijTheme {
/// The IDE and its version: `IntelliJ IDEA 2025.2`.
pub ide: String,
/// The plugin's name.
pub plugin: String,
/// What to import: the plugin's folder or jar.
pub path: IpcPath,
pub theme_id: String,
pub theme_name: String,
pub dark: bool,
/// The IDE is set to this theme.
pub active: bool,
}
/// Reads the themes of an IntelliJ theme plugin (a `.jar`, a Marketplace
/// `.zip`, an installed plugin's folder) or of a `.theme.json`.
pub fn read_source(path: &Path) -> IdeResult<IntellijThemeSource> {
read(path).map_err(IdeError::invalid)
}
/// The theme plugins of the JetBrains IDEs (and Android Studio) of this user,
/// the themes their IDEs are set to first.
pub fn installed_themes() -> Vec<InstalledIntellijTheme> {
let Some(home) = dirs::home_dir() else { return Vec::new() };
let layout = if cfg!(target_os = "macos") {
Layout::Mac
} else if cfg!(windows) {
Layout::Windows
} else {
Layout::Xdg
};
find_installed(&ide_dirs(layout, &home, |key| std::env::var_os(key)))
}
fn read(path: &Path) -> Result<IntellijThemeSource, String> {
if path.extension().is_some_and(|e| e.eq_ignore_ascii_case("json")) {
return loose_theme(path);
}
let mut jars = plugin_jars(path)?;
let plugin = descriptor(&mut jars)?;
let shown = || path.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
if plugin.themes.is_empty() {
return Err(format!("{} is not a theme plugin: its plugin.xml declares no themeProvider", shown()));
}
let mut themes = Vec::new();
let mut problem = None;
for (id, theme_path) in &plugin.themes {
match plugin_theme(&mut jars, id, theme_path) {
Ok(theme) => themes.push(theme),
Err(e) => problem = problem.or(Some(e)),
}
}
if themes.is_empty() {
return Err(problem.unwrap_or_else(|| format!("{} has no readable theme", shown())));
}
Ok(IntellijThemeSource {
name: plugin.name.unwrap_or_else(shown),
plugin_id: plugin.id,
version: plugin.version,
vendor: plugin.vendor,
themes,
})
}
fn plugin_theme(jars: &mut [Jar], id: &str, path: &str) -> Result<IntellijThemeFile, String> {
let bytes = find(jars, entry_name(path), MAX_FILE)?.ok_or_else(|| format!("{path} is missing from the plugin"))?;
let (theme, fields) = theme_json(&bytes).map_err(|e| format!("{path}: {e}"))?;
let editor_scheme = match string_field(&fields, "editorScheme") {
Some(scheme) => find(jars, entry_name(scheme), MAX_FILE)?.map(|b| text(&b)),
None => None,
};
Ok(IntellijThemeFile { id: id.to_owned(), theme, editor_scheme })
}
/// A `.theme.json` with the editor scheme beside it (its `editorScheme` names a path in the plugin's jar).
fn loose_theme(path: &Path) -> Result<IntellijThemeSource, String> {
let (theme, fields) = theme_json(&read_file(path)?).map_err(|e| format!("{}: {e}", path.display()))?;
// Any JSON object parses: an IntelliJ theme has UI colours or an editor scheme.
let scheme_ref = string_field(&fields, "editorScheme");
if !fields.get("ui").is_some_and(serde_json::Value::is_object) && scheme_ref.is_none() {
return Err(format!("{} is not an IntelliJ theme: it has no \"ui\" or \"editorScheme\"", path.display()));
}
let editor_scheme = scheme_ref
.and_then(|p| Path::new(p).file_name().map(|n| path.with_file_name(n)))
.filter(|p| p.is_file())
.map(|p| read_file(&p).map(|b| text(&b)))
.transpose()?;
let file = path.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
let id = file.strip_suffix(".theme.json").or_else(|| file.strip_suffix(".json")).unwrap_or(&file).to_owned();
Ok(IntellijThemeSource {
name: string_field(&fields, "name").map_or_else(|| id.clone(), str::to_owned),
plugin_id: None,
version: None,
vendor: string_field(&fields, "author").map(str::to_owned),
themes: vec![IntellijThemeFile { id, theme, editor_scheme }],
})
}
type JsonObject = serde_json::Map<String, serde_json::Value>;
/// The theme JSON, comments removed, and its fields, if it is an object.
fn theme_json(bytes: &[u8]) -> Result<(String, JsonObject), String> {
let json = super::theme::strip_json_comments(&text(bytes));
match serde_json::from_str::<serde_json::Value>(&json) {
Ok(serde_json::Value::Object(fields)) => Ok((json, fields)),
Ok(_) => Err("not a theme: expected a JSON object".into()),
Err(e) => Err(format!("invalid theme JSON: {e}")),
}
}
fn string_field<'a>(fields: &'a JsonObject, key: &str) -> Option<&'a str> {
fields.get(key)?.as_str().filter(|s| !s.trim().is_empty())
}
/// `/themes/x.theme.json` → `themes/x.theme.json`.
fn entry_name(path: &str) -> &str {
path.trim().trim_start_matches('/')
}
fn text(bytes: &[u8]) -> String {
let bytes = bytes.strip_prefix(b"\xEF\xBB\xBF").unwrap_or(bytes);
String::from_utf8_lossy(bytes).into_owned()
}
fn read_file(path: &Path) -> Result<Vec<u8>, String> {
let shown = |e: std::io::Error| format!("{}: {e}", path.display());
let file = std::fs::File::open(path).map_err(shown)?;
let mut buf = Vec::new();
file.take(MAX_FILE + 1).read_to_end(&mut buf).map_err(shown)?;
if buf.len() as u64 > MAX_FILE {
return Err(format!("{} is too large for a theme", path.display()));
}
Ok(buf)
}
trait ReadSeek: Read + Seek {}
impl<T: Read + Seek> ReadSeek for T {}
type Jar = zip::ZipArchive<Box<dyn ReadSeek>>;
fn open_jar(path: &Path) -> Result<Jar, String> {
let file = std::fs::File::open(path).map_err(|e| format!("{}: {e}", path.display()))?;
let reader: Box<dyn ReadSeek> = Box::new(std::io::BufReader::new(file));
zip::ZipArchive::new(reader).map_err(|e| format!("{} is not a jar or zip file ({e})", path.display()))
}
/// The jars of the plugin at `path`: a jar, a Marketplace zip (`<plugin>/lib/*.jar`)
/// or an installed plugin's folder (`lib/*.jar`).
fn plugin_jars(path: &Path) -> Result<Vec<Jar>, String> {
let not_plugin = || format!("{} is not an IntelliJ plugin: no {PLUGIN_XML} and no lib/*.jar", path.display());
if path.is_dir() {
let mut found: Vec<PathBuf> = [path.join("lib"), path.join("lib").join("modules")]
.iter()
.flat_map(|dir| std::fs::read_dir(dir).into_iter().flatten().flatten())
.map(|e| e.path())
.filter(|p| p.is_file() && is_jar(p))
.collect();
found.sort();
let jars: Vec<Jar> = found.iter().take(MAX_JARS).filter_map(|p| open_jar(p).ok()).collect();
return if jars.is_empty() { Err(not_plugin()) } else { Ok(jars) };
}
let mut archive = open_jar(path)?;
if archive.index_for_name(PLUGIN_XML).is_some() {
return Ok(vec![archive]);
}
let jars = nested_jars(&mut archive, MAX_JAR, MAX_NESTED);
if jars.is_empty() { Err(not_plugin()) } else { Ok(jars) }
}
/// The `lib` jars of a Marketplace zip, smallest first (a theme's own jar is small)
/// while they fit `max_total`; library jars too large or not jars at all are skipped.
fn nested_jars(archive: &mut Jar, max_jar: u64, max_total: u64) -> Vec<Jar> {
let mut nested: Vec<(u64, String)> = entry_names(archive)
.into_iter()
.filter(|n| is_lib_jar(n))
.filter_map(|n| Some((archive.by_name(&n).ok()?.size(), n)))
.filter(|(size, _)| *size <= max_jar)
.collect();
nested.sort();
let (mut jars, mut loaded) = (Vec::new(), 0u64);
for (size, name) in nested.into_iter().take(MAX_JARS) {
if loaded.saturating_add(size) > max_total {
break;
}
let Ok(Some(bytes)) = read_entry(archive, &name, max_jar) else { continue };
loaded += size;
let reader: Box<dyn ReadSeek> = Box::new(Cursor::new(bytes));
if let Ok(jar) = zip::ZipArchive::new(reader) {
jars.push(jar);
}
}
jars
}
fn entry_names(jar: &Jar) -> Vec<String> {
jar.file_names().filter_map(Result::ok).map(|n| n.into_owned()).collect()
}
fn is_jar(path: &Path) -> bool {
path.extension().is_some_and(|e| e.eq_ignore_ascii_case("jar"))
}
/// `lib/x.jar`, `<plugin>/lib/x.jar` or `<plugin>/lib/modules/x.jar` in a Marketplace zip.
fn is_lib_jar(name: &str) -> bool {
let parts: Vec<&str> = name.split('/').collect();
let n = parts.len();
name.to_ascii_lowercase().ends_with(".jar")
&& (2..=4).contains(&n)
&& (parts[n - 2] == "lib" || (n >= 3 && parts[n - 2] == "modules" && parts[n - 3] == "lib"))
}
fn read_entry(jar: &mut Jar, name: &str, max: u64) -> Result<Option<Vec<u8>>, String> {
let Ok(file) = jar.by_name(name) else { return Ok(None) };
if file.is_dir() {
return Ok(None);
}
if file.size() > max {
return Err(format!("{name} is too large"));
}
let mut buf = Vec::with_capacity(file.size() as usize);
file.take(max.saturating_add(1)).read_to_end(&mut buf).map_err(|e| format!("{name}: {e}"))?;
if buf.len() as u64 > max {
return Err(format!("{name} is too large"));
}
Ok(Some(buf))
}
fn find(jars: &mut [Jar], name: &str, max: u64) -> Result<Option<Vec<u8>>, String> {
for jar in jars.iter_mut() {
if let Some(bytes) = read_entry(jar, name, max)? {
return Ok(Some(bytes));
}
}
Ok(None)
}
/// What the plugin's descriptors say: `<idea-plugin>`'s `id`, `name`, `version`,
/// `vendor`, and its `themeProvider`s (id, path in the jar).
#[derive(Debug, Default, PartialEq)]
struct Descriptor {
id: Option<String>,
name: Option<String>,
version: Option<String>,
vendor: Option<String>,
themes: Vec<(String, String)>,
}
/// Reads `META-INF/plugin.xml` and the descriptors it includes (`<xi:include href>`), in any jar.
fn descriptor(jars: &mut [Jar]) -> Result<Descriptor, String> {
let main = find(jars, PLUGIN_XML, MAX_FILE)
.ok()
.flatten()
.ok_or_else(|| format!("not an IntelliJ plugin: no {PLUGIN_XML}"))?;
let mut out = Descriptor::default();
let mut includes = Vec::new();
parse_descriptor(&text(&main), true, &mut out, &mut includes).map_err(|e| format!("{PLUGIN_XML}: {e}"))?;
let mut read = std::collections::HashSet::from([PLUGIN_XML.to_owned()]);
while let Some(href) = includes.pop() {
let name = include_entry(&href);
if read.len() >= MAX_DESCRIPTORS || !read.insert(name.clone()) {
continue;
}
let Ok(Some(bytes)) = find(jars, &name, MAX_FILE) else { continue };
let _ = parse_descriptor(&text(&bytes), false, &mut out, &mut includes);
}
let mut seen = std::collections::HashSet::new();
out.themes.retain(|(id, _)| seen.insert(id.clone()));
Ok(out)
}
/// An `<xi:include href>` as a jar entry: `/META-INF/x.xml` from the jar's root, `x.xml` beside plugin.xml.
fn include_entry(href: &str) -> String {
let href = href.trim();
href.strip_prefix('/').map_or_else(|| format!("META-INF/{href}"), str::to_owned)
}
fn parse_descriptor(xml: &str, main: bool, out: &mut Descriptor, includes: &mut Vec<String>) -> Result<(), String> {
let mut reader = Reader::from_str(xml);
let mut stack: Vec<String> = Vec::new();
let mut text = String::new();
loop {
let event = reader.read_event().map_err(|e| e.to_string())?;
match event {
Event::Start(ref tag) | Event::Empty(ref tag) => {
let name = tag.local_name().as_ref().to_owned();
if name == "include" {
let href = tag.attributes().flatten().find(|a| a.key.local_name().as_ref() == "href");
if let Some(href) = href.and_then(|a| a.normalized_value(XmlVersion::Implicit1_0).ok()) {
includes.push(href.into_owned());
}
}
if name == "themeProvider" {
let (mut id, mut path) = (None, None);
for a in tag.attributes() {
let a = a.map_err(|e| e.to_string())?;
let value = a.normalized_value(XmlVersion::Implicit1_0).map_err(|e| e.to_string())?;
match a.key.local_name().as_ref() {
"id" => id = Some(value.trim().to_owned()),
"path" => path = Some(value.trim().to_owned()),
_ => {}
}
}
if let (Some(id), Some(path)) = (id, path)
&& !id.is_empty()
&& !path.is_empty()
{
out.themes.push((id, path));
}
}
if matches!(event, Event::Start(_)) {
stack.push(name);
text.clear();
}
}
Event::Text(t) => text.push_str(&t.xml_content(XmlVersion::Implicit1_0)),
Event::CData(t) => text.push_str(t.as_ref()),
Event::GeneralRef(r) => {
let name = r.as_ref();
let escaped = format!("&{name};");
text.push_str(&quick_xml::escape::unescape(&escaped).map_err(|e| e.to_string())?);
}
Event::End(_) => {
let name = stack.pop().ok_or("unbalanced element")?;
// Direct children of <idea-plugin>.
if main && stack.len() == 1 && stack[0] == "idea-plugin" {
let value = text.trim();
let slot = match name.as_str() {
"id" => Some(&mut out.id),
"name" => Some(&mut out.name),
"version" => Some(&mut out.version),
"vendor" => Some(&mut out.vendor),
_ => None,
};
if let Some(slot) = slot
&& slot.is_none()
&& !value.is_empty()
{
*slot = Some(value.to_owned());
}
}
text.clear();
}
Event::Eof => return if stack.is_empty() { Ok(()) } else { Err("unclosed element".into()) },
_ => {}
}
}
}
/// Where an OS keeps the IDEs' settings and plugins.
#[derive(Clone, Copy, Debug)]
enum Layout {
/// `~/Library/Application Support/<vendor>/<IDE>`, plugins in its `plugins/`.
Mac,
/// `%APPDATA%\<vendor>\<IDE>`, plugins in its `plugins\`.
Windows,
/// `$XDG_CONFIG_HOME/<vendor>/<IDE>`, plugins in `$XDG_DATA_HOME/<vendor>/<IDE>`.
Xdg,
}
#[derive(Debug, PartialEq)]
struct IdeDir {
label: String,
product: String,
version: (u32, u32),
config: PathBuf,
plugins: PathBuf,
}
/// The settings folders of JetBrains IDEs and Android Studio (2020.1 and later), newest first.
fn ide_dirs(layout: Layout, home: &Path, env: impl Fn(&str) -> Option<OsString>) -> Vec<IdeDir> {
let var = |key: &str| env(key).filter(|v| !v.is_empty()).map(PathBuf::from);
let (config_root, data_root) = match layout {
Layout::Mac => {
let root = home.join("Library").join("Application Support");
(root.clone(), root)
}
Layout::Windows => {
let root = var("APPDATA").unwrap_or_else(|| home.join("AppData").join("Roaming"));
(root.clone(), root)
}
Layout::Xdg => (
var("XDG_CONFIG_HOME").unwrap_or_else(|| home.join(".config")),
var("XDG_DATA_HOME").unwrap_or_else(|| home.join(".local").join("share")),
),
};
let mut out = Vec::new();
for vendor in ["JetBrains", "Google"] {
for entry in std::fs::read_dir(config_root.join(vendor)).into_iter().flatten().flatten() {
let name = entry.file_name().to_string_lossy().into_owned();
let Some((product, version, shown)) = product_dir(&name) else { continue };
let config = entry.path();
if !config.is_dir() {
continue;
}
let plugins = match layout {
Layout::Xdg => data_root.join(vendor).join(&name),
Layout::Mac | Layout::Windows => config.join("plugins"),
};
out.push(IdeDir {
label: format!("{} {shown}", product_label(product)),
product: product.to_owned(),
version,
config,
plugins,
});
}
}
out.sort_by(|a, b| b.version.cmp(&a.version).then_with(|| a.label.cmp(&b.label)));
out
}
/// `IntelliJIdea2025.2` → (`IntelliJIdea`, (2025, 2), `2025.2`).
fn product_dir(name: &str) -> Option<(&str, (u32, u32), &str)> {
let at = name.find(|c: char| c.is_ascii_digit())?;
let (product, version) = name.split_at(at);
if product.is_empty() || !product.chars().all(|c| c.is_ascii_alphabetic()) {
return None;
}
let (year, minor) = version.split_once('.')?;
let ok = |s: &str| !s.is_empty() && s.len() <= 4 && s.chars().all(|c| c.is_ascii_digit());
if year.len() != 4 || !ok(year) || !ok(minor) {
return None;
}
Some((product, (year.parse().ok()?, minor.parse().ok()?), version))
}
fn product_label(product: &str) -> &str {
match product {
"IntelliJIdea" => "IntelliJ IDEA",
"IdeaIC" => "IntelliJ IDEA Community Edition",
"PyCharmCE" => "PyCharm Community Edition",
"AndroidStudio" => "Android Studio",
"AndroidStudioPreview" => "Android Studio Preview",
other => other,
}
}
/// The theme ids `options/laf.xml` selects: the current one, and with "Sync with OS" the preferred light and dark.
fn active_theme_ids(config: &Path) -> Vec<String> {
let Ok(bytes) = read_file(&config.join("options").join("laf.xml")) else { return Vec::new() };
let xml = text(&bytes);
let mut reader = Reader::from_str(&xml);
let (mut ids, mut preferred, mut autodetect) = (Vec::new(), Vec::new(), false);
loop {
match reader.read_event() {
Ok(Event::Start(tag) | Event::Empty(tag)) => {
let attr = |key: &str| {
tag.attributes()
.flatten()
.find(|a| a.key.as_ref() == key)
.and_then(|a| a.normalized_value(XmlVersion::Implicit1_0).ok().map(|v| v.trim().to_owned()))
};
match tag.local_name().as_ref() {
"component" if attr("name").as_deref() == Some("LafManager") => {
autodetect = attr("autodetect").as_deref() == Some("true");
}
"laf" => ids.extend(attr("themeId")),
"preferred-light-laf" | "preferred-dark-laf" => preferred.extend(attr("themeId")),
_ => {}
}
}
Ok(Event::Eof) | Err(_) => break,
_ => {}
}
}
if autodetect {
ids.extend(preferred);
}
ids.retain(|id| !id.is_empty());
ids
}
fn find_installed(ides: &[IdeDir]) -> Vec<InstalledIntellijTheme> {
let mut out = Vec::new();
for ide in ides {
let active = active_theme_ids(&ide.config);
let mut plugins: Vec<PathBuf> = std::fs::read_dir(&ide.plugins)
.into_iter()
.flatten()
.flatten()
.map(|e| e.path())
.filter(|p| p.is_dir() || (p.is_file() && is_jar(p)))
.collect();
plugins.sort();
for plugin in plugins {
let Ok(mut jars) = plugin_jars(&plugin) else { continue };
let Ok(found) = descriptor(&mut jars) else { continue };
for (id, path) in &found.themes {
let Ok(Some(bytes)) = find(&mut jars, entry_name(path), MAX_FILE) else { continue };
let Ok((_, fields)) = theme_json(&bytes) else { continue };
let file = || plugin.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
out.push(InstalledIntellijTheme {
ide: ide.label.clone(),
plugin: found.name.clone().unwrap_or_else(file),
path: plugin.clone().into(),
theme_id: id.clone(),
theme_name: string_field(&fields, "name").unwrap_or(id).to_owned(),
dark: fields.get("dark").and_then(serde_json::Value::as_bool).unwrap_or(true),
active: active.contains(id),
});
}
}
}
// Stable: the IDEs' order (newest first) within each group.
out.sort_by_key(|t| !t.active);
out
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
const PLUGIN: &str = r#"<idea-plugin>
<id>com.example.deepsea</id>
<name>Deep Sea &amp; Friends</name>
<version>2.3</version>
<vendor email="[email protected]">Someone</vendor>
<description><![CDATA[<b>Dark</b> themes]]></description>
<depends>com.intellij.modules.platform</depends>
<extensions defaultExtensionNs="com.intellij">
<themeProvider id="deep-sea-id" path="/themes/DeepSea.theme.json"/>
<themeProvider id="shallow-id" path="themes/Shallow.theme.json" />
</extensions>
</idea-plugin>"#;
const DEEP: &str = r##"{
// comment
"name": "Deep Sea",
"dark": true,
"editorScheme": "/themes/DeepSea.xml",
"ui": { "*": { "background": "#101820" } }
}"##;
const SHALLOW: &str = r#"{ "name": "Shallow", "dark": false, "ui": {} }"#;
const SCHEME: &str = r#"<scheme name="Deep Sea" version="142" parent_scheme="Darcula"/>"#;
fn jar_bytes(entries: &[(&str, &[u8])]) -> Vec<u8> {
let mut zip = zip::ZipWriter::new(Cursor::new(Vec::new()));
for (name, data) in entries {
zip.start_file(*name, zip::write::SimpleFileOptions::default()).unwrap();
zip.write_all(data).unwrap();
}
zip.finish().unwrap().into_inner()
}
fn theme_jar() -> Vec<u8> {
jar_bytes(&[
("META-INF/MANIFEST.MF", b"Manifest-Version: 1.0\n"),
(PLUGIN_XML, PLUGIN.as_bytes()),
("themes/DeepSea.theme.json", DEEP.as_bytes()),
("themes/DeepSea.xml", SCHEME.as_bytes()),
("themes/Shallow.theme.json", SHALLOW.as_bytes()),
])
}
fn write(path: &Path, bytes: &[u8]) {
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
std::fs::write(path, bytes).unwrap();
}
fn expected_source() -> IntellijThemeSource {
IntellijThemeSource {
name: "Deep Sea & Friends".into(),
plugin_id: Some("com.example.deepsea".into()),
version: Some("2.3".into()),
vendor: Some("Someone".into()),
themes: vec![
IntellijThemeFile {
id: "deep-sea-id".into(),
theme: super::super::theme::strip_json_comments(DEEP),
editor_scheme: Some(SCHEME.into()),
},
IntellijThemeFile { id: "shallow-id".into(), theme: SHALLOW.into(), editor_scheme: None },
],
}
}
#[test]
fn reads_a_theme_jar_with_its_editor_scheme() {
let tmp = tempfile::tempdir().unwrap();
let jar = tmp.path().join("deep-sea.jar");
write(&jar, &theme_jar());
let source = read_source(&jar).unwrap();
assert_eq!(source, expected_source());
assert!(!source.themes[0].theme.contains("comment"));
}
#[test]
fn reads_a_marketplace_zip_and_an_installed_folder() {
let tmp = tempfile::tempdir().unwrap();
let other = jar_bytes(&[("META-INF/MANIFEST.MF", b"x"), ("a/B.class", b"\xCA\xFE")]);
let zip = tmp.path().join("DeepSea-2.3.zip");
write(&zip, &jar_bytes(&[("DeepSea/lib/a-util.jar", &other), ("DeepSea/lib/deep-sea.jar", &theme_jar())]));
assert_eq!(read_source(&zip).unwrap(), expected_source());
let dir = tmp.path().join("plugins").join("DeepSea");
write(&dir.join("lib").join("deep-sea.jar"), &theme_jar());
write(&dir.join("lib").join("a-util.jar"), &other);
assert_eq!(read_source(&dir).unwrap(), expected_source());
}
#[test]
fn skips_library_jars_it_cannot_or_need_not_read() {
let tmp = tempfile::tempdir().unwrap();
// Stored, not deflated: as large in the zip as its content.
let mut big = zip::ZipWriter::new(Cursor::new(Vec::new()));
let stored = zip::write::SimpleFileOptions::default().compression_method(zip::CompressionMethod::Stored);
big.start_file("lib/Data.class", stored).unwrap();
big.write_all(&[b'x'; 8192]).unwrap();
let big = big.finish().unwrap().into_inner();
let zip = tmp.path().join("DeepSea.zip");
write(
&zip,
&jar_bytes(&[
("DeepSea/lib/a-broken.jar", b"not a jar"),
("DeepSea/lib/b-big.jar", &big),
("DeepSea/lib/deep-sea.jar", &theme_jar()),
]),
);
assert_eq!(read_source(&zip).unwrap(), expected_source());
// The broken jar is skipped; a jar over the size limit, or past the total
// (smallest first), is not read, while the theme's small jar still is.
let mut archive = open_jar(&zip).unwrap();
let theme_len = theme_jar().len() as u64;
assert_eq!(nested_jars(&mut archive, u64::MAX, u64::MAX).len(), 2);
let one = nested_jars(&mut archive, theme_len + 64, u64::MAX);
assert_eq!(one.len(), 1);
assert!(one[0].index_for_name(PLUGIN_XML).is_some());
assert_eq!(nested_jars(&mut archive, u64::MAX, theme_len + 64).len(), 1);
assert!(nested_jars(&mut archive, 8, u64::MAX).is_empty());
}
#[test]
fn reads_a_loose_theme_json_with_the_scheme_beside_it() {
let tmp = tempfile::tempdir().unwrap();
let theme = tmp.path().join("DeepSea.theme.json");
write(&theme, DEEP.as_bytes());
write(&tmp.path().join("DeepSea.xml"), SCHEME.as_bytes());
let source = read_source(&theme).unwrap();
assert_eq!(source.name, "Deep Sea");
assert_eq!(source.themes.len(), 1);
assert_eq!(source.themes[0].id, "DeepSea");
assert_eq!(source.themes[0].editor_scheme.as_deref(), Some(SCHEME));
std::fs::remove_file(tmp.path().join("DeepSea.xml")).unwrap();
assert_eq!(read_source(&theme).unwrap().themes[0].editor_scheme, None);
}
#[test]
fn says_what_is_wrong_with_other_files() {
let tmp = tempfile::tempdir().unwrap();
let not_zip = tmp.path().join("notes.jar");
write(&not_zip, b"hello");
assert!(read_source(&not_zip).unwrap_err().message.contains("not a jar or zip"));
let other_plugin = tmp.path().join("tools.jar");
let xml = "<idea-plugin><name>Tools</name><actions/></idea-plugin>";
write(&other_plugin, &jar_bytes(&[(PLUGIN_XML, xml.as_bytes())]));
assert!(read_source(&other_plugin).unwrap_err().message.contains("not a theme plugin"));
let package = tmp.path().join("package.zip");
write(&package, &jar_bytes(&[("plugin.toml", b"id = \"x\"")]));
assert!(read_source(&package).unwrap_err().message.contains("not an IntelliJ plugin"));
let other = tmp.path().join("package.json");
write(&other, br#"{ "name": "app", "version": "1.0.0" }"#);
assert!(read_source(&other).unwrap_err().message.contains("is not an IntelliJ theme"));
let broken = tmp.path().join("broken.theme.json");
write(&broken, b"{ \"name\": ");
assert!(read_source(&broken).unwrap_err().message.contains("invalid theme JSON"));
let missing = tmp.path().join("missing.jar");
let xml = r#"<idea-plugin><name>M</name><themeProvider id="m" path="/m.theme.json"/></idea-plugin>"#;
write(&missing, &jar_bytes(&[(PLUGIN_XML, xml.as_bytes())]));
assert!(read_source(&missing).unwrap_err().message.contains("/m.theme.json is missing"));
}
#[test]
fn finds_theme_providers_in_included_descriptors() {
let tmp = tempfile::tempdir().unwrap();
let jar = tmp.path().join("split.jar");
let main = r#"<idea-plugin xmlns:xi="http://www.w3.org/2001/XInclude"><name>Split</name>
<xi:include href="/META-INF/themes.xml"/></idea-plugin>"#;
let part = r#"<idea-plugin xmlns:xi="http://www.w3.org/2001/XInclude"><extensions defaultExtensionNs="com.intellij">
<themeProvider id="s" path="/s.theme.json"/></extensions><name>Not this</name>
<xi:include href="more.xml"/><xi:include href="/META-INF/themes.xml"/></idea-plugin>"#;
let more = r#"<idea-plugin><themeProvider id="m" path="/m.theme.json"/></idea-plugin>"#;
let unrelated = r#"<idea-plugin><themeProvider id="x" path="/x.theme.json"/></idea-plugin>"#;
write(
&jar,
&jar_bytes(&[
("META-INF/themes.xml", part.as_bytes()),
("META-INF/more.xml", more.as_bytes()),
("META-INF/unrelated.xml", unrelated.as_bytes()),
(PLUGIN_XML, main.as_bytes()),
("s.theme.json", br#"{"name":"S"}"#),
("m.theme.json", br#"{"name":"M"}"#),
("x.theme.json", br#"{"name":"X"}"#),
]),
);
let source = read_source(&jar).unwrap();
assert_eq!(source.name, "Split");
// Only what plugin.xml includes (relative or from the root, once each) counts.
assert_eq!(source.themes.iter().map(|t| t.id.as_str()).collect::<Vec<_>>(), ["s", "m"]);
}
#[test]
fn parses_ide_folder_names() {
assert_eq!(product_dir("IntelliJIdea2025.2"), Some(("IntelliJIdea", (2025, 2), "2025.2")));
assert_eq!(product_dir("PyCharmCE2023.10"), Some(("PyCharmCE", (2023, 10), "2023.10")));
assert_eq!(product_dir("AndroidStudio2024.3"), Some(("AndroidStudio", (2024, 3), "2024.3")));
for name in
["Toolbox", "consentOptions", "IntelliJIdea2025", "Idea-2025.1", "2025.1", "Rider25.1", "Go Land2025.1"]
{
assert_eq!(product_dir(name), None, "{name}");
}
}
#[test]
fn locates_the_settings_and_plugins_of_each_os() {
let tmp = tempfile::tempdir().unwrap();
let home = tmp.path();
let mk = |p: PathBuf| {
std::fs::create_dir_all(&p).unwrap();
p
};
let mac = mk(home.join("Library/Application Support/JetBrains/IntelliJIdea2025.2"));
mk(home.join("Library/Application Support/JetBrains/Toolbox"));
let dirs = ide_dirs(Layout::Mac, home, |_| None);
assert_eq!(dirs.len(), 1);
assert_eq!(dirs[0].label, "IntelliJ IDEA 2025.2");
assert_eq!((dirs[0].config.clone(), dirs[0].plugins.clone()), (mac.clone(), mac.join("plugins")));
let appdata = mk(home.join("Roaming"));
let win = mk(appdata.join("JetBrains/PyCharm2024.3"));
let studio = mk(appdata.join("Google/AndroidStudio2025.1"));
let env = |k: &str| (k == "APPDATA").then(|| appdata.clone().into_os_string());
let dirs = ide_dirs(Layout::Windows, home, env);
assert_eq!(
dirs.iter().map(|d| d.label.as_str()).collect::<Vec<_>>(),
["Android Studio 2025.1", "PyCharm 2024.3"]
);
assert_eq!(dirs[1].plugins, win.join("plugins"));
assert_eq!(dirs[0].config, studio);
let config = mk(home.join(".config/JetBrains/IdeaIC2024.1"));
mk(home.join(".config/JetBrains/GoLand2025.2"));
let dirs = ide_dirs(Layout::Xdg, home, |_| None);
assert_eq!(dirs.iter().map(|d| d.product.as_str()).collect::<Vec<_>>(), ["GoLand", "IdeaIC"]);
assert_eq!(dirs[1].label, "IntelliJ IDEA Community Edition 2024.1");
assert_eq!(dirs[1].config, config);
assert_eq!(dirs[1].plugins, home.join(".local/share/JetBrains/IdeaIC2024.1"));
let xdg_data = home.join("data");
let env = |k: &str| (k == "XDG_DATA_HOME").then(|| xdg_data.clone().into_os_string());
assert_eq!(ide_dirs(Layout::Xdg, home, env)[1].plugins, xdg_data.join("JetBrains/IdeaIC2024.1"));
}
#[test]
fn reads_the_selected_theme_from_laf_xml() {
let tmp = tempfile::tempdir().unwrap();
let laf = tmp.path().join("options/laf.xml");
write(
&laf,
br#"<application>
<component name="LafManager">
<laf class-name="com.intellij.ide.ui.laf.UIThemeBasedLookAndFeelInfo" themeId="deep-sea-id" />
<preferred-light-laf themeId="shallow-id" />
</component>
</application>"#,
);
assert_eq!(active_theme_ids(tmp.path()), ["deep-sea-id"]);
let synced = std::fs::read_to_string(&laf)
.unwrap()
.replace(r#"name="LafManager""#, r#"name="LafManager" autodetect="true""#);
write(&laf, synced.as_bytes());
assert_eq!(active_theme_ids(tmp.path()), ["deep-sea-id", "shallow-id"]);
assert!(active_theme_ids(&tmp.path().join("missing")).is_empty());
}
#[test]
fn lists_installed_theme_plugins_the_ides_current_first() {
let tmp = tempfile::tempdir().unwrap();
let config = tmp.path().join("IntelliJIdea2025.2");
let plugins = config.join("plugins");
write(&plugins.join("DeepSea/lib/deep-sea.jar"), &theme_jar());
write(
&plugins.join("Tools/lib/tools.jar"),
&jar_bytes(&[(PLUGIN_XML, b"<idea-plugin><name>T</name></idea-plugin>")]),
);
write(&plugins.join("broken.jar"), b"not a jar");
write(
&config.join("options/laf.xml"),
br#"<application><component name="LafManager"><laf themeId="shallow-id"/></component></application>"#,
);
let ide = IdeDir {
label: "IntelliJ IDEA 2025.2".into(),
product: "IntelliJIdea".into(),
version: (2025, 2),
config: config.clone(),
plugins: plugins.clone(),
};
let found = find_installed(&[ide]);
let theme = |id: &str, name: &str, dark: bool, active: bool| InstalledIntellijTheme {
ide: "IntelliJ IDEA 2025.2".into(),
plugin: "Deep Sea & Friends".into(),
path: plugins.join("DeepSea").into(),
theme_id: id.into(),
theme_name: name.into(),
dark,
active,
};
assert_eq!(found, [theme("shallow-id", "Shallow", false, true), theme("deep-sea-id", "Deep Sea", true, false)]);
assert!(find_installed(&[]).is_empty());
}
}
+181 -22
View File
@@ -3,11 +3,14 @@
//! plugins, and revocations. The host keeps the last index it read
//! (`market-index.json`), checks signatures against it, and disables what it
//! revokes — at start-up and on every refresh (the desktop refreshes daily).
//! Release builds pin the official publisher to the keys they were built
//! with, so no index can sign `openidea.*` plugins with another key.
use super::{PluginHost, PluginSource, PluginStatus};
use super::{PackageFile, PluginEvent, PluginHost, PluginSource, PluginStatus, PluginUpdate, install};
use ide_core::{IdeError, IdeResult};
use ide_extension::Manifest;
use ide_extension::package::{self, Index, IndexEntry, PackageSignature, SIGNATURE_FILE};
use ide_extension::{MANIFEST_FILE, Manifest};
use semver::Version;
use serde::{Deserialize, Serialize};
use std::path::{Path, PathBuf};
@@ -15,11 +18,18 @@ use std::path::{Path, PathBuf};
/// the app, next to the signed packages of the official plugins.
pub const OFFICIAL_INDEX: &str = "https://github.com/l0ng-ai/openidea/releases/latest/download/plugins-index.json";
/// The publisher of the official plugins.
pub const OFFICIAL_PUBLISHER: &str = "openidea";
/// The last index read, in the plugins directory.
pub const INDEX_FILE: &str = "market-index.json";
const DOWNLOADS: &str = ".downloads";
const MAX_INDEX: usize = 16 << 20;
const MAX_PACKAGE: usize = 256 << 20;
const MAX_STAGED_FILES: usize = 256;
const MAX_STAGED_BYTES: usize = 16 << 20;
/// `moved` links followed before giving up (a loop, or a misconfiguration).
const MAX_MOVES: usize = 3;
/// A marketplace entry with what is installed.
#[derive(Clone, Debug, Serialize, Deserialize, specta::Type)]
@@ -29,6 +39,32 @@ pub struct MarketPlugin {
pub entry: IndexEntry,
/// Installed version, if any.
pub installed: Option<String>,
/// Whether this IDE can load `entry` (the newest version it can, else the newest).
pub compatible: bool,
}
impl MarketPlugin {
/// A version this IDE can load, newer than the one installed.
pub fn is_update(&self) -> bool {
let newer = |installed: &String| match (Version::parse(&self.entry.version), Version::parse(installed)) {
(Ok(offered), Ok(installed)) => offered > installed,
_ => false,
};
self.compatible && self.installed.as_ref().is_some_and(newer)
}
}
/// The official publisher's keys this IDE was built with
/// (`OPENIDEA_OFFICIAL_PLUGIN_KEYS`: base64, comma-separated). Development
/// builds have none: the index decides.
pub(super) fn built_in_official_keys() -> Vec<String> {
let keys = option_env!("OPENIDEA_OFFICIAL_PLUGIN_KEYS").unwrap_or_default();
keys.split(',').map(str::trim).filter(|k| !k.is_empty()).map(str::to_owned).collect()
}
fn pinned(mut index: Index, official_keys: &[String]) -> Index {
index.pin_publisher(OFFICIAL_PUBLISHER, "OpenIDEA", official_keys);
index
}
/// Reads `url`: `http(s)://…`, `file://…` or a local path.
@@ -56,6 +92,33 @@ pub fn read_url(url: &str, max: usize) -> Result<Vec<u8>, String> {
Ok(bytes)
}
/// The index at `url`, following `moved`, with downloads made absolute against
/// where it was read from.
fn read_index(url: &str) -> IdeResult<Index> {
let mut url = url.to_owned();
for _ in 0..=MAX_MOVES {
let bytes = read_url(&url, MAX_INDEX).map_err(IdeError::unavailable)?;
let text = String::from_utf8(bytes).map_err(|_| IdeError::invalid("the index is not UTF-8"))?;
let mut index = Index::parse(&text).map_err(IdeError::invalid)?;
match index.moved.take().filter(|m| !m.trim().is_empty()) {
Some(next) => {
let next = resolve(&url, &next);
if url.starts_with("https://") && !next.starts_with("https://") {
return Err(IdeError::invalid(format!("the marketplace moved to {next}, which isn't https")));
}
url = next;
}
None => {
for e in &mut index.plugins {
e.download = resolve(&url, &e.download);
}
return Ok(index);
}
}
}
Err(IdeError::unavailable(format!("the marketplace index moved more than {MAX_MOVES} times")))
}
/// `href` relative to the index at `base`: a URL, or a local path (with the
/// platform's separators, e.g. `C:\\market\\index.json`).
pub fn resolve(base: &str, href: &str) -> String {
@@ -75,9 +138,16 @@ pub fn resolve(base: &str, href: &str) -> String {
}
impl PluginHost {
pub(super) fn load_market_index(dir: &Path) -> Option<Index> {
pub(super) fn load_market_index(dir: &Path, official_keys: &[String]) -> Option<Index> {
let text = std::fs::read_to_string(dir.join(INDEX_FILE)).ok()?;
Index::parse(&text).map_err(|e| tracing::warn!("{INDEX_FILE}: {e}")).ok()
let index = Index::parse(&text).map_err(|e| tracing::warn!("{INDEX_FILE}: {e}")).ok()?;
Some(pinned(index, official_keys))
}
/// Pins the official publisher to `keys` (base64) instead of the built-in
/// ones, from the next index read.
pub fn set_official_keys(&self, keys: Vec<String>) {
*self.official_keys.write() = keys;
}
/// The marketplace index URL: the user's, else the default (none: no marketplace).
@@ -95,14 +165,26 @@ impl PluginHost {
self.save_state();
}
/// Reads the index again, keeps it, and applies its revocations.
/// The marketplace's plugins: read again when `refresh`, else the last index
/// read. Reads it when none has been read yet (first run, nothing cached).
pub fn market_listing(&self, refresh: bool) -> IdeResult<Vec<MarketPlugin>> {
if refresh || (self.market.read().is_none() && self.market_url().is_some()) {
self.refresh_market()
} else {
Ok(self.market_plugins())
}
}
/// Reads the index again, keeps it, and applies its revocations. Announces
/// updates of installed plugins it hasn't announced yet.
pub fn refresh_market(&self) -> IdeResult<Vec<MarketPlugin>> {
let url = self.market_url().ok_or_else(|| IdeError::unavailable("no plugin marketplace is configured"))?;
let bytes = read_url(&url, MAX_INDEX).map_err(IdeError::unavailable)?;
let text = String::from_utf8(bytes).map_err(|_| IdeError::invalid("the index is not UTF-8"))?;
let index = Index::parse(&text).map_err(IdeError::invalid)?;
let _ = std::fs::write(self.plugins_dir.join(INDEX_FILE), &text);
*self.market.write() = Some(index);
let index = read_index(&url)?;
if let Ok(json) = serde_json::to_string(&index) {
let _ = std::fs::write(self.plugins_dir.join(INDEX_FILE), json);
}
let newer = index.is_newer().then_some(index.format);
*self.market.write() = Some(pinned(index, &self.official_keys.read()));
// Trust may have changed: who signed what, what is revoked.
let installed: Vec<_> =
self.plugins.read().values().filter(|p| p.source == PluginSource::Installed).cloned().collect();
@@ -111,7 +193,41 @@ impl PluginHost {
plugin.meta.lock().publisher = publisher;
}
self.apply_revocations();
Ok(self.market_plugins())
if let Some(format) = newer {
return Err(IdeError::unavailable(format!(
"the marketplace needs a newer OpenIDEA (index format {format}); its revocations still apply"
)));
}
let plugins = self.market_plugins();
self.announce_updates(&plugins);
Ok(plugins)
}
/// Tells the windows about updates once per set of versions: a later
/// refresh offering the same ones says nothing.
fn announce_updates(&self, plugins: &[MarketPlugin]) {
let updates: Vec<PluginUpdate> = plugins
.iter()
.filter(|p| p.is_update())
.map(|p| PluginUpdate {
id: p.entry.id.clone(),
name: p.entry.name.clone(),
version: p.entry.version.clone(),
})
.collect();
let key: Vec<String> = updates.iter().map(|u| format!("{}@{}", u.id, u.version)).collect();
if std::mem::replace(&mut *self.announced_updates.lock(), key.clone()) != key && !updates.is_empty() {
self.emit(None, PluginEvent::UpdatesAvailable { updates });
}
}
/// Whether the package in `src` would replace the installed `id`'s copy
/// with one signed by another key than that copy, without that key's
/// endorsement: the user confirms the change of signer.
pub(super) fn signer_changed(&self, id: &str, src: &Path) -> bool {
let Some(current) = install::active_dir(&self.plugins_dir.join(id)) else { return false };
let market = self.market.read();
!package::signer_continues(market.as_ref(), read_signature(&current).as_ref(), read_signature(src).as_ref())
}
/// The plugins of the last index read.
@@ -119,16 +235,14 @@ impl PluginHost {
let Some(index) = self.market.read().clone() else { return Vec::new() };
let plugins = self.plugins.read();
index
.plugins
.iter()
.filter(|e| index.revocation(&e.id, &e.version).is_none())
.cloned()
.map(|entry| {
.offered()
.into_iter()
.map(|(entry, compatible)| {
let installed = plugins
.get(&entry.id)
.filter(|p| p.source == PluginSource::Installed)
.map(|p| p.manifest().version.clone());
MarketPlugin { entry, installed }
MarketPlugin { entry: entry.clone(), installed, compatible }
})
.collect()
}
@@ -139,12 +253,18 @@ impl PluginHost {
pub fn download_market_plugin(&self, id: &str) -> IdeResult<PathBuf> {
let url = self.market_url().ok_or_else(|| IdeError::unavailable("no plugin marketplace is configured"))?;
let index = self.market.read().clone().ok_or_else(|| IdeError::unavailable("refresh the marketplace first"))?;
let entry = index
.plugins
.iter()
.find(|e| e.id == id)
.cloned()
let (entry, compatible) = index
.offered()
.into_iter()
.find(|(e, _)| e.id == id)
.map(|(e, ok)| (e.clone(), ok))
.ok_or_else(|| IdeError::not_found(format!("{id} is not in the marketplace")))?;
if !compatible {
return Err(IdeError::unavailable(format!(
"{} {} needs plugin API {}: update OpenIDEA",
entry.name, entry.version, entry.engine
)));
}
let bytes = read_url(&resolve(&url, &entry.download), MAX_PACKAGE).map_err(IdeError::unavailable)?;
let downloads = self.plugins_dir.join(DOWNLOADS);
std::fs::create_dir_all(&downloads)?;
@@ -182,6 +302,45 @@ impl PluginHost {
Ok(dir)
}
/// Writes a package the UI generated (an imported IntelliJ theme) into a
/// folder next to the downloads, named after `name` (its plugin id); returns the folder.
/// The install page validates it like any other package.
pub fn stage_package(&self, name: &str, files: &[PackageFile]) -> IdeResult<PathBuf> {
ide_extension::validate_item_id(name, "package name").map_err(IdeError::invalid)?;
if !files.iter().any(|f| f.path == MANIFEST_FILE) {
return Err(IdeError::invalid(format!("a package needs a {MANIFEST_FILE}")));
}
let total: usize = files.iter().map(|f| f.text.len()).sum();
if files.len() > MAX_STAGED_FILES || total > MAX_STAGED_BYTES {
return Err(IdeError::invalid("the package is too large"));
}
let mut seen = std::collections::HashSet::new();
for f in files {
let valid = !f.path.is_empty()
&& f.path.len() <= 255
&& f.path.split('/').all(|part| {
!matches!(part, "" | "." | "..") && !part.contains(['\\', ':', '\0']) && !part.starts_with('.')
});
if !valid {
return Err(IdeError::invalid(format!("invalid package path {:?}", f.path)));
}
if !seen.insert(f.path.to_ascii_lowercase()) {
return Err(IdeError::invalid(format!("{} is in the package twice", f.path)));
}
}
// One folder per package, replaced by the next import of it: they don't pile up.
let dir = self.plugins_dir.join(DOWNLOADS).join(format!("{name}-generated"));
let _ = std::fs::remove_dir_all(&dir);
for f in files {
let path = dir.join(&f.path);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(&path, &f.text)?;
}
Ok(dir)
}
/// Files of the package in `dir` (what its declarations reference, and
/// the component): what a package file holds besides the manifest.
pub fn package_files_of(&self, dir: &Path) -> IdeResult<Vec<PathBuf>> {
+53 -6
View File
@@ -16,8 +16,12 @@
//! projects) and folders linked with `openidea-plugin link` (`links.json`).
mod api;
mod icon_extensions;
mod icon_patterns;
mod icon_settings;
mod icons;
mod install;
pub mod intellij;
mod market;
mod points;
mod providers;
@@ -32,7 +36,10 @@ pub mod web;
pub use api::{FileSecretStore, HostWorkspace, PluginWorkspace, SecretStore};
pub use icons::{map_asset as icon_theme_map_asset, sanitize_svg, svg_prefix as icon_theme_svg_prefix};
pub use market::{INDEX_FILE as MARKET_INDEX_FILE, MarketPlugin, OFFICIAL_INDEX as OFFICIAL_MARKET_INDEX};
pub use market::{
INDEX_FILE as MARKET_INDEX_FILE, MarketPlugin, OFFICIAL_INDEX as OFFICIAL_MARKET_INDEX, OFFICIAL_PUBLISHER,
read_url as read_market_url, resolve as resolve_market_href,
};
pub use points::{COMMANDS, CommandDecl, register_builtin_points};
pub(crate) use providers::editor_providers as editor_providers_for;
pub use providers::{ProviderKind, set_editor_providers};
@@ -238,12 +245,16 @@ pub struct Consent {
/// Optional capabilities the user accepted (`net:crates.io`, …). Required
/// ones are implied by consenting at all.
pub optional: Vec<String>,
/// The user accepted a package signed by another key than the installed
/// copy ([`InstallPreview::signer_changed`]).
#[serde(default)]
pub signer_change: bool,
}
impl Consent {
/// Accept everything the plugin asks for (tests, trusted official plugins).
pub fn all() -> Self {
Self { optional: vec!["*".into()] }
Self { optional: vec!["*".into()], signer_change: true }
}
fn accepts(&self, cap: &Capability) -> bool {
@@ -270,6 +281,14 @@ pub struct InstallPreview {
pub unsigned: bool,
/// The verified publisher.
pub publisher: Option<String>,
/// Replaces an installed copy signed with another key, which didn't
/// endorse this package's (or this one is unsigned): the user confirms it
/// ([`Consent::signer_change`]).
pub signer_changed: bool,
/// Something to confirm: a first install, an unsigned package, another
/// signer, or capabilities the installed version wasn't granted or
/// refused. Without, an update installs with the grants it has.
pub needs_consent: bool,
}
type ProjectPolicy = Arc<dyn Fn(&Path) -> bool + Send + Sync>;
@@ -316,6 +335,10 @@ pub struct PluginHost {
pins: Arc<RwLock<HashMap<(String, String), ProgramPin>>>,
/// The last marketplace index read: trust root for signatures, revocations.
market: RwLock<Option<ide_extension::package::Index>>,
/// The official publisher's keys the index is pinned to (none: as listed).
official_keys: RwLock<Vec<String>>,
/// `id@version` of the updates last announced.
announced_updates: Mutex<Vec<String>>,
/// The marketplace used while the user has set none (the app's official one).
default_market_url: RwLock<Option<String>>,
/// Updates not confirmed yet (the new version hasn't started once): what
@@ -354,9 +377,12 @@ impl PluginHost {
policy_pins.read().get(&(plugin.to_owned(), cap.to_string())).cloned()
}));
let shared = api::Shared::new(sink.clone(), plugins_dir.join(".data"), pins.clone(), ui.clone());
let market = RwLock::new(Self::load_market_index(&plugins_dir));
let official_keys = market::built_in_official_keys();
let market = RwLock::new(Self::load_market_index(&plugins_dir, &official_keys));
let host = Self {
market,
official_keys: RwLock::new(official_keys),
announced_updates: Mutex::default(),
default_market_url: RwLock::new(None),
updates: Mutex::default(),
project_policy: RwLock::new(Arc::new(|_| false)),
@@ -699,14 +725,27 @@ impl PluginHost {
}
let publisher = self.verified_publisher(src);
let m = &pkg.manifest;
let replaces = self.plugins.read().get(&m.id).map(|p| p.manifest().version.clone());
// An update shows what the installed version was granted.
let grant = replaces
.as_ref()
.and_then(|_| self.ledger.lock().get(&Subject::Installed(m.id.clone())).cloned())
.filter(|g| g.confirmed);
let granted = |c: &Capability| grant.as_ref().is_some_and(|g| g.has(c));
let mut capabilities: Vec<CapabilityInfo> =
m.capabilities.all().map(|c| CapabilityInfo::new(c, m.capabilities.is_optional(c), false)).collect();
m.capabilities.all().map(|c| CapabilityInfo::new(c, m.capabilities.is_optional(c), granted(c))).collect();
for extra in &set.capabilities {
if !m.declares(extra) {
capabilities.push(CapabilityInfo::new(extra, true, false));
capabilities.push(CapabilityInfo::new(extra, true, granted(extra)));
}
}
capabilities.sort_by(|a, b| b.risk.cmp(&a.risk).then(a.id.cmp(&b.id)));
let signer_changed = self.signer_changed(&m.id, src);
let answered = |c: &CapabilityInfo| {
c.granted || (c.optional && grant.as_ref().is_some_and(|g| g.declined.contains(&c.id)))
};
let needs_consent =
grant.is_none() || publisher.is_none() || signer_changed || !capabilities.iter().all(answered);
Ok(InstallPreview {
id: m.id.clone(),
name: m.name.clone(),
@@ -717,9 +756,11 @@ impl PluginHost {
capabilities,
contributions: counts(m),
warnings: set.warnings.clone(),
replaces: self.plugins.read().get(&m.id).map(|p| p.manifest().version.clone()),
replaces,
unsigned: publisher.is_none(),
publisher,
signer_changed,
needs_consent,
})
}
@@ -742,6 +783,11 @@ impl PluginHost {
if let Some(problem) = self.signature_problem(src) {
return Err(IdeError::invalid(format!("refusing {id}: {problem}")));
}
if !consent.signer_change && self.signer_changed(&id, src) {
return Err(IdeError::invalid(format!(
"refusing {id}: it is signed by another key than the installed version; confirm the change of signer"
)));
}
let version = pkg.manifest.version.clone();
if let Some(r) = self.market.read().as_ref().and_then(|i| i.revocation(&id, &version)) {
return Err(IdeError::invalid(format!("{id} {version} was revoked: {}", r.reason)));
@@ -2207,6 +2253,7 @@ fn placeholder_manifest(dir: &Path) -> Manifest {
homepage: None,
license: None,
change_notes: None,
declared_categories: Vec::new(),
engine: ide_extension::ENGINE_VERSION.into(),
legacy: false,
wasm: None,
+15
View File
@@ -292,6 +292,21 @@ impl PluginHost {
.collect()
}
/// The edits of an action `plugin_id` returned with `data`, now that the
/// user picked it: a call with the command budget, not the edit path's.
pub fn resolve_code_action(
&self,
plugin_id: &str,
doc: &ProviderDocument,
action: CodeAction,
) -> ide_core::IdeResult<CodeAction> {
let d = Arc::new(doc.clone());
let timeout = self.limits.timeout;
self.code_call(plugin_id, ide_plugin::Invocation::default(), "resolving the code action", move |rt, _| {
rt.resolve_code_action(&d, &action, timeout)
})
}
pub fn provide_definition(&self, doc: &ProviderDocument, at: ide_core::Pos) -> Vec<Location> {
let d = Arc::new(doc.clone());
self.edit_calls(ProviderKind::Definition, &doc.language, move |rt, timeout| {
+3 -1
View File
@@ -114,6 +114,7 @@ pub fn forwarded(event: &PluginEvent) -> bool {
| PluginEvent::ViewChanged { .. }
| PluginEvent::ViewMessage { .. }
| PluginEvent::OpenTerminal { .. }
| PluginEvent::OpenLocation { .. }
)
}
@@ -272,7 +273,8 @@ impl RemotePlugins {
report.failed.push(format!("{}: invalid plugin id", pkg.id));
continue;
}
let consent = Consent { optional: pkg.granted.clone() };
// The desktop asked the user, change of signer included.
let consent = Consent { optional: pkg.granted.clone(), signer_change: true };
if self.current_digest(&pkg.id).as_deref() == Some(pkg.digest.as_str()) {
if let Err(e) = self.follow_grants(&pkg, &consent) {
report.failed.push(format!("{}: {e}", pkg.id));
+1 -1
View File
@@ -134,7 +134,7 @@ impl ExtensionPoint for ThemesPoint {
/// Blanks out `//` and `/* */` comments outside strings, keeping newlines so
/// parse errors still point at the right line.
fn strip_json_comments(src: &str) -> String {
pub(super) fn strip_json_comments(src: &str) -> String {
let mut out = String::with_capacity(src.len());
let mut chars = src.chars().peekable();
let (mut in_str, mut escaped) = (false, false);
+22
View File
@@ -138,12 +138,24 @@ pub struct PluginsSnapshot {
pub ui: Vec<UiContribution>,
}
/// A newer version of an installed plugin in the marketplace.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PluginUpdate {
pub id: String,
pub name: String,
pub version: String,
}
/// Backend → UI notifications (Tauri event `ide://plugin`).
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(tag = "type", rename_all = "camelCase")]
pub enum PluginEvent {
/// Plugins, their status or their contributions changed: re-fetch `plugin_list`.
CommandsChanged,
/// The marketplace offers newer versions of installed plugins (announced
/// once per set of versions).
UpdatesAvailable { updates: Vec<PluginUpdate> },
#[serde(rename_all = "camelCase")]
Notification { plugin_id: String, level: NotificationLevel, title: String, body: String },
/// A plugin's status bar item was set (`item`) or removed.
@@ -180,6 +192,8 @@ pub enum PluginEvent {
args: Vec<String>,
cwd: ide_core::IpcPath,
},
#[serde(rename_all = "camelCase")]
OpenLocation { plugin_id: String, path: ide_core::IpcPath, line: u32, column: u32 },
/// Plugin languages, or the servers they may start, changed: re-fetch
/// `languages` (the host re-detects open documents: [`crate::Host::plugin_languages_changed`]).
LanguagesChanged,
@@ -212,6 +226,14 @@ impl PluginEditor {
}
}
/// A text file of a package the UI generated, at its `/`-separated path in the package.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PackageFile {
pub path: String,
pub text: String,
}
/// A plugin's status bar item.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
+10
View File
@@ -378,9 +378,19 @@ pub(super) fn view(pkg: &Package, decl: toml::Value) -> Result<UiDecl, String> {
}
let mut out = UiDecl::new(d.id.clone(), d.title.clone(), json(&d)?);
out.files = files;
if let Some(icon) = d.icon.as_deref().filter(|i| i.ends_with(".svg")) {
let path = std::path::PathBuf::from(icon);
let svg = super::icons::read_icon(pkg, &path)?;
out.assets.push((view_icon_asset(&d.id), svg));
out.files.push(path);
}
Ok(out)
}
pub fn view_icon_asset(id: &str) -> String {
format!("viewIcon/{id}")
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct CustomEditorDecl {
+286 -26
View File
@@ -34,6 +34,8 @@ pub struct Project {
pub lsp: Arc<LspManager>,
/// Git repositories: the one containing the project root and those nested below it.
pub repos: Arc<RwLock<Repos>>,
/// Explicitly opened external logs remain read-only and outside project indexing.
pub(crate) external_log_repos: RwLock<Vec<Arc<Repo>>>,
/// Incrementally maintained syntax trees of the open documents.
pub syntax: Arc<SyntaxCache>,
/// Tree-sitter structure queries (LSP fallback) over `syntax`.
@@ -56,6 +58,9 @@ pub struct Project {
pub(crate) watcher: Watcher,
/// Watchers of the Project Structure content roots outside the project folder.
pub(crate) content_watchers: Mutex<Vec<(PathBuf, Watcher)>>,
/// Watchers of folders outside the project that language servers registered file
/// watchers for (a virtualenv elsewhere, a path dependency).
lsp_watchers: Mutex<Vec<(PathBuf, Watcher)>>,
pub(crate) _scratch_watcher: Option<Watcher>,
/// A LightEdit window's loose files ([`Host::open_light`]): global settings only,
/// nothing indexed, no language servers, repositories or project services.
@@ -83,6 +88,7 @@ impl Host {
return Ok(current);
}
self.close_project(window);
self.wait_detached_closes();
let sink: SharedSink = (self.sink_for)(window);
// Before anything the project starts: its children get the project's environment.
let trusted = Arc::new(AtomicBool::new(self.trust.trusted(&root)?));
@@ -204,6 +210,7 @@ impl Host {
cfg.launch_error = Some(e);
None
});
ide_sdk::apply::configure_java_compiler(cfg, &data_dir, &effective.inspections);
ide_sdk::java::configure_jdtls(
cfg,
server_root,
@@ -218,9 +225,16 @@ impl Host {
ide_sdk::jdtls::launch_directly(cfg);
}
ide_sdk::apply::server_protocol_overrides(cfg, &effective);
// After the overrides: the PHP version they may set decides the storage.
ide_sdk::php::configure_intelephense_storage(cfg, &data_dir);
if cfg.name == "jdtls" {
let (excluded_poms, build_files) = jdtls_maven_import(root, &effective);
ide_sdk::apply::configure_maven_import(cfg, root, &excluded_poms, build_files.as_deref());
ide_sdk::apply::configure_gradle_import(
cfg,
effective.languages.java.gradle.linked_projects.as_deref(),
build_files.as_deref(),
);
}
}));
}
@@ -302,11 +316,16 @@ impl Host {
sink.clone(),
Box::new(move |batch| {
let changed: Vec<PathBuf> = batch.created.iter().chain(&batch.modified).cloned().collect();
index.update(&changed, &batch.removed);
let (ignored, ignored_gone) = ignored_changes(batch);
index.update(&[&changed[..], &ignored].concat(), &[&batch.removed[..], &ignored_gone].concat());
kotlin_bridge.files_changed(changed.iter().chain(&batch.removed));
lsp.project_files_changed(batch.created.iter().chain(&batch.removed));
lsp.watched_files_changed(&batch.created, &batch.modified, &batch.removed, &batch.excluded_dirs);
if composer_changed(&root, changed.iter().chain(&batch.removed)) {
lsp.reconfigure_soon();
}
// Atomic saves by external tools surface as `created`.
documents.on_disk_changed(&changed);
documents.on_disk_changed(&[&changed[..], &ignored].concat());
// A framework's root marker (`angular.json`) created or deleted: its
// templates gain or lose their mode. Markers in `node_modules` only
// count here when the folder isn't excluded (it is by default).
@@ -410,7 +429,16 @@ impl Host {
let content_watchers = extra_roots
.iter()
.filter_map(|r| {
let w = content_root_watcher(r, &excludes, &sink, &index, &documents, &history, Some(&kotlin_bridge))?;
let w = content_root_watcher(
r,
&excludes,
&sink,
&index,
&documents,
&history,
&lsp,
Some(&kotlin_bridge),
)?;
Some((r.clone(), w))
})
.collect();
@@ -443,6 +471,7 @@ impl Host {
index,
lsp,
repos,
external_log_repos: RwLock::default(),
structure: Arc::new(StructureService::new(syntax.clone())),
syntax,
detected_roots,
@@ -457,6 +486,7 @@ impl Host {
kotlin_bridge: Some(kotlin_bridge),
watcher,
content_watchers: Mutex::new(content_watchers),
lsp_watchers: Mutex::new(Vec::new()),
_scratch_watcher: scratch_watcher,
light: false,
});
@@ -466,6 +496,14 @@ impl Host {
weak.upgrade().map(|p| p.reference_candidates(q, cancel)).unwrap_or_default()
}));
}
{
let weak = Arc::downgrade(&project);
project.lsp.set_file_watch_roots_hook(Box::new(move |roots| {
if let Some(p) = weak.upgrade() {
p.watch_for_language_servers(roots);
}
}));
}
{
let weak = Arc::downgrade(&project);
project.lsp.set_project_excludes_hook(Box::new(move || {
@@ -488,12 +526,15 @@ impl Host {
root: &Path,
project_key: &str,
) -> (Arc<ide_db::DbManager>, Arc<ide_http::History>, Arc<ide_agent::AgentManager>) {
let db = Arc::new(ide_db::DbManager::new(
root.to_path_buf(),
self.data_dir.join("db-passwords.json"),
self.data_dir.join("db-history.json"),
self.ssh.clone(),
));
let db = Arc::new(
ide_db::DbManager::new(
root.to_path_buf(),
self.data_dir.join("db-passwords.json"),
self.data_dir.join("db-history.json"),
self.ssh.clone(),
)
.with_global_sources(self.data_dir.clone()),
);
let http_history = Arc::new(ide_http::History::new(self.data_dir.join("http-history").join(project_key)));
let agents = Arc::new(ide_agent::AgentManager::new(
ide_agent::AgentPaths {
@@ -535,6 +576,7 @@ impl Host {
return Ok(current);
}
self.close_project(window);
self.wait_detached_closes();
let sink: SharedSink = (self.sink_for)(window);
let settings = Arc::new(SettingsStore::load(&self.config_dir, None));
let effective = settings.effective();
@@ -567,6 +609,7 @@ impl Host {
index,
lsp,
repos: Arc::default(),
external_log_repos: RwLock::default(),
structure: Arc::new(StructureService::new(syntax.clone())),
syntax,
detected_roots: Arc::default(),
@@ -581,6 +624,7 @@ impl Host {
kotlin_bridge: None,
watcher,
content_watchers: Mutex::new(Vec::new()),
lsp_watchers: Mutex::new(Vec::new()),
_scratch_watcher: None,
light: true,
root: dir,
@@ -663,7 +707,14 @@ fn scan_repositories(
build_cache: &Path,
) {
let ignored = settings.effective().ignored_names();
let skip = |p: &Path| p.file_name().is_some_and(|n| ignored.is_ignored_name(&n.to_string_lossy()));
let skip = |p: &Path| {
p.file_name().is_some_and(|n| {
let name = n.to_string_lossy();
ignored.is_ignored_name(&name)
|| (ide_workspace::unity::GENERATED_DIRS.contains(&name.as_ref())
&& p.parent().is_some_and(ide_workspace::unity::is_project))
})
};
let mut roots = vec![root.to_path_buf()];
roots.extend(content_roots(&settings.effective(), root, build_cache));
match Repos::discover_roots(&roots, &skip) {
@@ -710,6 +761,51 @@ impl Project {
}
}
pub fn open_external_log(&self, path: &Path) -> IdeResult<PathBuf> {
let root = dunce::canonicalize(path)?;
let repo = Repo::discover(&root)?.ok_or_else(|| IdeError::unavailable("folder is not a Git worktree"))?;
let root = repo.root().to_path_buf();
repo.set_execution_allowed(false);
let mut external = self.external_log_repos.write();
if !external.iter().any(|r| r.root() == root) {
external.push(Arc::new(repo));
}
Ok(root)
}
/// Log read operations may also use an explicitly opened external worktree.
/// Mutations continue to use `with_repo_at` and cannot reach these repositories.
pub fn with_log_repo_at<T>(&self, at: Option<&Path>, f: impl FnOnce(&Repo) -> IdeResult<T>) -> IdeResult<T> {
if let Some(path) = at
&& self.repos().at_root(path).is_none()
{
let repo = self
.external_log_repos
.read()
.iter()
.find(|r| r.root() == path)
.cloned()
.ok_or_else(|| IdeError::unavailable("external Git log is not open"))?;
return f(&repo);
}
self.with_repo_at(at, f)
}
pub fn with_log_repo_of<T>(&self, path: &Path, f: impl FnOnce(&Repo) -> IdeResult<T>) -> IdeResult<T> {
if self.repos().for_path(path).is_some() {
return self.with_repo_of(path, f);
}
let repo = self
.external_log_repos
.read()
.iter()
.filter(|r| path.starts_with(r.root()))
.max_by_key(|r| r.root().components().count())
.cloned()
.ok_or_else(|| IdeError::unavailable("path is not in an open Git log"))?;
f(&repo)
}
/// Runs `f` with the repository `path` (a file or folder) belongs to.
pub fn with_repo_of<T>(&self, path: &Path, f: impl FnOnce(&Repo) -> IdeResult<T>) -> IdeResult<T> {
match self.repos().for_path(path) {
@@ -785,8 +881,14 @@ impl Project {
/// Effective source / test / resource / excluded roots (detected + Project Structure marks).
pub fn roots(&self) -> ProjectRoots {
let detected = with_gradle_roots(&self.root, &self.build_cache_dir(), &self.detected_roots.read());
linked_roots(&self.root, &self.settings.effective(), &detected)
let settings = self.settings.effective();
let detected = with_gradle_roots(
&self.root,
&self.build_cache_dir(),
&settings.languages.java.gradle,
&self.detected_roots.read(),
);
linked_roots(&self.root, &settings, &detected)
}
/// Where the Maven / Gradle tool windows cache what the build tools reported.
@@ -815,7 +917,8 @@ impl Project {
/// Project files of `q.languages` under `q.root` containing `q.word` as a whole
/// word (unsaved buffers included; excluded roots, ignored names and
/// `.gitignore`d files skipped): the files a references search opens on its server.
/// `.gitignore`d folders skipped, while `.gitignore`d files elsewhere count, as
/// in IDEA's project scope): the files a references search opens on its server.
pub fn reference_candidates(
&self,
q: &ide_lsp::CandidateQuery,
@@ -893,6 +996,7 @@ impl Project {
&self.index,
&self.documents,
&self.history,
&self.lsp,
self.kotlin_bridge.as_ref(),
) {
watchers.push((r, w));
@@ -924,7 +1028,7 @@ impl Project {
return encoding_rs::Encoding::for_label(loaded.encoding.as_bytes())
.ok_or_else(|| ide_core::IdeError::invalid("unknown file encoding"));
}
let bytes = self.with_repo_of(path, |r| r.file_bytes_at(path, rev))?.unwrap_or_default();
let bytes = self.with_log_repo_of(path, |r| r.file_bytes_at(path, rev))?.unwrap_or_default();
Ok(ide_vfs::detect_encoding(&bytes, fallback))
}
@@ -947,6 +1051,11 @@ const HISTORY_BATCH_LIMIT: usize = 500;
/// Local History for changes made outside the IDE (other tools, VCS, deletions).
/// Contents equal to the last snapshot (e.g. the IDE's own saves) are skipped.
/// The batch's ignored files (`.env`), split into those still on disk and those removed.
fn ignored_changes(batch: &ide_vfs::FsBatch) -> (Vec<PathBuf>, Vec<PathBuf>) {
batch.ignored.iter().cloned().partition(|p| p.symlink_metadata().is_ok())
}
fn record_external_changes(history: &LocalHistory, changed: &[PathBuf], removed: &[PathBuf]) {
for path in changed.iter().take(HISTORY_BATCH_LIMIT) {
if let Err(e) = history.record_file(path, "External change", false) {
@@ -980,7 +1089,7 @@ fn content_roots(settings: &ide_core::settings::Settings, root: &Path, build_cac
}
let mut out: Vec<PathBuf> = Vec::new();
let user = settings.project_structure.content_roots.iter().map(|r| PathBuf::from(r.trim()));
for p in user.chain(ide_build::gradle_external_roots(root, build_cache)) {
for p in user.chain(ide_build::gradle_external_roots(root, build_cache, &settings.languages.java.gradle)) {
if p.is_absolute() && p.is_dir() && !p.starts_with(root) && !out.contains(&p) {
out.push(p);
}
@@ -990,6 +1099,31 @@ fn content_roots(settings: &ide_core::settings::Settings, root: &Path, build_cac
/// Watches a content root outside the project: the index, open documents and
/// Local History follow its changes like the project folder's.
impl Project {
/// Watches `roots` (outside the project) for the language servers that asked to.
fn watch_for_language_servers(&self, roots: Vec<PathBuf>) {
let mut watchers = self.lsp_watchers.lock();
watchers.retain(|(r, _)| roots.contains(r));
for root in roots {
if watchers.iter().any(|(r, _)| *r == root) || !root.is_dir() {
continue;
}
let lsp = Arc::downgrade(&self.lsp);
let on_change = Box::new(move |batch: &ide_vfs::FsBatch| {
if let Some(lsp) = lsp.upgrade() {
lsp.watched_files_changed(&batch.created, &batch.modified, &batch.removed, &batch.excluded_dirs);
}
});
// Not project files: nothing else hears of them.
match Watcher::start(&root, Vec::new(), Arc::new(ide_core::NullSink), on_change) {
Ok(w) => watchers.push((root, w)),
Err(e) => tracing::warn!("language server watcher {}: {e}", root.display()),
}
}
}
}
#[allow(clippy::too_many_arguments)]
fn content_root_watcher(
dir: &Path,
excludes: &[String],
@@ -997,21 +1131,28 @@ fn content_root_watcher(
index: &Arc<ProjectIndex>,
documents: &Arc<Documents>,
history: &Arc<LocalHistory>,
lsp: &Arc<LspManager>,
kotlin_bridge: Option<&Arc<crate::kotlin_bridge::KotlinBridge>>,
) -> Option<Watcher> {
let kotlin_bridge = kotlin_bridge.cloned();
let (index, documents, history) = (index.clone(), documents.clone(), history.clone());
let (index, documents, history, lsp) = (index.clone(), documents.clone(), history.clone(), lsp.clone());
let root = dir.to_path_buf();
Watcher::start(
dir,
excludes.to_vec(),
sink.clone(),
Box::new(move |batch| {
let changed: Vec<PathBuf> = batch.created.iter().chain(&batch.modified).cloned().collect();
index.update(&changed, &batch.removed);
let (ignored, ignored_gone) = ignored_changes(batch);
index.update(&[&changed[..], &ignored].concat(), &[&batch.removed[..], &ignored_gone].concat());
// A workspace member's PHP language level follows its own composer.json.
if composer_changed(&root, changed.iter().chain(&batch.removed)) {
lsp.reconfigure_soon();
}
if let Some(bridge) = &kotlin_bridge {
bridge.files_changed(changed.iter().chain(&batch.removed));
}
documents.on_disk_changed(&changed);
documents.on_disk_changed(&[&changed[..], &ignored].concat());
record_external_changes(&history, &changed, &batch.removed);
}),
)
@@ -1019,6 +1160,13 @@ fn content_root_watcher(
.ok()
}
/// Whether the `composer.json` of `root` is among `paths`: the PHP language level follows it
/// (`ide_sdk::php`).
fn composer_changed<'a>(root: &Path, mut paths: impl Iterator<Item = &'a PathBuf>) -> bool {
let composer = root.join("composer.json");
paths.any(|p| *p == composer)
}
/// Where agent tasks create their worktrees: next to the project, `<parent>/<name>.worktrees`,
/// so they sit on the project's drive and outside its tree (not indexed, watched or shown in
/// its git status); `fallback` for a project at a filesystem root.
@@ -1045,31 +1193,93 @@ pub(crate) fn jdtls_maven_import(
settings: &ide_core::settings::Settings,
) -> (Vec<PathBuf>, Option<Vec<String>>) {
let maven = &settings.languages.java.maven;
if maven.unlinked_projects.is_empty() {
let gradle = &settings.languages.java.gradle;
if maven.unlinked_projects.is_empty() && gradle.linked_projects.is_none() {
return (Vec::new(), None);
}
let model = ide_build::maven::load(root);
let model = ide_build::maven::load_linked(root, maven);
let unlinked = ide_build::maven::UnlinkedProjects::from_model(&model, maven);
let excluded = unlinked.poms().map(PathBuf::from).collect();
let build_files = unlinked.import_poms(&model).map(|poms| {
let gradle = ide_build::gradle::load(root).and_then(|g| {
let poms = unlinked.import_poms(&model).or_else(|| {
gradle.linked_projects.as_ref().map(|_| {
model
.projects
.iter()
.filter(|p| !p.aggregated && !unlinked.contains(&p.pom))
.map(|p| p.pom.clone())
.collect()
})
});
let build_files = poms.map(|poms| {
let gradle = ide_build::gradle::load_linked(root, gradle).into_iter().filter_map(|g| {
let dir = root.join(&g.dir);
let names = ide_build::gradle::SETTINGS_FILES.iter().chain(ide_build::gradle::BUILD_FILES);
names.map(|n| dir.join(n)).find(|f| f.is_file())
});
poms.iter().map(|pom| root.join(pom)).chain(gradle).map(|f| ide_lsp::path_to_uri_string(&f)).collect()
let eclipse = if settings.languages.java.gradle.linked_projects.is_some() {
independent_eclipse_projects(root)
} else {
Vec::new()
};
poms.iter()
.map(|pom| root.join(pom))
.chain(gradle)
.chain(eclipse)
.map(|f| ide_lsp::path_to_uri_string(&f))
.collect()
});
(excluded, build_files)
}
/// Explicit Gradle imports must retain Eclipse projects without re-importing generated build metadata.
fn independent_eclipse_projects(root: &Path) -> Vec<PathBuf> {
let managed = |dir: &Path| {
std::iter::once("pom.xml")
.chain(ide_build::gradle::SETTINGS_FILES.iter().copied())
.chain(ide_build::gradle::BUILD_FILES.iter().copied())
.any(|name| dir.join(name).is_file())
};
if managed(root) {
return Vec::new();
}
let walker = ignore::WalkBuilder::new(root)
.standard_filters(false)
.filter_entry(move |entry| {
if !entry.file_type().is_some_and(|kind| kind.is_dir()) {
return false;
}
if entry.depth() > 0
&& (entry.file_name().to_string_lossy().starts_with('.')
|| matches!(
entry.file_name().to_str(),
Some("node_modules" | "target" | "build" | "dist" | "out" | "bin" | "vendor")
)
|| entry.path().parent().is_some_and(|parent| parent.join(".project").is_file()))
{
return false;
}
!managed(entry.path())
})
.build();
let mut projects: Vec<_> =
walker.flatten().map(|entry| entry.path().join(".project")).filter(|path| path.is_file()).collect();
projects.sort();
projects
}
/// The detected roots with the last Gradle sync's source sets first: Gradle evaluated what
/// static detection guesses. Folders outside the project keep their absolute path, which
/// `normalize` turns into the same form the UI compares (`Users/me/svc/src/main/java`).
fn with_gradle_roots(root: &Path, build_cache: &Path, detected: &[SourceFolder]) -> Vec<SourceFolder> {
fn with_gradle_roots(
root: &Path,
build_cache: &Path,
settings: &ide_core::settings::GradleSettings,
detected: &[SourceFolder],
) -> Vec<SourceFolder> {
use ide_build::gradle::ReportedRoot;
use ide_workspace::roots::RootKind;
let mut out: Vec<SourceFolder> = Vec::new();
for (dir, kind) in ide_build::gradle_source_roots(build_cache) {
for (dir, kind) in ide_build::gradle_source_roots(root, build_cache, settings) {
if !dir.is_dir() {
continue;
}
@@ -1239,6 +1449,37 @@ pub fn apply_git_executable(settings: &ide_core::settings::Settings) {
#[cfg(test)]
mod maven_linkage_tests {
#[test]
fn explicit_gradle_links_preserve_independent_eclipse_projects() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
for dir in
["linked", "unlinked", "eclipse", "nested/eclipse", "unlinked/subproject", "maven", "build/generated"]
{
std::fs::create_dir_all(root.join(dir)).unwrap();
std::fs::write(root.join(dir).join(".project"), "<projectDescription/>").unwrap();
}
for dir in ["linked", "unlinked"] {
std::fs::write(root.join(dir).join("settings.gradle"), "").unwrap();
}
std::fs::write(root.join("maven/pom.xml"), "<project/>").unwrap();
let mut settings = ide_core::settings::Settings::default();
settings.languages.java.gradle.linked_projects = Some(vec!["linked".into()]);
let (_, files) = super::jdtls_maven_import(root, &settings);
let files = files.unwrap();
let uri = |path: &str| ide_lsp::path_to_uri_string(&root.join(path));
assert!(files.contains(&uri("linked/settings.gradle")));
assert!(files.contains(&uri("maven/pom.xml")));
assert!(files.contains(&uri("eclipse/.project")));
assert!(files.contains(&uri("nested/eclipse/.project")));
assert!(!files.iter().any(|path| path.contains("unlinked") || path.contains("build/generated")));
assert!(!files.contains(&uri("linked/.project")));
assert!(!files.contains(&uri("maven/.project")));
settings.languages.java.gradle.linked_projects = Some(vec![]);
let (_, files) = super::jdtls_maven_import(root, &settings);
assert_eq!(files.unwrap(), [uri("maven/pom.xml"), uri("eclipse/.project"), uri("nested/eclipse/.project")]);
}
#[test]
fn jdtls_imports_the_linked_modules_and_the_gradle_build_once_a_reactor_module_is_unlinked() {
let tmp = tempfile::tempdir().unwrap();
@@ -1336,7 +1577,8 @@ mod gradle_roots_tests {
std::fs::write(cache.join("gradle.json"), json.to_string()).unwrap();
let guessed = SourceFolder { path: "src/it/java".into(), kind: RootKind::Source, detected_from: None };
let other = SourceFolder { path: "web".into(), kind: RootKind::Source, detected_from: None };
let out = with_gradle_roots(&root, &cache, &[guessed, other.clone()]);
std::fs::write(root.join("build.gradle"), "").unwrap();
let out = with_gradle_roots(&root, &cache, &Default::default(), &[guessed, other.clone()]);
let abs = |p: PathBuf| ide_workspace::roots::normalize(&s(p));
let got: Vec<(String, RootKind)> = out.iter().map(|f| (f.path.clone(), f.kind)).collect();
assert_eq!(
@@ -1435,3 +1677,21 @@ mod agent_worktree_tests {
assert_eq!(agent_worktrees_dir(Path::new("/"), fallback), fallback());
}
}
#[cfg(test)]
mod composer_tests {
use super::composer_changed;
use std::path::{Path, PathBuf};
#[test]
fn only_the_roots_own_composer_json_counts() {
let root = Path::new("/work/app");
let changed = |paths: &[&str]| {
let paths: Vec<PathBuf> = paths.iter().map(|p| root.join(p)).collect();
composer_changed(root, paths.iter())
};
assert!(changed(&["src/a.php", "composer.json"]));
assert!(!changed(&["composer.lock", "vendor/acme/lib/composer.json", "packages/b/composer.json"]));
assert!(!changed(&[]));
}
}
+66 -4
View File
@@ -1,6 +1,7 @@
//! System Settings › HTTP Proxy on the host: the global settings made current for
//! connections and child processes, the password kept in the saved passwords
//! (owner-only `proxy-password.json`) instead of `settings.toml`.
//! (the OS credential store) instead of `settings.toml`, or for the session
//! when there is no credential store.
use crate::Host;
use ide_core::IdeResult;
@@ -12,7 +13,9 @@ impl Host {
/// `settings` with the saved proxy password filled in, as the Settings dialog edits it.
pub fn with_proxy_password(&self, mut settings: Settings) -> Settings {
// A password typed into settings.toml by hand counts too, unless one was saved.
if let Some(password) = self.proxy_password.get(PASSWORD_KEY) {
let session = || self.proxy_session_password.lock().clone();
// The session copy is the latest write that could not be saved.
if let Some(password) = session().or_else(|| self.proxy_password.get(PASSWORD_KEY)) {
settings.proxy.password = password;
}
settings
@@ -40,11 +43,70 @@ impl Host {
_ => return Ok(Some(value)),
};
if let Some(password) = password {
match password.as_str().filter(|p| !p.is_empty()) {
Some(p) => self.proxy_password.set(PASSWORD_KEY, p)?,
let password = password.as_str().filter(|p| !p.is_empty());
let mut session = self.proxy_session_password.lock();
*session = None;
match password {
Some(p) => {
if let Err(e) = self.proxy_password.set(PASSWORD_KEY, p) {
tracing::warn!("the HTTP proxy password is kept for this session only: {}", e.message);
*session = Some(p.to_owned());
}
}
None => self.proxy_password.delete(PASSWORD_KEY),
}
}
Ok((key != "proxy.password").then_some(value))
}
}
#[cfg(test)]
mod tests {
use super::*;
use ide_ssh::secrets::{SecretFile, StoreError, StoreResult, Vault};
use std::sync::Arc;
/// A machine without a credential store (a headless Linux host).
struct NoStore;
impl Vault for NoStore {
fn available(&self) -> bool {
false
}
fn get(&self, _: &str, _: &str) -> StoreResult<String> {
Err(StoreError::NoDefaultStore)
}
fn set(&self, _: &str, _: &str, _: &str) -> StoreResult<()> {
Err(StoreError::NoDefaultStore)
}
fn delete(&self, _: &str, _: &str) -> StoreResult<()> {
Err(StoreError::NoDefaultStore)
}
}
#[tokio::test]
async fn without_a_credential_store_the_password_lasts_the_session() {
let tmp = tempfile::tempdir().unwrap();
let mut host = Host::new(crate::HostConfig {
config_dir: tmp.path().join("config"),
data_dir: tmp.path().join("data"),
local_data_dir: tmp.path().join("data"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new(|_| Arc::new(ide_core::NullSink)),
owner: |w| w,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
});
let path = tmp.path().join("data/proxy-password.json");
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
std::fs::write(&path, r#"{"httpProxy": "old"}"#).unwrap();
host.proxy_password = SecretFile::with_vault(path, |v| serde_json::from_value(v).ok(), Arc::new(NoStore));
let left = host.take_proxy_password("proxy.password", serde_json::json!("s3cret")).unwrap();
assert_eq!(left, None);
assert_eq!(host.with_proxy_password(Settings::default()).proxy.password, "s3cret");
host.take_proxy_password("proxy.password", serde_json::json!("")).unwrap();
assert_eq!(host.with_proxy_password(Settings::default()).proxy.password, "");
}
}
+3 -1
View File
@@ -188,7 +188,7 @@ impl Dispatcher {
ops::fs::create_file_bytes(&c.host, c.arg("path")?, c.arg("data")?).await
})
.add("fs_create_dir", |c| async move { ide_vfs::create_dir(&c.arg::<PathBuf>("path")?) })
.add("fs_rename", |c| async move { ide_vfs::rename(&c.arg::<PathBuf>("from")?, &c.arg::<PathBuf>("to")?) })
.add("fs_rename", |c| async move { ops::fs::rename(&c.host, c.arg("from")?, c.arg("to")?).await })
.add("fs_set_writable", |c| async move {
ops::fs::set_writable(&c.host, &c.window, c.arg("path")?, c.arg("writable")?)
})
@@ -257,6 +257,8 @@ impl Dispatcher {
&c.arg::<Option<Vec<String>>>("removedDetectedIds")?.unwrap_or_default(),
)
})
.add("run_without_derived", |c| async move { ops::run::without_derived(&c.host, &c.window, c.arg("configs")?) })
.add("run_apply_settings", |c| async move { ops::run::apply_settings(&c.host, &c.window, c.arg("configs")?) })
.add("run_system_env", |_| async move { Ok(ide_run::system_env()) })
.add("run_start", |c| async move {
let on_output = c.channel("onOutput")?;
+12
View File
@@ -177,6 +177,10 @@ pub(crate) fn read_login_env(shell: &str, timeout: std::time::Duration) -> Optio
}
}
}
// Nor what the last read found that this process lacks: children get it, the shell must not.
for (k, _) in ide_core::process::shell_env() {
cmd.env_remove(k);
}
ide_core::process::prepare(&mut cmd);
let mut child = cmd
.args(["-ilc", &format!("printf '{MARK}'; /usr/bin/env -0; printf '{MARK}'")])
@@ -257,6 +261,14 @@ mod tests {
std::fs::write(&rc, "export OPENIDEA_TEST_RC='two words'\n").unwrap();
let second = read_login_env(&shell, Duration::from_secs(10)).unwrap();
assert_eq!(get(&second, "OPENIDEA_TEST_RC").as_deref(), Some("two words"));
// What a read found goes to children, not into the next read: an rc file that no
// longer exports a variable drops it.
ide_core::process::set_shell_env(second);
std::fs::write(&rc, "export OPENIDEA_TEST_SEEN=\"${OPENIDEA_TEST_RC:-unset}\"\n").unwrap();
let third = read_login_env(&shell, Duration::from_secs(10)).unwrap();
assert_eq!(get(&third, "OPENIDEA_TEST_RC"), None);
assert_eq!(get(&third, "OPENIDEA_TEST_SEEN").as_deref(), Some("unset"));
}
#[test]
+42 -1
View File
@@ -217,15 +217,19 @@ pub fn restricted_command_allowed(cmd: &str) -> bool {
| "lsp_resend_diagnostics"
| "recent_projects"
| "remove_recent_project"
| "recent_project_groups"
| "set_recent_project_groups"
| "open_project"
| "close_project"
| "project_current"
| "project_roots"
| "index_status"
| "index_file_extensions"
| "jvm_file_presentations"
| "deploy_coverage"
| "deploy_test_connection"
| "find_files"
| "find_dirs"
| "find_symbols"
| "mark_recent"
| "remove_recent"
@@ -274,6 +278,15 @@ pub fn restricted_command_allowed(cmd: &str) -> bool {
| "plugin_set_theme_frame"
| "plugin_catalog"
| "plugin_assets"
| "plugin_market"
| "plugin_market_url"
| "plugin_set_market_url"
| "plugin_market_download"
| "plugin_unpack_package"
| "plugin_stage_package"
| "plugin_preview_install"
| "plugin_read_intellij_theme"
| "plugin_installed_intellij_themes"
| "notebook_open"
| "notebook_get"
| "notebook_edit"
@@ -295,7 +308,16 @@ pub fn restricted_command_allowed(cmd: &str) -> bool {
| "vcs_line_hunks"
| "vcs_file_at"
| "vcs_blame"
| "vcs_block_history"
| "vcs_log"
| "vcs_log_refs"
| "vcs_log_users"
| "vcs_log_details"
| "vcs_log_details_batch"
| "vcs_log_index"
| "vcs_revision_children"
| "vcs_revision_files"
| "vcs_open_external_log"
| "vcs_commit_files"
| "vcs_commits_changes"
| "vcs_branches"
@@ -319,12 +341,31 @@ pub fn restricted_command_allowed(cmd: &str) -> bool {
) || cmd.starts_with("plugin:")
}
/// Commands that change which plugin code runs: allowed where no project is open (the
/// welcome screen), otherwise only in a trusted project.
pub fn projectless_command_allowed(cmd: &str) -> bool {
matches!(
cmd,
"plugin_install_dir"
| "plugin_uninstall"
| "plugin_set_enabled"
| "plugin_grant"
| "plugin_set_capability"
| "plugin_roll_back"
| "plugin_reload"
)
}
impl crate::Host {
pub fn authorize_command(&self, window: &str, command: &str) -> IdeResult<()> {
if restricted_command_allowed(command) {
return Ok(());
}
self.project(window)?.require_trust()
match self.project(window) {
Ok(project) => project.require_trust(),
Err(_) if projectless_command_allowed(command) => Ok(()),
Err(error) => Err(error),
}
}
}
+51
View File
@@ -5,6 +5,7 @@
use ide_core::NullSink;
use ide_host::{Host, HostConfig, ops};
use std::sync::Arc;
use std::time::{Duration, Instant};
fn options(root: &std::path::Path) -> ide_terminal::TerminalOptions {
ide_terminal::TerminalOptions {
@@ -52,3 +53,53 @@ fn closing_a_project_closes_its_terminals_and_keeps_other_windows() {
host.terminals.kill_key("other");
});
}
/// The desktop closes a project on the event-loop thread while runtime workers may be
/// blocked waiting for that thread (IPC responses, window queries): closing must not
/// wait for workers that need the runtime, and exit still waits for them once free.
#[test]
fn detached_close_returns_while_runtime_workers_wait_for_the_closing_thread() {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap().join("project");
std::fs::create_dir_all(&root).unwrap();
let runtime = tokio::runtime::Builder::new_multi_thread().worker_threads(1).enable_all().build().unwrap();
let _guard = runtime.enter();
let host = Arc::new(Host::new(HostConfig {
config_dir: tmp.path().join("config"),
data_dir: tmp.path().join("data"),
local_data_dir: tmp.path().join("data"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new(|_| Arc::new(NullSink)),
owner: |w| w,
runtime: runtime.handle().clone(),
project_environment: false,
}));
host.trust.decide(&root, true, false).unwrap();
host.open_project("main", &root).unwrap();
// Occupy the only runtime worker until "the event loop" answers.
let (started_tx, started_rx) = std::sync::mpsc::channel();
let (release_tx, release_rx) = std::sync::mpsc::channel::<()>();
runtime.spawn(async move {
started_tx.send(()).unwrap();
let _ = release_rx.recv();
});
started_rx.recv().unwrap();
let (closed_tx, closed_rx) = std::sync::mpsc::channel();
let closer = host.clone();
std::thread::spawn(move || {
closer.close_project_detached("main");
let _ = closed_tx.send(());
});
let returned = closed_rx.recv_timeout(Duration::from_secs(5));
release_tx.send(()).unwrap();
assert!(returned.is_ok(), "closing waited for a runtime worker that waits for the closing thread");
assert!(host.project("main").is_err(), "the window no longer has the project");
host.open_project("main", &root).expect("the window opens a project once the close has finished");
let started = Instant::now();
host.shutdown_notebooks_on_exit();
assert!(started.elapsed() < Duration::from_secs(5), "exit joined the finished close instead of timing out");
}
+258 -9
View File
@@ -131,25 +131,44 @@ fn installs_from_before_versioning_keep_working_and_move_on_update() {
/// `hello` as `acme.hello` at `version`, packed and signed with `key`.
fn signed_hello(dir: &Path, version: &str, key: &PublisherKey, host: &PluginHost) -> (PathBuf, IndexEntry) {
let src = variant(&repo().join("crates/ide-plugin/tests/fixtures/hello"), dir, &format!("src-{version}"), |t| {
t.replace("id = \"hello\"", "id = \"acme.hello\"")
.replace("version = \"0.3.0\"", &format!("version = \"{version}\""))
});
signed(dir, "acme", version, key, host, |t| t.to_owned())
}
/// `hello` as `<publisher>.hello` at `version`, its manifest changed by
/// `edit`, packed and signed with `key`.
fn signed(
dir: &Path,
publisher: &str,
version: &str,
key: &PublisherKey,
host: &PluginHost,
edit: impl Fn(&str) -> String,
) -> (PathBuf, IndexEntry) {
let id = format!("{publisher}.hello");
let src =
variant(&repo().join("crates/ide-plugin/tests/fixtures/hello"), dir, &format!("src-{id}-{version}"), |t| {
edit(
&t.replace("id = \"hello\"", &format!("id = \"{id}\""))
.replace("version = \"0.3.0\"", &format!("version = \"{version}\"")),
)
});
let files = host.package_files_of(&src).unwrap();
let digest = package::package_digest(&src, &files).unwrap();
let sig = key.sign("acme", "acme.hello", version, &digest);
let out = dir.join(format!("acme.hello-{version}.oiplugin"));
let sig = key.sign(publisher, &id, version, &digest);
let out = dir.join(format!("{id}-{version}.oiplugin"));
package::pack(&src, &files, &out, Some(&sig)).unwrap();
let entry = IndexEntry {
id: "acme.hello".into(),
id: id.clone(),
name: "Hello".into(),
version: version.into(),
description: String::new(),
authors: vec![],
publisher: "acme".into(),
publisher: publisher.into(),
download: out.file_name().unwrap().to_string_lossy().into_owned(),
digest,
capabilities: vec![],
engine: "^1.0".into(),
categories: vec!["tools".into()],
};
(out, entry)
}
@@ -166,16 +185,39 @@ fn write_index(
publishers: vec![Publisher {
id: "acme".into(),
name: "ACME".into(),
keys: vec![KeyEntry { key: key.public_base64(), revoked: key_revoked }],
keys: vec![KeyEntry { revoked: key_revoked, ..KeyEntry::new(key.public_base64()) }],
}],
plugins,
revoked,
moved: None,
};
let path = dir.join("index.json");
std::fs::write(&path, serde_json::to_string(&index).unwrap()).unwrap();
path.to_string_lossy().into_owned()
}
/// First run: nothing cached yet, so listing the marketplace reads its index
/// instead of showing an empty one; later listings use the index read.
#[test]
fn the_first_marketplace_listing_reads_the_index() {
let f = fixture();
assert!(f.host.market_listing(false).unwrap().is_empty(), "no marketplace configured");
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, entry) = signed_hello(&market, "1.0.0", &key, &f.host);
let url = write_index(&market, &key, vec![entry], vec![], false);
f.host.set_market_url(Some(url.clone()));
let listed = f.host.market_listing(false).unwrap();
assert_eq!(listed.iter().map(|p| p.entry.id.as_str()).collect::<Vec<_>>(), ["acme.hello"]);
std::fs::remove_file(&url).unwrap();
assert_eq!(f.host.market_listing(false).unwrap().len(), 1, "the index read is kept");
assert!(f.host.market_listing(true).is_err(), "refresh reads the index again");
}
#[test]
fn signed_marketplace_plugins_install_and_revocations_disable_them() {
let f = fixture();
@@ -260,3 +302,210 @@ fn package_files_install_from_disk_unsigned() {
let info = f.host.install_from_dir(&dir, &Consent::all()).unwrap();
assert_eq!((info.status, info.publisher), (PluginStatus::Inactive, None));
}
#[test]
fn the_marketplace_offers_the_newest_version_this_ide_can_load() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, old) = signed_hello(&market, "1.0.0", &key, &f.host);
let (_, new) = signed_hello(&market, "2.0.0", &key, &f.host);
let next_major = ide_extension::ENGINE_VERSION.split('.').next().unwrap().parse::<u64>().unwrap() + 1;
let new = IndexEntry { engine: format!("^{next_major}.0"), ..new };
f.host.set_market_url(Some(write_index(&market, &key, vec![old.clone(), new.clone()], vec![], false)));
let listed = f.host.refresh_market().unwrap();
assert_eq!((listed.len(), listed[0].entry.version.as_str(), listed[0].compatible), (1, "1.0.0", true));
let dir = f.host.download_market_plugin("acme.hello").unwrap();
assert_eq!(f.host.preview_install(&dir).unwrap().version, "1.0.0");
// Only a version for a newer IDE: shown as such, not downloaded.
write_index(&market, &key, vec![new], vec![], false);
let listed = f.host.refresh_market().unwrap();
assert_eq!((listed[0].entry.version.as_str(), listed[0].compatible), ("2.0.0", false));
let err = f.host.download_market_plugin("acme.hello").unwrap_err();
assert!(err.message.contains("update OpenIDEA"), "{}", err.message);
}
#[test]
fn a_moved_marketplace_is_read_where_it_moved_to() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, entry) = signed_hello(&market, "1.0.0", &key, &f.host);
write_index(&market, &key, vec![entry], vec![], false);
// The old address points at the new one; downloads resolve against where the index is.
let old = f.work.join("old");
std::fs::create_dir_all(&old).unwrap();
std::fs::write(old.join("index.json"), r#"{"format": 1, "moved": "../market/index.json"}"#).unwrap();
f.host.set_market_url(Some(old.join("index.json").to_string_lossy().into_owned()));
let listed = f.host.refresh_market().unwrap();
assert_eq!(listed[0].entry.id, "acme.hello");
let dir = f.host.download_market_plugin("acme.hello").unwrap();
assert_eq!(f.host.preview_install(&dir).unwrap().version, "1.0.0");
// A loop gives up.
std::fs::write(old.join("index.json"), r#"{"format": 1, "moved": "index.json"}"#).unwrap();
let err = f.host.refresh_market().unwrap_err();
assert!(err.message.contains("moved more than"), "{}", err.message);
}
/// `dir/index.json` listing `publishers` and `plugins`; its path.
fn index_of(dir: &Path, publishers: Vec<Publisher>, plugins: Vec<IndexEntry>) -> String {
let index = Index { format: package::INDEX_FORMAT, publishers, plugins, ..Index::default() };
let path = dir.join("index.json");
std::fs::write(&path, serde_json::to_string(&index).unwrap()).unwrap();
path.to_string_lossy().into_owned()
}
fn publisher(id: &str, keys: Vec<KeyEntry>) -> Publisher {
Publisher { id: id.into(), name: id.into(), keys }
}
fn updates_announced(events: &Events) -> Vec<Vec<String>> {
events
.lock()
.iter()
.filter_map(|e| match e {
PluginEvent::UpdatesAvailable { updates } => {
Some(updates.iter().map(|u| format!("{} {}", u.id, u.version)).collect())
}
_ => None,
})
.collect()
}
/// What the install page sends for an update it doesn't need to show: the grants the installed version has.
fn as_granted(preview: &InstallPreview) -> Consent {
let optional = preview.capabilities.iter().filter(|c| c.optional && c.granted).map(|c| c.id.clone()).collect();
Consent { optional, signer_change: false }
}
#[test]
fn updates_keep_their_signer_and_need_no_consent_when_nothing_is_new() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let (k1, k2, k3) =
(PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([2; 32]), PublisherKey::from_seed([3; 32]));
let same = |t: &str| t.to_owned();
let (_, v1) = signed(&market, "acme", "1.0.0", &k1, &f.host, same);
f.host.set_market_url(Some(index_of(
&market,
vec![publisher("acme", vec![KeyEntry::new(k1.public_base64())])],
vec![v1.clone()],
)));
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let first = f.host.preview_install(&dir).unwrap();
assert!(first.needs_consent && !first.signer_changed, "a first install is confirmed");
f.host.install_from_dir(&dir, &Consent::default()).unwrap();
assert!(updates_announced(&f.events).is_empty());
// 1.1.0, same key and needs: announced once, installed with the grants it has.
let (_, v11) = signed(&market, "acme", "1.1.0", &k1, &f.host, same);
let keys1 = vec![KeyEntry::new(k1.public_base64())];
index_of(&market, vec![publisher("acme", keys1.clone())], vec![v1.clone(), v11.clone()]);
f.host.refresh_market().unwrap();
f.host.refresh_market().unwrap();
assert_eq!(updates_announced(&f.events), [["acme.hello 1.1.0"]], "once per set of versions");
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert_eq!(preview.replaces.as_deref(), Some("1.0.0"));
assert!(!preview.needs_consent && !preview.signer_changed);
assert!(preview.capabilities.iter().all(|c| c.granted), "shows what the installed version was granted");
f.host.install_from_dir(&dir, &as_granted(&preview)).unwrap();
assert!(f.host.market_plugins().iter().all(|p| !p.is_update()));
// 1.2.0 asks for something new: the user is asked.
let (_, v12) = signed(&market, "acme", "1.2.0", &k1, &f.host, |t| {
t.replace(
"required = [\"editor.read\", \"editor.write\"]",
"required = [\"editor.read\", \"editor.write\"]\noptional = [\"net:example.com\"]",
)
});
index_of(&market, vec![publisher("acme", keys1.clone())], vec![v12.clone()]);
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert!(preview.needs_consent && !preview.signer_changed);
f.host.install_from_dir(&dir, &Consent::default()).unwrap();
// 1.3.0 signed with a key the installed one didn't endorse: another signer, the user's call.
let (_, v13) = signed(&market, "acme", "1.3.0", &k2, &f.host, same);
let keys12 = vec![KeyEntry::new(k1.public_base64()), KeyEntry::new(k2.public_base64())];
index_of(&market, vec![publisher("acme", keys12)], vec![v13]);
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert!(preview.signer_changed && preview.needs_consent);
let err = f.host.install_from_dir(&dir, &as_granted(&preview)).unwrap_err();
assert!(err.message.contains("another key"), "{}", err.message);
assert_eq!(f.host.info_of("acme.hello").unwrap().version, "1.2.0", "nothing changed");
f.host.install_from_dir(&dir, &Consent { signer_change: true, ..as_granted(&preview) }).unwrap();
// 1.4.0 signed with k3, which k2 endorsed: a rotation the installed signer vouched for.
let (_, v14) = signed(&market, "acme", "1.4.0", &k3, &f.host, same);
let keys = vec![
KeyEntry { endorsement: Some(k2.endorse("acme", &k3.public_base64())), ..KeyEntry::new(k3.public_base64()) },
KeyEntry::new(k2.public_base64()),
];
index_of(&market, vec![publisher("acme", keys)], vec![v14]);
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert!(!preview.signer_changed && !preview.needs_consent);
f.host.install_from_dir(&dir, &as_granted(&preview)).unwrap();
assert_eq!(f.host.info_of("acme.hello").unwrap().version, "1.4.0");
}
#[test]
fn an_index_in_a_newer_format_still_revokes() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, entry) = signed_hello(&market, "1.0.0", &key, &f.host);
f.host.set_market_url(Some(write_index(&market, &key, vec![entry], vec![], false)));
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
f.host.install_from_dir(&dir, &Consent::all()).unwrap();
let newer = format!(
r#"{{"format": {}, "plugins": "a shape this IDE can't read",
"revoked": [{{"id": "acme.hello", "reason": "steals tokens"}}]}}"#,
package::INDEX_FORMAT + 1
);
std::fs::write(market.join("index.json"), newer).unwrap();
let err = f.host.refresh_market().unwrap_err();
assert!(err.message.contains("newer OpenIDEA"), "{}", err.message);
let info = f.host.info_of("acme.hello").unwrap();
assert_eq!((info.status, info.error.as_deref()), (PluginStatus::Failed, Some("revoked: steals tokens")));
assert!(f.host.market_plugins().is_empty());
}
#[test]
fn official_plugins_need_the_key_this_ide_was_built_with() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let (root, intruder) = (PublisherKey::from_seed([1; 32]), PublisherKey::from_seed([6; 32]));
f.host.set_official_keys(vec![root.public_base64()]);
// The index lists the intruder's key for the official publisher.
let (_, forged) = signed(&market, OFFICIAL_PUBLISHER, "1.0.0", &intruder, &f.host, |t| t.to_owned());
let keys = vec![KeyEntry::new(intruder.public_base64())];
f.host.set_market_url(Some(index_of(&market, vec![publisher(OFFICIAL_PUBLISHER, keys.clone())], vec![forged])));
f.host.refresh_market().unwrap();
let err = f.host.download_market_plugin("openidea.hello").unwrap_err();
assert!(err.message.contains("trusted publisher"), "{}", err.message);
// Signed with the built-in key it installs, whatever keys the index lists.
let (_, genuine) = signed(&market, OFFICIAL_PUBLISHER, "1.0.1", &root, &f.host, |t| t.to_owned());
index_of(&market, vec![publisher(OFFICIAL_PUBLISHER, keys)], vec![genuine]);
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("openidea.hello").unwrap();
assert_eq!(f.host.preview_install(&dir).unwrap().publisher.as_deref(), Some(OFFICIAL_PUBLISHER));
}
+115
View File
@@ -0,0 +1,115 @@
//! Explicit external logs allow history reads without adding a writable project root.
use ide_core::NullSink;
use ide_host::{Host, HostConfig, ops::vcs};
use std::{path::Path, process::Command, sync::Arc};
fn git(root: &Path, args: &[&str]) -> String {
let output = Command::new("git")
.current_dir(root)
.args(["-c", "user.name=Test", "-c", "[email protected]", "-c", "commit.gpgsign=false"])
.args(args)
.output()
.unwrap();
assert!(output.status.success(), "{}", String::from_utf8_lossy(&output.stderr));
String::from_utf8(output.stdout).unwrap().trim().to_owned()
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn external_history_is_readable_and_never_becomes_a_write_target() {
let temp = tempfile::tempdir().unwrap();
let base = dunce::canonicalize(temp.path()).unwrap();
let root = base.join("project");
let external = base.join("external");
std::fs::create_dir_all(&root).unwrap();
std::fs::create_dir_all(&external).unwrap();
git(&external, &["init", "-b", "main"]);
let deleted = external.join("deleted.txt");
std::fs::write(&deleted, "historical content\n").unwrap();
git(&external, &["add", "."]);
git(&external, &["commit", "-m", "initial"]);
let first = git(&external, &["rev-parse", "HEAD"]);
git(&external, &["rm", "deleted.txt"]);
git(&external, &["commit", "-m", "delete"]);
let head = git(&external, &["rev-parse", "HEAD"]);
let host = Host::new(HostConfig {
config_dir: base.join("config"),
data_dir: base.join("data"),
local_data_dir: base.join("data"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new(|_| Arc::new(NullSink)),
owner: |w| w,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
});
// A standalone editor needs no project, indexer, or writable VCS root.
vcs::vcs_open_external_log(&host, "standalone", external.clone()).await.unwrap();
let standalone = vcs::vcs_log(&host, "standalone", Some(external.clone()), Default::default()).await.unwrap();
assert_eq!(standalone[0].id, head);
let details = vcs::vcs_log_details_batch(
&host,
"standalone",
Some(external.clone()),
vec![head.clone(), first.clone()],
ide_core::RequestId(987),
)
.await
.unwrap();
assert!(details.iter().all(|d| d.signature == "N"));
assert_eq!(
vcs::vcs_revision_children(&host, "standalone", Some(external.clone()), first.clone(), String::new())
.await
.unwrap(),
vec!["deleted.txt"]
);
assert!(host.project("standalone").is_err());
host.trust.decide(&external, false, false).unwrap();
let restricted = host.open_project("restricted", &external).unwrap();
assert!(!restricted.is_trusted());
vcs::vcs_open_external_log(&host, "restricted", external.clone()).await.unwrap();
let restricted_log = vcs::vcs_log(
&host,
"restricted",
Some(external.clone()),
ide_vcs::LogQuery { request: Some(ide_core::RequestId(988)), summary_only: true, ..Default::default() },
)
.await
.unwrap();
assert_eq!(restricted_log[0].id, head);
assert_eq!(
vcs::vcs_log_details_batch(
&host,
"restricted",
Some(external.clone()),
vec![head.clone()],
ide_core::RequestId(989)
)
.await
.unwrap()[0]
.signature,
"N"
);
host.close_project("restricted");
let payload =
serde_json::to_vec(&serde_json::json!({ "repo": external, "revision": first, "path": "deleted.txt" })).unwrap();
let hex = payload.iter().map(|b| format!("{b:02x}")).collect::<String>();
let document =
vcs::revision_virtual_document(&host, "standalone", &format!("git-revision:{hex}/deleted.txt")).await.unwrap();
assert_eq!(document.text, "historical content\n");
host.trust.decide(&root, true, false).unwrap();
let project = host.open_project("main", &root).unwrap();
assert!(vcs::vcs_log(&host, "main", Some(external.clone()), Default::default()).await.is_err());
vcs::vcs_open_external_log(&host, "main", external.clone()).await.unwrap();
let log = vcs::vcs_log(&host, "main", Some(external.clone()), Default::default()).await.unwrap();
assert_eq!(log.iter().map(|c| c.id.as_str()).collect::<Vec<_>>(), vec![head.as_str(), first.as_str()]);
assert_eq!(
vcs::vcs_file_at(&host, "main", deleted.clone(), Some(first.clone())).await.unwrap().as_deref(),
Some("historical content\n")
);
assert!(project.repos().at_root(&external).is_none());
assert!(vcs::vcs_checkout(&host, "main", Some(external.clone()), first.clone(), false, None).await.is_err());
assert!(vcs::vcs_apply_patch(&host, "main", Some(external.clone()), String::new()).await.is_err());
assert_eq!(git(&external, &["rev-parse", "HEAD"]), head);
assert!(!deleted.exists());
host.close_project("main");
}
+49
View File
@@ -0,0 +1,49 @@
use ide_core::NullSink;
use ide_host::{Host, HostConfig, ops};
use std::sync::Arc;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn links_are_validated_persisted_and_unlinked_without_removing_files() {
let temp = tempfile::tempdir().unwrap();
let base = dunce::canonicalize(temp.path()).unwrap();
let root = base.join("workspace");
for dir in [root.join("a"), root.join("b"), base.join("external")] {
std::fs::create_dir_all(&dir).unwrap();
std::fs::write(dir.join("settings.gradle"), "rootProject.name = 'same-name'").unwrap();
}
let config = base.join("config");
let host = Host::new(HostConfig {
config_dir: config.clone(),
data_dir: base.join("data"),
local_data_dir: base.join("local"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new(|_| Arc::new(NullSink)),
owner: |window| window,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
});
host.open_project("main", &root).unwrap();
assert!(ops::build::gradle_link_projects(&host, "main", vec!["b".into()]).await.is_err());
host.set_project_trust(&root, true, false).await.unwrap();
let model = ops::build::gradle_link_projects(&host, "main", vec!["b/settings.gradle".into()]).await.unwrap();
assert_eq!(model.gradle.iter().map(|g| g.dir.as_str()).collect::<Vec<_>>(), ["a", "b"]);
let saved =
|| ide_workspace::SettingsStore::load(&config, Some(&root)).effective().languages.java.gradle.linked_projects;
assert_eq!(saved(), Some(vec!["a".into(), "b".into()]));
assert!(ops::build::gradle_link_projects(&host, "main", vec!["missing".into()]).await.is_err());
assert_eq!(saved(), Some(vec!["a".into(), "b".into()]));
let outside = base.join("external").to_string_lossy().into_owned();
let model = ops::build::gradle_link_projects(&host, "main", vec![outside.clone()]).await.unwrap();
assert_eq!(model.gradle.len(), 3);
let model = ops::build::gradle_unlink_projects(&host, "main", vec!["a".into(), "b".into(), outside]).await.unwrap();
assert!(model.gradle.is_empty());
assert_eq!(saved(), Some(vec![]));
for dir in [root.join("a"), root.join("b"), base.join("external")] {
assert!(dir.join("settings.gradle").is_file());
}
let model = ops::build::gradle_link_projects(&host, "main", vec!["b".into()]).await.unwrap();
assert_eq!(model.gradle.len(), 1);
assert_eq!(model.gradle[0].dir, "b");
host.shutdown().await;
}
+994
View File
@@ -0,0 +1,994 @@
//! Language smoke tests: the everyday code intelligence of each language — completion, go to
//! definition, Find Usages, hover, diagnostics with a quick fix, an intention, rename,
//! formatting, parameter info, the Structure view, auto-import, Optimize Imports and moving a
//! file — on its `fixtures/` project, through the `ops` the app's commands call and the
//! language server the app starts by default.
//!
//! `#[ignore]`d because they need the servers installed. CI's `Language smoke` job installs
//! pinned versions and runs one language per job:
//! `cargo test -p ide-host --test language_smoke -- --ignored --exact <language>`.
use ide_core::{ChangeSet, DocId, IdeEvent, Pos, Range, RequestId, TextChange};
use ide_host::{Host, HostConfig, ops};
use ide_lsp::types::{FormatOptions, GotoKind};
use std::collections::BTreeSet;
use std::future::Future;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
const WINDOW: &str = "main";
/// `word` on the first line of `file` that contains `line`.
#[derive(Clone, Copy, Debug)]
struct At {
file: &'static str,
line: &'static str,
word: &'static str,
}
const fn at(file: &'static str, line: &'static str, word: &'static str) -> At {
At { file, line, word }
}
/// A line typed after the first line of `file` containing `after`.
#[derive(Clone, Copy, Debug)]
struct Typed {
file: &'static str,
after: &'static str,
text: &'static str,
}
const fn typed(file: &'static str, after: &'static str, text: &'static str) -> Typed {
Typed { file, after, text }
}
/// One language's checklist. Every check names what it looks for, so a failure says which
/// operation broke.
struct Checks {
fixture: &'static str,
/// Files written into the project copy before it opens.
files: &'static [(&'static str, &'static str)],
/// How long the first answer may take (project import and indexing).
warm_up: Duration,
/// Go to definition from the first spot lands on the second.
definition: (At, At),
/// Find Usages of the symbol lists usages in these files.
references: (At, &'static [&'static str]),
/// Hover shows this text.
hover: (At, &'static str),
/// Completion at the end of the typed line offers this item.
completion: (Typed, &'static str),
/// The typed line is reported with this diagnostic; the quick fix with this title makes the
/// file contain the last text.
quick_fix: QuickFix,
/// With the word selected, this intention is offered with an edit or command.
intention: (At, &'static str),
/// Formatting turns the typed line into the expected one.
format: (Typed, &'static str),
indent: Indent,
/// Renaming the symbol to the new name edits these files.
rename: (At, &'static str, &'static [&'static str]),
/// Parameter info in a call being typed: a signature naming this parameter.
signature: (Typed, &'static str),
/// The Structure view of a file: these symbols, at any depth.
outline: (&'static str, &'static [&'static str]),
/// Completing a name the file doesn't import yet also imports it.
auto_import: AutoImport,
/// Optimize Imports, with an import typed after a line.
organize_imports: Option<(Typed, Imports)>,
/// Moving a file updates its references: (file, new path, a file that then contains this).
file_move: Option<(&'static str, &'static str, &'static str, &'static str)>,
}
#[derive(Clone, Copy, Debug)]
struct AutoImport {
/// The line typed, ending in the start of the name.
typed: Typed,
/// The completion item.
label: &'static str,
/// What the item's import adds to the file.
result: &'static str,
}
/// What Optimize Imports does with the typed import.
#[derive(Clone, Copy, Debug)]
enum Imports {
/// Removes it (unused).
Removed,
/// Sorts it before this line.
Before(&'static str),
}
#[derive(Clone, Copy, Debug)]
struct QuickFix {
typed: Typed,
diagnostic: &'static str,
title: &'static str,
result: &'static str,
}
/// The editor's indentation, sent with a format request.
#[derive(Clone, Copy, Debug)]
enum Indent {
Spaces(u32),
Tabs,
}
type Events = Arc<Mutex<Vec<IdeEvent>>>;
struct Smoke {
language: &'static str,
host: Arc<Host>,
root: PathBuf,
events: Events,
warm_up: Duration,
_tmp: tempfile::TempDir,
}
fn fixture_dir(name: &str) -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR")).join("../../fixtures").join(name)
}
fn copy_dir(from: &Path, to: &Path) {
std::fs::create_dir_all(to).unwrap();
for entry in std::fs::read_dir(from).unwrap() {
let entry = entry.unwrap();
let name = entry.file_name();
if matches!(name.to_str(), Some("node_modules" | "target" | "build" | ".gradle" | ".venv")) {
continue;
}
let target = to.join(&name);
if entry.file_type().unwrap().is_dir() {
copy_dir(&entry.path(), &target);
} else {
std::fs::copy(entry.path(), target).unwrap();
}
}
}
fn utf16_len(s: &str) -> u32 {
s.encode_utf16().count() as u32
}
/// The 0-based line index and text of the first line containing `needle`.
fn find_line<'a>(text: &'a str, needle: &str, what: &str) -> (u32, &'a str) {
text.lines()
.enumerate()
.find(|(_, l)| l.contains(needle))
.map(|(i, l)| (i as u32, l))
.unwrap_or_else(|| panic!("{what}: no line containing {needle:?}"))
}
/// The byte offset of `word` in `line`, not as part of a longer identifier (`parse` in `parser::parse`).
fn find_word(line: &str, word: &str) -> Option<usize> {
let ident = |c: char| c.is_alphanumeric() || c == '_';
line.match_indices(word).map(|(i, _)| i).find(|&i| {
let before = line[..i].chars().next_back();
let after = line[i + word.len()..].chars().next();
!(word.starts_with(ident) && before.is_some_and(ident)) && !(word.ends_with(ident) && after.is_some_and(ident))
})
}
impl Smoke {
async fn start(language: &'static str, checks: &Checks) -> Self {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap().join(checks.fixture);
copy_dir(&fixture_dir(checks.fixture), &root);
for (path, text) in checks.files {
let path = root.join(path);
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
std::fs::write(path, text).unwrap();
}
let events: Events = Arc::default();
let app = tmp.path().join("app");
let host = Arc::new(Host::new(HostConfig {
config_dir: app.join("config"),
data_dir: app.join("data"),
local_data_dir: app.join("data"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new({
let events = events.clone();
move |_| {
let events = events.clone();
Arc::new(move |e: IdeEvent| events.lock().unwrap().push(e))
}
}),
owner: |w| w,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
}));
host.trust.decide(&root, true, false).unwrap();
host.open_project(WINDOW, &root).unwrap();
Smoke { language, host, root, events, warm_up: checks.warm_up, _tmp: tmp }
}
fn fail(&self, check: &str, message: impl std::fmt::Display) -> ! {
let servers = ops::lsp::lsp_servers(&self.host, WINDOW).unwrap_or_default();
let servers: Vec<String> = servers.iter().map(|s| format!("{} ({:?})", s.name, s.status)).collect();
panic!("{} — {check}: {message}\nservers: {servers:?}", self.language)
}
/// Retries `f` until it succeeds (servers answer emptily while importing and indexing); on
/// timeout, fails with its last answer.
async fn retry<T, D: std::fmt::Debug, F: Future<Output = Result<T, D>>>(
&self,
check: &str,
timeout: Duration,
mut f: impl FnMut() -> F,
) -> T {
let deadline = Instant::now() + timeout;
loop {
let last = match f().await {
Ok(v) => return v,
Err(last) => last,
};
if Instant::now() > deadline {
self.fail(check, format_args!("timed out after {timeout:?}; last answer: {last:?}"));
}
tokio::time::sleep(Duration::from_millis(500)).await;
}
}
async fn doc(&self, file: &str) -> (DocId, String) {
let info = ops::doc::open(&self.host, WINDOW, self.root.join(file)).await.unwrap();
let info = ops::doc::snapshot(&self.host, WINDOW, info.id).await.unwrap();
(info.id, info.text)
}
async fn locate(&self, at: At) -> (DocId, Pos) {
let (doc, text) = self.doc(at.file).await;
let (line, content) = find_line(&text, at.line, at.file);
let col = find_word(content, at.word).unwrap_or_else(|| panic!("{at:?}: word not on the line"));
(doc, Pos::new(line, utf16_len(&content[..col])))
}
async fn word_range(&self, at: At) -> (DocId, Range) {
let (doc, start) = self.locate(at).await;
(doc, Range::new(start, Pos::new(start.line, start.col + utf16_len(at.word))))
}
/// Types a new line; returns the position at its end and the text before it.
async fn type_line(&self, t: Typed) -> (DocId, Pos, String) {
let (doc, text) = self.doc(t.file).await;
let (line, content) = find_line(&text, t.after, t.file);
let line_start: usize = text.lines().take(line as usize).map(|l| l.len() + 1).sum();
let end = utf16_len(&text[..line_start + content.len()]);
self.replace(doc, end, end, &format!("\n{}", t.text)).await;
let col = utf16_len(t.text);
(doc, Pos::new(line + 1, col), text)
}
async fn replace(&self, doc: DocId, from: u32, to: u32, insert: &str) {
let version = ops::doc::snapshot(&self.host, WINDOW, doc).await.unwrap().version;
let changes = ChangeSet { changes: vec![TextChange { from, to, insert: insert.into() }] };
ops::doc::edit(&self.host, WINDOW, doc, version, changes).unwrap();
}
async fn restore(&self, doc: DocId, text: &str) {
let current = ops::doc::snapshot(&self.host, WINDOW, doc).await.unwrap().text;
self.replace(doc, 0, utf16_len(&current), text).await;
}
fn path(&self, file: &str) -> PathBuf {
self.root.join(file)
}
fn files<P: AsRef<Path>>(&self, paths: impl IntoIterator<Item = P>) -> BTreeSet<String> {
paths
.into_iter()
.filter_map(|p| {
let p = p.as_ref().strip_prefix(&self.root).ok()?;
Some(p.to_string_lossy().replace('\\', "/"))
})
.collect()
}
async fn definition(&self, from: At, to: At) {
let (doc, pos) = self.locate(from).await;
let (_, target) = self.locate(to).await;
let want = self.path(to.file);
self.retry("go to definition", self.warm_up, || async {
let found = ops::lsp::lsp_goto(&self.host, WINDOW, doc, pos, GotoKind::Definition).await;
match &found {
Ok(locs) if locs.iter().any(|l| *l.path == *want && l.range.start.line == target.line) => Ok(()),
_ => Err(found),
}
})
.await;
}
async fn references(&self, of: At, files: &[&str]) {
let (doc, pos) = self.locate(of).await;
let want: BTreeSet<String> = files.iter().map(|f| f.to_string()).collect();
self.retry("find usages", self.warm_up, || async {
let found = ops::lsp::lsp_references(&self.host, WINDOW, RequestId(1), doc, pos, true, true).await;
match &found {
Ok(r) if want.is_subset(&self.files(r.locations.iter().map(|l| l.path.clone()))) => Ok(()),
_ => Err(found.map(|r| self.files(r.locations.into_iter().map(|l| l.path)))),
}
})
.await;
}
async fn hover(&self, over: At, shows: &str) {
let (doc, pos) = self.locate(over).await;
self.retry("hover", self.warm_up, || async {
let found = ops::lsp::lsp_hover(&self.host, WINDOW, RequestId(2), doc, pos).await;
match &found {
Ok(Some(h)) if h.contents.value.contains(shows) => Ok(()),
_ => Err(found.map(|h| h.map(|h| h.contents.value))),
}
})
.await;
}
async fn completion(&self, line: Typed, offers: &str) {
let (doc, pos, before) = self.type_line(line).await;
let trigger = line.text.ends_with('.').then(|| ".".to_string());
self.retry("completion", self.warm_up, || async {
let found = ops::lsp::lsp_completion(&self.host, WINDOW, RequestId(3), doc, pos, trigger.clone()).await;
match &found {
Ok(list)
if list.items.iter().any(|i| i.label == offers || i.filter_text.as_deref() == Some(offers)) =>
{
Ok(())
}
_ => Err(found.map(|l| l.items.into_iter().take(30).map(|i| i.label).collect::<Vec<_>>())),
}
})
.await;
self.restore(doc, &before).await;
}
async fn diagnostic(&self, file: &str, message: &str) -> Range {
let path = self.path(file);
self.retry("diagnostics", self.warm_up, || async {
let events = self.events.lock().unwrap();
let latest = events.iter().rev().filter_map(|e| match e {
IdeEvent::Diagnostics { path: p, items, source } if **p == *path => Some((source, items)),
_ => None,
});
let mut seen = Vec::new();
for (source, items) in latest {
if let Some(d) = items.iter().find(|d| d.message.contains(message)) {
return Ok(d.range);
}
seen.push(format!("{source}: {:?}", items.iter().map(|d| &d.message).collect::<Vec<_>>()));
}
Err(seen.into_iter().take(4).collect::<Vec<_>>())
})
.await
}
/// The code actions over `range` (all kinds, as the ⌥⏎ popup asks), the one titled `title`.
async fn action(&self, check: &str, doc: DocId, range: Range, title: &str) -> ide_lsp::types::CodeAction {
let action = self
.retry(check, self.warm_up, || async {
let found = ops::lsp::lsp_code_actions(&self.host, WINDOW, doc, range, vec![]).await;
match found.as_ref().ok().and_then(|a| a.iter().find(|a| a.title.contains(title))) {
Some(action) => Ok(action.clone()),
None => Err(found.map(|a| a.into_iter().map(|a| a.title).collect::<Vec<_>>())),
}
})
.await;
if action.edit.is_some() || action.command.is_some() {
return action;
}
ops::lsp::lsp_resolve_code_action(&self.host, WINDOW, doc, action)
.await
.unwrap_or_else(|e| self.fail(check, format_args!("resolving {title:?}: {e}")))
}
async fn quick_fix(&self, fix: QuickFix) {
let (doc, _, before) = self.type_line(fix.typed).await;
assert!(!before.contains(fix.result), "{fix:?}: the file already contains the result");
let range = self.diagnostic(fix.typed.file, fix.diagnostic).await;
let action = self.action("quick fix", doc, range, fix.title).await;
ops::lsp::lsp_execute_code_action(&self.host, WINDOW, RequestId(4), doc, action, vec![])
.await
.unwrap_or_else(|e| self.fail("quick fix", format_args!("applying {:?}: {e}", fix.title)));
self.retry("quick fix", Duration::from_secs(10), || async {
let text = self.doc(fix.typed.file).await.1;
if text.contains(fix.result) { Ok(()) } else { Err(text) }
})
.await;
self.restore(doc, &before).await;
}
async fn intention(&self, on: At, title: &str) {
let (doc, range) = self.word_range(on).await;
let action = self.action("intention", doc, range, title).await;
let edits = action.edit.as_ref().is_some_and(|e| !e.files.is_empty());
if !edits && action.command.is_none() {
self.fail("intention", format_args!("{title:?} has neither an edit nor a command: {action:?}"));
}
}
async fn format(&self, line: Typed, indent: Indent, formatted: &str) {
let (doc, _, before) = self.type_line(line).await;
assert!(!before.lines().any(|l| l == formatted), "{line:?}: the file already has {formatted:?}");
let (tab_size, insert_spaces) = match indent {
Indent::Spaces(n) => (n, true),
Indent::Tabs => (4, false),
};
let options = FormatOptions { tab_size, insert_spaces, on_save: false, tool: None };
let text = self
.retry("format", self.warm_up, || async {
let edits = ops::lsp::lsp_format(&self.host, WINDOW, doc, None, options.clone()).await;
let text = self.doc(line.file).await.1;
let mut document = ide_text::Document::new(&text);
match &edits {
Ok(edits) if !edits.is_empty() && document.apply_edits(edits).is_ok() => Ok(document.text()),
_ => Err(edits),
}
})
.await;
if !text.lines().any(|l| l == formatted) {
self.fail("format", format_args!("no line {formatted:?} in:\n{text}"));
}
self.restore(doc, &before).await;
}
async fn rename(&self, symbol: At, to: &str, files: &[&str]) {
let (doc, pos) = self.locate(symbol).await;
let prepared = ops::lsp::lsp_prepare_rename(&self.host, WINDOW, doc, pos).await;
if !matches!(prepared, Ok(Some(_))) {
self.fail("rename", format_args!("prepare rename: {prepared:?}"));
}
let renamed = ops::lsp::lsp_rename(&self.host, WINDOW, RequestId(5), doc, pos, to.into())
.await
.unwrap_or_else(|e| self.fail("rename", e));
let edited = self.files(renamed.edit.files.iter().filter_map(|f| match f {
ide_core::FileEdit::Edit { path, .. } => Some(path.clone()),
_ => None,
}));
let want: BTreeSet<String> = files.iter().map(|f| f.to_string()).collect();
if !want.is_subset(&edited) {
self.fail("rename", format_args!("edits {edited:?}, expected at least {want:?}: {:?}", renamed.edit));
}
let mut before = Vec::new();
for file in files {
before.push(self.doc(file).await.1.matches(to).count());
}
ops::lsp::apply_workspace_edit(&self.host, WINDOW, renamed.edit)
.await
.unwrap_or_else(|e| self.fail("rename", format_args!("applying the edit: {e}")));
for (file, before) in files.iter().zip(before) {
if self.doc(file).await.1.matches(to).count() <= before {
self.fail("rename", format_args!("{file} has no new {to:?} after the rename"));
}
}
}
async fn signature(&self, call: Typed, parameter: &str) {
let (doc, pos, before) = self.type_line(call).await;
self.retry("parameter info", self.warm_up, || async {
let found = ops::lsp::lsp_signature_help(&self.host, WINDOW, doc, pos).await;
match &found {
Ok(Some(h)) if h.signatures.iter().any(|s| s.label.contains(parameter)) => Ok(()),
_ => Err(found.map(|h| h.map(|h| h.signatures.into_iter().map(|s| s.label).collect::<Vec<_>>()))),
}
})
.await;
self.restore(doc, &before).await;
}
async fn outline(&self, file: &str, symbols: &[&str]) {
let (doc, _) = self.doc(file).await;
fn names(items: &[ide_core::OutlineItem], out: &mut BTreeSet<String>) {
for item in items {
out.insert(item.name.clone());
names(&item.children, out);
}
}
self.retry("structure", self.warm_up, || async {
let found = ops::doc::doc_outline(&self.host, WINDOW, doc).await;
let mut shown = BTreeSet::new();
if let Ok(items) = &found {
names(items, &mut shown);
}
if symbols.iter().all(|s| shown.contains(*s)) { Ok(()) } else { Err(shown) }
})
.await;
}
async fn auto_import(&self, a: AutoImport) {
let (doc, pos, before) = self.type_line(a.typed).await;
assert!(!before.contains(a.result), "{a:?}: the file already contains the result");
self.retry("auto-import completion", self.warm_up, || async {
let list = ops::lsp::lsp_completion(&self.host, WINDOW, RequestId(6), doc, pos, None).await;
let items = match &list {
Ok(l) => l.items.iter().filter(|i| i.label == a.label).cloned().collect::<Vec<_>>(),
Err(e) => return Err(format!("{e}")),
};
let text = self.doc(a.typed.file).await.1;
let mut imports = Vec::new();
for item in items {
// Servers compute the import when the item is resolved (on selection).
let item = if item.additional_edits.is_empty() {
ops::lsp::lsp_resolve_completion(&self.host, WINDOW, doc, item.clone()).await.unwrap_or(item)
} else {
item
};
let mut document = ide_text::Document::new(&text);
if document.apply_edits(&item.additional_edits).is_ok() && document.text().contains(a.result) {
return Ok(());
}
// Or accepting it runs a server command that edits the file (kotlin-lsp).
if let Some(command) = item.command.clone() {
let ran = ops::lsp::lsp_execute_command(&self.host, WINDOW, doc, command).await;
let text = self.doc(a.typed.file).await.1;
if ran.is_ok() && text.contains(a.result) {
return Ok(());
}
}
imports.push(format!("{:?} {:?}", item.additional_edits, item.command));
}
let labels: Vec<String> = list.map(|l| l.items.into_iter().take(20).map(|i| i.label).collect()).unwrap();
Err(format!("{:?} items' imports: {imports:?}; items: {labels:?}", a.label))
})
.await;
self.restore(doc, &before).await;
}
async fn organize_imports(&self, import: Typed, expect: Imports) {
let (doc, _, before) = self.type_line(import).await;
let typed = import.text.trim();
let organized = |text: &str| match expect {
Imports::Removed => !text.lines().any(|l| l.trim() == typed),
Imports::Before(line) => {
let at = |needle: &str| text.lines().position(|l| l.trim() == needle);
matches!((at(typed), at(line)), (Some(a), Some(b)) if a < b)
}
};
self.retry("optimize imports", self.warm_up, || async {
let text = self.doc(import.file).await.1;
// The whole file, as Optimize Imports asks (positions past the end fail on jdtls).
let last = text.split('\n').count() as u32 - 1;
let whole = Range::new(Pos::new(0, 0), Pos::new(last, utf16_len(text.rsplit('\n').next().unwrap_or(""))));
let kind = "source.organizeImports".to_string();
let found = ops::lsp::lsp_code_actions(&self.host, WINDOW, doc, whole, vec![kind.clone()]).await;
let actions: Vec<_> = match &found {
Ok(a) => a.iter().filter(|a| a.kind.as_ref().is_some_and(|k| k.starts_with(&kind))).cloned().collect(),
Err(e) => return Err(format!("{e}")),
};
let Some(action) = actions.iter().find(|a| a.is_preferred).or(actions.first()).cloned() else {
return Err(format!(
"no {kind} action: {:?}",
found.map(|a| a.into_iter().map(|a| a.title).collect::<Vec<_>>())
));
};
ops::lsp::lsp_apply_code_action(&self.host, WINDOW, doc, action).await.map_err(|e| e.to_string())?;
let text = self.doc(import.file).await.1;
if organized(&text) { Ok(()) } else { Err(text) }
})
.await;
self.restore(doc, &before).await;
}
async fn file_move(&self, from: &str, to: &str, referencing: &str, result: &str) {
// The servers know the referencing file (opened, as in an editor).
let _ = self.doc(referencing).await;
let moves = vec![ide_workspace::FileMove { from: self.path(from).into(), to: self.path(to).into() }];
let edit = self
.retry("move file", self.warm_up, || async {
let found =
ops::refactor::refactor_file_move_edits(&self.host, WINDOW, RequestId(7), moves.clone()).await;
let want = self.path(referencing);
match &found {
Ok(e)
if e.edit
.files
.iter()
.any(|f| matches!(f, ide_core::FileEdit::Edit { path, .. } if **path == *want)) =>
{
Ok(e.edit.clone())
}
_ => Err(found),
}
})
.await;
ops::refactor::refactor_move_files(&self.host, WINDOW, moves, edit, "Move".into())
.await
.unwrap_or_else(|e| self.fail("move file", e));
if !self.path(to).is_file() {
self.fail("move file", format_args!("{to} doesn't exist after the move"));
}
let text = self.doc(referencing).await.1;
if !text.contains(result) {
self.fail("move file", format_args!("{referencing} has no {result:?} after the move:\n{text}"));
}
}
async fn run(&self, c: &Checks) {
self.definition(c.definition.0, c.definition.1).await;
self.references(c.references.0, c.references.1).await;
self.hover(c.hover.0, c.hover.1).await;
self.completion(c.completion.0, c.completion.1).await;
self.quick_fix(c.quick_fix).await;
self.intention(c.intention.0, c.intention.1).await;
self.format(c.format.0, c.indent, c.format.1).await;
self.signature(c.signature.0, c.signature.1).await;
self.outline(c.outline.0, c.outline.1).await;
self.auto_import(c.auto_import).await;
if let Some((import, expect)) = c.organize_imports {
self.organize_imports(import, expect).await;
}
self.rename(c.rename.0, c.rename.1, c.rename.2).await;
// Last: it moves a file the other checks use.
if let Some((from, to, referencing, result)) = c.file_move {
self.file_move(from, to, referencing, result).await;
}
let shutdown = self.host.project(WINDOW).unwrap().lsp.clone();
let _ = tokio::time::timeout(Duration::from_secs(20), shutdown.shutdown_all()).await;
}
}
async fn smoke(language: &'static str, checks: Checks) {
let smoke = Smoke::start(language, &checks).await;
smoke.run(&checks).await;
}
const TYPESCRIPT: Checks = Checks {
fixture: "ts-app",
files: &[],
warm_up: Duration::from_secs(120),
definition: (
at("src/index.ts", "Total: ${formatMoney(total)}", "formatMoney"),
at("src/money.ts", "export function formatMoney", "formatMoney"),
),
references: (at("src/money.ts", "export function formatMoney", "formatMoney"), &["src/index.ts", "src/money.ts"]),
hover: (at("src/index.ts", "price: cents(8999)", "cents"), "cents(value: number): Cents"),
completion: (typed("src/index.ts", "const total = cart.total();", " cart."), "applyDiscount"),
quick_fix: QuickFix {
typed: typed("src/index.ts", "const total = cart.total();", " console.log(addMoney(total, total));"),
diagnostic: "Cannot find name 'addMoney'",
title: "import from \"./money.ts\"",
result: "import { addMoney, cents, formatMoney } from \"./money.ts\";",
},
intention: (at("src/index.ts", "const total = cart.total();", "cart.total()"), "Extract to constant"),
format: (
typed("src/index.ts", "const total = cart.total();", " const shown={total,count:cart.count}"),
" const shown = { total, count: cart.count }",
),
indent: Indent::Spaces(2),
rename: (
// From a usage in another file: the declaration is renamed too, not aliased in the import.
at("src/index.ts", "Total: ${formatMoney(total)}", "formatMoney"),
"formatPrice",
&["src/index.ts", "src/money.ts"],
),
signature: (typed("src/index.ts", "const total = cart.total();", " cart.add("), "product"),
outline: ("src/cart.ts", &["Cart", "add", "total"]),
auto_import: AutoImport {
typed: typed("src/index.ts", "const total = cart.total();", " addMon"),
label: "addMoney",
result: "addMoney",
},
organize_imports: Some((
typed(
"src/index.ts",
"import { cents, formatMoney } from \"./money.ts\";",
"import { readFileSync } from \"node:fs\";",
),
Imports::Removed,
)),
file_move: Some(("src/money.ts", "src/lib/money.ts", "src/index.ts", "\"./lib/money.ts\"")),
};
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "language smoke: needs vtsls"]
async fn typescript() {
smoke("typescript", TYPESCRIPT).await;
}
const PYTHON: Checks = Checks {
fixture: "python-app",
// Ruff (lint, quick fixes, formatting) starts in projects configured for it.
files: &[("ruff.toml", "")],
warm_up: Duration::from_secs(120),
definition: (
at("src/tasks/cli.py", "tasks = TaskList()", "TaskList"),
at("src/tasks/models.py", "class TaskList:", "TaskList"),
),
references: (
at("src/tasks/models.py", "class TaskList:", "TaskList"),
&["src/tasks/cli.py", "src/tasks/models.py"],
),
hover: (at("src/tasks/cli.py", "tasks.add(\"Write release notes\"", "add"), "def add("),
completion: (typed("src/tasks/cli.py", "tasks = build_demo()", " tasks."), "pending"),
quick_fix: QuickFix {
typed: typed("src/tasks/cli.py", "tasks = build_demo()", " print(Task)"),
diagnostic: "\"Task\" is not defined",
title: "from .models import Task",
result: "from .models import Priority, Task, TaskList",
},
// basedpyright has no refactorings; Ruff's fixes are the Python intentions.
intention: (at("src/tasks/models.py", "import os", "os"), "Remove unused import"),
format: (
typed("src/tasks/cli.py", "tasks = build_demo()", " shown2 = {'a':1,'b': 2}"),
" shown2 = {\"a\": 1, \"b\": 2}",
),
indent: Indent::Spaces(4),
rename: (
at("src/tasks/models.py", "class TaskList:", "TaskList"),
"TaskBoard",
&["src/tasks/cli.py", "src/tasks/models.py"],
),
signature: (typed("src/tasks/cli.py", "tasks = build_demo()", " tasks.add("), "title"),
outline: ("src/tasks/models.py", &["Priority", "Task", "TaskList", "pending"]),
auto_import: AutoImport {
typed: typed("src/tasks/cli.py", "tasks = build_demo()", " stamp = datetim"),
label: "datetime",
result: "import datetime",
},
organize_imports: Some((
typed("src/tasks/models.py", "from enum import IntEnum", "import abc"),
Imports::Before("import os"),
)),
// basedpyright has no `workspace/willRenameFiles`.
file_move: None,
};
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "language smoke: needs basedpyright and ruff"]
async fn python() {
smoke("python", PYTHON).await;
}
const RUST: Checks = Checks {
fixture: "rust-app",
files: &[],
warm_up: Duration::from_secs(180),
definition: (
at("src/main.rs", "match parser::parse(SAMPLE)", "parse"),
at("src/parser.rs", "pub fn parse(text: &str)", "parse"),
),
references: (at("src/parser.rs", "pub fn parse(text: &str)", "parse"), &["src/main.rs", "src/parser.rs"]),
hover: (at("src/main.rs", "inventory.take(&Sku", "take"), "pub fn take("),
completion: (typed("src/main.rs", "let mut inventory = Inventory::new();", " inventory."), "low_stock"),
quick_fix: QuickFix {
// rust-analyzer's own analysis, pulled as the file changes (no save, no `cargo check`).
typed: typed(
"src/main.rs",
"let mut inventory = Inventory::new();",
" let _blank = parser::ParseError { line: 1 };",
),
diagnostic: "missing structure fields",
title: "Fill struct fields",
result: "ParseError { line: 1, message",
},
intention: (at("src/main.rs", "match parser::parse(SAMPLE)", "parser::parse(SAMPLE)"), "Extract into variable"),
format: (
typed("src/main.rs", "let mut inventory = Inventory::new();", " let _spaced=( 1,2 );"),
" let _spaced = (1, 2);",
),
indent: Indent::Spaces(4),
rename: (at("src/parser.rs", "pub fn parse(text: &str)", "parse"), "parse_all", &["src/main.rs", "src/parser.rs"]),
signature: (typed("src/main.rs", "let mut inventory = Inventory::new();", " inventory.take("), "sku"),
outline: ("src/lib.rs", &["Inventory", "Item", "take"]),
auto_import: AutoImport {
typed: typed("src/main.rs", "let mut inventory = Inventory::new();", " let _seen = HashSe"),
label: "HashSet",
result: "use std::collections::HashSet;",
},
// rust-analyzer has no `source.organizeImports`.
organize_imports: None,
file_move: Some(("src/parser.rs", "src/csv.rs", "src/lib.rs", "pub mod csv;")),
};
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "language smoke: needs rust-analyzer"]
async fn rust() {
smoke("rust", RUST).await;
}
const GO: Checks = Checks {
fixture: "go-app",
files: &[],
warm_up: Duration::from_secs(180),
definition: (
at("cmd/server/main.go", "s := store.New()", "New"),
at("internal/store/store.go", "func New() *Store", "New"),
),
references: (
at("internal/store/store.go", "func New() *Store", "New"),
&["cmd/server/main.go", "internal/store/store.go"],
),
hover: (
at("cmd/server/main.go", "writeJSON(w, s.Search(", "Search"),
"func (s *store.Store) Search(q string) []store.Note",
),
completion: (typed("cmd/server/main.go", "s := store.New()", "\ts."), "Create"),
quick_fix: QuickFix {
typed: typed("cmd/server/main.go", "s := store.New()", "\tfmt.Println(s)"),
diagnostic: "undefined: fmt",
title: "\"fmt\"",
result: "\t\"fmt\"",
},
intention: (at("cmd/server/main.go", "s := store.New()", "store.New()"), "Extract"),
format: (typed("cmd/server/main.go", "s := store.New()", "\t_ = []int{1,2}"), "\t_ = []int{1, 2}"),
indent: Indent::Tabs,
rename: (
at("internal/store/store.go", "func New() *Store", "New"),
"Open",
&["cmd/server/main.go", "internal/store/store.go"],
),
signature: (typed("cmd/server/main.go", "s := store.New()", "\ts.Create("), "title"),
outline: ("internal/store/store.go", &["Store", "New", "(*Store).Create"]),
auto_import: AutoImport {
typed: typed("cmd/server/main.go", "s := store.New()", "\t_ = strings.ToUpp"),
label: "ToUpper",
result: "\t\"strings\"",
},
organize_imports: Some((typed("cmd/server/main.go", "\"strconv\"", "\t\"sort\""), Imports::Removed)),
// gopls renames packages, not files: moving a Go file changes no references.
file_move: None,
};
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "language smoke: needs gopls"]
async fn go() {
smoke("go", GO).await;
}
const JAVA: Checks = Checks {
fixture: "java-app",
files: &[],
warm_up: Duration::from_secs(300),
definition: (
at("src/main/java/com/example/orders/App.java", "book.place(\"alice\", \"Mechanical", "place"),
at("src/main/java/com/example/orders/OrderBook.java", "public Order place(", "place"),
),
references: (
at("src/main/java/com/example/orders/OrderBook.java", "public Order place(", "place"),
&["src/main/java/com/example/orders/App.java", "src/test/java/com/example/orders/OrderBookTest.java"],
),
hover: (
at("src/main/java/com/example/orders/App.java", "long revenue = book.revenueCents()", "revenueCents"),
"long com.example.orders.OrderBook.revenueCents()",
),
completion: (
typed("src/main/java/com/example/orders/App.java", "long revenue = book.revenueCents();", " book."),
"largest",
),
quick_fix: QuickFix {
typed: typed(
"src/main/java/com/example/orders/App.java",
"long revenue = book.revenueCents();",
" List<Order> all = book.byCustomer(\"alice\");",
),
diagnostic: "List cannot be resolved to a type",
title: "Import 'List' (java.util)",
result: "import java.util.List;",
},
intention: (
at("src/main/java/com/example/orders/App.java", "long revenue = book.revenueCents();", "book.revenueCents()"),
"Extract to local variable",
),
format: (
typed(
"src/main/java/com/example/orders/App.java",
"long revenue = book.revenueCents();",
" long doubled=revenue*2;",
),
" long doubled = revenue * 2;",
),
indent: Indent::Spaces(4),
rename: (
at("src/main/java/com/example/orders/OrderBook.java", "public Order place(", "place"),
"submit",
&["src/main/java/com/example/orders/App.java", "src/main/java/com/example/orders/OrderBook.java"],
),
signature: (
typed(
"src/main/java/com/example/orders/App.java",
"long revenue = book.revenueCents();",
" book.place(",
),
"customer",
),
outline: (
"src/main/java/com/example/orders/OrderBook.java",
&["OrderBook", "place(String, String, int, long)", "revenueCents()"],
),
auto_import: AutoImport {
typed: typed(
"src/main/java/com/example/orders/App.java",
"long revenue = book.revenueCents();",
" ArrayLis",
),
label: "ArrayList",
result: "import java.util.ArrayList;",
},
organize_imports: Some((
typed("src/main/java/com/example/orders/OrderBook.java", "import java.util.Optional;", "import java.util.Map;"),
Imports::Removed,
)),
file_move: Some((
"src/main/java/com/example/orders/OrderBook.java",
"src/main/java/com/example/orders/Ledger.java",
"src/main/java/com/example/orders/App.java",
"Ledger book",
)),
};
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "language smoke: needs jdtls and JDK 21"]
async fn java() {
smoke("java", JAVA).await;
}
const KOTLIN: Checks = Checks {
fixture: "kotlin-gradle",
files: &[],
warm_up: Duration::from_secs(420),
definition: (
at("src/main/kotlin/com/example/library/App.kt", "val library = Library()", "Library"),
at("src/main/kotlin/com/example/library/Library.kt", "class Library(", "Library"),
),
references: (
at("src/main/kotlin/com/example/library/Library.kt", "fun search(query: String)", "search"),
&["src/main/kotlin/com/example/library/App.kt", "src/main/kotlin/com/example/library/Library.kt"],
),
hover: (
at("src/main/kotlin/com/example/library/App.kt", "library.search(\"java\")", "search"),
"fun search(query: String): List<Book>",
),
completion: (
typed("src/main/kotlin/com/example/library/App.kt", "val library = Library()", " library."),
"lend",
),
quick_fix: QuickFix {
typed: typed(
"src/main/kotlin/com/example/library/App.kt",
"val library = Library()",
" val stamp = LocalDateTime.now()",
),
diagnostic: "Unresolved reference",
title: "Import",
result: "import java.time.LocalDateTime",
},
intention: (
at("src/main/kotlin/com/example/library/Library.kt", "fun giveBack(isbn: String): Boolean", "giveBack"),
"block body",
),
format: (
typed("src/main/kotlin/com/example/library/App.kt", "val library = Library()", " val pair=listOf(1,2)"),
" val pair = listOf(1, 2)",
),
indent: Indent::Spaces(4),
rename: (
at("src/main/kotlin/com/example/library/Library.kt", "fun search(query: String)", "search"),
"find",
&["src/main/kotlin/com/example/library/App.kt", "src/main/kotlin/com/example/library/Library.kt"],
),
signature: (
typed("src/main/kotlin/com/example/library/App.kt", "val library = Library()", " library.lend("),
"isbn",
),
outline: ("src/main/kotlin/com/example/library/Library.kt", &["Library", "lend", "search"]),
auto_import: AutoImport {
typed: typed(
"src/main/kotlin/com/example/library/App.kt",
"val library = Library()",
" val stamp = LocalDateTi",
),
label: "LocalDateTime",
result: "import java.time.LocalDateTime",
},
organize_imports: Some((
typed("src/main/kotlin/com/example/library/App.kt", "import java.time.LocalDate", "import java.util.UUID"),
Imports::Removed,
)),
// kotlin-lsp has no `workspace/willRenameFiles`.
file_move: None,
};
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "language smoke: needs kotlin-lsp, a JDK and Gradle"]
async fn kotlin() {
smoke("kotlin", KOTLIN).await;
}
+257 -32
View File
@@ -123,6 +123,19 @@ fn misbehave_pkg() -> PathBuf {
repo().join("crates/ide-plugin/tests/fixtures/misbehave")
}
/// Whether `f` holds within 5 s. A call a slow runner keeps waiting behind
/// another (or behind a new instance starting) holds once it caught up.
fn eventually(mut f: impl FnMut() -> bool) -> bool {
let until = Instant::now() + Duration::from_secs(5);
while !f() {
if Instant::now() > until {
return false;
}
std::thread::sleep(Duration::from_millis(20));
}
true
}
/// A theme package with a light and a dark theme, plus licence files, in `dir`.
fn write_theme_pair(dir: &Path) {
std::fs::create_dir_all(dir.join("t")).unwrap();
@@ -397,11 +410,11 @@ fn crashes_restart_with_backoff_until_the_circuit_opens() {
let info = f.host.info_of("misbehave").unwrap();
assert_eq!((info.status, info.stats.activations, info.error), (PluginStatus::Active, 2, None));
// Three in a row: stopped for the session, with growing waits between.
// Three in a row: stopped for the session, backing off between them (the wait
// doubles: `Limits::backoff`, tested where it is defined; wall-clock waits on CI
// are too noisy to compare).
f.take_events();
let mut waits = Vec::new();
for _ in 0..3 {
let started = Instant::now();
for crash in 0..3 {
let err = loop {
match f.run("misbehave", "panic", None) {
Err(e) if e.contains("restarts in") => std::thread::sleep(Duration::from_millis(20)),
@@ -409,9 +422,10 @@ fn crashes_restart_with_backoff_until_the_circuit_opens() {
}
};
assert!(err.contains("crashed"), "{err}");
waits.push(started.elapsed());
if crash < 2 {
assert!(f.run("misbehave", "echo", None).unwrap_err().contains("restarts in"), "backing off");
}
}
assert!(waits[2] >= backoff * 2 && waits[2] > waits[1], "{waits:?}");
let info = f.host.info_of("misbehave").unwrap();
assert_eq!(info.status, PluginStatus::Failed);
assert!(info.error.as_deref().is_some_and(|e| e.contains("3 times in a row")), "{:?}", info.error);
@@ -479,12 +493,15 @@ fn running_calls_can_be_cancelled() {
let f = f.clone();
std::thread::spawn(move || f.run("misbehave", "spin", None))
};
std::thread::sleep(Duration::from_millis(100));
let started = Instant::now();
f.host.cancel("misbehave");
// A cancel before the call runs (the instance may still be starting) does nothing: repeat it.
let until = Instant::now() + Duration::from_secs(5);
while !spinning.is_finished() {
assert!(Instant::now() < until, "the running call wasn't interrupted");
f.host.cancel("misbehave");
std::thread::sleep(Duration::from_millis(20));
}
let err = spinning.join().unwrap().unwrap_err();
assert!(err.contains("was cancelled"), "{err}");
assert!(started.elapsed() < Duration::from_secs(1));
assert_eq!(f.run("misbehave", "echo", Some("x")).unwrap().as_deref(), Some("x"), "no backoff");
let stats = f.host.info_of("misbehave").unwrap().stats;
assert_eq!((stats.crashes, stats.activations), (0, 2));
@@ -526,16 +543,22 @@ fn edit_path_calls_have_soft_and_hard_deadlines() {
let provider = host.syntax_provider();
let mis = LanguageId::new("mis");
let tokens = provider.highlights(&mis, "fast").unwrap();
// The first call starts the instance, which may take longer than the soft deadline.
let mut tokens = None;
assert!(eventually(|| {
tokens = provider.highlights(&mis, "fast");
tokens.is_some()
}));
let tokens = tokens.unwrap();
assert_eq!((tokens[0].start, tokens[0].end, tokens[0].kind.as_str()), (0, 4, "keyword"));
// Slower than the soft deadline: dropped (however late this thread gets
// to look), but the instance lives on.
assert!(provider.highlights(&mis, "sleep:150").is_none());
std::thread::sleep(Duration::from_millis(150));
assert!(provider.highlights(&mis, "fast").is_some());
assert!(eventually(|| provider.highlights(&mis, "fast").is_some()));
let info = host.info_of("misbehave").unwrap();
assert_eq!((info.stats.slow_calls, info.stats.crashes, info.stats.activations), (1, 0, 1));
assert!(info.stats.slow_calls >= 1, "{:?}", info.stats);
assert_eq!((info.stats.crashes, info.stats.activations), (0, 1));
// Past the hard deadline: interrupted; the next call gets a new instance.
assert!(provider.highlights(&mis, "spin").is_none());
@@ -544,7 +567,8 @@ fn edit_path_calls_have_soft_and_hard_deadlines() {
let info = host.info_of("misbehave").unwrap();
assert_eq!(info.stats.crashes, 1, "{:?}", info.error);
assert!(info.error.as_deref().is_some_and(|e| e.contains("timed out after 400ms")), "{:?}", info.error);
assert!(provider.highlights(&mis, "fast").is_some());
// The new instance may take longer to start than the soft deadline on a slow runner.
assert!(eventually(|| provider.highlights(&mis, "fast").is_some()));
assert_eq!(host.info_of("misbehave").unwrap().stats.activations, 2);
}
@@ -553,9 +577,10 @@ fn edit_path_calls_have_soft_and_hard_deadlines() {
#[test]
fn providers_of_several_plugins_run_concurrently() {
let tmp = tempfile::tempdir().unwrap();
// A budget no runner misses, so only the overlap is measured.
let limits = Limits {
edit_soft: Duration::from_millis(300),
edit_hard: Duration::from_secs(2),
edit_soft: Duration::from_secs(5),
edit_hard: Duration::from_secs(10),
fuel: u64::MAX,
..Limits::default()
};
@@ -584,16 +609,17 @@ fn providers_of_several_plugins_run_concurrently() {
text: text.into(),
};
let at = ide_core::Pos::new(0, 0);
// Activation is not what is measured.
assert_eq!(host.provide_completion(&doc("fast"), at, None).len(), 2);
// Activation is not what is measured: a first call that starts an instance
// can run past the edit-path budget, and its answer is dropped.
assert!(eventually(|| host.provide_completion(&doc("fast"), at, None).len() == 2));
let started = Instant::now();
let items = host.provide_completion(&doc("sleep:200"), at, None);
let items = host.provide_completion(&doc("sleep:500"), at, None);
let elapsed = started.elapsed();
let mut from: Vec<&str> = items.iter().map(|(p, _)| p.as_str()).collect();
from.sort();
assert_eq!(from, ["slow-a", "slow-b"], "both within the budget");
assert!(elapsed < Duration::from_millis(350), "one after the other would take 400ms: {elapsed:?}");
assert!(elapsed < Duration::from_millis(950), "one after the other would take 1000ms: {elapsed:?}");
assert_eq!(host.info_of("slow-b").unwrap().stats.slow_calls, 0);
}
@@ -613,7 +639,7 @@ fn idle_instances_are_reclaimed_and_schedules_reactivate() {
assert_eq!(f.host.instances(), 0);
let info = f.host.info_of("misbehave").unwrap();
assert_eq!((info.status, info.stats.reclaims), (PluginStatus::Inactive, 1));
assert_eq!(f.command_titles("misbehave").len(), 8, "contributions stay");
assert_eq!(f.command_titles("misbehave").len(), 10, "contributions stay");
assert_eq!(f.run("misbehave", "echo", Some("y")).unwrap().as_deref(), Some("y"));
assert_eq!(f.host.info_of("misbehave").unwrap().stats.activations, 2);
@@ -1064,7 +1090,16 @@ fn plugin_syntax_for_languages_without_a_grammar() {
let sol = LanguageId::new("solidity");
let text = "// c\ncontract A {\n uint256 public x;\n function f() public {}\n}\n";
assert!(provider.highlights(&LanguageId::new("rust"), text).is_none(), "only its own languages");
let tokens = provider.highlights(&sol, text).unwrap();
// The first call of a new instance can run past the edit-path budget on a
// slow machine; its answer is dropped and a later call asks again.
let deadline = Instant::now() + Duration::from_secs(10);
let tokens = loop {
match provider.highlights(&sol, text) {
Some(tokens) => break tokens,
None if Instant::now() < deadline => std::thread::sleep(Duration::from_millis(50)),
None => panic!("no highlights"),
}
};
let kind_at =
|s: &str| tokens.iter().find(|t| &text[t.start as usize..t.end as usize] == s).map(|t| t.kind.as_str());
assert_eq!(kind_at("contract"), Some("keyword"));
@@ -1642,22 +1677,21 @@ fn official_ai_completion_cancels_superseded_requests() {
};
let at = ide_core::Pos::new(1, 4);
let host = Arc::new(f);
// A slow request, superseded 200 ms later by the next keystroke's.
// A slow request, superseded by the next keystroke's once the server has it
// (starting the instance comes first, and takes longer on a slow runner).
let first = ide_core::CancellationToken::new();
let slow = {
let (host, doc, first) = (host.clone(), doc.clone(), first.clone());
std::thread::spawn(move || {
let started = Instant::now();
let r = host.host.provide_inline_completion(&doc, at, &first);
(r, started.elapsed())
})
std::thread::spawn(move || (host.host.provide_inline_completion(&doc, at, &first), Instant::now()))
};
std::thread::sleep(Duration::from_millis(200));
assert!(eventually(|| seen.lock().len() == 1), "the first request reaches the server");
let cancelled = Instant::now();
first.cancel();
let started = Instant::now();
let second = host.host.provide_inline_completion(&doc, at, &ide_core::CancellationToken::new());
let took = started.elapsed();
let (old, old_took) = slow.join().unwrap();
let (old, old_done) = slow.join().unwrap();
let old_took = old_done.saturating_duration_since(cancelled);
assert!(old.is_none(), "the superseded request yields nothing");
assert!(old_took < Duration::from_secs(1), "cancelled, not waited out: {old_took:?}");
assert_eq!(second.map(|(_, s)| s.insert_text).as_deref(), Some("a + b"));
@@ -1835,7 +1869,12 @@ fn official_mybatis_navigates_between_interface_and_xml() {
assert_eq!((target.path, target.range.start), (xml_path.clone(), ide_core::Pos::new(1, 14)));
let xdoc = doc(&xml_path, "xml", xml);
let to_java = f.host.provide_definition(&xdoc, ide_core::Pos::new(1, 16));
// Reads the interface from disk: on a slow machine that too can run past the budget.
let mut to_java = Vec::new();
assert!(eventually(|| {
to_java = f.host.provide_definition(&xdoc, ide_core::Pos::new(1, 16));
!to_java.is_empty()
}));
assert_eq!(
to_java.iter().map(|l| (l.path.clone(), l.range.start)).collect::<Vec<_>>(),
[(java_path.clone(), ide_core::Pos::new(3, 9))]
@@ -2006,6 +2045,83 @@ fn pinned_programs_fill_views_and_open_terminals() {
assert_eq!(args, ["follow", "build", "--lines=5"]);
}
#[test]
fn views_open_project_files_at_a_line() {
let f = fixture(Limits::default());
let info = f.install(&sample("jobs"));
assert_eq!(info.status, PluginStatus::Inactive, "{:?}", info.error);
std::fs::write(f.project.join("jobs.log"), "build\ndeploy\nfailed\n").unwrap();
let press = |button: &str| {
f.take_events();
let result = f.host.view_event("jobs", "jobs", ide_plugin::ViewEvent::Button { id: button.into() }, f.inv());
let opened = f.take_events().into_iter().find_map(|(client, e)| match e {
PluginEvent::OpenLocation { path, line, column, .. } => Some((client, path.0, line, column)),
_ => None,
});
(result, opened)
};
let (result, opened) = press("log");
assert!(result.is_ok(), "{result:?}");
let (client, path, line, column) = opened.expect("a location");
assert_eq!((client.as_deref(), line, column), (Some("main"), 2, 4));
assert_eq!(path.canonicalize().unwrap(), f.project.join("jobs.log").canonicalize().unwrap());
let (result, opened) = press("outside");
assert!(result.is_err(), "a path outside the project is refused");
assert!(opened.is_none());
f.host.set_capability("jobs", "project.read", false).unwrap();
let (result, opened) = press("log");
assert!(result.unwrap_err().message.contains("project.read"), "needs project.read");
assert!(opened.is_none());
}
#[test]
fn views_can_use_an_svg_icon_from_the_package() {
let f = fixture(Limits::default());
let tmp = tempfile::tempdir().unwrap();
let clean = r##"<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16"><rect width="16" height="16" fill="#08f"/></svg>"##;
let package = |dir: &str, icon: &str, svg: &str| {
let pkg = tmp.path().join(dir);
for (path, text) in [("panel/index.html", "<p>panel</p>"), ("icons/panel.svg", svg)] {
std::fs::create_dir_all(pkg.join(path).parent().unwrap()).unwrap();
std::fs::write(pkg.join(path), text).unwrap();
}
std::fs::write(
pkg.join("plugin.toml"),
format!(
r#"id = "panel-icon"
name = "Panel Icon"
version = "1.0.0"
engines.openidea = "^1.0"
[[contributes.views]]
id = "panel"
title = "Panel"
icon = "{icon}"
location = "bottom"
web = "panel/index.html"
"#
),
)
.unwrap();
pkg
};
assert!(f.host.install_from_dir(&package("missing", "icons/missing.svg", clean), &Consent::all()).is_err());
let scripted = clean.replace("<rect", "<script>alert(1)</script><rect");
assert!(f.host.install_from_dir(&package("scripted", "icons/panel.svg", &scripted), &Consent::all()).is_err());
let info = f.install(&package("clean", "icons/panel.svg", clean));
assert!(info.error.is_none(), "{:?}", info.error);
let view = f.host.ui_items("views").into_iter().find(|v| v.key == "panel").expect("the view");
assert_eq!(view.decl["icon"].as_str(), Some("icons/panel.svg"));
let assets = f.host.assets("panel-icon", "viewIcon/panel").unwrap();
assert_eq!(assets.len(), 1);
assert!(assets[0].1.contains("<rect"));
}
/// A plugin with web views and no code: its pages and their files are
/// installed with it and served only from the declared directories, while
/// it is enabled and only for the installed version; file icons and
@@ -2300,6 +2416,57 @@ fn a_zipped_plugin_folder_installs_like_the_folder() {
assert!(err.contains("is not an OpenIDEA plugin"), "{err}");
}
/// What the UI writes for an imported IntelliJ theme goes through the install page.
#[test]
fn a_generated_package_is_staged_for_the_install_page() {
let f = fixture(Limits::default());
let file = |path: &str, text: &str| PackageFile { path: path.into(), text: text.into() };
let manifest = r#"id = "intellij.deep-sea"
name = "Deep Sea"
version = "2.3.0"
engines.openidea = "^1.0"
[[contributes.themes]]
path = "themes/deep-sea.theme.json"
editor-scheme = "themes/deep-sea.scheme.json"
"#;
let files = [
file("plugin.toml", manifest),
file(
"themes/deep-sea.theme.json",
r##"{"id":"deep-sea","name":"Deep Sea","kind":"dark","tokens":{"--bg-app":"#101820"}}"##,
),
file("themes/deep-sea.scheme.json", r##"{"name":"Deep Sea","tokens":{"--syn-keyword":"#cc7832"}}"##),
];
let dir = f.host.stage_package("intellij.deep-sea", &files).unwrap();
let preview = f.host.preview_install(&dir).unwrap();
assert_eq!((preview.id.as_str(), preview.has_code, preview.unsigned), ("intellij.deep-sea", false, true));
assert!(preview.capabilities.is_empty());
f.install(&dir);
let active = f.host.set_theme(Some(("intellij.deep-sea", "deep-sea"))).unwrap().unwrap();
assert_eq!(active.theme.tokens["--bg-app"], "#101820");
assert_eq!(active.theme.tokens["--syn-keyword"], "#cc7832");
// Staging the same package again starts from an empty folder; importing it again replaces it.
assert_eq!(f.host.stage_package("intellij.deep-sea", &files[..1]).unwrap(), dir);
assert!(!dir.join("themes").exists());
let mut changed = files.clone();
changed[2] = file("themes/deep-sea.scheme.json", r##"{"tokens":{"--syn-keyword":"#ff8800"}}"##);
f.install(&f.host.stage_package("intellij.deep-sea", &changed).unwrap());
let active = f.host.set_theme(Some(("intellij.deep-sea", "deep-sea"))).unwrap().unwrap();
assert_eq!(active.theme.tokens["--syn-keyword"], "#ff8800");
for bad in ["../x", "/abs", "a//b", r"a\b", "C:x", ".hidden", "a/./b", ""] {
let err = f.host.stage_package("p", &[file("plugin.toml", manifest), file(bad, "x")]).unwrap_err().message;
assert!(err.contains("invalid package path"), "{bad:?}: {err}");
}
let err = f.host.stage_package("p", &[file("x.json", "{}")]).unwrap_err().message;
assert!(err.contains("needs a plugin.toml"), "{err}");
let err = f.host.stage_package("p", &[file("plugin.toml", ""), file("PLUGIN.toml", "")]).unwrap_err().message;
assert!(err.contains("twice"), "{err}");
assert!(f.host.stage_package("../p", &files).is_err());
}
/// Secrets live outside the plugin's directory (the OS keychain on the
/// desktop): uninstalling the plugin deletes the ones it declares.
#[test]
@@ -2328,3 +2495,61 @@ fn uninstalling_deletes_the_plugins_secrets() {
f.host.uninstall("openidea.ai-completion").unwrap();
assert!(keychain.0.lock().is_empty(), "{:?}", keychain.0.lock().keys().collect::<Vec<_>>());
}
/// Plugin API 1.0 leaves room to grow: event kinds are names, and the editor
/// providers carry snippets, imports, lazily computed quick fixes and tagged
/// problems.
#[test]
fn completion_quick_fixes_problems_and_event_kinds_of_api_1() {
let f = fixture(Limits::default());
let pkg = f._tmp.path().join("misbehave-providers");
copy_dir(&misbehave_pkg(), &pkg);
let manifest = std::fs::read_to_string(pkg.join("plugin.toml")).unwrap();
std::fs::write(
pkg.join("plugin.toml"),
format!("{manifest}\n[[contributes.providers]]\nkinds = [\"completion\", \"code-actions\"]\nlanguages = [\"text\"]\n"),
)
.unwrap();
f.install(&pkg);
let doc = |text: &str| ide_plugin::ProviderDocument {
path: f.project.join("a.txt"),
language: "text".into(),
version: 1,
text: text.into(),
};
let at = ide_core::Pos::new(0, 0);
// The first call starts the instance, which may take longer than the edit-path budget.
let mut items = Vec::new();
assert!(eventually(|| {
items = f.host.provide_completion(&doc("rich"), at, None);
!items.is_empty()
}));
let item = &items[0].1;
assert_eq!(
(item.insert_text.as_str(), item.snippet, item.filter_text.as_deref()),
("f(${1:x})$0", true, Some("fun"))
);
assert_eq!(item.additional_edits[0].new_text, "use f;\n");
// Offered without edits; computed when picked.
let actions = f.host.provide_code_actions(&doc("lazy"), ide_core::Range::default());
let (plugin, action) = actions.into_iter().next().expect("offered");
assert!(action.edits.is_empty() && action.data.as_deref() == Some("fix"));
let resolved = f.host.resolve_code_action(&plugin, &doc("lazy"), action.clone()).unwrap();
assert_eq!(resolved.edits[0].1[0].new_text, "fixed ");
let foreign = ide_plugin::CodeAction { data: Some("other".into()), ..action };
assert!(f.host.resolve_code_action(&plugin, &doc("lazy"), foreign).is_err());
// A kind this IDE doesn't know is refused, not a trap.
let err = f.host.run_command("misbehave", "subscribe", Some("file-deleted"), f.inv()).unwrap_err();
assert!(err.message.contains("no event kind"), "{}", err.message);
let err = f.host.run_command("misbehave", "subscribe", Some("file-saved"), f.inv()).unwrap_err();
assert!(err.message.contains("events"), "known, but not granted: {}", err.message);
// Tags the IDE knows, the tool that found it.
let path = f.project.join("a.txt");
f.host.run_command("misbehave", "diagnose", Some(&path.to_string_lossy()), f.inv()).unwrap();
let d = f.workspace.diagnostics.lock().get(&("misbehave".into(), path)).cloned().unwrap();
assert_eq!((d[0].unnecessary, d[0].deprecated, d[0].source.as_deref()), (true, false, Some("lint")));
}
+289
View File
@@ -0,0 +1,289 @@
//! Opening each `fixtures/` project the way the app does, through the `ops` its commands call:
//! the source and test roots detected from its build file, the index (files and symbols),
//! Search Everywhere / Go to Class / Go to Symbol answered from it and kept current as files
//! come and go, and the Maven / Gradle model.
//!
//! The `#[ignore]`d tests run Maven, Gradle, python3 and a JDK 21 (CI's `Language smoke`
//! java, kotlin and python jobs):
//! `cargo test -p ide-host --test project_smoke -- --ignored`.
use ide_host::{Host, HostConfig, ops};
use ide_index::SymbolFilter;
use ide_workspace::roots::RootKind;
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::time::{Duration, Instant};
const WINDOW: &str = "main";
struct Fixture {
dir: &'static str,
/// Detected roots: (project-relative path, kind).
roots: &'static [(&'static str, RootKind)],
/// Search Everywhere › Files: (query, file found).
file: (&'static str, &'static str),
/// Go to Class: (name, file declaring it).
class: (&'static str, &'static str),
/// Go to Symbol: (name, file declaring it).
symbol: (&'static str, &'static str),
/// A file created while the project is open: (path, text declaring the class `FreshWidget`).
added: (&'static str, &'static str),
}
struct Opened {
host: Arc<Host>,
root: PathBuf,
_tmp: tempfile::TempDir,
}
fn copy_dir(from: &Path, to: &Path) {
std::fs::create_dir_all(to).unwrap();
for entry in std::fs::read_dir(from).unwrap().flatten() {
let (src, dst) = (entry.path(), to.join(entry.file_name()));
if entry.file_type().unwrap().is_dir() {
copy_dir(&src, &dst);
} else {
std::fs::copy(&src, &dst).unwrap();
}
}
}
async fn open(fixture: &str) -> Opened {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap().join(fixture);
copy_dir(&Path::new(env!("CARGO_MANIFEST_DIR")).join("../../fixtures").join(fixture), &root);
let app = tmp.path().join("app");
let host = Arc::new(Host::new(HostConfig {
config_dir: app.join("config"),
data_dir: app.join("data"),
local_data_dir: app.join("data"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new(|_| Arc::new(|_| {})),
owner: |w| w,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
}));
host.trust.decide(&root, true, false).unwrap();
ops::project::open(&host, WINDOW, &root).await.unwrap();
Opened { host, root, _tmp: tmp }
}
/// Polls `check` until it holds; fails with its last answer.
async fn until<T: std::fmt::Debug>(what: &str, timeout: Duration, mut check: impl FnMut() -> Result<(), T>) {
let deadline = Instant::now() + timeout;
loop {
match check() {
Ok(()) => return,
Err(last) if Instant::now() > deadline => panic!("{what}: timed out after {timeout:?}; last: {last:?}"),
Err(_) => tokio::time::sleep(Duration::from_millis(100)).await,
}
}
}
impl Opened {
fn files(&self, query: &str) -> Vec<String> {
let found = ops::nav::find_files(&self.host, WINDOW, query.into(), 20).unwrap();
found.into_iter().map(|f| f.rel_path).collect()
}
fn symbols(&self, query: &str, filter: SymbolFilter) -> Vec<(String, String)> {
let found = ops::nav::find_symbols(&self.host, WINDOW, query.into(), filter, 20).unwrap();
found.into_iter().map(|s| (s.name, s.rel_path)).collect()
}
fn has_symbol(&self, name: &str, file: &str, filter: SymbolFilter) -> Result<(), Vec<(String, String)>> {
let found = self.symbols(name, filter);
if found.iter().any(|(n, f)| n == name && f == file) { Ok(()) } else { Err(found) }
}
}
async fn smoke(f: Fixture) {
let p = open(f.dir).await;
until("indexing", Duration::from_secs(30), || {
let s = ops::nav::index_status(&p.host, WINDOW).unwrap();
if !s.indexing && s.files > 0 && s.symbols > 0 { Ok(()) } else { Err(s) }
})
.await;
let roots = ops::project::roots(&p.host, WINDOW).unwrap().roots;
for (path, kind) in f.roots {
assert!(
roots.iter().any(|r| r.path == *path && r.kind == *kind),
"{}: no {kind:?} root {path:?} in {roots:?}",
f.dir
);
}
let files = p.files(f.file.0);
assert!(files.iter().any(|r| r == f.file.1), "{}: files for {:?}: {files:?}", f.dir, f.file.0);
if let Err(found) = p.has_symbol(f.class.0, f.class.1, SymbolFilter::Types) {
panic!("{}: Go to Class {:?}: {found:?}", f.dir, f.class.0);
}
if let Err(found) = p.has_symbol(f.symbol.0, f.symbol.1, SymbolFilter::All) {
panic!("{}: Go to Symbol {:?}: {found:?}", f.dir, f.symbol.0);
}
// The index follows the file system while the project is open.
let (added, text) = f.added;
std::fs::write(p.root.join(added), text).unwrap();
until("a new file's class", Duration::from_secs(20), || p.has_symbol("FreshWidget", added, SymbolFilter::Types))
.await;
std::fs::remove_file(p.root.join(added)).unwrap();
until("a deleted file's class", Duration::from_secs(20), || {
let found = p.symbols("FreshWidget", SymbolFilter::Types);
if found.is_empty() { Ok(()) } else { Err(found) }
})
.await;
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn rust_app() {
smoke(Fixture {
dir: "rust-app",
roots: &[("src", RootKind::Source), ("tests", RootKind::Test)],
file: ("pars", "src/parser.rs"),
class: ("Inventory", "src/lib.rs"),
symbol: ("low_stock", "src/lib.rs"),
added: ("src/fresh.rs", "pub struct FreshWidget;\n"),
})
.await;
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn ts_app() {
smoke(Fixture {
dir: "ts-app",
roots: &[("src", RootKind::Source), ("tests", RootKind::Test)],
file: ("cart", "src/cart.ts"),
class: ("Cart", "src/cart.ts"),
symbol: ("formatMoney", "src/money.ts"),
added: ("src/fresh.ts", "export class FreshWidget {}\n"),
})
.await;
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn python_app() {
smoke(Fixture {
dir: "python-app",
roots: &[("src", RootKind::Source), ("tests", RootKind::Test)],
file: ("models", "src/tasks/models.py"),
class: ("TaskList", "src/tasks/models.py"),
symbol: ("build_demo", "src/tasks/cli.py"),
added: ("src/tasks/fresh.py", "class FreshWidget:\n pass\n"),
})
.await;
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn go_app() {
smoke(Fixture {
dir: "go-app",
roots: &[("", RootKind::Source)],
file: ("store", "internal/store/store.go"),
class: ("Store", "internal/store/store.go"),
symbol: ("writeJSON", "cmd/server/main.go"),
added: ("internal/store/fresh.go", "package store\n\ntype FreshWidget struct{}\n"),
})
.await;
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn java_app() {
smoke(Fixture {
dir: "java-app",
roots: &[("src/main/java", RootKind::Source), ("src/test/java", RootKind::Test)],
file: ("OrderBook", "src/main/java/com/example/orders/OrderBook.java"),
class: ("OrderBook", "src/main/java/com/example/orders/OrderBook.java"),
symbol: ("revenueCents", "src/main/java/com/example/orders/OrderBook.java"),
added: (
"src/main/java/com/example/orders/FreshWidget.java",
"package com.example.orders;\n\npublic class FreshWidget {}\n",
),
})
.await;
let p = open("java-app").await;
let model = ops::build::build_model(&p.host, WINDOW).await.unwrap();
let maven = model.maven.expect("a Maven model");
assert_eq!(maven.projects.iter().map(|m| m.artifact_id.as_str()).collect::<Vec<_>>(), ["openidea-fixture-orders"]);
assert!(maven.projects[0].dependencies.iter().any(|d| d.artifact_id == "junit-jupiter"), "{maven:?}");
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn kotlin_gradle() {
smoke(Fixture {
dir: "kotlin-gradle",
roots: &[("src/main/kotlin", RootKind::Source), ("src/test/kotlin", RootKind::Test)],
file: ("Library", "src/main/kotlin/com/example/library/Library.kt"),
class: ("Library", "src/main/kotlin/com/example/library/Library.kt"),
symbol: ("giveBack", "src/main/kotlin/com/example/library/Library.kt"),
added: ("src/main/kotlin/com/example/library/Fresh.kt", "package com.example.library\n\nclass FreshWidget\n"),
})
.await;
let p = open("kotlin-gradle").await;
let model = ops::build::build_model(&p.host, WINDOW).await.unwrap();
assert_eq!(model.gradle.len(), 1, "one independent Gradle build");
let gradle = &model.gradle[0];
assert_eq!(gradle.projects.iter().map(|g| g.name.as_str()).collect::<Vec<_>>(), ["openidea-fixture-library"]);
}
/// Maven / Gradle sync (Reload in the build tool window): the build's own model.
async fn sync(fixture: &str, tool: ide_build::BuildTool) -> ide_build::BuildModel {
let p = open(fixture).await;
let progress = ops::Channel::new(|_: ide_build::SyncProgress| {});
let model = ops::build::build_sync(&p.host, WINDOW, tool, ide_core::RequestId(1), false, vec![], None, progress)
.await
.unwrap();
// The cached sync is what the next open shows.
let reloaded = ops::build::build_model(&p.host, WINDOW).await.unwrap();
assert_eq!(reloaded, model);
model
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "project smoke: needs Maven and a JDK"]
async fn maven_sync() {
let maven = sync("java-app", ide_build::BuildTool::Maven).await.maven.unwrap();
assert_eq!(maven.sync_error, None);
assert!(maven.synced_at.is_some());
let junit = maven.projects[0].dependencies.iter().find(|d| d.artifact_id == "junit-jupiter").unwrap();
// The resolved tree, not only the declaration.
assert!(!junit.children.is_empty(), "{junit:?}");
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "project smoke: needs Gradle and a JDK"]
async fn gradle_sync() {
let model = sync("kotlin-gradle", ide_build::BuildTool::Gradle).await;
assert_eq!(model.gradle.len(), 1, "one independent Gradle build");
let gradle = &model.gradle[0];
assert_eq!(gradle.sync_error, None);
assert!(gradle.synced_at.is_some());
let tasks: Vec<&str> = gradle.projects[0].tasks.iter().map(|t| t.name.as_str()).collect();
// The build's own tasks: the application and Kotlin plugins' too.
for task in ["run", "compileKotlin", "test"] {
assert!(tasks.contains(&task), "no {task} in {tasks:?}");
}
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "project smoke: needs python3"]
async fn python_venv() {
let p = open("python-app").await;
let made = std::process::Command::new("python3").args(["-m", "venv", ".venv"]).current_dir(&p.root).status();
assert!(made.unwrap().success());
// The project's virtualenv: listed first, and the interpreter the project uses.
let found = ops::sdk::sdk_detect(&p.host, WINDOW, ide_sdk::SdkKind::Python).await.unwrap();
assert_eq!(found.first().map(|s| s.source.as_str()), Some("venv"), "{found:?}");
let effective = ops::sdk::sdk_python_effective(&p.host, WINDOW).unwrap().expect("an interpreter");
assert!(effective.starts_with(p.root.join(".venv")), "{effective:?}");
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "project smoke: needs JDK 21"]
async fn jdk() {
let p = open("java-app").await;
let found = ops::sdk::sdk_detect(&p.host, WINDOW, ide_sdk::SdkKind::Jdk).await.unwrap();
assert!(found.iter().any(|s| s.version.as_deref().is_some_and(|v| v.starts_with("21"))), "{found:?}");
}
Loaded 100 of 959 files, more files were not shown because too many files have changed in this diff. Show more