Compare commits

...
Author SHA1 Message Date
l0ng-ai ee0952e32e test(evals): measure Java code intelligence as the user sees it
`pnpm eval` drives the production workbench in Chromium against a release
`openidea-server` and jdtls, and judges each scenario on what the user sees: time to
feedback, to the first result and to the final one, and the results against
`evals/java/golden/`. Scenarios cover navigation, usages, implementations, gutter
markers, code lenses, diagnostics, completion, quick documentation and rename, in
`fixtures/java-app` and a pinned spring-petclinic, warm and cold. The report
(`target/eval/report.json` / `.md`) names each failure's kind, a repro command and the
server commands in the slowest run's window; `--save-baseline` compares later runs.

- The frontend gets `data-*` hooks the scenarios read: usage / chooser rows carry their
  location, the usages popup its loading state and scope, a hint whether it is progress
  and what it answers.
- `smoke/tauriShim.ts`: the in-page Tauri runtime shared by the jsdom smoke tests and
  the evals; `perf/lib/bundle.ts` builds the perf bundle for both.
- `.github/tool-versions.env` pins the language-server versions for Language smoke and
  the evals.
- `.agents/skills/eval`: the measure → fix → prove loop for agents.
2026-11-11 22:39:59 +08:00
Vadim LazukandInf1mum a0c3249368 fix(linux): keep the first-map hint under Mutter's auto-maximize share (#2520)
Wayland has no primary monitor, so the first-map size hint fell back to scale 1 with no work-area limit and requested the saved physical size as logical. On a HiDPI output the first mapped area exceeded Mutter's 80% share and the window opened maximized before the saved geometry was restored. The hint now converts with the largest output scale and shrinks proportionally to stay within the tightest output's share; X11 keeps its primary-monitor path.

Fixes #2400

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 22:29:05 +08:00
l0ng-ai 757412cb73 fix(security): reach content roots in plugin quick fixes and keep diagrams local
- A plugin quick fix may change or open files in the project's content
  roots (a project window group's members, extra Project Structure
  folders), still never git's own files; before, it was disabled for
  every file outside the primary root.
- `git init` (New Project › Create Git repository) uses the configured
  git executable again.
- Mermaid output is cleaned before it is shown: remote image sources,
  external `url()`, `@import` and address strings in its CSS are
  removed, so `style`/`classDef` values and HTML labels fetch nothing.
- The reply image e2e test loads a web image only after its button is
  clicked; the post-checkout hook test runs in a trusted scope.
- Capability docs list `project.write` as high risk.
2026-11-11 22:18:48 +08:00
Fabio Nunes Cerqueira a497178b3e fix(security): close the findings of GHSA-442p-gxfg-qxv9
- Restricted Mode git: reads run without a signature verifier, hooks
  (including config-defined ones), an external diff, filters or a lazy
  fetch, also inside submodules; every repository and every git starts
  restricted until the project is trusted.
- project.write and plugin quick fixes cannot reach git's own files under
  any spelling or through links; project.write is a high-risk grant.
- Built-in web server: each served root has its own port and token; a
  file outside the project is served with its folder minus dot names, or
  alone when that folder is the home folder or above it; no fixed port.
- Images, references, Markdown links and language-server links reach no
  network share other than the one already open.
- Marketplace: entries whose id or version no manifest may carry are
  dropped; an index read over the network names nothing on this machine;
  network reads are bounded in size and time.
- Agent replies load remote images on a click; the Markdown preview and
  Mermaid configuration no longer fetch from other hosts.
2026-11-11 22:18:48 +08:00
ntbang0901 8b179992e8 feat(profiler): profile allocations and live objects of JVM runs (#2650)
The Profiler gets two more modes for JVM runs through async-profiler: allocations (where the program allocates, in bytes) and live objects (what is still alive at the end). A live-object profile collects garbage first: run executors get a beforeStop hook, bounded to 12 s, that runs a GC before the stop signal; a program that ends by itself says that unreachable objects may show as live.

Part 3 of #2217.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-11 22:13:38 +08:00
wangjunboit 85541b3663 feat(vcs): show a diff in a separate window (#2231)
Changes trees get Show Diff in New Window: the diff opens in its own native window (labelled diff-*, with the same capability as comparison windows), with the full diff toolbar, editable working-copy side and a save prompt for unsaved edits on close. Showing the same diff again reuses its window.

Co-authored-by: wangjunboit <[email protected]>
2026-11-11 22:07:04 +08:00
l0ng-ai 0418a2ce01 fix(java): explain a Maven build plugin that doesn't resolve when m2e only logs it (#2745)
When a build plugin such as maven-resources-plugin can't be resolved
(e.g. the default version is missing from a company mirror), m2e fails
the import's last step and every build but puts no marker on pom.xml,
so Problems stays empty and the "Maven project could not be configured"
notification never appeared. The notification now also fires for these
logged failures, once per plugin whichever way jdtls reports it.
2026-11-11 21:39:00 +08:00
l0ng-ai c8639b38ba fix(libraries): write only SDKs in angle brackets under External Libraries (#2743)
Every External Libraries group was shown as < name >, so a Maven or Gradle project listed hundreds of bracketed dependencies. As in IntelliJ IDEA, only SDKs (JDK, Python, Ruby, Go, Rust) keep the brackets; dependency groups such as Maven: g:a:v are shown as they are.

Refs #1977
2026-11-11 21:34:48 +08:00
Vadim LazukandInf1mum f04658612f fix(plugins): plugins reach a project window group's member files (#2739)
In a project window group plugins only saw the container: projects() and project-of() reported the container root, read/write/find refused member paths, Find in Files dropped member matches and the document indexer skipped member documents. The plugin host now treats the members like the editor and language servers do: projects() lists them, project-of() resolves to the deepest member, the sandbox path checks accept paths inside any member (files outside every root stay refused), and find-files, symbol search and the indexer cover member files. No WIT change.

Fixes #2506

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 21:31:23 +08:00
Qingandqtvbwfn d5cb379cae feat(services): show only services the user added (#2740)
The Services tool window listed services nobody added: an injected Docker connection, Spring Boot by default and every configuration that had ever run. Now only added types, connections and registries are listed, removal takes effect at once, an untouched injected Docker connection is dropped from older docker.json files, and the detail pane has a single scrollbar with console controls only (Run/Debug/Stop stay on the window toolbar).

Co-authored-by: qtvbwfn <[email protected]>
2026-11-11 21:28:41 +08:00
Muuandmuutot e3a818b607 feat(color-scheme): group Python color attributes as PyCharm does (#2735)
The Python page of Settings › Editor › Color Scheme listed its attributes flat. They now follow PyCharm's tree (String › f-string, Docstring, Braces and Operators, Functions, Parameters, Special names…), in its order; only display names change, so .icls files and existing overrides keep working.

Co-authored-by: muutot <[email protected]>
2026-11-11 21:26:31 +08:00
Vadim LazukandInf1mum 0b1872937a fix(lsp): hint when Vue 2 Options API template navigation lacks strictFunctionTypes (#2728)
Vue 2.7's Options API types only infer methods and computed into this under strictFunctionTypes; without it template go-to-definition and hover return nothing. When vue-language-server serves a Vue 2 project whose jsconfig/tsconfig lacks the option (and doesn't extend a base), the IDE now shows a warning with the fix.

Fixes #1163

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 21:25:01 +08:00
Vadim LazukandInf1mum f11cd361a7 fix(color-scheme): import .xml scheme files and tolerate real-world IDEA variants (#2731)
The import chooser only offered .icls, so schemes distributed as .xml (Material Theme UI) could not be picked. It now offers .xml too, and the parser accepts a BOM, an encoding declaration, a DOCTYPE, a scheme wrapped in a settings export and a scheme without a name (named after its file); a rejected file shows the reason.

Fixes #2714

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 21:23:38 +08:00
Vadim LazukandInf1mum 7034fe1640 fix(debugger): offer to reinstall Delve when it is too old for the Go toolchain (#2730)
A Delve older than the Go toolchain refuses the launch with "Version of Delve is too old for Go version …". The debugger now recognises it and offers Reinstall and Debug, which runs the managed go install …/dlv@latest and starts the session again.

Fixes #2721

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 21:23:03 +08:00
Vadim LazukandInf1mum fc264d3f5c fix(ui): stop double-clicking tree rows from selecting label text (#2726)
The second press of a double-click started the browser's word selection, which in the virtualized tree spilled onto the rows an expanding folder had just inserted. Tree cancels the default of repeated presses; the first press still selects and dblclick still activates.

Fixes #2713

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 21:22:02 +08:00
Vadim LazukandInf1mum 8e90341b3a feat(build-tools): a Skip Tests toggle for the Gradle tool window (#2734)
The Gradle tool window gets an optional Skip Tests toggle (off by default, per project): task runs and saved configurations started from the tree get -x test, unless the command line already excludes test or the build has no test task (where -x test would fail the run). A run that skips tests is not marked as a test run.

Fixes #2688

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 21:20:58 +08:00
l0ng-ai 3644956755 feat(terminal): activate the project's conda environment in new terminals (#2742)
Activate virtualenv only handled venvs, so a conda project interpreter left new terminals in (base). New terminals now run conda activate <prefix> through the installation's shell hook (cmd, PowerShell, bash, zsh, POSIX sh, fish), finding the installation from the env itself, the parent of envs/, CONDA_EXE or PATH.

Fixes #2697
2026-11-11 21:18:24 +08:00
l0ng-ai 90e7e3162e feat(mcp): run the IDE's tools without confirmation in brave mode (#2741)
Settings › Agents › IDE MCP Server gets Run tools without confirmation (brave mode): builds, run configurations and the other tools that change a project start without asking, so an unattended agent isn't left waiting on a confirmation. Read from the IDE's own settings only; the project layer already refuses mcpServer.*.

Fixes #2660
2026-11-11 21:17:24 +08:00
Yann 833ae4c3c4 feat(editor): add and remove UTF-8 BOM (#2717)
The encoding status menu of a UTF-8 file gets Add UTF-8 BOM / Remove UTF-8 BOM, and the status bar shows "UTF-8 BOM" for a file that has one. The change marks the document modified and is written on save (at once with auto-save).

Co-authored-by: YannABC <[email protected]>
2026-11-11 21:00:48 +08:00
Elmir Abkerimov 584baa3f71 feat(editor): code vision position per hint and a "no usages" switch (#2716)
Settings › Editor › Inlay Hints › Code Vision gets a Position for Usages and for Code author — Top (a row above the declaration, the default) or Right (after the end of the declaration's first line) — placed per hint, and Show "no usages" (on by default) to leave the usages hint off declarations nothing uses. The options sit under the hint they belong to.

Co-authored-by: w4terlaw <[email protected]>
2026-11-11 20:58:46 +08:00
Elmir Abkerimov 98010e6513 fix(editor): start every selected row at the text's left edge (#2724)
CodeMirror starts selection rows that run through a line start at the left padding rounded down to whole pixels, and a row that starts at a character at the real padding; with the 2.5px inset the two were half a pixel apart, a visible step on Retina. Rows that start at the real edge now move to the rounded one, and the sliver drawn on a line the selection ends at the start of is dropped after the rows are grown.

Co-authored-by: w4terlaw <[email protected]>
2026-11-11 20:56:38 +08:00
Muuandmuutot a3640f655f fix(perf): name language server memory rows by their workspace root (#2702)
The memory indicator labelled every language server row with the project folder, so with additional content roots the servers of each root showed as identical rows. Each row now names the server's own workspace root.

Co-authored-by: muutot <[email protected]>
2026-11-11 20:56:07 +08:00
Shohruh Ismoilovandshohruh-genius 8aaa7daf83 fix(project): keep workspace project icons stable on removal (#2699)
Removing a member from a workspace reset every detected project type, so the remaining members briefly showed folder icons and fell back to name order. The cached types of members still in the same workspace are kept while the rescan runs.

Co-authored-by: shohruh-genius <[email protected]>
2026-11-11 20:54:16 +08:00
Bin 6a9a970df4 fix(editor): use JSX comments for expression children (#2703)
Ctrl+/ on a JSX expression child such as {label} read the JavaScript comment tokens at the opening brace; it now wraps the line in the surrounding JSX comment pair, {/* … */}, and takes it back out.

Co-authored-by: ffgenius <[email protected]>
2026-11-11 20:53:53 +08:00
l0ng-ai a309b7e6fa fix(java): import Gradle builds on the JVM the IDE's Gradle runs use (#2736)
With Gradle JVM at Project JDK the IDE's Gradle runs use the project JDK (or an installed JDK the Gradle version supports), but the jdtls import ran Gradle on jdtls's own JDK 21, so Gradle < 8.5 failed to import and no dependency resolved. The import now takes JAVA_HOME and GRADLE_USER_HOME from gradle_build_env for Gradle builds.

Refs #2681
2026-11-11 20:52:49 +08:00
Zhu Minhaoandzmh69695328 6971936f78 feat(java): run jdtls's Gradle import with the build settings (#2701)
jdtls's Gradle import ran on its own defaults (project wrapper only, ~/.gradle, the server's JDK), so a project without gradlew was never imported and one whose caches live in a configured Gradle user home resolved no dependencies. Settings › Build Tools › Gradle now reach java.import.gradle.*: home with the wrapper disabled for a local distribution, plus user.home and java.home, resolved like the IDE's own Gradle runs and merged into the existing settings.

Co-authored-by: zmh69695328 <[email protected]>
2026-11-11 20:49:28 +08:00
l0ng-ai 9d781a2f83 fix(agents): start and delete tasks without waiting on slow git work (#2732)
Starting a task in the local checkout recorded the already-changed files through the full change listing (diff, numstat, a read of every untracked file, quadratic lookups); the snapshot now uses the status alone and the listing uses hashed lookups. Deleting a task now drops it from the list before its worktree folder is deleted.

Refs #2667, #2666
2026-11-11 20:46:13 +08:00
l0ng-ai 3260e5e224 fix(vcs): open the source file on F4 in the changes trees (#2729)
F4 in the Commit tool window opened the selected file, then the tree handled the same key as Enter and opened the diff preview, which took the focus. Tree takes an onJumpToSource handler for F4 / Cmd+Down; the commit and Git Log changed-files trees use it to open the file itself.

Fixes #2696
2026-11-11 20:38:27 +08:00
l0ng-ai ed575a00d9 docs: show sponsor QR codes at the same size with labels (#2725) 2026-11-11 20:33:59 +08:00
l0ng-ai 2b41e61ce9 docs: add Alipay and WeChat Pay sponsor QR codes (#2723) 2026-11-11 20:29:16 +08:00
l0ng-ai 908740b3f2 feat(plugins): a marketplace that helps pick plugins (#2718)
* feat(plugins): a marketplace that helps pick plugins

Rows show each plugin's description, publisher, downloads and an inline
Install / Update button; plugins without an icon get a tinted initial.
While browsing, the list opens with suggestions for the open project and
the maintainers' picks, then all community plugins by downloads; the
official plugins come last as examples. The first plugin is selected so
the details are never empty, and they show release notes, the homepage
and why a plugin is suggested.

The index gains optional homepage, notes, downloads, featured and suggest
fields; openidea-plugin index fills homepage and notes from the manifest.

* fix(plugins): plugin icons fill their tile like the initials

* feat(plugins): drop the Preview badge from the plugins page

* feat(plugins): open the plugins page on Available
2026-11-11 20:13:28 +08:00
l0ng-ai e702e4da7b chore: v0.1.42 2026-11-11 19:31:09 +08:00
arookieofc 91023de207 chore(ci): route heap dump dialogs through the platform wrapper; clippy 1.99 and Windows test fixes (#2711)
Heap dump open/save use the platform dialog wrapper (GTK fallback on Linux). ide-heap allows clippy 1.99's large_enum_variant on the hot bit table. dialog.test compares paths with forward slashes on Windows.
2026-11-11 19:30:15 +08:00
l0ng-ai 02ee2880fe fix: translate New Project popup and error text; green CI on main (#2715)
Translate the project popup actions and the non-empty target error; translate the JDK field label. Fix clippy in ide-jdwp, formatting, RUSTSEC-2026-0486 (rustix 1.1.6) and agent E2E tests that expected the removed floating Quote button.
2026-11-11 19:25:09 +08:00
l0ng-ai 5a316e81d6 fix: Compile File sees its module's classes; list external agent sessions per project off the main thread (#2707)
Compile File passed javac only the dependencies and upstream modules, so any file using
another class of its own module failed with "cannot find symbol". The module's own
output folders now come first on the classpath, and its source folders (generated ones
included; main ones for a test) are the -sourcepath with -implicit:none, so classes that
were never built resolve from source while only the file itself is written.

Listing sessions started outside OpenIDEA parsed every Claude Code transcript of every
project on a synchronous command, which runs on the main thread and froze the window
(1.2 GB of transcripts here). It now reads only the transcript folders of the open
project, keeps sessions whose working directory is in the project, reads Codex rollouts
line by line, and runs on the blocking pool; importing does the same.
2026-11-11 18:22:28 +08:00
l0ng-ai 7c2ca4d061 fix: build Maven again when outputs were deleted; main menu in a separate toolbar (#2706)
Build Project / Build Module skipped Maven when no input changed, even after the
modules' target/classes were deleted outside the IDE (mvn clean in a terminal).
The up-to-date record now also lists the output folders that existed after the
build, and a build is skipped only while they all still exist.

The "Always show the main menu" toggle duplicated Main menu › Merged with main
toolbar, showed on macOS where it does nothing, and kept the titles in a title
bar too narrow for them. It is replaced by a third Main menu choice, Separate
toolbar: the menu titles get their own row below the title bar and never
collapse into the hamburger button.
2026-11-11 18:09:01 +08:00
ntbang0901 8d09479a65 feat(debugger): heap snapshots of a debugged JVM (#2649)
Capture a heap dump of the debugged JVM over the attach listener (GC.heap_dump, after checking free disk space and the JDK's rules for suspended VMs) or open an .hprof file, and browse it in an editor tab: classes with shallow and retained sizes, biggest objects, dominators, the path from an object to a GC root, and a comparison of two snapshots. A snapshot captured at a stop adds a Retained Size column to that stop's Memory tab.

New crate ide-heap: HotSpot hprof parser (bounds-checked; unfinished or crafted dumps are refused), object graph, dominator tree (semi-NCA), retained sizes and GC-root paths. Snapshots live in private directories removed with their window; an opened dump is analyzed from a private copy so changes to the user's file can't affect the mapped data.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-11 17:43:07 +08:00
Bin 8fafde10b4 fix(editor): keep tab actions at the pane edge (#2682)
The trailing slot of the editor tab strip had flex: none, so the split / more buttons followed the last tab. An automatic left margin keeps them at the right edge of the pane without changing how wrapped tabs share space.

Co-authored-by: ffgenius <[email protected]>
2026-11-11 17:35:16 +08:00
Muhammad N. Fadhil b1cd917a96 fix(gradle): process resources before launches from the language server's classpath (#2678)
The Java language server copies resources unchanged into Buildship's bin/<source set>, so launches with "Build and run using: OpenIDEA" saw unfiltered resources (e.g. `version: @version@`). The Gradle sync now records which source sets Gradle changes resources for (filters, expand, renames, includes/excludes, extra sources); before such a launch the IDE runs only those processResources tasks and copies their output over the plain copies. Builds that copy resources as they are run no extra Gradle step; if Gradle fails, the launch continues with a hint.

Closes #2675

Co-authored-by: MuhammadNFadhil <[email protected]>
2026-11-11 17:33:43 +08:00
l0ng-ai 605d5f4985 feat(new-project): project generators for mainstream languages and frameworks (#2689)
File › New Project offers 38 generators grouped by language (Java, Spring Boot, Quarkus, Micronaut, Kotlin, Ktor, Scala, Python/Django/FastAPI/Flask, Node/Vite/Next.js/Nuxt/Angular/SvelteKit/Astro/NestJS/Express, Go, Rust, C/C++, Zig, .NET, PHP/Laravel/Symfony, Ruby/Rails, Dart/Flutter, Swift, Elixir/Phoenix, Haskell, static website, empty). Each is a declaration in generators/<id>.toml run by the new ide-scaffold crate: built-in templates, the framework's official CLI run non-interactively, or a project service zip (Spring Initializr, code.quarkus.io, Micronaut Launch) whose metadata becomes form fields. Projects are built in a staging directory and moved into place only on success. The dialog renders each form from its schema, flags missing toolchains with install hints and streams progress; created projects open trusted. New Project is also on the welcome screen and the title bar project menu.

Closes #709
2026-11-11 17:22:57 +08:00
l0ng-ai 7d52f058a7 chore: record ide-search's encoding_rs dev-dependency in Cargo.lock (#2695) 2026-11-11 17:17:30 +08:00
l0ng-ai c44558b4aa fix(lsp): report a server missing while a variable it requires is unset (#2671)
With pwsh installed but PSES_BUNDLE_PATH unset, PowerShell Editor Services was launched with a broken path and restarted in a loop with a crash notification each time. Language servers can now declare requires-env; while one of those variables is unset or empty (in the server's env and the IDE's environment) the server counts as missing and shows its install hint instead of starting. powershell.toml declares PSES_BUNDLE_PATH; the field is documented.
2026-11-11 17:16:11 +08:00
l0ng-ai 8455761181 fix(lsp): show only the crashed instance's stderr, without colour codes (#2668)
A language server crash notification quoted stderr verbatim: coloured output showed raw escape sequences, and since the buffer was never cleared between restarts each report repeated earlier instances' output. Stderr lines are now stripped of ANSI escapes when recorded and the buffer is cleared when a new instance starts; strip_ansi moves to ide-core::output_encoding so ide-build and ide-lsp share it.
2026-11-11 17:16:07 +08:00
l0ng-ai dabf9a954e fix(search): find text in GBK and UTF-16 files (#2665)
Closed files that are not UTF-8 were searched as raw bytes: Chinese in GBK files never matched and ASCII matches showed mojibake, UTF-16 files were skipped as binary, and comment/string-limited searches skipped non-UTF-8 files. The engine now checks UTF-8 while streaming and searches non-UTF-8 files again in text decoded like the editor does (BOM / UTF-16 as detected, else the project's fallback encoding). Find in Files, Search Everywhere › Text, MCP and plugin search share this path.
2026-11-11 17:14:42 +08:00
ntbang0901 a3d1e57f3e feat(debugger): memory view for a paused JVM (#2648)
Adds a Memory tab to the Debug tool window for JVM sessions: loaded classes with instance counts and shallow sizes (with deltas between refreshes), read over the HotSpot attach listener (`GC.class_histogram`), falling back to counting reachable objects through the debugger on JDKs that cannot answer while suspended. While paused, a class's instances, an instance's fields and its referrers are listed through a JDWP link (the Kotlin adapter's path, and a jdtls bundle `java/jvm-inspect` for java-debug). Debug launches started by the IDE get `-XX:+StartAttachListener` unless the user's VM options decide otherwise (not on Windows).

New crates: ide-jdwp, ide-jvm-attach, ide-jvm-inspect. The bundled jar rebuilds byte-for-byte from build.py against the pinned JDT LS plugins.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-11 17:10:21 +08:00
l0ng-ai 0ff3f188de feat(agents): add built-in Qoder (#2687)
Qoder's global CLI starts with `qodercli --acp` like Qoder CN; both editions share the icon.
2026-11-11 16:49:13 +08:00
wangfeiandhapput 1654d22756 feat(agents): add built-in Qoder CN support (#2647)
Qoder CN appears in the built-in agent list and starts with `qoderclicn --acp` through the existing ACP driver, with sign-in guidance and a row in the agent guides. Its icon is a 16px glyph with a dark-theme variant.

Co-authored-by: happut <[email protected]>
2026-11-11 16:46:45 +08:00
Emanyzwww 14d9cdff11 feat(app): show a What's New page after an update (#2664)
After an update the restarted app opens what the new version brought as an editor tab, and Help > What's New reopens it at any time. Its checkbox turns the automatic opening off for later updates; the same switch sits on System Settings > Updates next to the update checks. The update balloon keeps a one-paragraph summary.

The release workflow fills the updater manifest's notes from the release body without its Download section, the install records them, and a version installed outside the app reads them from the manifest. The first start of a different version is recognised from the last run version kept in settings.
2026-11-11 16:42:32 +08:00
Muuandmuutot f7a507a5db feat(vcs): allow rewriting history of pushed commits (#2644)
Edit Commit Message, Fixup, Squash Into, Drop Commit, Squash Commits and Interactively Rebase from Here are no longer disabled for pushed commits. A rewrite that reaches commits already on the upstream asks for confirmation first (pushing afterwards needs a force push); on a protected branch such a rewrite is refused, while its local commits can still be rewritten. Undo Commit follows the same rule.

Co-authored-by: muutot <[email protected]>
2026-11-11 16:41:45 +08:00
Charlie.Weiandcharli117 c67e07a4f9 feat(plugins): cancel plugin commands per request (#2620)
Plugin commands could only be cancelled as a whole plugin. Local IPC and remote RPC now take a request id: a cancelled request is dropped before the plugin is activated, a running command gets a short grace period to exit and is interrupted after it, and other calls to the same plugin keep running. The frontend runCommand takes an AbortSignal that settles the caller at once and forwards the cancellation.

Co-authored-by: charli117 <[email protected]>
2026-11-11 16:37:16 +08:00
misswellandmisswell 4e08893eae fix(run): build Maven modules through Maven on Rebuild (#2617)
A Rebuild launch of a Maven module used jdtls' java/buildWorkspace(true), which only recompiles sources: stale classes stayed in target/classes, and JDT rejected sources javac accepts (a package that doesn't match its directory), so Debug stopped with "Build has compile errors" although mvn clean compile succeeded. Rebuild now always delegates Maven modules to their Maven build (clean compile / clean test-compile), and a successful clean build clears the stale JDT diagnostics. Automatic and Make keep the JDT path.

Co-authored-by: misswell <[email protected]>
2026-11-11 16:33:19 +08:00
l0ng-ai 779070c3d7 feat(vcs): open diff follows the change tree selection; file buttons in the diff toolbar (#2628)
With a diff tab open from the Commit or Local Changes tree, selecting another file in the tree shows that file in the same tab and keeps focus in the tree; with no diff open, selecting opens nothing. The diff toolbar gets Compare Previous File / Compare Next File buttons when the diff has sibling files.

Closes #2492
2026-11-11 16:30:53 +08:00
l0ng-ai 4e86161c3c fix(run): debug Maven tests from the gutter and attach when Surefire starts the fork (#2684)
The run gutter and context menu offered Debug only for configurations with a debug plan or adapter, so Maven / Gradle tests (debugged through a JDWP agent) had none. Debugging a Maven test attached to the pinned Surefire port right away and java-debug's 30-second retry could run out while Maven recompiled; it now attaches when Maven prints the T E S T S banner just before starting the fork. A failed attach to a JVM started for the debugger now stops that run (as for Python), so its suspended JVM doesn't keep it alive.

Fixes #2656
2026-11-11 16:28:14 +08:00
arookieofc d7393b53f3 feat(menu): add always-show main menu option (#1521) (#2661)
* feat(menu): add always-show main menu option (#1521)

* fix(i18n): add zh catalog entries for always-show main menu strings
2026-11-11 16:22:52 +08:00
Deng Liuanddengliu 276a91973c fix(terminal): use editor search colors, keep the active match above the selection, and seed ⌘F (#2606)
The terminal Find bar painted matches with the old text-search tokens; it now uses --bg-search-match / --bg-search-match-current (blended over the panel background) like the editor. The active match's decoration goes to xterm's top layer so it stays visible above the selection the search places, and ⌘F with the Find bar already open seeds it from a fresh user selection while keeping the query when the selection is the current match.

Co-authored-by: dengliu <[email protected]>
2026-11-11 16:17:36 +08:00
l0ng-ai fc75b15519 perf(editor): find a type's overridden methods with one subtype walk (#2680)
The gutter's "is implemented / overridden" markers sent one implementation request per visible method, and jdtls builds the type hierarchy again for each, so a large interface in a big project kept it busy long enough to hold up navigation. When outline names carry the signature (jdtls), a type's methods are now resolved from one walk of its subtypes and their outlines, as the super-method markers are; used only while it takes fewer requests than it replaces, and a method whose name a subtype reuses with another signature (generic overloads) still gets its own request.

Refs #2314
2026-11-11 16:04:10 +08:00
arookieofc 8798c2a542 fix(remote): sync outside-root classpath entries for SSH Java launches (#2641) (#2669) 2026-11-11 15:55:30 +08:00
Deng Liuanddengliu 8ea2787490 fix(editor): keep selection within its own row when font glyphs exceed line height (#2674)
With a font whose glyph box is taller than the line height (e.g. font size 18–20 with the default 22px line height), selecting text on one line also painted the rows above and below: growToLineBoxes probed the rows at the glyph box's top and bottom edges, which then fell in the neighbouring rows, and .cm-openidea-selected painted an unclipped inline background at glyph height. Outer rows are now resolved at each marker's vertical midpoint and row seams align to the line boxes (block widgets and zero-width trailing markers handled), and the inline selection background is capped at 1lh in the editor, notebook cells and diff/merge viewers, with a plain colour fallback where lh is unsupported.

Co-authored-by: dengliu <[email protected]>
2026-11-11 15:54:26 +08:00
l0ng-ai dc4e8403de fix(java): keep a language server JDK that ran usable while it is unchanged (#2679)
A JDK whose java -version succeeded was probed again every 10 seconds; on a busy machine one probe could time out and the working JDK was reported as "cannot run on this machine". A successful probe now stays valid while the executable's size, modification time and permissions are unchanged; failed probes are still retried after 10 seconds, and a java that hangs on its first probe is still rejected.

Fixes #2470
2026-11-11 15:50:10 +08:00
l0ng-ai 4a0efbae25 fix(agents): keep a run or turn the reader opened open across remounts
A run's open state lived in its row, so it closed when the row remounted: when a
finished turn folded its work away, or when a run grew from one call to two.
Opened runs and turns are now remembered by their first entry for the
conversation's lifetime.
2026-11-11 15:36:22 +08:00
l0ng-ai cb3c0bafd7 fix(boot): count the startup timeout only while the window shows
A window behind others or on another Space gets no animation frames, and the
splash waits for two of them before startup finishes. The 30-second timeout kept
running meanwhile and reported a healthy but slow start, such as a cold dev
build, as "OpenIDEA could not start". It now pauses while the document is hidden
and resumes with the time left.
2026-11-11 15:34:30 +08:00
l0ng-ai 71072dbe4b fix(agents): pace streaming reasoning, cut only between graphemes, cap at 60 Hz
- Reasoning in view streams at the reply's pace instead of in bursts.
- The painted end of a streaming text falls between graphemes (Intl.Segmenter):
  an emoji with joiners or a skin tone, a flag or a letter with combining marks
  no longer flashes in parts. A chunk ending mid-cluster holds that fragment back
  until the rest arrives.
- The reveal commits at most at 60 Hz, so a 120 Hz display doesn't re-render the
  Markdown twice as often.
2026-11-11 15:33:53 +08:00
l0ng-ai 18b286e7e0 fix(agents): one spinner per conversation, counts on one line
Work in progress shimmers on its label (a run's counts and current call, a
running tool, "Thinking") instead of showing a spinner beside it; the working
row at the conversation's end is the only spinner. A run's counts keep their
line, and what runs now ellipsizes in the space left.
2026-11-11 15:32:40 +08:00
l0ng-ai 9474af4de1 fix(agents): steadier tool-call runs, working row in the stream, selection and scroll fixes
- Reasoning and tool calls fold into one run including running calls, so a run
  keeps its place while a call finishes. Its summary names what runs now; it no
  longer leads with "Reasoning". A folded thought shows its first line. An open
  run is bounded and follows its latest call.
- The working / starting / retrying row with its timer ends the conversation
  instead of sitting above the composer, for sessions and subagents alike.
- Selecting text no longer pops up a Quote button; Quote stays in the context menu.
- A selection clamped to the conversation ends in its last selectable text, not on
  its edge: WebKit painted a selection ending there into the composer, as a
  triple-click on the last paragraph showed.
- Sending a message while scrolled up returns to the end, where it shows.
2026-11-11 15:20:09 +08:00
arookieofc 8044d99ea0 feat(build): compile single Java files and skip unchanged Maven builds (#2476) (#2646)
* feat(build): compile a single Java file with javac (#2476)

* feat(build): add Compile File action for single Java sources (#2476)

* feat(build): skip unchanged Maven menu builds via fingerprint records (#2476)

* fix(build): drop shadowed fixture model in unix-only classpath test (#2476)

* fix(build): pass cancel into shared executeBuild helper (#2476)

* chore: retrigger CI after mass-cancel wave (#2646)

* chore: probe CI queue recovery (#2646)
2026-11-11 15:03:42 +08:00
Charlie.Weiandcharli117 0abaf6c777 fix(plugins): keep plugin settings that match their defaults (#2619)
Choosing a plugin setting's default value deleted the override, so in a project the global value showed through instead of the user's choice. The setting control now always writes the chosen value.

Co-authored-by: charli117 <[email protected]>
2026-11-11 14:45:40 +08:00
LU XIANGRUIandwildcatDownstairs 0d9b08ac8c fix(color-scheme): import WebStorm JavaScript and TypeScript colours from .icls (#2634)
WebStorm stores JavaScript and TypeScript colours under JS.* / TS.* keys, which the .icls importer did not map, and DEFAULT_SEMICOLON / _COMMA / _DOT all mapped to Braces and Operators, so a scheme recolouring semicolons painted every brace and paren. JS.* and TS.* now map onto the Language Defaults attributes with TS > JS > PY > defaults precedence, and the separator keys are no longer mapped.

Co-authored-by: wildcatDownstairs <[email protected]>
2026-11-11 14:45:32 +08:00
Maskerandzed1995 e54580265d fix(agents): show Codex ACP collaboration mode in the composer (#2631)
codex-acp reports its Default/Plan choice as a collaboration_mode config option, which the composer dropped because it only rendered model, mode, permissions, approval and thought_level. collaboration_mode is now one of the composer's ordered settings, so a Codex session can switch to Plan.

Co-authored-by: zed1995 <[email protected]>
2026-11-11 14:45:24 +08:00
LU XIANGRUIandwildcatDownstairs 4fa700b337 fix(editor): collapse the bookmark gutter fully in files without bookmarks (#2632)
The bookmark gutter's 2px side padding applied to the empty spacer element too, so the column kept 4px in files without bookmarks. The padding now applies only to elements that hold a marker, so the column is 0px until the file has a bookmark.

Co-authored-by: wildcatDownstairs <[email protected]>
2026-11-11 14:45:16 +08:00
DrPepperandmakise2060 173d5edcdc fix(run): render the Run console with the terminal font (#2615)
The Run console hard-coded the --font-mono stack and --fs-code, so Settings › Tools › Terminal › Font (family, fallback, size, or Use editor font) never reached process output. resolveMonoFont() now resolves the terminal font choice for both xterm and the DOM console, and the console's virtual row height follows the font size (round(size × 1.5), 20px at the default 13px) so rows don't overlap.

Co-authored-by: makise2060 <[email protected]>
2026-11-11 14:43:59 +08:00
Gabriel Gondim 4d60921f24 feat(database): list all databases on a SQL Server host (#2629)
SQL Server data sources without single-database mode now show a database level in the tree, like PostgreSQL: every online database the login can access, each expanding to its schemas, with the Schemas tab database picker and the Single database mode option. The backend already listed them; listsDatabases() only enabled the level for Postgres. A failed connect shows an error under the node instead of leaving it on Loading, and saving Properties reloads what the tree had loaded for that source so expanded nodes don't stay on Loading.

Closes #2600

Co-authored-by: GGVidal <[email protected]>
2026-11-11 14:40:52 +08:00
Paulem 7784f3cdef fix(dialog): fall back to a GTK file chooser on Linux when no portal or zenity is available (#2394)
On Linux sessions with neither an xdg-desktop-portal nor zenity (antiX), rfd's xdg-portal backend returned nothing and every Open / Save picker silently did nothing. All pickers now go through src/platform/dialog.ts; on Linux the backend checks once for a reachable portal or zenity and otherwise shows a GTK FileChooserDialog. Other platforms and sessions keep the desktop's dialog. dialog.test.ts keeps every picker (including Move Worktree, keymap import, Mermaid export and library sources) on the wrapper.

Fixes #2386

Co-authored-by: Paulem79 <[email protected]>
2026-11-11 14:25:03 +08:00
misswellandmisswell b010397711 fix(lsp): mark progress tasks that stop reporting (#757)
A $/progress task that reports nothing for 3 minutes (jdtls's Maven import waiting on a repository that never answers) kept the server Busy forever, which also held wait_idle and ReferencesIncomplete::Indexing. Such a task now stays open with its last message and fraction, marked "no progress for 180 s", and no longer counts as busy; a late report clears the mark and the same task finishes normally.

Co-authored-by: misswell <[email protected]>
2026-11-11 14:18:16 +08:00
arookieofc 108da537f4 feat(agents): list and import sessions started outside OpenIDEA (#2640)
* feat(agents): list and import sessions started outside OpenIDEA

Sessions started in a terminal, another editor or the CLIs never showed
up in OpenIDEA, so users had to re-explain their work (#312). The agent
history popup now lists them under Sessions from other apps: Claude Code
transcripts and Codex rollouts on this machine, with title, working
directory and age. Importing one creates a task whose first turn loads
and replays the transcript through the existing session/load path.

* i18n: add zh translation for 'Sessions from other apps'
2026-11-11 12:42:01 +08:00
Deng Liuanddengliu c658b6e59f feat(terminal): highlight visible occurrences of selected text (#2638)
Selecting text in the terminal highlights its other occurrences in the visible rows, like the editor's identifier-under-caret highlight. Only the viewport (plus a few rows of margin) is scanned, matches are cached per logical line and drawn in one overlay layer, recomputation is coalesced to one pass per animation frame, and highlights are cleared during heavy output and redrawn once it settles. Wrapped lines and wide characters map to the right cells; multi-line, very short or whitespace-only selections, the alternate buffer and Find-bar selections are skipped. Settings › Tools › Terminal › Highlight occurrences of selected text turns it off.

Co-authored-by: dengliu <[email protected]>
2026-11-11 12:41:33 +08:00
Deng Liuanddengliu 990a1c7a87 feat(agent): retry session setup after ACP authenticate on auth error (#2637)
When session/load or session/new fails with an authentication error and the agent advertises a non-terminal `agent` authentication method in `initialize`, the IDE now calls ACP `authenticate` with that method once per session setup and retries the failed call before falling back to the sign-in error. Terminal-login methods (`_meta.terminal-auth`) keep using the existing sign-in guidance.

Co-authored-by: dengliu <[email protected]>
2026-11-11 12:37:34 +08:00
l0ng-ai a58a7a0b8c fix(project): keep the project root icon outside workspaces (#2657)
Project type detection only runs for workspace members. A single project
root no longer takes the icon of its first matching build marker, which
labelled a Cargo + Vite repository as a Vite project.
2026-11-11 12:36:12 +08:00
arookieofc c4b47da9d1 fix(maven): link Windows path variants and warn on outside-root picks (#2639)
* fix(maven): link Windows path variants and warn on outside-root picks

The Add Maven Projects dialog hands back whatever form Windows uses:
drive-letter case may differ from the stored root and long paths come
back \?\ -prefixed, so in-root picks were silently dropped and the
button looked dead (#2626). Normalize both sides before comparing
(case-insensitively on Windows) and notify which picks were skipped
because Maven only links POMs inside the open project.

* i18n: tag Maven outside-root notice with tx and add zh entries
2026-11-11 12:33:15 +08:00
l0ng-ai 9d64134b68 fix(agents): keep list selection unbroken in agent replies (#2655)
Draw list markers as non-selectable pseudo-elements instead of native
markers. WebKit paints a selection box around each native marker, which
left a broken stripe down the list gutter when a reply was selected.
2026-11-11 12:32:27 +08:00
l0ng-ai ca945b13d2 feat(plugins): show available updates on the Installed tab (#2654)
Installed plugins with a newer, loadable marketplace version get an Update
badge in the list and an "Update to x.y.z" button in their details, which
installs it from the marketplace.
2026-11-11 12:32:22 +08:00
l0ng-ai 97c58cc808 fix(agents): keep a conversation drag-selection out of the composer (#2653)
Dragging a selection from the last message down past the conversation
highlighted the composer's text field. During such a drag everything outside
the conversation is made unselectable, but WebKit keeps editable text
selectable regardless of user-select, so the selection reached the textarea.
Text fields outside the selection scope now also drop out of hit testing while
the drag lasts.
2026-11-11 12:32:17 +08:00
l0ng-ai e158658afb fix(agents): drop the status bar's Changes toggle (#2652)
The session status bar ended with a Changes toggle that did the same as the
"N changed files" chip right below it. The chip opens and closes the Changes
pane and also shows the line counts, the pane has its own hide button, and the
status-bar toggle was disabled whenever there were no changes, so the status
bar now keeps only the Terminal button. The unused "Show changes" string goes
from the locale catalogs.
2026-11-11 12:32:12 +08:00
l0ng-ai db4bb52694 fix(agents): drop the running tool's name from the working row (#2645)
The tool in progress already shows as the last card of the conversation, and
next to it the row's elapsed time read as that tool's duration rather than the
turn's. The row now shows Working… with the turn's elapsed time, and still
shows retries and the starting state.
2026-11-11 12:31:37 +08:00
l0ng-ai 65bb39e08c test(plugins): give the official examples test a longer command budget (#2643)
The plugin_data_tools harness caps plugin commands at 250 ms to test interruption; the official examples' first command compiles and scans the project and ran over that on Windows runners. That test now uses a 10 s budget.
2026-11-11 12:08:25 +08:00
arookieofc 14e4881418 fix(debug): load run configuration .env files for debug launches (#2633)
Debug launches only copied the explicit env map into the DAP launch
arguments and dropped the configuration's .env files, which Run loads
via ide_run::envfile. Add a run_env_files command that loads the files
with the same loader over the system environment, and merge the result
under the explicit variables when building debug launches (generic,
plugin-hosted and java-debug paths); explicit variables still win and a
missing file fails the launch with the same error Run reports.

Fixes #753 (comment): .env applied in Run but ignored in Debug.
2026-11-11 10:41:16 +08:00
l0ng-ai 55bc093fe5 chore: v0.1.41 2026-11-11 09:51:47 +08:00
l0ng-ai 37ba544733 fix(run): run JUnit configurations with the build's JDK (#2627)
JUnit configurations in Maven and Gradle projects ran their build command without the project JDK, so with no JAVA_HOME the build failed with "Unable to locate a Java Runtime". apply_to_run_configs now gives junit configurations the Maven runner or Gradle build environment (JDK, MAVEN_OPTS, settings) without changing the command line, and ide-host applies it to junit configurations.
2026-11-11 09:42:19 +08:00
l0ng-ai 0fde75411a fix(search): show the results list when the preview is off (#2625)
Without the preview, `.content` has no definite height and the results pane's `flex: 1` (0% basis) collapsed it to its padding in the system WebKit of macOS 15, so Search Everywhere and Go to File showed no rows. The pane now uses an `auto` basis.
2026-11-11 09:39:59 +08:00
l0ng-ai c96206ce0c fix(agents): stop listing the ACP adapters of Claude Code and Codex; square quote button
- Settings › Coding Agents and the provider picker no longer list `claude-acp` and
  `codex-acp`: the native Claude Code and Codex drivers cover them. They still launch,
  and are listed again when an agent file customizes them or a task already uses them.
- The Quote button over a selection in a conversation sits in a small rounded-square
  toolbar instead of a round bubble.
2026-11-11 09:08:05 +08:00
Sergei Melnikov 2e4e5bb088 feat(vcs): show branch and incoming/outgoing commits beside repository roots (#2605)
Settings > Version Control > Git > "Show branch and incoming/outgoing commits
in the Project view" (off by default) shows each repository root's branch and
its outgoing/incoming commit counts beside the root in the Project view. Git is
queried only after a change inside .git or when the set of roots changes, one
round at a time.

Co-authored-by: pdasilem <[email protected]>
2026-11-11 08:46:16 +08:00
l0ng-ai fe69210ef0 fix(agents): clarify the MCP settings pages, their server form and import
- Rename the two MCP pages under Agents to IDE MCP Server (OpenIDEA's own
  server for outside clients) and External MCP Servers (servers coding
  agents use), and point the docs at Settings › Agents.
- Lay the Add/Configure MCP Server dialog out as a label column with
  padded, framed fields; the page's styles now use existing tokens, so
  text areas get a frame and text the right size and colour.
- Import is now Import from Project and reports how many servers it
  added from .mcp.json and .vscode/mcp.json, or that it found none.
2026-11-11 08:41:03 +08:00
l0ng-ai 0cfe0c82fe fix(settings): move plugin formatter choices into Code Style › Formatter (#2616)
Languages & Frameworks had a standalone Plugin Formatters page that, with no
plugin providing a formatter, only showed a Format on save toggle duplicating
Tools > Actions on Save and Editor > General. The per-language plugin
formatter choice now lives in Editor > Code Style > Formatter, appears only
when an enabled plugin provides a formatter, and follows the Code Style
scheme (Copy to Project / Copy to IDE include it). Docs and the plugin README
point to the new location.
2026-11-11 08:36:34 +08:00
l0ng-ai f21d895924 fix(window): zoom the window in one step instead of animating it (#2614)
Double-clicking the title bar stepped the window frame over 350 ms, waiting
for the webview to present each step; every step re-laid out the whole IDE,
so the window grew in visible jumps. It now moves to the work area, or back
to its remembered frame, in one non-animated resize. Restoring across
restarts and the AppKit unmaximize fallback are unchanged.
2026-11-11 08:29:13 +08:00
l0ng-ai e593d07405 fix(vcs): list files in the Compare tool window and open each diff in the editor (#2611)
The Compare tool window rendered the side-by-side diff inline next to the file
tree; in a sidebar the file names were cut off and the diff was a few pixels
tall. It now lists only the changed files, and double-click or Show Diff
opens the diff in an editor tab. Swap Sides and Jump to Source are
unchanged; the Changes Between dialog keeps its inline diff.
2026-11-11 08:26:29 +08:00
Sergei Melnikov daa7f96f7e feat(terminal): change the font size with Ctrl/Command+mouse wheel (#2604)
Ctrl/Command+mouse wheel in the terminal now steps its font size, like in
the editor, when Settings > Editor > General > "Change font size with
Ctrl/Command+Mouse Wheel" is on; with it off the wheel goes to the terminal.

Co-authored-by: pdasilem <[email protected]>
2026-11-11 08:24:18 +08:00
l0ng-ai 4f78a0f3c3 fix(tools): restart a tool's language servers in every open project after installing it (#2613)
Installing or updating a language server restarted it only in the project
the Settings window belongs to; other open projects kept the old version
running (e.g. jdtls 1.55 with 1.63 installed). A new lsp_restart_named
command restarts the servers of that name in every open project.

Refs #2507
2026-11-11 08:19:43 +08:00
l0ng-ai 7079e48716 fix(java): say why a Maven module has no code intelligence when a build plugin doesn't resolve (#2612)
When m2e cannot compute a module's build plan because a build plugin does
not resolve (e.g. a default plugin version a company mirror lacks), the module
is imported without its Java configuration and navigation finds nothing, with
only a pom.xml marker in Project Errors. jdtls now raises a one-time error
notification naming the pom and the plugin, what stops working, and the fix.

Refs #2542
2026-11-11 08:12:28 +08:00
SunZhiQiang 3e90b20d8d feat(agents): quote a mouse selection with a floating Quote button (#2519)
Ending a mouse selection in an agent conversation now shows a small floating
Quote button above it. It takes no focus, so Ctrl+C still copies, and it goes
away when the selection is cleared, the conversation scrolls, Escape is
pressed or something else is clicked. It runs the same agents.quoteReply
action as the context menu, which is unchanged.

Co-authored-by: panda-student <[email protected]>
2026-11-11 08:05:35 +08:00
l0ng-ai 9bd38ddca6 fix(http): read environment files in the encoding the editor detects (#2609)
Environment files that were not valid UTF-8 (Notepad ANSI/GBK, UTF-16)
were skipped silently, leaving only No Environment. They are now decoded with
the editor's detection (ide_vfs::load_text); only a missing file is skipped,
other read errors are reported.

Refs #2435
2026-11-11 07:47:19 +08:00
l0ng-ai c99a22c49c fix(java): rebuild when a launch's output folders were deleted outside the IDE (#2608)
Deleting a module's output folder outside the IDE (mvn clean in a terminal or
by an agent) left JDT's incremental build state current, so the build before
a launch compiled nothing and Run failed with NoClassDefFoundError. After the
incremental build, the launch classpath is checked for folders inside the
project that no longer exist; if any, the workspace is built fully once and
the classpath resolved again. Folders still missing afterwards are remembered
so later launches don't rebuild for them.

Refs #2476
2026-11-11 07:44:28 +08:00
l0ng-ai 41262a2878 test(e2e): give the streaming agent scroll tests the slow-test timeout (#2607)
The streaming tests run close to the 30 s test timeout on CI runners and
failed E2E on unrelated PRs; they now use test.slow().
2026-11-11 07:33:54 +08:00
l0ng-ai 31f84f1261 fix(plugins): find a program named with its extension on Windows (#2602)
resolve_program appended each PATHEXT extension to the name, so a name that
already carried one ("cargo.exe") was looked up as "cargo.exe.EXE" and never
found on Windows. A name with a PATHEXT extension is now also looked up as
written.
2026-11-11 07:33:03 +08:00
梁胜峰andVeng0923 9921ff759b feat(vcs): complete the push target branch from the remote's branches (#1680)
The Push dialog's target branch field was free text only. It now completes
from the branches the chosen remote already has (typing, Up/Down, click; a
typed origin/ prefix is accepted). Prefix matches come first, then other
contiguous matches, then looser ones. Enter only takes a row once one was
chosen, free text still creates a new branch, and the "New" hint shows only
for a name the remote doesn't have. The list widens to the longest name up
to 640px, with the full name as a tooltip beyond that.

Co-authored-by: Veng0923 <+[email protected]>
2026-11-11 07:07:40 +08:00
Andrea Basileandandreabasile 18744b530d feat(remote): forward ports of Compose services from forwardPorts (#2603)
A dev container's forwardPorts may name a port of another Compose service
("selenium:4444"). Those entries were skipped; they are now forwarded like the
container's own ports: the relay in the container connects to the service by
name, forwards are keyed by (host, port), and the Ports window shows them as
selenium:4444 -> localhost:4444. Service ports aren't saved with the ports added
by hand, since forwardPorts forwards them on every connection.

Refs #1083

Co-authored-by: andreabasile <[email protected]>
2026-11-11 06:59:53 +08:00
l0ng-ai f7eeaa4459 test(i18n): JUnit stays JUnit in every locale (#2601)
The JUnit run configuration type is called JUnit in every locale, a product
name like Docker; the catalog check that Chinese and Korean texts differ from
English now allows it.
2026-11-11 06:53:41 +08:00
Tiago R. Diana a231f02914 fix(php): start Laravel LSP without the unsupported --stdio argument (#2598)
Laravel LSP's `lsp` command takes no options and speaks over stdio; the
`--stdio` argument in the PHP language definition made it exit at startup
with "The \"--stdio\" option does not exist". A registry test now checks it
starts without arguments.

Co-authored-by: tiagodiana <[email protected]>
2026-11-11 06:51:57 +08:00
l0ng-ai b5c92181a5 feat(agents): roll back an agent task's changes (#2599)
The task's Changes pane gets Rollback: the selected files, or all of them, go
back to how they were when the task started, after a confirmation. Files the
user had already changed in the project's checkout when the task started are
copied aside then, so rolling back restores the user's own edits instead of
the last commit; files the agent created are removed. The copies stay with the
task (forks share them) and are removed with it. Rollback is refused while the
agent is working.

Refs #1777
2026-11-11 06:50:11 +08:00
l0ng-ai 3863d4dc75 feat(run): JUnit run configurations (#2597)
Run/Debug Configurations gets a JUnit type: a module and its tests (all, a
package, a class, a method or class-name patterns) with VM options. It runs
through the module's detected Maven or Gradle test configuration with the same
arguments as the gutter's Run on a test, so results show in the test runner
and Debug and Rerun Failed Tests work as for those runs. VM options go to
Surefire as -DargLine, and to Gradle's test JVMs through the init script.

Closes #2515
2026-11-11 05:57:11 +08:00
l0ng-ai b1bee808dd feat(dependencies): complete npm package versions in package.json (#2596)
Completion inside a version string under dependencies, devDependencies,
peerDependencies or optionalDependencies lists the package's versions from the
npm registry (abbreviated metadata, shared version cache), newest first,
without deprecated versions and without pre-releases when the package has
releases. Aliases, paths, URLs and workspace protocols are left alone; the
scanner tolerates a document being edited.

Closes #2558
2026-11-11 05:50:22 +08:00
l0ng-ai a8be534043 feat(vcs): move a worktree to another folder (#2595)
Move Worktree… in the Worktrees dialog and the Git tool window's Worktrees
tab moves the selected worktree's folder into the chosen folder, or to it when
it already has the worktree's folder name. An empty folder at the target is
replaced (Git would move the worktree inside it) and any other existing target
is refused. The main worktree and the one holding the open project can't be
moved.

Closes #1238
2026-11-11 05:41:42 +08:00
l0ng-ai b9aaef73d6 feat(lsp): lint JavaScript and TypeScript with ESLint (#2594)
Files below an ESLint configuration (eslint.config.*, .eslintrc*, or a
package.json with eslintConfig) in JavaScript, TypeScript, JSX, Vue, Svelte and
Astro are checked by vscode-eslint-language-server with the project's own
eslint. The server is a managed tool; its settings include the `experimental`
object it reads on every request, and its eslint/* requests get answers. Root
markers can name a JSON object (`package.json#eslintConfig`) as well as a TOML
table.

Closes #2451
2026-11-11 05:32:52 +08:00
Deng Liuanddengliu 4b0067993b feat(agents): configure MCP servers for coding agents (#789)
MCP servers for coding agents are kept in OpenIDEA's own settings: User
(settings.toml) and Workspace (.openidea/settings.toml), where a workspace
server replaces a user server of the same name. Enabled servers that match the
session's agent and the transports it negotiated (stdio, http, sse) are passed
in session/new, session/load and session/fork alongside the IDE's own server.
Settings > Agents > Model Context Protocol (MCP) lists, adds, edits, enables
and removes them; Import copies .mcp.json and .vscode/mcp.json entries into the
workspace settings once without changing those files.

Co-authored-by: dengliu <[email protected]>
2026-11-11 05:18:01 +08:00
l0ng-ai d2b0bd6e75 feat(http): convert requests to and from cURL, add environment files (#2592)
Convert to cURL and Copy copies the request at the caret as a cURL command with
the selected environment applied; pasting a `curl ...` command into a .http file
inserts the request it sends (headers, data variants, --json, -F multipart, -u,
-b, -A, -e, -G, POSIX and Windows line continuations). Add Environment to
Public / Private File opens the env file that applies to the request file,
creating it next to it when missing. The conversion lives in ide-http and
shares request preparation with running.

Refs #2435
2026-11-11 04:59:34 +08:00
l0ng-ai 3b7d72bd03 fix(navigation): show usages on a declaration a plugin adds targets to (#2591)
Go to Declaration on a declaration shows its usages only when the answer was the
declaration alone; a plugin definition merged into it (the MyBatis XML statement of a
mapper method) turned it into a chooser and the callers became unreachable. An answer
containing the declaration at the caret now shows usages too.

Fixes #1745
2026-11-11 04:50:57 +08:00
l0ng-ai 74e1a13428 fix(agents): offer each Claude model's effort levels before its first session (#2590)
The New Task page only knew the effort levels of models a session had run, so
switching to another model showed no effort choice. Claude Code discovery now
records every model's effort levels from its initialize response (a session's report
still wins) and runs again when an offered model's levels are unknown.

Fixes #1686
2026-11-11 04:45:30 +08:00
l0ng-aianddraylegend 3d6e983523 fix(agents): read a restored session's conversation after connecting (#2589)
A session tab restored with the project read its conversation while the agents
connection was still being set up; connecting then cleared the entry cache and
the tab kept spinning. The session now loads its conversation, subagent views and
changes once the connection is ready.

Fixes #2414

Co-authored-by: draylegend <[email protected]>
2026-11-11 04:38:12 +08:00
l0ng-ai 38b8b0d8df fix(maven): build with artifacts the local repository already has (#2588)
Maven 3.9 re-resolves a locally cached artifact whose recorded origin repository
is not declared in the build and fails when no declared repository has it. The
IDE's own Maven runs (Reload, Build, Rebuild, Build Artifacts) now track the
local repository under another file name, so such artifacts count as locally
installed; Maven goals the user runs keep plain Maven behaviour.

Refs #2293
2026-11-11 04:34:53 +08:00
l0ng-ai 6d82b94b69 fix(boot): name an outdated system web engine on the startup failure page (#2587)
On macOS 11 with the original Safari 14 the system web engine cannot parse the
Safari 16 bundle, and startup failed with only "Could not load startup script".
When the engine lacks Safari 16 features, the failure page now says the system
Safari (or WebKitGTK) is too old and how to update it.

Refs #2008
2026-11-11 04:30:57 +08:00
Sergei Melnikov 8ef5132b5e fix(vcs): update and push only the repository whose branch menu was used (#2584)
In a project with several Git repositories, Update and Push… on one repository's
current branch (Branches popup, Git Log branch menu, Git Log Update Selected) acted
on every repository because vcs.update / vcs.push were called without naming it.
They now pass the repository: only it is pulled, and only it is checked in the
multi-repository Push dialog. When the upstream branch was deleted on the remote,
the Push dialog lists the commits no remote branch contains instead of the last 50.

Co-authored-by: pdasilem <[email protected]>
2026-11-11 03:46:12 +08:00
l0ng-ai a82e0e2ec5 feat(vim): af / if function text objects (#2586)
af selects the function, method, constructor or lambda around the cursor and
if its body, from the syntax tree; whole-line functions are taken linewise, and
a repeat in Visual mode or a count takes the enclosing function. Mapped only in
Operator-pending and Visual mode, and restored after an rc reload.

Closes #2365
2026-11-11 03:44:31 +08:00
l0ng-ai 377a24375d feat(maven): generate sources and update folders (#2585)
jdtls adds a generated source folder only when it exists at import, so code from a
later generate-sources stayed unresolved. The Maven tool window gets Generate Sources
and Update Folders For All Projects (toolbar, Find Action) and a per-project variant in
the context menu: it runs generate-sources and, once a run succeeds, reloads the Maven
projects. run.runConfig accepts an onExit callback so a feature can wait for the run.

Fixes #2508
2026-11-11 03:37:35 +08:00
Mansur Akhmadiandmansur-akhmadi e6e5e4b6c6 fix(vfs): poll the whole tree when inotify runs out of watches (#1916)
On Linux each directory gets its own inotify watch. Once the watch limit ran out,
the directories past it never synced. Now a limit hit during startup fails the start
with the recorded reason, so the existing fallback polls the tree; a limit hit later
(background registration, new folders) swaps the OS watcher for a poller over the
whole tree, re-reading the batch that hit it so files already in a refused new folder
are reported. unwatched_reason() keeps surfacing the limit for the notification.

Refs #1915

Co-authored-by: mansur-akhmadi <[email protected]>
2026-11-11 03:28:49 +08:00
Arnaud Ritti 55fc760abb fix(editor): group plugin gutter links by line and icon (#2574)
Plugin line markers on the same line with the same resolved icon now render as one
gutter button; duplicate destinations from several providers are dropped and every
distinct target goes to the existing target chooser. Run markers and different icons
stay separate. The default `navigate` icon gets its own navigation glyph instead of
reusing the implementing-method one, as the WIT documents them as distinct icons.

Closes #2573

Co-authored-by: arnaud-ritti <[email protected]>
2026-11-11 03:27:21 +08:00
Andrea Basileandandreabasile 86e8bb5c4d fix(run): use the environment of the project root for a Django project in src/ (#2580) (#2580)
A Python project in a folder of the opened project, such as a Django project with
manage.py in src/, looked for its virtualenv or lock file only in that folder, so with
.venv, uv.lock or poetry.lock at the root it ran with the system python3. The lookup now
walks up to the opened project; virtualenv and Pixi paths found above get a ../ prefix.
Only a folder's own virtualenv makes it a pytest project, so folders borrowing the
root environment do not each get a pytest run.

Co-authored-by: andreabasile <[email protected]>
2026-11-11 03:22:32 +08:00
l0ng-ai afb7c8ea63 fix(refactor): make the preview and pull/push members dialogs resizable (#2583) (#2583)
Make the refactoring preview dialog (Rename, Safe Delete, Pull Up / Push Down) and the
Pull Members Up / Push Members Down dialog resizable. The preview had a fixed width and
a 320px list cap; the list now fills the dialog, the size is remembered and full paths
show as tooltips. The member table in the pull/push dialog takes the extra space.

Closes #2570
2026-11-11 03:20:20 +08:00
l0ng-ai f7aa81305a feat(project-view): option to hide the path beside project roots (#2579)
Project View › Options › Appearance gains Show Root Paths (on by default). Off, workspace roots and a single project's root show only their name; the path stays in the tooltip. Workspace roots now abbreviate the home folder as ~.

Fixes #2557
2026-11-11 02:18:14 +08:00
l0ng-ai 1a6fc7fd21 fix(debugger): show the output of Maven, Gradle and script debugging in the Debug console (#2578)
Debugging a configuration the Run feature starts with a JDWP agent (Maven, Gradle, Spring Boot, Tomcat, scripts set to the Java debugger) attaches the debugger to that process, and its output stayed in the Run tab while the Debug console was empty. The run feature now provides its sessions' output by process (`RunOutputService`), and a debug session attached to one follows it in its Console until the session is closed.

Fixes #2494
2026-11-11 02:14:27 +08:00
ntbang0901 835a9c883c feat(plugins): let plugins depend on each other and declare extension points (API 1.5) (#2383)
Plugins could not rely on each other: nothing said "this plugin needs that one", and a plugin had no way to let others extend it. Plugin API 1.5 adds both.

- `[[depends]]` in `plugin.toml` (version range, `optional`). A plugin whose required dependency is missing, disabled, the wrong version, cyclic or unable to run is shown as Blocked with the reason and starts once the dependency can run again. Plugins start in dependency order; disabling a plugin others need asks first. Granting or revoking a dependency's permissions also settles its dependents.
- `[[extension-points]]` declares points with typed fields. Dependents contribute to them from `plugin.toml` (checked at install time); the owner reads them through the read-only `extensions` interface (`host::extension_contributions`) and hears changes through the `extensions-changed` event.

The plugin details page shows what a plugin requires, who requires it, and its extension points with their contributors. Covered by the `ext-owner` fixture and `plugin_dependencies.rs`.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-11 02:05:41 +08:00
Vadim LazukandInf1mum d62e9855b1 feat(plugins): read a project file's bytes (API 1.5) (#2524)
`read-file` returns UTF-8 text only, so a plugin could not read a PDF, image, archive or font inside the open project, and `fs.read-bytes` needs a per-path grant. `workspace.project` gains `read-bytes(path)` with `read-file`'s rules (`project.read`, `resolve_in_root` confinement, `not-found` for missing paths, 16 MiB cap; always the file on disk). The Rust SDK exposes `Project::read_bytes` and the mock host `add_file_bytes`. Web custom editors get `document.bytes()` (at most 64 MiB, read-only).

Fixes #2347

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 01:50:10 +08:00
l0ng-ai 9660a47958 fix(i18n): say 依赖于 for project dependencies in Chinese (#2577)
The Project Structure dependency labels translated "depends on" as 取决于 ("is determined by"); they now read 依赖于.
2026-11-11 01:44:31 +08:00
l0ng-ai 2d404a8664 fix(i18n): restore the missing commas in the en and zh text catalogs (#2576)
Two merged changes (#2568, #2572) each appended entries next to "Recent color", and the textual merge dropped the comma between them, so en.json and zh.json no longer parsed.
2026-11-11 01:43:13 +08:00
aconeshanaandaconeshana 83cbc8ff5c fix(vcs): open the source file on double-click in the modal Local Changes (#2500)
In the modal Local Changes view (Git tool window), double-click and Enter always opened the diff. `openNode` in `CommitView.tsx` ignored the commit interface. The modal view now opens the source file and the non-modal Commit tool window keeps the diff. A new `vcs.commit.doubleClickOpensDiff` setting (unset = follow the interface) is exposed as Show on Double-Click → Source / Diff / Follow the Interface.

Co-authored-by: aconeshana <[email protected]>
2026-11-11 01:42:25 +08:00
Muuandmuutot eab165b4e0 fix(project-structure): list only configured projects as dependency candidates (#2572)
The Dependencies checklist offered every recently opened project as a candidate, which made the list long and noisy. A standalone project now lists only its configured dependency projects; in a workspace the other roots attached to the window stay candidates, and Add Project… picks a new one (the empty-state note says so).

Co-authored-by: muutot <[email protected]>
2026-11-11 01:41:48 +08:00
Muuandmuutot 5bdf4835b0 feat(editor): lean location chooser rows with the container column (#2482)
The location choosers (Choose Declaration / Implementation and the gutter marker targets) showed the name, a source-line preview, the file name and often a full absolute path on every row. Rows now have one layout: icon + name, the declaring container, and `path:line` right-aligned.

Co-authored-by: muutot <[email protected]>
2026-11-11 01:41:27 +08:00
Muuandmuutot 1acc007e47 feat(editor): fall back to type-hierarchy subtypes for gutter markers (#2434)
Servers that answer implementation requests with nothing (Pyrefly) still know the subtypes in the type hierarchy. When a type's implementation batch comes back empty, the gutter asks `typeHierarchy/subtypes` and uses its answer for the ↓ marker. The ↑ marker on class declarations is dropped.

Co-authored-by: muutot <[email protected]>
2026-11-11 01:40:50 +08:00
Bin 727dabe579 fix(editor): anchor the colour picker and complete its controls (#2568)
Colour swatches overlapped the Git change indicators, and the native colour picker could open at the window's top-left corner. The gutter spacing is adjusted and the native picker is replaced by a picker anchored beside the swatch (react-colorful) with screen sampling, a preview, recent colours and R/G/B/A%/HEX input. Markup files use the syntax tree, so colour-like template or HTML text no longer gets a swatch while embedded and inline styles still do.

Co-authored-by: ffgenius <[email protected]>
2026-11-11 01:40:07 +08:00
Muuandmuutot c70203b640 feat(ui): numeric mnemonics in the Select In popup (#2567)
The Select In popup (Alt+F1) labels its first nine selectable rows 1–9, and pressing the digit activates the row. `MenuList`/`Menu` gained an optional `numbers` flag and `showContextMenu` an options argument; only Select In opts in, and submenus never inherit the numbering.

Co-authored-by: muutot <[email protected]>
2026-11-11 01:39:21 +08:00
Muuandmuutot c088fa8b26 fix(editor): collapse empty icon gutters (#2566)
The run, npm-script, HTTP-endpoint and bookmark gutters reserved a column even when a file had none of their icons, pushing the code right by an empty strip. Each now emits a zero-width, non-interactive spacer marker when it has nothing to show, so the column collapses; files with icons are unchanged. A shared `gutterSpacer` factory keeps the spacer from reacting to clicks.

Co-authored-by: muutot <[email protected]>
2026-11-11 01:39:06 +08:00
l0ng-ai 481cf172b5 feat(java): offer every Java language level, 5 through 25 (#2575)
The Java language level settings offered only the LTS releases, and only those reached jdtls. Every level from 5 to 25 is now listed, and each maps to JDT's compliance value (1.5 to 1.8, then 9 to 25), so projects on 6 or 7 can be set up with a jdtls that still compiles them.

Fixes #2465
2026-11-11 01:29:26 +08:00
l0ng-ai ad7f03090a fix(python): name conda environments after their prefix on Windows (#2571)
Conda interpreters were labelled with the interpreter's grandparent folder, which on Windows (python.exe at the prefix root) named every env under envs/ "conda: envs" and the base env after the install folder; the colliding names made the interpreter popup show every full path. Env names and homes now come from the prefix itself (base for the installation), and creating a conda env on Windows finds python.exe at the prefix root instead of reporting it as not created.

Fixes #2512
2026-11-11 01:18:53 +08:00
l0ng-ai 0e3b35e1e5 fix(nav): answer unresolved names only from dependency roots (#2569)
When the language server found no declaration, Go to Declaration and the Ctrl/Cmd-hover link fell back to every index symbol with that name in the file's navigation roots, its own root included. The server already sees that root, so an unresolved member call such as r.save() offered every save in the project. The index fallback now searches only the roots the server can't see: configured dependency projects and extra content roots.
2026-11-11 01:15:36 +08:00
Shohruh Ismoilovandshohruh-genius 025e3e9255 fix(project): detect project types and honor workspace type sorting (#2522)
Workspace roots always sorted by name and only Maven/Gradle roots had a type icon. Project types are now detected from root build/SDK file names (no file contents or tools are read, up to four concurrent listings) through one registry (projectType.ts) that supplies names, icons, markers and refresh handling; roots show the type's icon, and Sort by Type groups roots by type with natural name order inside each type. Flutter and Vite logos come from Simple Icons (CC0, recorded in THIRD_PARTY_LICENSES.md).

Co-authored-by: shohruh-genius <[email protected]>
2026-11-11 01:07:40 +08:00
Deng Liuanddengliu d78a73452b feat(docker): Compose schema association and internal reference completion/navigation (#2455)
Docker Compose files get Compose Specification completion, hover and validation from yaml-language-server: languages/yaml.toml maps the pinned compose-spec schema to the standard Compose filename globs, and ide-sdk / ide-lsp feed content-detected Compose files and user overrides (Kept / Excluded as !path negations) into yaml.schemas, reconfiguring only when that set changes. The Compose editor completes and navigates (⌘-click / ⌘B) references to declared services, networks, volumes, configs and secrets in depends_on, links, volumes_from, extends.service, networks, volumes, configs and secrets.

Co-authored-by: dengliu <[email protected]>
2026-11-11 01:06:56 +08:00
黄贵波 eed2de8d66 fix(window): zoom to the work area in step with the webview frames (#1450)
Double-clicking the title bar ran NSWindow zoom:, and WebKit doesn't lay the page out during an animated resize, so the window grew empty and the content appeared only at the end. The backend now steps the frame itself over 350 ms (smoothstep), waiting for each webview presentation (_doAfterNextPresentationUpdate:, checked with respondsToSelector: and falling back to a fixed 16 ms step), so the page grows with the window. A window restored maximized by the window-state plugin goes back through AppKit's unmaximize; the frame to restore to is kept in window-zoom.json so a window left zoomed at quit can still restore. Marker lanes recompute at most every 120 ms during resize bursts.

Closes #1448

Co-authored-by: IsGuibo <[email protected]>
2026-11-11 01:05:17 +08:00
Muuandmuutot e3808fc7c2 feat(project): resolve navigation into attached dependency projects (#2448)
Settings › Project Structure › Dependencies lists other projects this one depends on (and, in a workspace, the roots each member depends on). Dependency projects join the content roots, so the index, Project view and run see them, and navigation resolves across them: nav_resolve_module maps an import path to a file in the file's navigation roots, and nav_index_definitions answers bare-name lookups filtered to those roots inside the index (so own-root matches can't evict dependency matches). navigation_roots is memoized for 1 s, and hand-written member_dependencies keys with the other path separator match on Windows.

Co-authored-by: muutot <[email protected]>
2026-11-11 01:01:09 +08:00
Budi Santosoandboenkkk 33786c14ba feat(db): Oracle thin driver and package nodes (#2129)
Oracle data sources now connect through the pure-Rust oracledb thin driver (pinned to 26.0.0-beta.5) instead of ODPI-C, so no Oracle Instant Client, ORACLE_HOME or Instant Client setting is needed; TNS descriptors and tnsnames.ora aliases still work in the Database field. The database tree gains a packages folder for Oracle schemas, and a package's DDL opens as its specification and body. The database guide's Oracle section is rewritten for the thin driver.

Co-authored-by: boenkkk <[email protected]>
2026-11-11 00:31:43 +08:00
ntbang0901 51ffd093b7 feat(keymap): import an IntelliJ IDEA keymap XML (#716)
Settings › Keymap › gear menu › Import IntelliJ IDEA Keymap… reads a keymap XML exported from IntelliJ IDEA (or one from its keymaps folder): the parent attribute picks the matching preset, IDEA action ids map through the table IdeaVim's :action uses (moved to src/platform/ideaActions.ts), Swing keystrokes, two-stroke chords and mouse shortcuts are converted, and the result is saved as a new custom keymap with a summary of actions that have no counterpart and skipped shortcuts.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-11 00:25:31 +08:00
Mansur Akhmadiandmansur-akhmadi 379709f13f feat(editor): rainbow indent guides (#1751)
Settings › Editor › General › Appearance › Rainbow indent guides (off by default, under Show indent guides): guide N is drawn in --syn-rainbow-((N-1) mod 5 + 1), so each level of a deep block has its own colour. With the per-guide strips this only swaps the image each guide uses (--openidea-guide-1…5 from theme.ts), so theme and scheme switches recolour open editors through CSS; code lens rows follow the setting.

Closes #1560

Co-authored-by: mansur-akhmadi <[email protected]>
2026-11-11 00:23:10 +08:00
SunZhiQiang 0311bd3bf8 feat(markdown): add full-screen preview and PNG export to Mermaid diagrams (#2460)
Drawn Mermaid diagrams in the Markdown preview and agent replies get Full screen and Download image buttons. Full screen fits and centres the drawing, pans on drag and zooms with Ctrl/Cmd+wheel; Esc closes it. Download saves a 2x PNG on the editor background through the native save dialog. reid() moved to svgIds.ts so the full-screen copy reuses it.

Co-authored-by: panda-student <[email protected]>
2026-11-11 00:17:20 +08:00
Deng Liuanddengliu df87c3ba1f feat(vcs): add git blame commit popup and per-commit gutter colors (#2484)
Annotate gutter lines get a background tint per commit, ordered from older (cool) to newer (warm) using the project colour tokens, and hovering highlights every line of that commit. Hovering a line opens a non-modal commit popup with the full hash (opens the commit in Git Log), author, date and the full message with issue and commit-hash links; commit metadata comes from the single-commit log fast path and is cached.

Co-authored-by: dengliu <[email protected]>
2026-11-11 00:15:29 +08:00
SunZhiQiang e8ef07f918 feat(vcs): drag unversioned files onto a changelist to add them (#2531)
The changes tree registered drag sources only for rows that already belonged to a changelist, so files under Unversioned Files could not be dragged. Drag sources and drop targets are now separate: every file or folder row drags, and dropping unversioned paths on a changelist runs Add to VCS (git add) before moving them into that list.

Co-authored-by: panda-student <[email protected]>
2026-11-11 00:12:24 +08:00
KangHyunTaeandrkdgusxo66 6a612a832c fix(vim): run a key typed right after a Normal-mode IME composition as a Vim command (#2564)
On WebKit, CodeMirror drops the first keydown within 100 ms of compositionend (meant for the IME's own Enter). In Normal mode that key never reached Vim, and a printable key was written into the document — e.g. Korean ㄱ, switch to English, quick j. A single ASCII character that arrives as text input in that window now runs as a Vim key through vimKeyFromEvent (so <Space>, <CR> and langmap apply); a keydown that does reach the editor clears the window, and keys that type no text are still dropped.

Fixes #2562

Co-authored-by: rkdgusxo66 <[email protected]>
2026-11-11 00:07:33 +08:00
DrPepperandmakise2060 e09cb2353f fix(workbench): keep the resize cursor while dragging a splitter (#2560)
A resize drag set document.body.style.cursor, so as soon as the pointer left the 1px handle the element under it (tree, editor, button) showed its own cursor mid-drag. The drag now holds the cursor on <html> with a data-drag-cursor rule that overrides descendant cursors, released on pointerup/pointercancel/window blur, with nested locks restoring the outer cursor. Tool-window gaps, split and stripe handles, the dialog resize grip and database / Git Log column resizing share it, and the handles use ew-resize / ns-resize throughout.

Co-authored-by: makise2060 <[email protected]>
2026-11-11 00:06:59 +08:00
l0ng-ai bf54006f05 fix(mybatis): find a mapper's XML by searching for its namespace (#2565)
Java → XML lookups read every XML file in the project to find the namespace; with hundreds of XML files that ran past the provider's 500 ms budget, so no gutter markers appeared and repeated timeouts disabled the plugin. The lookup now searches for namespace="…" with the host's text search and reads only the matching files, cached per namespace for 30 s. The plugin requires engine ^1.4 (0.4.1).

Fixes #2505
2026-11-11 00:05:33 +08:00
l0ng-ai ad39bda16f docs: add security policy for private vulnerability reports (#2449)
SECURITY.md asks for vulnerability reports through GitHub's private advisory form and says what to include; the issue chooser links the same form.
2026-11-11 00:01:58 +08:00
l0ng-ai 283b3bbeea fix(agent): show the details an agent sends with an error (#2563)
ACP agents report failures as a generic JSON-RPC error and put the cause in data.details (or a string data). Only the message was shown, so such failures read just "Internal error". The details are now appended unless the message already contains them.
2026-11-10 23:57:29 +08:00
l0ng-ai 51e4ec0ab9 fix(dap): keep output events whose text isn't UTF-8 (#2561)
netcoredbg on Windows embeds the debuggee's console output in the ANSI code page (GBK on a Chinese system). The message wasn't valid UTF-8, failed to parse and was dropped, so Debug console lines with non-ASCII text went missing. A message body that isn't UTF-8 is now decoded like other tool output (UTF-8, else the system code page) and parsed again.

Fixes #2489
2026-11-10 23:55:59 +08:00
ppxbandppxb 5593260635 fix(maven): run generated build-tool goals with the configured JDK (#2440)
A Maven goal or Gradle task run from the build tool window carried the JAVA_HOME the build model had when the tree was built, and that value was frozen into the (persisted) temporary configuration, so correcting the JDK in Settings only took effect after a restart, or not at all. For IDE-generated build-tool configurations JAVA_HOME is now derived: Settings (the Maven runner JRE, the Gradle JVM) decide at each launch, a saved configuration drops the derived value, and a dead path no longer outlives its setting. Configurations of the user's own, and Tomcat's JDK field, keep their JAVA_HOME.

Fixes #2433

Co-authored-by: ppxb <[email protected]>
2026-11-10 23:44:09 +08:00
l0ng-ai f4f648567f fix(agents): preserve upward scrolling during streaming replies (#2502)
Small upward scrolls while a reply streamed were still treated as being at the end, so each update pulled the view back down. Following pauses on upward wheel, touch or keyboard input before the browser scrolls, and queued updates can't resume it until the user scrolls down or jumps to the latest reply; streaming, resize and late diagram updates leave the viewport alone meanwhile.
2026-11-10 23:41:44 +08:00
l0ng-ai 7e365a3e0f fix(search): resizable preview split in Search Everywhere (#2496)
Search Everywhere had a fixed 220px preview and a separate results height cap. It now has a remembered vertical split (initially one third results, two thirds preview); the query keeps focus while resizing and the active result stays visible when the list shrinks. Action names line up with file names, shortcuts stay readable, and scope controls move to a second row in narrow popups.
2026-11-10 23:41:36 +08:00
l0ng-ai 7594a98e8c fix(search): use 16px file icons and spinners in Find and Usages (#2493)
Find results and Show Usages use the native 16px file/folder icons, and the four search indicators share the 16px eight-spoke animation, replacing 12/14px overrides; the preview and rerun controls get their proper icons.
2026-11-10 23:41:29 +08:00
l0ng-ai 839123caf2 fix(run): use 16px test status icons and the shared running indicator (#2487)
Run and Services test trees now use 16px status and filter icons and the shared eight-spoke running indicator. The fixed 11/13px glyphs made columns uneven, and a status change shifted the name; the running indicator now follows the tree's icon size.
2026-11-10 23:41:22 +08:00
l0ng-ai 9153c66849 fix(run): keep Run/Debug Configurations usable in narrow windows (#2485)
The Name field was squeezed and the dialog overflowed in narrow or zoomed windows because the sidebar, label column and header controls had fixed widths. Add a remembered sidebar splitter, shared label columns, and wrapping Name/storage and footer groups; Before launch actions stay together and both panes keep scrolling independently when the dialog is resized.
2026-11-10 23:41:15 +08:00
l0ng-ai 27ee6fde61 fix(vcs): show feedback for explicit Git commands (#2444)
Update could fail when incoming files collided with local untracked files and show nothing, because VCS popups were disabled. The shared Git command runners now mark progress and results as responses to a user command, so Update, Pull, Fetch, Push and local Git operations stay visible; background notifications and explicitly handled or cancelled failures stay quiet.
2026-11-10 23:41:06 +08:00
l0ng-ai 0574691d7e fix(format): reformat with the Code Style indent, not the detected one (#2554)
Without a per-language Tabs and Indents override, Reformat Code asked the
language server for the indent detected in the file. A file indented with 2
spaces (e.g. formatted elsewhere with Google style) then reformatted to 2
spaces and reported "already properly formatted", while a new file got 4.
Reformat now always asks for the Code Style indent (EditorConfig included);
detection only affects editing, and an indent chosen for the file in the
status bar still applies.

Fixes #2467
2026-11-10 23:37:29 +08:00
l0ng-ai 5948f8a7a3 fix(hierarchy): show no expand arrow on types without subtypes (#2553)
The type hierarchy loaded a level only when its node was expanded, so every
type showed an expand arrow, also one without subtypes. The level below each
shown type is now loaded ahead, four requests at a time, so a leaf shows as
one. Call hierarchies stay lazy.

Fixes #2471
2026-11-10 23:34:42 +08:00
1151 changed files with 77620 additions and 4681 deletions

No files matched your search

+83
View File
@@ -0,0 +1,83 @@
---
name: eval
description: Run the code-intelligence evals (`pnpm eval`), pick the worst failing scenario, diagnose it from the attached timeline, fix it, prove the fix with before / after numbers measured on this machine without regressing other scenarios, and open one PR. Use for "跑 eval", "优化 Java 代码智能", "eval 失败了", an issue labelled `eval`, or any change to Java navigation / usages / gutter markers that needs numbers.
---
# Eval loop: measure → fix → prove
`pnpm eval` drives the production workbench in Chromium against a release `openidea-server`
with real language servers (Java / jdtls for now) and judges every scenario on what the user
sees: time to feedback, to the first result, to the final one, and the results themselves vs
`evals/<language>/golden/`. One machine, before and after: numbers from different machines or
from the nightly host don't judge a fix.
## 0. Preconditions
- `jdtls` on PATH and `JAVA_HOME` pointing at a JDK of the pinned `JAVA` version (projects
compile against it, and the JDK scenarios' results are in its sources; jdtls itself runs on
a newer installed JDK when it needs one). Pinned versions: `.github/tool-versions.env`. A
mismatch prints a warning and the report records both: say so in the PR when it applies.
- First run builds `cargo build --release -p ide-host --bin openidea-server` and the perf
bundle; later iterations of a frontend-only change can use `--no-build` after a
`pnpm exec vite build --mode perf --outDir target/eval/dist`, a backend-only one after the
cargo build alone.
- Run evals one at a time on this machine, with nothing heavy (builds, other evals) beside them.
## 1. Baseline on the base commit
```sh
git switch --detach origin/main # or the fix branch's merge base
pnpm eval --save-baseline # target/eval/baseline.json, this machine only
```
Read `target/eval/report.json`. Each verdict has `status`, `failures` (`kind`: `error`,
`timeout`, `degraded`, `correctness`, `budget`, `golden`), `regressions`, `repro` (the command
that reruns it) and, for failures, `diagnostics`: every command sent to the server in the
slowest run's window (`cmd`, `start`, `ms`, `ok`, `bytes`, `items`), background tasks, server
status changes and the server's log tail.
## 2. Pick one problem
Worst first: wrong or missing results and silent degradation, then timeouts, then the
feedback budget (100 ms), then the slowest `completeMs`. One problem per PR. Say which
scenario and metric, with its value vs budget.
## 3. Diagnose from the timeline
- Commands in the window ordered by `start`: what ran, what waited on what, what's serial
that could overlap, which request returned many `items`/`bytes`, which failed.
- `feedbackMs` far above 16 ms with a fast `firstResultMs`: the UI shows nothing while it
waits (a hint or popup that only appears after a delay).
- A task still running across the window: background work competing with the request.
- For the frontend side rerun with `--trace` and open `target/eval/traces/<id>@<cond>.json`
in Chrome's Performance panel.
- Name the cause at class level (as in `issue-fix`): it usually affects sibling scenarios.
## 4. Fix, then rerun that scenario until it passes
```sh
pnpm eval --only <scenario> --condition <warm|cold> # the verdict's `repro`
```
Correctness is enforced, so a faster answer that drops, truncates or serves stale results
fails. Don't trade a budget miss for a degraded answer.
## 5. Full suite, then PR
```sh
pnpm eval # vs the baseline from step 1: no new failures or regressions
```
PR per `issue-fix` §6, plus in **How it was verified** the before / after table from
`target/eval/report.md` (both on this machine, commits named) and the tool versions line.
## Guard rails
- Goldens (`evals/*/golden/`) and budgets (`evals/budgets.json`) are human-owned. Don't edit
them, and don't run `--update-golden`, to make a scenario pass. When a golden looks wrong
(a tool bump, a deliberate behaviour change), stop and say so: a PR that touches them gets
`needs-decision`.
- Don't weaken a scenario's checks or timeouts in `evals/<language>/scenarios.ts` to pass.
Adding a scenario (for a case the sweep or a user found) is fine; record its golden with
`--update-golden --only <id>` and ask a person to review the file.
- A flaky timing (passes on rerun) is not a regression: rerun once before acting on one.
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/eval
+2
View File
@@ -0,0 +1,2 @@
custom:
- https://github.com/l0ng-ai/openidea#sponsor
+3
View File
@@ -3,3 +3,6 @@ contact_links:
- name: Download the latest release
url: https://github.com/l0ng-ai/openidea/releases/latest
about: Many problems are already fixed. Please check the latest version before reporting.
- name: Report a security vulnerability
url: https://github.com/l0ng-ai/openidea/security/advisories/new
about: Please report security problems privately, not in a public issue.
Binary file not shown.

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 97 KiB

+14
View File
@@ -0,0 +1,14 @@
# Language server and tool versions for Language smoke (.github/workflows/language-smoke.yml)
# and the code-intelligence evals (pnpm eval). Pinned so a release can't fail an unrelated
# PR or move a golden result; bump them deliberately (and re-record goldens: pnpm eval --update-golden).
VTSLS=0.3.0
TYPESCRIPT=6.0.3
BASEDPYRIGHT=1.40.2
RUFF=0.17.3
GOPLS=v0.24.0
JDTLS=1.63.0-202610191322
KOTLIN_LSP=263.7084.0
GRADLE=8.14.3
INTELEPHENSE=1.18.5
# The JDK projects compile against (JAVA_HOME): library navigation opens its sources.
JAVA=21
+4 -11
View File
@@ -24,6 +24,7 @@ on:
- "package.json"
- "pnpm-lock.yaml"
- ".github/workflows/language-smoke.yml"
- ".github/tool-versions.env"
pull_request:
branches: [main]
types: [opened, synchronize, reopened]
@@ -38,16 +39,6 @@ concurrency:
env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
# Pinned so a server release can't fail an unrelated PR; bump them deliberately.
VTSLS: 0.3.0
TYPESCRIPT: 6.0.3
BASEDPYRIGHT: 1.40.2
RUFF: 0.17.3
GOPLS: v0.24.0
JDTLS: 1.63.0-202610191322
KOTLIN_LSP: 263.7084.0
GRADLE: 8.14.3
INTELEPHENSE: 1.18.5
jobs:
smoke:
@@ -61,6 +52,8 @@ jobs:
timeout-minutes: 40
steps:
- uses: actions/checkout@v4
# Tool versions, shared with the evals (pnpm eval).
- run: grep -E '^[A-Z0-9_]+=' .github/tool-versions.env >> "$GITHUB_ENV"
- uses: dtolnay/rust-toolchain@stable
with: { components: "rust-analyzer, rustfmt" }
# One build cache for every language; main saves it.
@@ -96,7 +89,7 @@ jobs:
- if: matrix.language == 'java' || matrix.language == 'kotlin'
uses: actions/setup-java@v4
with: { distribution: temurin, java-version: "21" }
with: { distribution: temurin, java-version: "${{ env.JAVA }}" }
- if: matrix.language == 'java'
run: |
mkdir -p "$RUNNER_TEMP/jdtls"
+22 -1
View File
@@ -251,7 +251,28 @@ jobs:
core.setFailed(`latest.json would miss ${missing.join(", ")}`);
return;
}
const manifest = { version: tag.replace(/^v/, ""), notes: "", pub_date: new Date().toISOString(), platforms };
// The update's notes: the release body without its Download section. That
// section only matters on the release page, and What's new follows it.
const { data: release } = await github.rest.repos.getRelease({ owner, repo, release_id });
const withoutDownloadSection = (body) => {
const kept = [];
let skip = 0;
for (const line of body.split(/\r?\n/)) {
const heading = /^(#{1,6})\s+(.*)$/.exec(line);
if (heading) {
const level = heading[1].length;
if (!skip && level <= 3 && /^download$/i.test(heading[2].trim())) {
skip = level;
continue;
}
if (skip && level <= skip) skip = 0;
}
if (!skip) kept.push(line);
}
return kept.join("\n").trim();
};
const notes = withoutDownloadSection(String(release.body ?? ""));
const manifest = { version: tag.replace(/^v/, ""), notes, pub_date: new Date().toISOString(), platforms };
const old = assets.find((a) => a.name === "latest.json");
if (old) await github.rest.repos.deleteReleaseAsset({ owner, repo, asset_id: old.id });
await github.rest.repos.uploadReleaseAsset({
+1
View File
@@ -40,6 +40,7 @@ CLAUDE.local.md
.claude/skills/*
!.claude/skills/verify
!.claude/skills/issue-fix
!.claude/skills/eval
# Local reference screenshots
pycharm-screenshots/
+1
View File
@@ -9,6 +9,7 @@
"dist",
"target",
"src/bindings.ts",
"evals/*/golden",
"src/editor/core/cm/angular/parser*.ts",
"src/editor/core/cm/jsp/parser*.ts",
"src/editor/core/cm/astro/parser*.ts",
+11 -3
View File
@@ -24,8 +24,8 @@ Visual style: dark, low-contrast greys, one cool accent
Plugin docs: `plugins/README.md`, `plugins/themes.md`.
- `java/` — JDT LS extension bundles (`java-usages`, `kotlin-bridge`) and the patched
Lombok agent, embedded into the app by `ide-sdk` / `ide-tools`.
- `e2e/` — Playwright UI tests (`pnpm e2e`). `fixtures/` — sample projects per language
for manual and automated testing.
- `e2e/` — Playwright UI tests (`pnpm e2e`). `evals/` — code-intelligence evals (`pnpm eval`).
`fixtures/` — sample projects per language for manual and automated testing.
- `docs/` — the Mintlify documentation site (`docs.json` navigation, `en/` and `zh/` pages);
preview with `npx mint dev` in `docs/`.
@@ -59,9 +59,12 @@ Unclear product decisions → `needs-decision` and stop.
## Skills
Do not use Impeccable or its related skills in this repository, including globally installed copies.
Shared agent skills live in `.agents/skills/` (`.claude/skills/` links to them):
`verify` drives the running app through Tauri MCP and captures screenshot evidence;
`issue-fix` takes an issue from reproduction to one PR.
`issue-fix` takes an issue from reproduction to one PR;
`eval` takes a failing code-intelligence eval to a fix with before / after numbers.
## Commands
@@ -81,6 +84,11 @@ Shared agent skills live in `.agents/skills/` (`.claude/skills/` links to them):
`pnpm gen:bindings`.
- `pnpm perf` (UI interaction latency / frames / leaks), `pnpm perf:rust` (criterion
benches vs budgets). Budgets: `perf/budgets.json`.
- `pnpm eval [--only <id>] [--runs n] [--save-baseline]` — Java code intelligence (navigation,
usages, gutter markers) in the production workbench against a release `openidea-server` and
jdtls: time to feedback / first result / complete, results vs `evals/<language>/golden/`.
Report: `target/eval/report.json`. Goldens and `evals/budgets.json` are changed by people only.
Tool versions: `.github/tool-versions.env` (shared with Language smoke).
- `pnpm release:prepare <version>` — the release commit's changes: app version everywhere,
`Cargo.lock`, and the plugin API snapshot when `wit/` moved to a new version.
Generated
+164 -121
View File
@@ -1522,7 +1522,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d045de693cb712d0b22c6a64be5b953f67b3ce00ab5ad3dd5d8b441886ab8e1a"
dependencies = [
"quote",
"syn 2.0.119",
"syn 3.0.9",
]
[[package]]
@@ -1578,16 +1578,6 @@ dependencies = [
"syn 2.0.119",
]
[[package]]
name = "darling"
version = "0.13.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a01d95850c592940db9b8194bc39f4bc0e89dee5c4265e4b1807c34a9aba453c"
dependencies = [
"darling_core 0.13.4",
"darling_macro 0.13.4",
]
[[package]]
name = "darling"
version = "0.23.0"
@@ -1608,20 +1598,6 @@ dependencies = [
"darling_macro 0.24.1",
]
[[package]]
name = "darling_core"
version = "0.13.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "859d65a907b6852c9361e3185c862aae7fafd2887876799fa55f5f99dc40d610"
dependencies = [
"fnv",
"ident_case",
"proc-macro2",
"quote",
"strsim 0.10.0",
"syn 1.0.109",
]
[[package]]
name = "darling_core"
version = "0.23.0"
@@ -1631,7 +1607,7 @@ dependencies = [
"ident_case",
"proc-macro2",
"quote",
"strsim 0.11.1",
"strsim",
"syn 2.0.119",
]
@@ -1644,21 +1620,10 @@ dependencies = [
"ident_case",
"proc-macro2",
"quote",
"strsim 0.11.1",
"strsim",
"syn 3.0.9",
]
[[package]]
name = "darling_macro"
version = "0.13.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9c972679f83bdf9c42bd905396b6c3588a843a17f0f16dfcfa3e2c5d57441835"
dependencies = [
"darling_core 0.13.4",
"quote",
"syn 1.0.109",
]
[[package]]
name = "darling_macro"
version = "0.23.0"
@@ -2685,7 +2650,7 @@ checksum = "ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0"
dependencies = [
"cfg-if",
"libc",
"wasi",
"wasi 0.11.1+wasi-snapshot-preview1",
]
[[package]]
@@ -4180,7 +4145,7 @@ checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
[[package]]
name = "ide-agent"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"dunce",
"encoding_rs",
@@ -4204,7 +4169,7 @@ dependencies = [
[[package]]
name = "ide-analysis"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"ast-grep-core",
"ide-core",
@@ -4218,7 +4183,7 @@ dependencies = [
[[package]]
name = "ide-android"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"ide-core",
"serde",
@@ -4229,7 +4194,7 @@ dependencies = [
[[package]]
name = "ide-build"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"blake3",
"chardetng",
@@ -4247,7 +4212,7 @@ dependencies = [
[[package]]
name = "ide-cli"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"dirs 6.0.0",
"serde",
@@ -4261,7 +4226,7 @@ dependencies = [
[[package]]
name = "ide-clipboard"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"gtk",
"objc2 0.6.5",
@@ -4271,7 +4236,7 @@ dependencies = [
[[package]]
name = "ide-core"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"dunce",
"encoding_rs",
@@ -4288,7 +4253,7 @@ dependencies = [
[[package]]
name = "ide-coverage"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"ide-core",
"ide-run",
@@ -4302,15 +4267,20 @@ dependencies = [
[[package]]
name = "ide-dap"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"futures",
"ide-core",
"ide-heap",
"ide-jdwp",
"ide-jsonrpc",
"ide-jvm-attach",
"ide-jvm-inspect",
"ide-language",
"libc",
"memchr",
"parking_lot",
"percent-encoding",
"portable-pty",
"serde",
"serde_json",
@@ -4323,7 +4293,7 @@ dependencies = [
[[package]]
name = "ide-db"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"bb8",
"blake3",
@@ -4333,7 +4303,7 @@ dependencies = [
"ide-core",
"ide-http",
"ide-ssh",
"oracle",
"oracledb",
"parking_lot",
"reqwest",
"rustls",
@@ -4353,7 +4323,7 @@ dependencies = [
[[package]]
name = "ide-docker"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"ide-core",
"serde",
@@ -4366,10 +4336,11 @@ dependencies = [
[[package]]
name = "ide-extension"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"base64 0.22.1",
"blake3",
"dunce",
"ed25519-dalek",
"ide-core",
"parking_lot",
@@ -4386,7 +4357,7 @@ dependencies = [
[[package]]
name = "ide-fonts"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"objc2-core-foundation",
"objc2-core-text",
@@ -4398,7 +4369,7 @@ dependencies = [
[[package]]
name = "ide-github"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"ide-core",
"ide-vcs",
@@ -4409,9 +4380,27 @@ dependencies = [
"which",
]
[[package]]
name = "ide-heap"
version = "0.1.42"
dependencies = [
"criterion",
"ide-core",
"libc",
"memmap2",
"proptest",
"serde",
"serde_json",
"specta",
"tempfile",
"thiserror 2.0.21",
"tracing",
"uuid",
]
[[package]]
name = "ide-host"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"base64 0.22.1",
"blake3",
@@ -4431,9 +4420,11 @@ dependencies = [
"ide-db",
"ide-docker",
"ide-extension",
"ide-heap",
"ide-http",
"ide-index",
"ide-jsonrpc",
"ide-jvm-attach",
"ide-language",
"ide-lsp",
"ide-mcp",
@@ -4441,6 +4432,7 @@ dependencies = [
"ide-plugin",
"ide-remote",
"ide-run",
"ide-scaffold",
"ide-sdk",
"ide-search",
"ide-spell",
@@ -4477,9 +4469,10 @@ dependencies = [
[[package]]
name = "ide-http"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"ide-core",
"ide-vfs",
"parking_lot",
"reqwest",
"rustls",
@@ -4495,7 +4488,7 @@ dependencies = [
[[package]]
name = "ide-index"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"blake3",
"criterion",
@@ -4515,9 +4508,21 @@ dependencies = [
"tracing",
]
[[package]]
name = "ide-jdwp"
version = "0.1.42"
dependencies = [
"ide-jvm-inspect",
"proptest",
"tempfile",
"thiserror 2.0.21",
"tokio",
"tracing",
]
[[package]]
name = "ide-jsonrpc"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"encoding_rs",
"futures",
@@ -4532,9 +4537,29 @@ dependencies = [
"tracing",
]
[[package]]
name = "ide-jvm-attach"
version = "0.1.42"
dependencies = [
"libc",
"tempfile",
"thiserror 2.0.21",
]
[[package]]
name = "ide-jvm-inspect"
version = "0.1.42"
dependencies = [
"futures",
"serde",
"serde_json",
"specta",
"thiserror 2.0.21",
]
[[package]]
name = "ide-language"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"blake3",
"criterion",
@@ -4590,13 +4615,14 @@ dependencies = [
[[package]]
name = "ide-lsp"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"dunce",
"futures",
"globset",
"ide-core",
"ide-jsonrpc",
"ide-jvm-inspect",
"ide-language",
"ide-text",
"ide-vfs",
@@ -4616,7 +4642,7 @@ dependencies = [
[[package]]
name = "ide-mcp"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"bytes",
"dunce",
@@ -4635,7 +4661,7 @@ dependencies = [
[[package]]
name = "ide-notebook"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"futures",
"ide-core",
@@ -4658,7 +4684,7 @@ dependencies = [
[[package]]
name = "ide-plugin"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"blake3",
"dunce",
@@ -4681,9 +4707,10 @@ dependencies = [
[[package]]
name = "ide-profiler"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"ide-core",
"ide-jvm-attach",
"ide-run",
"serde",
"serde_json",
@@ -4694,7 +4721,7 @@ dependencies = [
[[package]]
name = "ide-remote"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"dunce",
"globset",
@@ -4713,7 +4740,7 @@ dependencies = [
[[package]]
name = "ide-run"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"dunce",
"encoding_rs",
@@ -4735,9 +4762,25 @@ dependencies = [
"zip",
]
[[package]]
name = "ide-scaffold"
version = "0.1.42"
dependencies = [
"ide-core",
"ide-tools",
"ide-vcs",
"insta",
"serde",
"serde_json",
"specta",
"tempfile",
"toml 0.9.12+spec-1.1.0",
"tracing",
]
[[package]]
name = "ide-sdk"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"dunce",
"ide-core",
@@ -4757,9 +4800,10 @@ dependencies = [
[[package]]
name = "ide-search"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"criterion",
"encoding_rs",
"globset",
"grep-matcher",
"grep-regex",
@@ -4777,7 +4821,7 @@ dependencies = [
[[package]]
name = "ide-spell"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"ide-core",
"parking_lot",
@@ -4790,7 +4834,7 @@ dependencies = [
[[package]]
name = "ide-ssh"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"dirs 6.0.0",
"flate2",
@@ -4811,7 +4855,7 @@ dependencies = [
[[package]]
name = "ide-syntax"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"ide-core",
"ide-language",
@@ -4826,7 +4870,7 @@ dependencies = [
[[package]]
name = "ide-terminal"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"ide-core",
"libc",
@@ -4839,7 +4883,7 @@ dependencies = [
[[package]]
name = "ide-text"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"criterion",
"ide-core",
@@ -4852,7 +4896,7 @@ dependencies = [
[[package]]
name = "ide-tools"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"flate2",
"hex",
@@ -4873,7 +4917,7 @@ dependencies = [
[[package]]
name = "ide-vcs"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"blake3",
"dunce",
@@ -4894,7 +4938,7 @@ dependencies = [
[[package]]
name = "ide-vfs"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"chardetng",
"dunce",
@@ -4918,11 +4962,11 @@ dependencies = [
[[package]]
name = "ide-web"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"dirs 6.0.0",
"ide-core",
"open",
"parking_lot",
"percent-encoding",
"serde",
"specta",
@@ -4934,7 +4978,7 @@ dependencies = [
[[package]]
name = "ide-workspace"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"blake3",
"dunce",
@@ -5751,7 +5795,7 @@ checksum = "1788edb87fdc09c7e26304471e2f5be8cdefb1b6930d6e3985fc02ff53bf86ee"
dependencies = [
"libc",
"log",
"wasi",
"wasi 0.11.1+wasi-snapshot-preview1",
"windows-sys 0.61.2",
]
@@ -6487,15 +6531,6 @@ dependencies = [
"memchr",
]
[[package]]
name = "odpic-sys"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "920b5474a5128a9f0232df5a0ffc50aaa5b077b29b8b06ab0131985ac82793ed"
dependencies = [
"cc",
]
[[package]]
name = "once_cell"
version = "1.21.4"
@@ -6521,7 +6556,7 @@ dependencies = [
[[package]]
name = "openidea"
version = "0.1.40"
version = "0.1.42"
dependencies = [
"base64 0.22.1",
"blake3",
@@ -6555,6 +6590,7 @@ dependencies = [
"ide-profiler",
"ide-remote",
"ide-run",
"ide-scaffold",
"ide-sdk",
"ide-search",
"ide-spell",
@@ -6609,29 +6645,24 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "04744f49eae99ab78e0d5c0b603ab218f515ea8cfe5a456d7629ad883a3b6e7d"
[[package]]
name = "oracle"
version = "0.6.3"
name = "oracledb"
version = "26.0.0-beta.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3db40fe6e4df881b683691ade5ef1f7b1afd52aefa115581f7b92855524d7ec0"
checksum = "27b98c7346f2bb5169ec4b2d4f066fb616529ac766b395440c5019fe88d1ef45"
dependencies = [
"cc",
"odpic-sys",
"once_cell",
"oracle_procmacro",
"paste",
"rustversion",
]
[[package]]
name = "oracle_procmacro"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ad247f3421d57de56a0d0408d3249d4b1048a522be2013656d92f022c3d8af27"
dependencies = [
"darling 0.13.4",
"proc-macro2",
"quote",
"syn 1.0.109",
"aes",
"base16ct",
"base64ct",
"cbc",
"chrono",
"pbkdf2",
"pkcs8",
"rand 0.10.3",
"rustls",
"sha2 0.11.1",
"uuid",
"webpki-roots",
"whoami",
]
[[package]]
@@ -7766,9 +7797,9 @@ dependencies = [
[[package]]
name = "rustix"
version = "1.1.5"
version = "1.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "891efababe418670775f199f0d233d84843c227a0949a883ce15b37c78d6629d"
checksum = "e5e76742e5294153d4dde1207dbec5bdd81eef6fabdc0d2118babadabfd858aa"
dependencies = [
"bitflags 2.13.2",
"errno",
@@ -8918,12 +8949,6 @@ dependencies = [
"unicode-properties",
]
[[package]]
name = "strsim"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "73473c0e59e6d5812c5dfe2a064a6444949f089e20eec9a2e5506596494e4623"
[[package]]
name = "strsim"
version = "0.11.1"
@@ -8960,7 +8985,6 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72b64191b275b66ffe2469e8af2c1cfe3bafa67b529ead792a6d0160888b4237"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
@@ -9183,7 +9207,7 @@ dependencies = [
"serde",
"serde_json",
"static_vcruntime",
"strsim 0.11.1",
"strsim",
"tauri-utils",
"tauri-winres",
"walkdir",
@@ -10723,6 +10747,15 @@ version = "0.11.1+wasi-snapshot-preview1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b"
[[package]]
name = "wasi"
version = "0.14.7+wasi-0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "883478de20367e224c0090af9cf5f9fa85bed63a95c1abf3afc5c083ebc06e8c"
dependencies = [
"wasip2",
]
[[package]]
name = "wasip2"
version = "1.0.4+wasi-0.2.12"
@@ -10732,6 +10765,15 @@ dependencies = [
"wit-bindgen",
]
[[package]]
name = "wasite"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "66fe902b4a6b8028a753d5424909b764ccf79b7a209eac9bf97e59cda9f71a42"
dependencies = [
"wasi 0.14.7+wasi-0.2.4",
]
[[package]]
name = "wasm-bindgen"
version = "0.2.129"
@@ -11315,6 +11357,7 @@ dependencies = [
"libc",
"libredox",
"objc2-system-configuration",
"wasite",
"web-sys",
]
+9 -2
View File
@@ -5,7 +5,7 @@ members = ["crates/*", "src-tauri"]
exclude = ["sdk", "plugins"]
[workspace.package]
version = "0.1.40"
version = "0.1.42"
edition = "2024"
rust-version = "1.96"
license = "Apache-2.0"
@@ -43,10 +43,15 @@ ide-ssh = { path = "crates/ide-ssh" }
ide-remote = { path = "crates/ide-remote" }
ide-host = { path = "crates/ide-host" }
ide-mcp = { path = "crates/ide-mcp" }
ide-scaffold = { path = "crates/ide-scaffold" }
ide-notebook = { path = "crates/ide-notebook" }
ide-extension = { path = "crates/ide-extension" }
ide-plugin = { path = "crates/ide-plugin" }
ide-cli = { path = "crates/ide-cli" }
ide-jvm-attach = { path = "crates/ide-jvm-attach" }
ide-jvm-inspect = { path = "crates/ide-jvm-inspect" }
ide-jdwp = { path = "crates/ide-jdwp" }
ide-heap = { path = "crates/ide-heap" }
# serialization & IPC typing
serde = { version = "1", features = ["derive"] }
@@ -82,6 +87,7 @@ trash = "5"
blake3 = "1"
dunce = "1"
dirs = "6"
memmap2 = "0.9"
zip = { version = "9.2.3", default-features = false, features = ["deflate"] }
# search, index, language
@@ -106,7 +112,8 @@ rustls-platform-verifier = "0.7"
# Until SQLx releases its libsqlite3-sys 0.39 update (RUSTSEC-2026-0459).
sqlx = { version = "0.9", git = "https://github.com/transact-rs/sqlx", rev = "10462fa4414c6c7f1ed1d39bb31678e3b726878c", default-features = false, features = ["runtime-tokio", "tls-rustls-ring-webpki", "sqlite", "postgres", "mysql"] }
tiberius = { version = "0.13", default-features = false, features = ["tds73", "rustls", "sql-browser-tokio"] }
oracle = { version = "0.6", default-features = false }
# Pure-Rust thin Oracle driver; pinned to the beta until it stabilizes.
oracledb = "=26.0.0-beta.5"
bb8 = "0.9"
uuid = { version = "1", features = ["v4"] }
# OS credential stores
+13
View File
@@ -19,6 +19,8 @@
<a href="#plugins">Plugins</a>
&nbsp;·&nbsp;
<a href="#community">Community</a>
&nbsp;·&nbsp;
<a href="#sponsor">Sponsor</a>
</p>
https://github.com/user-attachments/assets/234dd54c-1255-4b3a-b35c-533d78d737c0
@@ -68,6 +70,17 @@ the [official plugins](plugins/official). The plugin API is still a preview.
- Contributing: see [CONTRIBUTING.md](CONTRIBUTING.md) and the [contributor docs](https://openidea.sh/en/contribute/setup).
## Sponsor
If OpenIDEA saves you time, you can support its development with Alipay or WeChat Pay.
<table>
<tr>
<td align="center"><img src=".github/assets/sponsor-alipay.png" alt="Alipay QR code" width="200"><br>Alipay</td>
<td align="center"><img src=".github/assets/sponsor-wechat.png" alt="WeChat Pay QR code" width="200"><br>WeChat Pay</td>
</tr>
</table>
## License
[Apache 2.0](LICENSE). Bundled third-party assets are listed in [`THIRD_PARTY_LICENSES.md`](THIRD_PARTY_LICENSES.md).
+26
View File
@@ -0,0 +1,26 @@
# Security policy
## Reporting a vulnerability
Please do not report security problems in public issues, pull requests or discussions.
Report them privately through GitHub:
[Report a vulnerability](https://github.com/l0ng-ai/openidea/security/advisories/new)
(repository **Security** tab → **Report a vulnerability**). Only the maintainers can see
the report.
Include what you can of:
- the affected version, platform, and whether remote development (`openidea-server`)
or plugins are involved;
- steps to reproduce, or a proof of concept;
- the impact you expect, and any fix you suggest.
We will acknowledge the report, keep you updated in the advisory, and agree on a
disclosure date with you. Fixes are released as a new version, and the advisory is
published once users can update.
## Supported versions
Security fixes go into the latest release. Please check that the problem still occurs
there before reporting.
+6
View File
@@ -1,5 +1,11 @@
# Third-party licenses
## Project type icons
`src/ui/brands/*.svg` use the unmodified logo paths from
[Simple Icons](https://github.com/simple-icons/simple-icons), retrieved through
Iconify (`simple-icons:angular`, `simple-icons:astro`, `simple-icons:cplusplus`, `simple-icons:dart`, `simple-icons:django`, `simple-icons:dotnet`, `simple-icons:elixir`, `simple-icons:express`, `simple-icons:fastapi`, `simple-icons:flask`, `simple-icons:flutter`, `simple-icons:go`, `simple-icons:haskell`, `simple-icons:html5`, `simple-icons:kotlin`, `simple-icons:ktor`, `simple-icons:laravel`, `simple-icons:nestjs`, `simple-icons:nextdotjs`, `simple-icons:nodedotjs`, `simple-icons:nuxt`, `simple-icons:openjdk`, `simple-icons:phoenixframework`, `simple-icons:php`, `simple-icons:python`, `simple-icons:quarkus`, `simple-icons:ruby`, `simple-icons:rubyonrails`, `simple-icons:rust`, `simple-icons:scala`, `simple-icons:springboot`, `simple-icons:svelte`, `simple-icons:swift`, `simple-icons:symfony`, `simple-icons:vite`, `simple-icons:zig`) and filled with the brand colors (the standard icon grey for black logos). Licensed under [CC0 1.0 Universal](https://github.com/simple-icons/simple-icons/blob/develop/LICENSE.md).
## Project Lombok
`java/lombok/lombok-1.18.48-openidea.3.jar` contains Project Lombok v1.18.48 with the Equinox
+16
View File
@@ -164,6 +164,11 @@ impl Negotiated {
self.capabilities.pointer(&format!("/promptCapabilities/{name}")).and_then(Value::as_bool) == Some(true)
}
/// `mcpCapabilities.<name>` (`http`, `sse`).
pub fn mcp(&self, name: &str) -> bool {
self.capabilities.pointer(&format!("/mcpCapabilities/{name}")).and_then(Value::as_bool) == Some(true)
}
/// `sessionCapabilities.<name>` (`fork`, `resume`, `close`, `list`, `delete`).
pub fn session(&self, name: &str) -> bool {
self.capabilities.pointer(&format!("/sessionCapabilities/{name}")).is_some_and(|v| !v.is_null())
@@ -173,6 +178,17 @@ impl Negotiated {
pub fn meta(&self, pointer: &str) -> Option<&Value> {
self.capabilities.get("_meta")?.pointer(pointer)
}
/// First authentication method the agent can run over ACP (`type` absent or `"agent"`).
pub fn agent_auth_method(&self) -> Option<&str> {
self.auth_methods.iter().find_map(|m| {
let kind = m.get("type").and_then(Value::as_str).unwrap_or("agent");
(kind == "agent" && m.pointer("/_meta/terminal-auth").is_none())
.then(|| m.get("id").and_then(Value::as_str))
.flatten()
.filter(|id| !id.is_empty())
})
}
}
/// The client side of one agent connection: wraps the task's [`Handler`] and, once
+30
View File
@@ -135,6 +135,36 @@ fn the_shape_decides_the_version_not_the_number() {
assert_eq!((n.generation, n.load_session(), n.session("fork")), (2, false, false));
}
#[test]
fn picks_non_terminal_agent_auth_method() {
let v1 = Negotiated::from_initialize(&json!({
"protocolVersion": 1,
"authMethods": [
{ "id": "cli_login", "_meta": { "terminal-auth": { "command": "copilot", "args": ["login"] } } },
{ "id": "terminal_login", "type": "terminal", "args": ["login"] },
{ "id": "api_key", "type": "env_var" },
{ "id": "github_oauth", "name": "GitHub OAuth" }
]
}));
assert_eq!(v1.agent_auth_method(), Some("github_oauth"));
let v2 = Negotiated::from_initialize(&json!({
"protocolVersion": 2,
"info": { "name": "x", "version": "1" },
"authMethods": [
{ "type": "terminal", "methodId": "cli", "name": "CLI", "args": ["login"] },
{ "type": "agent", "methodId": "oauth", "name": "OAuth" }
]
}));
assert_eq!(v2.agent_auth_method(), Some("oauth"));
let terminal_only = Negotiated::from_initialize(&json!({
"protocolVersion": 1,
"authMethods": [{ "id": "cli", "type": "terminal", "args": ["login"] }]
}));
assert_eq!(terminal_only.agent_auth_method(), None);
}
#[test]
fn meta_and_auth_methods_pass_through() {
// The native shims advertise their extensions in `agentCapabilities._meta`.
+325
View File
@@ -0,0 +1,325 @@
//! Sessions the user started outside OpenIDEA (a terminal, another editor, the CLI):
//! scanned so they can be listed and imported as tasks instead of re-explained (#312).
//!
//! Only the native providers keep transcripts on disk in a readable form: Claude Code
//! (`<config>/projects/*/<id>.jsonl`, see `native::claude::history`) and Codex
//! (`<home>/.codex/sessions/**`, rollout files with a `session_meta` line).
use super::native::claude::history;
use serde::Serialize;
use specta::Type;
use std::io::BufRead;
use std::path::{Path, PathBuf};
/// A session on this machine that no task holds yet.
#[derive(Clone, Debug, Serialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct ExternalSession {
/// The [`crate::AgentProvider`] id that owns it (`claude`, `codex`).
pub provider: String,
/// The id resuming it names (`--resume`, `thread/resume`).
pub session_id: String,
pub title: String,
/// Where it ran, when the transcript names one.
pub cwd: Option<String>,
/// File modification time, ms since the epoch.
pub updated_ms: f64,
}
/// Modification time of `path` in ms, else 0.
fn mtime(path: &Path) -> f64 {
path.metadata()
.and_then(|m| m.modified())
.ok()
.and_then(|t| t.duration_since(std::time::UNIX_EPOCH).ok())
.map(|d| d.as_millis() as f64)
.unwrap_or(0.0)
}
/// Short display text: collapsed whitespace, truncated to `MAX_TITLE` chars.
fn short(text: &str) -> String {
const MAX_TITLE: usize = 80;
let flat: String = text.split_whitespace().collect::<Vec<_>>().join(" ");
if flat.chars().count() <= MAX_TITLE {
flat
} else {
format!("{}…", flat.chars().take(MAX_TITLE - 1).collect::<String>())
}
}
/// The text of a Claude Code transcript entry's message, when it carries prose.
fn message_text(e: &serde_json::Value) -> Option<String> {
match e.pointer("/message/content") {
Some(serde_json::Value::String(s)) => Some(s.clone()),
Some(serde_json::Value::Array(blocks)) => {
let text: Vec<&str> = blocks
.iter()
.filter(|b| b.get("type").and_then(serde_json::Value::as_str) == Some("text"))
.filter_map(|b| b.get("text").and_then(serde_json::Value::as_str))
.collect();
(!text.is_empty()).then(|| text.join("\n"))
}
_ => None,
}
}
/// Whether a session that ran in `cwd` belongs to the project at `root` (in it or below it).
fn in_project(cwd: Option<&str>, root: &Path) -> bool {
cwd.is_some_and(|cwd| Path::new(cwd).starts_with(root))
}
/// Claude Code transcripts of the project at `root`: `<config>/projects/<escaped cwd>/<id>.jsonl`.
/// Only the folders named after `root` or a folder below it are read, so other projects'
/// transcripts (often gigabytes) are never opened.
pub fn claude_sessions(config_dir: &Path, root: &Path) -> Vec<ExternalSession> {
let mut out = Vec::new();
let projects = config_dir.join("projects");
let escaped = history::escape(root);
let dirs: Vec<PathBuf> = std::fs::read_dir(&projects)
.into_iter()
.flatten()
.flatten()
.filter(|d| {
let name = d.file_name().to_string_lossy().into_owned();
name == escaped || name.starts_with(&format!("{escaped}-"))
})
.map(|d| d.path())
.collect();
for dir in dirs {
let files: Vec<PathBuf> = std::fs::read_dir(&dir)
.into_iter()
.flatten()
.flatten()
.map(|d| d.path())
.filter(|p| p.extension().is_some_and(|e| e == "jsonl"))
.collect();
for path in files {
let id = path.file_stem().and_then(|s| s.to_str()).unwrap_or_default().to_owned();
if id.is_empty() {
continue;
}
let entries = history::read(&path).unwrap_or_default();
if entries.is_empty() {
continue;
}
let title = history::title(&entries)
.or_else(|| entries.iter().filter(|e| history::is_turn_start(e)).filter_map(message_text).next())
.map(|t| short(&t))
.unwrap_or_else(|| format!("Claude Code session {}", &id[..id.len().min(8)]));
let cwd = entries
.iter()
.filter_map(|e| e.get("cwd").and_then(serde_json::Value::as_str))
.next()
.map(str::to_owned);
// Escaping is lossy (`a/b-c` and `a/b/c` share a folder): the transcript decides.
if !in_project(cwd.as_deref(), root) {
continue;
}
out.push(ExternalSession {
provider: "claude".to_owned(),
session_id: id,
title,
cwd,
updated_ms: mtime(&path),
});
}
}
out
}
/// The `session_meta` line of a Codex rollout file: `(id, cwd)`, across shapes.
fn codex_meta(line: &serde_json::Value) -> Option<(String, Option<String>)> {
let payload =
line.get("payload").filter(|p| p.get("type").and_then(serde_json::Value::as_str) == Some("session_meta"));
let inner = payload.and_then(|p| p.get("payload")).or_else(|| {
line.get("type").and_then(serde_json::Value::as_str).filter(|t| *t == "session_meta").and(Some(line))
})?;
let id = inner.get("id").and_then(serde_json::Value::as_str)?.to_owned();
let cwd = inner.get("cwd").and_then(serde_json::Value::as_str).map(str::to_owned);
Some((id, cwd))
}
/// A prose line of a Codex rollout file (a user message), across shapes.
fn codex_user_text(line: &serde_json::Value) -> Option<String> {
let payload = line.get("payload")?;
let ty = payload.get("type").and_then(serde_json::Value::as_str)?;
if !(ty == "user_message" || ty == "userMessage" || ty == "event_msg") {
return None;
}
let inner = payload.get("payload").unwrap_or(&serde_json::Value::Null);
for v in [payload, inner] {
if let Some(s) = v.get("message").and_then(serde_json::Value::as_str) {
return Some(s.to_owned());
}
if let Some(s) = v.get("text").and_then(serde_json::Value::as_str) {
return Some(s.to_owned());
}
}
None
}
/// One rollout file's session, or `None` when it names no `session_meta`.
fn codex_file(path: &Path) -> Option<ExternalSession> {
let file = std::fs::File::open(path).ok()?;
let mut id: Option<(String, Option<String>)> = None;
let mut title: Option<String> = None;
// `session_meta` is first, but cap the scan: giant rollouts must not stall listing.
for line in std::io::BufReader::new(file).lines().take(400) {
let Ok(line) = line else { break };
let Ok(v) = serde_json::from_str::<serde_json::Value>(&line) else { continue };
if id.is_none()
&& let Some(meta) = codex_meta(&v)
{
id = Some(meta);
}
if title.is_none()
&& let Some(t) = codex_user_text(&v)
{
title = Some(short(&t));
}
if id.is_some() && title.is_some() {
break;
}
}
let (session_id, cwd) = id?;
Some(ExternalSession {
provider: "codex".to_owned(),
title: title.unwrap_or_else(|| format!("Codex session {}", &session_id[..session_id.len().min(8)])),
session_id,
cwd,
updated_ms: mtime(path),
})
}
/// Collects `*.jsonl` below `dir`, up to 4 levels (`sessions/YYYY/MM/DD/<rollout>`).
fn rollout_files(dir: &Path, depth: usize, out: &mut Vec<PathBuf>) {
if depth == 0 {
return;
}
for entry in std::fs::read_dir(dir).into_iter().flatten().flatten() {
let path = entry.path();
if path.is_dir() {
rollout_files(&path, depth - 1, out);
} else if path.extension().is_some_and(|e| e == "jsonl") {
out.push(path);
}
}
}
/// Codex sessions of the project at `root`: `<home>/.codex/sessions/**`.
pub fn codex_sessions(codex_home: &Path, root: &Path) -> Vec<ExternalSession> {
let mut files = Vec::new();
rollout_files(&codex_home.join("sessions"), 4, &mut files);
files.iter().filter_map(|p| codex_file(p)).filter(|s| in_project(s.cwd.as_deref(), root)).collect()
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
use std::io::Write;
fn write(path: &Path, lines: &[serde_json::Value]) {
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
let mut f = std::fs::File::create(path).unwrap();
for line in lines {
writeln!(f, "{line}").unwrap();
}
}
fn claude_user(uuid: &str, text: &str) -> serde_json::Value {
json!({"type": "user", "uuid": uuid, "parentUuid": null,
"message": {"role": "user", "content": text}, "cwd": "/work/app"})
}
#[test]
fn lists_claude_transcripts_with_titles() {
let tmp = tempfile::tempdir().unwrap();
write(
&tmp.path().join("projects/-work-app/abc123.jsonl"),
&[
claude_user("u1", "fix the login bug"),
json!({"type": "assistant", "uuid": "a1", "parentUuid": "u1",
"message": {"role": "assistant", "content": []}}),
json!({"type": "ai-title", "aiTitle": "Login fix"}),
],
);
write(&tmp.path().join("projects/-work-app/empty.jsonl"), &[]);
let sessions = claude_sessions(tmp.path(), Path::new("/work/app"));
assert_eq!(sessions.len(), 1);
let s = &sessions[0];
assert_eq!((s.provider.as_str(), s.session_id.as_str()), ("claude", "abc123"));
assert_eq!(s.title, "Login fix");
assert_eq!(s.cwd.as_deref(), Some("/work/app"));
assert!(s.updated_ms > 0.0);
}
#[test]
fn claude_title_falls_back_to_the_first_prompt() {
let tmp = tempfile::tempdir().unwrap();
write(&tmp.path().join("projects/-work-app/def456.jsonl"), &[claude_user("u1", "hello there")]);
let sessions = claude_sessions(tmp.path(), Path::new("/work/app"));
assert_eq!(sessions.len(), 1);
assert_eq!(sessions[0].title, "hello there");
}
#[test]
fn lists_codex_rollouts_with_meta() {
let tmp = tempfile::tempdir().unwrap();
write(
&tmp.path().join("sessions/2026/01/02/rollout-01.jsonl"),
&[
json!({"timestamp": "t", "payload": {"type": "session_meta",
"payload": {"id": "thread-1", "cwd": "/work/app"}}}),
json!({"timestamp": "t", "payload": {"type": "user_message",
"payload": {"message": "add tests"}}}),
],
);
write(&tmp.path().join("sessions/notes.jsonl"), &[json!({"nope": true})]);
let sessions = codex_sessions(tmp.path(), Path::new("/work"));
assert_eq!(sessions.len(), 1);
let s = &sessions[0];
assert_eq!((s.provider.as_str(), s.session_id.as_str()), ("codex", "thread-1"));
assert_eq!(s.title, "add tests");
assert_eq!(s.cwd.as_deref(), Some("/work/app"));
}
#[test]
fn missing_directories_list_nothing() {
let tmp = tempfile::tempdir().unwrap();
assert!(claude_sessions(&tmp.path().join("nope"), Path::new("/work")).is_empty());
assert!(codex_sessions(&tmp.path().join("nope"), Path::new("/work")).is_empty());
}
#[test]
fn only_sessions_of_the_project_are_listed() {
let tmp = tempfile::tempdir().unwrap();
let user = |cwd: &str| {
json!({"type": "user", "uuid": "u1", "parentUuid": null,
"message": {"role": "user", "content": "hi"}, "cwd": cwd})
};
write(&tmp.path().join("projects/-work-app/in-root.jsonl"), &[user("/work/app")]);
write(&tmp.path().join("projects/-work-app-web/below.jsonl"), &[user("/work/app/web")]);
// `/work/app-old` escapes like a folder below `/work/app`; its transcript says otherwise.
write(&tmp.path().join("projects/-work-app-old/sibling.jsonl"), &[user("/work/app-old")]);
write(&tmp.path().join("projects/-other/elsewhere.jsonl"), &[user("/other")]);
let mut ids: Vec<String> =
claude_sessions(tmp.path(), Path::new("/work/app")).into_iter().map(|s| s.session_id).collect();
ids.sort();
assert_eq!(ids, ["below", "in-root"]);
let rollout = |name: &str, cwd: &str| {
write(
&tmp.path().join(format!("codex/sessions/2026/01/02/{name}.jsonl")),
&[json!({"payload": {"type": "session_meta", "payload": {"id": name, "cwd": cwd}}})],
)
};
rollout("mine", "/work/app/api");
rollout("theirs", "/other");
let codex: Vec<String> = codex_sessions(&tmp.path().join("codex"), Path::new("/work/app"))
.into_iter()
.map(|s| s.session_id)
.collect();
assert_eq!(codex, ["mine"]);
}
}
+288 -26
View File
@@ -10,6 +10,8 @@
mod acp;
mod config;
mod conversation;
mod external;
pub mod mcp;
mod model;
mod native;
mod provider;
@@ -18,6 +20,7 @@ mod skills;
mod timeline;
mod worktree;
pub use external::ExternalSession;
pub use model::*;
use conversation::{Conversation, SavedSubagent, Session};
@@ -91,6 +94,9 @@ type Connector = Arc<
+ Sync,
>;
/// Returns the configured MCP servers (User and Workspace) for this project.
pub type McpServersFn = dyn Fn() -> Vec<McpServer> + Send + Sync;
struct Shared {
paths: AgentPaths,
runtime: tokio::runtime::Handle,
@@ -104,6 +110,7 @@ struct Shared {
command_fetches: Mutex<HashMap<String, tokio::sync::watch::Receiver<bool>>>,
/// [`IDLE_RELEASE`], in milliseconds (shorter in tests).
idle_release_ms: AtomicU64,
mcp_servers_hook: RwLock<Option<Arc<McpServersFn>>>,
}
/// What an agent offered in its last session, for the New Task page.
@@ -131,6 +138,13 @@ impl Shared {
}
}
fn mcp_servers_for_session(&self, provider: &str, init: &acp::Negotiated) -> Vec<Value> {
let mut servers = mcp_servers_for(init.mcp_stdio(), &self.paths.project_root);
let configured = self.mcp_servers_hook.read().as_ref().map(|hook| hook()).unwrap_or_default();
servers.extend(mcp::acp_mcp_servers(&configured, provider, init));
servers
}
fn known_config(&self, provider: &str) -> Vec<AgentConfigOption> {
let mut config = self.known.lock().get(provider).map(|k| k.config.clone()).unwrap_or_default();
// Model-specific options are offered separately, never as provider-wide defaults.
@@ -163,16 +177,25 @@ impl Shared {
let shared = self.clone();
// Keep the lookup alive if a composer unmounts while it is waiting.
self.runtime.spawn(async move {
if let Ok((commands, options)) =
if let Ok(found) =
native::claude::discover(spec.clone(), shared.paths.project_root.clone()).await
{
let commands = config::commands_of(&json!({ "availableCommands": commands }), &spec.id);
let options = config::options_of(&json!({ "configOptions": options }));
let commands =
config::commands_of(&json!({ "availableCommands": found.commands }), &spec.id);
let options = config::options_of(&json!({ "configOptions": found.options }));
let levels: Vec<_> = found
.efforts
.into_iter()
.map(|(model, o)| {
let options = o.map(|o| config::options_of(&json!({ "configOptions": [o] })));
(model, options.unwrap_or_default())
})
.collect();
let _ = tokio::task::spawn_blocking(move || {
// Notifies subscribers itself when the discovery
// changed the cached config; unchanged results
// stay silent.
shared.remember_discovered(&spec.id, &options, &commands);
shared.remember_discovered(&spec.id, &options, &commands, &levels);
})
.await;
}
@@ -193,6 +216,18 @@ impl Shared {
self.known.lock().get(provider).is_some_and(|k| !k.config.is_empty())
}
/// Whether the thinking levels of every model the agent offers are known (#1686).
fn knows_all_levels(&self, provider: &str) -> bool {
let cache = self.known.lock();
let Some(known) = cache.get(provider) else { return false };
known
.config
.iter()
.filter(|o| o.category.as_deref() == Some("model"))
.flat_map(|o| &o.values)
.all(|v| known.thought_levels.contains_key(&v.value))
}
fn root_key(&self) -> String {
self.paths.project_root.to_string_lossy().into_owned()
}
@@ -205,11 +240,19 @@ impl Shared {
commands: Option<&[AgentCommand]>,
model: Option<&str>,
) {
self.remember_inner(provider, config, commands, model, false);
self.remember_inner(provider, config, commands, model, &[], false);
}
fn remember_discovered(&self, provider: &str, config: &[AgentConfigOption], commands: &[AgentCommand]) {
self.remember_inner(provider, Some(config), Some(commands), model_of(config).as_deref(), true);
/// `levels`: each model's thinking-level options, so a new task offers them for any model
/// it picks (#1686). A session's report for a model wins over these.
fn remember_discovered(
&self,
provider: &str,
config: &[AgentConfigOption],
commands: &[AgentCommand],
levels: &[(String, Vec<AgentConfigOption>)],
) {
self.remember_inner(provider, Some(config), Some(commands), model_of(config).as_deref(), levels, true);
}
fn remember_inner(
@@ -218,12 +261,19 @@ impl Shared {
config: Option<&[AgentConfigOption]>,
commands: Option<&[AgentCommand]>,
model: Option<&str>,
discovered_levels: &[(String, Vec<AgentConfigOption>)],
discovered: bool,
) {
// Keep the update and its disk write ordered; a session report wins over discovery.
let mut cache = self.known.lock();
let known = cache.entry(provider.to_owned()).or_default();
let mut changed = false;
for (model, levels) in discovered_levels {
if !known.thought_levels.contains_key(model) {
known.thought_levels.insert(model.clone(), levels.clone());
changed = true;
}
}
if let Some(c) = config.filter(|_| !discovered || known.config.is_empty()) {
if let Some(model) = model {
let levels: Vec<_> =
@@ -277,6 +327,9 @@ struct TaskFile {
/// In the project's own checkout: files already changed when the task started.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
before: BTreeMap<String, worktree::Stamp>,
/// The folder (in the task store) with copies of the `before` files, for rolling back.
#[serde(default, skip_serializing_if = "Option::is_none")]
saved: Option<String>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
subagents: Vec<SavedSubagent>,
/// Turns to drop from the session once it is loaded (a fork of an earlier point).
@@ -293,6 +346,7 @@ struct TaskState {
conversation: Conversation,
wanted: Vec<AgentConfigChoice>,
before: Arc<BTreeMap<String, worktree::Stamp>>,
saved: Option<String>,
pending_rollback: u32,
/// The user named the task; see `TaskFile::title_renamed`.
title_renamed: bool,
@@ -315,7 +369,7 @@ struct Live {
embedded: bool,
/// `session/load`: the process can be stopped between turns and the session reloaded.
resumable: bool,
mcp_stdio: bool,
init: acp::Negotiated,
}
struct Task {
@@ -385,6 +439,7 @@ impl AgentManager {
probes: Mutex::new(HashMap::new()),
command_fetches: Mutex::new(HashMap::new()),
idle_release_ms: AtomicU64::new(IDLE_RELEASE.as_millis() as u64),
mcp_servers_hook: RwLock::new(None),
});
let mut tasks = HashMap::new();
for file in std::fs::read_dir(&shared.paths.store_dir).into_iter().flatten().flatten() {
@@ -425,8 +480,11 @@ impl AgentManager {
/// The agents (custom ones re-read), whether each is installed, and what they offered last time.
pub fn providers(&self) -> Vec<AgentProvider> {
let specs = provider::load(self.shared.paths.custom_agents_dir.as_deref());
let in_use: std::collections::HashSet<String> =
self.tasks.read().values().map(|t| t.state.lock().task.provider.clone()).collect();
let list = specs
.iter()
.filter(|p| !provider::is_unlisted(p) || in_use.contains(&p.id))
.map(|p| {
let mut info = provider::describe(p);
info.config = self.shared.known_config(&p.id);
@@ -472,6 +530,11 @@ impl AgentManager {
self.shared.emit(AgentEvent::Providers);
}
/// Provides the user and workspace MCP servers configured in settings.
pub fn set_mcp_servers_hook(&self, hook: Box<McpServersFn>) {
*self.shared.mcp_servers_hook.write() = Some(Arc::from(hook));
}
/// The slash commands to complete for `provider`, in `task`'s directory or (without one)
/// in the project: what the agent offered last time there, unadvertised builtins,
/// and, for Claude Code, skills and custom commands on disk it has not listed.
@@ -488,7 +551,7 @@ impl AgentManager {
// merged without duplicating what `availableCommands` or the disk offer.
let mut offered = offered;
if task.is_none()
&& (offered.is_empty() || !self.shared.has_config(&provider))
&& (offered.is_empty() || !self.shared.has_config(&provider) || !self.shared.knows_all_levels(&provider))
&& let Some(spec) = self.shared.spec(&provider).filter(|s| s.driver == provider::Driver::Claude)
{
self.shared.fetch_settings(spec).await;
@@ -524,6 +587,93 @@ impl AgentManager {
Ok(timeline.entries.clone())
}
/// Session ids the tasks already hold (native drivers reuse the CLI's own id).
fn imported_session_ids(&self) -> HashSet<String> {
self.tasks.read().values().filter_map(|t| t.state.lock().session_id.clone()).collect()
}
/// Sessions of this project on this machine that no task holds yet, newest first. Reads
/// transcripts from disk: call it off the UI and command threads.
pub fn external_sessions(&self) -> Vec<ExternalSession> {
let mut out = Vec::new();
let root = self.shared.paths.project_root.clone();
if let Some(spec) = self.shared.spec("claude")
&& let Some(dir) = native::claude::cli::config_dir(&spec.env)
{
out.extend(external::claude_sessions(&dir, &root));
}
let home = self.shared.paths.home_dir.clone().or_else(home_dir);
if let Some(home) = home {
out.extend(external::codex_sessions(&home.join(".codex"), &root));
}
let known = self.imported_session_ids();
out.retain(|s| !known.contains(&s.session_id));
out.sort_by(|a, b| b.updated_ms.total_cmp(&a.updated_ms));
out.truncate(50);
out
}
/// Imports an external session as a task: its first turn loads and replays it (#312).
pub fn import_session(&self, provider: &str, session_id: &str) -> IdeResult<AgentTask> {
if !matches!(provider, "claude" | "codex") {
return Err(IdeError::invalid(format!("cannot import {provider} sessions")));
}
let spec = self.shared.spec(provider).ok_or_else(|| IdeError::invalid(format!("unknown agent {provider}")))?;
if self.imported_session_ids().contains(session_id) {
return Err(IdeError::invalid("that session is already a task"));
}
let found = self
.external_sessions()
.into_iter()
.find(|s| s.provider == provider && s.session_id == session_id)
.ok_or_else(|| IdeError::not_found(format!("unknown {provider} session {session_id}")))?;
let root = self.shared.paths.project_root.clone();
let cwd = found.cwd.map(PathBuf::from).filter(|p| p.is_absolute()).unwrap_or_else(|| root.clone());
let config = self.shared.known_config(&spec.id);
let now = now_ms();
let id = uuid::Uuid::new_v4().simple().to_string();
let task = AgentTask {
project_root: Some(IpcPath(root)),
id,
title: found.title,
provider: spec.id.clone(),
model: model_of(&config),
status: AgentStatus::Ready,
isolation: Isolation::Local,
cwd: IpcPath(cwd),
branch: None,
base: None,
base_commit: None,
error: None,
created_ms: now,
updated_ms: now,
config,
commands: self.shared.known_commands(&spec.id),
usage: None,
queue: Vec::new(),
capabilities: self.shared.probes.lock().get(&spec.id).map(|p| p.capabilities).unwrap_or_default(),
resets_ms: None,
retry: None,
background: Box::default(),
};
let file = TaskFile {
task: task.clone(),
session_id: Some(session_id.to_owned()),
entries: Vec::new(),
wanted: Vec::new(),
before: BTreeMap::new(),
saved: None,
subagents: Vec::new(),
pending_rollback: 0,
title_renamed: true,
};
let handle = Task::restore(file, self.shared.clone());
self.tasks.write().insert(task.id.clone(), handle.clone());
handle.emit_task();
handle.persist();
Ok(task)
}
/// Creates the task (and its worktree) and sends `req.prompt` as its first message.
pub async fn create(&self, req: NewAgentTask) -> IdeResult<AgentTask> {
let spec = self
@@ -576,9 +726,24 @@ impl AgentManager {
}));
}
let now = now_ms();
let id = uuid::Uuid::new_v4().simple().to_string();
// Copies of the files the user had changed already, for rolling back the task's changes to them.
let saved = if before.is_empty() {
None
} else {
let name = format!("{id}.before");
let (dir, at, files) = (self.shared.paths.store_dir.join(&name), cwd.clone(), before.clone());
match blocking(move || worktree::save_before(&at, &files, &dir)).await {
Ok(saved) => saved.then_some(name),
Err(e) => {
tracing::warn!("saving the changed files of agent task {id}: {e}");
None
}
}
};
let task = AgentTask {
project_root,
id: uuid::Uuid::new_v4().simple().to_string(),
id,
title,
provider: spec.id.clone(),
model: model_of(&config),
@@ -606,6 +771,7 @@ impl AgentManager {
entries: Vec::new(),
wanted: req.config,
before,
saved,
subagents: Vec::new(),
pending_rollback: 0,
title_renamed: false,
@@ -773,10 +939,16 @@ impl AgentManager {
return Err(IdeError::invalid("this agent can't copy a conversation from there"));
}
let cwd = source.state.lock().task.cwd.0.clone();
let provider_id = source.state.lock().task.provider.clone();
let mcp_servers = source.shared.mcp_servers_for_session(&provider_id, &live.init);
let cwd_s = dunce::canonicalize(&cwd).unwrap_or(cwd).to_string_lossy().into_owned();
let reply: Value = live
.peer
.request("session/fork", json!({ "sessionId": live.session, "cwd": cwd_s, "mcpServers": mcp_servers_for(live.mcp_stdio, &source.shared.paths.project_root) }), None)
.request(
"session/fork",
json!({ "sessionId": live.session, "cwd": cwd_s, "mcpServers": mcp_servers }),
None,
)
.await
.map_err(|e| IdeError::internal(source.describe(e)))?;
let session = reply
@@ -810,6 +982,7 @@ impl AgentManager {
entries,
wanted: s.wanted.clone(),
before: (*s.before).clone(),
saved: s.saved.clone(),
subagents: s.conversation.saved_subagents(),
pending_rollback: total - keep,
title_renamed: s.title_renamed,
@@ -1061,6 +1234,17 @@ impl AgentManager {
};
// A copy of the task (a fork) may work in the same worktree.
let shared = self.tasks.read().values().any(|t| t.id != id && t.state.lock().task.cwd.0 == cwd);
self.tasks.write().remove(id);
*task.deleted.lock() = true;
let _ = std::fs::remove_file(self.shared.paths.store_dir.join(format!("{id}.json")));
let saved = task.state.lock().saved.clone();
if let Some(saved) = saved
&& !self.tasks.read().values().any(|t| t.state.lock().saved.as_ref() == Some(&saved))
{
let _ = std::fs::remove_dir_all(self.shared.paths.store_dir.join(saved));
}
// The list drops the task now; removing a large worktree's folder can take a while.
self.shared.emit(AgentEvent::Removed { id: id.to_owned() });
if remove_worktree && !shared && isolation == Isolation::Worktree && cwd.exists() {
// On Windows the agent's processes lock the folder (their working directory,
// open files) until they have exited.
@@ -1070,10 +1254,6 @@ impl AgentManager {
let worktrees = self.shared.paths.worktrees_dir.clone();
blocking(move || worktree::remove(&root, &worktrees, &cwd)).await?;
}
self.tasks.write().remove(id);
*task.deleted.lock() = true;
let _ = std::fs::remove_file(self.shared.paths.store_dir.join(format!("{id}.json")));
self.shared.emit(AgentEvent::Removed { id: id.to_owned() });
Ok(())
}
@@ -1091,6 +1271,41 @@ impl AgentManager {
blocking(move || worktree::file_at(&cwd, &path, &base)).await
}
/// Rollback: puts the task's changed files among `rel_paths` (all with `None`) back as they
/// were when it started (files the user had changed then, as the user left them).
pub async fn rollback_changes(&self, id: &str, rel_paths: Option<Vec<String>>) -> IdeResult<()> {
let task = self.task(id)?;
if task.state.lock().busy {
return Err(IdeError::invalid("the agent is working: stop it before rolling back its changes"));
}
let (cwd, base) = task.cwd_and_base();
let (before, saved) = {
let s = task.state.lock();
(s.before.clone(), s.saved.as_ref().map(|name| self.shared.paths.store_dir.join(name)))
};
let restamped = blocking(move || {
let changes = worktree::without_untouched(worktree::changes(&cwd, &base)?, &before);
let mut restamped = Vec::new();
for change in changes.iter().filter(|c| rel_paths.as_ref().is_none_or(|p| p.contains(&c.rel_path))) {
worktree::restore(&cwd, &base, change, &before, saved.as_deref())?;
// Back to the user's own edits: not a change of the task any more.
if before.contains_key(&change.rel_path) {
restamped.push((change.rel_path.clone(), worktree::stamp(&change.path.0)));
}
}
Ok(restamped)
})
.await?;
if !restamped.is_empty() {
let mut s = task.state.lock();
let mut before = (*s.before).clone();
before.extend(restamped);
s.before = Arc::new(before);
}
task.persist();
Ok(())
}
/// Stops every agent (project closed).
pub fn shutdown(&self) {
for task in self.tasks.read().values() {
@@ -1116,6 +1331,7 @@ impl Task {
conversation,
wanted: file.wanted,
before: Arc::new(file.before),
saved: file.saved,
pending_rollback: file.pending_rollback,
title_renamed: file.title_renamed,
busy: false,
@@ -1228,6 +1444,7 @@ impl Task {
entries: s.conversation.main.entries.clone(),
wanted: s.wanted.clone(),
before: (*s.before).clone(),
saved: s.saved.clone(),
subagents: s.conversation.saved_subagents(),
pending_rollback: s.pending_rollback,
title_renamed: s.title_renamed,
@@ -1496,6 +1713,30 @@ impl Task {
})
}
/// Runs `method` and, on an auth error, tries the agent's own ACP `authenticate` once before retrying.
async fn request_with_auth(
&self,
peer: &acp::Conn,
init: &acp::Negotiated,
tried_auth: &mut bool,
method: &str,
params: Value,
) -> Result<Value, RpcError> {
let first = peer.request(method, params.clone(), None).await;
if !*tried_auth
&& let Err(RpcError::Server { code, ref message, .. }) = first
&& acp::auth_required(code, message)
&& let Some(method_id) = init.agent_auth_method()
{
*tried_auth = true;
match peer.request::<_, Value>("authenticate", json!({ "methodId": method_id }), None).await {
Ok(_) => return peer.request(method, params, None).await,
Err(e) => tracing::debug!(task = %self.id, method_id, error = %e, "agent authenticate failed"),
}
}
first
}
async fn ensure_session(self: &Arc<Self>) -> Result<Live, String> {
let _guard = self.connecting.lock().await;
if let Some(live) = self.live.lock().clone()
@@ -1533,16 +1774,17 @@ impl Task {
let init = peer.initialize(&client_info()).await.map_err(|e| self.describe(e))?;
let sign_in_message = spec.sign_in_message(&init.auth_methods);
let can_load = init.load_session();
let mcp_stdio = init.mcp_stdio();
let (capabilities, embedded) = capabilities_of(&init);
let mcp_servers = self.shared.mcp_servers_for_session(&provider_id, &init);
// Agents that compare canonical paths fail in a symlinked directory (macOS `/tmp`).
let cwd_s = dunce::canonicalize(&cwd).unwrap_or(cwd).to_string_lossy().into_owned();
let mut tried_auth = false;
let mut session = None;
if let (Some(id), true) = (saved, can_load) {
self.replaying.store(true, Ordering::Release);
let loaded: Result<Value, RpcError> =
peer.request("session/load", json!({ "sessionId": id, "cwd": cwd_s, "mcpServers": mcp_servers_for(mcp_stdio, &self.shared.paths.project_root) }), None).await;
let params = json!({ "sessionId": &id, "cwd": &cwd_s, "mcpServers": &mcp_servers });
let loaded = self.request_with_auth(&peer, &init, &mut tried_auth, "session/load", params).await;
tokio::time::sleep(REPLAY_GRACE).await;
self.replaying.store(false, Ordering::Release);
match loaded {
@@ -1554,12 +1796,9 @@ impl Task {
let (session_id, result) = match session {
Some(s) => s,
None => {
let v: Value = peer
.request(
"session/new",
json!({ "cwd": cwd_s, "mcpServers": mcp_servers_for(mcp_stdio, &self.shared.paths.project_root) }),
None,
)
let params = json!({ "cwd": &cwd_s, "mcpServers": &mcp_servers });
let v = self
.request_with_auth(&peer, &init, &mut tried_auth, "session/new", params)
.await
.map_err(|e| self.sign_in_error(&e, Some(&sign_in_message)).unwrap_or_else(|| self.describe(e)))?;
let id = v.get("sessionId").and_then(Value::as_str).ok_or("the agent returned no session")?.to_owned();
@@ -1651,7 +1890,7 @@ impl Task {
self.persist();
self.emit_task();
let live =
Live { sign_in_message, peer, session: session_id, capabilities, embedded, resumable: can_load, mcp_stdio };
Live { sign_in_message, peer, session: session_id, capabilities, embedded, resumable: can_load, init };
*self.live.lock() = Some(live.clone());
Ok(live)
}
@@ -1722,7 +1961,7 @@ impl Task {
format!("The agent process exited:\n{}", tail.join("\n"))
}
}
RpcError::Server { message, .. } => message,
RpcError::Server { message, data, .. } => with_details(message, data.as_ref()),
e => e.to_string(),
}
}
@@ -2180,6 +2419,24 @@ fn usage_limit(e: &RpcError) -> Option<Option<f64>> {
.then_some(None)
}
/// An error's message with the detail ACP agents put in `data` (`{"details": …}` or a string):
/// generic codes such as "Internal error" say nothing without it.
fn with_details(message: String, data: Option<&Value>) -> String {
let details = match data {
Some(Value::String(text)) => text.as_str(),
Some(data) => data.get("details").and_then(Value::as_str).unwrap_or_default(),
None => "",
}
.trim();
if details.is_empty() || message.contains(details) {
message
} else if message.trim().is_empty() {
details.to_owned()
} else {
format!("{message}: {details}")
}
}
fn client_info() -> Value {
json!({ "name": "OpenIDEA", "version": env!("CARGO_PKG_VERSION") })
}
@@ -2287,6 +2544,11 @@ fn now_ms() -> f64 {
std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map_or(0.0, |d| d.as_millis() as f64)
}
/// The user's home, for tools that keep state there (Codex sessions).
fn home_dir() -> Option<PathBuf> {
std::env::var_os("HOME").or_else(|| std::env::var_os("USERPROFILE")).map(PathBuf::from)
}
fn write_file(path: &Path, bytes: &[u8]) -> Result<(), String> {
if let Some(dir) = path.parent() {
std::fs::create_dir_all(dir).map_err(|e| e.to_string())?;
+507
View File
@@ -0,0 +1,507 @@
//! Model Context Protocol (MCP) server configurations stored in OpenIDEA's own settings
//! (global `settings.toml` and project `.openidea/settings.toml`), passed to ACP agents in
//! `session/new`, `session/load`, and `session/fork`, plus one-way import from `.mcp.json` /
//! `.vscode/mcp.json`.
use crate::acp::Negotiated;
use crate::{AgentMcpServer, McpKeyValue, McpScope, McpServer, McpServerInput, McpTransport};
use serde_json::{Value, json};
use std::collections::{BTreeMap, HashSet};
use std::path::Path;
/// Stable ID for a server in a given scope (`"user::<name>"` or `"workspace::<name>"`).
pub fn server_id(scope: McpScope, name: &str) -> String {
let prefix = match scope {
McpScope::User => "user",
McpScope::Workspace => "workspace",
};
format!("{prefix}::{name}")
}
/// Parses a server ID (`"user::<name>"` or `"workspace::<name>"`).
pub fn parse_server_id(id: &str) -> Option<(McpScope, &str)> {
let (prefix, name) = id.split_once("::")?;
if name.is_empty() {
return None;
}
match prefix {
"user" => Some((McpScope::User, name)),
"workspace" => Some((McpScope::Workspace, name)),
_ => None,
}
}
/// Converts the User (`settings.toml`) and Workspace (`.openidea/settings.toml`) MCP server maps
/// into the ordered list displayed in Settings › Agents › MCP Servers.
pub fn servers_from_settings(
user: &BTreeMap<String, AgentMcpServer>,
workspace: &BTreeMap<String, AgentMcpServer>,
) -> Vec<McpServer> {
user.iter()
.map(|(name, cfg)| to_mcp_server(McpScope::User, name, cfg))
.chain(workspace.iter().map(|(name, cfg)| to_mcp_server(McpScope::Workspace, name, cfg)))
.collect()
}
fn to_mcp_server(scope: McpScope, name: &str, cfg: &AgentMcpServer) -> McpServer {
McpServer {
id: server_id(scope, name),
name: name.to_owned(),
scope,
transport: cfg.transport,
enabled: cfg.enabled,
command: cfg.command.clone(),
args: cfg.args.clone(),
env: cfg.env.iter().map(|(k, v)| McpKeyValue { name: k.clone(), value: v.clone() }).collect(),
url: cfg.url.clone(),
headers: cfg.headers.iter().map(|(k, v)| McpKeyValue { name: k.clone(), value: v.clone() }).collect(),
agents: cfg.agents.clone(),
}
}
fn normalize_provider(id: &str) -> &str {
id.strip_suffix("-acp").unwrap_or(id)
}
fn matches_agent(agents: &[String], provider: &str) -> bool {
if agents.is_empty() {
return true;
}
let target = normalize_provider(provider);
agents.iter().any(|a| {
let trimmed = a.trim();
trimmed == provider || normalize_provider(trimmed) == target
})
}
/// Builds the ACP `mcpServers` array for `session/new`, `session/load`, and `session/fork`.
/// Workspace entries take precedence over User entries with the same name.
pub(crate) fn acp_mcp_servers(all: &[McpServer], provider: &str, init: &Negotiated) -> Vec<Value> {
let supports_stdio = init.mcp_stdio();
let supports_http = init.mcp("http");
let supports_sse = init.mcp("sse");
let mut seen = HashSet::new();
let mut servers = Vec::new();
let ordered =
all.iter().filter(|s| s.scope == McpScope::Workspace).chain(all.iter().filter(|s| s.scope == McpScope::User));
for srv in ordered {
if !srv.enabled || !matches_agent(&srv.agents, provider) || !seen.insert(srv.name.clone()) {
continue;
}
match srv.transport {
McpTransport::Stdio if supports_stdio => {
let Some(cmd) = srv.command.as_deref().filter(|c| !c.trim().is_empty()) else {
continue;
};
let env: Vec<Value> = srv.env.iter().map(|kv| json!({ "name": kv.name, "value": kv.value })).collect();
servers.push(json!({
"name": srv.name,
"command": cmd,
"args": srv.args,
"env": env,
}));
}
McpTransport::Http if supports_http => {
let Some(url) = srv.url.as_deref().filter(|u| !u.trim().is_empty()) else {
continue;
};
let headers: Vec<Value> =
srv.headers.iter().map(|kv| json!({ "name": kv.name, "value": kv.value })).collect();
servers.push(json!({
"type": "http",
"name": srv.name,
"url": url,
"headers": headers,
}));
}
McpTransport::Sse if supports_sse => {
let Some(url) = srv.url.as_deref().filter(|u| !u.trim().is_empty()) else {
continue;
};
let headers: Vec<Value> =
srv.headers.iter().map(|kv| json!({ "name": kv.name, "value": kv.value })).collect();
servers.push(json!({
"type": "sse",
"name": srv.name,
"url": url,
"headers": headers,
}));
}
_ => {}
}
}
servers
}
/// Applies a create or update operation to the User / Workspace MCP server maps.
pub fn apply_save(
user: &mut BTreeMap<String, AgentMcpServer>,
workspace: &mut BTreeMap<String, AgentMcpServer>,
has_workspace: bool,
input: McpServerInput,
) -> Result<(), String> {
let name = input.name.trim();
if name.is_empty() {
return Err("Server name is required".into());
}
if input.scope == McpScope::Workspace && !has_workspace {
return Err("Open a project to configure workspace MCP servers".into());
}
let cfg = match input.transport {
McpTransport::Stdio => {
let cmd = input.command.as_deref().map(str::trim).unwrap_or_default();
if cmd.is_empty() {
return Err("Command is required for stdio MCP servers".into());
}
let env = input
.env
.into_iter()
.filter_map(|kv| {
let k = kv.name.trim();
(!k.is_empty()).then(|| (k.to_owned(), kv.value))
})
.collect();
AgentMcpServer {
transport: McpTransport::Stdio,
enabled: input.enabled,
command: Some(cmd.to_owned()),
args: input.args,
env,
url: None,
headers: BTreeMap::new(),
agents: normalize_agents_list(input.agents),
}
}
McpTransport::Http | McpTransport::Sse => {
let url = input.url.as_deref().map(str::trim).unwrap_or_default();
if url.is_empty() {
return Err("URL is required for HTTP/SSE MCP servers".into());
}
let headers = input
.headers
.into_iter()
.filter_map(|kv| {
let k = kv.name.trim();
(!k.is_empty()).then(|| (k.to_owned(), kv.value))
})
.collect();
AgentMcpServer {
transport: input.transport,
enabled: input.enabled,
command: None,
args: Vec::new(),
env: BTreeMap::new(),
url: Some(url.to_owned()),
headers,
agents: normalize_agents_list(input.agents),
}
}
};
if let Some(prev_id) = input.id.as_deref()
&& let Some((prev_scope, prev_name)) = parse_server_id(prev_id)
&& (prev_scope != input.scope || prev_name != name)
{
match prev_scope {
McpScope::User => {
user.remove(prev_name);
}
McpScope::Workspace => {
workspace.remove(prev_name);
}
}
}
match input.scope {
McpScope::User => {
user.insert(name.to_owned(), cfg);
}
McpScope::Workspace => {
workspace.insert(name.to_owned(), cfg);
}
}
Ok(())
}
/// Enables or disables a User or Workspace MCP server by ID.
pub fn apply_set_enabled(
user: &mut BTreeMap<String, AgentMcpServer>,
workspace: &mut BTreeMap<String, AgentMcpServer>,
id: &str,
enabled: bool,
) -> Result<McpScope, String> {
let (scope, name) = parse_server_id(id).ok_or_else(|| format!("MCP server `{id}` not found"))?;
let map = match scope {
McpScope::User => user,
McpScope::Workspace => workspace,
};
let srv = map.get_mut(name).ok_or_else(|| format!("MCP server `{name}` not found"))?;
srv.enabled = enabled;
Ok(scope)
}
/// Removes a User or Workspace MCP server by ID.
pub fn apply_remove(
user: &mut BTreeMap<String, AgentMcpServer>,
workspace: &mut BTreeMap<String, AgentMcpServer>,
id: &str,
) -> Result<McpScope, String> {
let (scope, name) = parse_server_id(id).ok_or_else(|| format!("MCP server `{id}` not found"))?;
let map = match scope {
McpScope::User => user,
McpScope::Workspace => workspace,
};
if map.remove(name).is_none() {
return Err(format!("MCP server `{name}` not found"));
}
Ok(scope)
}
/// Imports MCP servers from `<workspace_root>/.mcp.json` and `<workspace_root>/.vscode/mcp.json`
/// into `workspace` without modifying the external files. Returns the number of newly added servers.
pub fn apply_import(workspace: &mut BTreeMap<String, AgentMcpServer>, workspace_root: &Path) -> usize {
let mut added = 0;
for rel in [".mcp.json", ".vscode/mcp.json"] {
let path = workspace_root.join(rel);
for (name, srv) in read_external_file(&path) {
if let std::collections::btree_map::Entry::Vacant(e) = workspace.entry(name) {
e.insert(srv);
added += 1;
}
}
}
added
}
fn normalize_agents_list(agents: Vec<String>) -> Vec<String> {
let mut out = Vec::new();
for a in agents {
let trimmed = a.trim();
if !trimmed.is_empty() && !out.iter().any(|existing: &String| existing == trimmed) {
out.push(trimmed.to_owned());
}
}
out
}
fn read_external_file(path: &Path) -> BTreeMap<String, AgentMcpServer> {
let Ok(bytes) = std::fs::read(path) else {
return BTreeMap::new();
};
let Ok(root) = serde_json::from_slice::<Value>(&bytes) else {
return BTreeMap::new();
};
let Some(servers_obj) = root.get("mcpServers").or_else(|| root.get("servers")).and_then(Value::as_object) else {
return BTreeMap::new();
};
let mut out = BTreeMap::new();
for (name, def) in servers_obj {
let name = name.trim();
let Some(obj) = def.as_object().filter(|_| !name.is_empty()) else {
continue;
};
let disabled = obj.get("disabled").and_then(Value::as_bool).unwrap_or(false)
|| obj.get("enabled").and_then(Value::as_bool) == Some(false);
let type_str = obj.get("type").and_then(Value::as_str).unwrap_or_default().to_ascii_lowercase();
let url = obj
.get("url")
.or_else(|| obj.get("serverUrl"))
.and_then(Value::as_str)
.map(str::trim)
.filter(|s| !s.is_empty())
.map(str::to_owned);
let command =
obj.get("command").and_then(Value::as_str).map(str::trim).filter(|s| !s.is_empty()).map(str::to_owned);
let transport = match type_str.as_str() {
"sse" => McpTransport::Sse,
"http" | "streamable-http" | "streamable_http" => McpTransport::Http,
"stdio" | "local" => McpTransport::Stdio,
_ if url.is_some() && command.is_none() => McpTransport::Http,
_ => McpTransport::Stdio,
};
match transport {
McpTransport::Stdio if command.is_none() => continue,
McpTransport::Http | McpTransport::Sse if url.is_none() => continue,
_ => {}
}
let args = obj
.get("args")
.and_then(Value::as_array)
.map(|arr| arr.iter().filter_map(|v| v.as_str().map(str::to_owned)).collect())
.unwrap_or_default();
out.insert(
name.to_owned(),
AgentMcpServer {
transport,
enabled: !disabled,
command,
args,
env: parse_key_values_map(obj.get("env")),
url,
headers: parse_key_values_map(obj.get("headers")),
agents: Vec::new(),
},
);
}
out
}
fn parse_key_values_map(val: Option<&Value>) -> BTreeMap<String, String> {
match val {
Some(Value::Object(map)) => map
.iter()
.filter_map(|(k, v)| {
let s = match v {
Value::String(s) => s.clone(),
Value::Number(n) => n.to_string(),
Value::Bool(b) => b.to_string(),
_ => return None,
};
Some((k.clone(), s))
})
.collect(),
Some(Value::Array(arr)) => arr
.iter()
.filter_map(|item| {
let obj = item.as_object()?;
let name = obj.get("name")?.as_str()?.to_owned();
let value = obj.get("value")?.as_str()?.to_owned();
Some((name, value))
})
.collect(),
_ => BTreeMap::new(),
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn crud_and_acp_filtering_by_scope_agent_and_transport() {
let mut user = BTreeMap::new();
let mut workspace = BTreeMap::new();
apply_save(
&mut user,
&mut workspace,
true,
McpServerInput {
id: None,
name: "shared".into(),
scope: McpScope::User,
transport: McpTransport::Stdio,
enabled: true,
command: Some("npx".into()),
args: vec!["-y".into(), "user-pkg".into()],
env: vec![McpKeyValue { name: "TOKEN".into(), value: "u1".into() }],
url: None,
headers: Vec::new(),
agents: Vec::new(),
},
)
.unwrap();
apply_save(
&mut user,
&mut workspace,
true,
McpServerInput {
id: None,
name: "shared".into(),
scope: McpScope::Workspace,
transport: McpTransport::Stdio,
enabled: true,
command: Some("npx".into()),
args: vec!["-y".into(), "workspace-pkg".into()],
env: Vec::new(),
url: None,
headers: Vec::new(),
agents: Vec::new(),
},
)
.unwrap();
apply_save(
&mut user,
&mut workspace,
true,
McpServerInput {
id: None,
name: "claude-http".into(),
scope: McpScope::User,
transport: McpTransport::Http,
enabled: true,
command: None,
args: Vec::new(),
env: Vec::new(),
url: Some("https://example.com/mcp".into()),
headers: vec![McpKeyValue { name: "Authorization".into(), value: "Bearer secret".into() }],
agents: vec!["claude".into()],
},
)
.unwrap();
let list = servers_from_settings(&user, &workspace);
assert_eq!(list.len(), 3);
// Negotiated with HTTP support: `claude` gets workspace `shared` + `claude-http`; `codex` only gets `shared`.
let init_http = Negotiated::from_initialize(&json!({
"agentCapabilities": { "mcpCapabilities": { "http": true, "sse": false } }
}));
let claude_servers = acp_mcp_servers(&list, "claude-acp", &init_http);
assert_eq!(claude_servers.len(), 2);
assert_eq!(claude_servers[0]["name"], "shared");
assert_eq!(claude_servers[0]["args"], json!(["-y", "workspace-pkg"]));
assert_eq!(claude_servers[1]["name"], "claude-http");
assert_eq!(claude_servers[1]["type"], "http");
let codex_servers = acp_mcp_servers(&list, "codex", &init_http);
assert_eq!(codex_servers.len(), 1);
assert_eq!(codex_servers[0]["name"], "shared");
// Disable workspace `shared` and remove user `claude-http`.
apply_set_enabled(&mut user, &mut workspace, "workspace::shared", false).unwrap();
apply_remove(&mut user, &mut workspace, "user::claude-http").unwrap();
let list = servers_from_settings(&user, &workspace);
let after = acp_mcp_servers(&list, "claude", &init_http);
assert_eq!(after.len(), 1);
assert_eq!(after[0]["args"], json!(["-y", "user-pkg"]));
}
#[test]
fn imports_from_mcp_json_and_vscode_mcp_json_without_modifying_them() {
let tmp = tempfile::tempdir().unwrap();
let mcp_json = tmp.path().join(".mcp.json");
let vscode_dir = tmp.path().join(".vscode");
std::fs::create_dir_all(&vscode_dir).unwrap();
let vscode_mcp = vscode_dir.join("mcp.json");
let raw_mcp = r#"{"mcpServers":{"sqlite":{"command":"uvx","args":["mcp-server-sqlite"]}}}"#;
let raw_vscode = r#"{"servers":{"remote":{"type":"http","url":"https://mcp.example.com"}}}"#;
std::fs::write(&mcp_json, raw_mcp).unwrap();
std::fs::write(&vscode_mcp, raw_vscode).unwrap();
let mut workspace = BTreeMap::new();
let added = apply_import(&mut workspace, tmp.path());
assert_eq!(added, 2);
assert_eq!(workspace["sqlite"].command.as_deref(), Some("uvx"));
assert_eq!(workspace["remote"].transport, McpTransport::Http);
assert_eq!(workspace["remote"].url.as_deref(), Some("https://mcp.example.com"));
// Re-importing does not duplicate or overwrite existing entries, and leaves files untouched.
workspace.get_mut("sqlite").unwrap().enabled = false;
assert_eq!(apply_import(&mut workspace, tmp.path()), 0);
assert!(!workspace["sqlite"].enabled);
assert_eq!(std::fs::read_to_string(&mcp_json).unwrap(), raw_mcp);
assert_eq!(std::fs::read_to_string(&vscode_mcp).unwrap(), raw_vscode);
}
}
+67
View File
@@ -535,3 +535,70 @@ pub enum AgentEvent {
/// The installed or configured agent providers changed.
Providers,
}
pub use ide_core::settings::{AgentMcpServer, McpTransport};
/// Where an MCP server configuration is stored (`settings.toml` vs `.openidea/settings.toml`).
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum McpScope {
User,
Workspace,
}
/// A named environment variable or HTTP header for an MCP server.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct McpKeyValue {
pub name: String,
pub value: String,
}
/// One configured Model Context Protocol (MCP) server.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct McpServer {
pub id: String,
pub name: String,
pub scope: McpScope,
pub transport: McpTransport,
pub enabled: bool,
pub command: Option<String>,
pub args: Vec<String>,
pub env: Vec<McpKeyValue>,
pub url: Option<String>,
pub headers: Vec<McpKeyValue>,
/// Agent provider ids this server is enabled for; empty means enabled for all agents.
#[serde(default)]
pub agents: Vec<String>,
}
/// Payload for creating or updating a user or workspace MCP server.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct McpServerInput {
#[serde(default)]
pub id: Option<String>,
pub name: String,
pub scope: McpScope,
pub transport: McpTransport,
#[serde(default = "default_true")]
pub enabled: bool,
#[serde(default)]
pub command: Option<String>,
#[serde(default)]
pub args: Vec<String>,
#[serde(default)]
pub env: Vec<McpKeyValue>,
#[serde(default)]
pub url: Option<String>,
#[serde(default)]
pub headers: Vec<McpKeyValue>,
/// Agent provider ids this server is enabled for; empty means enabled for all agents.
#[serde(default)]
pub agents: Vec<String>,
}
fn default_true() -> bool {
true
}
+1 -1
View File
@@ -135,7 +135,7 @@ impl Cli for Process {
}
/// Claude Code's configuration directory (`CLAUDE_CONFIG_DIR`, else `~/.claude`).
pub(super) fn config_dir(env: &[(String, String)]) -> Option<PathBuf> {
pub(crate) fn config_dir(env: &[(String, String)]) -> Option<PathBuf> {
let set = env
.iter()
.find(|(k, _)| k == "CLAUDE_CONFIG_DIR")
@@ -31,11 +31,11 @@ pub(super) fn transcript(config_dir: &Path, cwd: &Path, id: &str) -> PathBuf {
}
/// The project directory name Claude Code uses for `cwd`.
pub(super) fn escape(cwd: &Path) -> String {
pub(crate) fn escape(cwd: &Path) -> String {
cwd.to_string_lossy().chars().map(|c| if c.is_ascii_alphanumeric() { c } else { '-' }).collect()
}
pub(super) fn read(path: &Path) -> std::io::Result<Vec<Value>> {
pub(crate) fn read(path: &Path) -> std::io::Result<Vec<Value>> {
let text = std::fs::read_to_string(path)?;
Ok(text.lines().filter_map(|l| serde_json::from_str(l).ok()).collect())
}
@@ -111,7 +111,7 @@ fn block_ids<'a>(e: &'a Value, ty: &'a str, key: &'a str) -> impl Iterator<Item
/// Whether `e` is a message that started a turn: the user's own, not a tool result, a
/// steer, or text Claude Code added.
pub(super) fn is_turn_start(e: &Value) -> bool {
pub(crate) fn is_turn_start(e: &Value) -> bool {
if kind(e) != "user" || e.get("isMeta").and_then(Value::as_bool) == Some(true) {
return false;
}
@@ -149,7 +149,7 @@ pub(super) fn cut(chain: &[&Value], turns: usize) -> Result<Option<String>, Stri
}
/// The newest title Claude Code gave the session.
pub(super) fn title(entries: &[Value]) -> Option<String> {
pub(crate) fn title(entries: &[Value]) -> Option<String> {
entries
.iter()
.rev()
+16 -4
View File
@@ -7,8 +7,8 @@
//! <id>.json`): loading that session, from this connection or a fresh one, starts it that
//! way.
mod cli;
mod history;
pub(crate) mod cli;
pub(crate) mod history;
mod settings;
mod tools;
mod translate;
@@ -69,9 +69,17 @@ pub(crate) fn connect(
Ok(super::serve(shim, client))
}
/// What discovery found: ACP `availableCommands`, `configOptions` for the configured model,
/// and each model's effort option by model value (`None`: none).
pub(crate) struct Discovered {
pub commands: Vec<Value>,
pub options: Value,
pub efforts: Vec<(String, Option<Value>)>,
}
/// Discover the slash commands and the model, mode and effort options without a prompt,
/// ACP session, or pending session file.
pub(crate) async fn discover(spec: ProviderSpec, cwd: PathBuf) -> Result<(Vec<Value>, Value), String> {
pub(crate) async fn discover(spec: ProviderSpec, cwd: PathBuf) -> Result<Discovered, String> {
tokio::time::timeout(Duration::from_secs(5), async move {
let spec = tokio::task::spawn_blocking(move || (spec.resolve_for_launch(&cwd), cwd));
let (spec, cwd) = spec.await.map_err(|e| e.to_string())?;
@@ -100,7 +108,11 @@ pub(crate) async fn discover(spec: ProviderSpec, cwd: PathBuf) -> Result<(Vec<Va
.flatten()
.unwrap_or_default();
let settings = Settings::from_cli(&init, &applied);
Ok((settings.commands.clone(), settings.options()))
Ok(Discovered {
commands: settings.commands.clone(),
options: settings.options(),
efforts: settings.efforts_by_model(),
})
})
.await
.map_err(|_| "Claude Code command discovery timed out".to_owned())?
+38 -9
View File
@@ -132,19 +132,18 @@ impl Settings {
}).collect::<Vec<_>>(),
}));
}
let efforts = self.current_model().map(|m| m.efforts.as_slice()).unwrap_or_default();
if !efforts.is_empty() {
let mut values = vec![json!({ "value": "default", "name": "Default" })];
values.extend(efforts.iter().map(|e| json!({ "value": e, "name": capitalize(e) })));
let current = if efforts.contains(&self.effort) { self.effort.as_str() } else { "default" };
out.push(json!({
"id": EFFORT, "name": "Effort", "category": "thought_level", "type": "select",
"currentValue": current, "options": values,
}));
if let Some(effort) = self.current_model().and_then(|m| effort_option(m, &self.effort)) {
out.push(effort);
}
Value::Array(out)
}
/// Each model's effort option by model value (`None`: it has no effort levels): what a new
/// task offers once that model is picked, before any session has run it.
pub fn efforts_by_model(&self) -> Vec<(String, Option<Value>)> {
self.models.iter().map(|m| (m.value.clone(), effort_option(m, &self.effort))).collect()
}
pub fn valid(&self, option: &str, value: &str) -> bool {
self.options()
.as_array()
@@ -156,6 +155,20 @@ impl Settings {
}
}
/// `model`'s effort option, with `effort` selected when the model offers it; `None` without efforts.
fn effort_option(model: &Model, effort: &str) -> Option<Value> {
if model.efforts.is_empty() {
return None;
}
let mut values = vec![json!({ "value": "default", "name": "Default" })];
values.extend(model.efforts.iter().map(|e| json!({ "value": e, "name": capitalize(e) })));
let current = if model.efforts.iter().any(|e| e == effort) { effort } else { "default" };
Some(json!({
"id": EFFORT, "name": "Effort", "category": "thought_level", "type": "select",
"currentValue": current, "options": values,
}))
}
/// Slash commands as ACP `availableCommands`, without those that need Claude Code's terminal.
pub(super) fn commands(list: Option<&Value>, terminal_only: &[String]) -> Vec<Value> {
list.and_then(Value::as_array)
@@ -231,4 +244,20 @@ mod tests {
let cmds = commands(init().get("commands"), &["doctor".into()]);
assert_eq!(cmds, vec![json!({"name": "review", "description": "Review", "input": {"hint": "[pr]"}})]);
}
#[test]
fn every_model_offers_its_own_efforts() {
let s = Settings::from_cli(&init(), &json!({"applied": {"model": "claude-sonnet-5-5", "effort": "high"}}));
let efforts = s.efforts_by_model();
let values = |model: &str| {
let (_, o) = efforts.iter().find(|(m, _)| m == model).unwrap();
let o = o.as_ref().unwrap();
let values: Vec<_> = o["options"].as_array().unwrap().iter().map(|v| v["value"].clone()).collect();
(values, o["currentValue"].clone())
};
assert_eq!(values("default"), (vec![json!("default"), json!("low"), json!("high")], json!("high")));
// The configured effort only where the model offers it.
assert_eq!(values("opus"), (vec![json!("default"), json!("low"), json!("max")], json!("default")));
assert!(efforts.contains(&("claude-haiku-4-5".into(), None)), "no effort option without levels");
}
}
+58
View File
@@ -59,6 +59,8 @@ const BUILTIN: &[(&str, &str, Driver, &[&str], &str)] = &[
("cursor", "Cursor", Driver::Acp, &["cursor-agent", "acp"], "Run `cursor-agent login` in a terminal."),
("opencode", "OpenCode", Driver::Acp, &["opencode", "acp"], "Run `opencode auth login` in a terminal."),
("qwen", "Qwen Code", Driver::Acp, &["qwen", "--experimental-acp"], "Run `qwen` in a terminal and sign in."),
("qoder", "Qoder", Driver::Acp, &["qodercli", "--acp"], "Run `qodercli login` in a terminal."),
("qodercn", "Qoder CN", Driver::Acp, &["qoderclicn", "--acp"], "Run `qoderclicn login` in a terminal."),
("kimi", "Kimi CLI", Driver::Acp, &["kimi", "acp"], "Run `kimi` in a terminal and sign in."),
("goose", "Goose", Driver::Acp, &["goose", "acp"], "Run `goose configure` in a terminal."),
("auggie", "Auggie", Driver::Acp, &["auggie", "--acp"], "Run `auggie login` in a terminal."),
@@ -185,6 +187,14 @@ pub(crate) fn builtin_spec(id: &str) -> Option<ProviderSpec> {
builtins().into_iter().find(|s| s.id == id)
}
/// The ACP adapters of Claude Code and Codex, which the native drivers replace: they still
/// launch, but are listed only once customized through an agent file or used by a task.
const UNLISTED: &[&str] = &["claude-acp", "codex-acp"];
pub(crate) fn is_unlisted(spec: &ProviderSpec) -> bool {
UNLISTED.contains(&spec.id.as_str()) && builtin_spec(&spec.id).as_ref() == Some(spec)
}
/// Public launch metadata, keeping the built-in defaults separate from agent-file overrides.
pub(crate) fn describe(spec: &ProviderSpec) -> AgentProvider {
let builtin = builtin_spec(&spec.id);
@@ -401,6 +411,45 @@ mod tests {
use super::*;
use serde_json::json;
#[test]
fn qoder_global_and_cn_editions_are_both_built_in() {
let info = |id: &str| describe(&load(None).into_iter().find(|s| s.id == id).unwrap());
let (global, cn) = (info("qoder"), info("qodercn"));
assert_eq!((global.name.as_str(), global.command.as_str()), ("Qoder", "qodercli --acp"));
assert_eq!((cn.name.as_str(), cn.command.as_str()), ("Qoder CN", "qoderclicn --acp"));
assert_eq!(global.login_hint.as_deref(), Some("Run `qodercli login` in a terminal."));
}
#[test]
fn qodercn_is_available_without_a_file_and_preserves_existing_customizations() {
let dir = tempfile::tempdir().unwrap();
let qodercn = load(None).into_iter().find(|s| s.id == "qodercn").unwrap();
assert_eq!(qodercn.driver, Driver::Acp);
assert!(!is_unlisted(&qodercn));
let info = describe(&qodercn);
assert_eq!(info.name, "Qoder CN");
assert_eq!(info.command, "qoderclicn --acp");
assert_eq!(info.source, AgentSource::Builtin);
assert!(info.command_editable);
assert_eq!(info.login_hint.as_deref(), Some("Run `qoderclicn login` in a terminal."));
std::fs::write(
dir.path().join("qodercn.toml"),
"command = \"/custom/qoderclicn\"\nargs = [\"--acp\"]\n[env]\nK = \"v\"\n",
)
.unwrap();
let specs = load(Some(dir.path()));
assert_eq!(specs.iter().filter(|s| s.id == "qodercn").count(), 1);
let custom = specs.iter().find(|s| s.id == "qodercn").unwrap();
assert_eq!(custom.command, "/custom/qoderclicn");
assert_eq!(custom.env, [("K".into(), "v".into())]);
set_builtin_command(dir.path(), "qodercn", None).unwrap();
let reset = load(Some(dir.path())).into_iter().find(|s| s.id == "qodercn").unwrap();
assert_eq!(reset.command, "qoderclicn");
assert_eq!(reset.args, ["--acp"]);
assert_eq!(reset.env, custom.env);
}
#[test]
fn login_hints_cover_all_builtin_commands_and_preserve_separate_login_tools() {
for spec in builtins() {
@@ -591,6 +640,15 @@ mod tests {
set_builtin_env(dir.path(), "not-a-builtin", env).unwrap_err();
}
#[test]
fn acp_adapters_of_native_agents_are_unlisted_until_customized() {
let dir = tempfile::tempdir().unwrap();
let unlisted = |dir: &Path| load(Some(dir)).into_iter().filter(is_unlisted).map(|s| s.id).collect::<Vec<_>>();
assert_eq!(unlisted(dir.path()), ["claude-acp", "codex-acp"]);
set_builtin_env(dir.path(), "codex-acp", BTreeMap::from([("A".to_owned(), "1".to_owned())])).unwrap();
assert_eq!(unlisted(dir.path()), ["claude-acp"]);
}
#[test]
fn set_builtin_command_overrides_and_clears_a_builtin() {
let dir = tempfile::tempdir().unwrap();
+207 -5
View File
@@ -35,6 +35,9 @@ struct FakeAgent {
/// Without `loadSession` (a session lives only as long as its process).
no_load: bool,
auth_on_new: bool,
auth_on_load: bool,
authenticate_method: Option<String>,
authenticated: AtomicBool,
request_error: Option<(i64, String)>,
auth_methods: Vec<Value>,
/// Hold the late mode reply until the prompt's permission request is already waiting.
@@ -127,9 +130,9 @@ impl Handler for FakeAgent {
fn on_request(&self, method: &str, params: Value) -> Result<Value, (i64, String)> {
self.calls.lock().push(method.to_owned());
if (method == "session/new" && self.auth_on_new)
|| (method == "session/prompt" && params.pointer("/prompt/0/text") == Some(&json!("auth")))
{
let needs_auth = !self.authenticated.load(Ordering::Acquire)
&& ((method == "session/new" && self.auth_on_new) || (method == "session/load" && self.auth_on_load));
if needs_auth || (method == "session/prompt" && params.pointer("/prompt/0/text") == Some(&json!("auth"))) {
return Err(self
.request_error
.clone()
@@ -176,6 +179,16 @@ impl Handler for FakeAgent {
self.say(&session, "replayed history");
Ok(json!({}))
}
"authenticate" if self.authenticate_method.is_some() => {
let id = params.get("methodId").and_then(Value::as_str).unwrap_or_default();
self.calls.lock().push(format!("authenticate {id}"));
if Some(id) == self.authenticate_method.as_deref() {
self.authenticated.store(true, Ordering::Release);
Ok(json!({}))
} else {
Err((codes::INVALID_PARAMS, format!("unexpected methodId {id}")))
}
}
"session/set_config_option" | "session/set_mode" => {
self.calls.lock().push(format!("{method} {}", params.get("value").or(params.get("modeId")).unwrap()));
if params["configId"] == "model" && !self.model_options.lock().is_empty() {
@@ -696,6 +709,91 @@ async fn registry_sign_in_errors_use_the_advertised_terminal_login() {
}
}
#[tokio::test(flavor = "multi_thread")]
async fn authenticates_and_retries_session_setup_once_on_auth_error() {
let tmp = tempfile::tempdir().unwrap();
let methods = vec![
json!({"id": "cli_login", "_meta": {"terminal-auth": {"command": "copilot", "args": ["login"]}}}),
json!({"id": "github_oauth", "name": "GitHub OAuth"}),
];
let agent = Arc::new(FakeAgent {
auth_on_new: true,
authenticate_method: Some("github_oauth".into()),
request_error: Some((-32000, "Authentication required".into())),
auth_methods: methods.clone(),
..Default::default()
});
let mgr = manager(tmp.path(), agent.clone());
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("again"),
provider: "copilot".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
assert_eq!(
agent
.calls
.lock()
.iter()
.filter(|c| c.starts_with("session/") || c.starts_with("authenticate"))
.collect::<Vec<_>>(),
["session/new", "authenticate", "authenticate github_oauth", "session/new", "session/prompt"],
);
// Resuming after an IDE restart also authenticates once and retries `session/load`.
mgr.shutdown();
let agent2 = Arc::new(FakeAgent {
auth_on_load: true,
auth_on_new: true,
authenticate_method: Some("github_oauth".into()),
request_error: Some((-32000, "Authentication required".into())),
auth_methods: methods.clone(),
..Default::default()
});
let mgr2 = manager(tmp.path(), agent2.clone());
mgr2.prompt(&task.id, text("again"), false).await.unwrap();
tokio::time::sleep(Duration::from_millis(50)).await;
wait_for(&mgr2, &task.id, AgentStatus::Ready).await;
assert_eq!(
agent2
.calls
.lock()
.iter()
.filter(|c| c.starts_with("session/") || c.starts_with("authenticate"))
.collect::<Vec<_>>(),
["session/load", "authenticate", "authenticate github_oauth", "session/load", "session/prompt"],
);
// If `authenticate` fails during `session/load`, fallback to `session/new` does not try it again.
mgr2.shutdown();
let agent3 = Arc::new(FakeAgent {
auth_on_load: true,
auth_on_new: true,
request_error: Some((-32000, "Authentication required".into())),
auth_methods: methods,
..Default::default()
});
let mgr3 = manager(tmp.path(), agent3.clone());
mgr3.prompt(&task.id, text("again"), false).await.unwrap();
wait_for(&mgr3, &task.id, AgentStatus::Failed).await;
assert_eq!(
agent3
.calls
.lock()
.iter()
.filter(|c| c.starts_with("session/") || c.starts_with("authenticate"))
.collect::<Vec<_>>(),
["session/load", "authenticate", "session/new"],
);
mgr3.shutdown();
}
#[tokio::test(flavor = "multi_thread")]
async fn commands_are_remembered_per_project_and_merged_with_skills_on_disk() {
let tmp = tempfile::tempdir().unwrap();
@@ -2521,10 +2619,10 @@ async fn session_settings_win_over_discovery_in_memory_and_on_disk() {
let saved = options("saved", "low");
let discovered = options("discovered", "high");
if discovery_first {
mgr.shared.remember_discovered("claude", &discovered, &[]);
mgr.shared.remember_discovered("claude", &discovered, &[], &[]);
}
mgr.shared.remember("claude", Some(&saved), None, Some("saved"));
mgr.shared.remember_discovered("claude", &discovered, &[]);
mgr.shared.remember_discovered("claude", &discovered, &[], &[]);
let restored = manager(tmp.path(), Arc::new(FakeAgent::default()));
for mgr in [&mgr, &restored] {
assert_eq!(model_of(&mgr.shared.known_config("claude")).as_deref(), Some("saved"));
@@ -2533,6 +2631,31 @@ async fn session_settings_win_over_discovery_in_memory_and_on_disk() {
}
}
#[tokio::test(flavor = "multi_thread")]
async fn discovered_effort_levels_fill_models_no_session_has_reported() {
let tmp = tempfile::tempdir().unwrap();
let mgr = manager(tmp.path(), Arc::new(FakeAgent::default()));
let effort = |values: &[&str], current: &str| {
config::options_of(&json!({ "configOptions": [
{ "id": "effort", "category": "thought_level", "type": "select", "currentValue": current,
"options": values.iter().map(|v| json!({ "value": v })).collect::<Vec<_>>() }
] }))
};
let model = config::options_of(&json!({ "configOptions": [
{ "id": "model", "category": "model", "type": "select", "currentValue": "sonnet",
"options": [{ "value": "sonnet" }, { "value": "opus" }] }
] }));
// A session already reported sonnet's levels; discovery adds opus's.
let reported = [model.clone(), effort(&["low", "high"], "high")].concat();
mgr.shared.remember("claude", Some(&reported), None, Some("sonnet"));
let levels =
[("sonnet".to_owned(), effort(&["low"], "low")), ("opus".to_owned(), effort(&["low", "max"], "default"))];
mgr.shared.remember_discovered("claude", &model, &[], &levels);
let known = mgr.shared.known.lock();
assert_eq!(known["claude"].thought_levels["sonnet"], effort(&["low", "high"], "high"), "the session's report wins");
assert_eq!(known["claude"].thought_levels["opus"], effort(&["low", "max"], "default"));
}
#[test]
fn streamed_text_is_sent_as_appends() {
let mut streamed = HashMap::new();
@@ -2568,3 +2691,82 @@ fn streamed_text_is_sent_as_appends() {
let rewritten = entry_event("t", &mut streamed, None, 5, &agent(&format!("{}x!??", "é".repeat(4999))));
assert!(matches!(rewritten, AgentEvent::Entry { .. }), "a rewritten end is sent whole");
}
#[test]
fn error_messages_keep_the_agents_details() {
let details = json!({ "details": "spawn claude ENOENT" });
assert_eq!(with_details("Internal error".into(), Some(&details)), "Internal error: spawn claude ENOENT");
assert_eq!(
with_details("Internal error".into(), Some(&json!("model not found"))),
"Internal error: model not found"
);
assert_eq!(with_details(String::new(), Some(&details)), "spawn claude ENOENT");
assert_eq!(with_details("spawn claude ENOENT".into(), Some(&details)), "spawn claude ENOENT");
assert_eq!(with_details("Internal error".into(), Some(&json!({ "other": 1 }))), "Internal error");
assert_eq!(with_details("Internal error".into(), None), "Internal error");
}
#[tokio::test(flavor = "multi_thread")]
async fn rolling_back_restores_the_start_of_the_task_and_keeps_the_users_own_edits() {
let tmp = tempfile::tempdir().unwrap();
let dir = dunce::canonicalize(tmp.path()).unwrap();
let git = |args: &[&str]| {
let ok = std::process::Command::new("git")
.args(["-c", "user.name=t", "-c", "user.email=t@t", "-c", "commit.gpgsign=false"])
.args(args)
.current_dir(&dir)
.status()
.unwrap()
.success();
assert!(ok, "{args:?}");
};
let write = |rel: &str, text: &str| std::fs::write(dir.join(rel), text).unwrap();
let read = |rel: &str| std::fs::read_to_string(dir.join(rel)).unwrap();
git(&["init", "-q", "-b", "main"]);
write(".gitignore", "store/\nworktrees/\nagents/\nmodels.json\n");
write("mine.txt", "base\n");
write("other.txt", "base\n");
git(&["add", "."]);
git(&["commit", "-q", "-m", "init"]);
// The user's own uncommitted edit, before the task starts.
write("mine.txt", "my edit\n");
let mgr = manager(&dir, Arc::new(FakeAgent::default()));
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("Hello"),
provider: "claude".into(),
config: Vec::new(),
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::NeedsInput).await;
mgr.cancel(&task.id, None).unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
// What the agent did.
write("mine.txt", "my edit\nand the agent's\n");
write("other.txt", "agent\n");
write("new.txt", "agent\n");
async fn changed(mgr: &AgentManager, id: &str) -> Vec<String> {
mgr.changes(id).await.unwrap().into_iter().map(|c| c.rel_path).collect()
}
assert_eq!(changed(&mgr, &task.id).await, ["mine.txt", "new.txt", "other.txt"]);
mgr.rollback_changes(&task.id, Some(vec!["other.txt".into()])).await.unwrap();
assert_eq!(read("other.txt"), "base\n");
assert_eq!(changed(&mgr, &task.id).await, ["mine.txt", "new.txt"]);
// After a restart, from the saved copy.
mgr.shutdown();
let mgr = manager(&dir, Arc::new(FakeAgent::default()));
mgr.rollback_changes(&task.id, None).await.unwrap();
assert_eq!(read("mine.txt"), "my edit\n", "the user's edit stays");
assert!(!dir.join("new.txt").exists());
assert!(changed(&mgr, &task.id).await.is_empty());
mgr.delete(&task.id, false).await.unwrap();
assert!(!dir.join("store").join(format!("{}.before", task.id)).exists());
}
+94 -8
View File
@@ -14,8 +14,18 @@ pub(crate) struct Created {
pub base_commit: String,
}
/// The repository of `path`, free to execute: the host runs every agent command that
/// reaches a repository only in a trusted project.
fn discover(path: &Path) -> IdeResult<Option<Repo>> {
let repo = Repo::discover(path)?;
if let Some(repo) = &repo {
repo.set_execution_allowed(true);
}
Ok(repo)
}
fn repo_at(path: &Path) -> IdeResult<Repo> {
Repo::discover(path)?.ok_or_else(|| IdeError::unavailable(format!("{} is not in a git repository", path.display())))
discover(path)?.ok_or_else(|| IdeError::unavailable(format!("{} is not in a git repository", path.display())))
}
/// The current branch (or commit when detached) and its commit.
@@ -91,10 +101,11 @@ fn retrying(attempts: u32, pause: Duration, mut op: impl FnMut() -> std::io::Res
/// Files under `cwd` that differ from `base` (untracked files included), with line counts.
/// Outside a repository there is nothing to compare against: no changes.
pub(crate) fn changes(cwd: &Path, base: &str) -> IdeResult<Vec<AgentChange>> {
let Some(repo) = Repo::discover(cwd)? else { return Ok(Vec::new()) };
let Some(repo) = discover(cwd)? else { return Ok(Vec::new()) };
let root = repo.root().to_path_buf();
let stats = repo.numstat_with_worktree(base)?;
let stat = |rel: &str| stats.iter().find(|(p, ..)| p == rel).map_or((0, 0), |&(_, a, r)| (a, r));
let stats: std::collections::HashMap<String, (u32, u32)> =
repo.numstat_with_worktree(base)?.into_iter().map(|(p, a, r)| (p, (a, r))).collect();
let stat = |rel: &str| stats.get(rel).copied().unwrap_or((0, 0));
let mut out: Vec<AgentChange> = repo
.diff_with_worktree(base)?
.into_iter()
@@ -103,8 +114,9 @@ pub(crate) fn changes(cwd: &Path, base: &str) -> IdeResult<Vec<AgentChange>> {
AgentChange { path: IpcPath(root.join(&f.path)), rel_path: f.path, kind: f.kind, added, removed }
})
.collect();
let tracked: std::collections::HashSet<String> = out.iter().map(|c| c.rel_path.clone()).collect();
for s in repo.status()? {
if s.unstaged == Some(ChangeKind::Untracked) && !out.iter().any(|c| c.rel_path == s.rel_path) {
if s.unstaged == Some(ChangeKind::Untracked) && !tracked.contains(&s.rel_path) {
let added = std::fs::read_to_string(&*s.path).map_or(0, |t| t.lines().count() as u32);
out.push(AgentChange { path: s.path, rel_path: s.rel_path, kind: ChangeKind::Added, added, removed: 0 });
}
@@ -116,7 +128,7 @@ pub(crate) fn changes(cwd: &Path, base: &str) -> IdeResult<Vec<AgentChange>> {
/// Size and modification time of a file, `None` once it is gone.
pub(crate) type Stamp = Option<(u64, u128)>;
fn stamp(path: &Path) -> Stamp {
pub(crate) fn stamp(path: &Path) -> Stamp {
let meta = std::fs::metadata(path).ok()?;
let modified = meta.modified().ok()?.duration_since(UNIX_EPOCH).ok()?.as_nanos();
Some((meta.len(), modified))
@@ -124,7 +136,10 @@ fn stamp(path: &Path) -> Stamp {
/// The files already changed in `cwd` against `HEAD`, stamped so later edits to them show.
pub(crate) fn snapshot(cwd: &Path) -> IdeResult<BTreeMap<String, Stamp>> {
Ok(changes(cwd, "HEAD")?.into_iter().map(|c| (c.rel_path, stamp(&c.path))).collect())
// Paths and stamps only: no diff and no line counts, which read every untracked file
// and held up starting a task in a checkout with many of them.
let Some(repo) = Repo::discover(cwd)? else { return Ok(BTreeMap::new()) };
Ok(repo.status()?.into_iter().map(|s| (s.rel_path, stamp(&s.path))).collect())
}
/// Drops the changes that were there before the task and haven't been touched since.
@@ -132,8 +147,79 @@ pub(crate) fn without_untouched(changes: Vec<AgentChange>, before: &BTreeMap<Str
changes.into_iter().filter(|c| before.get(&c.rel_path).is_none_or(|s| *s != stamp(&c.path))).collect()
}
/// Files larger than this that were changed before the task keep no copy: they can't be rolled back.
const SAVED_MAX: u64 = 8 * 1024 * 1024;
fn saved_name(rel: &str) -> String {
use std::hash::{DefaultHasher, Hash, Hasher};
let mut hash = DefaultHasher::new();
rel.hash(&mut hash);
format!("{:016x}", hash.finish())
}
/// Copies the files in `before` (already changed in the checkout at `cwd` when the task started)
/// into `dir`, so rolling back the task's changes to them restores the user's own edits.
/// Whether anything was saved.
pub(crate) fn save_before(cwd: &Path, before: &BTreeMap<String, Stamp>, dir: &Path) -> IdeResult<bool> {
let Some(repo) = Repo::discover(cwd)? else { return Ok(false) };
let mut saved = false;
for (rel, stamp) in before {
let path = repo.root().join(rel);
if stamp.is_none() || !std::fs::metadata(&path).is_ok_and(|m| m.is_file() && m.len() <= SAVED_MAX) {
continue;
}
std::fs::create_dir_all(dir)?;
std::fs::copy(&path, dir.join(saved_name(rel)))?;
saved = true;
}
Ok(saved)
}
/// Rolls `change` back: to the copy saved when the task started when the file had changes
/// then (`before`, `saved` holding the copies), otherwise to `base` (removed when `base`
/// lacks it).
pub(crate) fn restore(
cwd: &Path,
base: &str,
change: &AgentChange,
before: &BTreeMap<String, Stamp>,
saved: Option<&Path>,
) -> IdeResult<()> {
let path = &change.path.0;
let content = match before.get(&change.rel_path) {
Some(None) => None,
Some(Some(_)) => {
let copy =
saved.map(|d| d.join(saved_name(&change.rel_path))).filter(|p| p.is_file()).ok_or_else(|| {
IdeError::invalid(format!(
"{} had changes of yours when the task started that weren't saved; roll it back in Git instead",
change.rel_path
))
})?;
Some(std::fs::read(copy)?)
}
None => {
let repo = Repo::discover(cwd)?.ok_or_else(|| IdeError::invalid("not in a git repository"))?;
repo.file_bytes_at(path, Some(base))?
}
};
match content {
Some(bytes) => {
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(path, bytes)?;
}
None => match std::fs::remove_file(path) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => return Err(e.into()),
_ => {}
},
}
Ok(())
}
pub(crate) fn file_at(cwd: &Path, path: &Path, rev: &str) -> IdeResult<Option<String>> {
let Some(repo) = Repo::discover(cwd)? else { return Ok(None) };
let Some(repo) = discover(cwd)? else { return Ok(None) };
let encoding = match ide_vfs::load_text(path) {
Ok(loaded) => encoding_rs::Encoding::for_label(loaded.encoding.as_bytes()).unwrap_or(encoding_rs::UTF_8),
Err(_) => ide_vfs::detect_encoding(&repo.file_bytes_at(path, Some(rev))?.unwrap_or_default(), None),
+110 -22
View File
@@ -79,23 +79,26 @@ fn scan(
Ok(())
}
/// `None` means an input cannot be resolved safely: build normally without reuse.
/// All filesystem work belongs on the caller's blocking thread.
pub fn launch_fingerprint(
/// One planned step as fingerprinted: resolved program, args, cwd and env.
type StepFingerprint = (PathBuf, Vec<String>, PathBuf, BTreeMap<String, String>);
/// Environment, planned steps and input paths shared by every Maven build
/// fingerprint: the filtered process environment plus the wrapper, settings
/// and JDK files the planned invocation reads.
struct Collected {
inputs: Vec<PathBuf>,
environment: BTreeMap<String, String>,
steps: Vec<StepFingerprint>,
inherited_hash: u64,
}
fn collect(
root: &Path,
model: &BuildModel,
module: &Path,
test: bool,
projects: &[&maven::MavenProject],
plan: &BuildPlan,
local_repo: Option<PathBuf>,
cancel: &CancellationToken,
) -> IdeResult<Option<LaunchFingerprint>> {
let Some(m) = &model.maven else { return Ok(None) };
let projects = maven_launch_projects(root, m, module, test);
if projects.is_empty() {
return Ok(None);
}
let Some(mut inputs) = maven::launch_inputs(root, &projects, local_repo) else { return Ok(None) };
let plan = launch_plan(root, model, module, test, BuildKind::Build)?;
) -> IdeResult<Option<Collected>> {
let Some(mut inputs) = maven::launch_inputs(root, projects, local_repo) else { return Ok(None) };
let mut environment: BTreeMap<String, String> = std::env::vars()
.map(|(key, value)| (key.to_ascii_uppercase(), value))
.filter(|(key, _)| {
@@ -165,26 +168,111 @@ pub fn launch_fingerprint(
Path::new(home).join(if cfg!(windows) { "bin/java.exe" } else { "bin/java" }),
]);
}
let steps: Vec<_> =
plan.steps.iter().map(|s| (process::program_path(&s.program, &s.env), &s.args, &s.cwd, &s.env)).collect();
let steps = plan
.steps
.iter()
.map(|s| (process::program_path(&s.program, &s.env), s.args.clone(), s.cwd.clone(), s.env.clone()))
.collect();
// Profiles and plugins can read arbitrary environment variables. Keep a signature
// of the inherited environment without persisting unrelated credentials.
let inherited: BTreeMap<_, _> = std::env::vars_os().collect();
let mut inherited_hash = std::hash::DefaultHasher::new();
inherited.hash(&mut inherited_hash);
let parameters =
serde_json::to_string(&(root, module, test, steps, environment, inherited_hash.finish(), projects))
.map_err(|e| IdeError::internal(e.to_string()))?;
Ok(Some(Collected { inputs, environment, steps, inherited_hash: inherited_hash.finish() }))
}
/// Scans every input, storing project inputs relative to the project.
fn scan_all(
root: &Path,
inputs: Vec<PathBuf>,
cancel: &CancellationToken,
) -> IdeResult<BTreeMap<PathBuf, Option<Stamp>>> {
let mut files = BTreeMap::new();
for path in inputs {
scan(root, &path, &mut files, &mut HashSet::new(), cancel)?;
}
// Store project inputs relative to the project, including each file's relative path.
let files = files.into_iter().map(|(p, stamp)| (p.strip_prefix(root).unwrap_or(&p).to_path_buf(), stamp)).collect();
Ok(Some(LaunchFingerprint { version: 2, parameters, files }))
Ok(files.into_iter().map(|(p, stamp)| (p.strip_prefix(root).unwrap_or(&p).to_path_buf(), stamp)).collect())
}
/// `None` means an input cannot be resolved safely: build normally without reuse.
/// All filesystem work belongs on the caller's blocking thread.
pub fn launch_fingerprint(
root: &Path,
model: &BuildModel,
module: &Path,
test: bool,
local_repo: Option<PathBuf>,
cancel: &CancellationToken,
) -> IdeResult<Option<LaunchFingerprint>> {
let Some(m) = &model.maven else { return Ok(None) };
let projects = maven_launch_projects(root, m, module, test);
if projects.is_empty() {
return Ok(None);
}
let plan = launch_plan(root, model, module, test, BuildKind::Build)?;
let Some(collected) = collect(root, &projects, &plan, local_repo)? else { return Ok(None) };
let Collected { inputs, environment, steps, inherited_hash } = collected;
let parameters = serde_json::to_string(&(root, module, test, steps, environment, inherited_hash, projects))
.map_err(|e| IdeError::internal(e.to_string()))?;
Ok(Some(LaunchFingerprint { version: 2, parameters, files: scan_all(root, inputs, cancel)? }))
}
/// Inputs fingerprint of a menu build (`Build Project` / `Build Module`): the
/// built modules' sources, resources and POM chain plus the exact planned
/// steps, so an unchanged project skips Maven entirely (#2476). Only `Build`
/// callers reuse it, never `Rebuild` or `Artifacts`; `None` disables reuse.
/// All filesystem work belongs on the caller's blocking thread.
pub fn menu_fingerprint(
root: &Path,
model: &BuildModel,
file: Option<&Path>,
cancel: &CancellationToken,
) -> IdeResult<Option<LaunchFingerprint>> {
let plan = plan(root, model, BuildKind::Build, file, None)?;
if plan.system != BuildSystem::Maven {
return Ok(None);
}
let Some(m) = &model.maven else { return Ok(None) };
let projects: Vec<&maven::MavenProject> = match file {
Some(file) => {
// The innermost linked project owning the file, as in single-file compilation.
let rel = rel_str(root, file);
let module = m
.projects
.iter()
.filter(|p| rel == p.dir || rel.starts_with(&format!("{}/", p.dir)) || p.dir.is_empty())
.max_by_key(|p| p.dir.len());
match module {
Some(project) if project.packaging != "pom" => {
maven_launch_projects(root, m, &subdir(root, &project.dir), true)
}
_ => return Ok(None),
}
}
// A project build walks every linked project.
None => m.projects.iter().collect(),
};
if projects.is_empty() {
return Ok(None);
}
let Some(collected) = collect(root, &projects, &plan, None)? else { return Ok(None) };
let Collected { inputs, environment, steps, inherited_hash } = collected;
let scope = file.map(|f| rel_str(root, f));
let parameters = serde_json::to_string(&("menu-build", scope, steps, environment, inherited_hash))
.map_err(|e| IdeError::internal(e.to_string()))?;
Ok(Some(LaunchFingerprint { version: 2, parameters, files: scan_all(root, inputs, cancel)? }))
}
impl LaunchFingerprint {
/// Folders (relative to the project) of the Maven projects whose POM is an input.
pub(super) fn project_dirs(&self) -> impl Iterator<Item = &Path> {
self.files
.keys()
.filter(|p| p.is_relative() && p.file_name().is_some_and(|n| n == "pom.xml"))
.map(|pom| pom.parent().unwrap_or(Path::new("")))
}
/// Relative paths whose content changed while `(length, mtime)` stayed the same
/// compared to `previous` (timestamp-preserving external edits, coarse-granularity
/// filesystems). Timestamp-incremental builders (Maven, the Kotlin plugin) would
@@ -0,0 +1,253 @@
//! Resources a Gradle build changes while copying them (filters, `expand`, renames, extra
//! sources). The Java language server copies them as they are into Buildship's
//! `bin/<source set>` folders, which launches run from; IntelliJ IDEA runs Gradle's copy.
//! Before a launch, Gradle processes them and its output replaces those plain copies, so
//! each resource is on the classpath once, as the build produces it.
use super::{BuildPlan, BuildStep, BuildSystem, Format, run, subdir};
use crate::BuildModel;
use crate::gradle::{self, GradleModel, ReportedProject};
use ide_core::{CancellationToken, IdeError, IdeResult, Severity};
use std::fs;
use std::path::{Path, PathBuf};
/// A source set whose resources Gradle processes, with the folder the launch reads them from.
#[derive(Debug, PartialEq)]
struct Target {
/// Task path from the root build (`:app:processResources`).
task: String,
/// Where the task writes the resources.
output: PathBuf,
/// The Java language server's output folder of the source set, on the launch classpath.
jdtls: PathBuf,
}
fn targets(reported: &[ReportedProject], class_paths: &[String]) -> Vec<Target> {
// buildSrc is built before the build, never addressable as `:buildSrc:<task>`.
let addressable = |p: &&ReportedProject| !p.build.as_deref().is_some_and(|b| b.ends_with(":buildSrc"));
let sets: Vec<_> = reported
.iter()
.filter(addressable)
.flat_map(|p| p.source_sets.iter().filter_map(move |s| Some((p, s, s.processed.as_ref()?))))
.collect();
if sets.is_empty() {
return Vec::new();
}
// Canonical, as Gradle reports its folders: jdtls may spell them differently.
let entries: Vec<PathBuf> = class_paths.iter().filter_map(|p| dunce::canonicalize(p).ok()).collect();
sets.into_iter()
.filter_map(|(p, set, resources)| {
// Buildship's output folder of the source set.
let jdtls = dunce::canonicalize(Path::new(&p.dir).join("bin").join(&set.name)).ok()?;
let project = if p.path == ":" { "" } else { p.path.as_str() };
entries.contains(&jdtls).then(|| Target {
task: format!("{project}:{}", resources.task),
output: PathBuf::from(&resources.dir),
jdtls,
})
})
.collect()
}
/// Before a JVM launch from the Java language server's output folders on `class_paths`:
/// runs the Gradle tasks that process resources of those folders' source sets (see the
/// module docs) and copies their output over the plain copies. `cache_dir` holds the
/// linked builds' last syncs. Returns the tasks run, none for builds that copy their
/// resources as they are.
pub fn process_launch_resources(
root: &Path,
model: &BuildModel,
cache_dir: &Path,
class_paths: &[String],
cancel: &CancellationToken,
) -> IdeResult<Vec<String>> {
// A composite also reports the builds it includes: the deepest build owning a folder runs its task.
let mut builds: Vec<&GradleModel> = model.gradle.iter().collect();
builds.sort_by_key(|g| std::cmp::Reverse(subdir(root, &g.dir).components().count()));
let mut done: Vec<PathBuf> = Vec::new();
let mut tasks = Vec::new();
for g in builds {
let reported = crate::sync::reported_gradle_projects(&gradle::cache_dir(cache_dir, &g.dir));
let targets: Vec<Target> =
targets(&reported, class_paths).into_iter().filter(|t| !done.contains(&t.jdtls)).collect();
if targets.is_empty() {
continue;
}
let mut step = BuildStep::new(&g.command, &["--console=plain"], &subdir(root, &g.dir), Format::Gradle);
step.args.extend(targets.iter().map(|t| t.task.clone()));
step.env = g.env.clone();
let plan = BuildPlan { system: BuildSystem::Gradle, title: "Process resources".into(), steps: vec![step] };
let result = run(root, &plan, cancel, &|_| {})?;
if !result.success {
let errors: Vec<&str> = result
.messages
.iter()
.filter(|m| m.severity == Severity::Error)
.take(5)
.map(|m| m.text.as_str())
.collect();
return Err(IdeError::internal(errors.join("\n")));
}
for t in targets {
if cancel.is_cancelled() {
return Err(IdeError::cancelled());
}
overlay(&t.output, &t.output, &t.jdtls)?;
done.push(t.jdtls);
tasks.push(t.task);
}
}
Ok(tasks)
}
/// Copies the files under `dir` to the same place under `to` (relative to `base`) where
/// they differ, so unchanged resources are not rewritten on every launch.
fn overlay(base: &Path, dir: &Path, to: &Path) -> IdeResult<()> {
let entries = match fs::read_dir(dir) {
// No resources: the task had nothing to copy.
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(()),
entries => entries?,
};
for entry in entries {
let entry = entry?;
let path = entry.path();
let kind = entry.file_type()?;
if kind.is_dir() {
overlay(base, &path, to)?;
} else if kind.is_file() {
let dest = to.join(path.strip_prefix(base).map_err(|e| IdeError::internal(e.to_string()))?);
if !same_content(&path, &dest) {
if let Some(parent) = dest.parent() {
fs::create_dir_all(parent)?;
}
fs::copy(&path, &dest)?;
}
}
}
Ok(())
}
fn same_content(a: &Path, b: &Path) -> bool {
match (fs::metadata(a), fs::metadata(b)) {
(Ok(m), Ok(n)) if m.len() == n.len() => matches!((fs::read(a), fs::read(b)), (Ok(x), Ok(y)) if x == y),
_ => false,
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::gradle::{ProcessedResources, ReportedSourceSet};
fn write(path: &Path, text: &str) {
fs::create_dir_all(path.parent().unwrap()).unwrap();
fs::write(path, text).unwrap();
}
fn set(name: &str, processed: Option<&Path>) -> ReportedSourceSet {
ReportedSourceSet {
name: name.into(),
test: name == "test",
processed: processed.map(|dir| ProcessedResources {
task: if name == "main" { "processResources".into() } else { "processTestResources".into() },
dir: dir.to_string_lossy().into_owned(),
}),
..Default::default()
}
}
fn project(path: &str, dir: &Path, sets: Vec<ReportedSourceSet>) -> ReportedProject {
ReportedProject {
path: path.into(),
dir: dir.to_string_lossy().into_owned(),
source_sets: sets,
..Default::default()
}
}
/// Only processed source sets whose jdtls output is on the classpath: the launched
/// module, upstream modules of the build, tests only for a test classpath; never buildSrc.
#[test]
fn targets_are_processed_sets_on_the_classpath() {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
let (app, lib, plain, src) = (root.join("app"), root.join("lib"), root.join("plain"), root.join("buildSrc"));
for dir in [&root, &app, &lib, &plain, &src] {
for set in ["main", "test"] {
fs::create_dir_all(dir.join("bin").join(set)).unwrap();
}
}
let out = |dir: &Path, set: &str| dir.join("build/resources").join(set);
let mut build_src = project(":buildSrc", &src, vec![set("main", Some(&out(&src, "main")))]);
build_src.build = Some(":buildSrc".into());
let reported = [
project(":", &root, vec![set("main", Some(&out(&root, "main")))]),
project(":app", &app, vec![set("main", Some(&out(&app, "main"))), set("test", Some(&out(&app, "test")))]),
project(":lib", &lib, vec![set("main", Some(&out(&lib, "main")))]),
project(":plain", &plain, vec![set("main", None)]),
build_src,
];
let entry = |dir: &Path, set: &str| dir.join("bin").join(set).to_string_lossy().into_owned();
let runtime = [
entry(&app, "main"),
// Spelled differently from Gradle's canonical folders.
format!("{}/", entry(&lib, "main")),
entry(&plain, "main"),
entry(&src, "main"),
"/m2/spring-core.jar".into(),
];
let tasks = |cp: &[String]| targets(&reported, cp).into_iter().map(|t| t.task).collect::<Vec<_>>();
assert_eq!(tasks(&runtime), [":app:processResources", ":lib:processResources"]);
let mut test = runtime.to_vec();
test.insert(0, entry(&app, "test"));
assert_eq!(tasks(&test), [":app:processResources", ":app:processTestResources", ":lib:processResources"]);
assert_eq!(tasks(&[entry(&root, "main")]), [":processResources"], "the root project");
let app_targets = targets(&reported, &runtime);
let t = &app_targets[0];
assert_eq!(
(t.output.as_path(), t.jdtls.as_path()),
(out(&app, "main").as_path(), app.join("bin/main").as_path())
);
let unprocessed = [project(":app", &app, vec![set("main", None)])];
assert!(targets(&unprocessed, &runtime).is_empty());
}
/// The processed copy replaces jdtls's plain one (the reported `@version@` case), new
/// files are added, identical ones are left alone and files Gradle doesn't produce stay.
#[test]
fn overlay_replaces_plain_copies_with_gradle_output() {
let tmp = tempfile::tempdir().unwrap();
let (output, jdtls) = (tmp.path().join("build/resources/main"), tmp.path().join("bin/main"));
write(&output.join("application.yaml"), "spring:\n application:\n version: 0.0.1\n");
write(&output.join("static/extra.txt"), "from an extra source");
write(&output.join("migrations/V1__init.sql"), "create table t (id int);");
write(&jdtls.join("application.yaml"), "spring:\n application:\n version: @version@\n");
write(&jdtls.join("migrations/V1__init.sql"), "create table t (id int);");
write(&jdtls.join("com/example/App.class"), "bytecode");
let unchanged = fs::metadata(jdtls.join("migrations/V1__init.sql")).unwrap().modified().unwrap();
std::thread::sleep(std::time::Duration::from_millis(20));
overlay(&output, &output, &jdtls).unwrap();
let read = |rel: &str| fs::read_to_string(jdtls.join(rel)).unwrap();
assert_eq!(read("application.yaml"), "spring:\n application:\n version: 0.0.1\n");
assert_eq!(read("static/extra.txt"), "from an extra source");
assert_eq!(read("com/example/App.class"), "bytecode");
let modified = fs::metadata(jdtls.join("migrations/V1__init.sql")).unwrap().modified().unwrap();
assert_eq!(modified, unchanged, "identical files are not rewritten");
overlay(&tmp.path().join("missing"), &tmp.path().join("missing"), &jdtls).unwrap();
}
#[test]
fn builds_without_processed_resources_run_nothing() {
let tmp = tempfile::tempdir().unwrap();
// Running this command would fail the call.
let missing = tmp.path().join("missing-gradle").to_string_lossy().into_owned();
let model =
BuildModel { gradle: vec![GradleModel { command: missing, ..Default::default() }], ..Default::default() };
let class_paths = [tmp.path().join("bin/main").to_string_lossy().into_owned()];
let tasks =
process_launch_resources(tmp.path(), &model, tmp.path(), &class_paths, &CancellationToken::new()).unwrap();
assert!(tasks.is_empty());
}
}
+55 -6
View File
@@ -4,9 +4,17 @@
//! for the Build tool window, the Problems view and the editor.
mod fingerprint;
mod launch_resources;
mod parse;
mod single;
mod up_to_date;
pub use fingerprint::{LaunchFingerprint, launch_fingerprint};
pub use fingerprint::{LaunchFingerprint, launch_fingerprint, menu_fingerprint};
pub use launch_resources::process_launch_resources;
pub use single::{
SingleFileTarget, compile_file_plan, find_javac, javac_step, single_file_classpath, single_file_target,
};
pub use up_to_date::{menu_up_to_date, record_menu_build};
use crate::process::{self, Stream};
use crate::{BuildModel, maven};
@@ -263,7 +271,12 @@ pub(super) fn maven_steps(
return Err(IdeError::not_found("no Maven project found"));
}
let step = |p: &maven::MavenProject, extra: &[String]| {
let mut s = BuildStep::new(&p.command, &["-B", "-Dstyle.color=never"], &subdir(root, &p.dir), Format::Maven);
let mut s = BuildStep::new(
&p.command,
&["-B", "-Dstyle.color=never", maven::LOCAL_REPO_TRACKING_ARG],
&subdir(root, &p.dir),
Format::Maven,
);
s.args.extend(p.options.iter().cloned());
s.args.extend(extra.iter().cloned());
s.args.extend(goals.iter().map(|g| (*g).to_owned()));
@@ -1044,9 +1057,21 @@ mod tests {
assert_eq!(p.system, BuildSystem::Maven);
assert_eq!(p.title, "demo");
assert_eq!(p.steps.len(), 1);
assert_eq!(p.steps[0].args, ["-B", "-Dstyle.color=never", "test-compile"]);
assert_eq!(p.steps[0].args, ["-B", "-Dstyle.color=never", maven::LOCAL_REPO_TRACKING_ARG, "test-compile"]);
let p = plan(&root, &model, BuildKind::Rebuild, Some(&root.join("core/src/main/java/A.java")), None).unwrap();
assert_eq!(p.steps[0].args, ["-B", "-Dstyle.color=never", "-pl", "core", "-am", "clean", "test-compile"]);
assert_eq!(
p.steps[0].args,
[
"-B",
"-Dstyle.color=never",
maven::LOCAL_REPO_TRACKING_ARG,
"-pl",
"core",
"-am",
"clean",
"test-compile"
]
);
assert_eq!(p.steps[0].cwd, root);
assert_eq!(p.title, "demo: build module 'core'");
@@ -1320,12 +1345,36 @@ mod tests {
assert_eq!(step.cwd, root);
assert_eq!(step.program, "mvnw");
assert_eq!(step.env["JAVA_HOME"], "/jdk");
assert_eq!(step.args, ["-B", "-Dstyle.color=never", "-s", "settings.xml", "-pl", "app", "-am", goal]);
assert_eq!(
step.args,
[
"-B",
"-Dstyle.color=never",
maven::LOCAL_REPO_TRACKING_ARG,
"-s",
"settings.xml",
"-pl",
"app",
"-am",
goal
]
);
// Rebuild cleans first, as IDEA's Rebuild does.
let plan = launch_plan(root, &model, &root.join("app"), test, BuildKind::Rebuild).unwrap();
assert_eq!(
plan.steps[0].args,
["-B", "-Dstyle.color=never", "-s", "settings.xml", "-pl", "app", "-am", "clean", goal]
[
"-B",
"-Dstyle.color=never",
maven::LOCAL_REPO_TRACKING_ARG,
"-s",
"settings.xml",
"-pl",
"app",
"-am",
"clean",
goal
]
);
}
}
+1 -26
View File
@@ -7,6 +7,7 @@
//! [`super::resolve`] turns them into IDE coordinates.
use ide_core::Severity;
pub(crate) use ide_core::output_encoding::strip_ansi;
use serde_json::Value;
/// Output dialect of a build step.
@@ -98,32 +99,6 @@ pub(crate) struct OutputParser {
pending: Option<Pending>,
}
/// Removes ANSI escape sequences (colour, cursor movement).
pub(crate) fn strip_ansi(s: &str) -> String {
if !s.contains('\u{1b}') {
return s.to_owned();
}
let mut out = String::with_capacity(s.len());
let mut chars = s.chars().peekable();
while let Some(c) = chars.next() {
if c != '\u{1b}' {
out.push(c);
continue;
}
if chars.peek() == Some(&'[') {
chars.next();
for c in chars.by_ref() {
if c.is_ascii_alphabetic() || c == '~' {
break;
}
}
} else {
chars.next();
}
}
out
}
fn num(s: &str) -> Option<u32> {
s.trim().parse().ok().filter(|n| *n > 0)
}
+331
View File
@@ -0,0 +1,331 @@
//! Single-file Java compilation with `javac` for the Compile File action (#2476).
//!
//! A module build (`-pl … -am`) still walks every upstream module, and packaging
//! walks the whole reactor; compiling one file only needs that file's module
//! outputs plus its dependency classpath, which is much faster to assemble.
use super::*;
use std::fs;
use std::process::Command as ProcCommand;
/// Output file of the `dependency:build-classpath` probe, relative to each module.
const CP_FILE: &str = "target/ide-compile-cp.txt";
/// A `.java` source and the module build that compiles it.
pub struct SingleFileTarget {
/// Absolute module directory.
pub module: PathBuf,
/// The source is under `src/test`: compile to test classes with the test classpath.
pub test: bool,
/// Absolute `target/classes` (or `target/test-classes`).
pub out_dir: PathBuf,
/// Reactor root building the module (owns the Maven command and options).
pub top: maven::MavenProject,
/// `-pl` value of the module relative to the reactor root; empty when the
/// module is the root itself.
pub pl: String,
/// `maven.compiler.release` (or equivalent) of the module's project.
pub java_version: Option<u32>,
}
/// Locates the linked Maven module owning a `.java` source file.
pub fn single_file_target(root: &Path, model: &BuildModel, file: &Path) -> IdeResult<SingleFileTarget> {
if file.extension().is_none_or(|ext| ext != "java") {
return Err(IdeError::invalid("only Java sources can be compiled as a single file"));
}
if !file.is_file() {
return Err(IdeError::not_found("no Java source found for single-file compilation"));
}
let Some(m) = &model.maven else {
return Err(IdeError::not_found("no linked Maven project found for module"));
};
// The innermost project whose directory contains the file owns it; the
// aggregator root (`""`) is a prefix of everything, so take the longest match.
let rel = rel_str(root, file);
let project = m
.projects
.iter()
.filter(|p| rel == p.dir || rel.starts_with(&format!("{}/", p.dir)) || p.dir.is_empty())
.max_by_key(|p| p.dir.len())
.ok_or_else(|| IdeError::not_found("no linked Maven project found for module"))?;
if project.packaging == "pom" {
// An aggregator owns no sources itself: a file that only matches it
// belongs to no linked module.
return Err(IdeError::not_found("no linked Maven project found for module"));
}
let module = subdir(root, &project.dir);
let top = maven_reactor(m, &project.dir)
.ok_or_else(|| IdeError::not_found("no linked Maven project found for module"))?;
let top_dir = subdir(root, &top.dir);
let pl = module.strip_prefix(&top_dir).map(|r| r.to_string_lossy().replace('\\', "/")).unwrap_or_default();
// Test sources see the test classpath and compile into test classes.
let parts: Vec<&str> = rel.split('/').collect();
let test = parts.windows(2).any(|w| w == ["src", "test"]);
let out_dir = module.join("target").join(if test { "test-classes" } else { "classes" });
Ok(SingleFileTarget { module, test, out_dir, top: top.clone(), pl, java_version: project.java_version })
}
/// `javac` for the module: its `JAVA_HOME`, else `PATH`.
pub fn find_javac(env: &BTreeMap<String, String>) -> String {
let home = env.get("JAVA_HOME").map(Path::new);
let exe = if cfg!(windows) { "javac.exe" } else { "javac" };
if let Some(javac) = home.map(|h| h.join("bin").join(exe)).filter(|p| p.is_file()) {
return javac.to_string_lossy().into_owned();
}
exe.to_owned()
}
/// Dependency classpath of the module: existing reactor class directories plus
/// `mvn dependency:build-classpath` (which resolves without compiling anything,
/// so `-am` stays cheap). Entries are platform-separated (`;` on Windows).
pub fn single_file_classpath(root: &Path, model: &BuildModel, target: &SingleFileTarget) -> IdeResult<String> {
let mut entries: Vec<String> = Vec::new();
// The module's own classes first: the file almost always uses classes next to it.
let own: &[&str] = if target.test { &["target/test-classes", "target/classes"] } else { &["target/classes"] };
for out in own {
let path = target.module.join(out);
if path.is_dir() {
entries.push(path.to_string_lossy().into_owned());
}
}
if let Some(m) = &model.maven {
// Upstream reactor modules first: their freshly built classes shadow
// installed snapshots of the same artifacts.
for project in maven_launch_projects(root, m, &target.module, target.test) {
let dir = subdir(root, &project.dir);
if dir == target.module {
continue;
}
for classes in ["target/classes", "target/test-classes"] {
let path = dir.join(classes);
if path.is_dir() {
entries.push(path.to_string_lossy().into_owned());
}
}
}
}
let probe = CP_FILE;
let mut args = vec!["-B", "-Dstyle.color=never", maven::LOCAL_REPO_TRACKING_ARG];
args.extend(target.top.options.iter().map(String::as_str));
if !target.pl.is_empty() {
args.extend(["-pl", target.pl.as_str(), "-am"]);
}
let output_arg = format!("-Dmdep.outputFile={probe}");
args.extend(["dependency:build-classpath", output_arg.as_str(), "-Dmdep.includeScope=test", "-q"]);
let top_dir = subdir(root, &target.top.dir);
let status = ProcCommand::new(&target.top.command)
.args(&args)
.envs(&target.top.env)
.current_dir(&top_dir)
.output()
.map_err(|e| IdeError::internal(format!("cannot resolve the compile classpath: {e}")))?;
if !status.status.success() {
let stderr = String::from_utf8_lossy(&status.stderr);
let tail: Vec<&str> = stderr.lines().filter(|l| !l.trim().is_empty()).collect();
let tail = tail.iter().rev().take(5).rev().cloned().collect::<Vec<_>>().join("\n");
return Err(IdeError::internal(format!("cannot resolve the compile classpath:\n{tail}")));
}
// With `-am` every visited module writes its own file; only this module's counts.
let cp_file = target.module.join(probe);
let content = fs::read_to_string(&cp_file).map_err(|e| {
IdeError::internal(format!("cannot resolve the compile classpath: {} ({e})", cp_file.display()))
})?;
let separator = if cfg!(windows) { ';' } else { ':' };
entries.extend(content.split([separator, '\n', '\r']).map(str::trim).filter(|e| !e.is_empty()).map(str::to_owned));
entries.dedup();
Ok(entries.join(&separator.to_string()))
}
/// The `javac` step compiling `file` into `target`'s output directory.
pub fn javac_step(target: &SingleFileTarget, file: &Path, classpath: &str, javac: &str) -> IdeResult<BuildStep> {
fs::create_dir_all(&target.out_dir)
.map_err(|e| IdeError::internal(format!("cannot create {} ({e})", target.out_dir.display())))?;
let mut step = BuildStep::new(javac, &[], &target.module, Format::Gradle);
// Raw `javac` prints `path:line: error: …`, the same form Gradle forwards.
step.args.push("-encoding".into());
step.args.push("UTF-8".into());
step.args.push("-d".into());
step.args.push(target.out_dir.to_string_lossy().into_owned());
step.args.push("-cp".into());
step.args.push(classpath.to_owned());
// Classes of the module that were never built (or are older than their source) are read
// from its sources; only `file` is written, so a test compile never puts main classes
// into test-classes.
let sources = source_roots(target);
if !sources.is_empty() {
let separator = if cfg!(windows) { ";" } else { ":" };
step.args.push("-sourcepath".into());
step.args.push(sources.iter().map(|p| p.to_string_lossy()).collect::<Vec<_>>().join(separator));
step.args.push("-implicit:none".into());
}
if let Some(version) = target.java_version {
step.args.push("--release".into());
step.args.push(version.to_string());
}
step.args.push(file.to_string_lossy().into_owned());
step.env = target.top.env.clone();
Ok(step)
}
/// The module's Java source folders that exist, generated ones included; test sources see
/// the main ones too.
fn source_roots(target: &SingleFileTarget) -> Vec<PathBuf> {
let scopes: &[(&str, &str)] = if target.test {
&[("src/test/java", "target/generated-test-sources"), ("src/main/java", "target/generated-sources")]
} else {
&[("src/main/java", "target/generated-sources")]
};
let mut roots = Vec::new();
for (src, generated) in scopes {
roots.push(target.module.join(src));
let generated = target.module.join(generated);
let mut dirs: Vec<PathBuf> =
fs::read_dir(&generated).into_iter().flatten().flatten().map(|d| d.path()).filter(|p| p.is_dir()).collect();
dirs.sort();
roots.extend(dirs);
}
roots.retain(|p| p.is_dir());
roots
}
/// Resolves the classpath and plans the `javac` invocation for one Java source.
pub fn compile_file_plan(root: &Path, model: &BuildModel, file: &Path) -> IdeResult<BuildPlan> {
let target = single_file_target(root, model, file)?;
let classpath = single_file_classpath(root, model, &target)?;
let javac = find_javac(&target.top.env);
let name = file.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
Ok(BuildPlan {
system: BuildSystem::Maven,
title: format!("Compile {name}"),
steps: vec![javac_step(&target, file, &classpath, &javac)?],
})
}
#[cfg(test)]
mod tests {
use super::*;
fn write(root: &Path, rel: &str, value: &str) {
let path = root.join(rel);
fs::create_dir_all(path.parent().unwrap()).unwrap();
fs::write(path, value).unwrap();
}
fn fixture(root: &Path) -> BuildModel {
write(
root,
"pom.xml",
r#"<project><groupId>demo</groupId><artifactId>parent</artifactId><version>1</version>
<packaging>pom</packaging><modules><module>lib</module><module>app</module></modules></project>"#,
);
write(
root,
"lib/pom.xml",
r#"<project><parent><groupId>demo</groupId><artifactId>parent</artifactId><version>1</version></parent>
<artifactId>lib</artifactId></project>"#,
);
write(
root,
"app/pom.xml",
r#"<project><parent><groupId>demo</groupId><artifactId>parent</artifactId><version>1</version></parent>
<artifactId>app</artifactId><dependencies><dependency><groupId>demo</groupId><artifactId>lib</artifactId><version>1</version></dependency></dependencies></project>"#,
);
write(root, "app/src/main/java/demo/Main.java", "class Main {}");
write(root, "app/src/test/java/demo/MainTest.java", "class MainTest {}");
write(root, "lib/target/classes/Lib.class", "built");
crate::load(root, &root.join("cache"), None)
}
#[test]
fn target_finds_the_owning_module_and_the_test_scope() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let main = single_file_target(root, &model, &root.join("app/src/main/java/demo/Main.java")).unwrap();
assert_eq!(main.module, root.join("app"));
assert!(!main.test);
assert_eq!(main.out_dir, root.join("app/target/classes"));
let test = single_file_target(root, &model, &root.join("app/src/test/java/demo/MainTest.java")).unwrap();
assert!(test.test);
assert_eq!(test.out_dir, root.join("app/target/test-classes"));
}
#[test]
fn target_rejects_non_java_missing_and_unlinked_files() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
write(root, "app/pom.xml", "<project/>");
assert!(single_file_target(root, &model, &root.join("app/pom.xml")).is_err());
assert!(single_file_target(root, &model, &root.join("app/src/main/java/demo/Missing.java")).is_err());
write(root, "other/Main.java", "class Main {}");
assert!(single_file_target(root, &model, &root.join("other/Main.java")).is_err());
}
#[test]
fn javac_prefers_the_module_jdk_and_scopes_test_sources() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let file = root.join("app/src/test/java/demo/MainTest.java");
let target = single_file_target(root, &model, &file).unwrap();
let jdk = root.join("jdk");
fs::create_dir_all(jdk.join("bin")).unwrap();
let exe = if cfg!(windows) { "javac.exe" } else { "javac" };
fs::write(jdk.join("bin").join(exe), "fake").unwrap();
let mut target = target;
target.top.env.insert("JAVA_HOME".into(), jdk.to_string_lossy().into_owned());
let step = javac_step(&target, &file, "/repo/a.jar", &find_javac(&target.top.env)).unwrap();
assert!(step.program.ends_with(exe));
assert!(target.out_dir.is_dir());
let args = step.args.join(" ");
assert!(args.contains("-encoding UTF-8"), "{args}");
assert!(args.contains("-d ") && args.contains("test-classes"), "{args}");
assert!(args.contains("-cp /repo/a.jar"), "{args}");
assert!(args.contains("MainTest.java"), "{args}");
// Test sources resolve their own and the main sources, without writing them.
let sourcepath = step.args.iter().position(|a| a == "-sourcepath").map(|i| step.args[i + 1].clone());
let roots = sourcepath.expect("a -sourcepath");
assert!(roots.contains("src/test/java") && roots.contains("src/main/java"), "{roots}");
assert!(step.args.iter().any(|a| a == "-implicit:none"), "{args}");
// No `JAVA_HOME`: plain `javac` from `PATH`.
assert_eq!(find_javac(&BTreeMap::new()), exe);
}
/// The classpath probe runs the project's Maven command with `-pl … -am`
/// `dependency:build-classpath`; the fake `mvnw` below only records its
/// arguments and writes a canned classpath file for the `-pl` module.
#[cfg(unix)]
#[test]
fn classpath_probe_writes_one_file_per_module_and_reads_the_compiled_one() {
use std::os::unix::fs::PermissionsExt;
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
fixture(root);
let script = root.join("mvnw");
fs::write(
&script,
"#!/bin/sh\npl='.'\nwhile [ $# -gt 0 ]; do\nif [ \"$1\" = \"-pl\" ]; then pl=\"$2\"; shift 2; else shift; fi\ndone\nmkdir -p \"$pl/target\"\nprintf '/repo/a.jar:/repo/b.jar' > \"$pl/target/ide-compile-cp.txt\"\n",
)
.unwrap();
fs::set_permissions(&script, fs::Permissions::from_mode(0o755)).unwrap();
let model = crate::load(root, &root.join("cache"), None);
let file = root.join("app/src/main/java/demo/Main.java");
let target = single_file_target(root, &model, &file).unwrap();
assert!(target.top.command.ends_with("mvnw"));
assert_eq!(target.pl, "app");
let cp = single_file_classpath(root, &model, &target).unwrap();
let separator = ':';
// Reactor class directories first, so fresh outputs shadow installed snapshots.
let lib = root.join("lib/target/classes").to_string_lossy().into_owned();
assert!(cp.starts_with(&lib), "{cp}");
// The module's own classes come first once they exist.
fs::create_dir_all(root.join("app/target/classes")).unwrap();
let cp = single_file_classpath(root, &model, &target).unwrap();
let own = root.join("app/target/classes").to_string_lossy().into_owned();
assert!(cp.starts_with(&format!("{own}:{lib}")), "{cp}");
assert!(cp.contains("/repo/a.jar"), "{cp}");
assert!(cp.contains("/repo/b.jar"), "{cp}");
assert!(cp.split(separator).count() >= 3, "{cp}");
}
}
+253
View File
@@ -0,0 +1,253 @@
//! Up-to-date checks for menu builds (`Build Project` / `Build Module`, #2476).
//!
//! Menu builds have no fingerprint record on the launch path, so a second
//! click always pays a full offline Maven run. This module stores one JSON
//! record per scope (project, or one module file's build) after a successful
//! build and compares it before the next build; on a match the build is
//! skipped entirely. Like launch reuse, timestamp-preserving external edits
//! are re-stamped first so Maven actually recompiles them on the next run.
use super::*;
use std::fs;
/// Record of the last successful menu build of one scope.
#[derive(serde::Serialize, serde::Deserialize)]
struct Record {
fingerprint: LaunchFingerprint,
/// Output folders of the built projects that existed after the build, relative to the
/// project. Deleting one outside the IDE (`mvn clean` in a terminal) leaves every input
/// unchanged, so a build is skipped only while they are all still there.
#[serde(default)]
outputs: Vec<PathBuf>,
}
/// The build's output folders that exist now, relative to `root`.
fn existing_outputs(root: &Path, fingerprint: &LaunchFingerprint) -> Vec<PathBuf> {
fingerprint
.project_dirs()
.flat_map(|dir| ["target/classes", "target/test-classes"].map(|out| dir.join(out)))
.filter(|rel| root.join(rel).is_dir())
.collect()
}
fn record_path(cache: &Path, root: &Path, file: Option<&Path>) -> PathBuf {
let base = cache.join("menu-build").join("records");
match file {
Some(f) => base.join(f.strip_prefix(root).unwrap_or(f)).with_extension("json"),
None => base.join("project.json"),
}
}
/// Re-stamps files whose content changed without advancing mtime (git
/// checkout, external sync): timestamp-incremental Maven would otherwise skip
/// them as "unchanged" and the rebuild would compile nothing.
fn restamp(root: &Path, edits: &[PathBuf]) {
let now = std::time::SystemTime::now();
for rel in edits {
let path = root.join(rel);
let _ = fs::File::options().write(true).open(&path).and_then(|f| f.set_modified(now)).or_else(|_| {
fs::OpenOptions::new()
.create(true)
.truncate(false)
.write(true)
.open(&path)
.and_then(|f| f.set_modified(now))
});
}
}
/// Whether the last successful menu build of this scope is still fresh: its
/// record exists and the fingerprint matches. On a mismatch caused by
/// timestamp-preserving external edits the files are re-stamped first, so the
/// rebuild Maven runs actually recompiles them.
pub fn menu_up_to_date(
cache: &Path,
root: &Path,
model: &BuildModel,
file: Option<&Path>,
cancel: &CancellationToken,
) -> bool {
let current = match menu_fingerprint(root, model, file, cancel) {
Ok(Some(current)) => current,
_ => return false,
};
let bytes = match fs::read(record_path(cache, root, file)) {
Ok(bytes) => bytes,
_ => return false,
};
let record: Record = match serde_json::from_slice(&bytes) {
Ok(record) => record,
_ => return false,
};
if record.fingerprint != current {
restamp(root, &record.fingerprint.mtime_preserved_edits(&current));
return false;
}
record.outputs.iter().all(|out| root.join(out).is_dir())
}
/// Records a successful menu build as the baseline for future up-to-date
/// checks. Failures are swallowed: a missing record only costs one Maven run.
pub fn record_menu_build(
cache: &Path,
root: &Path,
model: &BuildModel,
file: Option<&Path>,
cancel: &CancellationToken,
) {
let current = match menu_fingerprint(root, model, file, cancel) {
Ok(Some(current)) => current,
_ => return,
};
// A torn write fails to parse later and is treated as "no record", so a
// plain write is enough.
let path = record_path(cache, root, file);
if path.parent().is_some_and(|p| fs::create_dir_all(p).is_err()) {
return;
}
let outputs = existing_outputs(root, &current);
let bytes = match serde_json::to_vec(&Record { fingerprint: current, outputs }) {
Ok(bytes) => bytes,
_ => return,
};
let _ = fs::write(&path, bytes);
}
#[cfg(test)]
mod tests {
use super::*;
fn write(root: &Path, rel: &str, value: &str) {
let path = root.join(rel);
fs::create_dir_all(path.parent().unwrap()).unwrap();
fs::write(path, value).unwrap();
}
fn fixture(root: &Path) -> BuildModel {
maven_two_module(root, "int x;");
BuildModel { maven: Some(maven::load(root)), ..Default::default() }
}
fn maven_two_module(root: &Path, extra_app: &str) {
write(
root,
"pom.xml",
r#"<project><modelVersion>4.0.0</modelVersion><groupId>t</groupId><artifactId>root</artifactId><version>1</version><packaging>pom</packaging><modules><module>app</module><module>lib</module></modules></project>"#,
);
write(
root,
"app/pom.xml",
r#"<project><modelVersion>4.0.0</modelVersion><parent><groupId>t</groupId><artifactId>root</artifactId><version>1</version></parent><artifactId>app</artifactId><dependencies><dependency><groupId>t</groupId><artifactId>lib</artifactId><version>1</version></dependency></dependencies></project>"#,
);
write(
root,
"lib/pom.xml",
r#"<project><modelVersion>4.0.0</modelVersion><parent><groupId>t</groupId><artifactId>root</artifactId><version>1</version></parent><artifactId>lib</artifactId></project>"#,
);
write(root, "app/src/main/java/t/A.java", &format!("package t; public class A {{ {extra_app} }}"));
write(root, "lib/src/main/java/t/L.java", "package t; public class L {}");
}
#[test]
fn record_then_skip_without_changes() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let file = root.join("app/src/main/java/t/A.java");
let cancel = CancellationToken::new();
assert!(!menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
record_menu_build(&cache, root, &model, Some(&file), &cancel);
assert!(menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
// The project scope is independent of the module scope.
assert!(!menu_up_to_date(&cache, root, &model, None, &cancel));
record_menu_build(&cache, root, &model, None, &cancel);
assert!(menu_up_to_date(&cache, root, &model, None, &cancel));
}
#[test]
fn deleted_outputs_invalidate_the_record() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let cancel = CancellationToken::new();
for out in ["app/target/classes/t", "lib/target/classes/t", "app/target/test-classes"] {
fs::create_dir_all(root.join(out)).unwrap();
}
record_menu_build(&cache, root, &model, None, &cancel);
assert!(menu_up_to_date(&cache, root, &model, None, &cancel));
// `mvn clean` outside the IDE: no input changed, but the classes are gone.
fs::remove_dir_all(root.join("lib/target")).unwrap();
assert!(!menu_up_to_date(&cache, root, &model, None, &cancel));
fs::create_dir_all(root.join("lib/target/classes")).unwrap();
assert!(menu_up_to_date(&cache, root, &model, None, &cancel));
}
#[test]
fn edit_invalidates_and_rerecord_restores_skip() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let file = root.join("app/src/main/java/t/A.java");
let cancel = CancellationToken::new();
record_menu_build(&cache, root, &model, Some(&file), &cancel);
assert!(menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
write(root, "app/src/main/java/t/A.java", "package t; public class A { int y; }");
assert!(!menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
record_menu_build(&cache, root, &model, Some(&file), &cancel);
assert!(menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
}
#[test]
fn mtime_preserved_edit_forces_rebuild_and_restamps() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let rel = "app/src/main/java/t/A.java";
let file = root.join(rel);
let cancel = CancellationToken::new();
record_menu_build(&cache, root, &model, Some(&file), &cancel);
let before = fs::metadata(&file).unwrap().modified().unwrap();
// Same length, same mtime, different content: an external edit that
// preserved the timestamp.
write(root, rel, "package t; public class A { int z; }");
fs::File::options().write(true).open(&file).and_then(|f| f.set_modified(before)).unwrap();
assert!(!menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
assert!(fs::metadata(&file).unwrap().modified().unwrap() > before);
}
#[test]
fn corrupt_record_never_skips() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let file = root.join("app/src/main/java/t/A.java");
let cancel = CancellationToken::new();
fs::create_dir_all(cache.join("menu-build").join("records").join("app/src/main/java/t")).unwrap();
fs::write(cache.join("menu-build").join("records").join("app/src/main/java/t/A.json"), b"{not json").unwrap();
assert!(!menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
}
#[test]
fn unsupported_builds_never_skip() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
write(root, "other/Main.java", "public class Main {}");
let cache = root.join("cache");
let cancel = CancellationToken::new();
// A file outside any non-aggregator module has no fingerprint.
let stray = root.join("other/Main.java");
record_menu_build(&cache, root, &model, Some(&stray), &cancel);
assert!(!menu_up_to_date(&cache, root, &model, Some(&stray), &cancel));
}
}
+22 -1
View File
@@ -8,6 +8,24 @@ def canon = { File f -> f.canonicalPath }
// the identity path (`:lib:core`, `:buildSrc`) tells them apart and addresses their tasks
// from the root build. Internal API, hence the fallback.
def identity = { Project p -> try { p.identityPath.toString() } catch (ignored) { p.path } }
// The task copying a source set's resources and its output folder, when it changes them on the
// way (filters, `expand`, renames, includes / excludes, extra sources): jdtls copies resources as
// they are, so launches take them from Gradle. Internal API, hence the fallback.
def processedResources = { Project p, s ->
try {
def task = p.tasks.findByName(s.processResourcesTaskName)
if (task == null) return null
def specs = 0
def changes = task.mainSpec.sourcePaths.size() > 1
task.rootSpec.buildRootResolver().walk { r ->
specs++
changes |= !r.allCopyActions.isEmpty() || !r.allIncludes.isEmpty() || !r.allExcludes.isEmpty() ||
!r.allIncludeSpecs.isEmpty() || !r.allExcludeSpecs.isEmpty() || r.destPath.segments.length > 0
}
// The root and main specs exist for every task; more come from `from(…) { … }`.
(changes || specs > 2) ? [task: task.name, dir: canon(task.destinationDir)] : null
} catch (ignored) { null }
}
def describe = { Project p ->
def m = [path: identity(p), dir: canon(p.projectDir), buildDir: canon(p.layout.buildDirectory.get().asFile)]
@@ -26,7 +44,10 @@ def describe = { Project p ->
def test = s.name == 'test' || s.name == 'testFixtures' || suites.contains(s.name) ||
s.name.endsWith('Test') || s.name.endsWith('Tests') ||
(!src.isEmpty() && src.every { ideaTests.contains(it) })
[name: s.name, test: test, src: src, res: res]
def set = [name: s.name, test: test, src: src, res: res]
def processed = processedResources(p, s)
if (processed != null) set.processed = processed
set
}
}
try {
+13
View File
@@ -335,6 +335,19 @@ pub struct ReportedSourceSet {
/// Absolute resource folders.
#[serde(default)]
pub res: Vec<String>,
/// Set when Gradle changes the resources while copying them (filters, `expand`, renames,
/// includes / excludes, extra sources), which the Java language server's copy does not.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub processed: Option<ProcessedResources>,
}
/// The task that copies a source set's resources, and where it writes them.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct ProcessedResources {
/// Task name in its project (`processResources`).
pub task: String,
/// Absolute output folder.
pub dir: String,
}
const MODEL_LINE: &str = "@openidea-model\t";
+11 -1
View File
@@ -528,7 +528,17 @@ mod tests {
for (step, name) in plan.steps.iter().zip(["java", "kotlin"]) {
assert_eq!(step.cwd, root.join(name));
assert_eq!(step.program, format!("{name}-mvnw"));
assert_eq!(step.args, ["-B", "-Dstyle.color=never", "-o", "-Peditor", "test-compile"]);
assert_eq!(
step.args,
[
"-B",
"-Dstyle.color=never",
crate::maven::LOCAL_REPO_TRACKING_ARG,
"-o",
"-Peditor",
"test-compile"
]
);
assert_eq!(step.env.get("JAVA_HOME").map(String::as_str), Some("/jdk"));
}
}
+8
View File
@@ -18,6 +18,14 @@ use std::sync::{Mutex, OnceLock};
pub mod sources;
/// Lets the IDE's own Maven runs (Reload, Build) use what the local repository already
/// has. Maven 3.9 re-resolves an artifact whose recorded origin repository (`_remote.repositories`)
/// isn't declared in the build, such as a vendor jar downloaded through another project or
/// installed by hand, and fails when no declared repository has it. Tracking under another
/// file name makes such artifacts count as locally installed. Maven goals the user runs keep
/// plain Maven's behaviour.
pub const LOCAL_REPO_TRACKING_ARG: &str = "-Daether.enhancedLocalRepository.trackingFilename=openidea.repositories";
/// Phases shown under "Lifecycle" (IDEA's default list).
pub const LIFECYCLE: &[&str] =
&["clean", "validate", "compile", "test", "package", "verify", "install", "site", "deploy"];
+32 -4
View File
@@ -73,6 +73,9 @@ struct GradleCache {
/// The projects as Gradle evaluated them; empty in caches from before they were read.
#[serde(default)]
projects: Vec<ReportedProject>,
/// The projects report how their resources are processed; older caches are stale.
#[serde(default)]
resources: bool,
}
/// Init script printing every project, its source sets and Java version once configured.
@@ -244,11 +247,12 @@ pub fn apply_gradle_cache(root: &Path, cache_dir: &Path, model: &mut GradleModel
}
}
}
// A successful sync from before projects were cached has not seen computed includes.
let without_projects = cache.error.is_none() && cache.projects.is_empty();
// A successful sync from before projects were cached has not seen computed includes,
// nor, before resources were read, which launches need Gradle to process them.
let outdated = cache.error.is_none() && (cache.projects.is_empty() || !cache.resources);
model.synced_at = Some(cache.at_ms);
model.sync_error = cache.error;
model.stale = without_projects || cache.hash != hash_gradle_files(&gradle::build_files(root, model));
model.stale = outdated || cache.hash != hash_gradle_files(&gradle::build_files(root, model));
}
// ---------------------------------------------------------------- processes
@@ -341,7 +345,13 @@ pub fn sync_maven(
// `-T1` keeps mvnd serial: its default parallel build interleaves the
// `dependency:tree` output of concurrent modules, which the sync parses
// as one stream. mvn is serial anyway, so this only pins the daemon.
let mut args: Vec<String> = vec!["-B".into(), "-fae".into(), "-Dstyle.color=never".into(), "-T1".into()];
let mut args: Vec<String> = vec![
"-B".into(),
"-fae".into(),
"-Dstyle.color=never".into(),
"-T1".into(),
maven::LOCAL_REPO_TRACKING_ARG.into(),
];
args.extend(sync_maven_options(&options.maven_options));
if options.offline {
args.push("-o".into());
@@ -456,6 +466,7 @@ pub fn sync_gradle_model(
error: (!out.success).then(|| error_summary(&out.text)),
tasks: if out.success { gradle::parse_tasks(&out.text) } else { Vec::new() },
projects,
resources: true,
};
write_cache(&cache_dir, GRADLE_CACHE, &cache)
}
@@ -707,6 +718,7 @@ mod tests {
test,
src: src.iter().map(|p| core_src(p)).collect(),
res: res.iter().map(|p| core_src(p)).collect(),
processed: None,
};
// A settings script's own println comes first; task listing lines follow.
let out = format!(
@@ -731,6 +743,15 @@ mod tests {
);
let projects = parse_reported_projects(&out);
assert_eq!(projects.len(), 3, "the malformed line is skipped");
// As the init script prints a set whose resources Gradle filters or renames.
let processed = parse_reported_projects(&format!(
"@openidea-model\t{}",
r#"{"path":":","dir":"/p","sourceSets":[{"name":"main","test":false,"src":[],"res":[],"processed":{"task":"processResources","dir":"/p/build/resources/main"}}]}"#
));
assert_eq!(
processed[0].source_sets[0].processed,
Some(gradle::ProcessedResources { task: "processResources".into(), dir: "/p/build/resources/main".into() })
);
let src = |p: &str| PathBuf::from(core_src(p));
assert_eq!(
gradle::reported_source_roots(&projects),
@@ -796,12 +817,18 @@ mod tests {
..Default::default()
})
.collect(),
resources: true,
};
write_cache(&cache_dir, GRADLE_CACHE, &cache).unwrap();
apply_gradle_cache(&root, &cache_dir, &mut model);
assert_eq!(model.projects[1].tasks.len(), 1);
assert!(model.projects[0].tasks.iter().any(|t| t.name == "wrapper"), "root keeps its defaults");
assert!(!model.stale);
write_cache(&cache_dir, GRADLE_CACHE, &GradleCache { resources: false, ..cache.clone() }).unwrap();
let mut m = gradle::load(&root).unwrap();
apply_gradle_cache(&root, &cache_dir, &mut m);
assert!(m.stale, "synced before resource processing was read");
write_cache(&cache_dir, GRADLE_CACHE, &cache).unwrap();
let stale_after = |script: &str| {
std::fs::write(root.join("app/build.gradle"), script).unwrap();
@@ -835,6 +862,7 @@ mod tests {
dir: root.to_string_lossy().into_owned(),
..Default::default()
}],
resources: true,
..Default::default()
};
write_cache(&cache_dir, GRADLE_CACHE, &cache).unwrap();
+23
View File
@@ -73,6 +73,8 @@ pub struct EditorAppearanceSettings {
pub whitespace_inner: bool,
pub whitespace_trailing: bool,
pub indent_guides: bool,
/// Colour indent guides by depth (`--syn-rainbow-*`), while `indent_guides` is on.
pub rainbow_indent_guides: bool,
/// Right margin and visual guides from Code Style.
pub hard_wrap_guides: bool,
pub intention_bulb: bool,
@@ -95,6 +97,7 @@ impl Default for EditorAppearanceSettings {
whitespace_inner: true,
whitespace_trailing: true,
indent_guides: true,
rainbow_indent_guides: false,
hard_wrap_guides: true,
intention_bulb: true,
distraction_free_width: 80,
@@ -330,6 +333,23 @@ pub struct InlayHintsSettings {
pub code_vision_usages: bool,
/// Code vision: last author of a declaration (VCS blame) beside its usages.
pub code_vision_author: bool,
/// Code vision: "no usages" on declarations nothing uses (IDEA shows it; clicking searches everywhere).
pub code_vision_no_usages: bool,
/// Code vision: where the usages go, above the declaration or at the end of its line.
pub code_vision_usages_position: CodeVisionPosition,
/// Code vision: where the author goes, above the declaration or at the end of its line.
pub code_vision_author_position: CodeVisionPosition,
}
/// Editor › Inlay Hints › Code Vision position (IDEA's "Default position for metrics").
#[derive(Clone, Copy, Debug, Default, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum CodeVisionPosition {
/// A row of its own above the declaration.
#[default]
Top,
/// After the end of the declaration's first line.
Right,
}
impl Default for InlayHintsSettings {
@@ -341,6 +361,9 @@ impl Default for InlayHintsSettings {
disabled_languages: Vec::new(),
code_vision_usages: true,
code_vision_author: true,
code_vision_no_usages: true,
code_vision_usages_position: CodeVisionPosition::Top,
code_vision_author_position: CodeVisionPosition::Top,
}
}
}
+37
View File
@@ -273,6 +273,27 @@ pub enum DebugEvent {
change_type: String,
message: String,
},
/// The session's capabilities changed after it started: fetch them again.
Capabilities,
JvmBackend {
state: JvmBackendState,
},
}
/// The backend that reads a debugged JVM's objects (instances, fields, referrers).
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmBackendState {
Ready,
/// Turned off for the rest of the session: the debug adapter lacks what it needs.
Unsupported {
detail: String,
},
/// The Kotlin JDWP link only forwards now; inspecting stops.
Degraded {
detail: String,
},
Closed,
}
/// Where backend crates publish [`IdeEvent`]s. Cheap to clone via `Arc`.
@@ -294,3 +315,19 @@ pub struct NullSink;
impl EventSink for NullSink {
fn emit(&self, _: IdeEvent) {}
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn jvm_backend_events_json_shape() {
let degraded = DebugEvent::JvmBackend { state: JvmBackendState::Degraded { detail: "x".into() } };
assert_eq!(
serde_json::to_value(&degraded).unwrap(),
json!({"kind": "jvmBackend", "state": {"kind": "degraded", "detail": "x"}})
);
assert_eq!(serde_json::to_value(DebugEvent::Capabilities).unwrap(), json!({"kind": "capabilities"}));
}
}
+26
View File
@@ -21,6 +21,32 @@
use std::collections::BTreeMap;
/// Removes ANSI escape sequences (colour, cursor movement).
pub fn strip_ansi(s: &str) -> String {
if !s.contains('\u{1b}') {
return s.to_owned();
}
let mut out = String::with_capacity(s.len());
let mut chars = s.chars().peekable();
while let Some(c) = chars.next() {
if c != '\u{1b}' {
out.push(c);
continue;
}
if chars.peek() == Some(&'[') {
chars.next();
for c in chars.by_ref() {
if c.is_ascii_alphabetic() || c == '~' {
break;
}
}
} else {
chars.next();
}
}
out
}
/// Decodes complete lines or collected output from IDE-managed tools, preferring UTF-8.
/// Windows launcher/native diagnostics fall back to the system ANSI code page.
/// Callers must buffer incomplete lines so multibyte characters are never split.
+112
View File
@@ -72,6 +72,75 @@ pub fn kill_tree_of(pid: u32) {
let _ = pid;
}
/// Whether a process with this id exists. PID reuse makes this conservative: a reused id keeps
/// whatever was tied to the old process alive, a live process is never reported dead.
pub fn process_alive(pid: u32) -> bool {
// `kill(0, 0)` would signal the caller's own process group.
if pid == 0 {
return false;
}
#[cfg(unix)]
{
// Above `i32::MAX` the id would wrap to a negative value, which `kill` reads as a group.
let Ok(pid) = libc::pid_t::try_from(pid) else { return false };
// SAFETY: signal 0 delivers nothing; it only checks that the process exists and is reachable.
unsafe { libc::kill(pid, 0) == 0 || std::io::Error::last_os_error().raw_os_error() == Some(libc::EPERM) }
}
#[cfg(windows)]
{
use std::os::windows::io::{AsRawHandle, FromRawHandle, OwnedHandle};
use windows_sys::Win32::Foundation::{ERROR_ACCESS_DENIED, GetLastError, STILL_ACTIVE};
use windows_sys::Win32::System::Threading::{
GetExitCodeProcess, OpenProcess, PROCESS_QUERY_LIMITED_INFORMATION,
};
// SAFETY: plain call; a non-null result is a process handle owned by this function.
let handle = unsafe { OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, 0, pid) };
if handle.is_null() {
// A process we may not open still exists.
// SAFETY: reads the thread's last error, set by the call above.
return unsafe { GetLastError() } == ERROR_ACCESS_DENIED;
}
// SAFETY: `handle` is valid and nobody else owns it.
let handle = unsafe { OwnedHandle::from_raw_handle(handle) };
let mut code = 0u32;
// SAFETY: a live process handle opened with query access; one out-parameter.
let ok = unsafe { GetExitCodeProcess(handle.as_raw_handle(), &mut code) };
// A handle that cannot be queried is not evidence that the process is gone.
ok == 0 || code == STILL_ACTIVE as u32
}
#[cfg(not(any(unix, windows)))]
{
true
}
}
/// The program and everything it started: an adapter may report the launcher shell, whose
/// child is the debugged program, so stopping only the reported pid leaves the program alive.
#[cfg(unix)]
pub fn process_tree(pid: i32) -> Vec<i32> {
use std::collections::HashMap;
let Ok(out) = std::process::Command::new("ps").args(["-A", "-o", "pid=,ppid="]).output() else {
return vec![pid];
};
let mut children: HashMap<i32, Vec<i32>> = HashMap::new();
for line in String::from_utf8_lossy(&out.stdout).lines() {
let mut it = line.split_whitespace();
let (Some(p), Some(parent)) = (it.next(), it.next()) else { continue };
let (Ok(p), Ok(parent)) = (p.parse::<i32>(), parent.parse::<i32>()) else { continue };
children.entry(parent).or_default().push(p);
}
let mut tree = vec![pid];
let mut at = 0;
while at < tree.len() {
if let Some(kids) = children.get(&tree[at]) {
tree.extend(kids.iter().copied());
}
at += 1;
}
tree
}
/// [`kill_tree`] that first asks the group to exit: `SIGTERM`, then `grace`, then `SIGKILL`.
/// Git removes its `index.lock` on `SIGTERM`, where `SIGKILL` leaves it behind and the next
/// git command fails. Windows has no such signal to send from outside the child's console,
@@ -730,6 +799,30 @@ fn append_path_entries(path: &OsStr, extra: &OsStr) -> OsString {
mod tests {
use super::*;
#[test]
fn process_alive_tells_live_from_dead_pids() {
assert!(process_alive(std::process::id()));
assert!(!process_alive(0));
let mut cmd = if cfg!(windows) {
let mut cmd = Command::new("cmd");
cmd.args(["/C", "exit 0"]);
cmd
} else {
Command::new("true")
};
let mut child = cmd.spawn().unwrap();
let pid = child.id();
child.wait().unwrap();
drop(child);
assert!(!process_alive(pid));
}
#[cfg(unix)]
#[test]
fn process_alive_rejects_ids_that_are_not_a_single_process() {
assert!(!process_alive(u32::MAX));
}
#[cfg(unix)]
#[test]
fn kill_tree_of_reaches_what_the_child_started() {
@@ -1050,6 +1143,25 @@ mod tests {
}
assert_eq!(find_executable("nope", &search), None);
}
#[cfg(unix)]
#[test]
fn process_tree_lists_the_child_of_a_shell() {
let mut cmd = Command::new("sh");
own_group(&mut cmd).args(["-c", "sleep 5 & wait"]);
let mut child = cmd.spawn().unwrap();
let sh_pid = child.id() as i32;
// The shell forks `sleep` after it starts, so the tree grows a moment later.
let deadline = std::time::Instant::now() + Duration::from_secs(2);
let mut tree = process_tree(sh_pid);
while tree.len() < 2 && std::time::Instant::now() < deadline {
std::thread::sleep(Duration::from_millis(20));
tree = process_tree(sh_pid);
}
kill_tree(&mut child);
assert_eq!(tree[0], sh_pid, "the pid itself comes first");
assert!(tree.len() >= 2 && tree[1..].iter().all(|p| *p != sh_pid), "the shell's child follows: {tree:?}");
}
}
#[cfg(all(test, unix))]
+93 -1
View File
@@ -104,6 +104,11 @@ pub struct AgentSettings {
#[serde(deserialize_with = "agent_defaults")]
#[specta(type = BTreeMap<String, BTreeMap<String, String>>)]
pub defaults: BTreeMap<String, BTreeMap<String, String>>,
/// MCP servers passed to coding agents in `session/new`, `session/load` and `session/fork`,
/// keyed by server name (`[agents.mcpServers.<name>]`).
#[serde(deserialize_with = "or_default")]
#[specta(type = BTreeMap<String, AgentMcpServer>)]
pub mcp_servers: BTreeMap<String, AgentMcpServer>,
}
impl Default for AgentSettings {
@@ -121,6 +126,58 @@ impl Default for AgentSettings {
send_with: SendKey::Enter,
while_running: WhileRunning::Queue,
defaults: BTreeMap::new(),
mcp_servers: BTreeMap::new(),
}
}
}
/// Transport of an MCP server configured in Settings › Agents › MCP Servers.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum McpTransport {
#[default]
Stdio,
Http,
Sse,
}
/// An MCP server passed to coding agents in `session/new`, `session/load` and `session/fork`.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct AgentMcpServer {
#[serde(deserialize_with = "or_default")]
#[specta(type = McpTransport)]
pub transport: McpTransport,
pub enabled: bool,
#[serde(skip_serializing_if = "Option::is_none")]
#[specta(optional)]
pub command: Option<String>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub args: Vec<String>,
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub env: BTreeMap<String, String>,
#[serde(skip_serializing_if = "Option::is_none")]
#[specta(optional)]
pub url: Option<String>,
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub headers: BTreeMap<String, String>,
/// Agent provider ids this server is enabled for (e.g. `["claude", "codex"]`);
/// empty means enabled for all agents.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub agents: Vec<String>,
}
impl Default for AgentMcpServer {
fn default() -> Self {
Self {
transport: McpTransport::Stdio,
enabled: true,
command: None,
args: Vec::new(),
env: BTreeMap::new(),
url: None,
headers: BTreeMap::new(),
agents: Vec::new(),
}
}
}
@@ -204,6 +261,9 @@ pub enum WhileRunning {
pub struct McpServerSettings {
/// Serve the tools on 127.0.0.1 while the IDE runs.
pub enabled: bool,
/// Run the tools that change the project or start processes without asking first
/// (IDEA's brave mode). Read from the IDE's own settings only, never a project's.
pub run_without_confirmation: bool,
}
/// Absent profiles retain automatic discovery for existing projects.
@@ -396,6 +456,10 @@ pub struct GeneralSettings {
/// Where updates come from: `auto` (the CNB mirror for the Chinese UI, else GitHub) | `github` | `cnb`.
pub update_source: String,
pub auto_check_updates: bool,
/// System Settings › Updates: open the What's New page after the app starts a new version.
pub show_whats_new_after_update: bool,
/// The version the app last started with, so the first start after an update is recognisable.
pub last_run_version: String,
}
impl Default for GeneralSettings {
@@ -414,6 +478,8 @@ impl Default for GeneralSettings {
project_name: String::new(),
update_source: "auto".into(),
auto_check_updates: true,
show_whats_new_after_update: true,
last_run_version: String::new(),
}
}
}
@@ -517,7 +583,7 @@ pub struct AppearanceSettings {
pub project_colors_in_toolbar: bool,
pub dnd_with_alt_only: bool,
pub menu_icons: bool,
/// `hamburger` | `merged`: main menu presentation in the frameless title bar.
/// `hamburger` | `merged` | `toolbar` (a separate row): main menu presentation in the frameless title bar.
pub main_menu_style: String,
/// A lone Alt press focuses or opens the main menu (IDEA `ide.windows.main.menu.focus.on.alt`).
pub main_menu_alt: bool,
@@ -1273,6 +1339,11 @@ pub struct ProjectStructureSettings {
pub unmarked: Vec<String>,
/// Content roots besides the project folder (absolute paths), shown and searched with it.
pub content_roots: Vec<String>,
/// Other projects (absolute paths) this project depends on; indexed and navigable like content roots.
pub dependencies: Vec<String>,
/// Per-workspace-member dependencies: member root (IPC path) → the roots it depends on.
/// A member not listed here (or with an empty list) navigates only into itself.
pub member_dependencies: BTreeMap<String, Vec<String>>,
/// "Exclude files": `;`-separated name patterns (`*.log;tmp*`) excluded at any depth.
pub exclude_files: String,
/// Project compiler output directory (relative to project root or absolute; empty = default).
@@ -1446,6 +1517,27 @@ mod tests {
}
}
#[test]
fn double_click_opens_diff_is_unset_by_default_and_round_trips() {
let old: Settings = serde_json::from_str(r#"{"vcs":{"commit":{"nonModal":true}}}"#).unwrap();
assert_eq!(old.vcs.commit.double_click_opens_diff, None);
// The unset value shows up as `null` in the defaults: `SettingsStore::set` only
// accepts a key path it finds there.
let json = serde_json::to_value(Settings::default()).unwrap();
assert_eq!(json["vcs"]["commit"]["doubleClickOpensDiff"], serde_json::Value::Null);
for on in [true, false] {
let mut settings = Settings::default();
settings.vcs.commit.double_click_opens_diff = Some(on);
let json = serde_json::to_value(&settings).unwrap();
assert_eq!(json["vcs"]["commit"]["doubleClickOpensDiff"], on);
let restored: Settings = serde_json::from_value(json).unwrap();
assert_eq!(restored.vcs.commit.double_click_opens_diff, Some(on));
}
// Explicit `null` reads back as unset.
let cleared: Settings = serde_json::from_str(r#"{"vcs":{"commit":{"doubleClickOpensDiff":null}}}"#).unwrap();
assert_eq!(cleared.vcs.commit.double_click_opens_diff, None);
}
#[test]
fn agent_meters_default_and_round_trip() {
let old: Settings = serde_json::from_str(r#"{"agents":{"showCost":true}}"#).unwrap();
+12
View File
@@ -43,6 +43,7 @@ impl Default for VcsSettings {
pub struct GitSettings {
/// Show blame after the current editor line.
pub inline_blame: bool,
pub show_branch_in_project_view: bool,
/// Path to the git executable; empty = auto-detect.
pub executable: String,
/// Default of Update Project: `merge` | `rebase`.
@@ -74,6 +75,7 @@ impl Default for GitSettings {
fn default() -> Self {
Self {
inline_blame: false,
show_branch_in_project_view: false,
executable: String::new(),
update_method: "merge".into(),
auto_update_on_push_rejected: false,
@@ -115,6 +117,13 @@ pub struct CommitSettings {
pub wrap_at_margin: bool,
/// Plugin before-commit checks turned off (`<plugin id>/<check id>`); the rest run.
pub disabled_plugin_checks: Vec<String>,
/// Double-click / Enter in the changes view: `None` follows the interface (the
/// Commit tool window opens the diff, Local Changes the source file).
///
/// No `skip_serializing_if`: an unset value must still appear in
/// `Settings::default()`, or `SettingsStore::set` rejects the key as unknown.
#[specta(optional)]
pub double_click_opens_diff: Option<bool>,
}
impl Default for CommitSettings {
@@ -135,6 +144,7 @@ impl Default for CommitSettings {
show_right_margin: true,
wrap_at_margin: false,
disabled_plugin_checks: Vec::new(),
double_click_opens_diff: None,
}
}
}
@@ -312,6 +322,7 @@ pub struct TerminalSettings {
pub override_ide_shortcuts: bool,
pub shell_integration: bool,
pub highlight_hyperlinks: bool,
pub highlight_selection: bool,
pub option_as_meta: bool,
/// Escape moves focus to the editor instead of reaching the shell.
pub escape_to_editor: bool,
@@ -344,6 +355,7 @@ impl Default for TerminalSettings {
override_ide_shortcuts: true,
shell_integration: true,
highlight_hyperlinks: true,
highlight_selection: true,
option_as_meta: false,
escape_to_editor: false,
cursor_shape: "block".into(),
+5
View File
@@ -11,12 +11,17 @@ tempfile.workspace = true
ide-core.workspace = true
ide-language.workspace = true
ide-jsonrpc.workspace = true
ide-jvm-attach.workspace = true
ide-jvm-inspect.workspace = true
ide-jdwp.workspace = true
ide-heap.workspace = true
serde.workspace = true
serde_json.workspace = true
specta.workspace = true
tokio = { workspace = true, features = ["net"] }
parking_lot.workspace = true
memchr = "2"
percent-encoding = "2"
portable-pty.workspace = true
tracing.workspace = true
+48 -7
View File
@@ -246,13 +246,7 @@ impl Connection {
fn dispatch(inner: &Inner, incoming: &mpsc::UnboundedSender<Incoming>, body: &[u8]) {
tracing::trace!(target: "ide_dap::wire", "<-- {}", String::from_utf8_lossy(body));
let msg: Value = match serde_json::from_slice(body) {
Ok(v) => v,
Err(e) => {
tracing::warn!("dap: unparsable message: {e}");
return;
}
};
let Some(msg) = parse_message(body) else { return };
match msg.get("type").and_then(Value::as_str) {
Some("response") => {
let Some(request_seq) = msg.get("request_seq").and_then(Value::as_i64) else {
@@ -285,6 +279,25 @@ fn dispatch(inner: &Inner, incoming: &mpsc::UnboundedSender<Incoming>, body: &[u
}
}
/// A message body as JSON. Some adapters pass the debuggee's output through as it was
/// written: netcoredbg on Windows embeds console output in the ANSI code page (GBK on a
/// Chinese system), which isn't valid UTF-8 and made the whole `output` event unparsable,
/// so every line with non-ASCII text went missing. Such a body is decoded like other tool
/// output (UTF-8, else the system code page) before parsing.
fn parse_message(body: &[u8]) -> Option<Value> {
let error = match serde_json::from_slice(body) {
Ok(v) => return Some(v),
Err(e) => e,
};
if std::str::from_utf8(body).is_err()
&& let Ok(v) = serde_json::from_str(&ide_core::output_encoding::decode_tool_output(body))
{
return Some(v);
}
tracing::warn!("dap: unparsable message: {error}");
None
}
/// `message`, or the formatted `body.error` (`{format, variables}`) when present.
fn error_message(msg: &Value) -> String {
if let Some(err) = msg.pointer("/body/error")
@@ -306,6 +319,22 @@ fn error_message(msg: &Value) -> String {
mod tests {
use super::*;
#[test]
fn parses_output_events_with_non_utf8_text() {
// "中文" in GBK inside an output event, as netcoredbg sends it on a Chinese Windows.
let mut body = br#"{"seq":5,"type":"event","event":"output","body":{"category":"stdout","output":""#.to_vec();
body.extend_from_slice(&[0xd6, 0xd0, 0xce, 0xc4]);
body.extend_from_slice(br#"\n"}}"#);
let msg = parse_message(&body).expect("parsed");
assert_eq!(msg["event"], "output");
let output = msg["body"]["output"].as_str().unwrap();
assert!(output.ends_with('\n'));
// Off Windows (UTF-8 code page) the bytes decode lossily; the line still arrives.
#[cfg(not(windows))]
assert!(output.starts_with(char::REPLACEMENT_CHARACTER));
assert!(parse_message(b"{not json").is_none());
}
#[test]
fn formats_structured_errors() {
let msg =
@@ -313,4 +342,16 @@ mod tests {
assert_eq!(error_message(&msg), "cannot find a.py");
assert_eq!(error_message(&json!({"message": "nope"})), "nope");
}
#[test]
fn a_failed_launch_response_keeps_delves_too_old_error_verbatim() {
// The UI's repair offer keys off this exact wording (`missingAdapter`), so the
// conversion to the IPC error must not fold it into a generic message.
let message = "Failed to launch /p/api: Version of Delve is too old for Go version go1.27.0 \
(maximum supported version 1.26, suppress this error with --check-go-version=false)";
let e = ide_core::IdeError::from(DapError::Failed { command: "launch".into(), message: message.into() });
assert_eq!(e.kind, ide_core::error::ErrorKind::InvalidInput);
assert_eq!(e.message, format!("launch failed: {message}"));
assert!(e.message.contains("Version of Delve is too old for Go version go1.27.0"));
}
}
+100 -13
View File
@@ -12,6 +12,35 @@ use tokio::net::{TcpListener, TcpStream};
/// How long the relay waits for java-debug to attach once the JVM is connected.
const ATTACH_TIMEOUT: Duration = Duration::from_secs(60);
/// Loopback ports relaying a connected JVM, while their relay runs.
static RELAY_PORTS: std::sync::Mutex<Vec<u16>> = std::sync::Mutex::new(Vec::new());
fn relay_ports() -> std::sync::MutexGuard<'static, Vec<u16>> {
RELAY_PORTS.lock().unwrap_or_else(std::sync::PoisonError::into_inner)
}
/// Whether `port` is a relay of [`listen_for_jvm`]: a session attaching to it debugs a JVM that
/// connected to the IDE, so the JVM has no JDWP listener the memory view could find it by.
pub(crate) fn is_listen_relay(port: u16) -> bool {
relay_ports().contains(&port)
}
/// Registers a relay port for as long as it lives.
struct RelayPort(u16);
impl RelayPort {
fn new(port: u16) -> Self {
relay_ports().push(port);
Self(port)
}
}
impl Drop for RelayPort {
fn drop(&mut self) {
relay_ports().retain(|&p| p != self.0);
}
}
/// The address to listen on for `host`: the interface it names, or every interface when
/// it is not one of this machine's (a DNS name or a NAT address the JVM connects to).
async fn bind(host: &str, port: u16) -> IdeResult<TcpListener> {
@@ -40,11 +69,11 @@ pub async fn listen_for_jvm(host: &str, port: u16, cancel: CancellationToken) ->
let _ = jvm.set_nodelay(true);
let relay = TcpListener::bind(("127.0.0.1", 0)).await?;
let local = relay.local_addr()?.port();
tokio::spawn(relay_one(relay, jvm));
tokio::spawn(relay_one(relay, jvm, RelayPort::new(local)));
Ok(local)
}
async fn relay_one(relay: TcpListener, mut jvm: TcpStream) {
async fn relay_one(relay: TcpListener, mut jvm: TcpStream, _registered: RelayPort) {
let debugger = match tokio::time::timeout(ATTACH_TIMEOUT, relay.accept()).await {
Ok(Ok((s, _))) => s,
_ => return,
@@ -58,22 +87,64 @@ async fn relay_one(relay: TcpListener, mut jvm: TcpStream) {
}
#[cfg(test)]
mod tests {
pub(crate) mod tests {
use super::*;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
#[tokio::test]
async fn relays_the_jdwp_handshake_both_ways() {
let port = TcpListener::bind(("127.0.0.1", 0)).await.unwrap().local_addr().unwrap().port();
let waiting = tokio::spawn(listen_for_jvm("127.0.0.1", port, CancellationToken::new()));
// The remote JVM (`server=n`) connects to the IDE.
let mut jvm = loop {
if let Ok(s) = TcpStream::connect(("127.0.0.1", port)).await {
break s;
/// A remote JVM (`server=n`) connected to [`listen_for_jvm`], and the relay port. A port
/// picked by binding to 0 can be taken by a parallel test before `listen_for_jvm` binds it,
/// so a failed listen is retried on a fresh one.
pub(crate) async fn connected_jvm() -> (TcpStream, u16) {
for _ in 0..20 {
let port = TcpListener::bind(("127.0.0.1", 0)).await.unwrap().local_addr().unwrap().port();
if let Some(connected) = jvm_on(port).await {
return connected;
}
}
panic!("listen_for_jvm never accepted a JVM");
}
/// One attempt of [`connected_jvm`]; `None` when the port turned out to be taken.
async fn jvm_on(port: u16) -> Option<(TcpStream, u16)> {
let mut waiting = tokio::spawn(listen_for_jvm("127.0.0.1", port, CancellationToken::new()));
let mut jvm = None;
for _ in 0..100 {
// A listen that failed to bind leaves nothing to connect to.
if waiting.is_finished() {
break;
}
// macOS does not refuse a connect to a port a socket holds without listening: the
// SYN goes unanswered until the connect times out.
let connect = TcpStream::connect(("127.0.0.1", port));
if let Ok(Ok(s)) = tokio::time::timeout(Duration::from_secs(1), connect).await {
jvm = Some(s);
break;
}
tokio::time::sleep(Duration::from_millis(20)).await;
};
let local = waiting.await.unwrap().unwrap();
}
// A connection to another test's listener never reaches `listen_for_jvm`.
let relay = tokio::time::timeout(Duration::from_secs(2), &mut waiting).await;
if let (Some(jvm), Ok(Ok(Ok(relay)))) = (jvm, relay) {
return Some((jvm, relay));
}
waiting.abort();
None
}
#[tokio::test]
async fn a_port_another_socket_holds_is_given_up() {
// Bound but not listening, as a parallel test's outgoing connection holds its local port.
let held = tokio::net::TcpSocket::new_v4().unwrap();
held.bind(([127, 0, 0, 1], 0).into()).unwrap();
let port = held.local_addr().unwrap().port();
let attempt = tokio::time::timeout(Duration::from_secs(5), jvm_on(port)).await;
assert!(matches!(attempt, Ok(None)), "an attempt on a taken port gives up instead of waiting forever");
}
#[tokio::test]
async fn relays_the_jdwp_handshake_both_ways() {
// The remote JVM (`server=n`) connects to the IDE.
let (mut jvm, local) = connected_jvm().await;
// java-debug attaches to the relay and starts the handshake.
let mut debugger = TcpStream::connect(("127.0.0.1", local)).await.unwrap();
debugger.write_all(b"JDWP-Handshake").await.unwrap();
@@ -85,6 +156,22 @@ mod tests {
assert_eq!(&buf, b"JDWP-Handshake");
}
#[tokio::test]
async fn relay_port_is_known_while_the_relay_runs() {
let (jvm, local) = connected_jvm().await;
assert!(is_listen_relay(local));
let debugger = TcpStream::connect(("127.0.0.1", local)).await.unwrap();
drop((jvm, debugger));
for _ in 0..200 {
if !is_listen_relay(local) {
return;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
panic!("relay port {local} still registered after the relay ended");
}
#[tokio::test]
async fn waiting_is_cancellable() {
let cancel = CancellationToken::new();
+92
View File
@@ -0,0 +1,92 @@
//! The class table counted by the debugger (JDWP `InstanceCounts`) where no histogram can be
//! taken: JDK 8-16 while suspended, Windows, a remote JVM. Only while suspended, and slow on a
//! large heap (seconds on JDK 17+).
use super::inspect::Call;
use super::snapshot::diff_rows;
use super::{ClassCounts, JvmMemoryResult, JvmMemorySnapshot, JvmMemorySource, JvmMemoryState, JvmMemoryUnavailable};
use crate::session::Session;
use ide_core::IdeResult;
use ide_jvm_attach::HistogramRow;
use ide_jvm_inspect::ClassCount;
/// Reasons the debugger may be able to work around; the others (the program exited, the JVM
/// did not answer) would fail for it too.
pub(super) fn falls_back(reason: &JvmMemoryUnavailable) -> bool {
use JvmMemoryUnavailable as U;
matches!(
reason,
U::UnsafeWhileSuspended { .. }
| U::UnsupportedPlatform
| U::AttachedJvmNotFound
| U::ListenerNotRunning
| U::ListenModeNotSupported
| U::NoJvmProcess
)
}
/// `histogram` unless it is unavailable for a reason `session`'s inspector can work around.
pub(super) async fn or_debugger_counts(
state: &mut JvmMemoryState,
histogram: Result<JvmMemorySnapshot, JvmMemoryUnavailable>,
session: Option<&Session>,
) -> IdeResult<JvmMemoryResult> {
let reason = match histogram {
Ok(snapshot) => return Ok(JvmMemoryResult::Ready { snapshot }),
Err(reason) => reason,
};
let unavailable = |can_count_when_paused| Ok(JvmMemoryResult::Unavailable { reason, can_count_when_paused });
let Some(session) = session.filter(|_| falls_back(&reason)) else {
return unavailable(false);
};
let call = match Call::begin(session, None).await {
Ok(call) => call,
Err(super::JvmInspectUnavailable::NotSuspended) => return unavailable(true),
Err(_) => return unavailable(false),
};
let started = std::time::Instant::now();
let result = call.run(call.inspector.class_counts(&call.target)).await;
match call.finish(result) {
Ok(counts) => {
let elapsed_ms = u32::try_from(started.elapsed().as_millis()).unwrap_or(u32::MAX);
let previous = state.baseline(JvmMemorySource::Debugger);
let (snapshot, counts) = debugger_snapshot(counts, previous.as_deref(), elapsed_ms);
state.remember(JvmMemorySource::Debugger, counts);
Ok(JvmMemoryResult::Ready { snapshot })
}
Err(e) => {
tracing::debug!("debugger class counts: {e}");
let running = !session.suspended.get();
unavailable(running && session.jvm_inspector.lock().is_some())
}
}
}
/// Classes without instances are left out; most instances first.
pub(super) fn debugger_snapshot(
counts: Vec<ClassCount>,
previous: Option<&ClassCounts>,
elapsed_ms: u32,
) -> (JvmMemorySnapshot, ClassCounts) {
let rows = counts
.into_iter()
.filter(|c| c.instances > 0)
.map(|c| HistogramRow { name: c.name, module: None, instances: c.instances, bytes: 0 })
.collect();
let (mut rows, counts) = diff_rows(rows, previous);
for row in &mut rows {
row.delta_bytes = None;
}
rows.sort_by(|a, b| b.instances.cmp(&a.instances).then_with(|| a.name.cmp(&b.name)));
let snapshot = JvmMemorySnapshot {
total_instances: rows.iter().map(|r| r.instances).sum(),
total_bytes: 0,
rows,
java_version: String::new(),
taken_while_suspended: true,
has_baseline: previous.is_some(),
elapsed_ms,
source: JvmMemorySource::Debugger,
};
(snapshot, counts)
}
+293
View File
@@ -0,0 +1,293 @@
//! A heap dump of a debugged JVM, written over its attach listener into a snapshot directory.
use super::snapshot::{AttachProbe, JdkProbe, attach_failure, gone, pick_jvm_pid};
use super::{
JvmMemoryState, JvmMemoryUnavailable, JvmTarget, histogram_pid, keep_found, lookup, resolve_jvm,
safe_while_suspended, session_target,
};
use crate::session::Session;
use ide_core::{CancellationToken, IdeError, IdeResult};
use ide_jvm_attach::{AttachError, HeapDumpReply};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::future::Future;
use std::path::{Path, PathBuf};
use std::sync::Arc;
const DUMP_FILE: &str = "dump.hprof";
/// For a refusal whose reply holds no reason: empty, or only the line announcing the dump.
const NO_REASON: &str = "the JVM wrote no heap dump and gave no reason";
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmHeapUnavailable {
/// The reason a class table could not be read applies to the dump too.
Memory {
reason: JvmMemoryUnavailable,
},
/// Bytes.
NotEnoughDiskSpace {
needed: u64,
available: u64,
},
DumpFailed {
message: String,
},
DumpTimedOut {
seconds: u32,
},
}
#[derive(Clone, Debug, PartialEq)]
pub struct JvmHeapDump {
pub path: PathBuf,
/// The class histogram taken just before the dump.
pub histogram: Vec<ide_heap::HistogramEntry>,
/// Raw `VM.flags`; empty when the JVM did not answer.
pub vm_flags: String,
/// 0 when unknown.
pub java_major: u32,
/// Empty when unknown.
pub java_version: String,
pub pid: u32,
pub taken_while_suspended: bool,
pub dump_ms: u32,
}
#[derive(Clone, Debug, PartialEq)]
pub enum JvmHeapOutcome {
Ready(JvmHeapDump),
/// The snapshot directory has been deleted.
Unavailable(JvmHeapUnavailable),
/// The snapshot directory has been deleted.
Cancelled,
}
pub(crate) async fn session_heap_dump(
session: Arc<Session>,
dir: PathBuf,
cancel: CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
let launched = histogram_pid(&session);
let capture = {
let (dir, cancel) = (dir.clone(), cancel.clone());
async move {
let target = session_target(&session, launched);
let probe = |suspended| -> Arc<dyn AttachProbe> { Arc::new(JdkProbe { suspended }) };
let stopped = session.clone();
let suspended = move || capture_suspended(&stopped);
heap_dump_with(&session.jvm_memory, suspended, probe, target, dir, cancel).await
}
};
detached(capture, dir, cancel).await
}
/// The JVM can be suspended before the adapter reports the stop, so a session with a JDWP link
/// also counts the link's view.
pub(super) fn capture_suspended(session: &Session) -> bool {
session.suspended.get()
|| session
.jdwp_link
.lock()
.as_ref()
.is_some_and(|link| matches!(link.state(), ide_jdwp::LinkState::Suspended { .. }))
}
/// Runs `capture` in a task of its own, so a caller that stops waiting cannot release the
/// session's lock while the JVM is still writing the dump and let a class-table request run
/// next to it. On `cancel` returns at once and deletes `dir`.
pub(super) async fn detached(
capture: impl Future<Output = IdeResult<JvmHeapOutcome>> + Send + 'static,
dir: PathBuf,
cancel: CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
let task = tokio::spawn(capture);
tokio::select! {
joined = task => joined.map_err(|e| IdeError::internal(format!("JVM heap dump failed: {e}")))?,
() = cancel.cancelled() => {
discard_later(dir).await;
Ok(JvmHeapOutcome::Cancelled)
}
}
}
/// Holds `state`'s lock throughout, as a class-table request does; `suspended` and then `probe`
/// are read once it is held, and `suspended` again by [`take_heap_dump`]. Anything but `Ready`
/// deletes `dir`.
pub(super) async fn heap_dump_with(
state: &tokio::sync::Mutex<JvmMemoryState>,
suspended: impl Fn() -> bool + Send + Sync + 'static,
probe: impl FnOnce(bool) -> Arc<dyn AttachProbe>,
target: JvmTarget<'_>,
dir: PathBuf,
cancel: CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
// The capture's histogram never becomes the Memory view's baseline, so the state is unused.
let _turn = state.lock().await;
let outcome = capture(suspended, probe, target, dir.clone(), cancel).await;
if !matches!(outcome, Ok(JvmHeapOutcome::Ready(_))) {
discard_later(dir).await;
}
outcome
}
async fn capture(
suspended_now: impl Fn() -> bool + Send + Sync + 'static,
probe: impl FnOnce(bool) -> Arc<dyn AttachProbe>,
target: JvmTarget<'_>,
dir: PathBuf,
cancel: CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
if cancel.is_cancelled() {
return Ok(JvmHeapOutcome::Cancelled);
}
let suspended = suspended_now();
let lookup = match lookup(target) {
Ok(lookup) => lookup,
Err(reason) => return Ok(memory(reason)),
};
let probe = probe(suspended);
let (found, outcome) = tokio::task::spawn_blocking(move || match resolve_jvm(lookup, &*probe, suspended) {
Ok((tree, found)) => (found, take_heap_dump(&*probe, &tree, suspended, &suspended_now, &dir, &cancel)),
Err(reason) => (None, Ok(memory(reason))),
})
.await
.map_err(|e| IdeError::internal(format!("JVM heap dump failed: {e}")))?;
keep_found(target, found);
outcome
}
/// `tree` starts with the session's reported pid, followed by its descendants. `suspended` is
/// the stop read when the capture began; `suspended_now` reads it again. Anything but `Ready`
/// deletes `dir`.
pub(crate) fn take_heap_dump(
probe: &dyn AttachProbe,
tree: &[i32],
suspended: bool,
suspended_now: &dyn Fn() -> bool,
dir: &Path,
cancel: &CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
let outcome = dump(probe, tree, suspended, suspended_now, dir, cancel);
if !matches!(outcome, Ok(JvmHeapOutcome::Ready(_))) {
discard(dir);
}
outcome
}
fn dump(
probe: &dyn AttachProbe,
tree: &[i32],
suspended: bool,
suspended_now: &dyn Fn() -> bool,
dir: &Path,
cancel: &CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
use JvmHeapUnavailable as H;
use JvmMemoryUnavailable as U;
let Some(root) = tree.first().and_then(|&p| u32::try_from(p).ok()) else {
return Ok(memory(U::NoJvmProcess));
};
let Some(jvm) = pick_jvm_pid(tree, |p| probe.has_listener(p)) else {
return Ok(memory(gone(probe, root)));
};
let version = probe.version(jvm);
let java_major = version.as_ref().map_or(0, |v| v.major);
let safe = safe_while_suspended(version.as_ref());
let unsafe_stop = || Ok(memory(U::UnsafeWhileSuspended { java_major }));
if suspended && !safe {
return unsafe_stop();
}
// The program may stop during the histogram, and a stopped JDK below 17 hangs on the next
// request until it resumes.
let stopped_since = || !safe && suspended_now();
let text = match probe.histogram(jvm) {
Ok(text) => text,
Err(e) => return attach_failure(probe, jvm, e, "class histogram").map(memory),
};
let histogram: Vec<ide_heap::HistogramEntry> = ide_jvm_attach::parse_class_histogram(&text)
.rows
.into_iter()
.map(|r| ide_heap::HistogramEntry { name: r.name, instances: r.instances, bytes: r.bytes })
.collect();
let (instances, bytes) =
histogram.iter().fold((0u64, 0u64), |(i, b), e| (i.saturating_add(e.instances), b.saturating_add(e.bytes)));
let needed = ide_heap::capture_need(bytes, instances);
if let Some(available) = probe.free_bytes(dir)
&& available < needed
{
return Ok(JvmHeapOutcome::Unavailable(H::NotEnoughDiskSpace { needed, available }));
}
if stopped_since() {
return unsafe_stop();
}
// Without flags the index assumes the default object layout.
let vm_flags = probe.vm_flags(jvm).unwrap_or_else(|e| {
tracing::debug!("VM.flags of JVM {jvm}: {e}");
String::new()
});
if cancel.is_cancelled() {
return Ok(JvmHeapOutcome::Cancelled);
}
if stopped_since() {
return unsafe_stop();
}
let path = dir.join(DUMP_FILE);
let timeout = ide_jvm_attach::dump_timeout(bytes);
let started = std::time::Instant::now();
let reply = probe.heap_dump(jvm, &path, timeout);
let dump_ms = u32::try_from(started.elapsed().as_millis()).unwrap_or(u32::MAX);
if cancel.is_cancelled() {
return Ok(JvmHeapOutcome::Cancelled);
}
let failed = |message: String| Ok(JvmHeapOutcome::Unavailable(H::DumpFailed { message }));
match reply {
Ok(HeapDumpReply::Created { .. }) => {
if !std::fs::metadata(&path).is_ok_and(|m| m.is_file() && m.len() > 0) {
return failed(format!("the JVM reported a heap dump, but {} is missing or empty", path.display()));
}
Ok(JvmHeapOutcome::Ready(JvmHeapDump {
path,
histogram,
vm_flags,
java_major,
java_version: version.map(|v| v.text).unwrap_or_default(),
pid: jvm,
taken_while_suspended: suspended,
dump_ms,
}))
}
Ok(HeapDumpReply::Refused { message }) if message.is_empty() || message.starts_with("Dumping heap to") => {
failed(NO_REASON.into())
}
Ok(HeapDumpReply::Refused { message }) => failed(message),
Err(AttachError::Timeout) => {
let seconds = u32::try_from(timeout.as_secs()).unwrap_or(u32::MAX);
Ok(JvmHeapOutcome::Unavailable(H::DumpTimedOut { seconds }))
}
// A running JDK rejects an argument with a non-zero result code.
Err(AttachError::Command { message, .. }) => failed(message),
Err(e) => attach_failure(probe, jvm, e, "heap dump").map(memory),
}
}
fn memory(reason: JvmMemoryUnavailable) -> JvmHeapOutcome {
JvmHeapOutcome::Unavailable(JvmHeapUnavailable::Memory { reason })
}
fn discard(dir: &Path) {
if let Err(e) = std::fs::remove_dir_all(dir)
&& e.kind() != std::io::ErrorKind::NotFound
{
tracing::warn!("could not delete heap snapshot directory {}: {e}", dir.display());
}
}
/// A dump may be large, so it is deleted off the async threads.
async fn discard_later(dir: PathBuf) {
let _ = tokio::task::spawn_blocking(move || discard(&dir)).await;
}
+251
View File
@@ -0,0 +1,251 @@
//! A suspended JVM's instances, fields and referrers, read through the session's
//! [`JvmInspector`]. An object is only valid within the stop it was read in.
use super::launch_pid;
use crate::session::Session;
use ide_core::event::{DebugEvent, JvmBackendState};
use ide_core::{IdeError, IdeResult};
use ide_jvm_inspect::{
DEFAULT_REFERRERS, InspectError, InspectFuture, JvmField, JvmInspector, JvmReferrer, JvmValue, MAX_INSTANCES,
MAX_REFERRERS, ObjectId, VmTarget,
};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::sync::Arc;
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct JvmObjectRef {
/// The `generation` of the [`JvmInstancesResult`] the object came from.
pub generation: u64,
pub id: ObjectId,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmInspectUnavailable {
/// The session has no inspector: not a JVM session, not ready yet, or turned off.
NoBackend,
/// The debug adapter lacks what the inspector needs; turned off for the session.
Unsupported {
detail: String,
},
NotSuspended,
/// The object is from an earlier stop, or the program stopped or resumed meanwhile.
Stale,
Collected,
ClassNotLoaded,
VmNotFound,
/// Transient: ask again.
TimedOut,
BackendClosed,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "status", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmInstancesResult {
Ready {
/// Goes into the [`JvmObjectRef`] of each instance.
generation: u64,
instances: Vec<JvmValue>,
/// The limit was reached; there may be more.
truncated: bool,
},
Unavailable {
reason: JvmInspectUnavailable,
},
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "status", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmFieldsResult {
Ready { fields: Vec<JvmField> },
Unavailable { reason: JvmInspectUnavailable },
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "status", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmReferrersResult {
Ready { referrers: Vec<JvmReferrer>, truncated: bool },
Unavailable { reason: JvmInspectUnavailable },
}
/// The session's JVM for the inspector: java-debug's VM is found by the launched program's pid
/// or by the port of an attach (the thread lists of two JVMs can be identical).
pub(crate) fn vm_target(s: &Session) -> VmTarget {
let (pid, attach) = if s.request == "launch" {
(u32::try_from(launch_pid(s)).ok().filter(|&p| p > 0), None)
} else {
(None, s.attach_target.get())
};
VmTarget {
pid,
attach_host: attach.map(|(host, _)| host.clone()).filter(|h| !h.is_empty()),
attach_port: attach.map(|&(_, port)| port),
generation: s.suspended.ticket(),
}
}
pub(crate) fn set_inspector(s: &Session, inspector: Arc<dyn JvmInspector>) {
*s.jvm_inspector.lock() = Some(inspector);
s.emit(DebugEvent::Capabilities);
}
pub(crate) async fn instances(s: &Session, class_name: &str, limit: u32) -> IdeResult<JvmInstancesResult> {
let limit = limit.clamp(1, MAX_INSTANCES);
let call = match Call::begin(s, None).await {
Ok(call) => call,
Err(reason) => return Ok(JvmInstancesResult::Unavailable { reason }),
};
let generation = call.target.generation;
let result = call.run(call.inspector.instances(&call.target, class_name, limit)).await;
Ok(match settle(call.finish(result))? {
Ok(instances) => {
JvmInstancesResult::Ready { generation, truncated: instances.len() >= limit as usize, instances }
}
Err(reason) => JvmInstancesResult::Unavailable { reason },
})
}
pub(crate) async fn fields(s: &Session, object: JvmObjectRef) -> IdeResult<JvmFieldsResult> {
let call = match Call::begin(s, Some(object.generation)).await {
Ok(call) => call,
Err(reason) => return Ok(JvmFieldsResult::Unavailable { reason }),
};
let result = call.run(call.inspector.fields(&call.target, object.id)).await;
Ok(match settle(call.finish(result))? {
Ok(fields) => JvmFieldsResult::Ready { fields },
Err(reason) => JvmFieldsResult::Unavailable { reason },
})
}
pub(crate) async fn referrers(s: &Session, object: JvmObjectRef, limit: u32) -> IdeResult<JvmReferrersResult> {
let limit = if limit == 0 { DEFAULT_REFERRERS } else { limit.min(MAX_REFERRERS) };
let call = match Call::begin(s, Some(object.generation)).await {
Ok(call) => call,
Err(reason) => return Ok(JvmReferrersResult::Unavailable { reason }),
};
let result = call.run(call.inspector.referrers(&call.target, object.id, limit)).await;
Ok(match settle(call.finish(result))? {
Ok(referrers) => JvmReferrersResult::Ready { truncated: referrers.len() >= limit as usize, referrers },
Err(reason) => JvmReferrersResult::Unavailable { reason },
})
}
/// One inspector request. Holds the session's inspect lock and a clone of its inspector, which
/// [`Call::run`] lets go of when the session ends.
pub(super) struct Call<'s> {
session: &'s Session,
pub(super) inspector: Arc<dyn JvmInspector>,
pub(super) target: VmTarget,
_turn: tokio::sync::MutexGuard<'s, ()>,
}
impl<'s> Call<'s> {
/// `object_generation`: the stop the request's object was read in.
pub(super) async fn begin(
s: &'s Session,
object_generation: Option<u64>,
) -> Result<Call<'s>, JvmInspectUnavailable> {
// Checked under the lock: a request queued behind another must see the state it runs in.
let turn = s.inspect_lock.lock().await;
let Some(inspector) = s.jvm_inspector.lock().clone() else {
return Err(JvmInspectUnavailable::NoBackend);
};
if !s.suspended.get() {
return Err(JvmInspectUnavailable::NotSuspended);
}
let target = vm_target(s);
if object_generation.is_some_and(|g| g != target.generation) {
return Err(JvmInspectUnavailable::Stale);
}
Ok(Call { session: s, inspector, target, _turn: turn })
}
pub(super) async fn run<T>(&self, request: InspectFuture<'_, T>) -> Result<T, InspectError> {
tokio::select! {
result = request => result,
() = self.session.ended.cancelled() => Err(InspectError::Closed),
}
}
/// The result unless the program stopped or resumed while it was read: a request of several
/// commands can straddle a stop. `Unsupported` turns the backend off.
pub(super) fn finish<T>(self, result: Result<T, InspectError>) -> Result<T, InspectError> {
let s = self.session;
match result {
Err(InspectError::Unsupported(detail)) => {
turn_off(s, &self.inspector, &detail);
Err(InspectError::Unsupported(detail))
}
_ if !s.suspended.get() || s.suspended.ticket() != self.target.generation => Err(InspectError::Stale),
result => result,
}
}
}
/// An adapter that lacks what the inspector needs will not gain it during the session.
fn turn_off(s: &Session, inspector: &Arc<dyn JvmInspector>, detail: &str) {
let removed = {
let mut slot = s.jvm_inspector.lock();
// Not one installed since this request began.
let current = slot.as_ref().is_some_and(|i| std::ptr::addr_eq(Arc::as_ptr(i), Arc::as_ptr(inspector)));
current && slot.take().is_some()
};
if removed {
s.emit(DebugEvent::JvmBackend { state: JvmBackendState::Unsupported { detail: detail.to_owned() } });
s.emit(DebugEvent::Capabilities);
}
}
/// A failure the inspector could not classify is an error, not a reason shown to the user.
fn settle<T>(result: Result<T, InspectError>) -> IdeResult<Result<T, JvmInspectUnavailable>> {
use JvmInspectUnavailable as U;
let error = match result {
Ok(value) => return Ok(Ok(value)),
Err(error) => error,
};
Ok(Err(match error {
InspectError::NotSuspended => U::NotSuspended,
InspectError::Stale => U::Stale,
InspectError::Collected => U::Collected,
InspectError::ClassNotLoaded => U::ClassNotLoaded,
InspectError::VmNotFound => U::VmNotFound,
InspectError::Unsupported(detail) => U::Unsupported { detail },
// Also the Kotlin link's `Busy`: the backend stays.
InspectError::TimedOut => U::TimedOut,
InspectError::Closed => U::BackendClosed,
InspectError::Failed(message) => return Err(IdeError::internal(format!("JVM inspector: {message}"))),
}))
}
/// Reports a Kotlin link that stops inspecting or closes. Holds neither the link nor the
/// session: it ends once the session drops the link, and never drops it itself (after
/// `VirtualMachine.Dispose` the link is closed but still forwarding the reply).
pub(crate) fn watch_link(s: &Arc<Session>, link: &ide_jdwp::JdwpLink) {
let mut states = link.subscribe();
let session = Arc::downgrade(s);
tokio::spawn(async move {
let mut reported = None;
loop {
let state = match &*states.borrow_and_update() {
ide_jdwp::LinkState::Degraded(detail) => Some(JvmBackendState::Degraded { detail: detail.clone() }),
ide_jdwp::LinkState::Closed => Some(JvmBackendState::Closed),
_ => None,
};
if let Some(state) = state
&& reported.as_ref() != Some(&state)
{
let Some(s) = session.upgrade() else { return };
if s.ended.is_cancelled() || s.terminated.load(std::sync::atomic::Ordering::Acquire) {
return;
}
s.emit(DebugEvent::JvmBackend { state: state.clone() });
reported = Some(state);
}
if states.changed().await.is_err() {
return;
}
}
});
}
+338
View File
@@ -0,0 +1,338 @@
//! Class-table snapshots of a debugged JVM, read over the HotSpot attach listener or, where
//! that is unavailable, counted by the debugger; and the JVM's objects through the debugger.
mod fallback;
mod heap;
mod inspect;
mod scan;
mod snapshot;
#[cfg(test)]
pub(crate) mod test_support;
#[cfg(test)]
mod tests;
use crate::session::Session;
use crate::session::Suspension;
use ide_core::IdeResult;
use serde::{Deserialize, Serialize};
use specta::Type;
use std::collections::HashMap;
use std::sync::Arc;
use std::sync::atomic::{AtomicI64, Ordering};
use std::time::Duration;
pub use heap::*;
pub use inspect::{JvmFieldsResult, JvmInspectUnavailable, JvmInstancesResult, JvmObjectRef, JvmReferrersResult};
pub(crate) use inspect::{fields, instances, referrers, set_inspector, watch_link};
#[cfg(all(test, unix))]
pub(crate) use scan::watch_for_attached_jvm;
pub(crate) use scan::{JvmFinder, attach_target, find_attached_jvm, find_local_jvm, local_jdwp_port};
use snapshot::{AttachProbe, JdkProbe, take_snapshot};
pub const JVM_MEMORY_ADAPTERS: &[&str] = &["java-debug", crate::kotlin::ADAPTER];
pub const HISTOGRAM_TIMEOUT: Duration = Duration::from_secs(10);
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct JvmClassRow {
pub name: String,
pub module: Option<String>,
pub instances: u64,
pub bytes: u64,
/// Change since the previous snapshot; `None` on the first one and for ambiguous rows.
pub delta_instances: Option<i64>,
pub delta_bytes: Option<i64>,
/// Several classes share this name (different class loaders), so it has no delta.
pub ambiguous: bool,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct JvmMemorySnapshot {
/// Largest `bytes` first.
pub rows: Vec<JvmClassRow>,
pub total_instances: u64,
pub total_bytes: u64,
/// Empty when unknown.
pub java_version: String,
pub taken_while_suspended: bool,
/// A previous snapshot from the same source exists; deltas are against it.
pub has_baseline: bool,
/// Wall time of the histogram or count request.
pub elapsed_ms: u32,
pub source: JvmMemorySource,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum JvmMemorySource {
/// `GC.class_histogram -all`: sizes, and objects not yet collected.
Histogram,
/// JDWP `InstanceCounts`: reachable objects only, no sizes (`bytes` is 0).
Debugger,
}
#[derive(Clone, Copy, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmMemoryUnavailable {
/// An attach session whose JVM is remote, or local but not found by its JDWP port.
AttachedJvmNotFound,
/// An attach through the IDE's listen-mode relay: the JVM connected to the IDE, so it has
/// no JDWP listener to be found by.
ListenModeNotSupported,
UnsupportedPlatform,
NoJvmProcess,
ListenerNotRunning,
/// `java_major` is 0 when the version is unknown.
UnsafeWhileSuspended {
java_major: u32,
},
ProgramExited,
TimedOut,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(tag = "status", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmMemoryResult {
Ready {
snapshot: JvmMemorySnapshot,
},
Unavailable {
reason: JvmMemoryUnavailable,
/// The debugger could count the classes if the program were paused.
can_count_when_paused: bool,
},
}
/// Instances and bytes per class name, for the next snapshot's deltas; `None` for a name
/// several classes shared, so it gets no delta next time either.
type ClassCounts = HashMap<String, Option<(u64, u64)>>;
/// Per-session baseline; its mutex also keeps one class-table request per session at a time.
#[derive(Default)]
pub struct JvmMemoryState {
/// Shared with the blocking task, so a task that panics leaves it in place.
previous: Option<(JvmMemorySource, Arc<ClassCounts>)>,
}
impl JvmMemoryState {
/// The previous snapshot's counts if it came from `source`: counts from the other source
/// are not comparable (only reachable objects versus all of them).
pub(crate) fn baseline(&self, source: JvmMemorySource) -> Option<Arc<ClassCounts>> {
self.previous.as_ref().filter(|(s, _)| *s == source).map(|(_, counts)| counts.clone())
}
pub(crate) fn remember(&mut self, source: JvmMemorySource, counts: ClassCounts) {
self.previous = Some((source, Arc::new(counts)));
}
}
pub(crate) async fn session_classes(session: Arc<Session>) -> IdeResult<JvmMemoryResult> {
until_located(&session).await;
let launched = histogram_pid(&session);
// Detached, so a caller that gives up cannot release the lock while the JVM is still
// answering and let a second histogram request run next to it.
tokio::spawn(async move {
let target = session_target(&session, launched);
let probe = |suspended| -> Arc<dyn AttachProbe> { Arc::new(JdkProbe { suspended }) };
snapshot(&session.jvm_memory, &session.suspended, probe, target, Some(&session)).await
})
.await
.map_err(|e| ide_core::IdeError::internal(format!("JVM memory snapshot failed: {e}")))?
}
/// Bounds the wait of [`until_located`]; `configure` has its own timeouts, this only guards them.
const LOCATE_WAIT: Duration = Duration::from_secs(30);
/// Until `configure` has recorded where `session`'s JVM is. The Memory view asks as soon as the
/// session appears; looked up earlier, an attached JVM is reported as missing.
async fn until_located(session: &Session) {
let mut located = session.jvm_located.subscribe();
tokio::select! {
_ = located.wait_for(|v| *v) => {}
() = session.ended.cancelled() => {}
() = tokio::time::sleep(LOCATE_WAIT) => {}
}
}
/// `launched`: [`histogram_pid`] of `session`.
fn session_target(session: &Session, launched: Result<Option<i32>, JvmMemoryUnavailable>) -> JvmTarget<'_> {
match launched {
Ok(Some(pid)) => JvmTarget::Launched(pid),
Ok(None) => JvmTarget::Attached { pid: &session.jvm_pid, jdwp_port: session.jdwp_port.get().copied() },
Err(reason) => JvmTarget::Unavailable(reason),
}
}
/// The pid of a launch session's program (`None` for an attach session), or why no histogram
/// can be taken.
fn histogram_pid(session: &Session) -> Result<Option<i32>, JvmMemoryUnavailable> {
if !JVM_MEMORY_ADAPTERS.contains(&session.adapter.as_str()) {
return Err(JvmMemoryUnavailable::NoJvmProcess);
}
if !cfg!(unix) {
return Err(JvmMemoryUnavailable::UnsupportedPlatform);
}
if session.request == "launch" {
return match i32::try_from(launch_pid(session)).ok().filter(|&p| p > 0) {
Some(pid) => Ok(Some(pid)),
None => Err(JvmMemoryUnavailable::NoJvmProcess),
};
}
if session.jvm_pid.load(Ordering::Acquire) <= 0 {
if session.terminated.load(Ordering::Acquire) {
return Err(JvmMemoryUnavailable::ProgramExited);
}
if session.listen_relay.load(Ordering::Acquire) {
return Err(JvmMemoryUnavailable::ListenModeNotSupported);
}
}
Ok(None)
}
/// The JVM a launch session started: the one the IDE started for an app-launched (Kotlin)
/// session, else the program the adapter reported; 0 when neither is known.
pub(crate) fn launch_pid(session: &Session) -> i64 {
match session.jvm_pid.load(Ordering::Acquire) {
p if p > 0 => p,
_ => session.debuggee_pid.load(Ordering::Acquire),
}
}
#[derive(Clone, Copy)]
pub(crate) enum JvmTarget<'a> {
/// The program a launch session started: its pid and everything it started.
Launched(i32),
/// The JVM of an attach session. While `pid` is 0 it is looked up by `jdwp_port`, which is
/// `None` for a remote JVM.
Attached { pid: &'a AtomicI64, jdwp_port: Option<u16> },
/// Known without asking any JVM: no histogram can be taken, for this reason.
Unavailable(JvmMemoryUnavailable),
}
enum Lookup {
Tree(i32),
Known(i32),
Scan(u16),
}
/// `probe` is built from the suspension read once the lock is held: a request queued behind
/// another must not decide on a state from before the program hit a breakpoint. Where no
/// histogram can be taken, `debugger` (the session) may count the classes instead.
pub(crate) async fn snapshot(
state: &tokio::sync::Mutex<JvmMemoryState>,
suspended: &Suspension,
probe: impl FnOnce(bool) -> Arc<dyn AttachProbe>,
target: JvmTarget<'_>,
debugger: Option<&Session>,
) -> IdeResult<JvmMemoryResult> {
let mut state = state.lock().await;
let outcome = histogram(&mut state, suspended, probe, target).await?;
fallback::or_debugger_counts(&mut state, outcome, debugger).await
}
async fn histogram(
state: &mut JvmMemoryState,
suspended: &Suspension,
probe: impl FnOnce(bool) -> Arc<dyn AttachProbe>,
target: JvmTarget<'_>,
) -> IdeResult<Result<JvmMemorySnapshot, JvmMemoryUnavailable>> {
let suspended = suspended.get();
let lookup = match lookup(target) {
Ok(lookup) => lookup,
Err(reason) => return Ok(Err(reason)),
};
let probe = probe(suspended);
let previous = state.baseline(JvmMemorySource::Histogram);
let (found, outcome) = tokio::task::spawn_blocking(move || match resolve_jvm(lookup, &*probe, suspended) {
Ok((tree, found)) => (found, take_snapshot(&*probe, &tree, suspended, previous.as_deref())),
Err(reason) => (None, Ok(Err(reason))),
})
.await
.map_err(|e| ide_core::IdeError::internal(format!("JVM memory snapshot failed: {e}")))?;
keep_found(target, found);
Ok(outcome?.map(|(snapshot, counts)| {
state.remember(JvmMemorySource::Histogram, counts);
snapshot
}))
}
/// Where to look for `target`'s JVM, or why there is none to ask.
fn lookup(target: JvmTarget<'_>) -> Result<Lookup, JvmMemoryUnavailable> {
match target {
JvmTarget::Launched(pid) => Ok(Lookup::Tree(pid)),
JvmTarget::Attached { pid, jdwp_port } => match i32::try_from(pid.load(Ordering::Acquire)) {
Ok(p) if p > 0 => Ok(Lookup::Known(p)),
_ => jdwp_port.map(Lookup::Scan).ok_or(JvmMemoryUnavailable::AttachedJvmNotFound),
},
JvmTarget::Unavailable(reason) => Err(reason),
}
}
/// Blocking. The pids to pick the JVM from, and the pid a scan found (for [`keep_found`]).
fn resolve_jvm(
lookup: Lookup,
probe: &dyn AttachProbe,
suspended: bool,
) -> Result<(Vec<i32>, Option<i32>), JvmMemoryUnavailable> {
match lookup {
Lookup::Tree(pid) => Ok((pid_tree(pid), None)),
Lookup::Known(pid) => Ok((vec![pid], None)),
Lookup::Scan(port) => scan_for_jvm(probe, port, suspended).map(|pid| (vec![pid], Some(pid))),
}
}
fn keep_found(target: JvmTarget<'_>, found: Option<i32>) {
if let (Some(found), JvmTarget::Attached { pid, .. }) = (found, target) {
// The attach-time scan may have found it meanwhile; either answer is the same JVM.
let _ = pid.compare_exchange(0, i64::from(found), Ordering::AcqRel, Ordering::Acquire);
}
}
/// A suspended JDK below 17 hangs on an attach request, so while suspended only a JVM known to
/// be 17 or later is asked anything. Pass a version read without attaching (the `release` file).
fn safe_while_suspended(version: Option<&ide_jvm_attach::JavaVersion>) -> bool {
version.is_some_and(|v| ide_jvm_attach::histogram_safe_while_suspended(v.major))
}
/// The JVM listening on JDWP `port`. While suspended the scan reaches the session's own JVM
/// too, so JVMs unsafe to ask are skipped, and when one of them may be the session's the
/// answer is a refusal rather than "not found".
fn scan_for_jvm(probe: &dyn AttachProbe, port: u16, suspended: bool) -> Result<i32, JvmMemoryUnavailable> {
// Majors of the JVMs left out; 0 for an unknown version.
let skipped = std::cell::RefCell::new(std::collections::BTreeSet::new());
let may_query = |pid: u32| {
if !suspended {
return true;
}
let version = probe.version(pid);
let safe = safe_while_suspended(version.as_ref());
if !safe {
skipped.borrow_mut().insert(version.map_or(0, |v| v.major));
}
safe
};
match probe.find_by_jdwp_port(port, &may_query).and_then(|p| i32::try_from(p).ok()) {
Some(pid) if pid > 0 => Ok(pid),
_ => {
let skipped = skipped.into_inner();
Err(match skipped.first() {
None => JvmMemoryUnavailable::AttachedJvmNotFound,
Some(&major) => JvmMemoryUnavailable::UnsafeWhileSuspended {
java_major: if skipped.len() == 1 { major } else { 0 },
},
})
}
}
}
fn pid_tree(pid: i32) -> Vec<i32> {
#[cfg(unix)]
{
ide_core::process::process_tree(pid)
}
#[cfg(not(unix))]
{
vec![pid]
}
}
+146
View File
@@ -0,0 +1,146 @@
//! Finding the JVM behind an attach session by its JDWP port.
use super::safe_while_suspended;
use crate::session::Session;
use serde_json::Value;
use std::net::IpAddr;
use std::sync::Arc;
use std::sync::atomic::Ordering;
use std::time::Duration;
/// One try of the scan at attach time, and the limit per socket within it: a silent socket's
/// query thread lives that long, so a longer per-socket limit piles threads up across tries.
const ATTACH_SCAN_TRY: Duration = Duration::from_millis(500);
const ATTACH_SCAN_INTERVAL: Duration = Duration::from_millis(150);
/// The JVM's attach socket can appear a little after the JDWP "Listening" banner, which is
/// when the IDE attaches.
const ATTACH_SCAN_WINDOW: Duration = Duration::from_secs(3);
/// The single scan a refresh makes while the attach-time one found nothing.
pub(super) const RESCAN_PER_SOCKET: Duration = Duration::from_secs(1);
pub(super) const RESCAN_BUDGET: Duration = Duration::from_secs(2);
/// JDWP port of an attach to this machine (`hostName` loopback or absent): only such a JVM has
/// an attach socket the IDE can find.
pub(crate) fn local_jdwp_port(arguments: &Value) -> Option<u16> {
let local = match arguments.get("hostName") {
None | Some(Value::Null) => true,
Some(Value::String(host)) => {
let host = host.trim();
let ip = host.strip_prefix('[').and_then(|h| h.strip_suffix(']')).unwrap_or(host);
host.eq_ignore_ascii_case("localhost") || ip.parse::<IpAddr>().is_ok_and(|ip| ip.is_loopback())
}
Some(_) => false,
};
jdwp_port(arguments).filter(|_| local)
}
/// `port` of an attach request: a number or a numeric string, never 0.
fn jdwp_port(arguments: &Value) -> Option<u16> {
let port = match arguments.get("port")? {
Value::Number(n) => n.as_u64().and_then(|p| u16::try_from(p).ok()),
Value::String(s) => s.trim().parse().ok(),
_ => None,
};
port.filter(|&p| p != 0)
}
/// `hostName` (empty when absent) and `port` of an attach request, local or not.
pub(crate) fn attach_target(arguments: &Value) -> Option<(String, u16)> {
let host = arguments.get("hostName").and_then(Value::as_str).unwrap_or_default();
Some((host.to_owned(), jdwp_port(arguments)?))
}
/// Pid of the local JVM listening on a JDWP port (first argument), given whether the session is
/// suspended (second). Replaceable so tests never send attach requests to real JVMs.
pub(crate) type JvmFinder = Arc<dyn Fn(u16, bool) -> Option<u32> + Send + Sync>;
/// The [`JvmFinder`] of the attach-time scan.
pub(crate) fn find_local_jvm(port: u16, suspended: bool) -> Option<u32> {
let may_query = |pid| !suspended || safe_while_suspended(ide_jvm_attach::java_version_of(pid).as_ref());
ide_jvm_attach::find_jvm_by_jdwp_port(port, ATTACH_SCAN_TRY, ATTACH_SCAN_TRY, may_query)
}
/// Looks for the JVM behind a local attach: one try now, the rest in the background, so a JVM
/// that never opens an attach socket holds the attach up by one try at most. Call it before the
/// debugger connects: the JVM clears its JDWP listener address then, and a scan can no longer
/// tell it by its port.
pub(crate) async fn find_attached_jvm(session: &Arc<Session>, port: u16, finder: JvmFinder) {
let _ = session.jdwp_port.set(port);
let find = move |suspended: bool| finder(port, suspended);
let first = find.clone();
if let Ok(Some(pid)) = tokio::task::spawn_blocking(move || first(false)).await {
let _ = session.jvm_pid.compare_exchange(0, i64::from(pid), Ordering::AcqRel, Ordering::Acquire);
return;
}
drop(watch_for_attached_jvm(session, port, find, ATTACH_SCAN_INTERVAL, ATTACH_SCAN_WINDOW));
}
/// Retries `find` for the session's `jvm_pid`, holding the session weakly so an ended session
/// is not kept alive. `find` is told whether the session is suspended.
pub(crate) fn watch_for_attached_jvm<F>(
session: &Arc<Session>,
port: u16,
find: F,
interval: Duration,
window: Duration,
) -> tokio::task::JoinHandle<()>
where
F: Fn(bool) -> Option<u32> + Clone + Send + 'static,
{
let weak = Arc::downgrade(session);
tokio::spawn(async move {
let gate = || match weak.upgrade() {
Some(s) if !s.terminated.load(Ordering::Acquire) && !s.conn.is_closed() => {
if s.jvm_pid.load(Ordering::Acquire) > 0 {
ScanGate::Ended
} else {
ScanGate::Scan { suspended: s.suspended.get() }
}
}
_ => ScanGate::Ended,
};
let Some(pid) = retry_find(find, gate, interval, window).await else {
tracing::debug!("no local JVM found for JDWP port {port} yet");
return;
};
if let Some(s) = weak.upgrade() {
let _ = s.jvm_pid.compare_exchange(0, i64::from(pid), Ordering::AcqRel, Ordering::Acquire);
}
})
}
#[derive(Clone, Copy, Debug, PartialEq)]
pub(crate) enum ScanGate {
Scan { suspended: bool },
Ended,
}
/// Calls `find` on a blocking thread, on each try `gate` allows, until it returns a pid, `gate`
/// reports the end or `window` has passed.
pub(crate) async fn retry_find<F>(
find: F,
gate: impl Fn() -> ScanGate,
interval: Duration,
window: Duration,
) -> Option<u32>
where
F: Fn(bool) -> Option<u32> + Clone + Send + 'static,
{
let deadline = tokio::time::Instant::now() + window;
loop {
match gate() {
ScanGate::Ended => break,
ScanGate::Scan { suspended } => {
let find = find.clone();
if let Ok(Some(pid)) = tokio::task::spawn_blocking(move || find(suspended)).await {
return Some(pid);
}
}
}
if tokio::time::Instant::now() + interval >= deadline {
break;
}
tokio::time::sleep(interval).await;
}
None
}
+214
View File
@@ -0,0 +1,214 @@
//! One class histogram of a JVM and its deltas against the previous one.
use super::scan::{RESCAN_BUDGET, RESCAN_PER_SOCKET};
use super::{
ClassCounts, HISTOGRAM_TIMEOUT, JvmClassRow, JvmMemorySnapshot, JvmMemorySource, JvmMemoryUnavailable,
safe_while_suspended,
};
use ide_core::IdeResult;
use ide_jvm_attach::{AttachError, HeapDumpReply, HistogramRow, JavaVersion};
use std::collections::HashMap;
use std::path::Path;
use std::time::Duration;
/// Version probe for a JVM whose `release` file could not be read.
const VERSION_TIMEOUT: Duration = Duration::from_secs(2);
const FLAGS_TIMEOUT: Duration = Duration::from_secs(2);
/// The attach calls a snapshot or a heap dump makes, so tests can stand in for a JVM.
pub(crate) trait AttachProbe: Send + Sync {
fn has_listener(&self, pid: u32) -> bool;
/// Asks only the JVMs whose pid `may_query` accepts.
fn find_by_jdwp_port(&self, port: u16, may_query: &dyn Fn(u32) -> bool) -> Option<u32>;
fn version(&self, pid: u32) -> Option<JavaVersion>;
fn histogram(&self, pid: u32) -> Result<String, AttachError>;
fn alive(&self, pid: u32) -> bool;
fn vm_flags(&self, pid: u32) -> Result<String, AttachError>;
fn heap_dump(&self, pid: u32, path: &Path, timeout: Duration) -> Result<HeapDumpReply, AttachError>;
/// `None` where the free space is not known.
fn free_bytes(&self, dir: &Path) -> Option<u64>;
}
pub(super) struct JdkProbe {
pub(super) suspended: bool,
}
impl AttachProbe for JdkProbe {
fn has_listener(&self, pid: u32) -> bool {
ide_jvm_attach::has_listener(pid)
}
fn find_by_jdwp_port(&self, port: u16, may_query: &dyn Fn(u32) -> bool) -> Option<u32> {
ide_jvm_attach::find_jvm_by_jdwp_port(port, RESCAN_PER_SOCKET, RESCAN_BUDGET, may_query)
}
fn version(&self, pid: u32) -> Option<JavaVersion> {
ide_jvm_attach::java_version_of(pid).or_else(|| {
// The version decides whether attaching to a suspended VM hangs it, so never
// attach to learn it while suspended.
if self.suspended {
return None;
}
let out = ide_jvm_attach::jcmd(pid, "VM.version", VERSION_TIMEOUT).ok()?;
ide_jvm_attach::parse_vm_version(&out)
})
}
fn histogram(&self, pid: u32) -> Result<String, AttachError> {
ide_jvm_attach::jcmd(pid, "GC.class_histogram -all", HISTOGRAM_TIMEOUT)
}
fn alive(&self, pid: u32) -> bool {
alive(pid)
}
fn vm_flags(&self, pid: u32) -> Result<String, AttachError> {
ide_jvm_attach::vm_flags(pid, FLAGS_TIMEOUT)
}
fn heap_dump(&self, pid: u32, path: &Path, timeout: Duration) -> Result<HeapDumpReply, AttachError> {
ide_jvm_attach::heap_dump(pid, path, timeout)
}
fn free_bytes(&self, dir: &Path) -> Option<u64> {
ide_heap::free_bytes(dir)
}
}
fn alive(pid: u32) -> bool {
#[cfg(unix)]
{
// 0 and negative pids address process groups.
let Ok(pid) = i32::try_from(pid) else { return false };
// SAFETY: signal 0 only checks that the process exists and may be signalled.
pid > 0
&& (unsafe { libc::kill(pid, 0) } == 0
|| std::io::Error::last_os_error().raw_os_error() == Some(libc::EPERM))
}
#[cfg(not(unix))]
{
let _ = pid;
false
}
}
/// Why `pid`'s attach listener cannot answer: it stopped listening or the program ended.
pub(super) fn gone(probe: &dyn AttachProbe, pid: u32) -> JvmMemoryUnavailable {
if probe.alive(pid) { JvmMemoryUnavailable::ListenerNotRunning } else { JvmMemoryUnavailable::ProgramExited }
}
/// The reason for an attach request (`what`) to `jvm` that failed, or an error where no reason
/// fits.
pub(super) fn attach_failure(
probe: &dyn AttachProbe,
jvm: u32,
error: AttachError,
what: &str,
) -> IdeResult<JvmMemoryUnavailable> {
use JvmMemoryUnavailable as U;
match error {
AttachError::Timeout => Ok(U::TimedOut),
AttachError::Refused | AttachError::NoListener => Ok(gone(probe, jvm)),
AttachError::ForeignListener => {
tracing::warn!("attach socket of JVM {jvm} is no longer private to this user; not using it");
Ok(U::ListenerNotRunning)
}
AttachError::Unsupported => Ok(U::UnsupportedPlatform),
e => Err(ide_core::IdeError::internal(format!("{what} of JVM {jvm}: {e}"))),
}
}
pub(crate) fn pick_jvm_pid(tree: &[i32], has_listener: impl Fn(u32) -> bool) -> Option<u32> {
tree.iter().filter_map(|&p| u32::try_from(p).ok()).filter(|&p| p > 0).find(|&p| has_listener(p))
}
pub(crate) fn diff_rows(rows: Vec<HistogramRow>, previous: Option<&ClassCounts>) -> (Vec<JvmClassRow>, ClassCounts) {
let mut seen: HashMap<&str, u32> = HashMap::with_capacity(rows.len());
for r in &rows {
*seen.entry(r.name.as_str()).or_default() += 1;
}
let ambiguous: Vec<bool> = rows.iter().map(|r| seen[r.name.as_str()] > 1).collect();
drop(seen);
let mut counts = ClassCounts::with_capacity(rows.len());
let mut out: Vec<JvmClassRow> = rows
.into_iter()
.zip(ambiguous)
.map(|(r, ambiguous)| {
counts.insert(r.name.clone(), (!ambiguous).then_some((r.instances, r.bytes)));
let delta = match previous {
_ if ambiguous => None,
None => None,
Some(p) => match p.get(&r.name) {
Some(None) => None,
// Absent: new since the previous snapshot, so it grew from nothing.
before => {
let (instances, bytes) = before.copied().flatten().unwrap_or_default();
Some((delta(r.instances, instances), delta(r.bytes, bytes)))
}
},
};
JvmClassRow {
name: r.name,
module: r.module,
instances: r.instances,
bytes: r.bytes,
delta_instances: delta.map(|d| d.0),
delta_bytes: delta.map(|d| d.1),
ambiguous,
}
})
.collect();
out.sort_by(|a, b| b.bytes.cmp(&a.bytes).then_with(|| a.name.cmp(&b.name)));
(out, counts)
}
/// Exact while the difference fits in an i64 (two's complement).
fn delta(now: u64, before: u64) -> i64 {
now.wrapping_sub(before) as i64
}
/// `pid_tree` starts with the session's reported pid, followed by its descendants.
pub(crate) fn take_snapshot(
probe: &dyn AttachProbe,
pid_tree: &[i32],
suspended: bool,
previous: Option<&ClassCounts>,
) -> IdeResult<Result<(JvmMemorySnapshot, ClassCounts), JvmMemoryUnavailable>> {
use JvmMemoryUnavailable as U;
let Some(root) = pid_tree.first().and_then(|&p| u32::try_from(p).ok()) else {
return Ok(Err(U::NoJvmProcess));
};
let Some(jvm) = pick_jvm_pid(pid_tree, |p| probe.has_listener(p)) else {
return Ok(Err(gone(probe, root)));
};
let version = probe.version(jvm);
if suspended && !safe_while_suspended(version.as_ref()) {
return Ok(Err(U::UnsafeWhileSuspended { java_major: version.map_or(0, |v| v.major) }));
}
let started = std::time::Instant::now();
let text = match probe.histogram(jvm) {
Ok(text) => text,
Err(e) => return attach_failure(probe, jvm, e, "class histogram").map(Err),
};
let elapsed_ms = u32::try_from(started.elapsed().as_millis()).unwrap_or(u32::MAX);
let histogram = ide_jvm_attach::parse_class_histogram(&text);
if histogram.skipped_lines > 0 {
tracing::debug!("class histogram of JVM {jvm}: {} unreadable lines", histogram.skipped_lines);
}
let (rows, counts) = diff_rows(histogram.rows, previous);
let snapshot = JvmMemorySnapshot {
total_instances: rows.iter().map(|r| r.instances).sum(),
total_bytes: rows.iter().map(|r| r.bytes).sum(),
rows,
java_version: version.map(|v| v.text).unwrap_or_default(),
taken_while_suspended: suspended,
has_baseline: previous.is_some(),
elapsed_ms,
source: JvmMemorySource::Histogram,
};
Ok(Ok((snapshot, counts)))
}
@@ -0,0 +1,119 @@
//! A scripted [`JvmInspector`] and a session without an adapter behind it.
use crate::SessionOwner;
use crate::adapter::{AdapterSpec, Transport};
use crate::connection::Connection;
use crate::session::Session;
use ide_core::{DebugSessionId, IdeEvent, SharedSink};
use ide_jvm_inspect::{
ClassCount, InspectError, InspectFuture, JvmField, JvmInspector, JvmReferrer, JvmValue, ObjectId, VmTarget,
};
use parking_lot::Mutex;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use tokio::sync::Notify;
#[derive(Default)]
pub(crate) struct FakeInspector {
pub targets: Mutex<Vec<VmTarget>>,
/// The `limit` of each `instances` / `referrers` call.
pub limits: Mutex<Vec<u32>>,
pub calls: AtomicUsize,
/// Returned by every call when set.
pub error: Mutex<Option<InspectError>>,
pub counts: Mutex<Vec<ClassCount>>,
pub instances: Mutex<Vec<JvmValue>>,
/// Every call signals `entered`, then waits for `release`.
pub hold: bool,
pub entered: Notify,
pub release: Notify,
}
impl FakeInspector {
pub fn failing(error: InspectError) -> Self {
Self { error: Mutex::new(Some(error)), ..Self::default() }
}
pub fn calls(&self) -> usize {
self.calls.load(Ordering::SeqCst)
}
async fn answer<T: Send>(&self, target: &VmTarget, limit: Option<u32>, ok: T) -> Result<T, InspectError> {
self.calls.fetch_add(1, Ordering::SeqCst);
self.targets.lock().push(target.clone());
self.limits.lock().extend(limit);
if self.hold {
self.entered.notify_one();
self.release.notified().await;
}
match self.error.lock().clone() {
Some(e) => Err(e),
None => Ok(ok),
}
}
}
impl JvmInspector for FakeInspector {
fn class_counts<'a>(&'a self, target: &'a VmTarget) -> InspectFuture<'a, Vec<ClassCount>> {
Box::pin(self.answer(target, None, self.counts.lock().clone()))
}
fn instances<'a>(&'a self, target: &'a VmTarget, _: &'a str, limit: u32) -> InspectFuture<'a, Vec<JvmValue>> {
Box::pin(self.answer(target, Some(limit), self.instances.lock().clone()))
}
fn fields<'a>(&'a self, target: &'a VmTarget, _: ObjectId) -> InspectFuture<'a, Vec<JvmField>> {
Box::pin(self.answer(target, None, Vec::new()))
}
fn referrers<'a>(&'a self, target: &'a VmTarget, _: ObjectId, limit: u32) -> InspectFuture<'a, Vec<JvmReferrer>> {
Box::pin(self.answer(target, Some(limit), Vec::new()))
}
}
pub(crate) struct BareSession {
pub session: Arc<Session>,
pub events: Arc<Mutex<Vec<IdeEvent>>>,
/// The adapter's end of the connection, kept open.
_adapter: tokio::io::DuplexStream,
}
/// A java-debug session that no adapter answers; must be called inside a Tokio runtime.
pub(crate) fn bare_session(request: &str) -> BareSession {
bare_session_of("java-debug", request)
}
/// A session of adapter `name` that no adapter answers; must be called inside a Tokio runtime.
pub(crate) fn bare_session_of(name: &str, request: &str) -> BareSession {
let events: Arc<Mutex<Vec<IdeEvent>>> = Arc::default();
let sink_events = events.clone();
let sink: SharedSink = Arc::new(move |e: IdeEvent| sink_events.lock().push(e));
let (client, adapter) = tokio::io::duplex(1 << 10);
let (reader, writer) = tokio::io::split(client);
let (conn, _) = Connection::start(reader, writer);
let spec = AdapterSpec {
adapter: name.into(),
command: String::new(),
args: Vec::new(),
transport: Transport::Stdio,
install_hint: None,
cwd: std::env::temp_dir(),
env: Vec::new(),
server_host: None,
connect: None,
pinned: None,
};
let session = Session::new(
DebugSessionId(1),
SessionOwner::new("test", sink),
"test".into(),
name.into(),
None,
request.into(),
conn,
spec,
None,
None,
);
BareSession { session: Arc::new(session), events, _adapter: adapter }
}
File diff suppressed because it is too large. Load diff
+7
View File
@@ -9,6 +9,13 @@ use std::{
pub const ADAPTER: &str = "kotlin-debug-adapter";
/// The adapter attaches to a JVM the IDE starts, through the JDWP link. It resumes the VM while
/// constructing its debuggee, before its breakpoint hooks exist, so the link holds resumes back
/// until the attach is done.
pub(crate) fn link_options() -> ide_jdwp::LinkOptions {
ide_jdwp::LinkOptions { hold_resumes: true }
}
pub struct SourceMirror {
dir: tempfile::TempDir,
paths: HashMap<PathBuf, PathBuf>,
-166
View File
@@ -1,166 +0,0 @@
//! Keep a suspended JVM stopped until Kotlin's adapter has installed its listeners.
//! The official 0.4.4 adapter resumes VMStart while constructing its debuggee, before
//! installing breakpoint hooks. DAP attach completes only after those hooks exist.
use ide_core::IdeResult;
use std::{io, time::Duration};
use tokio::{
io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt},
net::{TcpListener, TcpStream},
sync::oneshot,
task::JoinHandle,
};
pub(crate) struct StartupRelay {
release: Option<oneshot::Sender<()>>,
task: JoinHandle<()>,
}
impl StartupRelay {
pub async fn start(host: String, port: u16) -> IdeResult<(Self, u16)> {
let listener = TcpListener::bind(("127.0.0.1", 0)).await?;
let local = listener.local_addr()?.port();
let (release, ready) = oneshot::channel();
let task = tokio::spawn(async move {
let connect = async {
let (adapter, _) = listener.accept().await?;
let jvm = TcpStream::connect((host.as_str(), port)).await?;
adapter.set_nodelay(true)?;
jvm.set_nodelay(true)?;
Ok::<_, io::Error>((adapter, jvm))
};
if let Ok(Ok((adapter, jvm))) = tokio::time::timeout(Duration::from_secs(60), connect).await {
let (mut ar, mut aw) = adapter.into_split();
let (mut jr, mut jw) = jvm.into_split();
let upstream = forward(&mut ar, &mut jw, ready);
let downstream = tokio::io::copy(&mut jr, &mut aw);
// Close both sockets as soon as either side disconnects.
tokio::select! {
r = upstream => if let Err(e) = r { tracing::debug!("Kotlin JDWP relay: {e}"); },
_ = downstream => {},
}
}
});
Ok((Self { release: Some(release), task }, local))
}
pub fn release(&mut self) {
if let Some(ready) = self.release.take() {
let _ = ready.send(());
}
}
}
impl Drop for StartupRelay {
fn drop(&mut self) {
self.task.abort();
}
}
async fn packet(reader: &mut (impl AsyncRead + Unpin)) -> io::Result<Vec<u8>> {
let mut header = [0; 11];
reader.read_exact(&mut header).await?;
let len = u32::from_be_bytes(header[..4].try_into().unwrap()) as usize;
if !(11..=16 * 1024 * 1024).contains(&len) {
return Err(io::Error::new(io::ErrorKind::InvalidData, "Invalid JDWP packet length"));
}
let mut packet = vec![0; len];
packet[..11].copy_from_slice(&header);
reader.read_exact(&mut packet[11..]).await?;
Ok(packet)
}
async fn forward(
reader: &mut (impl AsyncRead + Unpin),
writer: &mut (impl AsyncWrite + Unpin),
mut ready: oneshot::Receiver<()>,
) -> io::Result<()> {
let mut handshake = [0; 14];
reader.read_exact(&mut handshake).await?;
if &handshake != b"JDWP-Handshake" {
return Err(io::Error::new(io::ErrorKind::InvalidData, "Invalid JDWP handshake"));
}
writer.write_all(&handshake).await?;
let mut released = false;
let mut held: Vec<Vec<u8>> = Vec::new();
loop {
// Retain the in-flight read when attach completes, including partial packets.
let read = packet(reader);
tokio::pin!(read);
let packet = tokio::select! {
result = &mut read => result?,
result = &mut ready, if !released => {
result.map_err(|_| io::Error::new(io::ErrorKind::ConnectionAborted, "Kotlin attach cancelled"))?;
released = true;
for packet in held.drain(..) {
writer.write_all(&packet).await?;
}
read.await?
}
};
// JDWP VirtualMachine.Resume (1/9), ThreadReference.Resume (11/3).
// Other commands must proceed: attach itself queries threads and installs requests.
if !released && packet[8] == 0 && matches!((packet[9], packet[10]), (1, 9) | (11, 3)) {
if held.len() >= 1024 {
return Err(io::Error::new(io::ErrorKind::InvalidData, "Too many pending JDWP resumes"));
}
held.push(packet);
} else {
writer.write_all(&packet).await?;
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn command(id: u8, set: u8, cmd: u8) -> Vec<u8> {
vec![0, 0, 0, 11, 0, 0, 0, id, 0, set, cmd]
}
#[tokio::test]
async fn holds_resumes_but_allows_setup_and_preserves_partial_reads() {
let (mut adapter, mut reader) = tokio::io::duplex(4096);
let (mut writer, mut jvm) = tokio::io::duplex(4096);
let (release, ready) = oneshot::channel();
let task = tokio::spawn(async move { forward(&mut reader, &mut writer, ready).await });
adapter.write_all(b"JDWP-Handshake").await.unwrap();
let mut handshake = [0; 14];
jvm.read_exact(&mut handshake).await.unwrap();
assert_eq!(&handshake, b"JDWP-Handshake");
let resume = command(1, 1, 9);
let thread_resume = command(2, 11, 3);
let setup = command(3, 1, 4);
adapter.write_all(&resume).await.unwrap();
adapter.write_all(&thread_resume).await.unwrap();
adapter.write_all(&setup).await.unwrap();
assert_eq!(packet(&mut jvm).await.unwrap(), setup);
let next = command(4, 1, 9);
adapter.write_all(&next[..6]).await.unwrap();
release.send(()).unwrap();
assert_eq!(packet(&mut jvm).await.unwrap(), resume);
assert_eq!(packet(&mut jvm).await.unwrap(), thread_resume);
adapter.write_all(&next[6..]).await.unwrap();
assert_eq!(packet(&mut jvm).await.unwrap(), next);
drop(adapter);
assert!(task.await.unwrap().is_err());
}
#[tokio::test]
async fn dropping_session_closes_relay() {
let (relay, port) = StartupRelay::start("127.0.0.1".into(), 1).await.unwrap();
let mut adapter = TcpStream::connect(("127.0.0.1", port)).await.unwrap();
drop(relay);
let result = tokio::time::timeout(Duration::from_secs(1), adapter.read_u8()).await.unwrap();
assert!(result.is_err());
}
#[tokio::test]
async fn rejects_invalid_packet_lengths() {
for len in [0_u32, 10, 16 * 1024 * 1024 + 1] {
let mut bytes = command(1, 1, 9);
bytes[..4].copy_from_slice(&len.to_be_bytes());
assert_eq!(packet(&mut bytes.as_slice()).await.unwrap_err().kind(), io::ErrorKind::InvalidData);
}
}
}
+129 -12
View File
@@ -13,8 +13,8 @@ pub mod attach;
mod cargo;
pub mod connection;
pub mod jdwp;
mod jvm_memory;
pub mod kotlin;
mod kotlin_jdwp;
mod session;
use adapter::{AdapterLauncher, AdapterSpec, ProcessLauncher};
@@ -22,6 +22,7 @@ use connection::{Connection, DapError};
use futures::future::join_all;
use ide_core::{DebugSessionId, IdeError, IdeResult, SharedSink};
use ide_language::LanguageRegistry;
pub use jvm_memory::*;
use parking_lot::Mutex;
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
@@ -126,6 +127,14 @@ pub struct DebugCapabilities {
pub log_points: bool,
/// HotSwap ("Reload Changed Classes"): java-debug's `redefineClasses` request.
pub hot_code_replace: bool,
/// The session debugs a JVM, so it has a Memory view; [`DebugManager::jvm_memory_classes`]
/// says why when it has no data (platform, remote JVM, ...).
pub jvm_memory: bool,
/// The session has a JVM inspector: instances, fields and referrers while suspended. Can
/// change during the session ([`ide_core::event::DebugEvent::Capabilities`]).
pub jvm_instances: bool,
/// The session can be asked for a heap dump ([`DebugManager::jvm_heap_dump`]).
pub jvm_heap: bool,
}
/// Result of [`DebugManager::redefine_classes`].
@@ -302,6 +311,7 @@ pub struct DebugManager {
data_breakpoints: Mutex<Vec<DataBreakpoint>>,
/// In-flight stops, by session: a second Stop sets `stop_force`, the waiting one kills.
stopping: Mutex<HashMap<DebugSessionId, Arc<Session>>>,
jvm_finder: Mutex<jvm_memory::JvmFinder>,
}
/// rdbg (Ruby's debug gem) starts the debuggee itself and ignores `launch`'s program,
@@ -566,9 +576,21 @@ pub(crate) fn capabilities_from(caps: &Value) -> DebugCapabilities {
hit_conditional_breakpoints: has("supportsHitConditionalBreakpoints"),
log_points: has("supportsLogPoints"),
hot_code_replace: false,
jvm_memory: false,
jvm_instances: false,
jvm_heap: false,
}
}
pub(crate) fn session_capabilities(session: &Session) -> DebugCapabilities {
let mut caps = capabilities_from(&session.caps.lock());
caps.hot_code_replace = HOT_CODE_REPLACE_ADAPTERS.contains(&session.adapter.as_str());
caps.jvm_memory = JVM_MEMORY_ADAPTERS.contains(&session.adapter.as_str());
caps.jvm_instances = session.jvm_inspector.lock().is_some();
caps.jvm_heap = JVM_MEMORY_ADAPTERS.contains(&session.adapter.as_str());
caps
}
/// States of a file's line breakpoints that no session has been sent.
fn unsent(breakpoints: &[Breakpoint]) -> Vec<BreakpointState> {
breakpoints
@@ -657,9 +679,15 @@ impl DebugManager {
function_breakpoints: Mutex::default(),
data_breakpoints: Mutex::default(),
stopping: Mutex::default(),
jvm_finder: Mutex::new(Arc::new(jvm_memory::find_local_jvm)),
})
}
#[cfg(test)]
pub(crate) fn set_jvm_finder(&self, finder: jvm_memory::JvmFinder) {
*self.jvm_finder.lock() = finder;
}
fn session(&self, s: DebugSessionId) -> IdeResult<Arc<Session>> {
self.sessions.lock().get(&s).cloned().ok_or_else(|| no_session(s))
}
@@ -979,13 +1007,31 @@ impl DebugManager {
.await?;
*s.caps.lock() = if caps.is_object() { caps } else { json!({}) };
// The Kotlin adapter ignores the resolved classpath and project runtime on launch.
// The Kotlin adapters ignore the resolved classpath and project runtime on launch.
let request = if s.adapter == kotlin::ADAPTER && s.request == "launch" {
session::start_kotlin_debuggee(s, &mut arguments).await?;
"attach"
} else {
&s.request
};
// Both blocks below run before the Kotlin link replaces the JVM's address with its own.
if request == "attach"
&& let Some(target) = jvm_memory::attach_target(&arguments)
{
let _ = s.attach_target.set(target);
}
if cfg!(unix)
&& s.request == "attach"
&& JVM_MEMORY_ADAPTERS.contains(&s.adapter.as_str())
&& let Some(port) = jvm_memory::local_jdwp_port(&arguments)
{
if jdwp::is_listen_relay(port) {
s.listen_relay.store(true, Ordering::Release);
} else {
let finder = self.jvm_finder.lock().clone();
jvm_memory::find_attached_jvm(s, port, finder).await;
}
}
if s.adapter == kotlin::ADAPTER {
let host = arguments
.get("hostName")
@@ -998,11 +1044,14 @@ impl DebugManager {
.and_then(|p| u16::try_from(p).ok())
.filter(|p| *p != 0)
.ok_or_else(|| IdeError::invalid("Invalid Kotlin JDWP port"))?;
let (relay, port) = kotlin_jdwp::StartupRelay::start(host, port).await?;
*s.kotlin_relay.lock() = Some(relay);
let (link, port) = ide_jdwp::JdwpLink::start(host, port, kotlin::link_options()).await?;
*s.jvm_inspector.lock() = Some(Arc::new(ide_jdwp::LinkInspector::new(link.clone())));
jvm_memory::watch_link(s, &link);
*s.jdwp_link.lock() = Some(link);
arguments["hostName"] = json!("127.0.0.1");
arguments["port"] = json!(port);
}
s.jvm_located.send_replace(true);
// Launch/attach may wait for a slow runtime or a remote target. Let the adapter report
// failure; Stop cancels configuration and a closed connection fails the pending reply.
// Many adapters answer only after `configurationDone`, so don't wait for the reply yet.
@@ -1097,8 +1146,8 @@ impl DebugManager {
reply.await.unwrap_or(Err(DapError::Closed))?;
}
}
if let Some(relay) = s.kotlin_relay.lock().as_mut() {
relay.release();
if let Some(link) = s.jdwp_link.lock().as_ref() {
link.release();
}
Ok(())
}
@@ -1331,8 +1380,9 @@ impl DebugManager {
}
pub async fn threads(&self, s: DebugSessionId) -> IdeResult<Vec<ThreadInfo>> {
let body = self.session(s)?.conn.request_timeout("threads", Value::Null, REQUEST_TIMEOUT).await?;
Ok(body
let session = self.session(s)?;
let body = session.conn.request_timeout("threads", Value::Null, REQUEST_TIMEOUT).await?;
let threads: Vec<ThreadInfo> = body
.get("threads")
.and_then(Value::as_array)
.map(|t| {
@@ -1343,7 +1393,8 @@ impl DebugManager {
})
.collect()
})
.unwrap_or_default())
.unwrap_or_default();
Ok(threads)
}
pub async fn stack_trace(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<Vec<StackFrame>> {
@@ -1472,8 +1523,11 @@ impl DebugManager {
pub async fn resume(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
let ticket = session.suspended.ticket();
let body = session.conn.request_timeout("continue", json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await?;
// DAP: a response without the flag continued every thread.
let all = body.get("allThreadsContinued").and_then(Value::as_bool).unwrap_or(true);
session.suspended.resumed(ticket, thread_id, all);
session.emit(ide_core::event::DebugEvent::Continued { thread_id, all_threads_continued: all });
Ok(())
}
@@ -1513,7 +1567,10 @@ impl DebugManager {
StepKind::Out => "stepOut",
};
let session = self.session(s)?;
let ticket = session.suspended.ticket();
session.conn.request_timeout(command, json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await?;
// A step runs its thread; other threads stopped by their own events stay stopped.
session.suspended.resumed(ticket, thread_id, false);
session.emit(ide_core::event::DebugEvent::Continued { thread_id, all_threads_continued: true });
Ok(())
}
@@ -1611,10 +1668,12 @@ impl DebugManager {
/// Step into a specific call of the current line (Smart Step Into).
pub async fn step_in_target(&self, s: DebugSessionId, thread_id: i64, target_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
let ticket = session.suspended.ticket();
session
.conn
.request_timeout("stepIn", json!({ "threadId": thread_id, "targetId": target_id }), REQUEST_TIMEOUT)
.await?;
session.suspended.resumed(ticket, thread_id, false);
session.emit(ide_core::event::DebugEvent::Continued { thread_id, all_threads_continued: true });
Ok(())
}
@@ -1648,10 +1707,68 @@ impl DebugManager {
}
pub fn capabilities(&self, s: DebugSessionId) -> IdeResult<DebugCapabilities> {
Ok(session_capabilities(&*self.session(s)?))
}
/// Class table of the session's JVM (`GC.class_histogram -all`, or counted by the debugger
/// while suspended where no histogram can be taken), with deltas against the session's
/// previous snapshot from the same source.
pub async fn jvm_memory_classes(&self, s: DebugSessionId) -> IdeResult<JvmMemoryResult> {
let session = self.session(s)?;
let mut caps = capabilities_from(&session.caps.lock());
caps.hot_code_replace = HOT_CODE_REPLACE_ADAPTERS.contains(&session.adapter.as_str());
Ok(caps)
jvm_memory::session_classes(session).await
}
/// Writes a heap dump of the session's JVM to `dir/dump.hprof`, one class-table or dump
/// request per session at a time. Anything but `Ready` deletes `dir`, unless `s` is not a
/// session. On `cancel` the result is `Cancelled` at once, while a dump the JVM is writing
/// still holds the session's turn until the JVM answers.
pub async fn jvm_heap_dump(
&self,
s: DebugSessionId,
dir: PathBuf,
cancel: ide_core::CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
let session = self.session(s)?;
jvm_memory::session_heap_dump(session, dir, cancel).await
}
/// Gives the session the backend for [`Self::jvm_memory_instances`] and the debugger's class
/// counts; the UI is told to reload the session's capabilities.
pub fn set_jvm_inspector(
&self,
s: DebugSessionId,
inspector: Arc<dyn ide_jvm_inspect::JvmInspector>,
) -> IdeResult<()> {
let session = self.session(s)?;
if session.ended.is_cancelled() {
return Err(no_session(s));
}
jvm_memory::set_inspector(&session, inspector);
Ok(())
}
/// Up to `limit` (clamped to `1..=MAX_INSTANCES`) instances of `class_name` while suspended.
pub async fn jvm_memory_instances(
&self,
s: DebugSessionId,
class_name: String,
limit: u32,
) -> IdeResult<JvmInstancesResult> {
jvm_memory::instances(&*self.session(s)?, &class_name, limit).await
}
pub async fn jvm_memory_fields(&self, s: DebugSessionId, object: JvmObjectRef) -> IdeResult<JvmFieldsResult> {
jvm_memory::fields(&*self.session(s)?, object).await
}
/// Objects that point at `object`; `limit` 0 means `DEFAULT_REFERRERS`, at most `MAX_REFERRERS`.
pub async fn jvm_memory_referrers(
&self,
s: DebugSessionId,
object: JvmObjectRef,
limit: u32,
) -> IdeResult<JvmReferrersResult> {
jvm_memory::referrers(&*self.session(s)?, object, limit).await
}
/// HotSwap: reloads the classes compiled since the session started (or since the
+315 -42
View File
@@ -5,6 +5,8 @@ use crate::adapter::AdapterSpec;
use crate::connection::{Connection, DapError, Incoming};
use crate::{Breakpoint, BreakpointState, DataBreakpoint, DebugManager, FunctionBreakpoint, SessionOwner};
use ide_core::event::DebugEvent;
#[cfg(unix)]
use ide_core::process::process_tree;
use ide_core::{DebugSessionId, IdeEvent, TaskId};
use parking_lot::Mutex;
use serde_json::{Value, json};
@@ -62,6 +64,15 @@ pub(crate) struct Session {
pub stop_force: AtomicBool,
/// `systemProcessId` from the adapter's `process` event: the program being debugged.
pub debuggee_pid: AtomicI64,
/// The JVM an app-launched (Kotlin) session started, or the local JVM an attach session
/// found by its JDWP port. Separate from `debuggee_pid`, which decides how Stop signals the
/// program and is 0 for Kotlin.
pub jvm_pid: AtomicI64,
/// JDWP port of an attach to a local JVM, to look the JVM up again; unset for a remote one.
pub jdwp_port: std::sync::OnceLock<u16>,
/// The attach goes through the listen-mode relay of [`crate::jdwp::listen_for_jvm`].
pub listen_relay: AtomicBool,
pub suspended: Suspension,
/// Pids signalled for the current stop: the reported pid and everything it started (an
/// adapter may report the launcher shell, whose child is the program).
#[cfg_attr(not(unix), allow(dead_code))]
@@ -75,8 +86,105 @@ pub(crate) struct Session {
/// Resolved path → the path as the IDE knows it, for adapters given resolved paths.
resolved_paths: Mutex<HashMap<PathBuf, PathBuf>>,
progress: Mutex<HashMap<String, TaskId>>,
pub kotlin_relay: Mutex<Option<crate::kotlin_jdwp::StartupRelay>>,
/// Relay between a Kotlin adapter and its JVM. Dropping it closes both sockets.
pub jdwp_link: parking_lot::Mutex<Option<ide_jdwp::JdwpLink>>,
pub jvm_inspector: parking_lot::Mutex<Option<Arc<dyn ide_jvm_inspect::JvmInspector>>>,
/// One inspector request per session at a time.
pub inspect_lock: tokio::sync::Mutex<()>,
/// Cancelled when the session ends, so a request still waiting on the inspector lets go of
/// it (and of the JDWP link it holds).
pub ended: ide_core::CancellationToken,
/// `hostName` (empty when absent) and `port` of the session's `attach` request, as the IDE
/// was given them: Kotlin then replaces them with the link's. Unlike `jdwp_port`, also set
/// for a remote JVM.
pub attach_target: std::sync::OnceLock<(String, u16)>,
/// Set once `configure` has recorded where the program's JVM is (the fields above, the
/// Kotlin link), just before the `launch`/`attach` request goes out.
pub jvm_located: watch::Sender<bool>,
pub kotlin_mirror: Mutex<Option<crate::kotlin::SourceMirror>>,
pub jvm_memory: tokio::sync::Mutex<crate::JvmMemoryState>,
}
/// Which threads of the program are suspended: each from its `stopped` event until a
/// `continued` event or a successful resume/step request for it. Never cleared elsewhere, since
/// a stale "suspended" only makes a later check refuse. fwcd and java-debug stop a breakpoint's
/// thread alone, so a thread can run while another stays stopped.
///
/// A resume reply ends only the stops taken before the request was sent: a thread's next
/// `stopped` event can overtake the reply. The check and the removal happen under one lock, so
/// no stop can land between them and be lost.
#[derive(Default)]
pub(crate) struct Suspension(Mutex<Stops>);
#[derive(Default)]
struct Stops {
/// Counts stops, and moves back to an earlier stopped thread: an object read under one
/// ticket is stale under another.
ticket: u64,
/// Threads stopped alone by their event, oldest first, with the ticket of that stop.
threads: Vec<(i64, u64)>,
/// The ticket of a stop of every thread (`allThreadsStopped`, or no thread id): any resume
/// ends it.
all: Option<u64>,
}
impl Stops {
fn latest(&self) -> Option<u64> {
self.threads.last().map(|&(_, ticket)| ticket).max(self.all)
}
/// Ends the stops taken by ticket `upto` of `thread`, or of every thread.
fn release(&mut self, thread: Option<i64>, every: bool, upto: u64) {
let latest = self.latest();
if every || thread.is_none() || self.all.is_some_and(|t| t <= upto) {
self.all = self.all.filter(|&t| t > upto);
self.threads.retain(|&(_, t)| t > upto);
} else {
self.threads.retain(|&(id, t)| Some(id) != thread || t > upto);
}
// Back on an earlier thread's stop, as the JDWP link moves to it with a new generation.
let now = self.latest();
if now.is_some() && now != latest {
self.ticket += 1;
}
}
}
impl Suspension {
/// Whether some thread is stopped.
pub fn get(&self) -> bool {
let stops = self.0.lock();
stops.all.is_some() || !stops.threads.is_empty()
}
/// A `stopped` event; `all`: its `allThreadsStopped`.
pub fn stopped(&self, thread: Option<i64>, all: bool) {
let mut stops = self.0.lock();
stops.ticket += 1;
let ticket = stops.ticket;
match thread.filter(|_| !all) {
Some(thread) => {
stops.threads.retain(|&(id, _)| id != thread);
stops.threads.push((thread, ticket));
}
None => stops.all = Some(ticket),
}
}
/// A `continued` event; `all`: its `allThreadsContinued`.
pub fn continued(&self, thread: Option<i64>, all: bool) {
self.0.lock().release(thread, all, u64::MAX);
}
/// Take before sending a resume-like request and pass to [`Self::resumed`].
pub fn ticket(&self) -> u64 {
self.0.lock().ticket
}
/// After a successful resume-like request for `thread`; `all`: every thread ran.
pub fn resumed(&self, ticket: u64, thread: i64, all: bool) {
self.0.lock().release(Some(thread), all, ticket);
}
}
/// Adapters that only match files by their resolved path: Godot maps a script to
@@ -125,6 +233,10 @@ impl Session {
terminated: AtomicBool::new(false),
stop_force: AtomicBool::new(false),
debuggee_pid: AtomicI64::new(0),
jvm_pid: AtomicI64::new(0),
jdwp_port: std::sync::OnceLock::new(),
listen_relay: AtomicBool::new(false),
suspended: Suspension::default(),
stop_pids: parking_lot::Mutex::new(Vec::new()),
#[cfg(windows)]
stop_tree: Mutex::new(None),
@@ -132,8 +244,14 @@ impl Session {
breakpoint_ids: Mutex::new(HashMap::new()),
resolved_paths: Mutex::new(HashMap::new()),
progress: Mutex::new(HashMap::new()),
kotlin_relay: Mutex::new(None),
jdwp_link: parking_lot::Mutex::new(None),
jvm_inspector: parking_lot::Mutex::new(None),
inspect_lock: tokio::sync::Mutex::default(),
ended: ide_core::CancellationToken::new(),
attach_target: std::sync::OnceLock::new(),
jvm_located: watch::channel(false).0,
kotlin_mirror: Mutex::new(None),
jvm_memory: tokio::sync::Mutex::default(),
}
}
@@ -294,7 +412,10 @@ impl Session {
/// Kills the adapter and every debuggee we started.
pub fn kill_processes(&self) {
self.kotlin_relay.lock().take();
self.ended.cancel();
// The inspector holds a clone of the link, so both go.
self.jvm_inspector.lock().take();
self.jdwp_link.lock().take();
#[cfg(unix)]
if self.request == "launch" {
for group in launch_groups(self) {
@@ -351,32 +472,6 @@ pub(crate) async fn remember_debuggee(s: &Arc<Session>) {
let _ = s;
}
/// The program and everything it started: an adapter may report the launcher shell, whose
/// child is the debugged program, so stopping only the reported pid leaves the program alive.
#[cfg(unix)]
fn process_tree(pid: i32) -> Vec<i32> {
use std::collections::HashMap;
let Ok(out) = std::process::Command::new("ps").args(["-A", "-o", "pid=,ppid="]).output() else {
return vec![pid];
};
let mut children: HashMap<i32, Vec<i32>> = HashMap::new();
for line in String::from_utf8_lossy(&out.stdout).lines() {
let mut it = line.split_whitespace();
let (Some(p), Some(parent)) = (it.next(), it.next()) else { continue };
let (Ok(p), Ok(parent)) = (p.parse::<i32>(), parent.parse::<i32>()) else { continue };
children.entry(parent).or_default().push(p);
}
let mut tree = vec![pid];
let mut at = 0;
while at < tree.len() {
if let Some(kids) = children.get(&tree[at]) {
tree.extend(kids.iter().copied());
}
at += 1;
}
tree
}
/// Asks the program itself to exit (`force` = SIGKILL) without ending the adapter session:
/// its output keeps flowing over DAP while it shuts down. The first call remembers the pid
/// tree, because the reported pid may die before its program does.
@@ -600,18 +695,27 @@ async fn on_event(mgr: &Arc<DebugManager>, s: &Arc<Session>, event: &str, mut bo
s.emit(DebugEvent::Initialized);
}
"stopped" => {
let all = body.get("allThreadsStopped").and_then(Value::as_bool).unwrap_or(false);
s.suspended.stopped(int("threadId"), all);
let reason = str_("reason").unwrap_or_else(|| "pause".into());
mgr.restore_breakpoints(s).await;
s.emit(DebugEvent::Stopped {
thread_id: int("threadId").unwrap_or(0),
reason: str_("reason").unwrap_or_else(|| "pause".into()),
reason,
description: str_("description").or_else(|| str_("text")),
all_threads_stopped: body.get("allThreadsStopped").and_then(Value::as_bool).unwrap_or(false),
all_threads_stopped: all,
});
}
"continued" => {
let all = body.get("allThreadsContinued").and_then(Value::as_bool);
// DAP: an event without the flag continues its thread alone. The UI keeps treating
// it as all threads; the suspension, which holds only threads that stopped, need not.
s.suspended.continued(int("threadId"), all.unwrap_or(false));
s.emit(DebugEvent::Continued {
thread_id: int("threadId").unwrap_or(0),
all_threads_continued: all.unwrap_or(true),
});
}
"continued" => s.emit(DebugEvent::Continued {
thread_id: int("threadId").unwrap_or(0),
all_threads_continued: body.get("allThreadsContinued").and_then(Value::as_bool).unwrap_or(true),
}),
"output" => {
let category = str_("category").unwrap_or_else(|| "console".into());
if category == "telemetry" {
@@ -917,6 +1021,19 @@ fn forward_output<R: AsyncRead + Unpin + Send + 'static>(s: &Session, mut stream
});
}
/// The attach listener must be on the command line before the user's arguments, which may
/// override it; the memory view needs it because the JVM is started by the IDE, not attached to.
/// Not on Windows (`windows`), which the memory view does not support.
fn kotlin_jvm_args(user: Vec<String>, windows: bool) -> Vec<String> {
if windows || !ide_jvm_attach::wants_listener_flag(&user.join(" ")) {
return user;
}
let mut args = Vec::with_capacity(user.len() + 1);
args.push(ide_jvm_attach::START_LISTENER_FLAG.to_owned());
args.extend(user);
args
}
/// Launch with the IDE's resolved runtime and classpath; the adapter only attaches.
pub(crate) async fn start_kotlin_debuggee(s: &Arc<Session>, args: &mut Value) -> ide_core::IdeResult<()> {
use tokio::io::{AsyncBufReadExt, BufReader};
@@ -931,7 +1048,7 @@ pub(crate) async fn start_kotlin_debuggee(s: &Arc<Session>, args: &mut Value) ->
std::env::join_paths(strings("classPaths")).map_err(|e| ide_core::IdeError::invalid(e.to_string()))?;
let mut cmd = tokio::process::Command::new(java);
ide_core::process::prepare(cmd.as_std_mut());
cmd.args(strings("vmArguments"))
cmd.args(kotlin_jvm_args(strings("vmArguments"), cfg!(windows)))
// Let the JVM pick its own port (`:0`) and report it below: pre-picking a free
// port here races with other processes that may take it before the JVM binds.
.arg("-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=127.0.0.1:0")
@@ -965,11 +1082,12 @@ pub(crate) async fn start_kotlin_debuggee(s: &Arc<Session>, args: &mut Value) ->
// forced stop), the session's job on Windows, so a stop reaches what it forks.
ide_core::process::own_group(cmd.as_std_mut());
let mut child = cmd.spawn()?;
#[cfg(windows)]
if let Some(pid) = child.id()
&& let Some(job) = s.job.lock().as_ref()
{
job.assign_pid(pid);
if let Some(pid) = child.id() {
s.jvm_pid.store(i64::from(pid), Ordering::Release);
#[cfg(windows)]
if let Some(job) = s.job.lock().as_ref() {
job.assign_pid(pid);
}
}
if let Some(err) = child.stderr.take() {
forward_output(s, err, "stderr");
@@ -1010,7 +1128,162 @@ fn parse_jdwp_port(line: &str) -> Option<u16> {
#[cfg(test)]
mod tests {
use super::parse_jdwp_port;
use super::{Suspension, kotlin_jvm_args, parse_jdwp_port};
#[test]
fn a_resume_reply_clears_the_stop_it_was_sent_for() {
let s = Suspension::default();
assert!(!s.get());
s.stopped(Some(1), true);
let ticket = s.ticket();
s.resumed(ticket, 1, true);
assert!(!s.get());
}
#[test]
fn a_stop_after_the_resume_request_keeps_the_session_suspended() {
let s = Suspension::default();
s.stopped(Some(1), true);
let ticket = s.ticket();
// The next breakpoint's `stopped` event overtakes the reply to `continue`.
s.continued(Some(1), true);
s.stopped(Some(1), true);
s.resumed(ticket, 1, true);
assert!(s.get());
// A reply for the new stop still clears it.
s.resumed(s.ticket(), 1, true);
assert!(!s.get());
}
#[test]
fn a_reply_after_a_continued_event_changes_nothing() {
let s = Suspension::default();
s.stopped(Some(1), true);
let ticket = s.ticket();
s.continued(Some(1), true);
s.resumed(ticket, 1, true);
assert!(!s.get());
s.stopped(Some(1), true);
assert!(s.get());
}
#[test]
fn concurrent_stops_are_never_lost() {
let s = std::sync::Arc::new(Suspension::default());
s.stopped(Some(1), true);
let ticket = s.ticket();
let stopping = {
let s = s.clone();
std::thread::spawn(move || (0..10_000).for_each(|_| s.stopped(Some(1), true)))
};
for _ in 0..10_000 {
s.resumed(ticket, 1, true);
}
stopping.join().unwrap();
// However they interleave, a resume for the old ticket never clears a later stop.
assert!(s.get());
assert_eq!(s.ticket(), ticket + 10_000);
}
/// fwcd and java-debug stop a breakpoint's thread alone (`SUSPEND_EVENT_THREAD`): 7, then 8.
fn two_threads_stopped() -> Suspension {
let s = Suspension::default();
s.stopped(Some(7), false);
s.stopped(Some(8), false);
s
}
#[test]
fn continuing_the_latest_stopped_thread_keeps_the_other_as_a_new_stop() {
let s = two_threads_stopped();
let ticket = s.ticket();
s.continued(Some(8), false);
assert!(s.get());
// Objects read on thread 8's stop are stale.
assert_eq!(s.ticket(), ticket + 1);
s.continued(Some(7), false);
assert!(!s.get());
}
#[test]
fn continuing_an_earlier_stopped_thread_keeps_the_latest_stop() {
let s = two_threads_stopped();
let ticket = s.ticket();
s.continued(Some(7), false);
assert!(s.get());
assert_eq!(s.ticket(), ticket);
}
#[test]
fn continuing_all_threads_ends_every_stop() {
let s = two_threads_stopped();
s.continued(Some(7), true);
assert!(!s.get());
let s = two_threads_stopped();
s.continued(None, false);
assert!(!s.get());
}
#[test]
fn continuing_a_thread_that_did_not_stop_changes_nothing() {
let s = two_threads_stopped();
let ticket = s.ticket();
s.continued(Some(5), false);
assert!(s.get());
assert_eq!(s.ticket(), ticket);
}
#[test]
fn a_step_reply_resumes_its_thread_alone() {
let s = two_threads_stopped();
let ticket = s.ticket();
s.resumed(ticket, 8, false);
assert!(s.get());
assert_eq!(s.ticket(), ticket + 1);
s.resumed(s.ticket(), 7, true);
assert!(!s.get());
}
#[test]
fn a_thread_stopping_again_before_its_resume_reply_stays_stopped() {
let s = two_threads_stopped();
let ticket = s.ticket();
// Thread 7's step ends, and its `stopped` event overtakes the reply to `next`.
s.stopped(Some(7), false);
s.resumed(ticket, 7, false);
s.continued(Some(8), false);
assert!(s.get());
}
#[test]
fn a_stop_of_every_thread_ends_with_any_resume() {
let s = Suspension::default();
s.stopped(Some(7), false);
s.stopped(Some(8), true);
s.continued(Some(8), false);
assert!(!s.get());
// A stop without a thread.
s.stopped(None, false);
s.resumed(s.ticket(), 5, false);
assert!(!s.get());
}
#[test]
fn kotlin_launch_starts_attach_listener_first() {
assert_eq!(
kotlin_jvm_args(vec!["-Xmx1g".into()], false),
vec!["-XX:+StartAttachListener".to_string(), "-Xmx1g".into()]
);
assert_eq!(
kotlin_jvm_args(vec!["-XX:+DisableAttachMechanism".into()], false),
vec!["-XX:+DisableAttachMechanism".to_string()]
);
}
#[test]
fn kotlin_launch_on_windows_gets_no_attach_listener() {
assert_eq!(kotlin_jvm_args(vec!["-Xmx1g".into()], true), vec!["-Xmx1g".to_string()]);
}
#[test]
fn reads_the_port_the_jvm_chose() {
+471 -17
View File
@@ -1,5 +1,7 @@
//! Protocol flow against an in-memory fake adapter.
mod jvm_inspect;
use super::*;
use crate::adapter::{AdapterIo, LaunchFuture};
use ide_core::IdeEvent;
@@ -120,6 +122,14 @@ struct Script {
supports_terminate: bool,
/// Answer `setBreakpoints` only once this many are waiting (a client that waits for each never gets one).
hold_set_breakpoints: usize,
/// Send a `continued` event right after the first `stopped` one.
continued_after_stop: bool,
/// Close the connection when a `pause` request arrives (the adapter dies).
hang_up_on_pause: bool,
/// Answer `continue` with this body and keep the program running, instead of letting it exit.
continue_reply: Option<Value>,
/// Answer `initialize` only after this long (a slow adapter keeps the session connecting).
delay_initialize: Option<Duration>,
initialized_delay: Duration,
launch_delay: Duration,
/// Leave startup pending before `initialized` (false) or after configuration (true).
@@ -144,6 +154,10 @@ impl Default for Script {
output_after_terminate: false,
supports_terminate: true,
hold_set_breakpoints: 0,
continued_after_stop: false,
hang_up_on_pause: false,
continue_reply: None,
delay_initialize: None,
initialized_delay: Duration::ZERO,
launch_delay: Duration::ZERO,
pending_startup: None,
@@ -210,6 +224,9 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
}
match msg["command"].as_str().unwrap_or_default() {
"initialize" => {
if let Some(delay) = script.delay_initialize {
tokio::time::sleep(delay).await;
}
ok!(
msg,
json!({
@@ -341,6 +358,9 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
send!(
json!({"type": "event", "event": "stopped", "body": {"reason": "breakpoint", "threadId": 1, "allThreadsStopped": true}})
);
if script.continued_after_stop {
send!(json!({"type": "event", "event": "continued", "body": {"threadId": 1}}));
}
send!(
json!({"type": "event", "event": "breakpoint", "body": {"reason": "changed", "breakpoint": {"id": 1, "verified": false, "message": "moved", "line": 9, "source": {"path": script.source_path}}}})
);
@@ -372,11 +392,13 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
{"label": ""},
]})
),
"continue" if script.continue_reply.is_some() => ok!(msg, script.continue_reply.clone().unwrap()),
"continue" => {
ok!(msg, json!({"allThreadsContinued": true}));
send!(json!({"type": "event", "event": "exited", "body": {"exitCode": 3}}));
send!(json!({"type": "event", "event": "terminated"}));
}
"pause" if script.hang_up_on_pause => break,
"stepIn" | "stepOut" | "pause" => ok!(msg, Value::Null),
"redefineClasses" => {
send!(
@@ -430,6 +452,8 @@ fn harness(script: Script) -> Harness {
Arc::new(LanguageRegistry::from_configs(Vec::new())),
Arc::new(FakeLauncher { log: log.clone(), script }),
);
// A local attach must not send requests to the developer's own JVMs.
manager.set_jvm_finder(Arc::new(|_, _| None));
Harness { manager, log, events, owner }
}
@@ -1352,6 +1376,36 @@ async fn hot_swap_only_for_java_debug() {
assert_eq!(failed.error_message.as_deref(), Some("boom"));
}
/// A loopback port nothing listens on until `listen` is called on the socket, and that no parallel
/// test can bind meanwhile. A port picked by binding to 0 and closing the listener can be taken
/// by another test before the code under test connects to it.
fn reserved_port() -> (tokio::net::TcpSocket, u16) {
// Without SO_REUSEADDR, which every `TcpListener::bind` sets: no other socket may bind it.
let socket = tokio::net::TcpSocket::new_v4().unwrap();
socket.bind(([127, 0, 0, 1], 0).into()).unwrap();
let port = socket.local_addr().unwrap().port();
(socket, port)
}
/// An editor that starts listening on the reserved port after `delay` and serves one session.
#[cfg(unix)]
fn fake_editor(socket: tokio::net::TcpSocket, delay: Duration, log: Log) {
tokio::spawn(async move {
tokio::time::sleep(delay).await;
let listener = socket.listen(1).unwrap();
let (stream, _) = listener.accept().await.unwrap();
// Closed once its session is accepted, so a later start finds no editor open.
drop(listener);
fake_adapter(stream, log, Script::default()).await;
});
}
#[tokio::test]
async fn a_reserved_port_cannot_be_taken_by_a_parallel_test() {
let (_socket, port) = reserved_port();
assert!(tokio::net::TcpListener::bind(("127.0.0.1", port)).await.is_err(), "port {port} is still free to bind");
}
fn godot_registry(port: u16, command: &str, args: &[&str]) -> Arc<LanguageRegistry> {
let lang = ide_language::LanguageConfig {
id: ide_core::LanguageId::new("gdscript"),
@@ -1419,7 +1473,7 @@ async fn godot_connects_to_the_editor_adapter() {
#[tokio::test]
async fn godot_without_editor_or_executable_says_what_to_do() {
let port = std::net::TcpListener::bind(("127.0.0.1", 0)).unwrap().local_addr().unwrap().port();
let (_no_editor, port) = reserved_port();
let (owner, _events) = recording_owner("main");
let manager = DebugManager::new(godot_registry(port, "openidea-no-such-godot", &[]));
let err = manager.start(godot_launch(&std::env::temp_dir()), owner).await.unwrap_err();
@@ -1432,15 +1486,9 @@ async fn godot_without_editor_or_executable_says_what_to_do() {
#[cfg(unix)]
#[tokio::test]
async fn godot_starts_a_headless_editor_when_none_is_open() {
let port = std::net::TcpListener::bind(("127.0.0.1", 0)).unwrap().local_addr().unwrap().port();
let (editor, port) = reserved_port();
let log: Log = Arc::default();
let server_log = log.clone();
tokio::spawn(async move {
tokio::time::sleep(std::time::Duration::from_millis(300)).await;
let listener = tokio::net::TcpListener::bind(("127.0.0.1", port)).await.unwrap();
let (stream, _) = listener.accept().await.unwrap();
fake_adapter(stream, server_log, Script::default()).await;
});
fake_editor(editor, Duration::from_millis(300), log.clone());
let (owner, _events) = recording_owner("main");
// `sleep` stands in for `godot --headless --editor --path ${root} …`.
let manager = DebugManager::new(godot_registry(port, "sleep", &["30"]));
@@ -1643,7 +1691,7 @@ async fn a_plugin_language_runs_its_debug_adapter_from_the_approved_program() {
std::fs::write(&program, "#!/bin/sh\nexec sleep 30\n").unwrap();
std::fs::set_permissions(&program, std::fs::Permissions::from_mode(0o755)).unwrap();
let port = std::net::TcpListener::bind(("127.0.0.1", 0)).unwrap().local_addr().unwrap().port();
let (editor, port) = reserved_port();
let mut gdscript = parse_definition(include_str!("../../../languages/gdscript.toml")).unwrap();
let debugger = gdscript.debugger.as_mut().unwrap();
debugger.args.clear();
@@ -1660,13 +1708,7 @@ async fn a_plugin_language_runs_its_debug_adapter_from_the_approved_program() {
let _language = store.add("acme.godot".into(), gdscript);
let log: Log = Arc::default();
let server_log = log.clone();
tokio::spawn(async move {
tokio::time::sleep(std::time::Duration::from_millis(300)).await;
let listener = tokio::net::TcpListener::bind(("127.0.0.1", port)).await.unwrap();
let (stream, _) = listener.accept().await.unwrap();
fake_adapter(stream, server_log, Script::default()).await;
});
fake_editor(editor, Duration::from_millis(300), log.clone());
let (owner, _events) = recording_owner("main");
let manager = DebugManager::new(registry.clone());
let id = manager.start(godot_launch(&dir), owner).await.unwrap();
@@ -2018,6 +2060,418 @@ fn delve_leftovers_are_removed_without_touching_other_files() {
std::fs::remove_dir_all(&dir).unwrap();
}
fn is_suspended(h: &Harness, id: DebugSessionId) -> bool {
h.manager.session(id).unwrap().suspended.get()
}
#[tokio::test]
async fn stopped_event_marks_session_suspended() {
let h = harness(Script::default());
let id = h.start(config()).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
assert!(is_suspended(&h, id));
}
#[tokio::test]
async fn resume_clears_suspended_without_continued_event() {
let h = harness(Script::default());
let id = h.start(config()).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
// The fake adapter ends the session after `continue`, so keep the session alive here.
let session = h.manager.session(id).unwrap();
assert!(session.suspended.get());
h.manager.resume(id, 1).await.unwrap();
assert!(!session.suspended.get());
}
/// After the fake adapter's stop of every thread ends, threads 7 then 8 stop alone, as a
/// breakpoint does on fwcd and java-debug (`SUSPEND_EVENT_THREAD`).
async fn threads_stopped(script: Script) -> (Harness, Arc<Session>, DebugSessionId) {
let h = harness(script);
let id = h.start(config()).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let session = h.manager.session(id).unwrap();
session.suspended.continued(Some(1), true);
session.suspended.stopped(Some(7), false);
session.suspended.stopped(Some(8), false);
(h, session, id)
}
#[tokio::test]
async fn resume_of_one_thread_keeps_the_others_suspended() {
let only_it = Script { continue_reply: Some(json!({"allThreadsContinued": false})), ..Script::default() };
let (h, session, id) = threads_stopped(only_it).await;
h.manager.resume(id, 7).await.unwrap();
assert!(session.suspended.get());
h.manager.resume(id, 8).await.unwrap();
assert!(!session.suspended.get());
}
#[tokio::test]
async fn resume_reply_without_the_flag_continues_every_thread() {
let (h, session, id) = threads_stopped(Script { continue_reply: Some(json!({})), ..Script::default() }).await;
h.manager.resume(id, 7).await.unwrap();
assert!(!session.suspended.get());
}
#[tokio::test]
async fn step_of_one_thread_keeps_the_others_suspended() {
let (h, session, id) = threads_stopped(Script::default()).await;
let ticket = session.suspended.ticket();
h.manager.step(id, 8, StepKind::Into).await.unwrap();
assert!(session.suspended.get());
// Thread 8 was the latest stop: what was read on it is stale.
assert_eq!(session.suspended.ticket(), ticket + 1);
h.manager.step_in_target(id, 7, 1).await.unwrap();
assert!(!session.suspended.get());
}
#[tokio::test]
async fn continued_event_clears_suspended() {
let h = harness(Script { continued_after_stop: true, ..Script::default() });
let id = h.start(config()).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Continued { .. })).await;
assert!(!is_suspended(&h, id));
}
fn java_config(request: &str) -> LaunchConfig {
LaunchConfig { adapter: "java-debug".into(), request: request.into(), extra_json: None, ..config() }
}
fn remote_attach(adapter: &str, project: &std::path::Path) -> LaunchConfig {
LaunchConfig {
adapter: adapter.into(),
cwd: Some(project.to_owned().into()),
extra_json: Some(json!({"projectRoot": project, "hostName": "10.0.0.5", "port": 5005}).to_string()),
..java_config("attach")
}
}
#[tokio::test]
async fn attach_session_reports_capability() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
for adapter in ["java-debug", kotlin::ADAPTER] {
let id = h.start(remote_attach(adapter, project.path())).await.unwrap();
let caps = h.manager.capabilities(id).unwrap();
assert!(caps.jvm_memory, "{adapter}");
assert_eq!(caps.jvm_heap, caps.jvm_memory, "{adapter}");
}
let id = h.start(config()).await.unwrap();
let caps = h.manager.capabilities(id).unwrap();
assert!(!caps.jvm_memory);
assert_eq!(caps.jvm_heap, caps.jvm_memory);
assert_eq!(
h.manager.jvm_memory_classes(id).await.unwrap(),
JvmMemoryResult::Unavailable { reason: JvmMemoryUnavailable::NoJvmProcess, can_count_when_paused: false }
);
}
#[cfg(unix)]
#[tokio::test]
async fn ended_attach_session_without_pid_reports_program_exited() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(remote_attach("java-debug", project.path())).await.unwrap();
h.manager.session(id).unwrap().terminated.store(true, Ordering::Release);
assert_eq!(
h.manager.jvm_memory_classes(id).await.unwrap(),
JvmMemoryResult::Unavailable { reason: JvmMemoryUnavailable::ProgramExited, can_count_when_paused: false }
);
}
#[cfg(unix)]
#[tokio::test]
async fn listen_mode_attach_is_not_scanned() {
let h = harness(Script::default());
let scans = Arc::new(Mutex::new(Vec::new()));
let counted = scans.clone();
h.manager.set_jvm_finder(Arc::new(move |port, _| {
counted.lock().push(port);
None
}));
let (_jvm, relay) = jdwp::tests::connected_jvm().await;
let mut cfg = java_config("attach");
cfg.extra_json = Some(json!({"hostName": "127.0.0.1", "port": relay}).to_string());
let id = h.start(cfg).await.unwrap();
assert_eq!(
h.manager.jvm_memory_classes(id).await.unwrap(),
JvmMemoryResult::Unavailable {
reason: JvmMemoryUnavailable::ListenModeNotSupported,
can_count_when_paused: false
}
);
tokio::time::sleep(Duration::from_millis(50)).await;
assert!(scans.lock().is_empty());
}
#[cfg(unix)]
#[tokio::test]
async fn remote_host_is_not_scanned() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(remote_attach("java-debug", project.path())).await.unwrap();
let session = h.manager.session(id).unwrap();
assert_eq!(session.jdwp_port.get(), None);
assert_eq!(
h.manager.jvm_memory_classes(id).await.unwrap(),
JvmMemoryResult::Unavailable {
reason: JvmMemoryUnavailable::AttachedJvmNotFound,
can_count_when_paused: false
}
);
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 0);
}
/// The Memory view asks as soon as the session appears, while its JVM is still being looked up:
/// answering then would claim there is no JVM to ask.
#[cfg(unix)]
#[tokio::test]
async fn memory_asked_while_an_attach_connects_waits_for_its_jvm() {
let h = harness(Script { delay_initialize: Some(Duration::from_millis(300)), ..Script::default() });
let project = tempfile::tempdir().unwrap();
let (manager, owner) = (h.manager.clone(), h.owner.clone());
let config = remote_attach("java-debug", project.path());
let start = tokio::spawn(async move { manager.start(config, owner).await });
h.wait_event(|e| matches!(e, DebugEvent::Started { .. })).await;
let id = DebugSessionId(*event_sessions(&h.events).first().unwrap());
let session = h.manager.session(id).unwrap();
assert!(session.attach_target.get().is_none(), "asked too late to cover the connecting session");
h.manager.jvm_memory_classes(id).await.unwrap();
assert!(session.attach_target.get().is_some(), "answered before the attach was looked at");
start.await.unwrap().unwrap();
}
/// A JVM clears its JDWP listener address once a debugger connects, so only a scan made before
/// the `attach` request can tell it by its port.
#[cfg(unix)]
#[tokio::test]
async fn local_attach_finds_its_jvm_before_the_debugger_connects() {
let h = harness(Script::default());
let log = h.log.clone();
h.manager.set_jvm_finder(Arc::new(move |port, _| {
std::thread::sleep(Duration::from_millis(50));
let attached = log.lock().iter().any(|m| m["command"] == "attach");
(port == 5005 && !attached).then_some(4242)
}));
let mut cfg = java_config("attach");
cfg.extra_json = Some(json!({"hostName": "127.0.0.1", "port": 5005}).to_string());
let id = h.start(cfg).await.unwrap();
let session = h.manager.session(id).unwrap();
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 4242);
}
#[cfg(unix)]
#[tokio::test]
async fn local_attach_remembers_its_jdwp_port() {
let h = harness(Script::default());
let ports = Arc::new(Mutex::new(Vec::new()));
let asked = ports.clone();
h.manager.set_jvm_finder(Arc::new(move |port, _| {
asked.lock().push(port);
Some(4242)
}));
let mut cfg = java_config("attach");
cfg.extra_json = Some(json!({"hostName": "localhost", "port": "5005"}).to_string());
let id = h.start(cfg).await.unwrap();
let session = h.manager.session(id).unwrap();
assert_eq!(session.jdwp_port.get(), Some(&5005));
assert_eq!(h.request("attach")["arguments"]["port"], "5005");
for _ in 0..200 {
if session.jvm_pid.load(Ordering::Acquire) != 0 {
break;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 4242);
assert_eq!(ports.lock().first(), Some(&5005));
}
/// A fake `find_jvm_by_jdwp_port` for the JVM 4242, recording whether each scan ran while
/// suspended. A JVM not `safe_while_suspended` (JDK below 17) is skipped by a suspended scan.
#[cfg(unix)]
fn counting_find(
safe_while_suspended: bool,
) -> (Arc<Mutex<Vec<bool>>>, impl Fn(bool) -> Option<u32> + Clone + Send + 'static) {
let scans = Arc::new(Mutex::new(Vec::new()));
let counted = scans.clone();
(scans, move |suspended| {
counted.lock().push(suspended);
(!suspended || safe_while_suspended).then_some(4242)
})
}
#[cfg(unix)]
async fn suspended_remote_attach(h: &Harness) -> Arc<Session> {
let project = tempfile::tempdir().unwrap();
let id = h.start(remote_attach("java-debug", project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let session = h.manager.session(id).unwrap();
assert!(session.suspended.get());
session
}
#[cfg(unix)]
#[tokio::test]
async fn background_scan_skips_an_old_jdk_while_suspended() {
let h = harness(Script::default());
let session = suspended_remote_attach(&h).await;
let (scans, find) = counting_find(false);
let task =
jvm_memory::watch_for_attached_jvm(&session, 5005, find, Duration::from_millis(5), Duration::from_secs(5));
tokio::time::sleep(Duration::from_millis(100)).await;
assert!(!scans.lock().is_empty() && scans.lock().iter().all(|&suspended| suspended));
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 0);
session.suspended.continued(None, true);
tokio::time::timeout(Duration::from_secs(2), task).await.unwrap().unwrap();
assert_eq!(scans.lock().last(), Some(&false));
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 4242);
}
#[cfg(unix)]
#[tokio::test]
async fn background_scan_finds_a_jdk_17_or_later_while_suspended() {
let h = harness(Script::default());
let session = suspended_remote_attach(&h).await;
let (scans, find) = counting_find(true);
let task =
jvm_memory::watch_for_attached_jvm(&session, 5005, find, Duration::from_millis(5), Duration::from_secs(5));
tokio::time::timeout(Duration::from_secs(2), task).await.unwrap().unwrap();
assert_eq!(*scans.lock(), [true]);
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 4242);
}
#[cfg(unix)]
#[tokio::test]
async fn background_scan_keeps_a_known_pid_and_stops_with_the_session() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(remote_attach("java-debug", project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let session = h.manager.session(id).unwrap();
session.suspended.continued(None, true);
session.jvm_pid.store(7, Ordering::Release);
let watch = |find| {
jvm_memory::watch_for_attached_jvm(&session, 5005, find, Duration::from_millis(5), Duration::from_secs(5))
};
let (scans, find) = counting_find(true);
tokio::time::timeout(Duration::from_secs(2), watch(find)).await.unwrap().unwrap();
assert!(scans.lock().is_empty(), "a known pid needs no scan");
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 7);
// A refresh stores the pid while this scan is still running.
session.jvm_pid.store(0, Ordering::Release);
let racing = session.clone();
let find = move |_| {
racing.jvm_pid.store(7, Ordering::Release);
Some(4242)
};
let task =
jvm_memory::watch_for_attached_jvm(&session, 5005, find, Duration::from_millis(5), Duration::from_secs(5));
tokio::time::timeout(Duration::from_secs(2), task).await.unwrap().unwrap();
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 7, "a pid stored first is kept");
h.manager.stop(id).await.unwrap();
let (scans, find) = counting_find(true);
tokio::time::timeout(Duration::from_secs(2), watch(find)).await.unwrap().unwrap();
assert!(scans.lock().is_empty());
}
#[cfg(unix)]
#[tokio::test]
async fn unknown_pid_reports_no_jvm_process() {
let h = harness(Script::default());
let id = h.start(java_config("launch")).await.unwrap();
assert!(h.manager.capabilities(id).unwrap().jvm_memory);
assert_eq!(
h.manager.jvm_memory_classes(id).await.unwrap(),
JvmMemoryResult::Unavailable { reason: JvmMemoryUnavailable::NoJvmProcess, can_count_when_paused: false }
);
}
fn kotlin_attach(project: &std::path::Path, host: &str, port: u16) -> LaunchConfig {
LaunchConfig {
adapter: kotlin::ADAPTER.into(),
cwd: Some(project.to_owned().into()),
extra_json: Some(json!({"projectRoot": project, "hostName": host, "port": port}).to_string()),
..java_config("attach")
}
}
#[tokio::test]
async fn kotlin_session_installs_link_inspector() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_attach(project.path(), "127.0.0.1", 5005)).await.unwrap();
let session = h.manager.session(id).unwrap();
assert!(session.jdwp_link.lock().is_some());
assert!(session.jvm_inspector.lock().is_some());
}
#[tokio::test]
async fn attach_target_recorded_before_kotlin_rewrite() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_attach(project.path(), "10.0.0.5", 5005)).await.unwrap();
let session = h.manager.session(id).unwrap();
assert_eq!(session.attach_target.get(), Some(&("10.0.0.5".to_owned(), 5005)));
let sent = &h.request("attach")["arguments"];
assert_eq!(sent["hostName"], "127.0.0.1");
let link_port = sent["port"].as_u64().unwrap();
assert!(link_port != 0 && link_port != 5005);
}
#[tokio::test]
async fn java_attach_records_target() {
let h = harness(Script::default());
let mut cfg = java_config("attach");
cfg.extra_json = Some(json!({"hostName": "localhost", "port": 5005}).to_string());
let id = h.start(cfg).await.unwrap();
let session = h.manager.session(id).unwrap();
assert_eq!(session.attach_target.get(), Some(&("localhost".to_owned(), 5005)));
assert!(session.jdwp_link.lock().is_none());
assert!(session.jvm_inspector.lock().is_none());
}
#[tokio::test]
async fn attach_target_recorded_for_any_adapter() {
let h = harness(Script::default());
let mut cfg = java_config("attach");
// "x" is not a registered adapter; debugpy has no JVM link or memory support.
cfg.adapter = "debugpy".into();
cfg.extra_json = Some(json!({"hostName": "h", "port": 7}).to_string());
let id = h.start(cfg).await.unwrap();
assert_eq!(h.manager.session(id).unwrap().attach_target.get(), Some(&("h".to_owned(), 7)));
}
#[tokio::test]
async fn launch_sessions_have_no_attach_target() {
let h = harness(Script::default());
let id = h.start(java_config("launch")).await.unwrap();
assert_eq!(h.manager.session(id).unwrap().attach_target.get(), None);
}
#[tokio::test]
async fn closing_the_session_drops_the_link() {
let h = harness(Script { hang_up_on_pause: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_attach(project.path(), "127.0.0.1", 5005)).await.unwrap();
let session = h.manager.session(id).unwrap();
assert!(session.jdwp_link.lock().is_some() && session.jvm_inspector.lock().is_some());
let _ = session.conn.request_timeout("pause", json!({"threadId": 1}), Duration::from_secs(2)).await;
for _ in 0..200 {
if h.manager.session(id).is_err() {
break;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
assert!(session.jdwp_link.lock().is_none());
assert!(session.jvm_inspector.lock().is_none());
}
#[cfg(unix)]
#[tokio::test]
async fn rdbg_drains_test_output_after_dap_closes_before_killing_the_process() {
+401
View File
@@ -0,0 +1,401 @@
//! Instances, fields and referrers through a session's JVM inspector.
use super::*;
use crate::jvm_memory::test_support::FakeInspector;
use ide_core::event::JvmBackendState;
use ide_jvm_inspect::{InspectError, JvmValue, VmTarget};
use tokio::io::AsyncReadExt;
fn debug_events(h: &Harness) -> Vec<DebugEvent> {
h.events
.lock()
.iter()
.filter_map(|e| match e {
IdeEvent::Debug { event, .. } => Some(event.clone()),
_ => None,
})
.collect()
}
fn backend_events(h: &Harness) -> Vec<JvmBackendState> {
debug_events(h)
.into_iter()
.filter_map(|e| match e {
DebugEvent::JvmBackend { state } => Some(state),
_ => None,
})
.collect()
}
/// A java-debug session stopped at its breakpoint, inspected by `fake`.
async fn stopped_with(h: &Harness, config: LaunchConfig, fake: &Arc<FakeInspector>) -> DebugSessionId {
let id = h.start(config).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
h.manager.set_jvm_inspector(id, fake.clone()).unwrap();
id
}
fn object(id: u64) -> JvmValue {
JvmValue::Object { id, type_name: "A".into() }
}
const NOT_SUSPENDED: JvmInstancesResult =
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::NotSuspended };
#[tokio::test]
async fn instances_need_a_suspended_session() {
let h = harness(Script { continued_after_stop: true, ..Script::default() });
let id = h.start(java_config("launch")).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Continued { .. })).await;
let fake = Arc::new(FakeInspector::default());
h.manager.set_jvm_inspector(id, fake.clone()).unwrap();
assert_eq!(h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap(), NOT_SUSPENDED);
assert_eq!(fake.calls(), 0);
}
#[tokio::test]
async fn instances_without_an_inspector_have_no_backend() {
let h = harness(Script::default());
let id = h.start(java_config("launch")).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
assert_eq!(
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::NoBackend }
);
}
#[tokio::test]
async fn object_from_previous_stop_is_stale() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::default());
fake.instances.lock().push(object(7));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let JvmInstancesResult::Ready { generation, instances, truncated } =
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap()
else {
panic!("instances unavailable");
};
assert_eq!((generation, instances, truncated), (1, vec![object(7)], false));
h.manager.session(id).unwrap().suspended.stopped(None, true);
let stale = JvmObjectRef { generation, id: 7 };
assert_eq!(
h.manager.jvm_memory_fields(id, stale).await.unwrap(),
JvmFieldsResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
assert_eq!(
h.manager.jvm_memory_referrers(id, stale, 10).await.unwrap(),
JvmReferrersResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
assert_eq!(fake.calls(), 1, "a stale object never reaches the inspector");
let current = JvmObjectRef { generation: 2, id: 7 };
assert_eq!(h.manager.jvm_memory_fields(id, current).await.unwrap(), JvmFieldsResult::Ready { fields: Vec::new() });
}
#[tokio::test]
async fn instances_stay_readable_while_another_thread_is_stopped() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::default());
fake.instances.lock().push(object(7));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let session = h.manager.session(id).unwrap();
// Threads 20, 21, then 22 stop alone (SUSPEND_EVENT_THREAD).
session.suspended.continued(Some(1), true);
for thread in [20, 21, 22] {
session.suspended.stopped(Some(thread), false);
}
let instances = || h.manager.jvm_memory_instances(id, "A".into(), 10);
let result = instances().await.unwrap();
let JvmInstancesResult::Ready { generation, .. } = result else { panic!("{result:?}") };
let read_on_22 = JvmObjectRef { generation, id: 7 };
// An earlier thread resumes: the stop on 22 goes on, and so does what was read on it.
session.suspended.continued(Some(20), false);
assert_eq!(
h.manager.jvm_memory_fields(id, read_on_22).await.unwrap(),
JvmFieldsResult::Ready { fields: Vec::new() }
);
// The latest one resumes: 21 is still stopped, but what was read on 22 is stale.
session.suspended.continued(Some(22), false);
assert_eq!(
h.manager.jvm_memory_fields(id, read_on_22).await.unwrap(),
JvmFieldsResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
let result = instances().await.unwrap();
assert!(matches!(result, JvmInstancesResult::Ready { .. }), "{result:?}");
}
/// Starts `instances` on a held fake inspector; returns once the inspector was entered.
async fn held_request(
h: &Harness,
id: DebugSessionId,
fake: &Arc<FakeInspector>,
) -> tokio::task::JoinHandle<IdeResult<JvmInstancesResult>> {
let manager = h.manager.clone();
let request = tokio::spawn(async move { manager.jvm_memory_instances(id, "A".into(), 10).await });
tokio::time::timeout(Duration::from_secs(2), fake.entered.notified()).await.unwrap();
request
}
#[tokio::test]
async fn stop_during_request_is_stale() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector { hold: true, ..FakeInspector::default() });
fake.instances.lock().push(object(7));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let request = held_request(&h, id, &fake).await;
h.manager.session(id).unwrap().suspended.stopped(None, true);
fake.release.notify_one();
assert_eq!(
request.await.unwrap().unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
}
#[tokio::test]
async fn resume_during_request_is_stale() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector { hold: true, ..FakeInspector::default() });
let id = stopped_with(&h, java_config("launch"), &fake).await;
let request = held_request(&h, id, &fake).await;
h.manager.session(id).unwrap().suspended.continued(None, true);
fake.release.notify_one();
assert_eq!(
request.await.unwrap().unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
}
#[tokio::test]
async fn one_request_at_a_time_per_session() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector { hold: true, ..FakeInspector::default() });
let id = stopped_with(&h, java_config("launch"), &fake).await;
let first = held_request(&h, id, &fake).await;
let manager = h.manager.clone();
let second = tokio::spawn(async move { manager.jvm_memory_instances(id, "B".into(), 10).await });
tokio::time::sleep(Duration::from_millis(50)).await;
assert_eq!(fake.calls(), 1, "the second request waits for the first");
fake.release.notify_one();
first.await.unwrap().unwrap();
tokio::time::timeout(Duration::from_secs(2), fake.entered.notified()).await.unwrap();
fake.release.notify_one();
second.await.unwrap().unwrap();
assert_eq!(fake.calls(), 2);
}
#[tokio::test]
async fn session_end_lets_go_of_a_waiting_inspector() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector { hold: true, ..FakeInspector::default() });
let id = stopped_with(&h, java_config("launch"), &fake).await;
let request = held_request(&h, id, &fake).await;
h.manager.session(id).unwrap().kill_processes();
assert_eq!(
tokio::time::timeout(Duration::from_secs(2), request).await.unwrap().unwrap().unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::BackendClosed }
);
assert_eq!(Arc::strong_count(&fake), 1, "no clone of the inspector outlives the session");
}
#[tokio::test]
async fn target_carries_pid_for_launch() {
let h = harness(Script { debuggee_pid: Some(4242), java_process_id: true, ..Script::default() });
let fake = Arc::new(FakeInspector::default());
let id = stopped_with(&h, java_config("launch"), &fake).await;
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap();
assert_eq!(
*fake.targets.lock(),
[VmTarget { pid: Some(4242), attach_host: None, attach_port: None, generation: 1 }]
);
}
#[tokio::test]
async fn target_carries_attach_port_and_host() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let fake = Arc::new(FakeInspector::default());
let id = stopped_with(&h, remote_attach("java-debug", project.path()), &fake).await;
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap();
assert_eq!(
*fake.targets.lock(),
[VmTarget { pid: None, attach_host: Some("10.0.0.5".into()), attach_port: Some(5005), generation: 1 }]
);
}
#[tokio::test]
async fn unsupported_backend_turns_instances_off() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::failing(InspectError::Unsupported("x".into())));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let before = debug_events(&h).len();
assert_eq!(
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::Unsupported { detail: "x".into() } }
);
let after: Vec<_> = debug_events(&h).split_off(before);
assert!(
matches!(
after.as_slice(),
[DebugEvent::JvmBackend { state: JvmBackendState::Unsupported { detail } }, DebugEvent::Capabilities]
if detail == "x"
),
"{after:?}"
);
assert!(!h.manager.capabilities(id).unwrap().jvm_instances);
assert_eq!(
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::NoBackend }
);
assert_eq!(fake.calls(), 1);
}
/// `error` is reported but leaves the backend on.
async fn keeps_the_backend(error: InspectError, reason: JvmInspectUnavailable) {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::failing(error));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let object = JvmObjectRef { generation: 1, id: 7 };
assert_eq!(h.manager.jvm_memory_fields(id, object).await.unwrap(), JvmFieldsResult::Unavailable { reason });
assert!(h.manager.capabilities(id).unwrap().jvm_instances);
assert!(backend_events(&h).is_empty());
assert!(h.manager.session(id).unwrap().jvm_inspector.lock().is_some());
}
#[tokio::test]
async fn collected_keeps_the_backend() {
keeps_the_backend(InspectError::Collected, JvmInspectUnavailable::Collected).await;
}
#[tokio::test]
async fn timed_out_keeps_the_backend() {
keeps_the_backend(InspectError::TimedOut, JvmInspectUnavailable::TimedOut).await;
}
#[tokio::test]
async fn failed_inspection_is_an_error() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::failing(InspectError::Failed("boom".into())));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let err = h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap_err();
assert!(err.to_string().contains("boom"), "{err}");
assert!(h.manager.capabilities(id).unwrap().jvm_instances);
}
#[tokio::test]
async fn limits_are_clamped() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::default());
let id = stopped_with(&h, java_config("launch"), &fake).await;
for limit in [5000, 0] {
h.manager.jvm_memory_instances(id, "A".into(), limit).await.unwrap();
}
let object = JvmObjectRef { generation: 1, id: 7 };
for limit in [0, 900, 20] {
h.manager.jvm_memory_referrers(id, object, limit).await.unwrap();
}
assert_eq!(*fake.limits.lock(), [1000, 1, 50, 200, 20]);
}
#[tokio::test]
async fn a_full_page_is_truncated() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::default());
fake.instances.lock().extend([object(1), object(2)]);
let id = stopped_with(&h, java_config("launch"), &fake).await;
let truncated = |r: JvmInstancesResult| match r {
JvmInstancesResult::Ready { truncated, .. } => truncated,
other => panic!("{other:?}"),
};
assert!(truncated(h.manager.jvm_memory_instances(id, "A".into(), 2).await.unwrap()));
assert!(!truncated(h.manager.jvm_memory_instances(id, "A".into(), 3).await.unwrap()));
}
#[tokio::test]
async fn capability_follows_the_inspector() {
let h = harness(Script::default());
let id = h.start(java_config("launch")).await.unwrap();
assert!(!h.manager.capabilities(id).unwrap().jvm_instances);
let before = debug_events(&h).len();
h.manager.set_jvm_inspector(id, Arc::new(FakeInspector::default())).unwrap();
assert!(h.manager.capabilities(id).unwrap().jvm_instances);
assert!(matches!(debug_events(&h).split_off(before).as_slice(), [DebugEvent::Capabilities]));
}
#[tokio::test]
async fn remote_attach_counts_classes_through_the_debugger() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let fake = Arc::new(FakeInspector::default());
fake.counts.lock().push(ide_jvm_inspect::ClassCount { name: "A".into(), instances: 3 });
let id = stopped_with(&h, remote_attach("java-debug", project.path()), &fake).await;
let result = h.manager.jvm_memory_classes(id).await.unwrap();
assert!(
matches!(&result, JvmMemoryResult::Ready { snapshot }
if snapshot.source == JvmMemorySource::Debugger && snapshot.rows.len() == 1),
"{result:?}"
);
assert_eq!(fake.targets.lock()[0].attach_port, Some(5005));
}
const HANDSHAKE: &[u8; 14] = b"JDWP-Handshake";
/// A Kotlin attach whose link reaches a fake JVM: (adapter end, JVM end) after both handshakes.
async fn linked_kotlin(h: &Harness) -> (DebugSessionId, tokio::net::TcpStream, tokio::net::TcpStream) {
let jvm = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_attach(project.path(), "127.0.0.1", jvm.local_addr().unwrap().port())).await.unwrap();
let link_port = h.request("attach")["arguments"]["port"].as_u64().unwrap() as u16;
let mut adapter = tokio::net::TcpStream::connect(("127.0.0.1", link_port)).await.unwrap();
adapter.write_all(HANDSHAKE).await.unwrap();
let (mut jvm, _) = tokio::time::timeout(Duration::from_secs(2), jvm.accept()).await.unwrap().unwrap();
let mut buf = [0; 14];
jvm.read_exact(&mut buf).await.unwrap();
jvm.write_all(HANDSHAKE).await.unwrap();
adapter.read_exact(&mut buf).await.unwrap();
(id, adapter, jvm)
}
/// A command the adapter sends with an id from the link's own range: the link then only forwards.
async fn degrade(adapter: &mut tokio::net::TcpStream) {
let mut packet = 11u32.to_be_bytes().to_vec();
packet.extend(ide_jdwp::OWN_IDS.start.to_be_bytes());
packet.extend([0, 1, 1]);
adapter.write_all(&packet).await.unwrap();
}
async fn wait_backend(h: &Harness, want: impl Fn(&JvmBackendState) -> bool) {
h.wait_event(|e| matches!(e, DebugEvent::JvmBackend { state } if want(state))).await;
}
#[tokio::test]
async fn kotlin_link_reports_degraded_then_closed() {
let h = harness(Script::default());
let (id, mut adapter, jvm) = linked_kotlin(&h).await;
degrade(&mut adapter).await;
wait_backend(&h, |s| matches!(s, JvmBackendState::Degraded { .. })).await;
drop(jvm);
wait_backend(&h, |s| *s == JvmBackendState::Closed).await;
assert_eq!(backend_events(&h).len(), 2, "{:?}", backend_events(&h));
let session = h.manager.session(id).unwrap();
assert!(session.jdwp_link.lock().is_some(), "a closed link is kept until the session ends");
}
#[tokio::test]
async fn kotlin_link_is_quiet_once_the_program_ended() {
let h = harness(Script::default());
let (id, _adapter, jvm) = linked_kotlin(&h).await;
let session = h.manager.session(id).unwrap();
// The adapter reported the end; the session keeps its link until it shuts down.
session.terminated.store(true, Ordering::Release);
drop(jvm);
for _ in 0..100 {
if session.jdwp_link.lock().as_ref().is_some_and(|l| l.state() == ide_jdwp::LinkState::Closed) {
break;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
tokio::time::sleep(Duration::from_millis(50)).await;
assert!(backend_events(&h).is_empty(), "{:?}", backend_events(&h));
}
+1 -1
View File
@@ -20,7 +20,7 @@ parking_lot.workspace = true
futures.workspace = true
sqlx.workspace = true
tiberius.workspace = true
oracle.workspace = true
oracledb.workspace = true
bb8.workspace = true
tokio-util = { workspace = true, features = ["compat"] }
dirs.workspace = true
+3 -3
View File
@@ -417,7 +417,7 @@ fn binary_type(name: &str, text: bool) -> bool {
}
/// Bytes as text: UTF-8, else hex (grids) or with invalid sequences replaced (catalog names).
fn bytes_text(bytes: Vec<u8>, mode: Cells) -> String {
pub(crate) fn bytes_text(bytes: Vec<u8>, mode: Cells) -> String {
match (String::from_utf8(bytes), mode) {
(Ok(s), _) => s,
(Err(e), Cells::Display) => hex(e.as_bytes()),
@@ -1042,7 +1042,7 @@ pub enum Held {
Mysql(Box<bb8::PooledConnection<'static, MysqlManager>>),
SqlServer(Box<bb8::PooledConnection<'static, MssqlConnectionManager>>),
Clickhouse(crate::clickhouse::ClickhousePool),
Oracle(crate::oracle::OracleConnection),
Oracle(Box<crate::oracle::OracleConnection>),
}
impl Held {
@@ -1058,7 +1058,7 @@ impl Held {
}
Held::Clickhouse(p.clone())
}
Pool::Oracle(p) => Held::Oracle(p.get().await?),
Pool::Oracle(p) => Held::Oracle(Box::new(p.get().await?)),
})
}
+41 -3
View File
@@ -57,6 +57,7 @@ impl DbTableKind {
pub enum DbObjectKind {
Function,
Procedure,
Package,
Trigger,
Sequence,
}
@@ -92,12 +93,21 @@ pub struct SequenceInfo {
pub data_type: String,
}
/// A schema's routines (procedures and functions) and sequences.
/// An Oracle package (its procedures and functions are not listed separately).
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbPackageInfo")]
pub struct PackageInfo {
pub name: String,
}
/// A schema's routines (procedures and functions), packages and sequences.
#[derive(Debug, Clone, PartialEq, Eq, Default, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbSchemaObjects")]
pub struct SchemaObjects {
pub routines: Vec<RoutineInfo>,
pub packages: Vec<PackageInfo>,
pub sequences: Vec<SequenceInfo>,
}
@@ -296,7 +306,7 @@ fn through_tunnel(e: IdeError, source: &DataSource, via: Option<&str>) -> IdeErr
/// Options › Run keep-alive query: pings the pool until it is closed or dropped
/// (a dropped SSH tunnel or an idle-killing proxy stays alive meanwhile).
fn keep_alive(pool: &Arc<Pool>, every: Duration) {
pub(crate) fn keep_alive(pool: &Arc<Pool>, every: Duration) {
let pool = Arc::downgrade(pool);
tokio::spawn(async move {
let mut tick = tokio::time::interval(every);
@@ -853,6 +863,14 @@ impl DbManager {
})
.collect();
}
if let Some(q) = sql::packages(driver, schema) {
out.packages = pool
.rows(&q, MAX_META_ROWS)
.await?
.into_iter()
.filter_map(|r| Some(PackageInfo { name: r.into_iter().next().flatten()? }))
.collect();
}
if let Some(q) = sql::sequences(driver, schema) {
out.sequences = pool
.rows(&q, MAX_META_ROWS)
@@ -870,9 +888,29 @@ impl DbManager {
Ok(out)
}
/// The definition of a routine (every overload), trigger or sequence.
/// The definition of a routine (every overload), package (spec and body), trigger or sequence.
pub async fn object_ddl(&self, id: &str, schema: &str, kind: DbObjectKind, name: &str) -> IdeResult<String> {
let pool = self.pool(id).await?;
if kind == DbObjectKind::Package {
let mut defs: Vec<String> = Vec::new();
for body in [false, true] {
let Some(q) = sql::package_ddl(pool.driver(), schema, name, body) else { continue };
match pool.rows(&q, MAX_META_ROWS).await {
Ok(rows) => defs.extend(rows.into_iter().filter_map(|r| r.into_iter().next().flatten())),
// A spec-only package has no body: the spec alone still opens.
Err(e) if body => tracing::warn!("package body {schema}.{name}: {}", e.message),
Err(e) => return Err(e),
}
}
if defs.is_empty() {
return Err(IdeError::not_found(format!("{schema}.{name}")));
}
return Ok(defs
.iter()
.map(|d| format!("{};\n", d.trim_end().trim_end_matches(';')))
.collect::<Vec<_>>()
.join("\n"));
}
let (q, column) = sql::object_ddl(pool.driver(), schema, kind, name)
.ok_or_else(|| IdeError::invalid(format!("no {kind:?} objects in this database")))?;
let defs: Vec<String> = pool
File diff suppressed because it is too large. Load diff
+1 -3
View File
@@ -20,7 +20,7 @@ pub enum DbDriver {
/// Microsoft SQL Server.
SqlServer,
Clickhouse,
/// Oracle Database (via Oracle Instant Client).
/// Oracle Database (pure-Rust thin driver, no client library).
Oracle,
}
@@ -88,8 +88,6 @@ pub struct DbOptions {
pub auto_disconnect_secs: Option<u32>,
/// PostgreSQL: show only the configured database, its schemas right under the data source.
pub single_database: bool,
/// Oracle: directory of the Oracle Instant Client (empty: `ORACLE_HOME` / system search path).
pub instant_client: String,
}
/// Schemas tab: the databases / schemas the tree shows. Names may use `*` and `?`.
+39
View File
@@ -664,6 +664,32 @@ pub fn routines(driver: DbDriver, schema: &str) -> Option<String> {
}
}
/// Rows: `name` of a schema's packages (Oracle only; their procedures and functions are not
/// listed separately, as in IDEA).
pub fn packages(driver: DbDriver, schema: &str) -> Option<String> {
match driver {
DbDriver::Oracle => Some(format!(
"SELECT object_name FROM all_objects WHERE owner = {} AND object_type = 'PACKAGE' \
AND object_name NOT LIKE 'BIN$%' ORDER BY 1",
quote_literal(driver, schema)
)),
_ => None,
}
}
/// The statement reading a package's specification (`body` false) or body as DDL.
pub fn package_ddl(driver: DbDriver, schema: &str, name: &str, body: bool) -> Option<String> {
match driver {
DbDriver::Oracle => Some(format!(
"SELECT DBMS_METADATA.GET_DDL('{}', {}, {}) FROM DUAL",
if body { "PACKAGE_BODY" } else { "PACKAGE" },
quote_literal(driver, name),
quote_literal(driver, schema)
)),
_ => None,
}
}
/// Rows: `name, data type` of a schema's sequences (PostgreSQL, Oracle).
pub fn sequences(driver: DbDriver, schema: &str) -> Option<String> {
match driver {
@@ -963,6 +989,7 @@ mod tests {
assert!(completion_columns(DbDriver::Oracle).contains("all_tab_columns"));
assert!(triggers(DbDriver::Oracle, "hr", "t").contains("all_triggers"));
assert!(routines(DbDriver::Oracle, "hr").unwrap().contains("all_objects"));
assert!(packages(DbDriver::Oracle, "hr").unwrap().contains("object_type = 'PACKAGE'"));
assert!(sequences(DbDriver::Oracle, "hr").unwrap().contains("all_sequences"));
}
@@ -984,6 +1011,17 @@ mod tests {
let (q, col) = object_ddl(DbDriver::Oracle, "hr", K::Procedure, "p").unwrap();
assert!(q.contains("DBMS_METADATA.GET_DDL('PROCEDURE'") && col == 0);
assert!(object_ddl(DbDriver::Oracle, "hr", K::Sequence, "q").unwrap().0.contains("all_sequences"));
assert!(
package_ddl(DbDriver::Oracle, "hr", "pkg", false)
.unwrap()
.contains("DBMS_METADATA.GET_DDL('PACKAGE', 'pkg', 'hr')")
);
assert!(
package_ddl(DbDriver::Oracle, "hr", "pkg", true)
.unwrap()
.contains("DBMS_METADATA.GET_DDL('PACKAGE_BODY', 'pkg', 'hr')")
);
assert!(package_ddl(DbDriver::Postgres, "public", "pkg", false).is_none());
}
#[test]
@@ -1001,6 +1039,7 @@ mod tests {
fn schema_objects_per_dialect() {
use crate::DbObjectKind as K;
assert!(routines(DbDriver::Sqlite, "main").is_none());
assert!(packages(DbDriver::Sqlite, "main").is_none());
assert!(sequences(DbDriver::Mysql, "db").is_none());
assert!(routines(DbDriver::Postgres, "public").unwrap().contains("n.nspname = 'public'"));
assert_eq!(
+44 -17
View File
@@ -136,24 +136,19 @@ pub struct DockerRegistry {
pub username: String,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct DockerSettings {
#[serde(default = "default_connections")]
#[serde(default)]
pub connections: Vec<DockerConnection>,
#[serde(default)]
pub registries: Vec<DockerRegistry>,
}
impl Default for DockerSettings {
fn default() -> Self {
Self { connections: default_connections(), registries: Vec::new() }
}
}
/// Before anything is configured the CLI's own daemon is shown, as before connections existed.
fn default_connections() -> Vec<DockerConnection> {
vec![DockerConnection { id: "default".into(), name: "Docker".into(), endpoint: DockerEndpoint::Default }]
/// Whether `c` is the connection older versions injected when nothing was configured:
/// Services lists only connections the user added, so this one is dropped on load.
fn is_injected_default(c: &DockerConnection) -> bool {
c.id == "default" && c.name == "Docker" && c.endpoint == DockerEndpoint::Default
}
const SETTINGS_FILE: &str = "docker.json";
@@ -161,10 +156,17 @@ const PASSWORDS_FILE: &str = "docker-passwords.json";
pub fn load_settings(data_dir: &Path) -> DockerSettings {
match std::fs::read(data_dir.join(SETTINGS_FILE)) {
Ok(bytes) => serde_json::from_slice(&bytes).unwrap_or_else(|e| {
tracing::warn!("{SETTINGS_FILE}: {e}");
DockerSettings::default()
}),
// A file written before the opt-in Services rework carries the injected default
// (saved along with a registry or another connection); drop it unless edited.
Ok(bytes) => serde_json::from_slice(&bytes)
.map(|mut s: DockerSettings| {
s.connections.retain(|c| !is_injected_default(c));
s
})
.unwrap_or_else(|e| {
tracing::warn!("{SETTINGS_FILE}: {e}");
DockerSettings::default()
}),
Err(_) => DockerSettings::default(),
}
}
@@ -373,7 +375,8 @@ mod tests {
fn settings_round_trip_and_passwords() {
let tmp = tempfile::tempdir().unwrap();
let dir = tmp.path();
assert_eq!(load_settings(dir).connections[0].endpoint, DockerEndpoint::Default);
// Nothing configured: no connection is invented.
assert!(load_settings(dir).connections.is_empty());
let settings = DockerSettings {
connections: vec![DockerConnection {
id: "c1".into(),
@@ -395,8 +398,32 @@ mod tests {
assert!(auth_for(dir, "postgres").is_none());
save_settings(dir, &DockerSettings { registries: Vec::new(), ..settings }).unwrap();
assert_eq!(registry_password(dir, "r1"), None);
// An empty list stays empty (the default connection is only for a missing file).
// An empty list stays empty.
save_settings(dir, &DockerSettings { connections: Vec::new(), registries: Vec::new() }).unwrap();
assert!(load_settings(dir).connections.is_empty());
}
#[test]
fn load_drops_the_injected_default_connection() {
let tmp = tempfile::tempdir().unwrap();
let dir = tmp.path();
let injected =
|| DockerConnection { id: "default".into(), name: "Docker".into(), endpoint: DockerEndpoint::Default };
let registry =
DockerRegistry { id: "r1".into(), name: "GHCR".into(), address: "ghcr.io".into(), username: "me".into() };
// Saved untouched by an older version (e.g. along with a registry): dropped on load.
save_settings(dir, &DockerSettings { connections: vec![injected()], registries: vec![registry.clone()] })
.unwrap();
let loaded = load_settings(dir);
assert!(loaded.connections.is_empty());
assert_eq!(loaded.registries, vec![registry]);
// A connection the user renamed or re-pointed stays theirs.
let renamed = DockerConnection { name: "Local engine".into(), ..injected() };
save_settings(dir, &DockerSettings { connections: vec![renamed.clone()], registries: Vec::new() }).unwrap();
assert_eq!(load_settings(dir).connections, vec![renamed]);
let repointed =
DockerConnection { endpoint: DockerEndpoint::Socket { path: "/var/run/docker.sock".into() }, ..injected() };
save_settings(dir, &DockerSettings { connections: vec![repointed.clone()], registries: Vec::new() }).unwrap();
assert_eq!(load_settings(dir).connections, vec![repointed]);
}
}
+17 -2
View File
@@ -71,7 +71,6 @@ impl Capability {
match self {
Self::ProjectRead | Self::EditorRead | Self::Storage | Self::Events | Self::VcsRead => Risk::Low,
Self::EditorWrite
| Self::ProjectWrite
| Self::DatabaseSources
| Self::VcsWrite
| Self::Net(_)
@@ -80,7 +79,14 @@ impl Capability {
| Self::Secrets
| Self::InputEvents
| Self::Clipboard => Risk::Medium,
Self::IdeCommands | Self::Ipc(_) | Self::Process(_) | Self::FsRead(_) | Self::FsWrite(_) => Risk::High,
// A trusted project's own files name the programs the IDE starts (its settings, its
// build scripts), so writing them is running code.
Self::ProjectWrite
| Self::IdeCommands
| Self::Ipc(_)
| Self::Process(_)
| Self::FsRead(_)
| Self::FsWrite(_) => Risk::High,
}
}
@@ -402,6 +408,15 @@ mod tests {
assert!(Capability::Storage.auto_migrates() && !Capability::EditorWrite.auto_migrates());
}
/// A trusted project's files name programs the IDE starts by itself: the git and language
/// server executables in `.openidea/settings.toml`, its build scripts. Writing them is
/// running code, and the install dialog says so.
#[test]
fn writing_project_files_is_a_high_risk_grant() {
assert_eq!(Capability::ProjectWrite.risk(), Risk::High);
assert!(!Capability::ProjectWrite.auto_migrates());
}
#[test]
fn host_patterns() {
assert!(host_matches("crates.io", "crates.io"));
+7 -1
View File
@@ -420,7 +420,13 @@ pub fn resolve_program(name: &str, path_var: Option<&OsStr>) -> Result<PathBuf,
return if candidate.is_file() { Ok(candidate.to_path_buf()) } else { Err(format!("{name}: no such program")) };
}
let exts: Vec<String> = if cfg!(windows) {
std::env::var("PATHEXT").unwrap_or_else(|_| ".EXE;.CMD;.BAT".into()).split(';').map(str::to_owned).collect()
let pathext = std::env::var("PATHEXT").unwrap_or_else(|_| ".EXE;.CMD;.BAT".into());
let pathext: Vec<String> = pathext.split(';').filter(|e| !e.is_empty()).map(str::to_owned).collect();
// A name given with its extension (`cargo.exe`) is looked up as written.
let has_ext = candidate.extension().is_some_and(|e| {
pathext.iter().any(|x| x.trim_start_matches('.').eq_ignore_ascii_case(&e.to_string_lossy()))
});
if has_ext { vec![String::new()] } else { pathext }
} else {
vec![String::new()]
};
+8 -1
View File
@@ -19,8 +19,10 @@ mod index;
pub mod ledger;
mod manifest;
pub mod package;
mod plugin_point;
mod point;
mod registry;
mod resolve;
mod ui;
mod when;
@@ -30,7 +32,12 @@ pub use manifest::{
ActivationEvent, CATEGORIES, Capabilities, ENGINE_VERSION, EVENT_KINDS, LEGACY_API, MANIFEST_FILE, MAX_CATEGORIES,
Manifest, check_relative, engine_compatible, validate_item_id, validate_plugin_id, validate_point_id,
};
pub use plugin_point::{FieldType, PointDecl};
pub use point::{Commit, Disposable, ExtensionPoint, MAX_PACKAGE_FILE, Package, Placement, Prepared, decode};
pub use registry::{ContributionInfo, ExtensionRegistry, LOCALES_DIR, LOCALES_POINT, PreparedSet};
pub use registry::{
ChangeListener, Contribution, ContributionInfo, DeclaredPoint, ExtensionRegistry, LOCALES_DIR, LOCALES_POINT,
PreparedSet,
};
pub use resolve::{Blocked, Dependency, DependencyState, Node, Plan, dependency_state, dependents, plan};
pub use ui::{UiContribution, UiDecl, UiPoint, UiStore, to_json};
pub use when::validate_when;
+136 -2
View File
@@ -26,6 +26,8 @@
//! activated at start-up, as before.
use crate::capability::Capability;
use crate::plugin_point::PointDecl;
use crate::resolve::Dependency;
use serde::Deserialize;
use std::collections::BTreeSet;
use std::path::{Component, Path, PathBuf};
@@ -66,6 +68,10 @@ pub struct Manifest {
pub wasm: Option<PathBuf>,
pub activation: Vec<ActivationEvent>,
pub capabilities: Capabilities,
/// `[[depends]]`: plugins this one needs (API 1.5).
pub depends: Vec<Dependency>,
/// `[[extension-points]]`: what other plugins may contribute to this one (API 1.5).
pub extension_points: Vec<PointDecl>,
/// `contributes.<point>` → its declarations, validated by the extension points.
pub contributes: toml::Table,
}
@@ -151,6 +157,7 @@ pub const EVENT_KINDS: &[&str] = &[
"file-closed",
"settings-changed",
"input-mode-changed",
"extensions-changed",
"file-opened",
"file-saved",
"files-changed",
@@ -210,6 +217,10 @@ struct Raw {
#[serde(default)]
permissions: Option<Vec<String>>,
#[serde(default)]
depends: Vec<Dependency>,
#[serde(default)]
extension_points: Vec<PointDecl>,
#[serde(default)]
contributes: toml::Table,
}
@@ -281,6 +292,8 @@ impl Manifest {
wasm: raw.wasm,
activation,
capabilities,
depends: raw.depends,
extension_points: raw.extension_points,
contributes: raw.contributes,
};
m.validate()?;
@@ -357,6 +370,9 @@ impl Manifest {
{
self.needs_api(&format!("`{event}`"), 4)?;
}
if matches!(event, ActivationEvent::Event(kind) if kind == "extensions-changed") {
self.needs_api("`on-event:extensions-changed`", 5)?;
}
if matches!(event, ActivationEvent::Event(kind) if kind == "files-changed") {
self.needs_api("`on-event:files-changed`", 3)?;
}
@@ -367,6 +383,7 @@ impl Manifest {
for c in &self.declared_categories {
validate_item_id(c, "category")?;
}
self.validate_relations()?;
for (point, decls) in &self.contributes {
validate_point_id(point)?;
if !matches!(decls, toml::Value::Array(_) | toml::Value::Table(_)) {
@@ -376,6 +393,42 @@ impl Manifest {
Ok(())
}
/// `[[depends]]` and `[[extension-points]]`.
fn validate_relations(&self) -> Result<(), String> {
if !self.depends.is_empty() {
self.needs_api("`[[depends]]`", 5)?;
}
if !self.extension_points.is_empty() {
self.needs_api("`[[extension-points]]`", 5)?;
}
let mut required = BTreeSet::new();
for dep in &self.depends {
validate_plugin_id(&dep.id)?;
if dep.id == self.id {
return Err(format!("`depends`: {} cannot depend on itself", self.id));
}
if let Some(req) = &dep.version
&& semver::VersionReq::parse(req).is_err()
{
return Err(format!(
"`depends`: {:?} is not a version requirement like \"^1.2\" (plugin {})",
req, dep.id
));
}
if !required.insert(&dep.id) {
return Err(format!("`depends`: {} is listed twice", dep.id));
}
}
let mut declared = BTreeSet::new();
for point in &self.extension_points {
point.validate(&self.id)?;
if !declared.insert(&point.id) {
return Err(format!("extension point {} is declared twice", point.id));
}
}
Ok(())
}
/// The declarations of one extension point: `[[contributes.x]]` gives
/// several, `[contributes.x]` one.
pub fn declarations(&self, point: &str) -> Vec<toml::Value> {
@@ -472,11 +525,12 @@ pub fn validate_item_id(id: &str, what: &str) -> Result<(), String> {
}
/// Extension point ids: `camelCase` or `kebab-case` words, dots allowed (`iconThemes`, `menus`).
/// Points declared by plugins are `<plugin id>.<name>`, which may be longer and contain `_`.
pub fn validate_point_id(id: &str) -> Result<(), String> {
let ok = !id.is_empty()
&& id.len() <= 64
&& id.len() <= 128
&& id.starts_with(|c: char| c.is_ascii_lowercase())
&& id.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '-' | '.'));
&& id.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '-' | '.' | '_'));
if ok { Ok(()) } else { Err(format!("invalid contribution point {id:?}")) }
}
@@ -711,6 +765,86 @@ mod api14_tests {
}
}
#[cfg(test)]
mod relations_tests {
use super::*;
const HEAD: &str = "id = \"acme.app\"\nname = \"App\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.5\"\n";
fn parse(rest: &str) -> Result<Manifest, String> {
Manifest::parse(&format!("{HEAD}{rest}"))
}
#[test]
fn parses_depends_and_extension_points() {
let m = parse(
r#"
[[depends]]
id = "acme.base"
version = "^0.2"
[[depends]]
id = "acme.extra"
optional = true
[[extension-points]]
id = "acme.app.hooks"
required = ["name"]
fields = { name = "string" }
[[contributes."acme.base.things"]]
name = "x"
"#,
)
.unwrap();
assert_eq!(m.depends.len(), 2);
assert_eq!(m.depends[0].version.as_deref(), Some("^0.2"));
assert!(m.depends[1].optional);
assert_eq!(m.extension_points[0].id, "acme.app.hooks");
assert_eq!(m.declarations("acme.base.things").len(), 1);
}
#[test]
fn the_new_keys_need_api_1_5() {
let older =
"id = \"acme.app\"\nname = \"App\"\nversion = \"1.0.0\"\nwasm = \"p.wasm\"\nengines.openidea = \"^1.4\"\n";
let err = Manifest::parse(&format!("{older}[[depends]]\nid = \"acme.base\"\n")).unwrap_err();
assert!(err.contains("new in plugin API 1.5"), "{err}");
let err = Manifest::parse(&format!("{older}[[extension-points]]\nid = \"acme.app.x\"\n")).unwrap_err();
assert!(err.contains("new in plugin API 1.5"), "{err}");
let err = Manifest::parse(&format!("{older}activation = [\"on-event:extensions-changed\"]\n")).unwrap_err();
assert!(err.contains("new in plugin API 1.5"), "{err}");
let current = older.replace("^1.4", "^1.5");
assert!(Manifest::parse(&format!("{current}activation = [\"on-event:extensions-changed\"]\n")).is_ok());
}
#[test]
fn refuses_bad_dependencies() {
assert!(parse("[[depends]]\nid = \"acme.app\"\n").unwrap_err().contains("itself"));
assert!(parse("[[depends]]\nid = \"Bad Id\"\n").unwrap_err().contains("invalid plugin id"));
assert!(
parse("[[depends]]\nid = \"acme.base\"\nversion = \"one\"\n").unwrap_err().contains("version requirement")
);
let twice = "[[depends]]\nid = \"acme.base\"\n[[depends]]\nid = \"acme.base\"\n";
assert!(parse(twice).unwrap_err().contains("listed twice"));
assert!(parse("[[depends]]\nid = \"acme.base\"\nunknown = 1\n").is_err());
}
#[test]
fn refuses_points_outside_the_plugins_namespace() {
let err = parse("[[extension-points]]\nid = \"other.thing\"\n").unwrap_err();
assert!(err.contains("acme.app.<name>"), "{err}");
let twice = "[[extension-points]]\nid = \"acme.app.x\"\n[[extension-points]]\nid = \"acme.app.x\"\n";
assert!(parse(twice).unwrap_err().contains("declared twice"));
}
#[test]
fn plugin_ids_with_underscores_work_as_contribution_keys() {
assert!(validate_point_id("my_tools.base.things").is_ok());
assert!(validate_point_id("Bad").is_err());
}
}
#[cfg(test)]
mod commands_api_tests {
use super::*;
+110 -3
View File
@@ -352,6 +352,22 @@ pub struct IndexEntry {
/// the IDE before showing it.
#[serde(default)]
pub icon: Option<PluginIcon>,
/// Where to read more: the manifest's homepage, else the registry's source repository.
#[serde(default)]
pub homepage: Option<String>,
/// This version's release notes (the manifest's `change-notes`).
#[serde(default)]
pub notes: Option<String>,
/// Downloads of the plugin's packages, as counted by whoever serves them.
#[serde(default)]
pub downloads: Option<u64>,
/// Picked by the marketplace's maintainers.
#[serde(default)]
pub featured: bool,
/// Project-relative paths whose presence makes the plugin worth suggesting
/// (`pom.xml`, `config/routes.rb`).
#[serde(default)]
pub suggest: Vec<String>,
}
impl IndexEntry {
@@ -389,10 +405,31 @@ impl Index {
/// revocations, publisher keys (revoked ones) and `moved` — and lists no
/// plugins ([`Index::is_newer`]): the IDE needs updating to install from it.
pub fn parse(text: &str) -> Result<Self, String> {
let (index, dropped) = Self::parse_dropping(text)?;
if let Some(first) = dropped.first() {
// One line per index, and a bounded one: whoever wrote the index chose the text.
let id: String = first.id.chars().take(80).collect();
tracing::warn!(
"index: {} entries are not offered, their id or version is not one a manifest may carry (first: {id:?})",
dropped.len()
);
}
Ok(index)
}
/// [`Index::parse`], with the entries it does not offer: those whose id or
/// version no manifest may carry, so no IDE could install them.
pub fn parse_dropping(text: &str) -> Result<(Self, Vec<IndexEntry>), String> {
let value: serde_json::Value = serde_json::from_str(text).map_err(|e| format!("index: {e}"))?;
let format = value.get("format").and_then(serde_json::Value::as_u64).ok_or("index: `format` is missing")?;
if format == u64::from(INDEX_FORMAT) {
return serde_json::from_value(value).map_err(|e| format!("index: {e}"));
let mut index: Index = serde_json::from_value(value).map_err(|e| format!("index: {e}"))?;
// What no manifest may carry is not offered: the id and version also name files on disk.
let (offered, dropped) = std::mem::take(&mut index.plugins)
.into_iter()
.partition(|e| crate::validate_plugin_id(&e.id).is_ok() && e.semver().is_some());
index.plugins = offered;
return Ok((index, dropped));
}
if format < u64::from(INDEX_FORMAT) {
return Err(format!("index format {format} is not supported (this IDE reads {INDEX_FORMAT})"));
@@ -401,13 +438,14 @@ impl Index {
let items = value.get(field).and_then(serde_json::Value::as_array);
items.into_iter().flatten().filter_map(|v| serde_json::from_value(v.clone()).ok()).collect()
}
Ok(Index {
let index = Index {
format: u32::try_from(format).unwrap_or(u32::MAX),
publishers: each(&value, "publishers"),
plugins: Vec::new(),
revoked: each(&value, "revoked"),
moved: value.get("moved").and_then(serde_json::Value::as_str).map(str::to_owned),
})
};
Ok((index, Vec::new()))
}
/// Written in a format newer than this IDE reads: it lists no plugins.
@@ -685,9 +723,33 @@ mod tests {
engine: "^1.0".into(),
categories: Vec::new(),
icon: None,
homepage: None,
notes: None,
downloads: None,
featured: false,
suggest: Vec::new(),
}
}
#[test]
fn listing_details_are_optional() {
let mut json = serde_json::to_value(entry("acme.a", "acme")).unwrap();
for key in ["homepage", "notes", "downloads", "featured", "suggest"] {
json.as_object_mut().unwrap().remove(key);
}
assert_eq!(serde_json::from_value::<IndexEntry>(json).unwrap(), entry("acme.a", "acme"));
let full = IndexEntry {
homepage: Some("https://example.com".into()),
notes: Some("Fixes".into()),
downloads: Some(42),
featured: true,
suggest: vec!["pom.xml".into()],
..entry("acme.a", "acme")
};
let back: IndexEntry = serde_json::from_str(&serde_json::to_string(&full).unwrap()).unwrap();
assert_eq!(back, full);
}
fn version(id: &str, version: &str, engine: &str) -> IndexEntry {
IndexEntry { version: version.into(), engine: engine.into(), ..entry(id, "acme") }
}
@@ -868,6 +930,51 @@ mod tests {
}
}
#[test]
fn an_index_offers_only_the_ids_and_versions_a_manifest_may_carry() {
let invalid_ids = [
"../x",
"../../escape",
"acme/../x",
r"acme.\..\x",
"acme./../../../escape",
"/abs",
r"C:\x",
"ACME.x",
"acme.",
"acme..x",
"acme.x y",
"",
];
let invalid_versions =
["../1.0.0", "1.0.0/../../../escape", r"1.0.0\..\x", "1.0.0-..", " 1.0.0", "1.0", "latest", ""];
let valid = [("acme.x", "1.0.0"), ("acme.tools-2_b", "1.0.0-rc.1+b.2")];
let mut listed: Vec<(&str, &str)> = valid.to_vec();
listed.extend(invalid_ids.iter().map(|id| (*id, "1.0.0")));
listed.extend(invalid_versions.iter().map(|version| ("acme.x", *version)));
let plugins: Vec<serde_json::Value> = listed
.iter()
.map(|(id, version)| {
serde_json::json!({
"id": id, "name": "X", "version": version, "publisher": "acme",
"download": "x.oiplugin", "digest": "d", "engine": "^1.0",
})
})
.collect();
let text = serde_json::json!({ "format": INDEX_FORMAT, "plugins": plugins }).to_string();
let pairs = |entries: &[IndexEntry]| -> Vec<(String, String)> {
entries.iter().map(|e| (e.id.clone(), e.version.clone())).collect()
};
let owned = |pairs: &[(&str, &str)]| -> Vec<(String, String)> {
pairs.iter().map(|(id, version)| ((*id).to_owned(), (*version).to_owned())).collect()
};
assert_eq!(pairs(&Index::parse(&text).unwrap().plugins), owned(&valid));
let (index, dropped) = Index::parse_dropping(&text).unwrap();
assert_eq!(pairs(&index.plugins), owned(&valid));
assert_eq!(pairs(&dropped), owned(&listed[valid.len()..]));
}
#[test]
fn a_newer_index_still_revokes_and_moves() {
let newer = format!(
+192
View File
@@ -0,0 +1,192 @@
//! Extension points a plugin declares for other plugins to contribute to
//! (`[[extension-points]]`), and the check of a contribution against them.
use serde::{Deserialize, Serialize};
use specta::Type;
use std::collections::BTreeMap;
const MAX_FIELDS: usize = 32;
const MAX_ID: usize = 128;
/// Longest string a contribution may carry in one field.
const MAX_TEXT: usize = 64 * 1024;
const MAX_LIST: usize = 1000;
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "kebab-case")]
pub enum FieldType {
String,
Boolean,
Integer,
Number,
StringList,
}
/// `[[extension-points]]`: a kind of contribution the declaring plugin's code reads.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(deny_unknown_fields)]
pub struct PointDecl {
/// `<plugin id>.<name>`.
pub id: String,
#[serde(default)]
pub description: String,
/// Fields a contribution must have.
#[serde(default)]
pub required: Vec<String>,
/// Every field a contribution may have; a contribution with another key is refused.
#[serde(default)]
pub fields: BTreeMap<String, FieldType>,
}
impl PointDecl {
pub fn validate(&self, plugin_id: &str) -> Result<(), String> {
let prefix = format!("{plugin_id}.");
let name = self.id.strip_prefix(&prefix).unwrap_or("");
let name_ok = name.chars().next().is_some_and(|c| c.is_ascii_lowercase() || c.is_ascii_digit())
&& name.chars().all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || matches!(c, '-' | '_' | '.'));
if !name_ok || self.id.len() > MAX_ID {
return Err(format!(
"invalid extension point {:?}: use \"{plugin_id}.<name>\" with a-z, 0-9, '.', '-', '_' in the name",
self.id
));
}
if self.description.len() > 500 {
return Err(format!("extension point {}: `description` is longer than 500 characters", self.id));
}
if self.fields.len() > MAX_FIELDS {
return Err(format!("extension point {}: at most {MAX_FIELDS} fields", self.id));
}
for field in self.fields.keys() {
let ok = field.chars().next().is_some_and(|c| c.is_ascii_lowercase())
&& field.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '-' | '_'))
&& field.len() <= 64;
if !ok {
return Err(format!("extension point {}: invalid field name {field:?}", self.id));
}
}
for field in &self.required {
if !self.fields.contains_key(field) {
return Err(format!("extension point {}: required field {field:?} is not in `fields`", self.id));
}
}
Ok(())
}
/// Checks one contribution and returns it as JSON for the declaring plugin's code.
pub fn check(&self, value: &toml::Value) -> Result<serde_json::Value, String> {
let toml::Value::Table(table) = value else {
return Err("a contribution is a table".into());
};
for key in table.keys() {
if !self.fields.contains_key(key) {
let known: Vec<&str> = self.fields.keys().map(String::as_str).collect();
return Err(format!("unknown field {key:?} (this extension point has: {})", known.join(", ")));
}
}
for key in &self.required {
if !table.contains_key(key) {
return Err(format!("missing required field {key:?}"));
}
}
let mut out = serde_json::Map::new();
for (key, value) in table {
out.insert(key.clone(), convert(key, self.fields[key], value)?);
}
Ok(serde_json::Value::Object(out))
}
}
fn convert(key: &str, ty: FieldType, value: &toml::Value) -> Result<serde_json::Value, String> {
use serde_json::Value as J;
let wrong = |expected: &str| format!("field {key:?} must be {expected}");
match (ty, value) {
(FieldType::String, toml::Value::String(s)) => text(key, s).map(J::from),
(FieldType::Boolean, toml::Value::Boolean(b)) => Ok(J::from(*b)),
(FieldType::Integer, toml::Value::Integer(i)) => Ok(J::from(*i)),
(FieldType::Number, toml::Value::Integer(i)) => Ok(J::from(*i)),
(FieldType::Number, toml::Value::Float(f)) if f.is_finite() => Ok(J::from(*f)),
(FieldType::StringList, toml::Value::Array(items)) => {
if items.len() > MAX_LIST {
return Err(format!("field {key:?} has more than {MAX_LIST} items"));
}
items
.iter()
.map(|item| match item {
toml::Value::String(s) => text(key, s).map(J::from),
_ => Err(wrong("a list of strings")),
})
.collect::<Result<Vec<_>, _>>()
.map(J::Array)
}
(FieldType::String, _) => Err(wrong("a string")),
(FieldType::Boolean, _) => Err(wrong("true or false")),
(FieldType::Integer, _) => Err(wrong("an integer")),
(FieldType::Number, _) => Err(wrong("a number")),
(FieldType::StringList, _) => Err(wrong("a list of strings")),
}
}
fn text(key: &str, s: &str) -> Result<String, String> {
if s.len() > MAX_TEXT {
return Err(format!("field {key:?} is longer than {} KiB", MAX_TEXT >> 10));
}
Ok(s.to_owned())
}
#[cfg(test)]
mod tests {
use super::*;
fn decl(src: &str) -> PointDecl {
toml::from_str(src).unwrap()
}
const COMPLETIONS: &str = r#"
id = "acme.base.completions"
description = "Extra completions"
required = ["label"]
fields = { label = "string", detail = "string", weight = "integer", scopes = "string-list" }
"#;
#[test]
fn the_id_is_namespaced_by_the_declaring_plugin() {
assert!(decl(COMPLETIONS).validate("acme.base").is_ok());
let err = decl(COMPLETIONS).validate("acme.other").unwrap_err();
assert!(err.contains("acme.other.<name>"), "{err}");
assert!(decl("id = \"acme.base.\"").validate("acme.base").is_err());
assert!(decl("id = \"acme.base.Bad Name\"").validate("acme.base").is_err());
}
#[test]
fn required_fields_must_be_declared() {
let err = decl("id = \"p.x\"\nrequired = [\"a\"]").validate("p").unwrap_err();
assert!(err.contains("required field \"a\""), "{err}");
}
#[test]
fn checks_a_contribution_and_converts_it_to_json() {
let point = decl(COMPLETIONS);
let contribution: toml::Value =
toml::from_str("label = \"ew.x\"\nweight = 3\nscopes = [\"xml\", \"java\"]").unwrap();
let json = point.check(&contribution).unwrap();
assert_eq!(json, serde_json::json!({"label": "ew.x", "weight": 3, "scopes": ["xml", "java"]}));
}
#[test]
fn refuses_what_the_point_does_not_declare() {
let point = decl(COMPLETIONS);
let check = |src: &str| point.check(&toml::from_str::<toml::Value>(src).unwrap()).unwrap_err();
assert!(check("detail = \"x\"").contains("missing required field \"label\""));
let err = check("label = \"a\"\nextra = 1");
assert!(err.contains("unknown field \"extra\"") && err.contains("label"), "{err}");
assert!(check("label = 1").contains("must be a string"));
assert!(check("label = \"a\"\nweight = \"3\"").contains("must be an integer"));
assert!(check("label = \"a\"\nscopes = [1]").contains("list of strings"));
}
#[test]
fn numbers_take_integers_and_floats() {
let point = decl("id = \"p.x\"\nfields = { ratio = \"number\" }");
assert!(point.check(&toml::from_str::<toml::Value>("ratio = 2").unwrap()).is_ok());
assert!(point.check(&toml::from_str::<toml::Value>("ratio = 0.5").unwrap()).is_ok());
}
}
+281 -6
View File
@@ -1,6 +1,7 @@
//! Extension points plus the contributions committed to them, accounted per plugin.
use crate::capability::Capability;
use crate::plugin_point::PointDecl;
use crate::point::{Disposable, ExtensionPoint, Package, Placement, Prepared};
use parking_lot::{Mutex, RwLock};
use serde::{Deserialize, Serialize};
@@ -23,7 +24,7 @@ pub struct ContributionInfo {
}
struct Entry {
point: &'static str,
point: String,
key: Option<String>,
label: String,
disposable: Disposable,
@@ -38,7 +39,7 @@ pub struct PreparedSet {
pub files: Vec<PathBuf>,
/// Declarations for extension points this host does not know (skipped).
pub warnings: Vec<String>,
items: Vec<(&'static str, Prepared)>,
items: Vec<(String, Prepared)>,
}
impl PreparedSet {
@@ -61,11 +62,41 @@ impl std::fmt::Debug for PreparedSet {
}
}
/// What one plugin contributed to an extension point another plugin declared.
#[derive(Clone, Debug, PartialEq)]
pub struct Contribution {
/// The contributing plugin.
pub plugin: String,
/// The declaration as JSON, checked against the point's fields.
pub data: serde_json::Value,
}
/// An extension point a plugin declared, with who contributes to it.
#[derive(Clone, Debug, PartialEq)]
pub struct DeclaredPoint {
pub owner: String,
pub decl: PointDecl,
/// Plugin ids, in commit order, once each.
pub contributors: Vec<String>,
}
#[derive(Default)]
struct Dynamic {
points: BTreeMap<String, (String, PointDecl)>,
contributions: BTreeMap<String, Vec<Contribution>>,
}
/// Called with (point id, owner plugin) after contributions to a declared point changed.
pub type ChangeListener = Arc<dyn Fn(&str, &str) + Send + Sync>;
#[derive(Default)]
pub struct ExtensionRegistry {
points: RwLock<BTreeMap<&'static str, Arc<dyn ExtensionPoint>>>,
/// plugin id → its contributions in commit order.
entries: Mutex<BTreeMap<String, Vec<Entry>>>,
/// Points declared by plugins and what was contributed to them.
dynamic: Arc<Mutex<Dynamic>>,
listener: Arc<RwLock<Option<ChangeListener>>>,
}
impl ExtensionRegistry {
@@ -97,6 +128,111 @@ impl ExtensionRegistry {
self.points.read().get(id).cloned()
}
/// Registers the function told when contributions to a plugin-declared point change.
pub fn set_change_listener(&self, listener: ChangeListener) {
*self.listener.write() = Some(listener);
}
/// The plugin that declared `point`, while it is committed.
pub fn point_owner(&self, point: &str) -> Option<String> {
self.dynamic.lock().points.get(point).map(|(owner, _)| owner.clone())
}
/// What plugins contributed to `point`, in commit order. Empty unless a plugin declares it.
pub fn contributions_to(&self, point: &str) -> Vec<Contribution> {
let dynamic = self.dynamic.lock();
if !dynamic.points.contains_key(point) {
return Vec::new();
}
dynamic.contributions.get(point).cloned().unwrap_or_default()
}
/// The points `owner` declared, with their contributors.
pub fn declared_points(&self, owner: &str) -> Vec<DeclaredPoint> {
let dynamic = self.dynamic.lock();
dynamic
.points
.iter()
.filter(|(_, (o, _))| o == owner)
.map(|(id, (owner, decl))| {
let mut contributors: Vec<String> = Vec::new();
for c in dynamic.contributions.get(id).into_iter().flatten() {
if !contributors.contains(&c.plugin) {
contributors.push(c.plugin.clone());
}
}
DeclaredPoint { owner: owner.clone(), decl: decl.clone(), contributors }
})
.collect()
}
/// Makes `pkg`'s `[[extension-points]]` known: they take effect with the set's commit.
fn prepare_declarations(&self, pkg: &Package) -> Vec<(String, Prepared)> {
pkg.manifest
.extension_points
.iter()
.map(|decl| {
let (dynamic, owner, decl) = (self.dynamic.clone(), pkg.id().to_owned(), decl.clone());
let id = decl.id.clone();
let prepared = Prepared::new(format!("Extension point {id}"), move || {
let id = decl.id.clone();
dynamic.lock().points.insert(id.clone(), (owner.clone(), decl));
Ok(Disposable::new(move || {
dynamic.lock().points.remove(&id);
}))
})
.key(format!("extension-point:{id}"));
(id, prepared)
})
.collect()
}
/// Validates `pkg`'s contributions to a point some plugin declares, or `None`
/// when no plugin declares `point` (the caller then skips it as unknown).
fn prepare_contributions(&self, pkg: &Package, point: &str) -> Option<Result<Vec<Prepared>, String>> {
let declared = self.dynamic.lock().points.get(point).cloned();
let own =
pkg.manifest.extension_points.iter().find(|d| d.id == point).map(|d| (pkg.id().to_owned(), d.clone()));
let (owner, decl) = declared.or(own)?;
if owner != pkg.id() && !pkg.manifest.depends.iter().any(|d| d.id == owner) {
return Some(Err(format!("{point} is declared by {owner}: add it to [[depends]]")));
}
let listener = self.listener.clone();
let result = pkg
.manifest
.declarations(point)
.into_iter()
.enumerate()
.map(|(i, value)| {
let data = decl.check(&value).map_err(|e| format!("contributions[{i}]: {e}"))?;
let label = decl
.required
.first()
.and_then(|field| data.get(field))
.and_then(|v| v.as_str())
.map_or_else(|| point.to_owned(), str::to_owned);
let contribution = Contribution { plugin: pkg.id().to_owned(), data };
let (dynamic, listener, point) = (self.dynamic.clone(), listener.clone(), point.to_owned());
Ok(Prepared::new(label, move || {
dynamic.lock().contributions.entry(point.clone()).or_default().push(contribution.clone());
notify(&dynamic, &listener, &point);
Ok(Disposable::new(move || {
{
let mut guard = dynamic.lock();
if let Some(list) = guard.contributions.get_mut(&point)
&& let Some(pos) = list.iter().position(|c| *c == contribution)
{
list.remove(pos);
}
}
notify(&dynamic, &listener, &point);
}))
}))
})
.collect();
Some(result)
}
/// Validates every `contributes.<point>` declaration of `pkg`. Fails as a
/// whole on the first invalid declaration; unknown points are skipped with
/// a warning (forward compatibility).
@@ -108,9 +244,24 @@ impl ExtensionRegistry {
warnings: Vec::new(),
items: Vec::new(),
};
// Declared points first: contributions to them (the plugin's own too) commit after.
set.items.extend(self.prepare_declarations(pkg));
for point_id in pkg.manifest.contributes.keys() {
let Some(point) = self.point(point_id) else {
let msg = format!("contributes.{point_id}: not supported by this version of the IDE; skipped");
if let Some(result) = self.prepare_contributions(pkg, point_id) {
let prepared = result.map_err(|e| format!("contributes.{point_id}: {e}"))?;
set.items.extend(prepared.into_iter().map(|p| (point_id.clone(), p)));
continue;
}
// Point ids are `<owner id>.<name>`, so the dependency that owns it can be named.
let owner =
pkg.manifest.depends.iter().find(|d| point_id.strip_prefix(&format!("{}.", d.id)).is_some());
let msg = match owner {
Some(owner) => {
format!("contributes.{point_id}: {} is not running, so this is skipped until it is", owner.id)
}
None => format!("contributes.{point_id}: not supported by this version of the IDE; skipped"),
};
tracing::warn!(plugin = %pkg.id(), "{msg}");
set.warnings.push(msg);
continue;
@@ -129,7 +280,7 @@ impl ExtensionRegistry {
set.files.push(f.clone());
}
}
set.items.push((point.id(), prepared));
set.items.push((point.id().to_owned(), prepared));
}
}
if let Some(point) = self.point(LOCALES_POINT)
@@ -144,7 +295,7 @@ impl ExtensionRegistry {
set.files.push(f.clone());
}
}
set.items.push((point.id(), prepared));
set.items.push((point.id().to_owned(), prepared));
}
Ok(set)
}
@@ -201,7 +352,7 @@ impl ExtensionRegistry {
.lock()
.get(plugin)
.map(|list| {
list.iter().map(|e| ContributionInfo { point: e.point.into(), label: e.label.clone() }).collect()
list.iter().map(|e| ContributionInfo { point: e.point.clone(), label: e.label.clone() }).collect()
})
.unwrap_or_default()
}
@@ -217,6 +368,14 @@ impl ExtensionRegistry {
}
}
fn notify(dynamic: &Mutex<Dynamic>, listener: &RwLock<Option<ChangeListener>>, point: &str) {
let owner = dynamic.lock().points.get(point).map(|(owner, _)| owner.clone());
let callback = listener.read().clone();
if let (Some(owner), Some(callback)) = (owner, callback) {
callback(point, &owner);
}
}
fn strip_prefix<'a>(e: &'a str, point: &str) -> &'a str {
e.strip_prefix(&format!("contributes.{point}: ")).unwrap_or(e)
}
@@ -310,6 +469,122 @@ mod tests {
assert!(reg.contributions("p").is_empty());
}
fn pkg_of(id: &str, body: &str) -> Package {
let src = format!("id = \"{id}\"\nname = \"N\"\nversion = \"1.0.0\"\nengines.openidea = \"^1.5\"\n{body}");
Package::new(Manifest::parse(&src).unwrap(), PathBuf::from("/nonexistent"))
}
const BASE: &str = r#"
[[extension-points]]
id = "acme.base.things"
required = ["name"]
fields = { name = "string", weight = "integer" }
"#;
const PLUGIN: &str =
"[[depends]]\nid = \"acme.base\"\n\n[[contributes.\"acme.base.things\"]]\nname = \"a\"\nweight = 2\n";
fn commit(reg: &ExtensionRegistry, pkg: &Package) {
reg.commit(reg.prepare(pkg).unwrap()).unwrap();
}
#[test]
fn a_plugin_declares_a_point_and_a_dependent_contributes_to_it() {
let reg = ExtensionRegistry::new();
commit(&reg, &pkg_of("acme.base", BASE));
assert_eq!(reg.point_owner("acme.base.things").as_deref(), Some("acme.base"));
assert!(reg.contributions_to("acme.base.things").is_empty());
commit(&reg, &pkg_of("acme.more", PLUGIN));
let got = reg.contributions_to("acme.base.things");
assert_eq!(got.len(), 1);
assert_eq!(got[0].plugin, "acme.more");
assert_eq!(got[0].data, serde_json::json!({"name": "a", "weight": 2}));
assert_eq!(
reg.contributions("acme.more")[0],
ContributionInfo { point: "acme.base.things".into(), label: "a".into() }
);
let declared = reg.declared_points("acme.base");
assert_eq!(declared.len(), 1);
assert_eq!(declared[0].contributors, ["acme.more"]);
assert!(reg.dispose_plugin("acme.more"));
assert!(reg.contributions_to("acme.base.things").is_empty());
assert!(reg.dispose_plugin("acme.base"));
assert_eq!(reg.point_owner("acme.base.things"), None);
}
#[test]
fn contributions_disappear_from_view_while_the_owner_is_gone() {
let reg = ExtensionRegistry::new();
commit(&reg, &pkg_of("acme.base", BASE));
commit(&reg, &pkg_of("acme.more", PLUGIN));
reg.dispose_plugin("acme.base");
assert!(reg.contributions_to("acme.base.things").is_empty(), "nobody may read a point that is not declared");
commit(&reg, &pkg_of("acme.base", BASE));
assert_eq!(reg.contributions_to("acme.base.things").len(), 1, "back with the owner");
}
#[test]
fn contributing_needs_a_dependency_on_the_owner() {
let reg = ExtensionRegistry::new();
commit(&reg, &pkg_of("acme.base", BASE));
let err =
reg.prepare(&pkg_of("acme.rogue", "[[contributes.\"acme.base.things\"]]\nname = \"x\"\n")).unwrap_err();
assert!(err.contains("acme.base") && err.contains("[[depends]]"), "{err}");
}
#[test]
fn a_contribution_is_checked_against_the_declared_fields() {
let reg = ExtensionRegistry::new();
commit(&reg, &pkg_of("acme.base", BASE));
let bad = "[[depends]]\nid = \"acme.base\"\n[[contributes.\"acme.base.things\"]]\nweight = 1\n";
let err = reg.prepare(&pkg_of("acme.more", bad)).unwrap_err();
assert_eq!(err, "contributes.acme.base.things: contributions[0]: missing required field \"name\"");
}
#[test]
fn a_point_nobody_declares_is_skipped_with_a_warning() {
let reg = ExtensionRegistry::new();
let set = reg.prepare(&pkg_of("acme.more", PLUGIN)).unwrap();
assert!(set.is_empty());
assert!(set.warnings[0].contains("acme.base.things"), "{:?}", set.warnings);
}
#[test]
fn a_missing_owner_is_named_instead_of_blaming_the_ide_version() {
let reg = ExtensionRegistry::new();
let set = reg.prepare(&pkg_of("acme.more", PLUGIN)).unwrap();
let warning = &set.warnings[0];
assert!(warning.contains("acme.base") && warning.contains("not running"), "{warning}");
assert!(!warning.contains("this version of the IDE"), "{warning}");
let unrelated = "[[contributes.\"acme.nobody.things\"]]\nname = \"x\"\n";
let set = reg.prepare(&pkg_of("acme.more", unrelated)).unwrap();
assert!(set.warnings[0].contains("this version of the IDE"), "{:?}", set.warnings);
}
#[test]
fn a_plugin_may_contribute_to_its_own_point() {
let reg = ExtensionRegistry::new();
let own = format!("{BASE}\n[[contributes.\"acme.base.things\"]]\nname = \"builtin\"\n");
commit(&reg, &pkg_of("acme.base", &own));
assert_eq!(reg.contributions_to("acme.base.things")[0].data["name"], "builtin");
}
#[test]
fn the_owner_hears_about_changes_to_its_point() {
let reg = ExtensionRegistry::new();
let heard: Arc<Mutex<Vec<(String, String)>>> = Arc::default();
let sink = heard.clone();
reg.set_change_listener(Arc::new(move |point, owner| sink.lock().push((point.into(), owner.into()))));
commit(&reg, &pkg_of("acme.base", BASE));
assert!(heard.lock().is_empty(), "declaring is not a contribution");
commit(&reg, &pkg_of("acme.more", PLUGIN));
reg.dispose_plugin("acme.more");
let expected = ("acme.base.things".to_owned(), "acme.base".to_owned());
assert_eq!(*heard.lock(), [expected.clone(), expected]);
}
#[test]
fn invalid_declarations_fail_prepare_and_unknown_points_warn() {
let (reg, _) = setup();
+433
View File
@@ -0,0 +1,433 @@
//! Which plugins can run, and in which order, given what they depend on.
use semver::{Version, VersionReq};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::collections::{BTreeMap, BTreeSet};
use std::fmt;
/// `[[depends]]` of a manifest.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(deny_unknown_fields)]
pub struct Dependency {
pub id: String,
/// Semver requirement on the dependency's version; none accepts any.
#[serde(default)]
pub version: Option<String>,
/// The plugin runs without it; when present it starts first.
#[serde(default)]
pub optional: bool,
}
impl Dependency {
/// `acme.base ^1.0`.
pub fn describe(&self) -> String {
match &self.version {
Some(v) => format!("{} {v}", self.id),
None => self.id.clone(),
}
}
}
/// A plugin as the resolver sees it.
#[derive(Clone, Copy, Debug)]
pub struct Node<'a> {
pub id: &'a str,
pub version: &'a str,
pub depends: &'a [Dependency],
/// Not switched off by the user.
pub enabled: bool,
}
/// Why an enabled plugin cannot run.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase", tag = "kind")]
pub enum Blocked {
/// A required plugin is not installed.
Missing { requirement: String },
/// A required plugin is switched off.
Disabled { id: String },
/// A required plugin is installed in a version that does not satisfy the requirement.
Version { requirement: String, found: String },
/// A required plugin cannot run itself.
Dependency { id: String },
/// The plugin is part of a dependency cycle (`a → b → a`).
Cycle { ring: Vec<String> },
}
impl fmt::Display for Blocked {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Missing { requirement } => write!(f, "Requires {requirement}, which is not installed"),
Self::Disabled { id } => write!(f, "Requires {id}, which is disabled"),
Self::Version { requirement, found } => {
write!(f, "Requires {requirement}, but version {found} is installed")
}
Self::Dependency { id } => write!(f, "Requires {id}, which cannot run"),
Self::Cycle { ring } => write!(f, "Part of a dependency cycle: {}", ring.join(" → ")),
}
}
}
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct Plan {
/// Plugins that can run, every plugin after the plugins it depends on.
pub order: Vec<String>,
/// Enabled plugins that cannot run. Disabled plugins are in neither list.
pub blocked: BTreeMap<String, Blocked>,
}
/// Decides, for every enabled plugin, whether its required plugins allow it to
/// run and the order to start them in (the counterpart of IntelliJ's
/// `PluginSetBuilder`: dependency graph, topological order, cycle report).
pub fn plan(nodes: &[Node<'_>]) -> Plan {
let by_id: BTreeMap<&str, &Node<'_>> = nodes.iter().map(|n| (n.id, n)).collect();
let enabled: Vec<&Node<'_>> = by_id.values().copied().filter(|n| n.enabled).collect();
let mut blocked: BTreeMap<String, Blocked> = BTreeMap::new();
for node in &enabled {
for dep in node.depends.iter().filter(|d| !d.optional) {
if let Some(reason) = direct_reason(dep, by_id.get(dep.id.as_str()).copied()) {
blocked.entry(node.id.to_owned()).or_insert(reason);
}
}
}
// Cycles among plugins whose requirements are otherwise present.
let candidates: BTreeSet<&str> = enabled.iter().map(|n| n.id).filter(|id| !blocked.contains_key(*id)).collect();
let graph: Graph<'_> = candidates
.iter()
.map(|id| {
let deps = by_id[id].depends.iter().filter(|d| !d.optional).map(|d| d.id.as_str());
(*id, deps.filter(|d| candidates.contains(d)).collect())
})
.collect();
for component in components(&graph) {
let cyclic = component.len() > 1 || graph[component[0]].contains(&component[0]);
if cyclic {
let ring = ring_of(&component, &graph);
for id in component {
blocked.insert(id.to_owned(), Blocked::Cycle { ring: ring.clone() });
}
}
}
// A plugin that needs a blocked plugin is blocked too.
loop {
let mut changed = false;
for node in &enabled {
if blocked.contains_key(node.id) {
continue;
}
if let Some(dep) = node.depends.iter().find(|d| !d.optional && blocked.contains_key(&d.id)) {
blocked.insert(node.id.to_owned(), Blocked::Dependency { id: dep.id.clone() });
changed = true;
}
}
if !changed {
break;
}
}
let runnable: BTreeSet<&str> = enabled.iter().map(|n| n.id).filter(|id| !blocked.contains_key(*id)).collect();
Plan { order: topological(&runnable, &by_id), blocked }
}
/// Plugins that need `id`, directly or through other plugins, whether or not
/// they are enabled. Optional dependencies do not count.
pub fn dependents(nodes: &[Node<'_>], id: &str) -> Vec<String> {
let mut found: BTreeSet<String> = BTreeSet::new();
let mut frontier = vec![id.to_owned()];
while let Some(current) = frontier.pop() {
for node in nodes {
if node.depends.iter().any(|d| !d.optional && d.id == current) && found.insert(node.id.to_owned()) {
frontier.push(node.id.to_owned());
}
}
}
found.remove(id);
found.into_iter().collect()
}
/// How one dependency stands right now, for the plugin details page.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum DependencyState {
Satisfied,
Missing,
Disabled,
/// Installed, wrong version.
Version,
/// Installed and enabled, but cannot run itself.
Blocked,
}
pub fn dependency_state(dep: &Dependency, found: Option<&Node<'_>>, found_blocked: bool) -> DependencyState {
match direct_reason(dep, found) {
Some(Blocked::Missing { .. }) => DependencyState::Missing,
Some(Blocked::Disabled { .. }) => DependencyState::Disabled,
Some(Blocked::Version { .. }) => DependencyState::Version,
_ if found_blocked => DependencyState::Blocked,
_ => DependencyState::Satisfied,
}
}
fn direct_reason(dep: &Dependency, found: Option<&Node<'_>>) -> Option<Blocked> {
let Some(target) = found else {
return Some(Blocked::Missing { requirement: dep.describe() });
};
if !target.enabled {
return Some(Blocked::Disabled { id: dep.id.clone() });
}
let want = dep.version.as_deref()?;
let satisfied = VersionReq::parse(want)
.ok()
.zip(Version::parse(target.version).ok())
.is_some_and(|(req, have)| req.matches(&have));
(!satisfied).then(|| Blocked::Version { requirement: dep.describe(), found: target.version.to_owned() })
}
/// Plugin id → the plugin ids it requires.
type Graph<'a> = BTreeMap<&'a str, Vec<&'a str>>;
/// Strongly connected components (Tarjan) of `graph`.
fn components<'a>(graph: &Graph<'a>) -> Vec<Vec<&'a str>> {
struct State<'a, 'g> {
index: usize,
stack: Vec<&'a str>,
info: BTreeMap<&'a str, (usize, usize, bool)>,
out: Vec<Vec<&'a str>>,
graph: &'g Graph<'a>,
}
fn visit<'a>(s: &mut State<'a, '_>, v: &'a str) {
s.info.insert(v, (s.index, s.index, true));
s.index += 1;
s.stack.push(v);
for &w in &s.graph[v] {
match s.info.get(w).copied() {
None => {
visit(s, w);
let low = s.info[w].1;
let entry = s.info.get_mut(v).expect("visited");
entry.1 = entry.1.min(low);
}
Some((index, _, true)) => {
let entry = s.info.get_mut(v).expect("visited");
entry.1 = entry.1.min(index);
}
Some(_) => {}
}
}
let (index, low, _) = s.info[v];
if index == low {
let mut component = Vec::new();
while let Some(w) = s.stack.pop() {
s.info.get_mut(w).expect("visited").2 = false;
component.push(w);
if w == v {
break;
}
}
component.sort_unstable();
s.out.push(component);
}
}
let mut state = State { index: 0, stack: Vec::new(), info: BTreeMap::new(), out: Vec::new(), graph };
for &id in graph.keys() {
if !state.info.contains_key(id) {
visit(&mut state, id);
}
}
state.out
}
/// One cycle through `component`, from its smallest id back to it.
fn ring_of(component: &[&str], graph: &Graph<'_>) -> Vec<String> {
let start = component[0];
let mut ring = vec![start.to_owned()];
let mut current = start;
let mut seen: BTreeSet<&str> = BTreeSet::from([start]);
loop {
let mut next: Vec<&str> = graph[current].iter().copied().filter(|d| component.contains(d)).collect();
next.sort_unstable();
// Prefer closing the ring; otherwise take the smallest unseen member.
let step =
if next.contains(&start) { start } else { *next.iter().find(|d| !seen.contains(**d)).unwrap_or(&start) };
ring.push(step.to_owned());
if step == start {
return ring;
}
seen.insert(step);
current = step;
}
}
/// Kahn's algorithm over the runnable plugins; ties break by id. Optional
/// dependencies order too, but never block: leftovers (an optional edge
/// closing a loop) are appended by id.
fn topological(runnable: &BTreeSet<&str>, by_id: &BTreeMap<&str, &Node<'_>>) -> Vec<String> {
let mut waiting: BTreeMap<&str, BTreeSet<&str>> = runnable
.iter()
.map(|id| {
let deps = by_id[id].depends.iter().map(|d| d.id.as_str()).filter(|d| runnable.contains(d) && d != id);
(*id, deps.collect())
})
.collect();
let mut order: Vec<String> = Vec::with_capacity(runnable.len());
while let Some(next) = waiting.iter().find(|(_, deps)| deps.is_empty()).map(|(id, _)| *id) {
waiting.remove(next);
for deps in waiting.values_mut() {
deps.remove(next);
}
order.push(next.to_owned());
}
order.extend(waiting.keys().map(|id| (*id).to_owned()));
order
}
#[cfg(test)]
mod tests {
use super::*;
fn dep(id: &str) -> Dependency {
Dependency { id: id.into(), version: None, optional: false }
}
fn dep_v(id: &str, version: &str) -> Dependency {
Dependency { id: id.into(), version: Some(version.into()), optional: false }
}
fn optional(id: &str) -> Dependency {
Dependency { id: id.into(), version: None, optional: true }
}
fn node<'a>(id: &'a str, depends: &'a [Dependency]) -> Node<'a> {
Node { id, version: "1.0.0", depends, enabled: true }
}
#[test]
fn dependencies_start_first_whatever_the_ids_sort_like() {
let (a, z) = ([dep("z.base")], []);
let plan = plan(&[node("a.app", &a), node("z.base", &z)]);
assert_eq!(plan.order, ["z.base", "a.app"]);
assert!(plan.blocked.is_empty());
}
#[test]
fn independent_plugins_keep_id_order() {
let plan = plan(&[node("b", &[]), node("a", &[]), node("c", &[])]);
assert_eq!(plan.order, ["a", "b", "c"]);
}
#[test]
fn a_missing_requirement_blocks_with_the_requirement_text() {
let needs = [dep_v("acme.base", "^1.0")];
let plan = plan(&[node("acme.app", &needs)]);
assert!(plan.order.is_empty());
assert_eq!(plan.blocked["acme.app"], Blocked::Missing { requirement: "acme.base ^1.0".into() });
assert_eq!(plan.blocked["acme.app"].to_string(), "Requires acme.base ^1.0, which is not installed");
}
#[test]
fn a_disabled_requirement_blocks_but_the_disabled_plugin_itself_is_not_blocked() {
let needs = [dep("acme.base")];
let mut base = node("acme.base", &[]);
base.enabled = false;
let plan = plan(&[base, node("acme.app", &needs)]);
assert_eq!(plan.blocked["acme.app"], Blocked::Disabled { id: "acme.base".into() });
assert!(!plan.blocked.contains_key("acme.base"));
assert!(plan.order.is_empty());
}
#[test]
fn the_installed_version_must_satisfy_the_requirement() {
let needs = [dep_v("acme.base", "^0.2")];
let mut base = node("acme.base", &[]);
base.version = "0.3.0";
let plan_bad = plan(&[base, node("acme.app", &needs)]);
assert_eq!(
plan_bad.blocked["acme.app"],
Blocked::Version { requirement: "acme.base ^0.2".into(), found: "0.3.0".into() }
);
assert_eq!(plan_bad.blocked["acme.app"].to_string(), "Requires acme.base ^0.2, but version 0.3.0 is installed");
base.version = "0.2.7";
let plan_ok = plan(&[base, node("acme.app", &needs)]);
assert!(plan_ok.blocked.is_empty());
assert_eq!(plan_ok.order, ["acme.base", "acme.app"]);
}
#[test]
fn blocking_propagates_to_everything_downstream() {
let (b, c) = ([dep("a")], [dep("b")]);
let plan = plan(&[node("b", &b), node("c", &c)]);
assert_eq!(plan.blocked["b"], Blocked::Missing { requirement: "a".into() });
assert_eq!(plan.blocked["c"], Blocked::Dependency { id: "b".into() });
assert!(plan.order.is_empty());
}
#[test]
fn an_optional_requirement_never_blocks_but_orders_when_present() {
let app = [optional("acme.extra")];
let alone = plan(&[node("acme.app", &app)]);
assert_eq!(alone.order, ["acme.app"]);
assert!(alone.blocked.is_empty());
let with = plan(&[node("acme.app", &app), node("acme.extra", &[])]);
assert_eq!(with.order, ["acme.extra", "acme.app"]);
let mut off = node("acme.extra", &[]);
off.enabled = false;
let disabled = plan(&[node("acme.app", &app), off]);
assert_eq!(disabled.order, ["acme.app"]);
assert!(disabled.blocked.is_empty());
}
#[test]
fn a_cycle_blocks_its_members_and_names_the_ring() {
let (a, b, d) = ([dep("b")], [dep("a")], [dep("a")]);
let plan = plan(&[node("a", &a), node("b", &b), node("d", &d)]);
let ring = Blocked::Cycle { ring: vec!["a".into(), "b".into(), "a".into()] };
assert_eq!(plan.blocked["a"], ring);
assert_eq!(plan.blocked["b"], ring);
assert_eq!(ring.to_string(), "Part of a dependency cycle: a → b → a");
assert_eq!(plan.blocked["d"], Blocked::Dependency { id: "a".into() });
assert!(plan.order.is_empty());
}
#[test]
fn a_plugin_requiring_itself_is_a_cycle() {
let a = [dep("a")];
let plan = plan(&[node("a", &a)]);
assert_eq!(plan.blocked["a"], Blocked::Cycle { ring: vec!["a".into(), "a".into()] });
}
#[test]
fn an_optional_loop_does_not_block_and_still_yields_every_plugin() {
let (a, b) = ([optional("b")], [optional("a")]);
let plan = plan(&[node("a", &a), node("b", &b)]);
assert!(plan.blocked.is_empty());
assert_eq!(plan.order.len(), 2);
}
#[test]
fn dependents_are_transitive_and_ignore_optional_edges() {
let (b, c, d) = ([dep("a")], [dep("b")], [optional("a")]);
let nodes = [node("a", &[]), node("b", &b), node("c", &c), node("d", &d)];
assert_eq!(dependents(&nodes, "a"), ["b", "c"]);
assert_eq!(dependents(&nodes, "c"), Vec::<String>::new());
}
#[test]
fn dependency_state_tells_the_details_page_what_to_say() {
let want = dep_v("base", "^1.0");
assert_eq!(dependency_state(&want, None, false), DependencyState::Missing);
let mut base = node("base", &[]);
assert_eq!(dependency_state(&want, Some(&base), false), DependencyState::Satisfied);
assert_eq!(dependency_state(&want, Some(&base), true), DependencyState::Blocked);
base.version = "2.0.0";
assert_eq!(dependency_state(&want, Some(&base), false), DependencyState::Version);
base.enabled = false;
assert_eq!(dependency_state(&want, Some(&base), false), DependencyState::Disabled);
}
}
+10 -2
View File
@@ -102,6 +102,14 @@ fn parse<T>(r: serde_json::Result<T>) -> IdeResult<T> {
r.map_err(|e| IdeError::new(ide_core::error::ErrorKind::Protocol, format!("unexpected gh output: {e}")))
}
/// The repository of `root`, free to execute: the host runs the `gh_*` commands only in a
/// trusted project.
fn trusted_repo(root: &Path) -> IdeResult<Repo> {
let repo = Repo::discover(root)?.ok_or_else(|| IdeError::invalid("not a git repository"))?;
repo.set_execution_allowed(true);
Ok(repo)
}
/// Whether gh is installed and signed in, and which GitHub repository `root` maps to.
pub fn availability(root: &Path) -> GhAvailability {
if gh_executable().is_none() {
@@ -233,7 +241,7 @@ pub fn create(
return Err(IdeError::invalid("choose a base branch"));
}
if req.push {
let repo = Repo::discover(root)?.ok_or_else(|| IdeError::invalid("not a git repository"))?;
let repo = trusted_repo(root)?;
repo.push(false, true, cancel, on_output)?;
}
let mut args = vec!["pr", "create", "--title", req.title.trim(), "--body", &req.body, "--base", req.base.trim()];
@@ -354,7 +362,7 @@ pub fn prepare_diff(
cancel: &CancellationToken,
on_output: &dyn Fn(String),
) -> IdeResult<PrDiffRevisions> {
let repo = Repo::discover(root)?.ok_or_else(|| IdeError::invalid("not a git repository"))?;
let repo = trusted_repo(root)?;
if !repo.has_commit(head_oid) || !repo.has_commit(base_oid) {
let remotes = repo.remotes()?;
let remote = remote_for(&remotes, name_with_owner)
+28
View File
@@ -0,0 +1,28 @@
[package]
name = "ide-heap"
description = "HotSpot .hprof parser and heap-snapshot analysis for the debugger's JVM Memory view"
version.workspace = true
edition.workspace = true
license.workspace = true
[dependencies]
ide-core.workspace = true
memmap2.workspace = true
serde.workspace = true
serde_json.workspace = true
specta.workspace = true
thiserror.workspace = true
tracing.workspace = true
uuid.workspace = true
[target.'cfg(unix)'.dependencies]
libc = "0.2"
[dev-dependencies]
criterion = { version = "0.5", default-features = false }
proptest.workspace = true
tempfile.workspace = true
[[bench]]
name = "heap"
harness = false
+100
View File
@@ -0,0 +1,100 @@
//! `cargo bench -p ide-heap`: whole-dump analysis and the dominator tree at a million objects.
use criterion::{Criterion, black_box, criterion_group, criterion_main};
use ide_core::CancellationToken;
use ide_heap::testing::{MiniHeap, XorShift};
use ide_heap::{AnalyzeOptions, BasicType, HeapError, HeapPhase, HeapRootKind, Store, analyze, dominators};
const CHAINS: u64 = 100_000;
const CHAIN_LEN: u64 = 5;
const NODES: u64 = CHAINS * CHAIN_LEN;
/// Every object takes one slot: node `k` at slot `2k`, its payload at `2k + 1`.
const BASE: u64 = 0x1_0000_0000;
const SLOT: u64 = 0x20;
fn node_addr(k: u64) -> u64 {
BASE + 2 * k * SLOT
}
fn payload_addr(k: u64) -> u64 {
node_addr(k) + SLOT
}
/// 100k rooted chains of five `Node { next, payload }`, each node with its own `byte[16]`. Every
/// seventh node points at the payload of the same position in the next chain instead, so that
/// payload has two owners and its own one is garbage: 1M objects in all.
fn million_objects() -> MiniHeap {
let mut heap = MiniHeap::new();
heap.class("Node", Some("java.lang.Object"), &[("next", BasicType::Object), ("payload", BasicType::Object)]);
for k in 0..NODES {
let next = if (k + 1) % CHAIN_LEN == 0 { 0 } else { node_addr(k + 1) };
let payload = if k % 7 == 0 { payload_addr((k + CHAIN_LEN) % NODES) } else { payload_addr(k) };
heap.object(node_addr(k), "Node", &[("next", next), ("payload", payload)]);
heap.prim(payload_addr(k), BasicType::Byte, 16);
if k % CHAIN_LEN == 0 {
heap.root(HeapRootKind::JniGlobal, node_addr(k));
}
}
heap
}
/// CSR of a random tree on `nodes` nodes rooted at 0 plus `extra` random edges, targets of each
/// node in scrambled order.
fn random_graph(nodes: u32, extra: u32, seed: u64) -> (Vec<u32>, Vec<u32>) {
let mut rng = XorShift::new(seed);
let mut edges: Vec<(u32, u32)> = (1..nodes).map(|v| (rng.below(v), v)).collect();
edges.extend((0..extra).map(|_| (rng.below(nodes), rng.below(nodes))));
for i in (1..edges.len()).rev() {
edges.swap(i, rng.below(i as u32 + 1) as usize);
}
let mut start = vec![0u32; nodes as usize + 1];
for &(from, _) in &edges {
start[from as usize + 1] += 1;
}
for v in 0..nodes as usize {
start[v + 1] += start[v];
}
let mut cursor = start.clone();
let mut out = vec![0u32; edges.len()];
for &(from, to) in &edges {
out[cursor[from as usize] as usize] = to;
cursor[from as usize] += 1;
}
(start, out)
}
fn no_progress(_: HeapPhase, _: u64, _: u64) -> Result<(), HeapError> {
Ok(())
}
fn bench(c: &mut Criterion) {
let dir = tempfile::tempdir().expect("temp dir");
let hprof = dir.path().join("dump.hprof");
million_objects().write_to(&hprof);
let (start, out) = random_graph(1_000_000, 500_000, 0x5EED_D0D0);
let cancel = CancellationToken::new();
let opts = AnalyzeOptions { in_memory: true, ..AnalyzeOptions::default() };
let summary = analyze(&hprof, dir.path(), opts, &no_progress, &cancel).expect("analyze").summary;
assert_eq!(summary.objects, 2 * NODES, "{summary:?}");
assert_eq!(summary.unreachable_objects, NODES.div_ceil(7), "{summary:?}");
let mut g = c.benchmark_group("heap");
g.sample_size(10);
g.bench_function("index_1m_objects", |b| {
b.iter(|| {
let opts = AnalyzeOptions { in_memory: true, ..AnalyzeOptions::default() };
let index = analyze(&hprof, dir.path(), opts, &no_progress, &cancel).expect("analyze");
black_box(index.summary.objects)
})
});
g.bench_function("dominators_1m_random", |b| {
b.iter(|| {
let d = dominators(&start, &out, 0, &Store::Ram, &no_progress, &cancel).expect("dominators");
black_box(d.reachable)
})
});
g.finish();
}
criterion_group!(benches, bench);
criterion_main!(benches);
+202
View File
@@ -0,0 +1,202 @@
//! Ties the passes together: graph, dominators, shallow and retained sizes, the class table and
//! paths to GC roots, into a [`HeapIndex`].
mod classes;
mod paths;
mod retained;
use std::collections::HashMap;
use std::fs::File;
use std::path::Path;
use std::time::Instant;
use ide_core::CancellationToken;
use memmap2::Mmap;
use crate::column::Store;
use crate::dominators::{Dominators, dominators};
use crate::error::HeapError;
use crate::files::INDEX_FILES;
use crate::graph::{HeapGraph, build_graph};
use crate::index::{HeapIndex, HeapSummary, write_summary};
use crate::progress::{CHECK_EVERY, HeapPhase, Progress};
use crate::shallow::{HistogramEntry, ShallowModel, VmLayout};
#[derive(Clone, Debug, Default)]
pub struct AnalyzeOptions {
/// The JVM's class histogram from the same stop as the dump.
pub histogram: Option<Vec<HistogramEntry>>,
/// `VM.flags` output of the JVM that wrote the dump.
pub vm_flags: Option<String>,
pub java_major: Option<u32>,
/// Keep every array in RAM and write nothing to `index_dir`.
pub in_memory: bool,
}
/// Analyzes the dump at `hprof`, writing the index columns and `summary.json` into `index_dir`
/// (which must exist and may hold the dump itself). `progress` gets every [`HeapPhase`] in order;
/// an `Err` from it, a cancel or any other error removes every index file from `index_dir` and
/// leaves the rest of the directory alone.
pub fn analyze(
hprof: &Path,
index_dir: &Path,
opts: AnalyzeOptions,
progress: &dyn Fn(HeapPhase, u64, u64) -> Result<(), HeapError>,
cancel: &CancellationToken,
) -> Result<HeapIndex, HeapError> {
let store = if opts.in_memory { Store::Ram } else { Store::Dir(index_dir.to_path_buf()) };
let result = run(hprof, &store, opts, progress, cancel);
// `run` has dropped every column by now: Windows cannot delete a mapped file.
if result.is_err() && !matches!(store, Store::Ram) {
remove_index_files(index_dir);
}
result
}
fn run(
hprof: &Path,
store: &Store,
opts: AnalyzeOptions,
report: &dyn Fn(HeapPhase, u64, u64) -> Result<(), HeapError>,
cancel: &CancellationToken,
) -> Result<HeapIndex, HeapError> {
let started = Instant::now();
let progress = Progress::new(report, cancel);
let file = File::open(hprof)?;
// SAFETY: nothing writes the dump while it is analyzed or viewed: a capture finishes writing
// it before analysis and deletes it only with the snapshot that holds this index.
let dump = unsafe { Mmap::map(&file)? };
let graph = build_graph(&dump, store, report, cancel)?;
let Dominators { idom, preorder, .. } =
dominators(&graph.out_start, &graph.out, graph.root, store, report, cancel)?;
let nodes = u64::from(graph.root) + 1;
// Seven walks over the nodes, roughly: sizes, retained, child counts, child sorting, class
// counts and the tree walk, which visits every node twice.
let mut meter = Meter::start(&progress, HeapPhase::Retained, 7 * nodes)?;
let layout = VmLayout::from_vm_flags(opts.vm_flags.as_deref().unwrap_or(""), opts.java_major);
let model = ShallowModel::new(&graph, layout, opts.histogram.as_deref(), cancel)?;
let (shallow8, retained) = retained::sizes(&graph, &model, &idom, &preorder, store, &mut meter)?;
drop(preorder);
let (kids_start, kids) = retained::children(&idom, &retained, graph.root, store, &mut meter)?;
let tree = retained::Tree { idom, shallow8, retained, kids_start, kids };
let (classes, totals) = classes::table(&graph, &tree, store, &mut meter)?;
meter.finish()?;
let mut meter = Meter::start(&progress, HeapPhase::Paths, nodes + graph.out.len() as u64)?;
let gc_parent = paths::gc_parents(&graph, store, &mut meter)?;
meter.finish()?;
progress.report(HeapPhase::Saving, 0, 1)?;
let HeapGraph {
ids,
types,
offsets,
roots,
classes: class_info,
class_slot,
thread_objects,
weak_edges_skipped,
id_size,
..
} = graph;
let summary = HeapSummary {
objects: totals.reachable_objects + totals.unreachable_objects,
reachable_objects: totals.reachable_objects,
reachable_bytes: totals.reachable_bytes,
unreachable_objects: totals.unreachable_objects,
unreachable_bytes: totals.unreachable_bytes,
classes: class_info.iter().filter(|c| c.id != 0).count() as u32,
weak_edges_skipped,
size_source: model.source,
analysis_ms: u32::try_from(started.elapsed().as_millis()).unwrap_or(u32::MAX),
};
if let Store::Dir(dir) = store {
write_summary(dir, &summary)?;
}
let mut root_kinds = HashMap::with_capacity(roots.len());
for (object, kind, serial) in roots {
root_kinds.entry(object).or_insert((kind, serial));
}
progress.report(HeapPhase::Saving, 1, 1)?;
let retained::Tree { idom, shallow8, retained, kids_start, kids } = tree;
Ok(HeapIndex {
hprof: dump,
ids,
types,
offsets,
shallow8,
retained,
idom,
kids_start,
kids,
gc_parent,
classes,
class_info,
class_slot,
roots: root_kinds,
thread_objects,
id_size,
summary,
})
}
/// Removes what an analysis may have written; a file that is not there is fine.
fn remove_index_files(dir: &Path) {
for name in INDEX_FILES {
let path = dir.join(name);
if let Err(err) = std::fs::remove_file(&path)
&& err.kind() != std::io::ErrorKind::NotFound
{
tracing::warn!("could not remove {}: {err}", path.display());
}
}
}
/// Steps of one phase: reports progress every [`CHECK_EVERY`] steps, and each loop looks at the
/// token before its first step.
pub(super) struct Meter<'a> {
progress: &'a Progress<'a>,
phase: HeapPhase,
/// An estimate; `done` is clamped to it.
total: u64,
done: u64,
next_report: u64,
}
impl<'a> Meter<'a> {
fn start(progress: &'a Progress<'a>, phase: HeapPhase, total: u64) -> Result<Self, HeapError> {
progress.report(phase, 0, total)?;
Ok(Self { progress, phase, total, done: 0, next_report: CHECK_EVERY })
}
pub(super) fn begin_loop(&self) -> Result<(), HeapError> {
self.progress.check()
}
pub(super) fn token(&self) -> &'a CancellationToken {
self.progress.token()
}
#[inline]
pub(super) fn tick(&mut self) -> Result<(), HeapError> {
self.advance(1)
}
pub(super) fn advance(&mut self, steps: u64) -> Result<(), HeapError> {
self.done += steps;
if self.done < self.next_report {
return Ok(());
}
self.next_report = self.done + CHECK_EVERY;
self.progress.report(self.phase, self.done.min(self.total), self.total)
}
fn finish(self) -> Result<(), HeapError> {
self.progress.report(self.phase, self.total, self.total)
}
}
#[cfg(test)]
mod tests;
+117
View File
@@ -0,0 +1,117 @@
//! The class table: counts and shallow sizes per type slot, retained sizes per type from a walk of
//! the dominator tree, and the reachable / unreachable totals of the summary.
use super::Meter;
use super::retained::Tree;
use crate::column::{NONE, Store};
use crate::error::HeapError;
use crate::graph::{HeapGraph, PRIM_SLOTS};
use crate::hprof::BasicType;
use crate::index::ClassStat;
use crate::names::display_name;
/// Instances and arrays only: class objects are left out.
#[derive(Default)]
pub(super) struct Totals {
pub(super) reachable_objects: u64,
pub(super) reachable_bytes: u64,
pub(super) unreachable_objects: u64,
pub(super) unreachable_bytes: u64,
}
pub(super) fn table(
graph: &HeapGraph,
tree: &Tree,
store: &Store,
meter: &mut Meter,
) -> Result<(Vec<ClassStat>, Totals), HeapError> {
let slots = graph.classes.len() + PRIM_SLOTS as usize;
let mut classes: Vec<ClassStat> = (0..slots as u32)
.map(|slot| ClassStat { name: slot_name(graph, slot), count: 0, shallow: 0, retained: 0, unreachable_count: 0 })
.collect();
let mut totals = Totals::default();
meter.begin_loop()?;
for v in 0..graph.root as usize {
meter.tick()?;
let slot = graph.types[v];
let bytes = u64::from(tree.shallow8[v]) * 8;
let stat = &mut classes[slot as usize];
let instance = slot != graph.class_slot;
if tree.idom[v] == NONE {
stat.unreachable_count += 1;
if instance {
totals.unreachable_objects += 1;
totals.unreachable_bytes += bytes;
}
} else {
stat.count += 1;
stat.shallow += bytes;
if instance {
totals.reachable_objects += 1;
totals.reachable_bytes += bytes;
}
}
}
retained_by_type(graph, tree, &mut classes, store, meter)?;
Ok((classes, totals))
}
/// Adds to each type the retained size of its instances that have no instance of the same type
/// above them in the dominator tree. The walk keeps no stack: it goes up through `idom` and
/// resumes a node's children from `cursor`, while `active[t]` counts the instances of type `t` on
/// the path from the root.
fn retained_by_type(
graph: &HeapGraph,
tree: &Tree,
classes: &mut [ClassStat],
store: &Store,
meter: &mut Meter,
) -> Result<(), HeapError> {
let root = graph.root;
let mut cursor = store.scratch::<u32>(root as usize + 1, 0)?;
let mut active = vec![0u32; classes.len()];
cursor[root as usize] = tree.kids_start[root as usize];
let mut v = root;
meter.begin_loop()?;
loop {
meter.tick()?;
let next = cursor[v as usize];
if next < tree.kids_start[v as usize + 1] {
cursor[v as usize] = next + 1;
let child = tree.kids[next as usize];
let slot = graph.types[child as usize] as usize;
if active[slot] == 0 {
classes[slot].retained += tree.retained[child as usize];
}
active[slot] += 1;
cursor[child as usize] = tree.kids_start[child as usize];
v = child;
} else if v == root {
return Ok(());
} else {
active[graph.types[v as usize] as usize] -= 1;
v = tree.idom[v as usize];
}
}
}
fn slot_name(graph: &HeapGraph, slot: u32) -> String {
match graph.classes.get(slot as usize) {
Some(class) => display_name(&class.name),
None => graph.prim_type(slot).map_or_else(String::new, |ty| prim_array_name(ty).to_owned()),
}
}
fn prim_array_name(ty: BasicType) -> &'static str {
match ty {
BasicType::Object => "java.lang.Object[]",
BasicType::Boolean => "boolean[]",
BasicType::Char => "char[]",
BasicType::Float => "float[]",
BasicType::Double => "double[]",
BasicType::Byte => "byte[]",
BasicType::Short => "short[]",
BasicType::Int => "int[]",
BasicType::Long => "long[]",
}
}
+32
View File
@@ -0,0 +1,32 @@
//! Parents on shortest paths from the GC roots, by breadth-first search over strong edges.
use super::Meter;
use crate::column::{Column, NONE, Store};
use crate::error::HeapError;
use crate::files::GC_PARENT_FILE;
use crate::graph::HeapGraph;
pub(super) fn gc_parents(graph: &HeapGraph, store: &Store, meter: &mut Meter) -> Result<Column<u32>, HeapError> {
let root = graph.root;
let nodes = root as usize + 1;
let mut parent = store.column::<u32>(GC_PARENT_FILE, nodes, NONE)?;
let mut queue = store.scratch::<u32>(nodes, 0)?;
parent[root as usize] = root;
queue[0] = root;
let (mut head, mut tail) = (0, 1);
meter.begin_loop()?;
while head < tail {
meter.tick()?;
let v = queue[head];
head += 1;
for &w in graph.out_edges(v) {
meter.tick()?;
if parent[w as usize] == NONE {
parent[w as usize] = v;
queue[tail] = w;
tail += 1;
}
}
}
Ok(parent)
}
+209
View File
@@ -0,0 +1,209 @@
//! Shallow and retained sizes, and the dominator tree as child lists sorted by retained size.
use std::cmp::Reverse;
use super::Meter;
use crate::column::{Column, NONE, Store};
use crate::error::HeapError;
use crate::files::{KIDS_FILE, KIDS_START_FILE, RETAINED_FILE, SHALLOW8_FILE};
use crate::graph::{HeapGraph, accumulate};
use crate::progress::CHECK_EVERY;
use crate::shallow::ShallowModel;
/// The dominator tree with sizes; see [`crate::HeapIndex`] for each column.
pub(super) struct Tree {
pub(super) idom: Column<u32>,
pub(super) shallow8: Column<u32>,
pub(super) retained: Column<u64>,
pub(super) kids_start: Column<u32>,
pub(super) kids: Column<u32>,
}
/// `(shallow8, retained)` of every node, the root included. `preorder` is the DFS preorder of the
/// reachable nodes, from the root.
pub(super) fn sizes(
graph: &HeapGraph,
model: &ShallowModel,
idom: &[u32],
preorder: &[u32],
store: &Store,
meter: &mut Meter,
) -> Result<(Column<u32>, Column<u64>), HeapError> {
let nodes = idom.len();
let mut shallow8 = store.column::<u32>(SHALLOW8_FILE, nodes, 0)?;
let mut retained = store.column::<u64>(RETAINED_FILE, nodes, 0)?;
meter.begin_loop()?;
for v in 0..graph.root {
meter.tick()?;
let units = u32::try_from(model.size_of(graph, v).div_ceil(8)).unwrap_or(u32::MAX);
shallow8[v as usize] = units;
if idom[v as usize] != NONE {
retained[v as usize] = u64::from(units) * 8;
}
}
// A dominator is a DFS ancestor, so in reverse preorder every node is complete before it is
// added to its dominator.
meter.begin_loop()?;
for &v in preorder.iter().skip(1).rev() {
meter.tick()?;
let own = retained[v as usize];
retained[idom[v as usize] as usize] += own;
}
Ok((shallow8, retained))
}
/// `(kids_start, kids)`: the children of every reachable node in the dominator tree, as CSR.
pub(super) fn children(
idom: &[u32],
retained: &[u64],
root: u32,
store: &Store,
meter: &mut Meter,
) -> Result<(Column<u32>, Column<u32>), HeapError> {
let nodes = idom.len();
// Counted into `starts[d + 1]`; filled through `starts[d]` as a cursor, which leaves it at d's
// end until the column is shifted one slot.
let mut starts = store.column::<u32>(KIDS_START_FILE, nodes + 1, 0)?;
let mut longest = 0;
meter.begin_loop()?;
for (v, &d) in idom.iter().enumerate() {
meter.tick()?;
if d != NONE && v != root as usize {
let count = &mut starts[d as usize + 1];
*count += 1;
longest = longest.max(*count);
}
}
let total = accumulate(&mut starts, meter.token())?;
let mut kids = store.column::<u32>(KIDS_FILE, total as usize, 0)?;
meter.begin_loop()?;
for (v, &d) in idom.iter().enumerate() {
meter.tick()?;
if d != NONE && v != root as usize {
let cursor = &mut starts[d as usize];
kids[*cursor as usize] = v as u32;
*cursor += 1;
}
}
starts.copy_within(0..nodes, 1);
starts[0] = 0;
sort_children(&mut kids, &starts, retained, longest as usize, CHECK_EVERY as usize, store, meter)?;
Ok((starts, kids))
}
/// Sorts each child list by retained size descending, then by node. A list longer than `run` is
/// sorted in runs of `run` that are then merged, so no single step takes longer than one run.
fn sort_children(
kids: &mut [u32],
starts: &[u32],
retained: &[u64],
longest: usize,
run: usize,
store: &Store,
meter: &mut Meter,
) -> Result<(), HeapError> {
let key = |&k: &u32| (Reverse(retained[k as usize]), k);
let mut buffer = if longest > run { store.scratch::<u32>(longest, 0)? } else { Column::ram(0, 0) };
meter.begin_loop()?;
for v in 0..starts.len() - 1 {
meter.tick()?;
let list = &mut kids[starts[v] as usize..starts[v + 1] as usize];
if list.len() <= run {
list.sort_unstable_by_key(key);
meter.advance(list.len() as u64)?;
continue;
}
for chunk in list.chunks_mut(run) {
meter.begin_loop()?;
chunk.sort_unstable_by_key(key);
meter.advance(chunk.len() as u64)?;
}
let len = list.len();
let (mut from, mut to) = (list, &mut buffer[..len]);
let mut in_buffer = false;
let mut width = run;
while width < len {
meter.begin_loop()?;
for start in (0..len).step_by(2 * width) {
let mid = (start + width).min(len);
let end = (start + 2 * width).min(len);
merge(&from[start..mid], &from[mid..end], &mut to[start..end], &key, meter)?;
}
std::mem::swap(&mut from, &mut to);
in_buffer = !in_buffer;
width *= 2;
}
if in_buffer {
to.copy_from_slice(from);
}
}
Ok(())
}
fn merge<K: Ord>(
left: &[u32],
right: &[u32],
out: &mut [u32],
key: &impl Fn(&u32) -> K,
meter: &mut Meter,
) -> Result<(), HeapError> {
let (mut i, mut j) = (0, 0);
for slot in out.iter_mut() {
meter.tick()?;
let take_left = j == right.len() || (i < left.len() && key(&left[i]) <= key(&right[j]));
if take_left {
*slot = left[i];
i += 1;
} else {
*slot = right[j];
j += 1;
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use ide_core::CancellationToken;
use super::*;
use crate::error::HeapError;
use crate::progress::{HeapPhase, Progress};
use crate::testing::XorShift;
fn sorted_with_run(lists: &[Vec<u32>], retained: &[u64], run: usize) -> Vec<u32> {
let report = |_, _, _| Ok::<(), HeapError>(());
let cancel = CancellationToken::new();
let progress = Progress::new(&report, &cancel);
let mut meter = Meter::start(&progress, HeapPhase::Retained, 1).unwrap();
let mut starts = vec![0u32];
for list in lists {
starts.push(starts.last().unwrap() + list.len() as u32);
}
let mut kids: Vec<u32> = lists.concat();
let longest = lists.iter().map(Vec::len).max().unwrap_or(0);
sort_children(&mut kids, &starts, retained, longest, run, &Store::Ram, &mut meter).unwrap();
kids
}
#[test]
fn long_child_lists_are_merged_from_sorted_runs() {
let mut rng = XorShift::new(7);
let retained: Vec<u64> = (0..500).map(|_| u64::from(rng.below(40))).collect();
let mut nodes: Vec<u32> = (0..500).collect();
for i in (1..nodes.len()).rev() {
nodes.swap(i, rng.below(i as u32 + 1) as usize);
}
// Lists below, at and above the run length, and one that ends on a partial run.
let lists = vec![nodes[..3].to_vec(), nodes[3..7].to_vec(), nodes[7..400].to_vec(), nodes[400..].to_vec()];
for run in [1, 4, 7, 64, 1000] {
let mut expected = Vec::new();
for list in &lists {
let mut list = list.clone();
list.sort_by_key(|&k| (Reverse(retained[k as usize]), k));
expected.extend(list);
}
assert_eq!(sorted_with_run(&lists, &retained, run), expected, "run {run}");
}
}
}
+265
View File
@@ -0,0 +1,265 @@
use std::cell::Cell;
use std::collections::BTreeSet;
use std::path::PathBuf;
use super::*;
use crate::column::{Column, NONE};
use crate::files::INDEX_FILES;
use crate::hprof::{BasicType, HeapRootKind};
use crate::index::{ClassStat, HeapSummary};
use crate::shallow::SizeSource;
use crate::testing::{MiniHeap, sample};
const DUMP: &str = "dump.hprof";
const ROOT: Option<&str> = Some("java.lang.Object");
const NODE_FIELDS: [(&str, BasicType); 2] = [("next", BasicType::Object), ("v", BasicType::Long)];
fn no_progress(_: HeapPhase, _: u64, _: u64) -> Result<(), HeapError> {
Ok(())
}
fn in_memory() -> AnalyzeOptions {
AnalyzeOptions { in_memory: true, ..AnalyzeOptions::default() }
}
/// A temp dir holding `heap` as `dump.hprof`, the way a capture lays out its snapshot dir.
fn dump_dir(heap: &MiniHeap) -> (tempfile::TempDir, PathBuf) {
let dir = tempfile::tempdir().unwrap();
let dump = dir.path().join(DUMP);
heap.write_to(&dump);
(dir, dump)
}
fn analyze_with(heap: &MiniHeap, opts: AnalyzeOptions) -> HeapIndex {
let (dir, dump) = dump_dir(heap);
analyze(&dump, dir.path(), opts, &no_progress, &CancellationToken::new()).unwrap()
}
fn file_names(dir: &Path) -> BTreeSet<String> {
std::fs::read_dir(dir).unwrap().map(|e| e.unwrap().file_name().into_string().unwrap()).collect()
}
fn node(index: &HeapIndex, addr: u64) -> u32 {
index.ids.rank(addr).unwrap_or_else(|| panic!("{addr:#x} is not in the dump"))
}
fn shallow(index: &HeapIndex, addr: u64) -> u64 {
u64::from(index.shallow8[node(index, addr) as usize]) * 8
}
fn retained(index: &HeapIndex, addr: u64) -> u64 {
index.retained[node(index, addr) as usize]
}
fn idom(index: &HeapIndex, addr: u64) -> u32 {
index.idom[node(index, addr) as usize]
}
fn class<'a>(index: &'a HeapIndex, name: &str) -> &'a ClassStat {
let mut rows = index.classes.iter().filter(|c| c.name == name);
let row = rows.next().unwrap_or_else(|| panic!("no class {name}"));
assert!(rows.next().is_none(), "two classes named {name}");
row
}
fn stat(c: &ClassStat) -> (u64, u64, u64, u64) {
(c.count, c.shallow, c.retained, c.unreachable_count)
}
fn kids_of(index: &HeapIndex, v: u32) -> &[u32] {
&index.kids[index.kids_start[v as usize] as usize..index.kids_start[v as usize + 1] as usize]
}
const H: u64 = 0x1000;
const N1: u64 = 0x1100;
const N2: u64 = 0x1200;
const N3: u64 = 0x1300;
const L: u64 = 0x1400;
const R: u64 = 0x1500;
const W: u64 = 0x1600;
const N4: u64 = 0x1700;
#[test]
fn sample_retained_sizes() {
let index = analyze_with(&sample(), in_memory());
let shallows: Vec<u64> = [H, N1, L, W, R].iter().map(|&a| shallow(&index, a)).collect();
assert_eq!(shallows, [24, 24, 32, 24, 32]);
let retained_sizes: Vec<u64> = [L, N3, N1, N2, H, W].iter().map(|&a| retained(&index, a)).collect();
assert_eq!(retained_sizes, [32, 56, 24, 24, 72, 24]);
let root = index.root();
assert_eq!(
[idom(&index, N1), idom(&index, N2), idom(&index, N3), idom(&index, L)],
[node(&index, H), node(&index, H), root, node(&index, N3)]
);
assert_eq!((idom(&index, R), idom(&index, N4)), (NONE, NONE));
assert_eq!((retained(&index, R), retained(&index, N4)), (0, 0));
// Children by retained size descending, ties by node.
assert_eq!(kids_of(&index, node(&index, H)), [node(&index, N1), node(&index, N2)]);
let top = kids_of(&index, root);
assert_eq!(top[..2], [node(&index, H), node(&index, N3)]);
assert!(top.windows(2).all(|p| (index.retained[p[0] as usize], p[1]) > (index.retained[p[1] as usize], p[0])));
assert_eq!(index.retained[root as usize], top.iter().map(|&k| index.retained[k as usize]).sum::<u64>());
}
#[test]
fn sample_class_table() {
let index = analyze_with(&sample(), in_memory());
assert_eq!(stat(class(&index, "Node")), (3, 72, 104, 1));
assert_eq!(stat(class(&index, "Holder")), (1, 24, 72, 0));
assert_eq!(stat(class(&index, "long[]")), (1, 32, 32, 0));
assert_eq!(stat(class(&index, "java.lang.ref.WeakReference")), (1, 24, 24, 0));
let objects = class(&index, "java.lang.Object[]");
assert_eq!((objects.count, objects.unreachable_count), (0, 1));
// Indexed by type slot, primitive array slots included.
assert_eq!(index.classes.len(), index.class_info.len() + crate::graph::PRIM_SLOTS as usize);
let slot = index.types[node(&index, H) as usize] as usize;
assert_eq!(index.classes[slot].name, "Holder");
assert_eq!(index.class_info[slot].name, "Holder");
}
#[test]
fn sample_summary() {
let index = analyze_with(&sample(), in_memory());
let s = &index.summary;
assert_eq!(
(s.reachable_objects, s.reachable_bytes, s.unreachable_objects, s.unreachable_bytes, s.weak_edges_skipped),
(6, 152, 2, 56, 1)
);
assert_eq!(s.objects, 8);
assert_eq!(s.classes, 7);
assert_eq!(s.size_source, SizeSource::Layout);
}
#[test]
fn sample_gc_parents() {
let index = analyze_with(&sample(), in_memory());
let parent = |addr| index.gc_parent[node(&index, addr) as usize];
let root = index.root();
assert_eq!(index.gc_parent[root as usize], root);
assert_eq!([parent(H), parent(N3), parent(W)], [root; 3]);
assert_eq!([parent(N1), parent(L)], [node(&index, H), node(&index, N3)]);
assert_eq!((parent(R), parent(N4)), (NONE, NONE));
assert_eq!(index.roots[&node(&index, N3)], (HeapRootKind::JavaFrame, Some(1)));
}
#[test]
fn cycle_is_retained_once() {
let mut heap = MiniHeap::new();
heap.class("Node", ROOT, &NODE_FIELDS)
.object(0xA000, "Node", &[("next", 0xB000)])
.object(0xB000, "Node", &[("next", 0xA000)])
.root(HeapRootKind::JniGlobal, 0xA000);
let index = analyze_with(&heap, in_memory());
assert_eq!((retained(&index, 0xA000), retained(&index, 0xB000)), (48, 24));
assert_eq!(class(&index, "Node").retained, 48);
}
#[test]
fn deep_chain_retained_by_class() {
// A small stack proves no pass recurses along the chain.
std::thread::Builder::new()
.stack_size(256 << 10)
.spawn(|| {
const NODES: u64 = 1_000_000;
let addr = |i: u64| 0x10_0000 + i * 0x100;
let mut heap = MiniHeap::new();
heap.class("Node", ROOT, &NODE_FIELDS);
for i in 0..NODES {
let next = if i + 1 < NODES { addr(i + 1) } else { 0 };
heap.object(addr(i), "Node", &[("next", next)]);
}
heap.root(HeapRootKind::JniGlobal, addr(0));
let index = analyze_with(&heap, in_memory());
assert_eq!(retained(&index, addr(0)), 24_000_000);
assert_eq!(retained(&index, addr(NODES - 1)), 24);
assert_eq!(class(&index, "Node").retained, 24_000_000);
assert_eq!(index.summary.reachable_objects, NODES);
})
.unwrap()
.join()
.unwrap();
}
#[test]
fn histogram_sizes_flow_into_retained() {
let histogram = vec![HistogramEntry { name: "Node".to_owned(), instances: 3, bytes: 96 }];
let index = analyze_with(&sample(), AnalyzeOptions { histogram: Some(histogram), ..in_memory() });
assert_eq!(shallow(&index, N3), 32);
assert_eq!(retained(&index, N3), 64);
assert_eq!(index.summary.size_source, SizeSource::Jvm);
}
#[test]
fn vm_flags_of_an_unknown_jdk_set_the_layout() {
// Without compressed oops a Node is 12 + 8 + 8 = 28 → 32.
let opts = AnalyzeOptions { vm_flags: Some("-XX:MaxHeapSize=42949672960".to_owned()), ..in_memory() };
let index = analyze_with(&sample(), opts);
assert_eq!(shallow(&index, N4), 32);
}
#[test]
fn cancel_stops_analysis() {
let (dir, dump) = dump_dir(&sample());
let cancel = CancellationToken::new();
let phases = Cell::new(Vec::new());
let progress = |phase, _, _| {
let mut seen = phases.take();
seen.push(phase);
phases.set(seen);
if phase == HeapPhase::Dominators {
cancel.cancel();
}
Ok(())
};
let result = analyze(&dump, dir.path(), AnalyzeOptions::default(), &progress, &cancel);
assert!(matches!(result, Err(HeapError::Cancelled)), "{:?}", result.err());
assert_eq!(phases.take().last(), Some(&HeapPhase::Dominators), "stops at the first check");
// The dump the capture analyzes in place stays; every file the analysis wrote is gone.
assert_eq!(file_names(dir.path()), BTreeSet::from([DUMP.to_owned()]));
}
#[test]
fn progress_error_stops_analysis_and_removes_its_files() {
let (dir, dump) = dump_dir(&sample());
// The last report, once every index file is written.
let progress = |phase, done, _| match (phase, done) {
(HeapPhase::Saving, 1) => Err(HeapError::NotEnoughDiskSpace { needed: 2, available: 1 }),
_ => Ok(()),
};
let result = analyze(&dump, dir.path(), AnalyzeOptions::default(), &progress, &CancellationToken::new());
assert!(matches!(result, Err(HeapError::NotEnoughDiskSpace { .. })), "{:?}", result.err());
assert_eq!(file_names(dir.path()), BTreeSet::from([DUMP.to_owned()]));
}
#[test]
fn writes_index_files() {
let (dir, dump) = dump_dir(&sample());
let index = analyze(&dump, dir.path(), AnalyzeOptions::default(), &no_progress, &CancellationToken::new()).unwrap();
let names = file_names(dir.path());
for name in [
"types.u32",
"shallow8.u32",
"retained.u64",
"idom.u32",
"kids_start.u32",
"kids.u32",
"gc_parent.u32",
"offsets.u64",
"summary.json",
] {
assert!(names.contains(name), "{name} missing from {names:?}");
}
assert!(names.iter().any(|n| n.starts_with("ids.")), "{names:?}");
// Nothing but the index files and the dump: scratch arrays are gone.
let unexpected: Vec<_> = names.iter().filter(|n| *n != DUMP && !INDEX_FILES.contains(&n.as_str())).collect();
assert!(unexpected.is_empty(), "{unexpected:?}");
let summary: HeapSummary =
serde_json::from_slice(&std::fs::read(dir.path().join("summary.json")).unwrap()).unwrap();
assert_eq!(summary, index.summary);
drop(index);
let retained = Column::<u64>::open(&dir.path().join("retained.u64")).unwrap();
let in_ram = analyze_with(&sample(), in_memory());
assert_eq!(&*retained, &*in_ram.retained);
}
+306
View File
@@ -0,0 +1,306 @@
//! Fixed-width arrays that live in RAM or in memory-mapped files, so the analysis arrays of a
//! large heap do not count against the process's anonymous memory.
use std::fs::{File, OpenOptions};
use std::io;
use std::marker::PhantomData;
use std::mem::size_of;
use std::ops::{Deref, DerefMut};
use std::path::{Path, PathBuf};
use memmap2::{Mmap, MmapMut};
/// Index value meaning "no object" in every `u32` column.
pub const NONE: u32 = u32::MAX;
/// # Safety
/// Every bit pattern of `Self` is a valid value and `Self` has no padding, so a mapped file's
/// bytes can be viewed as `[Self]` and a file can be written from one.
pub unsafe trait Pod: Copy + 'static {}
unsafe impl Pod for u8 {}
unsafe impl Pod for u32 {}
unsafe impl Pod for u64 {}
enum Backing<T> {
Ram(Vec<T>),
Map(MmapMut),
ReadOnly(Mmap),
}
/// A fixed-length array of `T`. File contents use the machine's byte order: the arrays are a
/// cache for this machine and are never moved to another one.
pub struct Column<T: Pod> {
backing: Backing<T>,
/// Scratch file to delete on drop where it could not be unlinked while mapped.
delete_on_drop: Option<PathBuf>,
_marker: PhantomData<T>,
}
fn byte_len<T: Pod>(len: usize) -> io::Result<usize> {
len.checked_mul(size_of::<T>())
.filter(|&bytes| bytes <= isize::MAX as usize)
.ok_or_else(|| io::Error::new(io::ErrorKind::InvalidInput, "column is too long"))
}
fn is_zero<T: Pod>(value: &T) -> bool {
// SAFETY: `Pod` types have no padding, so all `size_of::<T>()` bytes are initialized.
let bytes = unsafe { std::slice::from_raw_parts((value as *const T).cast::<u8>(), size_of::<T>()) };
bytes.iter().all(|&b| b == 0)
}
impl<T: Pod> Column<T> {
fn new(backing: Backing<T>, delete_on_drop: Option<PathBuf>) -> Self {
Self { backing, delete_on_drop, _marker: PhantomData }
}
pub fn ram(len: usize, fill: T) -> Self {
Self::new(Backing::Ram(vec![fill; len]), None)
}
/// Creates (or truncates) `path` and maps it; the file stays after the column is dropped.
pub fn file(path: &Path, len: usize, fill: T) -> io::Result<Self> {
let bytes = byte_len::<T>(len)?;
let file = OpenOptions::new().read(true).write(true).create(true).truncate(true).open(path)?;
match Self::map_new(&file, len, bytes, fill) {
Ok(column) => Ok(column),
Err(err) => {
drop(file);
let _ = std::fs::remove_file(path);
Err(err)
}
}
}
/// A temporary column in `dir` that never outlives the process. On unix the file is unlinked
/// right after mapping, so even a crash leaves nothing behind; elsewhere (Windows cannot delete
/// a mapped file) it is deleted on drop.
pub fn scratch(dir: &Path, len: usize, fill: T) -> io::Result<Self> {
let bytes = byte_len::<T>(len)?;
if bytes == 0 {
return Ok(Self::ram(0, fill));
}
let path = dir.join(format!("scratch-{}.tmp", uuid::Uuid::new_v4().simple()));
let file = OpenOptions::new().read(true).write(true).create_new(true).open(&path)?;
let mut column = match Self::map_new(&file, len, bytes, fill) {
Ok(column) => column,
Err(err) => {
drop(file);
let _ = std::fs::remove_file(&path);
return Err(err);
}
};
// If the unlink fails the drop-time delete is the fallback, so the file is never left behind.
if !cfg!(unix) || std::fs::remove_file(&path).is_err() {
column.delete_on_drop = Some(path);
}
Ok(column)
}
/// Maps an existing file read-only; writing through the column panics.
pub fn open(path: &Path) -> io::Result<Self> {
let file = File::open(path)?;
let bytes = file.metadata()?.len();
if bytes % size_of::<T>() as u64 != 0 {
return Err(io::Error::new(
io::ErrorKind::InvalidData,
format!("{} is {bytes} bytes, not a multiple of {}", path.display(), size_of::<T>()),
));
}
if bytes == 0 {
return Ok(Self::ram(0, zero_value()));
}
// SAFETY: the file is private to the snapshot directory; nothing truncates it while mapped.
let map = unsafe { Mmap::map(&file)? };
Ok(Self::new(Backing::ReadOnly(map), None))
}
fn map_new(file: &File, len: usize, bytes: usize, fill: T) -> io::Result<Self> {
if bytes == 0 {
return Ok(Self::ram(0, fill));
}
file.set_len(bytes as u64)?;
// SAFETY: as in `open`; the file was just created by this process.
let map = unsafe { MmapMut::map_mut(file)? };
let mut column = Self::new(Backing::Map(map), None);
// A fresh file reads as zeros, and writing a zero fill would fault in every page.
if !is_zero(&fill) {
debug_assert_eq!(column.len(), len);
column.fill(fill);
}
Ok(column)
}
}
fn zero_value<T: Pod>() -> T {
// SAFETY: every bit pattern of a `Pod` type is a valid value.
unsafe { std::mem::zeroed() }
}
impl<T: Pod> Deref for Column<T> {
type Target = [T];
fn deref(&self) -> &[T] {
match &self.backing {
Backing::Ram(v) => v,
// SAFETY: maps are page-aligned, `T` is `Pod` and the length is a multiple of its width.
Backing::Map(m) => unsafe { std::slice::from_raw_parts(m.as_ptr().cast(), m.len() / size_of::<T>()) },
Backing::ReadOnly(m) => unsafe { std::slice::from_raw_parts(m.as_ptr().cast(), m.len() / size_of::<T>()) },
}
}
}
impl<T: Pod> DerefMut for Column<T> {
fn deref_mut(&mut self) -> &mut [T] {
match &mut self.backing {
Backing::Ram(v) => v,
// SAFETY: as in `deref`; `&mut self` makes the access exclusive.
Backing::Map(m) => unsafe {
std::slice::from_raw_parts_mut(m.as_mut_ptr().cast(), m.len() / size_of::<T>())
},
Backing::ReadOnly(_) => panic!("column opened from a file is read-only"),
}
}
}
impl<T: Pod> Drop for Column<T> {
fn drop(&mut self) {
if let Some(path) = self.delete_on_drop.take() {
// Unmap before deleting: Windows refuses to delete a mapped file.
self.backing = Backing::Ram(Vec::new());
let _ = std::fs::remove_file(path);
}
}
}
/// Where a computation puts its arrays: result columns are named files in `Dir`, temporary arrays
/// are scratch columns in the same directory.
pub enum Store {
Ram,
Dir(PathBuf),
}
impl Store {
/// `name` is the whole file name (`types.u32`); it is used as given.
pub fn column<T: Pod>(&self, name: &str, len: usize, fill: T) -> io::Result<Column<T>> {
match self {
Store::Ram => Ok(Column::ram(len, fill)),
Store::Dir(dir) => Column::file(&dir.join(name), len, fill),
}
}
pub fn scratch<T: Pod>(&self, len: usize, fill: T) -> io::Result<Column<T>> {
match self {
Store::Ram => Ok(Column::ram(len, fill)),
Store::Dir(dir) => Column::scratch(dir, len, fill),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn dir_is_empty(dir: &Path) -> bool {
std::fs::read_dir(dir).unwrap().next().is_none()
}
#[test]
fn file_column_round_trips() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("x.u32");
{
let mut col = Column::<u32>::file(&path, 4, 7).unwrap();
assert_eq!(&*col, &[7, 7, 7, 7]);
col.copy_from_slice(&[1, 2, 3, 4]);
}
let col = Column::<u32>::open(&path).unwrap();
assert_eq!(&*col, &[1, 2, 3, 4]);
}
#[test]
fn scratch_column_leaves_no_file() {
let dir = tempfile::tempdir().unwrap();
{
let mut col = Column::<u64>::scratch(dir.path(), 1024, 9).unwrap();
col[1023] = 5;
assert_eq!((col[0], col[1023]), (9, 5));
// Windows cannot delete a mapped file, so only unix unlinks while the map is alive.
#[cfg(unix)]
assert!(dir_is_empty(dir.path()));
}
assert!(dir_is_empty(dir.path()));
}
#[test]
fn zero_fill_and_non_zero_fill() {
let dir = tempfile::tempdir().unwrap();
let zeros = Column::<u64>::file(&dir.path().join("z.u64"), 3000, 0).unwrap();
assert!(zeros.iter().all(|&v| v == 0));
let ones = Column::<u8>::file(&dir.path().join("o.u8"), 10_000, 0xAB).unwrap();
assert!(ones.iter().all(|&v| v == 0xAB));
let ram = Column::<u32>::ram(5, NONE);
assert_eq!(&*ram, &[NONE; 5]);
}
#[test]
fn empty_columns_work_everywhere() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("e.u32");
assert_eq!(Column::<u32>::ram(0, 1).len(), 0);
assert_eq!(Column::<u32>::file(&path, 0, 1).unwrap().len(), 0);
assert_eq!(Column::<u32>::open(&path).unwrap().len(), 0);
assert_eq!(Column::<u32>::scratch(dir.path(), 0, 1).unwrap().len(), 0);
}
#[test]
fn open_rejects_a_length_that_is_not_a_multiple_of_the_width() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("odd.u32");
std::fs::write(&path, [0u8; 6]).unwrap();
let err = Column::<u32>::open(&path).err().unwrap();
assert_eq!(err.kind(), io::ErrorKind::InvalidData);
}
#[test]
fn file_rejects_a_byte_length_that_overflows() {
let dir = tempfile::tempdir().unwrap();
let err = Column::<u64>::file(&dir.path().join("big.u64"), usize::MAX, 0).err().unwrap();
assert_eq!(err.kind(), io::ErrorKind::InvalidInput);
assert!(dir_is_empty(dir.path()));
}
#[test]
#[should_panic(expected = "read-only")]
fn opened_columns_are_read_only() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("r.u32");
drop(Column::<u32>::file(&path, 2, 0).unwrap());
let mut col = Column::<u32>::open(&path).unwrap();
col[0] = 1;
}
#[test]
fn store_names_files_exactly_as_given() {
let dir = tempfile::tempdir().unwrap();
let store = Store::Dir(dir.path().to_path_buf());
let mut col = store.column::<u32>("types.u32", 3, 0).unwrap();
col[2] = 8;
drop(col);
let names: Vec<_> =
std::fs::read_dir(dir.path()).unwrap().map(|e| e.unwrap().file_name().into_string().unwrap()).collect();
assert_eq!(names, ["types.u32"]);
assert_eq!(&*Column::<u32>::open(&dir.path().join("types.u32")).unwrap(), &[0, 0, 8]);
}
#[test]
fn store_scratch_is_anonymous_and_ram_store_touches_no_disk() {
let dir = tempfile::tempdir().unwrap();
let store = Store::Dir(dir.path().to_path_buf());
drop(store.scratch::<u32>(16, 1).unwrap());
assert!(dir_is_empty(dir.path()));
let ram = Store::Ram.column::<u8>("ignored.u8", 4, 2).unwrap();
assert_eq!(&*ram, &[2; 4]);
assert_eq!(&*Store::Ram.scratch::<u8>(2, 3).unwrap(), &[3, 3]);
}
}
+59
View File
@@ -0,0 +1,59 @@
//! Class-by-class differences between two snapshots.
use std::collections::HashMap;
use serde::{Deserialize, Serialize};
use crate::index::HeapIndex;
/// One class name in two snapshots; a side without the name has zeros.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct HeapClassDiff {
pub name: String,
pub base_count: u64,
pub count: u64,
pub base_shallow: u64,
pub shallow: u64,
pub base_retained: u64,
pub retained: u64,
/// Several classes have this name on one side or both, so the sides cannot be matched: each
/// side shows the sum over its classes, and there is no difference to show.
pub ambiguous: bool,
}
/// Joined by class name, the only key two dumps share. By the change in retained size
/// descending (none for an ambiguous name), then by name.
pub fn compare(base: &HeapIndex, current: &HeapIndex) -> Vec<HeapClassDiff> {
let mut by_name: HashMap<String, HeapClassDiff> = HashMap::new();
for (index, is_base) in [(base, true), (current, false)] {
for row in index.class_rows() {
let diff = by_name.entry(row.name).or_insert_with_key(|name| HeapClassDiff {
name: name.clone(),
base_count: 0,
count: 0,
base_shallow: 0,
shallow: 0,
base_retained: 0,
retained: 0,
ambiguous: false,
});
let (count, shallow, retained) = if is_base {
(&mut diff.base_count, &mut diff.base_shallow, &mut diff.base_retained)
} else {
(&mut diff.count, &mut diff.shallow, &mut diff.retained)
};
*count = count.saturating_add(row.count);
*shallow = shallow.saturating_add(row.shallow);
*retained = retained.saturating_add(row.retained);
diff.ambiguous |= row.ambiguous;
}
}
let change = |d: &HeapClassDiff| if d.ambiguous { 0 } else { d.retained.abs_diff(d.base_retained) };
let mut diffs: Vec<HeapClassDiff> = by_name.into_values().collect();
diffs.sort_by(|a, b| change(b).cmp(&change(a)).then_with(|| a.name.cmp(&b.name)));
diffs
}
#[cfg(test)]
mod tests;
+78
View File
@@ -0,0 +1,78 @@
use super::*;
use crate::hprof::HeapRootKind;
use crate::test_index::analyzed;
use crate::testing::{MiniHeap, sample, sample_with_two_more_nodes};
fn diff<'a>(diffs: &'a [HeapClassDiff], name: &str) -> &'a HeapClassDiff {
let mut rows = diffs.iter().filter(|d| d.name == name);
let row = rows.next().unwrap_or_else(|| panic!("no diff for {name}"));
assert!(rows.next().is_none(), "two diffs for {name}");
row
}
fn sort_key(d: &HeapClassDiff) -> u64 {
if d.ambiguous { 0 } else { d.retained.abs_diff(d.base_retained) }
}
fn assert_sorted(diffs: &[HeapClassDiff]) {
assert!(diffs.windows(2).all(|p| (sort_key(&p[1]), &p[0].name) < (sort_key(&p[0]), &p[1].name)), "{diffs:#?}");
}
#[test]
fn compare_counts_by_class() {
let base = analyzed(&sample());
let mut current = sample_with_two_more_nodes();
current
.class("Extra", Some("java.lang.Object"), &[])
.object(0x4000, "Extra", &[])
.root(HeapRootKind::JniGlobal, 0x4000);
let diffs = compare(&base, &analyzed(&current));
let node = diff(&diffs, "Node");
assert_eq!((node.base_count, node.count), (3, 5));
assert_eq!(node.shallow - node.base_shallow, 48);
assert_eq!((node.base_retained, node.retained), (104, 152));
assert!(!node.ambiguous);
let extra = diff(&diffs, "Extra");
assert_eq!((extra.base_count, extra.base_shallow, extra.base_retained, extra.count), (0, 0, 0, 1));
assert_eq!(diffs[0].name, "Node");
let holder = diff(&diffs, "Holder");
assert_eq!((holder.base_count, holder.count, holder.base_retained, holder.retained), (1, 1, 72, 72));
assert_sorted(&diffs);
// A class only in the base snapshot.
let gone = compare(&analyzed(&current), &base);
let extra = diff(&gone, "Extra");
assert_eq!((extra.base_count, extra.count, extra.shallow, extra.retained), (1, 0, 0, 0));
assert_sorted(&gone);
}
fn two_foos(extra: bool) -> MiniHeap {
let mut heap = sample();
heap.class("Foo", Some("java.lang.Object"), &[])
.object(0x3000, "Foo", &[])
.class("Foo", Some("java.lang.Object"), &[])
.object(0x3100, "Foo", &[]);
if extra {
heap.object(0x3200, "Foo", &[]);
}
for addr in [0x3000, 0x3100, 0x3200] {
heap.root(HeapRootKind::JniGlobal, addr);
}
heap
}
#[test]
fn compare_gives_no_diff_for_a_name_of_several_classes() {
let diffs = compare(&analyzed(&two_foos(false)), &analyzed(&two_foos(true)));
let foo = diff(&diffs, "Foo");
assert!(foo.ambiguous);
assert_eq!((foo.base_count, foo.count), (2, 3));
assert!(diffs.iter().filter(|d| d.name != "Foo").all(|d| !d.ambiguous));
assert_sorted(&diffs);
// Ambiguous on one side only is still ambiguous.
let diffs = compare(&analyzed(&sample()), &analyzed(&two_foos(false)));
let foo = diff(&diffs, "Foo");
assert!(foo.ambiguous);
assert_eq!((foo.base_count, foo.count), (0, 2));
}
+137
View File
@@ -0,0 +1,137 @@
//! Disk-space estimates and the per-snapshot directories under the app's cache.
//!
//! A snapshot directory holds the index columns and, for a capture, the dump itself. Its `owner`
//! file names the host process that created it, so a host that starts later can remove the
//! directories of hosts that died without cleaning up.
use std::fs;
use std::io::{self, Read};
use std::path::{Path, PathBuf};
use std::time::{Duration, SystemTime};
/// Space to leave free on top of an estimate.
pub const DISK_MARGIN: u64 = 1 << 30;
/// hprof ≈ `bytes + 16 × instances`; index columns and scratch arrays ≈ `90 × instances`.
const CAPTURE_BYTES_PER_INSTANCE: u64 = 106;
const INDEX_BYTES_PER_OBJECT: u64 = 90;
const OWNER_FILE: &str = "owner";
const TMP_SUFFIX: &str = ".tmp";
/// A directory still being created by a live host has no `owner` for a moment; one that stays
/// unfinished for longer than this was abandoned.
const TMP_GRACE: Duration = Duration::from_secs(60 * 60);
/// Bytes of free space that a dump of `heap_bytes` bytes holding `instances` instances needs,
/// including its index.
pub fn capture_need(heap_bytes: u64, instances: u64) -> u64 {
heap_bytes.saturating_add(CAPTURE_BYTES_PER_INSTANCE.saturating_mul(instances)).saturating_add(DISK_MARGIN)
}
/// Bytes of free space that indexing a dump of `objects` objects needs.
pub fn index_need(objects: u64) -> u64 {
INDEX_BYTES_PER_OBJECT.saturating_mul(objects).saturating_add(DISK_MARGIN)
}
/// Bytes available to this process on the volume holding `dir`; `None` where it is not checked
/// (not unix) or cannot be read, so callers go ahead rather than refuse.
pub fn free_bytes(dir: &Path) -> Option<u64> {
#[cfg(unix)]
{
use std::os::unix::ffi::OsStrExt;
let path = std::ffi::CString::new(dir.as_os_str().as_bytes()).ok()?;
// SAFETY: `path` is NUL-terminated and `stat` is only read after `statvfs` filled it in.
let stat = unsafe {
let mut stat = std::mem::MaybeUninit::<libc::statvfs>::zeroed();
if libc::statvfs(path.as_ptr(), stat.as_mut_ptr()) != 0 {
return None;
}
stat.assume_init()
};
// The field widths differ between unix targets.
#[allow(clippy::unnecessary_cast)]
Some((stat.f_bavail as u64).saturating_mul(stat.f_frsize as u64))
}
#[cfg(not(unix))]
{
let _ = dir;
None
}
}
/// `<local_data_dir>/cache/heap-snapshots`.
pub fn snapshots_root(local_data_dir: &Path) -> PathBuf {
local_data_dir.join("cache").join("heap-snapshots")
}
/// Creates `<root>/<uuid>/` holding an `owner` file with this process's pid. The directory is
/// built as `<uuid>.tmp` and renamed, so another host scanning `root` never sees it without its
/// owner and removes it as abandoned.
pub fn create_snapshot_dir(root: &Path) -> io::Result<PathBuf> {
fs::create_dir_all(root)?;
let name = uuid::Uuid::new_v4().to_string();
let building = root.join(format!("{name}{TMP_SUFFIX}"));
let dir = root.join(&name);
create_private_dir(&building)?;
let published =
fs::write(building.join(OWNER_FILE), std::process::id().to_string()).and_then(|()| fs::rename(&building, &dir));
if let Err(err) = published {
let _ = fs::remove_dir_all(&building);
return Err(err);
}
Ok(dir)
}
/// A dump holds whatever was in the JVM's memory, passwords and tokens included, so on unix only
/// the owner may enter the directory. Windows keeps the ACL inherited from the user's data dir.
fn create_private_dir(path: &Path) -> io::Result<()> {
#[cfg(unix)]
{
use std::os::unix::fs::DirBuilderExt;
fs::DirBuilder::new().mode(0o700).create(path)
}
#[cfg(not(unix))]
{
fs::DirBuilder::new().create(path)
}
}
/// Removes the directories under `root` whose owner file is missing, unreadable or names a pid
/// for which `alive` is false, and unfinished `*.tmp` ones older than an hour. Only directories
/// directly under `root` are touched; a symlink there is left alone.
pub fn remove_stale(root: &Path, alive: impl Fn(u32) -> bool) {
remove_stale_at(root, alive, SystemTime::now());
}
fn remove_stale_at(root: &Path, alive: impl Fn(u32) -> bool, now: SystemTime) {
let Ok(entries) = fs::read_dir(root) else { return };
for entry in entries.flatten() {
let path = entry.path();
// `symlink_metadata` does not follow: a link out of `root` is not a directory here.
let Ok(meta) = fs::symlink_metadata(&path) else { continue };
if !meta.file_type().is_dir() {
continue;
}
let stale = if path.extension().is_some_and(|ext| ext == "tmp") {
meta.modified()
.ok()
.and_then(|modified| now.duration_since(modified).ok())
.is_some_and(|age| age >= TMP_GRACE)
} else {
read_owner(&path).is_none_or(|pid| !alive(pid))
};
if stale && let Err(err) = fs::remove_dir_all(&path) {
tracing::warn!(dir = %path.display(), %err, "could not remove a stale heap snapshot directory");
}
}
}
fn read_owner(dir: &Path) -> Option<u32> {
let mut text = String::new();
fs::File::open(dir.join(OWNER_FILE)).ok()?.take(32).read_to_string(&mut text).ok()?;
text.trim().parse().ok()
}
#[cfg(test)]
mod tests;
+137
View File
@@ -0,0 +1,137 @@
use std::time::{Duration, SystemTime};
use super::*;
fn names(dir: &Path) -> Vec<String> {
let mut names: Vec<_> =
std::fs::read_dir(dir).unwrap().map(|e| e.unwrap().file_name().into_string().unwrap()).collect();
names.sort();
names
}
fn dir_with_owner(root: &Path, name: &str, owner: Option<&str>) {
let dir = root.join(name);
std::fs::create_dir(&dir).unwrap();
std::fs::write(dir.join("dump.hprof"), b"x").unwrap();
if let Some(owner) = owner {
std::fs::write(dir.join(OWNER_FILE), owner).unwrap();
}
}
#[test]
fn estimates() {
assert_eq!(capture_need(664_018_520, 20_087_506), 664_018_520 + 106 * 20_087_506 + (1 << 30));
assert_eq!(index_need(10), 900 + (1 << 30));
}
#[test]
fn estimates_saturate_instead_of_wrapping() {
assert_eq!(capture_need(u64::MAX, u64::MAX), u64::MAX);
assert_eq!(index_need(u64::MAX), u64::MAX);
}
#[cfg(unix)]
#[test]
fn not_enough_space_for_estimate() {
let dir = tempfile::tempdir().unwrap();
let free = free_bytes(dir.path()).expect("statvfs works on a temp dir");
assert!(free > 0);
assert!(capture_need(u64::MAX / 4, 0) > free);
}
#[cfg(unix)]
#[test]
fn free_bytes_of_a_missing_dir_is_unknown() {
let dir = tempfile::tempdir().unwrap();
assert_eq!(free_bytes(&dir.path().join("missing")), None);
}
#[cfg(not(unix))]
#[test]
fn free_space_is_not_checked_off_unix() {
assert_eq!(free_bytes(Path::new(".")), None);
}
#[test]
fn snapshots_root_is_under_the_cache() {
let local = Path::new("local");
assert_eq!(snapshots_root(local), local.join("cache").join("heap-snapshots"));
}
#[test]
fn snapshot_dir_records_its_owner_and_leaves_no_tmp() {
let root = tempfile::tempdir().unwrap();
let missing_parent = root.path().join("cache").join("heap-snapshots");
let dir = create_snapshot_dir(&missing_parent).unwrap();
assert_eq!(dir.parent(), Some(missing_parent.as_path()));
assert!(uuid::Uuid::parse_str(dir.file_name().unwrap().to_str().unwrap()).is_ok());
assert_eq!(std::fs::read_to_string(dir.join(OWNER_FILE)).unwrap(), std::process::id().to_string());
assert_eq!(names(&missing_parent), [dir.file_name().unwrap().to_str().unwrap()]);
assert_ne!(create_snapshot_dir(&missing_parent).unwrap(), dir);
}
#[cfg(unix)]
#[test]
fn snapshot_dir_is_private_to_the_owner() {
use std::os::unix::fs::PermissionsExt;
let root = tempfile::tempdir().unwrap();
let dir = create_snapshot_dir(root.path()).unwrap();
assert_eq!(std::fs::metadata(&dir).unwrap().permissions().mode() & 0o777, 0o700);
}
#[test]
fn stale_dirs_removed_on_start() {
let root = tempfile::tempdir().unwrap();
dir_with_owner(root.path(), "a", Some("999999"));
dir_with_owner(root.path(), "b", Some(&std::process::id().to_string()));
dir_with_owner(root.path(), "c", None);
remove_stale(root.path(), |pid| pid == std::process::id());
assert_eq!(names(root.path()), ["b"]);
}
#[test]
fn owner_that_is_not_a_pid_is_stale() {
let root = tempfile::tempdir().unwrap();
dir_with_owner(root.path(), "empty", Some(""));
dir_with_owner(root.path(), "word", Some("not-a-pid"));
dir_with_owner(root.path(), "negative", Some("-1"));
dir_with_owner(root.path(), "padded", Some(&format!(" {}\n", std::process::id())));
remove_stale(root.path(), |pid| pid == std::process::id());
assert_eq!(names(root.path()), ["padded"]);
}
#[test]
fn stale_scan_tolerates_a_missing_root_and_leaves_loose_files_alone() {
let root = tempfile::tempdir().unwrap();
remove_stale(&root.path().join("missing"), |_| false);
std::fs::write(root.path().join("notes.txt"), b"keep").unwrap();
remove_stale(root.path(), |_| false);
assert_eq!(names(root.path()), ["notes.txt"]);
}
#[test]
fn unfinished_dir_is_left_alone_for_an_hour() {
let root = tempfile::tempdir().unwrap();
// Another host may be between `create_dir` and the rename of its own snapshot dir.
dir_with_owner(root.path(), "young.tmp", None);
let now = SystemTime::now();
remove_stale_at(root.path(), |_| false, now);
assert_eq!(names(root.path()), ["young.tmp"]);
remove_stale_at(root.path(), |_| false, now + TMP_GRACE - Duration::from_secs(60));
assert_eq!(names(root.path()), ["young.tmp"]);
remove_stale_at(root.path(), |_| false, now + TMP_GRACE + Duration::from_secs(60));
assert!(names(root.path()).is_empty());
}
#[cfg(unix)]
#[test]
fn stale_scan_never_follows_a_symlink_out_of_the_root() {
let outside = tempfile::tempdir().unwrap();
std::fs::write(outside.path().join("precious"), b"x").unwrap();
let root = tempfile::tempdir().unwrap();
std::os::unix::fs::symlink(outside.path(), root.path().join("link")).unwrap();
remove_stale(root.path(), |_| false);
remove_stale_at(root.path(), |_| false, SystemTime::now() + 2 * TMP_GRACE);
assert_eq!(names(outside.path()), ["precious"]);
}
+247
View File
@@ -0,0 +1,247 @@
//! Dominator tree of the object graph, by Lengauer–Tarjan with path compression and no recursion:
//! the DFS "stack" is the chain of DFS parents, and `eval` reverses the links of the path it
//! compresses instead of keeping a stack.
//!
//! Between the DFS and the final translation every array is indexed by DFS number, so the root is
//! 0 and a vertex's DFS ancestors all have smaller numbers.
use ide_core::CancellationToken;
use crate::column::{Column, NONE, Store};
use crate::error::HeapError;
use crate::files::IDOM_FILE;
use crate::graph::accumulate;
use crate::progress::{CHECK_EVERY, HeapPhase, Progress};
const PHASE: HeapPhase = HeapPhase::Dominators;
pub struct Dominators {
/// Immediate dominator of each node, saved as `idom.u32` in the store; [`NONE`] for
/// unreachable nodes, `root` for `root`.
pub idom: Column<u32>,
/// Nodes in DFS preorder from `root`, `reachable` entries.
pub preorder: Column<u32>,
pub reachable: u32,
}
/// The dominators of the graph whose node `v` has targets `out[out_start[v]..out_start[v + 1]]`,
/// from `root` (which must be a node). `progress` gets `Dominators` from `(0, total)` to
/// `(total, total)`; an `Err` from it stops the computation with that error.
pub fn dominators(
out_start: &[u32],
out: &[u32],
root: u32,
store: &Store,
progress: &dyn Fn(HeapPhase, u64, u64) -> Result<(), HeapError>,
cancel: &CancellationToken,
) -> Result<Dominators, HeapError> {
compute(out_start, out, root, store, progress, cancel).map(|(dominators, _)| dominators)
}
/// [`dominators`], plus the number of steps of work it took.
fn compute(
out_start: &[u32],
out: &[u32],
root: u32,
store: &Store,
progress: &dyn Fn(HeapPhase, u64, u64) -> Result<(), HeapError>,
cancel: &CancellationToken,
) -> Result<(Dominators, u64), HeapError> {
let nodes = out_start.len().saturating_sub(1);
if nodes >= NONE as usize {
return Err(HeapError::TooLarge);
}
assert!((root as usize) < nodes, "root {root} is not one of the {nodes} nodes");
// Three walks of at most `nodes` vertices each: DFS, semidominators, immediate dominators.
let span = nodes as u64;
let mut work = Work::new(Progress::new(progress, cancel), 3 * span);
work.report(0)?;
// DFS: `pre` maps nodes to DFS numbers and `vertex` back; `cursor[d]` is the next edge of
// DFS vertex d to follow.
let mut pre = store.scratch::<u32>(nodes, NONE)?;
let mut vertex = store.scratch::<u32>(nodes, 0)?;
let mut parent = store.scratch::<u32>(nodes, 0)?;
let mut cursor = store.scratch::<u32>(nodes, 0)?;
pre[root as usize] = 0;
vertex[0] = root;
cursor[0] = out_start[root as usize];
let (mut reachable, mut d) = (1u32, 0u32);
loop {
work.tick()?;
let edge = cursor[d as usize];
if edge < out_start[vertex[d as usize] as usize + 1] {
cursor[d as usize] = edge + 1;
let w = out[edge as usize] as usize;
if pre[w] == NONE {
let next = reachable as usize;
pre[w] = reachable;
vertex[next] = w as u32;
parent[next] = d;
cursor[next] = out_start[w];
d = reachable;
reachable += 1;
work.done = u64::from(reachable);
}
} else if d == 0 {
break;
} else {
d = parent[d as usize];
}
}
let r = reachable as usize;
work.report(span)?;
// Predecessors by DFS number, as CSR: counted into `starts[w + 1]`, then filled through
// `starts[w]` as a cursor, which leaves it at w's end until the column is shifted one slot.
let mut semi = cursor;
let mut label = store.scratch::<u32>(r, 0)?;
let mut starts = store.scratch::<u32>(r + 1, 0)?;
for d in 0..r {
work.tick()?;
semi[d] = d as u32;
label[d] = d as u32;
for &w in targets(out_start, out, vertex[d]) {
work.tick()?;
starts[pre[w as usize] as usize + 1] += 1;
}
}
let edges = accumulate(&mut starts, cancel)?;
let mut preds = store.scratch::<u32>(edges as usize, 0)?;
for d in 0..r {
work.tick()?;
for &w in targets(out_start, out, vertex[d]) {
work.tick()?;
let slot = &mut starts[pre[w as usize] as usize];
preds[*slot as usize] = d as u32;
*slot += 1;
}
}
starts.copy_within(0..r, 1);
starts[0] = 0;
drop(pre);
work.report(span)?;
// Semidominators in reverse preorder, linking each finished vertex to its parent. Vertex w
// waits in its semidominator's bucket until a child of that semidominator is linked; then
// `eval` gives w's idom, or a vertex whose idom is also w's, fixed up in preorder below.
// `idom[v]` is the next vertex in v's bucket until v leaves the bucket.
let mut ancestor = store.scratch::<u32>(r, NONE)?;
let mut bucket = store.scratch::<u32>(r, NONE)?;
let mut idom = store.scratch::<u32>(r, NONE)?;
for w in (1..r).rev() {
work.done = span + (r - w) as u64;
work.tick()?;
let mut s = semi[w];
for &v in &preds[starts[w] as usize..starts[w + 1] as usize] {
work.tick()?;
let u = eval(v, &mut ancestor, &mut label, &semi, &mut work)?;
s = s.min(semi[u as usize]);
}
semi[w] = s;
idom[w] = bucket[s as usize];
bucket[s as usize] = w as u32;
let p = parent[w];
ancestor[w] = p;
let mut v = std::mem::replace(&mut bucket[p as usize], NONE);
while v != NONE {
work.tick()?;
let next = idom[v as usize];
let u = eval(v, &mut ancestor, &mut label, &semi, &mut work)?;
idom[v as usize] = if semi[u as usize] < semi[v as usize] { u } else { p };
v = next;
}
}
drop((label, starts, preds, ancestor, bucket, parent));
work.report(2 * span)?;
// Immediate dominators, translated back to nodes; in preorder, so a deferred vertex's
// stand-in already has its final idom.
let mut node_idom = store.column::<u32>(IDOM_FILE, nodes, NONE)?;
let mut preorder = store.scratch::<u32>(r, 0)?;
idom[0] = 0;
node_idom[root as usize] = root;
preorder[0] = root;
for w in 1..r {
work.done = 2 * span + w as u64;
work.tick()?;
if idom[w] != semi[w] {
idom[w] = idom[idom[w] as usize];
}
node_idom[vertex[w] as usize] = vertex[idom[w] as usize];
preorder[w] = vertex[w];
}
work.report(3 * span)?;
Ok((Dominators { idom: node_idom, preorder, reachable }, work.steps))
}
fn targets<'a>(out_start: &[u32], out: &'a [u32], v: u32) -> &'a [u32] {
let v = v as usize;
&out[out_start[v] as usize..out_start[v + 1] as usize]
}
/// Counts steps of work and reports progress, which also looks at the cancellation token, at
/// every phase boundary and after every [`CHECK_EVERY`] steps, whichever loop they are in.
struct Work<'a> {
progress: Progress<'a>,
total: u64,
/// Progress to report at the next check; the loops keep it current.
done: u64,
steps: u64,
next_report: u64,
}
impl<'a> Work<'a> {
fn new(progress: Progress<'a>, total: u64) -> Self {
Self { progress, total, done: 0, steps: 0, next_report: CHECK_EVERY }
}
fn report(&mut self, done: u64) -> Result<(), HeapError> {
self.done = done;
self.next_report = self.steps + CHECK_EVERY;
self.progress.report(PHASE, done, self.total)
}
#[inline]
fn tick(&mut self) -> Result<(), HeapError> {
self.steps += 1;
if self.steps < self.next_report { Ok(()) } else { self.report(self.done) }
}
}
/// The vertex of smallest semidominator on the forest path from `v` up to, not including, its
/// tree's root (`v` itself when it is a root), compressing that path so the root becomes the
/// ancestor of every vertex on it. The walk up reverses the `ancestor` links so the walk down can
/// fold labels nearest-root first, as the recursive compress does, without a stack.
fn eval(v: u32, ancestor: &mut [u32], label: &mut [u32], semi: &[u32], work: &mut Work) -> Result<u32, HeapError> {
let at = |x: u32| x as usize;
if ancestor[at(v)] == NONE {
return Ok(v);
}
let (mut x, mut below) = (v, NONE);
while ancestor[at(ancestor[at(x)])] != NONE {
work.tick()?;
let up = ancestor[at(x)];
ancestor[at(x)] = below;
below = x;
x = up;
}
// `x` is the root's child, already compressed; `below` is the vertex under it.
let root = ancestor[at(x)];
let mut best = label[at(x)];
while below != NONE {
work.tick()?;
let y = at(below);
if semi[at(best)] < semi[at(label[y])] {
label[y] = best;
} else {
best = label[y];
}
below = ancestor[y];
ancestor[y] = root;
}
Ok(label[at(v)])
}
#[cfg(test)]
mod tests;
+348
View File
@@ -0,0 +1,348 @@
use std::cell::RefCell;
use std::time::Instant;
use proptest::collection::vec;
use proptest::prelude::*;
use super::*;
use crate::NONE;
use crate::graph::build_graph;
use crate::testing::{XorShift, sample};
fn no_progress(_: HeapPhase, _: u64, _: u64) -> Result<(), HeapError> {
Ok(())
}
/// CSR of `edges` over `n` nodes, targets in edge order.
fn csr(n: usize, edges: &[(u32, u32)]) -> (Vec<u32>, Vec<u32>) {
let mut start = vec![0u32; n + 1];
for &(from, _) in edges {
start[from as usize + 1] += 1;
}
for v in 0..n {
start[v + 1] += start[v];
}
let mut cursor = start.clone();
let mut out = vec![0u32; edges.len()];
for &(from, to) in edges {
out[cursor[from as usize] as usize] = to;
cursor[from as usize] += 1;
}
(start, out)
}
fn run(n: usize, edges: &[(u32, u32)], root: u32) -> Dominators {
let (start, out) = csr(n, edges);
dominators(&start, &out, root, &Store::Ram, &no_progress, &CancellationToken::new()).unwrap()
}
fn preds(n: usize, edges: &[(u32, u32)]) -> Vec<Vec<u32>> {
let mut preds = vec![Vec::new(); n];
for &(from, to) in edges {
preds[to as usize].push(from);
}
preds
}
/// Immediate dominators from iterated `Dom` sets (one `u64` bitset per node, `n <= 64`).
fn dataflow_idom(n: usize, edges: &[(u32, u32)], root: u32) -> Vec<u32> {
let (start, out) = csr(n, edges);
let mut reachable = vec![false; n];
let mut todo = vec![root];
reachable[root as usize] = true;
while let Some(v) = todo.pop() {
for &w in &out[start[v as usize] as usize..start[v as usize + 1] as usize] {
if !std::mem::replace(&mut reachable[w as usize], true) {
todo.push(w);
}
}
}
let preds = preds(n, edges);
let mut dom = vec![u64::MAX; n];
dom[root as usize] = 1 << root;
let mut changed = true;
while changed {
changed = false;
for v in (0..n).filter(|&v| reachable[v] && v != root as usize) {
let meet =
preds[v].iter().filter(|&&p| reachable[p as usize]).fold(u64::MAX, |acc, &p| acc & dom[p as usize]);
let next = meet | 1 << v;
if next != dom[v] {
dom[v] = next;
changed = true;
}
}
}
(0..n)
.map(|v| {
if !reachable[v] {
NONE
} else if v == root as usize {
root
} else {
// The strict dominator dominated by all the others has the most dominators itself.
let strict = dom[v] & !(1 << v);
(0..n as u32).filter(|&d| strict & 1 << d != 0).max_by_key(|&d| dom[d as usize].count_ones()).unwrap()
}
})
.collect()
}
/// Cooper–Harvey–Kennedy iteration over reverse postorder from an explicit-stack DFS.
fn chk_idom(n: usize, edges: &[(u32, u32)], root: u32) -> Vec<u32> {
let (start, out) = csr(n, edges);
let mut post_number = vec![NONE; n];
let mut postorder = Vec::new();
let mut seen = vec![false; n];
let mut stack = vec![(root, start[root as usize])];
seen[root as usize] = true;
while let Some(&mut (v, ref mut next)) = stack.last_mut() {
if *next < start[v as usize + 1] {
let w = out[*next as usize];
*next += 1;
if !std::mem::replace(&mut seen[w as usize], true) {
stack.push((w, start[w as usize]));
}
} else {
post_number[v as usize] = postorder.len() as u32;
postorder.push(v);
stack.pop();
}
}
let preds = preds(n, edges);
let mut idom = vec![NONE; n];
idom[root as usize] = root;
let intersect = |idom: &[u32], mut a: u32, mut b: u32| {
while a != b {
while post_number[a as usize] < post_number[b as usize] {
a = idom[a as usize];
}
while post_number[b as usize] < post_number[a as usize] {
b = idom[b as usize];
}
}
a
};
let mut changed = true;
while changed {
changed = false;
for &v in postorder.iter().rev().filter(|&&v| v != root) {
let mut new = NONE;
for &p in &preds[v as usize] {
if idom[p as usize] != NONE {
new = if new == NONE { p } else { intersect(&idom, p, new) };
}
}
if new != idom[v as usize] {
idom[v as usize] = new;
changed = true;
}
}
}
idom
}
#[test]
fn diamond() {
let d = run(5, &[(0, 1), (0, 2), (1, 3), (2, 3), (3, 4)], 0);
assert_eq!(&*d.idom, &[0, 0, 0, 0, 3]);
assert_eq!(d.reachable, 5);
assert_eq!(&*d.preorder, &[0, 1, 3, 4, 2]);
}
#[test]
fn unreachable_is_none() {
let d = run(6, &[(0, 1), (0, 2), (1, 3), (2, 3), (3, 4), (5, 3)], 0);
assert_eq!(&*d.idom, &[0, 0, 0, 0, 3, NONE]);
assert_eq!(d.reachable, 5);
assert_eq!(d.preorder.len(), 5);
}
#[test]
fn cycle_back_to_root() {
let d = run(3, &[(0, 1), (1, 2), (2, 1), (2, 0)], 0);
assert_eq!(&*d.idom, &[0, 0, 1]);
}
#[test]
fn root_need_not_be_node_zero() {
let d = run(4, &[(3, 1), (1, 0), (3, 2), (2, 0), (0, 3)], 3);
assert_eq!(&*d.idom, &[3, 3, 3, 3]);
assert_eq!(d.preorder[0], 3);
}
#[test]
fn sample_heap_dominators() {
let bytes = sample().bytes();
let g = build_graph(&bytes, &Store::Ram, &no_progress, &CancellationToken::new()).unwrap();
let d = dominators(&g.out_start, &g.out, g.root, &Store::Ram, &no_progress, &CancellationToken::new()).unwrap();
let node = |id: u64| g.ids.rank(id).unwrap();
let idom_of = |id: u64| d.idom[node(id) as usize];
assert_eq!(idom_of(0x1100), node(0x1000), "N1 → H");
assert_eq!(idom_of(0x1200), node(0x1000), "N2 → H");
assert_eq!(idom_of(0x1300), g.root, "N3 → root");
assert_eq!(idom_of(0x1400), node(0x1300), "L → N3");
assert_eq!(idom_of(0x1500), NONE, "R is unreachable");
assert_eq!(idom_of(0x1700), NONE, "N4 only through the weak referent");
assert_eq!(d.idom[g.root as usize], g.root);
}
#[test]
fn idom_is_written_to_the_store() {
let dir = tempfile::tempdir().unwrap();
let (start, out) = csr(3, &[(0, 1), (1, 2)]);
let store = Store::Dir(dir.path().to_path_buf());
let d = dominators(&start, &out, 0, &store, &no_progress, &CancellationToken::new()).unwrap();
drop(d);
assert_eq!(&*Column::<u32>::open(&dir.path().join("idom.u32")).unwrap(), &[0, 0, 1]);
let names: Vec<_> = std::fs::read_dir(dir.path()).unwrap().map(|e| e.unwrap().file_name()).collect();
assert_eq!(names, ["idom.u32"], "scratch columns leave no files");
}
#[test]
fn reports_dominators_progress_and_stops_on_cancel() {
let (start, out) = csr(3, &[(0, 1), (1, 2)]);
let calls = RefCell::new(Vec::new());
let record = |phase, done, total| {
calls.borrow_mut().push((phase, done, total));
Ok(())
};
dominators(&start, &out, 0, &Store::Ram, &record, &CancellationToken::new()).unwrap();
let calls = calls.into_inner();
let total = calls[0].2;
assert_eq!(calls.first(), Some(&(HeapPhase::Dominators, 0, total)));
assert_eq!(calls.last(), Some(&(HeapPhase::Dominators, total, total)));
assert!(calls.windows(2).all(|w| w[0].1 <= w[1].1), "progress never goes back: {calls:?}");
let cancel = CancellationToken::new();
let cancel_on_first = |_, _, _| {
cancel.cancel();
Ok(())
};
let result = dominators(&start, &out, 0, &Store::Ram, &cancel_on_first, &cancel);
assert!(matches!(result, Err(HeapError::Cancelled)));
let refuse = |_, _, _| Err(HeapError::TooLarge);
let result = dominators(&start, &out, 0, &Store::Ram, &refuse, &CancellationToken::new());
assert!(matches!(result, Err(HeapError::TooLarge)));
}
#[test]
fn cancel_during_idom_phase() {
// Long enough that the last phase alone passes a check interval.
const NODES: u32 = (CHECK_EVERY + CHECK_EVERY / 2) as u32;
let edges: Vec<_> = (1..NODES).map(|v| (v - 1, v)).collect();
let (start, out) = csr(NODES as usize, &edges);
let span = u64::from(NODES);
let cancel = CancellationToken::new();
let calls = RefCell::new(Vec::new());
let cancel_past_semidominators = |_, done, _| {
calls.borrow_mut().push(done);
if done > 2 * span {
cancel.cancel();
}
Ok(())
};
let result = dominators(&start, &out, 0, &Store::Ram, &cancel_past_semidominators, &cancel);
assert!(matches!(result, Err(HeapError::Cancelled)));
let calls = calls.into_inner();
assert!(calls.contains(&(2 * span)), "the semidominator phase finished: {calls:?}");
let last = *calls.last().unwrap();
assert!(last > 2 * span && last < 3 * span, "stopped inside the idom phase: {calls:?}");
}
proptest! {
#[test]
fn matches_dataflow_on_random_graphs(n in 2usize..64, edges in vec((0u32..64, 0u32..64), 0..192)) {
let edges: Vec<_> = edges.into_iter().map(|(a, b)| (a % n as u32, b % n as u32)).collect();
let d = run(n, &edges, 0);
prop_assert_eq!(d.idom.to_vec(), dataflow_idom(n, &edges, 0));
prop_assert_eq!(d.preorder.len(), d.reachable as usize);
}
}
#[test]
fn matches_chk_on_a_large_random_graph() {
const NODES: u32 = 200_000;
const EXTRA_EDGES: u32 = 100_000;
let mut rng = XorShift::new(0x5EED_D0D0);
let mut edges: Vec<_> = (1..NODES).map(|v| (rng.below(v), v)).collect();
edges.extend((0..EXTRA_EDGES).map(|_| (rng.below(NODES), rng.below(NODES))));
// Visit edges in a scrambled order so the DFS tree differs from the generating tree.
for i in (1..edges.len()).rev() {
edges.swap(i, rng.below(i as u32 + 1) as usize);
}
let started = Instant::now();
let d = run(NODES as usize, &edges, 0);
let elapsed = started.elapsed();
let expected = chk_idom(NODES as usize, &edges, 0);
let mismatches = d.idom.iter().zip(&expected).filter(|(a, b)| a != b).count();
assert_eq!(mismatches, 0);
assert_eq!(d.reachable, NODES);
eprintln!("dominators of {NODES} random nodes: {elapsed:?}");
}
/// A doubly linked list whose items an array list also holds: super-root 0 → list 1 → node 0,
/// 0 → array list 2 → element array 3 → every item; node j → item j, next, prev.
fn shared_list_and_array(k: u32) -> (usize, Vec<(u32, u32)>) {
let node = |j: u32| 4 + 2 * j;
let item = |j: u32| 5 + 2 * j;
let mut edges = vec![(0, 1), (0, 2), (2, 3), (1, node(0))];
for j in 0..k {
edges.push((node(j), item(j)));
if j + 1 < k {
edges.push((node(j), node(j + 1)));
}
if j > 0 {
edges.push((node(j), node(j - 1)));
}
edges.push((3, item(j)));
}
((4 + 2 * k) as usize, edges)
}
#[test]
fn shared_list_and_array_is_near_linear() {
let (n, edges) = shared_list_and_array(29);
assert_eq!(run(n, &edges, 0).idom.to_vec(), dataflow_idom(n, &edges, 0));
let (n, edges) = shared_list_and_array(3_000);
assert_eq!(run(n, &edges, 0).idom.to_vec(), chk_idom(n, &edges, 0));
const K: u32 = 200_000;
let (n, edges) = shared_list_and_array(K);
let (start, out) = csr(n, &edges);
let started = Instant::now();
let (d, steps) = compute(&start, &out, 0, &Store::Ram, &no_progress, &CancellationToken::new()).unwrap();
let elapsed = started.elapsed();
let item = |j: u32| (5 + 2 * j) as usize;
assert!((0..K).all(|j| d.idom[item(j)] == 0), "every item is held by both collections");
let m = edges.len() as u64;
let bound = (n as u64 + m) * u64::from(n.ilog2());
eprintln!("shared list k={K}: {steps} steps (n={n}, m={m}, bound {bound}), {elapsed:?}");
assert!(steps <= bound, "{steps} steps for n={n}, m={m}");
}
#[test]
fn deep_chain_has_linear_dominators() {
// A small stack proves the algorithm does not recurse along the chain.
std::thread::Builder::new()
.stack_size(256 << 10)
.spawn(|| {
const NODES: u32 = 1_000_000;
let mut edges: Vec<_> = (1..NODES).map(|v| (v - 1, v)).collect();
edges.push((0, NODES - 1));
// Makes `eval` compress the whole chain at once.
edges.push((NODES - 1, 1));
let started = Instant::now();
let d = run(NODES as usize, &edges, 0);
eprintln!("dominators of a {NODES}-node chain: {:?}", started.elapsed());
assert_eq!(d.reachable, NODES);
assert_eq!(d.idom[0], 0);
assert_eq!(d.idom[NODES as usize - 1], 0);
let wrong = (1..NODES - 1).filter(|&v| d.idom[v as usize] != v - 1).count();
assert_eq!(wrong, 0);
})
.unwrap()
.join()
.unwrap();
}
+26
View File
@@ -0,0 +1,26 @@
/// Offsets (`at`) are byte positions in the `.hprof` file.
#[derive(Debug, thiserror::Error)]
pub enum HeapError {
#[error("the dump ends early at byte {at}")]
Truncated { at: u64 },
#[error("not a heap dump (no JAVA PROFILE header)")]
NotHprof,
#[error("unsupported identifier size {0}")]
UnsupportedIdSize(u32),
#[error("unknown heap sub-record tag {tag:#04x} at byte {at}")]
UnknownSubRecord { tag: u8, at: u64 },
#[error("unknown value type {ty} at byte {at}")]
UnknownType { ty: u8, at: u64 },
#[error("class {id:#x} used at byte {at} is not in the dump")]
MissingClass { id: u64, at: u64 },
#[error("the heap has too many objects to analyze")]
TooLarge,
#[error("the dump's class and field names are too long; it is not a dump a JVM wrote")]
NamesTooLarge,
#[error("not enough disk space: {needed} bytes needed, {available} free")]
NotEnoughDiskSpace { needed: u64, available: u64 },
#[error("cancelled")]
Cancelled,
#[error("{0}")]
Io(#[from] std::io::Error),
}
+34
View File
@@ -0,0 +1,34 @@
//! Names of the files an analysis writes into its index directory. They live in one list so a
//! failed analysis removes exactly these and never the dump or anything else sharing the directory.
pub(crate) const IDS_BITS_FILE: &str = "ids.bits.u64";
pub(crate) const IDS_WRANK_FILE: &str = "ids.wrank.u32";
pub(crate) const IDS_TABLE_FILE: &str = "ids.table.u32";
pub(crate) const IDS_CHUNKS_FILE: &str = "ids.chunks.u32";
pub(crate) const IDS_SORTED_FILE: &str = "ids.sorted.u64";
pub(crate) const TYPES_FILE: &str = "types.u32";
pub(crate) const OFFSETS_FILE: &str = "offsets.u64";
pub(crate) const IDOM_FILE: &str = "idom.u32";
pub(crate) const SHALLOW8_FILE: &str = "shallow8.u32";
pub(crate) const RETAINED_FILE: &str = "retained.u64";
pub(crate) const KIDS_START_FILE: &str = "kids_start.u32";
pub(crate) const KIDS_FILE: &str = "kids.u32";
pub(crate) const GC_PARENT_FILE: &str = "gc_parent.u32";
pub(crate) const SUMMARY_FILE: &str = "summary.json";
pub(crate) const INDEX_FILES: &[&str] = &[
IDS_BITS_FILE,
IDS_WRANK_FILE,
IDS_TABLE_FILE,
IDS_CHUNKS_FILE,
IDS_SORTED_FILE,
TYPES_FILE,
OFFSETS_FILE,
IDOM_FILE,
SHALLOW8_FILE,
RETAINED_FILE,
KIDS_START_FILE,
KIDS_FILE,
GC_PARENT_FILE,
SUMMARY_FILE,
];
+274
View File
@@ -0,0 +1,274 @@
//! The object graph of a heap dump in CSR form: one node per object, plus a super-root.
mod classes;
mod link;
use std::collections::hash_map::Entry;
use std::collections::{HashMap, HashSet};
use ide_core::CancellationToken;
use self::classes::{ClassTable, IdHash};
use self::link::Linker;
use crate::column::{Column, Store};
use crate::error::HeapError;
use crate::files::{OFFSETS_FILE, TYPES_FILE};
use crate::hprof::{BasicType, HeapRecord, HeapRootKind, Record, parse_header, scan, scan_segment};
use crate::ids::{IdMap, IdMapBuilder};
use crate::progress::{CHECK_EVERY, HeapPhase, Progress};
/// Number of primitive array type slots, one per non-object [`BasicType`].
pub const PRIM_SLOTS: u32 = 8;
pub struct ClassInfo {
pub id: u64,
/// The hprof name with `/` turned into `.` (`java.lang.String`, `[Ljava.lang.Object;`).
pub name: String,
/// Index in [`HeapGraph::classes`]. Chains always end: a super cycle in the dump is cut where it closes.
pub super_index: Option<u32>,
/// This class's own instance fields, in dump order.
pub fields: Vec<(String, BasicType)>,
/// `(name, object id)` of reference statics; a null static has object id 0.
pub statics: Vec<(String, u64)>,
pub static_bytes: u64,
/// Index in `fields` of `java.lang.ref.Reference.referent`, on that class only.
pub weak_field: Option<usize>,
}
pub struct HeapGraph {
pub ids: IdMap,
/// Type slot of each object: class index for instances and object arrays,
/// [`Self::class_slot`] for class objects, [`Self::prim_slot`] for primitive arrays.
pub types: Column<u32>,
/// Array length, 0 for instances and classes.
pub lens: Column<u32>,
/// File offset of each object's record, for [`crate::record_at`].
pub offsets: Column<u64>,
/// `n + 2` entries: node `v`'s targets are `out[out_start[v]..out_start[v + 1]]`.
pub out_start: Column<u32>,
pub out: Column<u32>,
/// The super-root, node `n`, with an edge to every rooted object.
pub root: u32,
/// `(object, kind, thread serial)` of every root record whose object is in the dump.
pub roots: Vec<(u32, HeapRootKind, Option<u32>)>,
/// Class dumps in dump order, then a synthetic `java.lang.Class` when the dump has none.
pub classes: Vec<ClassInfo>,
/// Type slot of class objects (`java.lang.Class`).
pub class_slot: u32,
/// Thread serial → thread object, from `ROOT THREAD OBJECT` records.
pub thread_objects: HashMap<u32, u32>,
/// `Reference.referent` edges left out of the graph.
pub weak_edges_skipped: u64,
pub id_size: u8,
pub timestamp_ms: u64,
}
impl HeapGraph {
pub fn prim_slot(&self, ty: BasicType) -> u32 {
self.classes.len() as u32 + (ty as u32 - BasicType::Boolean as u32)
}
/// The element type of primitive array slot `slot`, or `None` for a class slot.
pub fn prim_type(&self, slot: u32) -> Option<BasicType> {
let offset = slot.checked_sub(self.classes.len() as u32).filter(|&o| o < PRIM_SLOTS)?;
BasicType::from_u8(offset as u8 + BasicType::Boolean as u8)
}
pub fn out_edges(&self, v: u32) -> &[u32] {
let v = v as usize;
&self.out[self.out_start[v] as usize..self.out_start[v + 1] as usize]
}
}
/// Builds the graph of the dump in `buf`. Strong edges only: `Reference.referent` is counted in
/// [`HeapGraph::weak_edges_skipped`] instead. `progress` gets `Reading` by bytes, then
/// `Linking` by objects, starting with `(Linking, 0, objects)` once the object count is known;
/// an `Err` from it stops the build with that error.
pub fn build_graph(
buf: &[u8],
store: &Store,
progress: &dyn Fn(HeapPhase, u64, u64) -> Result<(), HeapError>,
cancel: &CancellationToken,
) -> Result<HeapGraph, HeapError> {
let progress = Progress::new(progress, cancel);
let header = parse_header(buf)?;
let id_size = header.id_size;
let len = buf.len() as u64;
// Pass 1: names, classes, roots and every object id.
progress.report(HeapPhase::Reading, 0, len)?;
let mut names: IdHash<&[u8]> = IdHash::default();
let mut class_names: IdHash<u64> = IdHash::default();
let mut dumps = Vec::new();
let mut class_by_id: IdHash<u32> = IdHash::default();
let mut raw_roots = Vec::new();
let mut segments = Vec::new();
let mut builder = IdMapBuilder::new();
let (mut step, mut pos) = (0, header.body_start as u64);
scan(buf, &header, &mut |record| {
progress.step(step, HeapPhase::Reading, pos, len)?;
step += 1;
match record {
Record::Utf8 { id, text } => {
names.insert(id, text);
}
Record::LoadClass { class_id, name_id } => {
class_names.insert(class_id, name_id);
}
Record::Segment(range) => {
pos = range.start as u64;
segments.push(range);
}
Record::Heap { at, record } => {
pos = at as u64;
match record {
HeapRecord::Root { kind, id, thread_serial } => raw_roots.push((id, kind, thread_serial)),
HeapRecord::Class(dump) => {
builder.insert(dump.id);
if let Entry::Vacant(slot) = class_by_id.entry(dump.id) {
slot.insert(dumps.len() as u32);
dumps.push(dump);
}
}
HeapRecord::Instance { id, .. }
| HeapRecord::ObjectArray { id, .. }
| HeapRecord::PrimitiveArray { id, .. } => builder.insert(id),
}
}
}
Ok(())
})?;
progress.report(HeapPhase::Reading, len, len)?;
let ids = builder.build(store, cancel)?;
let table = classes::resolve(dumps, class_by_id, &names, &class_names, id_size, buf.len())?;
drop((names, class_names));
let n = ids.len();
let objects = u64::from(n);
progress.report(HeapPhase::Linking, 0, objects)?;
let mut roots = Vec::new();
let mut root_targets = Vec::new();
let mut thread_objects = HashMap::new();
let (mut seen_records, mut seen_targets) = (HashSet::new(), HashSet::new());
for (i, &(id, kind, serial)) in raw_roots.iter().enumerate() {
progress.step(i as u64, HeapPhase::Linking, 0, objects)?;
let Some(object) = ids.rank(id).filter(|_| id != 0) else { continue };
if seen_records.insert((object, kind as u8, serial)) {
roots.push((object, kind, serial));
}
if seen_targets.insert(object) {
root_targets.push(object);
}
if let (HeapRootKind::ThreadObject, Some(serial)) = (kind, serial) {
thread_objects.entry(serial).or_insert(object);
}
}
drop((raw_roots, seen_records, seen_targets));
let linker = Linker::new(&ids, &table, id_size);
let nodes = n as usize;
let mut types = store.column::<u32>(TYPES_FILE, nodes, 0)?;
let mut lens = store.scratch::<u32>(nodes, 0)?;
let mut offsets = store.column::<u64>(OFFSETS_FILE, nodes, 0)?;
let mut out_start = store.scratch::<u32>(nodes + 2, 0)?;
// Both passes visit every object record once; report them as one walk over the objects.
let mut visited = 0u64;
let tick = |visited: &mut u64| {
let result = progress.step(*visited, HeapPhase::Linking, (*visited / 2).min(objects), objects);
*visited += 1;
result
};
// Pass 2: node columns and out-degrees, counted into `out_start[v + 1]`.
let mut weak_edges_skipped = 0;
for segment in &segments {
scan_segment(buf, segment.clone(), id_size, &mut |at, record| {
let Some(node) = linker.node(at, &record)? else { return Ok(()) };
tick(&mut visited)?;
let v = node.index as usize;
types[v] = node.ty;
lens[v] = node.len;
offsets[v] = at as u64;
let mut degree = 0u64;
linker.edges(&record, &mut |_, weak| {
if weak {
weak_edges_skipped += 1;
} else {
degree += 1;
}
});
let count = &mut out_start[v + 1];
*count = u32::try_from(u64::from(*count) + degree).map_err(|_| HeapError::TooLarge)?;
Ok(())
})?;
}
out_start[nodes + 1] = u32::try_from(root_targets.len()).map_err(|_| HeapError::TooLarge)?;
let edges = accumulate(&mut out_start, cancel)?;
// Pass 3: `out_start[v]` is node v's write cursor; once filled it holds v's end, so the
// column is shifted one slot right to become the starts again.
let mut out = store.scratch::<u32>(edges as usize, 0)?;
for segment in &segments {
scan_segment(buf, segment.clone(), id_size, &mut |at, record| {
let Some(node) = linker.node(at, &record)? else { return Ok(()) };
tick(&mut visited)?;
let v = node.index as usize;
let mut cursor = out_start[v] as usize;
linker.edges(&record, &mut |target, weak| {
if !weak && let Some(slot) = out.get_mut(cursor) {
*slot = target;
cursor += 1;
}
});
out_start[v] = cursor as u32;
Ok(())
})?;
}
let first_root_edge = out_start[nodes] as usize;
for (slot, &target) in out[first_root_edge..].iter_mut().zip(&root_targets) {
*slot = target;
}
out_start[nodes] = edges;
out_start.copy_within(0..nodes + 1, 1);
out_start[0] = 0;
progress.report(HeapPhase::Linking, objects, objects)?;
drop(linker);
let ClassTable { classes, class_slot, .. } = table;
Ok(HeapGraph {
ids,
types,
lens,
offsets,
out_start,
out,
root: n,
roots,
classes,
class_slot,
thread_objects,
weak_edges_skipped,
id_size,
timestamp_ms: header.timestamp_ms,
})
}
/// Turns per-node counts into running totals in place (`counts[k]` becomes the sum of
/// `counts[..=k]`) and returns the last one; past `u32::MAX` the graph is `TooLarge`.
pub(crate) fn accumulate(counts: &mut [u32], cancel: &CancellationToken) -> Result<u32, HeapError> {
let mut sum = 0u32;
for chunk in counts.chunks_mut(CHECK_EVERY as usize) {
if cancel.is_cancelled() {
return Err(HeapError::Cancelled);
}
for count in chunk {
sum = sum.checked_add(*count).ok_or(HeapError::TooLarge)?;
*count = sum;
}
}
Ok(sum)
}
#[cfg(test)]
mod tests;
+188
View File
@@ -0,0 +1,188 @@
//! Class dumps resolved into [`ClassInfo`]s, plus where the references sit in instance data.
use std::collections::HashMap;
use std::hash::BuildHasherDefault;
use super::{ClassInfo, PRIM_SLOTS};
use crate::column::NONE;
use crate::error::HeapError;
use crate::hprof::{BasicType, ClassDump};
use crate::ids::ChunkHasher;
/// Keyed by hprof id (an address).
pub(super) type IdHash<V> = HashMap<u64, V, BuildHasherDefault<ChunkHasher>>;
const REFERENCE: &str = "java.lang.ref.Reference";
const JAVA_LANG_CLASS: &str = "java.lang.Class";
/// Not computed yet, in [`ClassTable::fielded`].
const UNSET: u32 = NONE - 1;
pub(super) struct ClassTable {
pub(super) classes: Vec<ClassInfo>,
/// Class id → class index.
pub(super) by_id: IdHash<u32>,
pub(super) class_slot: u32,
/// Per class: `(offset, is the referent)` of each reference field in the class's own part of
/// the instance data.
pub(super) refs: Vec<Vec<(usize, bool)>>,
/// Per class: bytes of its own part of the instance data.
pub(super) own_bytes: Vec<usize>,
/// Per class: the nearest class up its chain (itself included) whose own part is not empty, or
/// `NONE`. Walking these instead of every super keeps a walk bounded by the data it reads.
pub(super) fielded: Vec<u32>,
}
/// The longest name a JVM gives a class or field (a constant pool UTF8 has a `u2` length).
const MAX_NAME_BYTES: usize = 65_535;
/// Name bytes a dump may decode beyond 4 per byte of the file.
const NAME_BUDGET_FLOOR: usize = 64 << 20;
/// Names decoded from the UTF8 records within a budget: every field of every class may name the
/// same record, so a small crafted file could otherwise ask for terabytes and abort the IDE.
struct Names<'m, 'b> {
records: &'m IdHash<&'b [u8]>,
left: usize,
}
impl Names<'_, '_> {
/// Empty for an id with no record.
fn text(&mut self, id: u64) -> Result<String, HeapError> {
let Some(bytes) = self.records.get(&id) else { return Ok(String::new()) };
if bytes.len() > MAX_NAME_BYTES || bytes.len() > self.left {
return Err(HeapError::NamesTooLarge);
}
self.left -= bytes.len();
Ok(String::from_utf8_lossy(bytes).into_owned())
}
}
/// `dumps` holds one dump per class id, and `by_id` its position; `names` are the UTF8 records
/// and `class_names` the LOAD CLASS class id → name id pairs. `file_len` bounds the name bytes.
pub(super) fn resolve(
dumps: Vec<ClassDump>,
by_id: IdHash<u32>,
names: &IdHash<&[u8]>,
class_names: &IdHash<u64>,
id_size: u8,
file_len: usize,
) -> Result<ClassTable, HeapError> {
// Room for the synthetic java.lang.Class and the primitive slots in a `u32` type id.
if dumps.len() > (u32::MAX - PRIM_SLOTS - 1) as usize {
return Err(HeapError::TooLarge);
}
let mut names = Names { records: names, left: file_len.saturating_mul(4).saturating_add(NAME_BUDGET_FLOOR) };
let mut supers: Vec<Option<u32>> =
dumps.iter().map(|d| if d.super_id == 0 { None } else { by_id.get(&d.super_id).copied() }).collect();
cut_super_cycles(&mut supers);
let mut classes = Vec::with_capacity(dumps.len() + 1);
let mut refs = Vec::with_capacity(dumps.len() + 1);
let mut own_bytes = Vec::with_capacity(dumps.len() + 1);
for (dump, super_index) in dumps.into_iter().zip(supers) {
let name = match class_names.get(&dump.id) {
Some(&name_id) if names.records.contains_key(&name_id) => names.text(name_id)?.replace('/', "."),
_ => format!("{:#x}", dump.id),
};
let fields =
dump.fields.iter().map(|&(n, ty)| Ok((names.text(n)?, ty))).collect::<Result<Vec<_>, HeapError>>()?;
let weak_field = if name == REFERENCE {
fields.iter().position(|(n, ty)| n == "referent" && *ty == BasicType::Object)
} else {
None
};
let mut offset = 0;
let mut class_refs = Vec::new();
for (k, &(_, ty)) in fields.iter().enumerate() {
if ty == BasicType::Object {
class_refs.push((offset, weak_field == Some(k)));
}
offset += ty.hprof_size(id_size);
}
refs.push(class_refs);
own_bytes.push(offset);
classes.push(ClassInfo {
id: dump.id,
name,
super_index,
fields,
statics: dump
.statics
.iter()
.map(|&(n, object)| Ok((names.text(n)?, object)))
.collect::<Result<_, HeapError>>()?,
static_bytes: dump.static_bytes,
weak_field,
});
}
let class_slot = match classes.iter().position(|c| c.name == JAVA_LANG_CLASS) {
Some(slot) => slot as u32,
None => {
classes.push(ClassInfo {
id: 0,
name: JAVA_LANG_CLASS.to_owned(),
super_index: None,
fields: Vec::new(),
statics: Vec::new(),
static_bytes: 0,
weak_field: None,
});
refs.push(Vec::new());
own_bytes.push(0);
classes.len() as u32 - 1
}
};
let fielded = nearest_fielded(&classes, &own_bytes);
Ok(ClassTable { classes, by_id, class_slot, refs, own_bytes, fielded })
}
/// A dump may claim `A extends B extends A`; the edge that closes a cycle is dropped so every
/// walk up the chain ends. Iterative, so a long chain cannot overflow the stack.
fn cut_super_cycles(supers: &mut [Option<u32>]) {
const NEW: u8 = 0;
const ON_PATH: u8 = 1;
const DONE: u8 = 2;
let mut state = vec![NEW; supers.len()];
let mut path = Vec::new();
for start in 0..supers.len() {
let mut c = start;
while state[c] == NEW {
state[c] = ON_PATH;
path.push(c);
match supers[c] {
Some(s) if state[s as usize] == ON_PATH => {
supers[c] = None;
break;
}
Some(s) => c = s as usize,
None => break,
}
}
for p in path.drain(..) {
state[p] = DONE;
}
}
}
fn nearest_fielded(classes: &[ClassInfo], own_bytes: &[usize]) -> Vec<u32> {
let mut fielded: Vec<u32> =
own_bytes.iter().enumerate().map(|(c, &b)| if b > 0 { c as u32 } else { UNSET }).collect();
let mut path = Vec::new();
for start in 0..classes.len() {
let mut c = start;
let found = loop {
if fielded[c] != UNSET {
break fielded[c];
}
path.push(c);
match classes[c].super_index {
Some(s) => c = s as usize,
None => break NONE,
}
};
for p in path.drain(..) {
fielded[p] = found;
}
}
fielded
}
+107
View File
@@ -0,0 +1,107 @@
//! Turns one heap sub-record into its node (index, type, length) and its outgoing edges. Both
//! linking passes go through here, so the degrees counted in the first match the edges written
//! in the second.
use super::classes::ClassTable;
use crate::column::NONE;
use crate::error::HeapError;
use crate::hprof::{BasicType, HeapRecord};
use crate::ids::IdMap;
pub(super) struct Node {
pub(super) index: u32,
pub(super) ty: u32,
pub(super) len: u32,
}
pub(super) struct Linker<'a> {
ids: &'a IdMap,
table: &'a ClassTable,
/// Per class: the node of its class object, `NONE` for the synthetic `java.lang.Class`.
class_objects: Vec<u32>,
id_size: u8,
}
impl<'a> Linker<'a> {
pub(super) fn new(ids: &'a IdMap, table: &'a ClassTable, id_size: u8) -> Self {
let class_objects =
table.classes.iter().map(|c| if c.id == 0 { NONE } else { ids.rank(c.id).unwrap_or(NONE) }).collect();
Self { ids, table, class_objects, id_size }
}
/// `None` for root records. `at` is the record's offset, for the error of an instance or array
/// whose class is not in the dump.
pub(super) fn node(&self, at: usize, record: &HeapRecord<'_>) -> Result<Option<Node>, HeapError> {
let class_index = |class_id: u64| {
self.table.by_id.get(&class_id).copied().ok_or(HeapError::MissingClass { id: class_id, at: at as u64 })
};
let (id, ty, len) = match record {
HeapRecord::Root { .. } => return Ok(None),
HeapRecord::Class(c) => (c.id, self.table.class_slot, 0),
HeapRecord::Instance { id, class_id, .. } => (*id, class_index(*class_id)?, 0),
HeapRecord::ObjectArray { id, class_id, len, .. } => (*id, class_index(*class_id)?, *len),
HeapRecord::PrimitiveArray { id, ty, len, .. } => {
let prim_base = self.table.classes.len() as u32;
(*id, prim_base + (*ty as u32 - BasicType::Boolean as u32), *len)
}
};
Ok(self.ids.rank(id).map(|index| Node { index, ty, len }))
}
/// Calls `edge(target, is_referent)` for every reference to an object in the dump, in field
/// order; `is_referent` marks `Reference.referent`, which callers leave out of the graph.
pub(super) fn edges(&self, record: &HeapRecord<'_>, edge: &mut impl FnMut(u32, bool)) {
let mut to = |id: u64, weak: bool| {
if let Some(target) = self.target(id) {
edge(target, weak);
}
};
match record {
HeapRecord::Root { .. } | HeapRecord::PrimitiveArray { .. } => {}
HeapRecord::Class(c) => {
to(c.super_id, false);
to(c.loader_id, false);
for &(_, object) in &c.statics {
to(object, false);
}
}
HeapRecord::ObjectArray { data, .. } => {
for item in data.chunks_exact(usize::from(self.id_size)) {
to(read_id(item, 0, self.id_size).unwrap_or(0), false);
}
}
HeapRecord::Instance { class_id, data, .. } => {
let Some(&class) = self.table.by_id.get(class_id) else { return };
let mut base = 0;
let mut next = self.table.fielded[class as usize];
'chain: while next != NONE && base < data.len() {
let c = next as usize;
for &(offset, weak) in &self.table.refs[c] {
let Some(id) = read_id(data, base + offset, self.id_size) else { break 'chain };
to(id, weak);
}
base += self.table.own_bytes[c];
next = self.table.classes[c].super_index.map_or(NONE, |s| self.table.fielded[s as usize]);
}
let object = self.class_objects[class as usize];
if object != NONE {
edge(object, false);
}
}
}
}
/// Id 0 is null; ids not in the dump have no node.
fn target(&self, id: u64) -> Option<u32> {
if id == 0 { None } else { self.ids.rank(id) }
}
}
/// The big-endian id at `at`, or `None` when `data` ends first.
fn read_id(data: &[u8], at: usize, id_size: u8) -> Option<u64> {
let bytes = data.get(at..at.checked_add(usize::from(id_size))?)?;
Some(match *bytes {
[a, b, c, d] => u64::from(u32::from_be_bytes([a, b, c, d])),
_ => u64::from_be_bytes(bytes.try_into().ok()?),
})
}
+479
View File
@@ -0,0 +1,479 @@
use std::cell::RefCell;
use proptest::prelude::*;
use super::*;
use crate::hprof::{HeapRecord, record_at};
use crate::testing::{HprofWriter, MiniHeap, sample};
const H: u64 = 0x1000;
const N1: u64 = 0x1100;
const N2: u64 = 0x1200;
const N3: u64 = 0x1300;
const L: u64 = 0x1400;
const R: u64 = 0x1500;
const W: u64 = 0x1600;
const N4: u64 = 0x1700;
fn no_progress(_: HeapPhase, _: u64, _: u64) -> Result<(), HeapError> {
Ok(())
}
fn build(bytes: &[u8]) -> Result<HeapGraph, HeapError> {
build_graph(bytes, &Store::Ram, &no_progress, &CancellationToken::new())
}
fn index(g: &HeapGraph, id: u64) -> u32 {
g.ids.rank(id).unwrap_or_else(|| panic!("{id:#x} is not in the graph"))
}
fn class(g: &HeapGraph, name: &str) -> (u32, u64) {
let slot = g.classes.iter().position(|c| c.name == name).unwrap_or_else(|| panic!("no class {name}"));
(slot as u32, g.classes[slot].id)
}
/// Targets of `id`'s edges, as ids, in edge order.
fn out_of(g: &HeapGraph, id: u64) -> Vec<u64> {
g.out_edges(index(g, id)).iter().map(|&t| g.ids.id_at(t)).collect()
}
fn root_targets(g: &HeapGraph) -> Vec<u64> {
g.out_edges(g.root).iter().map(|&t| g.ids.id_at(t)).collect()
}
fn assert_csr(g: &HeapGraph) {
let n = g.ids.len();
assert_eq!(g.root, n);
assert_eq!(g.out_start.len(), n as usize + 2);
assert_eq!(g.out_start[0], 0);
assert!(g.out_start.windows(2).all(|w| w[0] <= w[1]), "out_start is not monotone");
assert_eq!(g.out_start[n as usize + 1] as usize, g.out.len());
assert!(g.out.iter().all(|&t| t < n), "edge to a node outside the objects");
assert_eq!((g.types.len(), g.lens.len(), g.offsets.len()), (n as usize, n as usize, n as usize));
let slots = g.classes.len() as u32 + PRIM_SLOTS;
assert!(g.types.iter().all(|&t| t < slots), "type outside the slots");
for c in &g.classes {
let mut steps = 0;
let mut cursor = c.super_index;
while let Some(s) = cursor {
steps += 1;
assert!(steps <= g.classes.len(), "super chain of {} does not end", c.name);
cursor = g.classes[s as usize].super_index;
}
}
}
#[test]
fn edges_follow_fields_and_skip_weak_referents() {
let g = build(&sample().bytes()).unwrap();
assert_csr(&g);
let (_, holder) = class(&g, "Holder");
let (_, node) = class(&g, "Node");
let (_, weak) = class(&g, "java.lang.ref.WeakReference");
assert_eq!(out_of(&g, H), [N1, N2, holder]);
assert_eq!(out_of(&g, N1), [N3, node]);
assert_eq!(out_of(&g, N3), [L, node]);
assert_eq!(out_of(&g, W), [weak]);
assert_eq!(out_of(&g, N4), [node]);
assert_eq!(out_of(&g, L), Vec::<u64>::new());
assert_eq!(g.weak_edges_skipped, 1);
let (reference, _) = class(&g, "java.lang.ref.Reference");
assert_eq!(g.classes[reference as usize].weak_field, Some(0));
assert!(g.classes.iter().filter(|c| c.name != "java.lang.ref.Reference").all(|c| c.weak_field.is_none()));
}
#[test]
fn object_arrays_skip_null_and_unknown() {
let mut heap = sample();
heap.array_of(0x1800, "[Ljava.lang.Object;", &[N2, 0, 0x9990, N1, 0xDEAD_BEEF]);
let g = build(&heap.bytes()).unwrap();
assert_csr(&g);
assert_eq!(out_of(&g, R), [N1]);
assert_eq!(out_of(&g, 0x1800), [N2, N1]);
}
#[test]
fn super_root_points_at_each_root_once() {
let mut heap = sample();
heap.root(HeapRootKind::JniGlobal, H).root(HeapRootKind::MonitorUsed, H).root(HeapRootKind::JniLocal, 0x9990);
let g = build(&heap.bytes()).unwrap();
assert_csr(&g);
let targets = root_targets(&g);
assert_eq!(targets.iter().filter(|&&t| t == H).count(), 1);
assert!(targets.contains(&N3) && targets.contains(&W));
assert!(!targets.contains(&N1) && !targets.contains(&R) && !targets.contains(&N4));
let classes = g.classes.iter().filter(|c| c.id != 0).count();
assert_eq!(targets.len(), 3 + classes, "H, N3, W and every sticky class");
let mut sorted = targets.clone();
sorted.sort_unstable();
sorted.dedup();
assert_eq!(sorted.len(), targets.len());
let h = index(&g, H);
let of_h: Vec<_> = g.roots.iter().filter(|r| r.0 == h).map(|r| (r.1, r.2)).collect();
assert_eq!(of_h, [(HeapRootKind::JniGlobal, None), (HeapRootKind::MonitorUsed, None)]);
assert!(g.roots.contains(&(index(&g, N3), HeapRootKind::JavaFrame, Some(1))));
assert!(g.roots.iter().all(|r| r.0 < g.root));
}
#[test]
fn class_dump_edges_cover_statics() {
let mut w = HprofWriter::new();
w.string(1, "java/lang/Object")
.load_class(0x100, 1)
.class(0x100, 0, &[], &[])
.string(2, "Config")
.string(3, "INSTANCE")
.string(4, "EMPTY")
.string(5, "LOST")
.load_class(0x200, 2)
.class_with_loader(0x200, 0x100, 0x1200, &[], &[(3, N1), (4, 0), (5, 0x9990)])
.instance(N1, 0x100, &[])
.instance(0x1200, 0x100, &[]);
let g = build(&w.finish()).unwrap();
assert_csr(&g);
assert_eq!(out_of(&g, 0x200), [0x100, 0x1200, N1]);
assert_eq!(out_of(&g, 0x100), Vec::<u64>::new());
let (config, _) = class(&g, "Config");
let info = &g.classes[config as usize];
let statics: Vec<_> = info.statics.iter().map(|(n, id)| (n.as_str(), *id)).collect();
assert_eq!(statics, [("INSTANCE", N1), ("EMPTY", 0), ("LOST", 0x9990)]);
assert_eq!(info.super_index, Some(class(&g, "java.lang.Object").0));
}
#[test]
fn missing_class_is_an_error() {
let mut w = HprofWriter::new();
w.string(1, "java/lang/Object").load_class(0x100, 1).class(0x100, 0, &[], &[]).instance(N1, 0x999, &[]);
let bytes = w.finish();
let at = instance_offset(&bytes, N1);
let err = build(&bytes).err().unwrap();
assert!(matches!(err, HeapError::MissingClass { id: 0x999, at: a } if a == at as u64), "{err:?}");
let mut w = HprofWriter::new();
w.obj_array(0x10, 0x999, &[]);
assert!(matches!(build(&w.finish()), Err(HeapError::MissingClass { id: 0x999, .. })));
}
fn instance_offset(bytes: &[u8], id: u64) -> usize {
let header = crate::hprof::parse_header(bytes).unwrap();
let mut found = None;
crate::hprof::scan(bytes, &header, &mut |r| {
if let crate::hprof::Record::Heap { at, record: HeapRecord::Instance { id: i, .. } } = r
&& i == id
{
found = Some(at);
}
Ok(())
})
.unwrap();
found.unwrap()
}
#[test]
fn cancel_during_linking_returns_cancelled() {
let bytes = sample().bytes();
let cancelled = CancellationToken::new();
cancelled.cancel();
let result = build_graph(&bytes, &Store::Ram, &no_progress, &cancelled);
assert!(matches!(result, Err(HeapError::Cancelled)));
let cancel = CancellationToken::new();
let phases = RefCell::new(Vec::new());
let result = build_graph(
&bytes,
&Store::Ram,
&|phase, done, total| {
phases.borrow_mut().push((phase, done, total));
if phase == HeapPhase::Linking {
cancel.cancel();
}
Ok(())
},
&cancel,
);
assert!(matches!(result, Err(HeapError::Cancelled)));
let phases = phases.into_inner();
assert_eq!(phases.first(), Some(&(HeapPhase::Reading, 0, bytes.len() as u64)));
assert_eq!(phases.last(), Some(&(HeapPhase::Linking, 0, 15)), "stops at the first Linking report");
}
#[test]
fn progress_error_stops_the_build() {
let bytes = sample().bytes();
let fail = |phase, _, total| {
if phase == HeapPhase::Linking {
Err(HeapError::NotEnoughDiskSpace { needed: total * 90, available: 0 })
} else {
Ok(())
}
};
let result = build_graph(&bytes, &Store::Ram, &fail, &CancellationToken::new());
assert!(matches!(result, Err(HeapError::NotEnoughDiskSpace { needed: 1350, available: 0 })));
}
#[test]
fn progress_reads_bytes_then_links_objects() {
let bytes = sample().bytes();
let phases = RefCell::new(Vec::new());
let g = build_graph(
&bytes,
&Store::Ram,
&|phase, done, total| {
phases.borrow_mut().push((phase, done, total));
Ok(())
},
&CancellationToken::new(),
)
.unwrap();
let phases = phases.into_inner();
let len = bytes.len() as u64;
let objects = u64::from(g.root);
assert_eq!(objects, 15, "7 classes and 8 objects");
assert_eq!(phases.first(), Some(&(HeapPhase::Reading, 0, len)));
assert!(phases.contains(&(HeapPhase::Reading, len, len)));
let linking: Vec<_> = phases.iter().filter(|p| p.0 == HeapPhase::Linking).collect();
assert_eq!(linking.first(), Some(&&(HeapPhase::Linking, 0, objects)));
assert_eq!(linking.iter().filter(|p| ***p == (HeapPhase::Linking, 0, objects)).count(), 1, "{linking:?}");
assert_eq!(linking.last(), Some(&&(HeapPhase::Linking, objects, objects)));
assert!(phases.iter().all(|&(_, done, total)| done <= total));
}
#[test]
fn types_lengths_and_offsets() {
let bytes = sample().bytes();
let g = build(&bytes).unwrap();
let (holder, holder_id) = class(&g, "Holder");
let (node, _) = class(&g, "Node");
let (array, _) = class(&g, "[Ljava.lang.Object;");
let (java_lang_class, _) = class(&g, "java.lang.Class");
assert_eq!(g.class_slot, java_lang_class);
let ty = |id| g.types[index(&g, id) as usize];
let len = |id| g.lens[index(&g, id) as usize];
assert_eq!([ty(H), ty(N1), ty(R), ty(holder_id)], [holder, node, array, java_lang_class]);
assert_eq!(ty(L), g.prim_slot(BasicType::Long));
assert_eq!(g.prim_type(ty(L)), Some(BasicType::Long));
assert_eq!(g.prim_type(holder), None);
assert_eq!(g.prim_type(g.classes.len() as u32 + PRIM_SLOTS), None);
assert_eq!(g.prim_slot(BasicType::Boolean), g.classes.len() as u32);
assert_eq!(g.prim_slot(BasicType::Long), g.classes.len() as u32 + PRIM_SLOTS - 1);
assert_eq!([len(H), len(L), len(R), len(holder_id)], [0, 2, 3, 0]);
for i in 0..g.root {
let id = match record_at(&bytes, g.offsets[i as usize] as usize, g.id_size).unwrap() {
HeapRecord::Class(c) => c.id,
HeapRecord::Instance { id, .. } | HeapRecord::ObjectArray { id, .. } => id,
HeapRecord::PrimitiveArray { id, .. } => id,
other => panic!("offset of object {i} points at {other:?}"),
};
assert_eq!(id, g.ids.id_at(i));
}
assert_eq!((g.id_size, g.timestamp_ms), (8, 1_700_000_000_000));
let holder_info = &g.classes[holder as usize];
let fields: Vec<_> = holder_info.fields.iter().map(|(n, t)| (n.as_str(), *t)).collect();
assert_eq!(fields, [("a", BasicType::Object), ("b", BasicType::Object)]);
assert_eq!(holder_info.super_index, Some(class(&g, "java.lang.Object").0));
assert_eq!(g.classes[class(&g, "java.lang.Object").0 as usize].super_index, None);
}
#[test]
fn dump_without_java_lang_class_gets_a_synthetic_slot() {
let mut w = HprofWriter::new();
w.string(1, "java/lang/Object").load_class(0x100, 1).class(0x100, 0, &[], &[]).instance(N1, 0x100, &[]);
let g = build(&w.finish()).unwrap();
assert_csr(&g);
assert_eq!(g.classes.len(), 2);
let synthetic = &g.classes[g.class_slot as usize];
assert_eq!((synthetic.id, synthetic.name.as_str(), synthetic.super_index), (0, "java.lang.Class", None));
assert_eq!(g.types[index(&g, 0x100) as usize], g.class_slot);
assert_eq!(g.types[index(&g, N1) as usize], 0);
}
#[test]
fn classes_without_a_name_record_get_their_address() {
let mut w = HprofWriter::new();
w.class(0x100, 0, &[(9, BasicType::Object)], &[])
.instance(N1, 0x100, &N2.to_be_bytes())
.instance(N2, 0x100, &[0; 8]);
let g = build(&w.finish()).unwrap();
assert_eq!(g.classes[0].name, "0x100");
assert_eq!(g.classes[0].fields, [(String::new(), BasicType::Object)]);
assert_eq!(out_of(&g, N1), [N2, 0x100]);
}
#[test]
fn thread_objects_map_serials_to_threads() {
let mut heap = sample();
heap.class("java.lang.Thread", Some("java.lang.Object"), &[])
.object(0x2000, "java.lang.Thread", &[])
.object(0x2100, "java.lang.Thread", &[])
.root_in_thread(HeapRootKind::ThreadObject, 0x2000, 1)
.root_in_thread(HeapRootKind::ThreadObject, 0x2100, 7)
.root_in_thread(HeapRootKind::ThreadObject, 0x9990, 9);
let g = build(&heap.bytes()).unwrap();
let expected: HashMap<u32, u32> = [(1, index(&g, 0x2000)), (7, index(&g, 0x2100))].into();
assert_eq!(g.thread_objects, expected);
assert!(g.roots.contains(&(index(&g, 0x2100), HeapRootKind::ThreadObject, Some(7))));
}
#[test]
fn inherited_fields_follow_the_class_first_layout() {
let mut heap = MiniHeap::new();
heap.class("A", Some("java.lang.Object"), &[("a", BasicType::Object), ("n", BasicType::Int)])
.class("Empty", Some("A"), &[])
.class("B", Some("Empty"), &[("flag", BasicType::Byte), ("b", BasicType::Object)])
.object(0x10, "B", &[("a", 0x20), ("b", 0x30), ("flag", 1)])
.object(0x20, "A", &[])
.object(0x30, "A", &[]);
let g = build(&heap.bytes()).unwrap();
assert_csr(&g);
assert_eq!(out_of(&g, 0x10), [0x30, 0x20, class(&g, "B").1]);
}
#[test]
fn super_class_cycles_are_cut() {
let mut w = HprofWriter::new();
w.string(1, "A")
.string(2, "B")
.string(3, "S")
.string(4, "f")
.load_class(0x100, 1)
.load_class(0x200, 2)
.load_class(0x300, 3)
.class(0x100, 0x200, &[(4, BasicType::Object)], &[])
.class(0x200, 0x100, &[(4, BasicType::Object)], &[])
.class(0x300, 0x300, &[], &[])
.instance(0x10, 0x100, &[[0; 7].as_slice(), &[0x20], &[0; 7], &[0x30]].concat())
.instance(0x20, 0x300, &[])
.instance(0x30, 0x300, &[0; 64]);
let g = build(&w.finish()).unwrap();
assert_csr(&g);
assert_eq!(out_of(&g, 0x10), [0x20, 0x30, 0x100]);
assert_eq!(out_of(&g, 0x30), [0x300]);
}
#[test]
fn short_instance_data_stops_at_the_last_whole_field() {
let mut w = HprofWriter::new();
w.string(1, "P")
.string(2, "x")
.load_class(0x100, 1)
.class(0x100, 0, &[(2, BasicType::Object), (2, BasicType::Object)], &[])
.instance(0x10, 0x100, &[[0; 7].as_slice(), &[0x20], &[0; 5]].concat())
.instance(0x20, 0x100, &[]);
let g = build(&w.finish()).unwrap();
assert_csr(&g);
assert_eq!(out_of(&g, 0x10), [0x20, 0x100]);
}
#[test]
fn id_size_4_dump_links() {
let mut w = HprofWriter::with_id_size(4);
w.string(1, "Node")
.string(2, "next")
.load_class(0x100, 1)
.class(0x100, 0, &[(2, BasicType::Object)], &[(2, 0x20)])
.instance(0x10, 0x100, &0x20u32.to_be_bytes())
.instance(0x20, 0x100, &[0; 4])
.obj_array(0x30, 0x100, &[0x10, 0, 0x20])
.root(HeapRootKind::JniGlobal, 0x30);
let g = build(&w.finish()).unwrap();
assert_csr(&g);
assert_eq!(g.id_size, 4);
assert_eq!(out_of(&g, 0x10), [0x20, 0x100]);
assert_eq!(out_of(&g, 0x30), [0x10, 0x20]);
assert_eq!(out_of(&g, 0x100), [0x20]);
assert_eq!(root_targets(&g), [0x30]);
}
#[test]
fn dir_store_keeps_types_and_offsets_only() {
let dir = tempfile::tempdir().unwrap();
// The id map's own files (`ids.*`) depend on its layout; MiniHeap's class ids are unaligned.
let names = || {
let mut names: Vec<_> = std::fs::read_dir(dir.path())
.unwrap()
.map(|e| e.unwrap().file_name().into_string().unwrap())
.filter(|name| !name.starts_with("ids."))
.collect();
names.sort();
names
};
let g =
build_graph(&sample().bytes(), &Store::Dir(dir.path().to_path_buf()), &no_progress, &CancellationToken::new())
.unwrap();
assert_eq!(out_of(&g, H).len(), 3);
let kept = ["offsets.u64", "types.u32"];
assert!(dir.path().join("ids.sorted.u64").exists());
#[cfg(unix)]
assert_eq!(names(), kept);
drop(g);
assert_eq!(names(), kept);
}
#[test]
fn empty_dump_has_only_the_super_root() {
let g = build(&HprofWriter::new().finish()).unwrap();
assert_csr(&g);
assert_eq!((g.root, g.out.len(), g.classes.len()), (0, 0, 1));
assert_eq!(g.class_slot, 0);
}
#[test]
fn edge_count_past_u32_is_too_large() {
let cancel = CancellationToken::new();
let mut counts = [0, 2, 0, 3, 1];
assert_eq!(accumulate(&mut counts, &cancel).unwrap(), 6);
assert_eq!(counts, [0, 2, 2, 5, 6]);
let mut full = [0, u32::MAX - 1, 1];
assert_eq!(accumulate(&mut full, &cancel).unwrap(), u32::MAX);
let mut over = [0, u32::MAX, 1];
assert!(matches!(accumulate(&mut over, &cancel), Err(HeapError::TooLarge)));
cancel.cancel();
assert!(matches!(accumulate(&mut [0, 1], &cancel), Err(HeapError::Cancelled)));
}
/// A class with `fields` instance fields, every one named by the same `name_len`-byte UTF8 record.
fn shared_long_field_names(name_len: usize, fields: usize) -> Vec<u8> {
let mut w = HprofWriter::new();
let long = "f".repeat(name_len);
w.string(1, "java/lang/Object").load_class(0x100, 1).class(0x100, 0, &[], &[]);
w.string(2, "Wide").string(3, &long).load_class(0x200, 2);
w.class(0x200, 0x100, &vec![(3, BasicType::Int); fields], &[]);
w.finish()
}
#[test]
fn names_that_would_decode_far_larger_than_the_file_are_refused() {
// 128 MiB of names from a file of about 70 KB.
let bytes = shared_long_field_names(32 << 10, 4000);
assert!(bytes.len() < 100 << 10, "{}", bytes.len());
let err = build(&bytes).err().expect("refused");
assert!(matches!(err, HeapError::NamesTooLarge), "{err:?}");
}
#[test]
fn a_name_longer_than_the_jvm_allows_is_refused() {
let mut w = HprofWriter::new();
w.string(1, &"C".repeat(70_000)).load_class(0x100, 1).class(0x100, 0, &[], &[]);
assert!(matches!(build(&w.finish()), Err(HeapError::NamesTooLarge)));
}
#[test]
fn real_sized_names_shared_by_many_fields_are_fine() {
let g = build(&shared_long_field_names(40, 4000)).unwrap();
assert_eq!(g.classes[class(&g, "Wide").0 as usize].fields.len(), 4000);
}
proptest! {
#[test]
fn build_graph_never_panics_on_mutated_dumps(flips in proptest::collection::vec((0usize..4096, any::<u8>()), 1..12)) {
let mut bytes = sample().bytes();
for (pos, value) in flips {
let at = pos % bytes.len();
bytes[at] = value;
}
if let Ok(g) = build(&bytes) {
assert_csr(&g);
}
}
}
Loaded 100 of 1151 files, more files were not shown because too many files have changed in this diff. Show more