Compare commits

..
Author SHA1 Message Date
l0ng-ai 440f5384ad perf(agents): make switching to a long conversation tab cheap
A kept-alive agent tab is hidden with `content-visibility: hidden`, and WebKit
recomputes style for the whole subtree when it is shown again. With a long
conversation that is tens of thousands of nodes, so Ctrl+Tab (or any switch)
to it blocked the main thread for 200–500 ms.

- Folded sections (tool call runs, tool output, reasoning, subagent task and
  result) render their content only once opened, or while Find in
  Conversation is open so their text is still found.
- Conversation rows skip rendering while off screen (`content-visibility:
  auto`) once they have been laid out, keeping their remembered height so
  scrolling up doesn't jump; a width change lays them out again.
2026-11-09 11:00:54 +08:00
2407 changed files with 16455 additions and 205792 deletions

No files matched your search

-83
View File
@@ -1,83 +0,0 @@
---
name: eval
description: Run the code-intelligence evals (`pnpm eval`), pick the worst failing scenario, diagnose it from the attached timeline, fix it, prove the fix with before / after numbers measured on this machine without regressing other scenarios, and open one PR. Use for "跑 eval", "优化 Java 代码智能", "eval 失败了", an issue labelled `eval`, or any change to Java navigation / usages / gutter markers that needs numbers.
---
# Eval loop: measure → fix → prove
`pnpm eval` drives the production workbench in Chromium against a release `openidea-server`
with real language servers (Java / jdtls for now) and judges every scenario on what the user
sees: time to feedback, to the first result, to the final one, and the results themselves vs
`evals/<language>/golden/`. One machine, before and after: numbers from different machines or
from the nightly host don't judge a fix.
## 0. Preconditions
- `jdtls` on PATH and `JAVA_HOME` pointing at a JDK of the pinned `JAVA` version (projects
compile against it, and the JDK scenarios' results are in its sources; jdtls itself runs on
a newer installed JDK when it needs one). Pinned versions: `.github/tool-versions.env`. A
mismatch prints a warning and the report records both: say so in the PR when it applies.
- First run builds `cargo build --release -p ide-host --bin openidea-server` and the perf
bundle; later iterations of a frontend-only change can use `--no-build` after a
`pnpm exec vite build --mode perf --outDir target/eval/dist`, a backend-only one after the
cargo build alone.
- Run evals one at a time on this machine, with nothing heavy (builds, other evals) beside them.
## 1. Baseline on the base commit
```sh
git switch --detach origin/main # or the fix branch's merge base
pnpm eval --save-baseline # target/eval/baseline.json, this machine only
```
Read `target/eval/report.json`. Each verdict has `status`, `failures` (`kind`: `error`,
`timeout`, `degraded`, `correctness`, `budget`, `golden`), `regressions`, `repro` (the command
that reruns it) and, for failures, `diagnostics`: every command sent to the server in the
slowest run's window (`cmd`, `start`, `ms`, `ok`, `bytes`, `items`), background tasks, server
status changes and the server's log tail.
## 2. Pick one problem
Worst first: wrong or missing results and silent degradation, then timeouts, then the
feedback budget (100 ms), then the slowest `completeMs`. One problem per PR. Say which
scenario and metric, with its value vs budget.
## 3. Diagnose from the timeline
- Commands in the window ordered by `start`: what ran, what waited on what, what's serial
that could overlap, which request returned many `items`/`bytes`, which failed.
- `feedbackMs` far above 16 ms with a fast `firstResultMs`: the UI shows nothing while it
waits (a hint or popup that only appears after a delay).
- A task still running across the window: background work competing with the request.
- For the frontend side rerun with `--trace` and open `target/eval/traces/<id>@<cond>.json`
in Chrome's Performance panel.
- Name the cause at class level (as in `issue-fix`): it usually affects sibling scenarios.
## 4. Fix, then rerun that scenario until it passes
```sh
pnpm eval --only <scenario> --condition <warm|cold> # the verdict's `repro`
```
Correctness is enforced, so a faster answer that drops, truncates or serves stale results
fails. Don't trade a budget miss for a degraded answer.
## 5. Full suite, then PR
```sh
pnpm eval # vs the baseline from step 1: no new failures or regressions
```
PR per `issue-fix` §6, plus in **How it was verified** the before / after table from
`target/eval/report.md` (both on this machine, commits named) and the tool versions line.
## Guard rails
- Goldens (`evals/*/golden/`) and budgets (`evals/budgets.json`) are human-owned. Don't edit
them, and don't run `--update-golden`, to make a scenario pass. When a golden looks wrong
(a tool bump, a deliberate behaviour change), stop and say so: a PR that touches them gets
`needs-decision`.
- Don't weaken a scenario's checks or timeouts in `evals/<language>/scenarios.ts` to pass.
Adding a scenario (for a case the sweep or a user found) is fine; record its golden with
`--update-golden --only <id>` and ask a person to review the file.
- A flaky timing (passes on rerun) is not a regression: rerun once before acting on one.
-1
View File
@@ -1 +0,0 @@
../../.agents/skills/eval
-2
View File
@@ -1,2 +0,0 @@
custom:
- https://github.com/l0ng-ai/openidea#sponsor
-3
View File
@@ -3,6 +3,3 @@ contact_links:
- name: Download the latest release
url: https://github.com/l0ng-ai/openidea/releases/latest
about: Many problems are already fixed. Please check the latest version before reporting.
- name: Report a security vulnerability
url: https://github.com/l0ng-ai/openidea/security/advisories/new
about: Please report security problems privately, not in a public issue.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 86 KiB

-14
View File
@@ -1,14 +0,0 @@
# Language server and tool versions for Language smoke (.github/workflows/language-smoke.yml)
# and the code-intelligence evals (pnpm eval). Pinned so a release can't fail an unrelated
# PR or move a golden result; bump them deliberately (and re-record goldens: pnpm eval --update-golden).
VTSLS=0.3.0
TYPESCRIPT=6.0.3
BASEDPYRIGHT=1.40.2
RUFF=0.17.3
GOPLS=v0.24.0
JDTLS=1.63.0-202610191322
KOTLIN_LSP=263.7084.0
GRADLE=8.14.3
INTELEPHENSE=1.18.5
# The JDK projects compile against (JAVA_HOME): library navigation opens its sources.
JAVA=21
+1 -1
View File
@@ -103,7 +103,7 @@ jobs:
if: runner.os == 'Linux'
run: git diff --exit-code src/bindings.ts
# The remote development server ships as a static Linux binary (release-build.yml):
# The remote development server ships as a static Linux binary (release.yml):
# keep that build working.
server:
name: Remote server (linux musl)
+12 -20
View File
@@ -24,7 +24,6 @@ on:
- "package.json"
- "pnpm-lock.yaml"
- ".github/workflows/language-smoke.yml"
- ".github/tool-versions.env"
pull_request:
branches: [main]
types: [opened, synchronize, reopened]
@@ -39,6 +38,15 @@ concurrency:
env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
# Pinned so a server release can't fail an unrelated PR; bump them deliberately.
VTSLS: 0.3.0
TYPESCRIPT: 6.0.3
BASEDPYRIGHT: 1.40.2
RUFF: 0.17.3
GOPLS: v0.24.0
JDTLS: 1.63.0-202610191322
KOTLIN_LSP: 263.7084.0
GRADLE: 8.14.3
jobs:
smoke:
@@ -46,14 +54,11 @@ jobs:
strategy:
fail-fast: false
matrix:
language: [typescript, python, rust, go, java, kotlin, php, c, cpp]
# C/C++ debug with GDB, whose DAP interpreter needs 14+: 22.04 ships 12.1, 24.04 ships 15.
runs-on: ${{ (matrix.language == 'c' || matrix.language == 'cpp') && 'ubuntu-24.04' || 'ubuntu-22.04' }}
language: [typescript, python, rust, go, java, kotlin]
runs-on: ubuntu-22.04
timeout-minutes: 40
steps:
- uses: actions/checkout@v4
# Tool versions, shared with the evals (pnpm eval).
- run: grep -E '^[A-Z0-9_]+=' .github/tool-versions.env >> "$GITHUB_ENV"
- uses: dtolnay/rust-toolchain@stable
with: { components: "rust-analyzer, rustfmt" }
# One build cache for every language; main saves it.
@@ -69,12 +74,6 @@ jobs:
- if: matrix.language == 'typescript'
run: npm install --global "@vtsls/language-server@$VTSLS" "typescript@$TYPESCRIPT"
# The smoke builds the c-app fixture and debugs it with GDB's own DAP interpreter.
- if: matrix.language == 'c' || matrix.language == 'cpp'
run: |
sudo apt-get update
sudo apt-get install -y gdb
- if: matrix.language == 'python'
uses: actions/setup-python@v5
with: { python-version: "3.12" }
@@ -89,7 +88,7 @@ jobs:
- if: matrix.language == 'java' || matrix.language == 'kotlin'
uses: actions/setup-java@v4
with: { distribution: temurin, java-version: "${{ env.JAVA }}" }
with: { distribution: temurin, java-version: "21" }
- if: matrix.language == 'java'
run: |
mkdir -p "$RUNNER_TEMP/jdtls"
@@ -109,13 +108,6 @@ jobs:
ln -s "$RUNNER_TEMP/kotlin-lsp/kotlin-lsp.sh" "$RUNNER_TEMP/bin/kotlin-lsp"
echo "$RUNNER_TEMP/bin" >> "$GITHUB_PATH"
# Xdebug for the debug smoke; Composer installs the fixture's Pest for the run smoke.
- if: matrix.language == 'php'
uses: shivammathur/setup-php@v2
with: { php-version: "8.5", coverage: xdebug, tools: composer }
- if: matrix.language == 'php'
run: npm install --global "intelephense@$INTELEPHENSE"
- run: cargo test --locked -p ide-host --test language_smoke --no-run
- run: cargo test --locked -p ide-host --test language_smoke -- --ignored --exact ${{ matrix.language }}
- if: matrix.language == 'java'
-119
View File
@@ -1,119 +0,0 @@
name: Nightly
# Twice a day, builds the newest main commit that passed CI as a prerelease tagged with its own
# version, so remote development finds the server of that version. The `nightly` release holds
# the manifest the nightly app updates from; its tag marks the commit last built, and a run with
# nothing newer stops there. Only the latest builds are kept.
on:
schedule:
- cron: "17 2,14 * * *"
workflow_dispatch:
inputs:
force:
description: "Build even if nothing is newer than the last nightly"
type: boolean
default: false
permissions:
contents: read
concurrency:
group: nightly
env:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
jobs:
prepare:
permissions:
actions: read
contents: write
if: github.repository == 'l0ng-ai/openidea'
runs-on: ubuntu-latest
outputs:
sha: ${{ steps.commit.outputs.sha }}
tag: ${{ steps.release.outputs.tag }}
release_id: ${{ steps.release.outputs.id }}
steps:
- id: commit
env:
FORCE: ${{ inputs.force }}
run: |
sha="$(gh run list --workflow ci.yml --branch main --event push --status success --limit 1 \
--json headSha --jq '.[0].headSha // ""')"
if [ -z "$sha" ]; then
echo "::notice::no main commit has passed CI"
exit 0
fi
# No `nightly` tag yet (the first run): compare fails and the build goes on.
status="$(gh api "repos/$GH_REPO/compare/nightly...$sha" --jq .status 2>/dev/null)" || status=ahead
if [ "$status" != ahead ] && [ "$FORCE" != true ]; then
echo "::notice::$sha is not newer than the last nightly"
exit 0
fi
echo "sha=$sha" >> "$GITHUB_OUTPUT"
- uses: actions/checkout@v5
if: steps.commit.outputs.sha != ''
with: { ref: "${{ steps.commit.outputs.sha }}" }
- uses: actions/setup-node@v5
if: steps.commit.outputs.sha != ''
with: { node-version: 22 }
- uses: dtolnay/rust-toolchain@stable
if: steps.commit.outputs.sha != ''
- id: release
if: steps.commit.outputs.sha != ''
env:
SHA: ${{ steps.commit.outputs.sha }}
run: |
version="$(node scripts/nightly-prepare.ts "$(date -u +%Y%m%d%H%M)")"
tag="v$version"
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git commit -am "chore: $tag"
git tag "$tag"
git push origin "$tag"
id="$(gh api -X POST "repos/$GH_REPO/releases" -f tag_name="$tag" -f name="OpenIDEA Nightly $version" \
-F draft=true -F prerelease=true -f body="Built from $SHA." --jq .id)"
echo "tag=$tag" >> "$GITHUB_OUTPUT"
echo "id=$id" >> "$GITHUB_OUTPUT"
binaries:
permissions:
contents: write
needs: prepare
if: needs.prepare.outputs.tag != ''
uses: ./.github/workflows/release-build.yml
with:
tag: ${{ needs.prepare.outputs.tag }}
release_id: ${{ needs.prepare.outputs.release_id }}
secrets: inherit
publish:
permissions:
contents: write
needs: [prepare, binaries]
runs-on: ubuntu-latest
env:
TAG: ${{ needs.prepare.outputs.tag }}
SHA: ${{ needs.prepare.outputs.sha }}
KEEP: 14
steps:
- name: Publish the build
run: gh release edit "$TAG" --tag "$TAG" --draft=false
- name: Point the updater at it
run: |
gh release download "$TAG" -p latest.json -D "$RUNNER_TEMP"
gh release view nightly >/dev/null 2>&1 ||
gh release create nightly --prerelease --latest=false --target "$SHA" --title "OpenIDEA Nightly" \
--notes "The update manifest of the nightly builds, and the commit last built."
gh release upload nightly "$RUNNER_TEMP/latest.json" --clobber
gh api -X PATCH "repos/$GH_REPO/git/refs/tags/nightly" -f sha="$SHA" -F force=true --silent
# Failed builds' drafts go too, once they are this old.
- name: Delete old builds
run: |
gh release list --limit 200 --json tagName,createdAt --jq \
'[.[] | select(.tagName | contains("-nightly."))] | sort_by(.createdAt) | reverse | .[env.KEEP | tonumber:][].tagName' |
while read -r tag; do
gh release delete "$tag" --cleanup-tag --yes
done
-283
View File
@@ -1,283 +0,0 @@
name: Release build
# Builds a tag's desktop apps, remote servers and updater manifest into its draft release.
# Stable releases (release.yml) and nightly builds (nightly.yml) both run it.
on:
workflow_call:
inputs:
tag: { type: string, required: true }
release_id: { type: string, required: true }
permissions:
contents: read
env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
TAG: ${{ inputs.tag }}
jobs:
plugin-keys:
permissions: {}
runs-on: ubuntu-latest
outputs:
keys: ${{ steps.keys.outputs.keys }}
steps:
# The official plugins' public key (from PLUGIN_SIGNING_KEY, the seed
# `openidea-plugin keygen` writes), built into the app: their packages
# must carry its signature whatever the marketplace index lists. Without
# the secret the app takes the index's word, as development builds do.
- id: keys
env:
SIGNING_KEY: ${{ secrets.PLUGIN_SIGNING_KEY }}
run: |
if [ -n "$SIGNING_KEY" ]; then
seed="$(printf %s "$SIGNING_KEY" | base64 -d | od -An -tx1 | tr -d ' \n')"
key="$(printf '302e020100300506032b657004220420%s' "$seed" | xxd -r -p |
openssl pkey -inform DER -pubout -outform DER | tail -c 32 | base64)"
echo "keys=$key" >> "$GITHUB_OUTPUT"
fi
build:
permissions:
contents: write
needs: plugin-keys
name: Build (${{ matrix.name }})
env:
OPENIDEA_OFFICIAL_PLUGIN_KEYS: ${{ needs.plugin-keys.outputs.keys }}
strategy:
fail-fast: false
matrix:
include:
- name: macOS arm64
os: macos-latest
target: aarch64-apple-darwin
- name: macOS x64
os: macos-latest
target: x86_64-apple-darwin
- name: Linux x64
os: ubuntu-22.04
target: x86_64-unknown-linux-gnu
- name: Linux arm64
os: ubuntu-22.04-arm
target: aarch64-unknown-linux-gnu
- name: Windows arm64
os: windows-11-arm
target: aarch64-pc-windows-msvc
- name: Windows x64
os: windows-latest
target: x86_64-pc-windows-msvc
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v5
with: { ref: "${{ env.TAG }}" }
- uses: pnpm/action-setup@v5
with: { version: 11 }
- uses: actions/setup-node@v5
with: { node-version: 22, cache: pnpm }
- uses: dtolnay/rust-toolchain@stable
with: { targets: "${{ matrix.target }}" }
- uses: Swatinem/rust-cache@v2
with: { key: "${{ matrix.target }}" }
- name: Linux deps
if: runner.os == 'Linux'
run: |
sudo apt-get update
sudo apt-get install -y libwebkit2gtk-4.1-dev libayatana-appindicator3-dev librsvg2-dev libxdo-dev libssl-dev patchelf
- run: pnpm install --frozen-lockfile
# Signing is optional: only secrets that are set get exported, so forks and
# unconfigured repos produce unsigned builds instead of failing.
- name: Signing env
shell: bash
env:
APPLE_CERTIFICATE: ${{ runner.os == 'macOS' && secrets.APPLE_CERTIFICATE || '' }}
APPLE_CERTIFICATE_PASSWORD: ${{ runner.os == 'macOS' && secrets.APPLE_CERTIFICATE_PASSWORD || '' }}
APPLE_SIGNING_IDENTITY: ${{ runner.os == 'macOS' && secrets.APPLE_SIGNING_IDENTITY || '' }}
APPLE_ID: ${{ runner.os == 'macOS' && secrets.APPLE_ID || '' }}
APPLE_PASSWORD: ${{ runner.os == 'macOS' && secrets.APPLE_PASSWORD || '' }}
APPLE_TEAM_ID: ${{ runner.os == 'macOS' && secrets.APPLE_TEAM_ID || '' }}
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
run: |
for v in APPLE_CERTIFICATE APPLE_CERTIFICATE_PASSWORD APPLE_SIGNING_IDENTITY APPLE_ID \
APPLE_PASSWORD APPLE_TEAM_ID TAURI_SIGNING_PRIVATE_KEY TAURI_SIGNING_PRIVATE_KEY_PASSWORD; do
[ -n "${!v}" ] && { echo "$v<<__EOF__"; echo "${!v}"; echo "__EOF__"; } >> "$GITHUB_ENV"
done
true
- uses: tauri-apps/tauri-action@v0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
releaseId: ${{ inputs.release_id }}
tagName: ${{ env.TAG }}
args: --target ${{ matrix.target }}
# The parallel builds would race on one latest.json; `updater-manifest` writes it once.
uploadUpdaterJson: false
# tauri-action notarizes the .app but not the DMG around it; notarize and
# staple the DMG too, then replace the uploaded asset.
- name: Notarize DMG
if: runner.os == 'macOS' && env.APPLE_ID != ''
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
for dmg in target/${{ matrix.target }}/release/bundle/dmg/*.dmg; do
xcrun notarytool submit "$dmg" --apple-id "$APPLE_ID" --password "$APPLE_PASSWORD" \
--team-id "$APPLE_TEAM_ID" --wait
xcrun stapler staple "$dmg"
# Release assets have no spaces: GitHub stored the first upload with dots.
asset="$RUNNER_TEMP/$(basename "$dmg" | tr ' ' .)"
cp "$dmg" "$asset"
gh release upload "$TAG" "$asset" --clobber
done
# Portable Windows builds (#1857): on Windows Tauri embeds the
# frontend in the exe and WebView2 is a system component, so zipping
# the raw exe gives an unpack-and-run alternative to the installers.
- name: Portable zip
if: runner.os == 'Windows'
shell: pwsh
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
$target = "${{ matrix.target }}"
$arch = if ($target -like "x86_64*") { "x64" } else { "arm64" }
$version = $env:TAG -replace '^v', ''
$asset = "OpenIDEA_${version}_windows_${arch}_portable.zip"
$dir = "target/$target/release"
$files = @("$dir/openidea.exe")
if (Test-Path "$dir/WebView2Loader.dll") {
$files += "$dir/WebView2Loader.dll"
}
Compress-Archive -Path $files -DestinationPath $asset -Force
$hash = (Get-FileHash $asset -Algorithm SHA256).Hash.ToLower()
"$hash $asset" | Out-File -Encoding ascii "$asset.sha256"
gh release upload $env:TAG $asset "$asset.sha256" --clobber
# The remote development server, per host platform it can run on. The desktop
# app downloads the one for the host (ide-tools `server`) and installs it over SSH.
# Linux builds are static (musl), so they run whatever the host's glibc.
server:
permissions:
contents: write
name: Server (${{ matrix.os }}-${{ matrix.arch }})
strategy:
fail-fast: false
matrix:
include:
- { runner: ubuntu-22.04, target: x86_64-unknown-linux-musl, os: linux, arch: x86_64 }
- { runner: ubuntu-24.04-arm, target: aarch64-unknown-linux-musl, os: linux, arch: aarch64 }
- { runner: macos-latest, target: aarch64-apple-darwin, os: macos, arch: aarch64 }
- { runner: macos-latest, target: x86_64-apple-darwin, os: macos, arch: x86_64 }
runs-on: ${{ matrix.runner }}
steps:
- uses: actions/checkout@v5
with: { ref: "${{ env.TAG }}" }
- uses: dtolnay/rust-toolchain@stable
with: { targets: "${{ matrix.target }}" }
- uses: Swatinem/rust-cache@v2
with: { key: "server-${{ matrix.target }}" }
- name: musl toolchain
if: runner.os == 'Linux'
run: sudo apt-get update && sudo apt-get install -y musl-tools
- name: Build
run: |
# musl-tools only ships `musl-gcc`; cc-rs would look for `<arch>-linux-musl-gcc`.
if [ "$RUNNER_OS" = Linux ]; then export CC_${TARGET//-/_}=musl-gcc; fi
cargo build --release --locked -p ide-host --bin openidea-server --target "$TARGET"
env:
TARGET: ${{ matrix.target }}
- name: Upload
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
ASSET: openidea-server-${{ matrix.os }}-${{ matrix.arch }}.gz
run: |
gzip -9 -c target/${{ matrix.target }}/release/openidea-server > "$ASSET"
if command -v sha256sum >/dev/null; then sha256sum "$ASSET"; else shasum -a 256 "$ASSET"; fi > "$ASSET.sha256"
gh release upload "$TAG" "$ASSET" "$ASSET.sha256" --clobber
# The updater manifest (latest.json), built once from the signed bundles every build uploaded.
# Fails when a platform is missing, so a release never ships a manifest without it.
updater-manifest:
permissions:
contents: write
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/github-script@v8
env:
RELEASE_ID: ${{ inputs.release_id }}
with:
script: |
const { owner, repo } = context.repo;
const release_id = Number(process.env.RELEASE_ID);
const tag = process.env.TAG;
const assets = await github.paginate(github.rest.repos.listReleaseAssets, { owner, repo, release_id });
const sigs = assets.filter((a) => a.name.endsWith(".sig"));
if (sigs.length === 0) {
core.notice("no signed bundles (TAURI_SIGNING_PRIVATE_KEY unset): skipping latest.json");
return;
}
// Bundle suffix → updater platform keys, as tauri-action names them.
// Nightly versions don't fit MSI or RPM, so those builds offer NSIS on Windows.
const nightly = tag.includes("-nightly.");
const windows = (arch, msi, nsis) => nightly
? [[nsis, [`windows-${arch}`, `windows-${arch}-nsis`]]]
: [[msi, [`windows-${arch}`, `windows-${arch}-msi`]], [nsis, [`windows-${arch}-nsis`]]];
const kinds = [
[/_aarch64\.AppImage$/, ["linux-aarch64", "linux-aarch64-appimage"]],
[/_arm64\.deb$/, ["linux-aarch64-deb"]],
...(nightly ? [] : [[/\.aarch64\.rpm$/, ["linux-aarch64-rpm"]]]),
[/_amd64\.AppImage$/, ["linux-x86_64", "linux-x86_64-appimage"]],
[/_amd64\.deb$/, ["linux-x86_64-deb"]],
...(nightly ? [] : [[/\.x86_64\.rpm$/, ["linux-x86_64-rpm"]]]),
[/_aarch64\.app\.tar\.gz$/, ["darwin-aarch64", "darwin-aarch64-app"]],
[/_x64\.app\.tar\.gz$/, ["darwin-x86_64", "darwin-x86_64-app"]],
...windows("aarch64", /_arm64_en-US\.msi$/, /_arm64-setup\.exe$/),
...windows("x86_64", /_x64_en-US\.msi$/, /_x64-setup\.exe$/),
];
const platforms = {};
for (const sig of sigs) {
const bundle = sig.name.slice(0, -".sig".length);
const kind = kinds.find(([re]) => re.test(bundle));
if (!kind) continue;
const { data } = await github.rest.repos.getReleaseAsset({
owner, repo, asset_id: sig.id, headers: { accept: "application/octet-stream" },
});
const signature = Buffer.from(data).toString("utf8").trim();
const url = `https://github.com/${owner}/${repo}/releases/download/${tag}/${encodeURIComponent(bundle)}`;
for (const key of kind[1]) platforms[key] = { signature, url };
}
const missing = kinds.flatMap(([, keys]) => keys).filter((k) => !platforms[k]);
if (missing.length) {
core.setFailed(`latest.json would miss ${missing.join(", ")}`);
return;
}
// The update's notes: the release body without its Download section. That
// section only matters on the release page, and What's new follows it.
const { data: release } = await github.rest.repos.getRelease({ owner, repo, release_id });
const withoutDownloadSection = (body) => {
const kept = [];
let skip = 0;
for (const line of body.split(/\r?\n/)) {
const heading = /^(#{1,6})\s+(.*)$/.exec(line);
if (heading) {
const level = heading[1].length;
if (!skip && level <= 3 && /^download$/i.test(heading[2].trim())) {
skip = level;
continue;
}
if (skip && level <= skip) skip = 0;
}
if (!skip) kept.push(line);
}
return kept.join("\n").trim();
};
const notes = withoutDownloadSection(String(release.body ?? ""));
const manifest = { version: tag.replace(/^v/, ""), notes, pub_date: new Date().toISOString(), platforms };
const old = assets.find((a) => a.name === "latest.json");
if (old) await github.rest.repos.deleteReleaseAsset({ owner, repo, asset_id: old.id });
await github.rest.repos.uploadReleaseAsset({
owner, repo, release_id, name: "latest.json",
data: JSON.stringify(manifest, null, 2),
headers: { "content-type": "application/json" },
});
core.info(`latest.json: ${Object.keys(platforms).sort().join(", ")}`);
+229 -24
View File
@@ -1,7 +1,7 @@
name: Release
on:
push:
tags: ["v*", "!v*-nightly.*"]
tags: ["v*"]
workflow_dispatch:
inputs:
tag:
@@ -9,7 +9,7 @@ on:
required: true
permissions:
contents: read
contents: write
concurrency:
group: release-${{ github.event.inputs.tag || github.ref_name }}
@@ -23,12 +23,10 @@ env:
jobs:
# Creates (or reuses) one draft release that every platform build uploads into.
create-release:
if: ${{ !contains(github.event.inputs.tag || github.ref_name, '-nightly.') }}
permissions:
contents: write
runs-on: ubuntu-latest
outputs:
release_id: ${{ steps.release.outputs.id }}
official_plugin_keys: ${{ steps.plugin-keys.outputs.keys }}
steps:
- uses: actions/checkout@v5
with: { ref: "${{ env.TAG }}" }
@@ -49,6 +47,20 @@ jobs:
echo "::error::plugin API $(api wit) ships without a snapshot ($snapshot is $(api "$snapshot")): copy wit/*.wit to $snapshot/"
exit 1
fi
# The official plugins' public key (from PLUGIN_SIGNING_KEY, the seed
# `openidea-plugin keygen` writes), built into the app: their packages
# must carry its signature whatever the marketplace index lists. Without
# the secret the app takes the index's word, as development builds do.
- id: plugin-keys
env:
SIGNING_KEY: ${{ secrets.PLUGIN_SIGNING_KEY }}
run: |
if [ -n "$SIGNING_KEY" ]; then
seed="$(printf %s "$SIGNING_KEY" | base64 -d | od -An -tx1 | tr -d ' \n')"
key="$(printf '302e020100300506032b657004220420%s' "$seed" | xxd -r -p |
openssl pkey -inform DER -pubout -outform DER | tail -c 32 | base64)"
echo "keys=$key" >> "$GITHUB_OUTPUT"
fi
- id: release
uses: actions/github-script@v8
with:
@@ -62,26 +74,161 @@ jobs:
core.setOutput("id", existing.id);
return;
}
// Nightly builds are prereleases: the notes span stable release to stable release.
const previous = releases.find((r) => !r.draft && !r.prerelease);
const { data: notes } = await github.rest.repos.generateReleaseNotes({
owner, repo, tag_name: tag, previous_tag_name: previous?.tag_name,
});
const { data } = await github.rest.repos.createRelease({
owner, repo, tag_name: tag, name: `OpenIDEA ${tag}`,
draft: true, prerelease: tag.includes("-"), body: notes.body,
draft: true, prerelease: tag.includes("-"), generate_release_notes: true,
});
core.setOutput("id", data.id);
binaries:
permissions:
contents: write
build:
needs: create-release
uses: ./.github/workflows/release-build.yml
with:
tag: ${{ github.event.inputs.tag || github.ref_name }}
release_id: ${{ needs.create-release.outputs.release_id }}
secrets: inherit
name: Build (${{ matrix.name }})
env:
OPENIDEA_OFFICIAL_PLUGIN_KEYS: ${{ needs.create-release.outputs.official_plugin_keys }}
strategy:
fail-fast: false
matrix:
include:
- name: macOS arm64
os: macos-latest
target: aarch64-apple-darwin
- name: macOS x64
os: macos-latest
target: x86_64-apple-darwin
- name: Linux x64
os: ubuntu-22.04
target: x86_64-unknown-linux-gnu
- name: Linux arm64
os: ubuntu-22.04-arm
target: aarch64-unknown-linux-gnu
- name: Windows arm64
os: windows-11-arm
target: aarch64-pc-windows-msvc
- name: Windows x64
os: windows-latest
target: x86_64-pc-windows-msvc
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v5
with: { ref: "${{ env.TAG }}" }
- uses: pnpm/action-setup@v5
with: { version: 11 }
- uses: actions/setup-node@v5
with: { node-version: 22, cache: pnpm }
- uses: dtolnay/rust-toolchain@stable
with: { targets: "${{ matrix.target }}" }
- uses: Swatinem/rust-cache@v2
with: { key: "${{ matrix.target }}" }
- name: Linux deps
if: runner.os == 'Linux'
run: |
sudo apt-get update
sudo apt-get install -y libwebkit2gtk-4.1-dev libayatana-appindicator3-dev librsvg2-dev libxdo-dev libssl-dev patchelf
- run: pnpm install --frozen-lockfile
# Signing is optional: only secrets that are set get exported, so forks and
# unconfigured repos produce unsigned builds instead of failing.
- name: Signing env
shell: bash
env:
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
APPLE_CERTIFICATE_PASSWORD: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
APPLE_SIGNING_IDENTITY: ${{ secrets.APPLE_SIGNING_IDENTITY }}
APPLE_ID: ${{ secrets.APPLE_ID }}
APPLE_PASSWORD: ${{ secrets.APPLE_PASSWORD }}
APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }}
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
run: |
for v in APPLE_CERTIFICATE APPLE_CERTIFICATE_PASSWORD APPLE_SIGNING_IDENTITY APPLE_ID \
APPLE_PASSWORD APPLE_TEAM_ID TAURI_SIGNING_PRIVATE_KEY TAURI_SIGNING_PRIVATE_KEY_PASSWORD; do
[ -n "${!v}" ] && { echo "$v<<__EOF__"; echo "${!v}"; echo "__EOF__"; } >> "$GITHUB_ENV"
done
true
- uses: tauri-apps/tauri-action@v0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
releaseId: ${{ needs.create-release.outputs.release_id }}
tagName: ${{ env.TAG }}
args: --target ${{ matrix.target }}
# The parallel builds would race on one latest.json; `updater-manifest` writes it once.
uploadUpdaterJson: false
# tauri-action notarizes the .app but not the DMG around it; notarize and
# staple the DMG too, then replace the uploaded asset.
- name: Notarize DMG
if: runner.os == 'macOS' && env.APPLE_ID != ''
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
for dmg in target/${{ matrix.target }}/release/bundle/dmg/*.dmg; do
xcrun notarytool submit "$dmg" --apple-id "$APPLE_ID" --password "$APPLE_PASSWORD" \
--team-id "$APPLE_TEAM_ID" --wait
xcrun stapler staple "$dmg"
gh release upload "$TAG" "$dmg" --clobber
done
# Portable Windows builds (#1857): on Windows Tauri embeds the
# frontend in the exe and WebView2 is a system component, so zipping
# the raw exe gives an unpack-and-run alternative to the installers.
- name: Portable zip
if: runner.os == 'Windows'
shell: pwsh
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
$target = "${{ matrix.target }}"
$arch = if ($target -like "x86_64*") { "x64" } else { "arm64" }
$version = $env:TAG -replace '^v', ''
$asset = "OpenIDEA_${version}_windows_${arch}_portable.zip"
$dir = "target/$target/release"
$files = @("$dir/openidea.exe")
if (Test-Path "$dir/WebView2Loader.dll") {
$files += "$dir/WebView2Loader.dll"
}
Compress-Archive -Path $files -DestinationPath $asset -Force
$hash = (Get-FileHash $asset -Algorithm SHA256).Hash.ToLower()
"$hash $asset" | Out-File -Encoding ascii "$asset.sha256"
gh release upload $env:TAG $asset "$asset.sha256" --clobber
# The remote development server, per host platform it can run on. The desktop
# app downloads the one for the host (ide-tools `server`) and installs it over SSH.
# Linux builds are static (musl), so they run whatever the host's glibc.
server:
needs: create-release
name: Server (${{ matrix.os }}-${{ matrix.arch }})
strategy:
fail-fast: false
matrix:
include:
- { runner: ubuntu-22.04, target: x86_64-unknown-linux-musl, os: linux, arch: x86_64 }
- { runner: ubuntu-24.04-arm, target: aarch64-unknown-linux-musl, os: linux, arch: aarch64 }
- { runner: macos-latest, target: aarch64-apple-darwin, os: macos, arch: aarch64 }
- { runner: macos-latest, target: x86_64-apple-darwin, os: macos, arch: x86_64 }
runs-on: ${{ matrix.runner }}
steps:
- uses: actions/checkout@v5
with: { ref: "${{ env.TAG }}" }
- uses: dtolnay/rust-toolchain@stable
with: { targets: "${{ matrix.target }}" }
- uses: Swatinem/rust-cache@v2
with: { key: "server-${{ matrix.target }}" }
- name: musl toolchain
if: runner.os == 'Linux'
run: sudo apt-get update && sudo apt-get install -y musl-tools
- name: Build
run: |
# musl-tools only ships `musl-gcc`; cc-rs would look for `<arch>-linux-musl-gcc`.
if [ "$RUNNER_OS" = Linux ]; then export CC_${TARGET//-/_}=musl-gcc; fi
cargo build --release --locked -p ide-host --bin openidea-server --target "$TARGET"
env:
TARGET: ${{ matrix.target }}
- name: Upload
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
ASSET: openidea-server-${{ matrix.os }}-${{ matrix.arch }}.gz
run: |
gzip -9 -c target/${{ matrix.target }}/release/openidea-server > "$ASSET"
if command -v sha256sum >/dev/null; then sha256sum "$ASSET"; else shasum -a 256 "$ASSET"; fi > "$ASSET.sha256"
gh release upload "$TAG" "$ASSET" "$ASSET.sha256" --clobber
# The official plugins, signed, and the marketplace index released apps read
# (releases/latest/download/plugins-index.json). They are not in the
@@ -93,8 +240,6 @@ jobs:
# openidea-plugin tool for Linux x64, which that workflow and the registry's
# checks run.
plugins:
permissions:
contents: write
needs: create-release
name: Official plugins
runs-on: ubuntu-22.04
@@ -158,10 +303,70 @@ jobs:
gh release upload "$TAG" "$out"/*.oiplugin "$out/plugins-index.json" \
"$out/openidea-plugin-linux-x86_64.gz" --clobber
# The updater manifest (latest.json), built once from the signed bundles every build uploaded.
# Fails when a platform is missing, so a release never ships a manifest without it.
updater-manifest:
needs: [create-release, build]
runs-on: ubuntu-latest
steps:
- uses: actions/github-script@v8
env:
RELEASE_ID: ${{ needs.create-release.outputs.release_id }}
with:
script: |
const { owner, repo } = context.repo;
const release_id = Number(process.env.RELEASE_ID);
const tag = process.env.TAG;
const assets = await github.paginate(github.rest.repos.listReleaseAssets, { owner, repo, release_id });
const sigs = assets.filter((a) => a.name.endsWith(".sig"));
if (sigs.length === 0) {
core.notice("no signed bundles (TAURI_SIGNING_PRIVATE_KEY unset): skipping latest.json");
return;
}
// Bundle suffix → updater platform keys, as tauri-action names them.
const kinds = [
[/_aarch64\.AppImage$/, ["linux-aarch64", "linux-aarch64-appimage"]],
[/_arm64\.deb$/, ["linux-aarch64-deb"]],
[/\.aarch64\.rpm$/, ["linux-aarch64-rpm"]],
[/_amd64\.AppImage$/, ["linux-x86_64", "linux-x86_64-appimage"]],
[/_amd64\.deb$/, ["linux-x86_64-deb"]],
[/\.x86_64\.rpm$/, ["linux-x86_64-rpm"]],
[/_aarch64\.app\.tar\.gz$/, ["darwin-aarch64", "darwin-aarch64-app"]],
[/_x64\.app\.tar\.gz$/, ["darwin-x86_64", "darwin-x86_64-app"]],
[/_arm64_en-US\.msi$/, ["windows-aarch64", "windows-aarch64-msi"]],
[/_arm64-setup\.exe$/, ["windows-aarch64-nsis"]],
[/_x64_en-US\.msi$/, ["windows-x86_64", "windows-x86_64-msi"]],
[/_x64-setup\.exe$/, ["windows-x86_64-nsis"]],
];
const platforms = {};
for (const sig of sigs) {
const bundle = sig.name.slice(0, -".sig".length);
const kind = kinds.find(([re]) => re.test(bundle));
if (!kind) continue;
const { data } = await github.rest.repos.getReleaseAsset({
owner, repo, asset_id: sig.id, headers: { accept: "application/octet-stream" },
});
const signature = Buffer.from(data).toString("utf8").trim();
const url = `https://github.com/${owner}/${repo}/releases/download/${tag}/${encodeURIComponent(bundle)}`;
for (const key of kind[1]) platforms[key] = { signature, url };
}
const missing = kinds.flatMap(([, keys]) => keys).filter((k) => !platforms[k]);
if (missing.length) {
core.setFailed(`latest.json would miss ${missing.join(", ")}`);
return;
}
const manifest = { version: tag.replace(/^v/, ""), notes: "", pub_date: new Date().toISOString(), platforms };
const old = assets.find((a) => a.name === "latest.json");
if (old) await github.rest.repos.deleteReleaseAsset({ owner, repo, asset_id: old.id });
await github.rest.repos.uploadReleaseAsset({
owner, repo, release_id, name: "latest.json",
data: JSON.stringify(manifest, null, 2),
headers: { "content-type": "application/json" },
});
core.info(`latest.json: ${Object.keys(platforms).sort().join(", ")}`);
publish:
permissions:
contents: write
needs: [create-release, binaries, plugins]
needs: [create-release, build, server, plugins, updater-manifest]
runs-on: ubuntu-latest
steps:
- uses: actions/github-script@v8
-1
View File
@@ -40,7 +40,6 @@ CLAUDE.local.md
.claude/skills/*
!.claude/skills/verify
!.claude/skills/issue-fix
!.claude/skills/eval
# Local reference screenshots
pycharm-screenshots/
+1 -3
View File
@@ -9,12 +9,10 @@
"dist",
"target",
"src/bindings.ts",
"evals/*/golden",
"src/editor/core/cm/angular/parser*.ts",
"src/editor/core/cm/jsp/parser*.ts",
"src/editor/core/cm/astro/parser*.ts",
"src/editor/core/cm/django/parser*.ts",
"plugins/themes.md",
"docs/plugins/themes.md",
"pnpm-lock.yaml",
"**/*.toml",
"fixtures",
+4 -14
View File
@@ -21,13 +21,11 @@ Visual style: dark, low-contrast greys, one cool accent
- `crates/ide-host` — the `openidea-server` binary for remote development and the
`openidea-plugin` CLI (plugin starter templates live in its `templates.rs`).
- `wit/` — plugin interface definitions; `plugins/` — sample and bundled plugins.
Plugin docs: `plugins/README.md`, `plugins/themes.md`.
Plugin docs: `plugins/README.md`, `docs/plugins/themes.md`.
- `java/` — JDT LS extension bundles (`java-usages`, `kotlin-bridge`) and the patched
Lombok agent, embedded into the app by `ide-sdk` / `ide-tools`.
- `e2e/` — Playwright UI tests (`pnpm e2e`). `evals/` — code-intelligence evals (`pnpm eval`).
`fixtures/` — sample projects per language for manual and automated testing.
- `docs/` — the Mintlify documentation site (`docs.json` navigation, `en/` and `zh/` pages);
preview with `npx mint dev` in `docs/`.
- `e2e/` — Playwright UI tests (`pnpm e2e`). `fixtures/` — sample projects per language
for manual and automated testing.
## Rules
@@ -59,12 +57,9 @@ Unclear product decisions → `needs-decision` and stop.
## Skills
Do not use Impeccable or its related skills in this repository, including globally installed copies.
Shared agent skills live in `.agents/skills/` (`.claude/skills/` links to them):
`verify` drives the running app through Tauri MCP and captures screenshot evidence;
`issue-fix` takes an issue from reproduction to one PR;
`eval` takes a failing code-intelligence eval to a fix with before / after numbers.
`issue-fix` takes an issue from reproduction to one PR.
## Commands
@@ -84,11 +79,6 @@ Shared agent skills live in `.agents/skills/` (`.claude/skills/` links to them):
`pnpm gen:bindings`.
- `pnpm perf` (UI interaction latency / frames / leaks), `pnpm perf:rust` (criterion
benches vs budgets). Budgets: `perf/budgets.json`.
- `pnpm eval [--only <id>] [--runs n] [--save-baseline]` — Java code intelligence (navigation,
usages, gutter markers) in the production workbench against a release `openidea-server` and
jdtls: time to feedback / first result / complete, results vs `evals/<language>/golden/`.
Report: `target/eval/report.json`. Goldens and `evals/budgets.json` are changed by people only.
Tool versions: `.github/tool-versions.env` (shared with Language smoke).
- `pnpm release:prepare <version>` — the release commit's changes: app version everywhere,
`Cargo.lock`, and the plugin API snapshot when `wit/` moved to a new version.
Generated
+143 -289
View File
File diff suppressed because it is too large. Load diff
+5 -16
View File
@@ -5,7 +5,7 @@ members = ["crates/*", "src-tauri"]
exclude = ["sdk", "plugins"]
[workspace.package]
version = "0.1.43"
version = "0.1.39"
edition = "2024"
rust-version = "1.96"
license = "Apache-2.0"
@@ -43,15 +43,10 @@ ide-ssh = { path = "crates/ide-ssh" }
ide-remote = { path = "crates/ide-remote" }
ide-host = { path = "crates/ide-host" }
ide-mcp = { path = "crates/ide-mcp" }
ide-scaffold = { path = "crates/ide-scaffold" }
ide-notebook = { path = "crates/ide-notebook" }
ide-extension = { path = "crates/ide-extension" }
ide-plugin = { path = "crates/ide-plugin" }
ide-cli = { path = "crates/ide-cli" }
ide-jvm-attach = { path = "crates/ide-jvm-attach" }
ide-jvm-inspect = { path = "crates/ide-jvm-inspect" }
ide-jdwp = { path = "crates/ide-jdwp" }
ide-heap = { path = "crates/ide-heap" }
# serialization & IPC typing
serde = { version = "1", features = ["derive"] }
@@ -68,7 +63,6 @@ parking_lot = "0.12"
rayon = "1"
dashmap = "6"
futures = "0.3"
tokio-tungstenite = { version = "0.28", default-features = false, features = ["connect", "__rustls-tls"] }
# errors & diagnostics
thiserror = "2"
@@ -88,8 +82,7 @@ trash = "5"
blake3 = "1"
dunce = "1"
dirs = "6"
memmap2 = "0.9"
zip = { version = "9.2.3", default-features = false, features = ["deflate"] }
zip = { version = "9", default-features = false, features = ["deflate"] }
# search, index, language
grep-searcher = "0.1"
@@ -110,11 +103,9 @@ portable-pty = "0.9"
reqwest = { version = "0.13", default-features = false, features = ["rustls-no-provider", "http2", "charset", "socks", "gzip", "deflate", "brotli"] }
rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12", "logging"] }
rustls-platform-verifier = "0.7"
# Until SQLx releases its libsqlite3-sys 0.39 update (RUSTSEC-2026-0459).
sqlx = { version = "0.9", git = "https://github.com/l0ng-ai/sqlx", rev = "9894985a1babc7e6bc0437da4fd43e3143184073", default-features = false, features = ["runtime-tokio", "tls-rustls-ring-webpki", "sqlite", "postgres", "mysql"] }
sqlx = { version = "0.9", default-features = false, features = ["runtime-tokio", "tls-rustls-ring-webpki", "sqlite", "postgres", "mysql"] }
tiberius = { version = "0.13", default-features = false, features = ["tds73", "rustls", "sql-browser-tokio"] }
# Pure-Rust thin Oracle driver; pinned to the beta until it stabilizes.
oracledb = "=26.0.0-beta.5"
oracle = { version = "0.6", default-features = false }
bb8 = "0.9"
uuid = { version = "1", features = ["v4"] }
# OS credential stores
@@ -140,9 +131,7 @@ debug = false
[profile.release]
codegen-units = 1
# Thin LTO: fat LTO's whole-program link outgrew the 16 GB release runners
# (Linux arm64 was killed mid-link from v0.1.39 on).
lto = "thin"
lto = true
opt-level = 3
# Unwind, so a panic fails one task or command instead of quitting the IDE.
panic = "unwind"
+9 -18
View File
@@ -12,15 +12,11 @@
<p align="center">
<a href="https://github.com/l0ng-ai/openidea/releases/latest"><b>Download</b></a>
&nbsp;·&nbsp;
<a href="https://openidea.sh">Docs</a>
&nbsp;·&nbsp;
<a href="#benchmark">Benchmark</a>
&nbsp;·&nbsp;
<a href="#plugins">Plugins</a>
&nbsp;·&nbsp;
<a href="#community">Community</a>
&nbsp;·&nbsp;
<a href="#sponsor">Sponsor</a>
</p>
https://github.com/user-attachments/assets/234dd54c-1255-4b3a-b35c-533d78d737c0
@@ -29,7 +25,9 @@ OpenIDEA brings the IntelliJ IDEA way of working — its keymap, its tool window
its search-everything habits — to a native app with a Rust core.
Editor, code insight, debugger, Git, terminal, database and HTTP clients,
for 60+ languages. Builds for macOS, Windows and Linux.
for 60+ languages. Builds for macOS, Windows and Linux. On Windows, if you'd
rather skip the installer, each release also ships a portable zip — unpack it
and run `openidea.exe`.
## Benchmark
@@ -58,7 +56,7 @@ themes, file icons and languages. They run as sandboxed WebAssembly components;
themes, icon themes and languages need no code at all.
Install them from Settings › Plugins, or start your own with
`openidea-plugin new` — see [Writing OpenIDEA plugins](https://openidea.sh/en/plugins) and
`openidea-plugin new` — see [Writing OpenIDEA plugins](plugins/README.md) and
the [official plugins](plugins/official). The plugin API is still a preview.
## Community
@@ -66,20 +64,13 @@ the [official plugins](plugins/official). The plugin API is still a preview.
- [Discord](https://discord.gg/2Zy4yn7gP) — questions, feedback and release news.
- WeChat group — scan to join:
<img src=".github/assets/wechat-group.png" alt="WeChat group QR code" width="200">
<img src="docs/assets/wechat-group.png" alt="WeChat group QR code" width="200">
- Contributing: see [CONTRIBUTING.md](CONTRIBUTING.md) and the [contributor docs](https://openidea.sh/en/contribute/setup).
### Contributors
## Sponsor
If OpenIDEA saves you time, you can support its development with Alipay or WeChat Pay.
<table>
<tr>
<td align="center"><img src=".github/assets/sponsor-alipay.png" alt="Alipay QR code" width="200"><br>Alipay</td>
<td align="center"><img src=".github/assets/sponsor-wechat.png" alt="WeChat Pay QR code" width="200"><br>WeChat Pay</td>
</tr>
</table>
<a href="https://github.com/l0ng-ai/openidea/graphs/contributors">
<img src="https://contrib.rocks/image?repo=l0ng-ai/openidea&max=100" alt="OpenIDEA contributors">
</a>
## License
-26
View File
@@ -1,26 +0,0 @@
# Security policy
## Reporting a vulnerability
Please do not report security problems in public issues, pull requests or discussions.
Report them privately through GitHub:
[Report a vulnerability](https://github.com/l0ng-ai/openidea/security/advisories/new)
(repository **Security** tab → **Report a vulnerability**). Only the maintainers can see
the report.
Include what you can of:
- the affected version, platform, and whether remote development (`openidea-server`)
or plugins are involved;
- steps to reproduce, or a proof of concept;
- the impact you expect, and any fix you suggest.
We will acknowledge the report, keep you updated in the advisory, and agree on a
disclosure date with you. Fixes are released as a new version, and the advisory is
published once users can update.
## Supported versions
Security fixes go into the latest release. Please check that the problem still occurs
there before reporting.
+2 -12
View File
@@ -1,11 +1,5 @@
# Third-party licenses
## Project type icons
`src/ui/brands/*.svg` use the unmodified logo paths from
[Simple Icons](https://github.com/simple-icons/simple-icons), retrieved through
Iconify (`simple-icons:angular`, `simple-icons:astro`, `simple-icons:cplusplus`, `simple-icons:dart`, `simple-icons:django`, `simple-icons:dotnet`, `simple-icons:elixir`, `simple-icons:express`, `simple-icons:fastapi`, `simple-icons:flask`, `simple-icons:flutter`, `simple-icons:go`, `simple-icons:haskell`, `simple-icons:html5`, `simple-icons:kotlin`, `simple-icons:ktor`, `simple-icons:laravel`, `simple-icons:nestjs`, `simple-icons:nextdotjs`, `simple-icons:nodedotjs`, `simple-icons:nuxt`, `simple-icons:openjdk`, `simple-icons:phoenixframework`, `simple-icons:php`, `simple-icons:python`, `simple-icons:quarkus`, `simple-icons:ruby`, `simple-icons:rubyonrails`, `simple-icons:rust`, `simple-icons:scala`, `simple-icons:springboot`, `simple-icons:svelte`, `simple-icons:swift`, `simple-icons:symfony`, `simple-icons:vite`, `simple-icons:zig`) and filled with the brand colors (the standard icon grey for black logos). Licensed under [CC0 1.0 Universal](https://github.com/simple-icons/simple-icons/blob/develop/LICENSE.md).
## Project Lombok
`java/lombok/lombok-1.18.48-openidea.3.jar` contains Project Lombok v1.18.48 with the Equinox
@@ -268,7 +262,7 @@ END OF TERMS AND CONDITIONS
The Solarized palettes in `src/styles/tokens.css` are adapted from
[Ethan Schoonover’s Solarized](https://github.com/altercation/solarized/tree/62f656a02f93c5190a8753159e34b385588d5ff3).
MIT license:
See [theme sources and mapping](docs/themes.md). MIT license:
```text
Copyright (c) 2011 Ethan Schoonover
@@ -296,11 +290,7 @@ THE SOFTWARE.
The One palettes and semantic syntax mapping in `src/styles/tokens.css` are
adapted from GitHub’s Atom One syntax and UI themes.
Source revisions:
[one-light-syntax d845790](https://github.com/atom/one-light-syntax/tree/d84579027410c576086dfca14d934c4bd74b0438),
[one-dark-syntax 9c96f44](https://github.com/atom/one-dark-syntax/tree/9c96f4454362267ac45322063e193ccf9d2debb1),
[one-light-ui a27bce3](https://github.com/atom/one-light-ui/tree/a27bce35088ecd323e44f5c2510bcf2d4e9efcf8),
[one-dark-ui 18c2143](https://github.com/atom/one-dark-ui/tree/18c2143d41e17b337fff6dc7024562c4d999be17).
Source revisions are listed in [theme sources and mapping](docs/themes.md).
Both syntax themes use the following MIT notice:
```text
-102
View File
@@ -1,102 +0,0 @@
//! A short-lived ACP session to learn models, effort levels and other config options
//! before the user's first task (Cursor and other ACP agents do not expose these in
//! `initialize` alone).
use super::{Conn, Dialect};
use crate::config;
use crate::model::AgentConfigOption;
use crate::provider::ProviderSpec;
use crate::{Shared, client_info};
use parking_lot::Mutex;
use serde_json::{Value, json};
use std::sync::Arc;
use std::time::Duration;
const DISCOVER_TIMEOUT: Duration = if cfg!(test) { Duration::from_secs(3) } else { Duration::from_secs(120) };
/// Opens `session/new`, reads `configOptions`, and records per-model `thought_level` choices.
pub(crate) async fn run(shared: &Arc<Shared>, spec: &ProviderSpec) -> Result<(), String> {
let root = shared.paths.project_root.clone();
let launch = spec.clone();
let launch = tokio::task::spawn_blocking({
let root = root.clone();
move || launch.resolve_for_launch(&root)
})
.await
.map_err(|e| e.to_string())?;
let cwd = dunce::canonicalize(&root).unwrap_or(root);
let stderr: Arc<Mutex<Vec<String>>> = Arc::default();
let lines = stderr.clone();
let on_stderr: Box<dyn Fn(String) + Send + Sync> = Box::new(move |l| lines.lock().push(l));
let client = Dialect::wrap(Arc::new(ide_jsonrpc::NoopHandler));
let peer = (shared.connector)(&launch, &cwd, client.clone(), on_stderr)
.map_err(|e| format!("Couldn't start {} (`{}`): {e}", launch.name, launch.command))?;
let conn = Conn::new(peer.clone(), client);
let work = async {
conn.initialize(&client_info()).await.map_err(|e| e.to_string())?;
let cwd_s = cwd.to_string_lossy().into_owned();
// No project MCP during discovery: Cursor can spend a long time spawning servers.
let new: Value = conn
.request("session/new", json!({ "cwd": cwd_s, "mcpServers": [] }), None)
.await
.map_err(|e| e.to_string())?;
let mut options = config::options_of(&new);
config::add_client_options(&launch.id, &mut options);
let model_option = options.iter().find(|o| o.category.as_deref() == Some("model"));
let effort = levels_for_model_from_slice(&options);
let levels: Vec<(String, Vec<AgentConfigOption>)> = match (model_option, effort) {
(Some(model), Some(effort)) if !model.values.is_empty() => {
model.values.iter().map(|v| (v.value.clone(), effort.clone())).collect()
}
(Some(model), Some(effort)) => model.current.clone().map(|m| vec![(m, effort)]).unwrap_or_default(),
_ => Vec::new(),
};
let option_ids: Vec<_> = options.iter().map(|o| o.id.as_str()).collect();
tracing::info!(
"agent {}: settings discovery got {} config option(s) {:?}, {} model effort list(s)",
launch.id,
options.len(),
option_ids,
levels.len()
);
shared.remember_discovered(&launch.id, &options, &[], &levels);
Ok::<(), String>(())
};
let result = tokio::time::timeout(DISCOVER_TIMEOUT, work).await;
peer.kill();
match result {
Ok(Ok(())) => Ok(()),
Ok(Err(e)) => Err(match stderr.lock().last() {
Some(line) => format!("{e}: {line}"),
None => e,
}),
Err(_) => {
let tail = stderr.lock().join("\n");
if tail.is_empty() {
Err("the agent did not answer in time".into())
} else {
Err(format!("the agent did not answer in time:\n{tail}"))
}
}
}
}
fn is_thought_level(o: &AgentConfigOption) -> bool {
o.category.as_deref() == Some("thought_level")
|| matches!(o.id.as_str(), "effort" | "reasoning_effort" | "thinking" | "reasoning")
}
fn levels_for_model_from_slice(options: &[AgentConfigOption]) -> Option<Vec<AgentConfigOption>> {
let levels: Vec<_> = options
.iter()
.filter(|o| is_thought_level(o))
.cloned()
.map(|mut o| {
if o.category.as_deref() != Some("thought_level") {
o.category = Some("thought_level".into());
}
o
})
.collect();
(!levels.is_empty()).then_some(levels)
}
-1
View File
@@ -107,7 +107,6 @@ fn manager(dir: &Path, agent: Arc<FakeV2Agent>) -> AgentManager {
models_file: dir.join("models.json"),
custom_agents_dir: None,
home_dir: None,
env_secrets: None,
},
tokio::runtime::Handle::current(),
connector,
-20
View File
@@ -14,7 +14,6 @@
//! - Agent → client ([`Dialect`]): `session/update`s are normalized ([`normalize`]) and
//! permission requests get the v1 `toolCall`.
pub(crate) mod discover;
mod normalize;
use ide_jsonrpc::{Handler, Peer, RpcError, codes};
@@ -58,9 +57,6 @@ fn client_capabilities() -> Value {
"terminal": false,
// Subagents as sessions of their own (ACP subagents RFD; v2 baseline).
"subagents": {},
// Cursor exposes model parameters (e.g. Composer `fast`) as separate config options
// only when the client advertises this flag (undocumented Cursor ACP extension).
"_meta": { "parameterizedModelPicker": true },
})
}
@@ -168,11 +164,6 @@ impl Negotiated {
self.capabilities.pointer(&format!("/promptCapabilities/{name}")).and_then(Value::as_bool) == Some(true)
}
/// `mcpCapabilities.<name>` (`http`, `sse`).
pub fn mcp(&self, name: &str) -> bool {
self.capabilities.pointer(&format!("/mcpCapabilities/{name}")).and_then(Value::as_bool) == Some(true)
}
/// `sessionCapabilities.<name>` (`fork`, `resume`, `close`, `list`, `delete`).
pub fn session(&self, name: &str) -> bool {
self.capabilities.pointer(&format!("/sessionCapabilities/{name}")).is_some_and(|v| !v.is_null())
@@ -182,17 +173,6 @@ impl Negotiated {
pub fn meta(&self, pointer: &str) -> Option<&Value> {
self.capabilities.get("_meta")?.pointer(pointer)
}
/// First authentication method the agent can run over ACP (`type` absent or `"agent"`).
pub fn agent_auth_method(&self) -> Option<&str> {
self.auth_methods.iter().find_map(|m| {
let kind = m.get("type").and_then(Value::as_str).unwrap_or("agent");
(kind == "agent" && m.pointer("/_meta/terminal-auth").is_none())
.then(|| m.get("id").and_then(Value::as_str))
.flatten()
.filter(|id| !id.is_empty())
})
}
}
/// The client side of one agent connection: wraps the task's [`Handler`] and, once
+2 -105
View File
@@ -93,82 +93,9 @@ fn the_negotiating_request_carries_both_versions() {
assert!(p["capabilities"].is_object());
assert_eq!(p["clientCapabilities"]["subagents"], json!({}));
assert_eq!(p["clientCapabilities"]["terminal"], false);
assert_eq!(
p["clientCapabilities"]["_meta"]["parameterizedModelPicker"], true,
"Cursor and similar agents need parameterized model options (e.g. fast on/off)"
);
// What we sent in the capture is what claude-agent-acp negotiated v2 and v1 from.
let sent = frames(TURN, "out")[0]["params"]["clientCapabilities"].clone();
let mut expected = sent;
expected["_meta"] = p["clientCapabilities"]["_meta"].clone();
assert_eq!(expected, p["clientCapabilities"]);
}
#[tokio::test(flavor = "multi_thread")]
async fn discovery_reads_effort_and_fast_from_session_new_then_stops_the_agent() {
#[derive(Default)]
struct DiscoveryAgent {
peer: std::sync::OnceLock<Peer>,
calls: parking_lot::Mutex<Vec<(String, Value)>>,
}
impl Handler for DiscoveryAgent {
fn on_notification(&self, _: &str, _: Value) {}
fn on_request(&self, method: &str, params: Value) -> Result<Value, (i64, String)> {
self.calls.lock().push((method.to_owned(), params));
match method {
"initialize" => Ok(json!({
"protocolVersion": 1,
"agentCapabilities": { "loadSession": true }
})),
"session/new" => Ok(json!({
"sessionId": "discover",
"configOptions": [
{"id": "model", "category": "model", "type": "select", "currentValue": "m1",
"options": [{"value": "m1", "name": "One"}, {"value": "m2", "name": "Two"}]},
{"id": "effort", "type": "select", "currentValue": "medium",
"options": [{"value": "low"}, {"value": "medium"}, {"value": "high"}]},
{"id": "fast", "type": "select", "currentValue": "false",
"options": [{"value": "false", "name": "Off"}, {"value": "true", "name": "On"}]}
]
})),
_ => Ok(json!({})),
}
}
}
let agent = Arc::new(DiscoveryAgent::default());
let tmp = tempfile::tempdir().unwrap();
let agent_for_connector = agent.clone();
let connector: crate::Connector = Arc::new(move |_, _, client, _| {
let (ca, cb) = tokio::io::duplex(1 << 20);
let (car, caw) = tokio::io::split(ca);
let (cbr, cbw) = tokio::io::split(cb);
let agent_peer = Peer::connect_framed(cbr, cbw, agent_for_connector.clone(), Framing::Lines);
let _ = agent_for_connector.peer.set(agent_peer);
Ok(Peer::connect_framed(car, caw, client, Framing::Lines))
});
let mgr = crate::AgentManager::with_connector(
crate::AgentPaths {
project_root: tmp.path().to_path_buf(),
store_dir: tmp.path().join("store"),
worktrees_dir: tmp.path().join("worktrees"),
models_file: tmp.path().join("models.json"),
custom_agents_dir: None,
home_dir: None,
env_secrets: None,
},
tokio::runtime::Handle::current(),
connector,
);
mgr.commands("cursor", None).await.unwrap();
let calls = agent.calls.lock();
assert_eq!(calls.iter().map(|(m, _)| m.as_str()).collect::<Vec<_>>(), ["initialize", "session/new"]);
assert_eq!(calls[1].1["mcpServers"], json!([]));
let cursor = mgr.providers().into_iter().find(|p| p.id == "cursor").unwrap();
let fast = cursor.config.iter().find(|o| o.id == "fast").expect("fast option");
assert_eq!(fast.category.as_deref(), Some("model_config"));
assert_eq!(fast.current.as_deref(), Some("false"));
assert!(cursor.thought_levels.contains_key("m1") && cursor.thought_levels.contains_key("m2"));
assert!(cursor.thought_levels["m1"].iter().any(|o| o.category.as_deref() == Some("thought_level")));
let sent = &frames(TURN, "out")[0]["params"];
assert_eq!(sent["clientCapabilities"], p["clientCapabilities"]);
}
#[test]
@@ -208,36 +135,6 @@ fn the_shape_decides_the_version_not_the_number() {
assert_eq!((n.generation, n.load_session(), n.session("fork")), (2, false, false));
}
#[test]
fn picks_non_terminal_agent_auth_method() {
let v1 = Negotiated::from_initialize(&json!({
"protocolVersion": 1,
"authMethods": [
{ "id": "cli_login", "_meta": { "terminal-auth": { "command": "copilot", "args": ["login"] } } },
{ "id": "terminal_login", "type": "terminal", "args": ["login"] },
{ "id": "api_key", "type": "env_var" },
{ "id": "github_oauth", "name": "GitHub OAuth" }
]
}));
assert_eq!(v1.agent_auth_method(), Some("github_oauth"));
let v2 = Negotiated::from_initialize(&json!({
"protocolVersion": 2,
"info": { "name": "x", "version": "1" },
"authMethods": [
{ "type": "terminal", "methodId": "cli", "name": "CLI", "args": ["login"] },
{ "type": "agent", "methodId": "oauth", "name": "OAuth" }
]
}));
assert_eq!(v2.agent_auth_method(), Some("oauth"));
let terminal_only = Negotiated::from_initialize(&json!({
"protocolVersion": 1,
"authMethods": [{ "id": "cli", "type": "terminal", "args": ["login"] }]
}));
assert_eq!(terminal_only.agent_auth_method(), None);
}
#[test]
fn meta_and_auth_methods_pass_through() {
// The native shims advertise their extensions in `agentCapabilities._meta`.
+4 -86
View File
@@ -42,38 +42,9 @@ pub(crate) fn bypass_approvals(config: &[AgentConfigOption]) -> bool {
let selected = |category: &str| {
config.iter().find(|o| o.category.as_deref() == Some(category)).and_then(|o| o.current.as_deref())
};
["mode", "permissions", "approval"]
.into_iter()
.any(|category| selected(category).zip(bypass_value(category)).is_some_and(|(value, bypass)| bypass(value)))
}
/// For an option of `category` that can approve everything, what tells its approving values.
fn bypass_value(category: &str) -> Option<fn(&str) -> bool> {
match category {
"mode" => Some(is_bypass_mode),
"permissions" => Some(|v| matches!(v.to_ascii_lowercase().as_str(), "on" | "true")),
"approval" => Some(|v| v.eq_ignore_ascii_case("bypass")),
_ => None,
}
}
/// Moves options that approve everything without asking to a value that asks: the agent's
/// default or ask mode, else its first one that asks.
pub(crate) fn ask_for_approvals(config: &mut [AgentConfigOption]) {
for option in config {
let Some(bypass) = option.category.as_deref().and_then(bypass_value) else { continue };
if option.current.as_deref().is_some_and(|c| !bypass(c)) {
continue;
}
let asks = |v: &&AgentConfigValue| !bypass(&v.value);
let value = ["default", "ask"]
.iter()
.find_map(|preferred| option.values.iter().filter(asks).find(|v| v.value == *preferred))
.or_else(|| option.values.iter().find(asks));
if let Some(value) = value {
option.current = Some(value.value.clone());
}
}
selected("mode").is_some_and(is_bypass_mode)
|| selected("permissions").is_some_and(|v| matches!(v.to_ascii_lowercase().as_str(), "on" | "true"))
|| selected("approval").is_some_and(|v| v.eq_ignore_ascii_case("bypass"))
}
/// The mode that approves everything without asking (`bypassPermissions`, `yolo`, `full-access`…), if the agent has one.
@@ -136,15 +107,7 @@ pub(crate) fn parse_options(options: &Value) -> Vec<AgentConfigOption> {
Some(AgentConfigOption {
name: str_of(o, "name").unwrap_or_else(|| id.clone()),
description: str_of(o, "description"),
category: str_of(o, "category").or_else(|| {
matches!(id.as_str(), "model" | "mode")
.then(|| id.clone())
.or_else(|| {
matches!(id.as_str(), "effort" | "reasoning_effort" | "thinking" | "reasoning")
.then(|| "thought_level".into())
})
.or_else(|| matches!(id.as_str(), "fast").then(|| "model_config".into()))
}),
category: str_of(o, "category").or_else(|| matches!(id.as_str(), "model" | "mode").then(|| id.clone())),
current: str_of(o, "currentValue"),
values,
via: AgentConfigVia::Option,
@@ -370,51 +333,6 @@ mod tests {
assert!(bypass_approvals(&url_mode));
}
#[test]
fn asking_for_approvals_keeps_a_mode_that_asks_and_leaves_one_that_does_not() {
let mode = |current: &str, values: &[&str]| {
options_of(&json!({"modes": {"currentModeId": current,
"availableModes": values.iter().map(|v| json!({"id": v})).collect::<Vec<_>>()}}))
};
let codex = ["read-only", "ask", "auto", "full-access"];
for (current, expected) in [("auto", "auto"), ("read-only", "read-only"), ("full-access", "ask")] {
let mut c = mode(current, &codex);
ask_for_approvals(&mut c);
assert_eq!(c[0].current.as_deref(), Some(expected), "{current}");
assert!(!bypass_approvals(&c));
}
let mut claude = mode("bypassPermissions", &["acceptEdits", "default", "plan", "bypassPermissions"]);
ask_for_approvals(&mut claude);
assert_eq!(claude[0].current.as_deref(), Some("default"));
let mut other = mode("yolo", &["yolo", "code", "architect"]);
ask_for_approvals(&mut other);
assert_eq!(other[0].current.as_deref(), Some("code"));
let mut c = options_of(&json!({"configOptions": [
{"id": "allow_all", "category": "permissions", "type": "select", "currentValue": "on",
"options": [{"value": "on"}, {"value": "off"}]},
{"id": "approval", "category": "approval", "type": "select", "currentValue": "bypass",
"options": [{"value": "bypass"}, {"value": "ask"}]}
]}));
ask_for_approvals(&mut c);
assert_eq!(ids(&c), [("allow_all", Some("off")), ("approval", Some("ask"))]);
}
#[test]
fn effort_and_fast_categories_are_inferred_from_ids() {
let c = options_of(&json!({
"configOptions": [
{"id": "effort", "type": "select", "currentValue": "high",
"options": [{"value": "low"}, {"value": "high"}]},
{"id": "fast", "type": "select", "currentValue": "false",
"options": [{"value": "false", "name": "Off"}, {"value": "true", "name": "On"}]}
]
}));
assert_eq!(c.len(), 2);
assert_eq!(c[0].category.as_deref(), Some("thought_level"));
assert_eq!(c[1].category.as_deref(), Some("model_config"));
}
#[test]
fn command_sources_in_updates_and_stored_tasks() {
let update = json!({"availableCommands": [
-325
View File
@@ -1,325 +0,0 @@
//! Sessions the user started outside OpenIDEA (a terminal, another editor, the CLI):
//! scanned so they can be listed and imported as tasks instead of re-explained (#312).
//!
//! Only the native providers keep transcripts on disk in a readable form: Claude Code
//! (`<config>/projects/*/<id>.jsonl`, see `native::claude::history`) and Codex
//! (`<home>/.codex/sessions/**`, rollout files with a `session_meta` line).
use super::native::claude::history;
use serde::Serialize;
use specta::Type;
use std::io::BufRead;
use std::path::{Path, PathBuf};
/// A session on this machine that no task holds yet.
#[derive(Clone, Debug, Serialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct ExternalSession {
/// The [`crate::AgentProvider`] id that owns it (`claude`, `codex`).
pub provider: String,
/// The id resuming it names (`--resume`, `thread/resume`).
pub session_id: String,
pub title: String,
/// Where it ran, when the transcript names one.
pub cwd: Option<String>,
/// File modification time, ms since the epoch.
pub updated_ms: f64,
}
/// Modification time of `path` in ms, else 0.
fn mtime(path: &Path) -> f64 {
path.metadata()
.and_then(|m| m.modified())
.ok()
.and_then(|t| t.duration_since(std::time::UNIX_EPOCH).ok())
.map(|d| d.as_millis() as f64)
.unwrap_or(0.0)
}
/// Short display text: collapsed whitespace, truncated to `MAX_TITLE` chars.
fn short(text: &str) -> String {
const MAX_TITLE: usize = 80;
let flat: String = text.split_whitespace().collect::<Vec<_>>().join(" ");
if flat.chars().count() <= MAX_TITLE {
flat
} else {
format!("{}…", flat.chars().take(MAX_TITLE - 1).collect::<String>())
}
}
/// The text of a Claude Code transcript entry's message, when it carries prose.
fn message_text(e: &serde_json::Value) -> Option<String> {
match e.pointer("/message/content") {
Some(serde_json::Value::String(s)) => Some(s.clone()),
Some(serde_json::Value::Array(blocks)) => {
let text: Vec<&str> = blocks
.iter()
.filter(|b| b.get("type").and_then(serde_json::Value::as_str) == Some("text"))
.filter_map(|b| b.get("text").and_then(serde_json::Value::as_str))
.collect();
(!text.is_empty()).then(|| text.join("\n"))
}
_ => None,
}
}
/// Whether a session that ran in `cwd` belongs to the project at `root` (in it or below it).
fn in_project(cwd: Option<&str>, root: &Path) -> bool {
cwd.is_some_and(|cwd| Path::new(cwd).starts_with(root))
}
/// Claude Code transcripts of the project at `root`: `<config>/projects/<escaped cwd>/<id>.jsonl`.
/// Only the folders named after `root` or a folder below it are read, so other projects'
/// transcripts (often gigabytes) are never opened.
pub fn claude_sessions(config_dir: &Path, root: &Path) -> Vec<ExternalSession> {
let mut out = Vec::new();
let projects = config_dir.join("projects");
let escaped = history::escape(root);
let dirs: Vec<PathBuf> = std::fs::read_dir(&projects)
.into_iter()
.flatten()
.flatten()
.filter(|d| {
let name = d.file_name().to_string_lossy().into_owned();
name == escaped || name.starts_with(&format!("{escaped}-"))
})
.map(|d| d.path())
.collect();
for dir in dirs {
let files: Vec<PathBuf> = std::fs::read_dir(&dir)
.into_iter()
.flatten()
.flatten()
.map(|d| d.path())
.filter(|p| p.extension().is_some_and(|e| e == "jsonl"))
.collect();
for path in files {
let id = path.file_stem().and_then(|s| s.to_str()).unwrap_or_default().to_owned();
if id.is_empty() {
continue;
}
let entries = history::read(&path).unwrap_or_default();
if entries.is_empty() {
continue;
}
let title = history::title(&entries)
.or_else(|| entries.iter().filter(|e| history::is_turn_start(e)).filter_map(message_text).next())
.map(|t| short(&t))
.unwrap_or_else(|| format!("Claude Code session {}", &id[..id.len().min(8)]));
let cwd = entries
.iter()
.filter_map(|e| e.get("cwd").and_then(serde_json::Value::as_str))
.next()
.map(str::to_owned);
// Escaping is lossy (`a/b-c` and `a/b/c` share a folder): the transcript decides.
if !in_project(cwd.as_deref(), root) {
continue;
}
out.push(ExternalSession {
provider: "claude".to_owned(),
session_id: id,
title,
cwd,
updated_ms: mtime(&path),
});
}
}
out
}
/// The `session_meta` line of a Codex rollout file: `(id, cwd)`, across shapes.
fn codex_meta(line: &serde_json::Value) -> Option<(String, Option<String>)> {
let payload =
line.get("payload").filter(|p| p.get("type").and_then(serde_json::Value::as_str) == Some("session_meta"));
let inner = payload.and_then(|p| p.get("payload")).or_else(|| {
line.get("type").and_then(serde_json::Value::as_str).filter(|t| *t == "session_meta").and(Some(line))
})?;
let id = inner.get("id").and_then(serde_json::Value::as_str)?.to_owned();
let cwd = inner.get("cwd").and_then(serde_json::Value::as_str).map(str::to_owned);
Some((id, cwd))
}
/// A prose line of a Codex rollout file (a user message), across shapes.
fn codex_user_text(line: &serde_json::Value) -> Option<String> {
let payload = line.get("payload")?;
let ty = payload.get("type").and_then(serde_json::Value::as_str)?;
if !(ty == "user_message" || ty == "userMessage" || ty == "event_msg") {
return None;
}
let inner = payload.get("payload").unwrap_or(&serde_json::Value::Null);
for v in [payload, inner] {
if let Some(s) = v.get("message").and_then(serde_json::Value::as_str) {
return Some(s.to_owned());
}
if let Some(s) = v.get("text").and_then(serde_json::Value::as_str) {
return Some(s.to_owned());
}
}
None
}
/// One rollout file's session, or `None` when it names no `session_meta`.
fn codex_file(path: &Path) -> Option<ExternalSession> {
let file = std::fs::File::open(path).ok()?;
let mut id: Option<(String, Option<String>)> = None;
let mut title: Option<String> = None;
// `session_meta` is first, but cap the scan: giant rollouts must not stall listing.
for line in std::io::BufReader::new(file).lines().take(400) {
let Ok(line) = line else { break };
let Ok(v) = serde_json::from_str::<serde_json::Value>(&line) else { continue };
if id.is_none()
&& let Some(meta) = codex_meta(&v)
{
id = Some(meta);
}
if title.is_none()
&& let Some(t) = codex_user_text(&v)
{
title = Some(short(&t));
}
if id.is_some() && title.is_some() {
break;
}
}
let (session_id, cwd) = id?;
Some(ExternalSession {
provider: "codex".to_owned(),
title: title.unwrap_or_else(|| format!("Codex session {}", &session_id[..session_id.len().min(8)])),
session_id,
cwd,
updated_ms: mtime(path),
})
}
/// Collects `*.jsonl` below `dir`, up to 4 levels (`sessions/YYYY/MM/DD/<rollout>`).
fn rollout_files(dir: &Path, depth: usize, out: &mut Vec<PathBuf>) {
if depth == 0 {
return;
}
for entry in std::fs::read_dir(dir).into_iter().flatten().flatten() {
let path = entry.path();
if path.is_dir() {
rollout_files(&path, depth - 1, out);
} else if path.extension().is_some_and(|e| e == "jsonl") {
out.push(path);
}
}
}
/// Codex sessions of the project at `root`: `<home>/.codex/sessions/**`.
pub fn codex_sessions(codex_home: &Path, root: &Path) -> Vec<ExternalSession> {
let mut files = Vec::new();
rollout_files(&codex_home.join("sessions"), 4, &mut files);
files.iter().filter_map(|p| codex_file(p)).filter(|s| in_project(s.cwd.as_deref(), root)).collect()
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
use std::io::Write;
fn write(path: &Path, lines: &[serde_json::Value]) {
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
let mut f = std::fs::File::create(path).unwrap();
for line in lines {
writeln!(f, "{line}").unwrap();
}
}
fn claude_user(uuid: &str, text: &str) -> serde_json::Value {
json!({"type": "user", "uuid": uuid, "parentUuid": null,
"message": {"role": "user", "content": text}, "cwd": "/work/app"})
}
#[test]
fn lists_claude_transcripts_with_titles() {
let tmp = tempfile::tempdir().unwrap();
write(
&tmp.path().join("projects/-work-app/abc123.jsonl"),
&[
claude_user("u1", "fix the login bug"),
json!({"type": "assistant", "uuid": "a1", "parentUuid": "u1",
"message": {"role": "assistant", "content": []}}),
json!({"type": "ai-title", "aiTitle": "Login fix"}),
],
);
write(&tmp.path().join("projects/-work-app/empty.jsonl"), &[]);
let sessions = claude_sessions(tmp.path(), Path::new("/work/app"));
assert_eq!(sessions.len(), 1);
let s = &sessions[0];
assert_eq!((s.provider.as_str(), s.session_id.as_str()), ("claude", "abc123"));
assert_eq!(s.title, "Login fix");
assert_eq!(s.cwd.as_deref(), Some("/work/app"));
assert!(s.updated_ms > 0.0);
}
#[test]
fn claude_title_falls_back_to_the_first_prompt() {
let tmp = tempfile::tempdir().unwrap();
write(&tmp.path().join("projects/-work-app/def456.jsonl"), &[claude_user("u1", "hello there")]);
let sessions = claude_sessions(tmp.path(), Path::new("/work/app"));
assert_eq!(sessions.len(), 1);
assert_eq!(sessions[0].title, "hello there");
}
#[test]
fn lists_codex_rollouts_with_meta() {
let tmp = tempfile::tempdir().unwrap();
write(
&tmp.path().join("sessions/2026/01/02/rollout-01.jsonl"),
&[
json!({"timestamp": "t", "payload": {"type": "session_meta",
"payload": {"id": "thread-1", "cwd": "/work/app"}}}),
json!({"timestamp": "t", "payload": {"type": "user_message",
"payload": {"message": "add tests"}}}),
],
);
write(&tmp.path().join("sessions/notes.jsonl"), &[json!({"nope": true})]);
let sessions = codex_sessions(tmp.path(), Path::new("/work"));
assert_eq!(sessions.len(), 1);
let s = &sessions[0];
assert_eq!((s.provider.as_str(), s.session_id.as_str()), ("codex", "thread-1"));
assert_eq!(s.title, "add tests");
assert_eq!(s.cwd.as_deref(), Some("/work/app"));
}
#[test]
fn missing_directories_list_nothing() {
let tmp = tempfile::tempdir().unwrap();
assert!(claude_sessions(&tmp.path().join("nope"), Path::new("/work")).is_empty());
assert!(codex_sessions(&tmp.path().join("nope"), Path::new("/work")).is_empty());
}
#[test]
fn only_sessions_of_the_project_are_listed() {
let tmp = tempfile::tempdir().unwrap();
let user = |cwd: &str| {
json!({"type": "user", "uuid": "u1", "parentUuid": null,
"message": {"role": "user", "content": "hi"}, "cwd": cwd})
};
write(&tmp.path().join("projects/-work-app/in-root.jsonl"), &[user("/work/app")]);
write(&tmp.path().join("projects/-work-app-web/below.jsonl"), &[user("/work/app/web")]);
// `/work/app-old` escapes like a folder below `/work/app`; its transcript says otherwise.
write(&tmp.path().join("projects/-work-app-old/sibling.jsonl"), &[user("/work/app-old")]);
write(&tmp.path().join("projects/-other/elsewhere.jsonl"), &[user("/other")]);
let mut ids: Vec<String> =
claude_sessions(tmp.path(), Path::new("/work/app")).into_iter().map(|s| s.session_id).collect();
ids.sort();
assert_eq!(ids, ["below", "in-root"]);
let rollout = |name: &str, cwd: &str| {
write(
&tmp.path().join(format!("codex/sessions/2026/01/02/{name}.jsonl")),
&[json!({"payload": {"type": "session_meta", "payload": {"id": name, "cwd": cwd}}})],
)
};
rollout("mine", "/work/app/api");
rollout("theirs", "/other");
let codex: Vec<String> = codex_sessions(&tmp.path().join("codex"), Path::new("/work/app"))
.into_iter()
.map(|s| s.session_id)
.collect();
assert_eq!(codex, ["mine"]);
}
}
File diff suppressed because it is too large. Load diff
-507
View File
@@ -1,507 +0,0 @@
//! Model Context Protocol (MCP) server configurations stored in OpenIDEA's own settings
//! (global `settings.toml` and project `.openidea/settings.toml`), passed to ACP agents in
//! `session/new`, `session/load`, and `session/fork`, plus one-way import from `.mcp.json` /
//! `.vscode/mcp.json`.
use crate::acp::Negotiated;
use crate::{AgentMcpServer, McpKeyValue, McpScope, McpServer, McpServerInput, McpTransport};
use serde_json::{Value, json};
use std::collections::{BTreeMap, HashSet};
use std::path::Path;
/// Stable ID for a server in a given scope (`"user::<name>"` or `"workspace::<name>"`).
pub fn server_id(scope: McpScope, name: &str) -> String {
let prefix = match scope {
McpScope::User => "user",
McpScope::Workspace => "workspace",
};
format!("{prefix}::{name}")
}
/// Parses a server ID (`"user::<name>"` or `"workspace::<name>"`).
pub fn parse_server_id(id: &str) -> Option<(McpScope, &str)> {
let (prefix, name) = id.split_once("::")?;
if name.is_empty() {
return None;
}
match prefix {
"user" => Some((McpScope::User, name)),
"workspace" => Some((McpScope::Workspace, name)),
_ => None,
}
}
/// Converts the User (`settings.toml`) and Workspace (`.openidea/settings.toml`) MCP server maps
/// into the ordered list displayed in Settings › Agents › MCP Servers.
pub fn servers_from_settings(
user: &BTreeMap<String, AgentMcpServer>,
workspace: &BTreeMap<String, AgentMcpServer>,
) -> Vec<McpServer> {
user.iter()
.map(|(name, cfg)| to_mcp_server(McpScope::User, name, cfg))
.chain(workspace.iter().map(|(name, cfg)| to_mcp_server(McpScope::Workspace, name, cfg)))
.collect()
}
fn to_mcp_server(scope: McpScope, name: &str, cfg: &AgentMcpServer) -> McpServer {
McpServer {
id: server_id(scope, name),
name: name.to_owned(),
scope,
transport: cfg.transport,
enabled: cfg.enabled,
command: cfg.command.clone(),
args: cfg.args.clone(),
env: cfg.env.iter().map(|(k, v)| McpKeyValue { name: k.clone(), value: v.clone() }).collect(),
url: cfg.url.clone(),
headers: cfg.headers.iter().map(|(k, v)| McpKeyValue { name: k.clone(), value: v.clone() }).collect(),
agents: cfg.agents.clone(),
}
}
fn normalize_provider(id: &str) -> &str {
id.strip_suffix("-acp").unwrap_or(id)
}
fn matches_agent(agents: &[String], provider: &str) -> bool {
if agents.is_empty() {
return true;
}
let target = normalize_provider(provider);
agents.iter().any(|a| {
let trimmed = a.trim();
trimmed == provider || normalize_provider(trimmed) == target
})
}
/// Builds the ACP `mcpServers` array for `session/new`, `session/load`, and `session/fork`.
/// Workspace entries take precedence over User entries with the same name.
pub(crate) fn acp_mcp_servers(all: &[McpServer], provider: &str, init: &Negotiated) -> Vec<Value> {
let supports_stdio = init.mcp_stdio();
let supports_http = init.mcp("http");
let supports_sse = init.mcp("sse");
let mut seen = HashSet::new();
let mut servers = Vec::new();
let ordered =
all.iter().filter(|s| s.scope == McpScope::Workspace).chain(all.iter().filter(|s| s.scope == McpScope::User));
for srv in ordered {
if !srv.enabled || !matches_agent(&srv.agents, provider) || !seen.insert(srv.name.clone()) {
continue;
}
match srv.transport {
McpTransport::Stdio if supports_stdio => {
let Some(cmd) = srv.command.as_deref().filter(|c| !c.trim().is_empty()) else {
continue;
};
let env: Vec<Value> = srv.env.iter().map(|kv| json!({ "name": kv.name, "value": kv.value })).collect();
servers.push(json!({
"name": srv.name,
"command": cmd,
"args": srv.args,
"env": env,
}));
}
McpTransport::Http if supports_http => {
let Some(url) = srv.url.as_deref().filter(|u| !u.trim().is_empty()) else {
continue;
};
let headers: Vec<Value> =
srv.headers.iter().map(|kv| json!({ "name": kv.name, "value": kv.value })).collect();
servers.push(json!({
"type": "http",
"name": srv.name,
"url": url,
"headers": headers,
}));
}
McpTransport::Sse if supports_sse => {
let Some(url) = srv.url.as_deref().filter(|u| !u.trim().is_empty()) else {
continue;
};
let headers: Vec<Value> =
srv.headers.iter().map(|kv| json!({ "name": kv.name, "value": kv.value })).collect();
servers.push(json!({
"type": "sse",
"name": srv.name,
"url": url,
"headers": headers,
}));
}
_ => {}
}
}
servers
}
/// Applies a create or update operation to the User / Workspace MCP server maps.
pub fn apply_save(
user: &mut BTreeMap<String, AgentMcpServer>,
workspace: &mut BTreeMap<String, AgentMcpServer>,
has_workspace: bool,
input: McpServerInput,
) -> Result<(), String> {
let name = input.name.trim();
if name.is_empty() {
return Err("Server name is required".into());
}
if input.scope == McpScope::Workspace && !has_workspace {
return Err("Open a project to configure workspace MCP servers".into());
}
let cfg = match input.transport {
McpTransport::Stdio => {
let cmd = input.command.as_deref().map(str::trim).unwrap_or_default();
if cmd.is_empty() {
return Err("Command is required for stdio MCP servers".into());
}
let env = input
.env
.into_iter()
.filter_map(|kv| {
let k = kv.name.trim();
(!k.is_empty()).then(|| (k.to_owned(), kv.value))
})
.collect();
AgentMcpServer {
transport: McpTransport::Stdio,
enabled: input.enabled,
command: Some(cmd.to_owned()),
args: input.args,
env,
url: None,
headers: BTreeMap::new(),
agents: normalize_agents_list(input.agents),
}
}
McpTransport::Http | McpTransport::Sse => {
let url = input.url.as_deref().map(str::trim).unwrap_or_default();
if url.is_empty() {
return Err("URL is required for HTTP/SSE MCP servers".into());
}
let headers = input
.headers
.into_iter()
.filter_map(|kv| {
let k = kv.name.trim();
(!k.is_empty()).then(|| (k.to_owned(), kv.value))
})
.collect();
AgentMcpServer {
transport: input.transport,
enabled: input.enabled,
command: None,
args: Vec::new(),
env: BTreeMap::new(),
url: Some(url.to_owned()),
headers,
agents: normalize_agents_list(input.agents),
}
}
};
if let Some(prev_id) = input.id.as_deref()
&& let Some((prev_scope, prev_name)) = parse_server_id(prev_id)
&& (prev_scope != input.scope || prev_name != name)
{
match prev_scope {
McpScope::User => {
user.remove(prev_name);
}
McpScope::Workspace => {
workspace.remove(prev_name);
}
}
}
match input.scope {
McpScope::User => {
user.insert(name.to_owned(), cfg);
}
McpScope::Workspace => {
workspace.insert(name.to_owned(), cfg);
}
}
Ok(())
}
/// Enables or disables a User or Workspace MCP server by ID.
pub fn apply_set_enabled(
user: &mut BTreeMap<String, AgentMcpServer>,
workspace: &mut BTreeMap<String, AgentMcpServer>,
id: &str,
enabled: bool,
) -> Result<McpScope, String> {
let (scope, name) = parse_server_id(id).ok_or_else(|| format!("MCP server `{id}` not found"))?;
let map = match scope {
McpScope::User => user,
McpScope::Workspace => workspace,
};
let srv = map.get_mut(name).ok_or_else(|| format!("MCP server `{name}` not found"))?;
srv.enabled = enabled;
Ok(scope)
}
/// Removes a User or Workspace MCP server by ID.
pub fn apply_remove(
user: &mut BTreeMap<String, AgentMcpServer>,
workspace: &mut BTreeMap<String, AgentMcpServer>,
id: &str,
) -> Result<McpScope, String> {
let (scope, name) = parse_server_id(id).ok_or_else(|| format!("MCP server `{id}` not found"))?;
let map = match scope {
McpScope::User => user,
McpScope::Workspace => workspace,
};
if map.remove(name).is_none() {
return Err(format!("MCP server `{name}` not found"));
}
Ok(scope)
}
/// Imports MCP servers from `<workspace_root>/.mcp.json` and `<workspace_root>/.vscode/mcp.json`
/// into `workspace` without modifying the external files. Returns the number of newly added servers.
pub fn apply_import(workspace: &mut BTreeMap<String, AgentMcpServer>, workspace_root: &Path) -> usize {
let mut added = 0;
for rel in [".mcp.json", ".vscode/mcp.json"] {
let path = workspace_root.join(rel);
for (name, srv) in read_external_file(&path) {
if let std::collections::btree_map::Entry::Vacant(e) = workspace.entry(name) {
e.insert(srv);
added += 1;
}
}
}
added
}
fn normalize_agents_list(agents: Vec<String>) -> Vec<String> {
let mut out = Vec::new();
for a in agents {
let trimmed = a.trim();
if !trimmed.is_empty() && !out.iter().any(|existing: &String| existing == trimmed) {
out.push(trimmed.to_owned());
}
}
out
}
fn read_external_file(path: &Path) -> BTreeMap<String, AgentMcpServer> {
let Ok(bytes) = std::fs::read(path) else {
return BTreeMap::new();
};
let Ok(root) = serde_json::from_slice::<Value>(&bytes) else {
return BTreeMap::new();
};
let Some(servers_obj) = root.get("mcpServers").or_else(|| root.get("servers")).and_then(Value::as_object) else {
return BTreeMap::new();
};
let mut out = BTreeMap::new();
for (name, def) in servers_obj {
let name = name.trim();
let Some(obj) = def.as_object().filter(|_| !name.is_empty()) else {
continue;
};
let disabled = obj.get("disabled").and_then(Value::as_bool).unwrap_or(false)
|| obj.get("enabled").and_then(Value::as_bool) == Some(false);
let type_str = obj.get("type").and_then(Value::as_str).unwrap_or_default().to_ascii_lowercase();
let url = obj
.get("url")
.or_else(|| obj.get("serverUrl"))
.and_then(Value::as_str)
.map(str::trim)
.filter(|s| !s.is_empty())
.map(str::to_owned);
let command =
obj.get("command").and_then(Value::as_str).map(str::trim).filter(|s| !s.is_empty()).map(str::to_owned);
let transport = match type_str.as_str() {
"sse" => McpTransport::Sse,
"http" | "streamable-http" | "streamable_http" => McpTransport::Http,
"stdio" | "local" => McpTransport::Stdio,
_ if url.is_some() && command.is_none() => McpTransport::Http,
_ => McpTransport::Stdio,
};
match transport {
McpTransport::Stdio if command.is_none() => continue,
McpTransport::Http | McpTransport::Sse if url.is_none() => continue,
_ => {}
}
let args = obj
.get("args")
.and_then(Value::as_array)
.map(|arr| arr.iter().filter_map(|v| v.as_str().map(str::to_owned)).collect())
.unwrap_or_default();
out.insert(
name.to_owned(),
AgentMcpServer {
transport,
enabled: !disabled,
command,
args,
env: parse_key_values_map(obj.get("env")),
url,
headers: parse_key_values_map(obj.get("headers")),
agents: Vec::new(),
},
);
}
out
}
fn parse_key_values_map(val: Option<&Value>) -> BTreeMap<String, String> {
match val {
Some(Value::Object(map)) => map
.iter()
.filter_map(|(k, v)| {
let s = match v {
Value::String(s) => s.clone(),
Value::Number(n) => n.to_string(),
Value::Bool(b) => b.to_string(),
_ => return None,
};
Some((k.clone(), s))
})
.collect(),
Some(Value::Array(arr)) => arr
.iter()
.filter_map(|item| {
let obj = item.as_object()?;
let name = obj.get("name")?.as_str()?.to_owned();
let value = obj.get("value")?.as_str()?.to_owned();
Some((name, value))
})
.collect(),
_ => BTreeMap::new(),
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn crud_and_acp_filtering_by_scope_agent_and_transport() {
let mut user = BTreeMap::new();
let mut workspace = BTreeMap::new();
apply_save(
&mut user,
&mut workspace,
true,
McpServerInput {
id: None,
name: "shared".into(),
scope: McpScope::User,
transport: McpTransport::Stdio,
enabled: true,
command: Some("npx".into()),
args: vec!["-y".into(), "user-pkg".into()],
env: vec![McpKeyValue { name: "TOKEN".into(), value: "u1".into() }],
url: None,
headers: Vec::new(),
agents: Vec::new(),
},
)
.unwrap();
apply_save(
&mut user,
&mut workspace,
true,
McpServerInput {
id: None,
name: "shared".into(),
scope: McpScope::Workspace,
transport: McpTransport::Stdio,
enabled: true,
command: Some("npx".into()),
args: vec!["-y".into(), "workspace-pkg".into()],
env: Vec::new(),
url: None,
headers: Vec::new(),
agents: Vec::new(),
},
)
.unwrap();
apply_save(
&mut user,
&mut workspace,
true,
McpServerInput {
id: None,
name: "claude-http".into(),
scope: McpScope::User,
transport: McpTransport::Http,
enabled: true,
command: None,
args: Vec::new(),
env: Vec::new(),
url: Some("https://example.com/mcp".into()),
headers: vec![McpKeyValue { name: "Authorization".into(), value: "Bearer secret".into() }],
agents: vec!["claude".into()],
},
)
.unwrap();
let list = servers_from_settings(&user, &workspace);
assert_eq!(list.len(), 3);
// Negotiated with HTTP support: `claude` gets workspace `shared` + `claude-http`; `codex` only gets `shared`.
let init_http = Negotiated::from_initialize(&json!({
"agentCapabilities": { "mcpCapabilities": { "http": true, "sse": false } }
}));
let claude_servers = acp_mcp_servers(&list, "claude-acp", &init_http);
assert_eq!(claude_servers.len(), 2);
assert_eq!(claude_servers[0]["name"], "shared");
assert_eq!(claude_servers[0]["args"], json!(["-y", "workspace-pkg"]));
assert_eq!(claude_servers[1]["name"], "claude-http");
assert_eq!(claude_servers[1]["type"], "http");
let codex_servers = acp_mcp_servers(&list, "codex", &init_http);
assert_eq!(codex_servers.len(), 1);
assert_eq!(codex_servers[0]["name"], "shared");
// Disable workspace `shared` and remove user `claude-http`.
apply_set_enabled(&mut user, &mut workspace, "workspace::shared", false).unwrap();
apply_remove(&mut user, &mut workspace, "user::claude-http").unwrap();
let list = servers_from_settings(&user, &workspace);
let after = acp_mcp_servers(&list, "claude", &init_http);
assert_eq!(after.len(), 1);
assert_eq!(after[0]["args"], json!(["-y", "user-pkg"]));
}
#[test]
fn imports_from_mcp_json_and_vscode_mcp_json_without_modifying_them() {
let tmp = tempfile::tempdir().unwrap();
let mcp_json = tmp.path().join(".mcp.json");
let vscode_dir = tmp.path().join(".vscode");
std::fs::create_dir_all(&vscode_dir).unwrap();
let vscode_mcp = vscode_dir.join("mcp.json");
let raw_mcp = r#"{"mcpServers":{"sqlite":{"command":"uvx","args":["mcp-server-sqlite"]}}}"#;
let raw_vscode = r#"{"servers":{"remote":{"type":"http","url":"https://mcp.example.com"}}}"#;
std::fs::write(&mcp_json, raw_mcp).unwrap();
std::fs::write(&vscode_mcp, raw_vscode).unwrap();
let mut workspace = BTreeMap::new();
let added = apply_import(&mut workspace, tmp.path());
assert_eq!(added, 2);
assert_eq!(workspace["sqlite"].command.as_deref(), Some("uvx"));
assert_eq!(workspace["remote"].transport, McpTransport::Http);
assert_eq!(workspace["remote"].url.as_deref(), Some("https://mcp.example.com"));
// Re-importing does not duplicate or overwrite existing entries, and leaves files untouched.
workspace.get_mut("sqlite").unwrap().enabled = false;
assert_eq!(apply_import(&mut workspace, tmp.path()), 0);
assert!(!workspace["sqlite"].enabled);
assert_eq!(std::fs::read_to_string(&mcp_json).unwrap(), raw_mcp);
assert_eq!(std::fs::read_to_string(&vscode_mcp).unwrap(), raw_vscode);
}
}
+1 -166
View File
@@ -384,10 +384,6 @@ pub enum AgentEntry {
Permission {
id: u32,
title: String,
/// The shell command the request asks to run, from its structured input; `None` for
/// any other request, whatever its title says.
#[serde(default)]
command: Option<String>,
options: Vec<AgentPermissionOption>,
/// Set when the request asks `AskUserQuestion`'s questions; answered with a JSON object
/// of question to answer instead of an option.
@@ -488,8 +484,7 @@ pub struct NewAgentTask {
pub base: Option<String>,
}
/// A file the task changed: against [`AgentTask::base_commit`], or against `HEAD` in the
/// grouped view.
/// A file the task changed, relative to [`AgentTask::base_commit`].
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentChange {
@@ -498,44 +493,6 @@ pub struct AgentChange {
pub kind: ide_vcs::ChangeKind,
pub added: u32,
pub removed: u32,
/// Path before a rename; the left side of its diff reads this path.
pub old_path: Option<String>,
}
/// A commit made while the task ran, with the files it changed against its first parent.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentCommit {
pub id: String,
pub short_id: String,
pub summary: String,
pub message: String,
pub time_ms: f64,
/// First parent, or `None` for a root commit.
pub parent: Option<String>,
pub files: Vec<AgentChange>,
}
/// Which comparison a rollback of the task's changes uses.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum AgentChangesView {
/// Everything since the commit the task started from.
#[default]
SinceStart,
/// The uncommitted files against `HEAD`; commits made since the start stay.
Group,
}
/// A task's uncommitted files and the commits made since it started.
#[derive(Clone, Debug, Default, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentChanges {
/// Files changed against `HEAD`, less those changed before the task started and untouched since.
pub uncommitted: Vec<AgentChange>,
/// Commits from the task's start (exclusive) to `HEAD` (inclusive), newest first. Empty
/// for tasks working in the project's own checkout.
pub commits: Vec<AgentCommit>,
}
/// Streamed to the window that subscribed (`AgentManager::subscribe`).
@@ -561,15 +518,6 @@ pub enum AgentEvent {
index: u32,
entry: AgentEntry,
},
/// Text was appended to entry `index` (an agent message or a thought): it read the first
/// `from` UTF-16 units of its text when the previous event for it was sent.
Append {
task: String,
subagent: Option<String>,
index: u32,
from: u32,
text: String,
},
/// An agent reported the usage limits of its account (shared by every task of `provider`).
Limits {
provider: String,
@@ -578,116 +526,3 @@ pub enum AgentEvent {
/// The installed or configured agent providers changed.
Providers,
}
pub use ide_core::settings::{AgentMcpServer, McpTransport};
/// Where an MCP server configuration is stored (`settings.toml` vs `.openidea/settings.toml`).
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum McpScope {
User,
Workspace,
}
/// A named environment variable or HTTP header for an MCP server.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct McpKeyValue {
pub name: String,
pub value: String,
}
/// One configured Model Context Protocol (MCP) server.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct McpServer {
pub id: String,
pub name: String,
pub scope: McpScope,
pub transport: McpTransport,
pub enabled: bool,
pub command: Option<String>,
pub args: Vec<String>,
pub env: Vec<McpKeyValue>,
pub url: Option<String>,
pub headers: Vec<McpKeyValue>,
/// Agent provider ids this server is enabled for; empty means enabled for all agents.
#[serde(default)]
pub agents: Vec<String>,
}
/// Payload for creating or updating a user or workspace MCP server.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct McpServerInput {
#[serde(default)]
pub id: Option<String>,
pub name: String,
pub scope: McpScope,
pub transport: McpTransport,
#[serde(default = "default_true")]
pub enabled: bool,
#[serde(default)]
pub command: Option<String>,
#[serde(default)]
pub args: Vec<String>,
#[serde(default)]
pub env: Vec<McpKeyValue>,
#[serde(default)]
pub url: Option<String>,
#[serde(default)]
pub headers: Vec<McpKeyValue>,
/// Agent provider ids this server is enabled for; empty means enabled for all agents.
#[serde(default)]
pub agents: Vec<String>,
}
fn default_true() -> bool {
true
}
/// Payload for creating a custom ACP agent (always the ACP driver); the id
/// becomes the `<config>/agents/<id>.toml` file name.
#[derive(Clone, Debug, Deserialize, Type)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct CustomAgentInput {
pub id: String,
pub name: String,
pub command: String,
#[serde(default)]
pub args: Vec<String>,
#[serde(default)]
pub env: BTreeMap<String, String>,
}
/// What importing one entry of a JetBrains `acp.json` file did.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AcpImportOutcome {
/// The `agent_servers` key the entry was imported from.
pub name: String,
/// The id its agent file would get (also when skipped).
pub id: String,
pub saved: bool,
/// Why the entry was skipped; empty when it was saved.
#[serde(default)]
pub reason: String,
}
/// What accepting a worktree task's changes copied into the local checkout.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentAcceptOutcome {
/// Relative paths copied into the checkout (uncommitted), including ones already identical.
pub accepted: Vec<String>,
/// Files left alone because the checkout has its own edits to them.
pub skipped: Vec<AgentAcceptSkipped>,
}
/// A file accept left alone, with why.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentAcceptSkipped {
pub path: String,
pub reason: String,
}
+1 -1
View File
@@ -135,7 +135,7 @@ impl Cli for Process {
}
/// Claude Code's configuration directory (`CLAUDE_CONFIG_DIR`, else `~/.claude`).
pub(crate) fn config_dir(env: &[(String, String)]) -> Option<PathBuf> {
pub(super) fn config_dir(env: &[(String, String)]) -> Option<PathBuf> {
let set = env
.iter()
.find(|(k, _)| k == "CLAUDE_CONFIG_DIR")
@@ -31,11 +31,11 @@ pub(super) fn transcript(config_dir: &Path, cwd: &Path, id: &str) -> PathBuf {
}
/// The project directory name Claude Code uses for `cwd`.
pub(crate) fn escape(cwd: &Path) -> String {
pub(super) fn escape(cwd: &Path) -> String {
cwd.to_string_lossy().chars().map(|c| if c.is_ascii_alphanumeric() { c } else { '-' }).collect()
}
pub(crate) fn read(path: &Path) -> std::io::Result<Vec<Value>> {
pub(super) fn read(path: &Path) -> std::io::Result<Vec<Value>> {
let text = std::fs::read_to_string(path)?;
Ok(text.lines().filter_map(|l| serde_json::from_str(l).ok()).collect())
}
@@ -111,7 +111,7 @@ fn block_ids<'a>(e: &'a Value, ty: &'a str, key: &'a str) -> impl Iterator<Item
/// Whether `e` is a message that started a turn: the user's own, not a tool result, a
/// steer, or text Claude Code added.
pub(crate) fn is_turn_start(e: &Value) -> bool {
pub(super) fn is_turn_start(e: &Value) -> bool {
if kind(e) != "user" || e.get("isMeta").and_then(Value::as_bool) == Some(true) {
return false;
}
@@ -149,7 +149,7 @@ pub(super) fn cut(chain: &[&Value], turns: usize) -> Result<Option<String>, Stri
}
/// The newest title Claude Code gave the session.
pub(crate) fn title(entries: &[Value]) -> Option<String> {
pub(super) fn title(entries: &[Value]) -> Option<String> {
entries
.iter()
.rev()
+4 -16
View File
@@ -7,8 +7,8 @@
//! <id>.json`): loading that session, from this connection or a fresh one, starts it that
//! way.
pub(crate) mod cli;
pub(crate) mod history;
mod cli;
mod history;
mod settings;
mod tools;
mod translate;
@@ -69,17 +69,9 @@ pub(crate) fn connect(
Ok(super::serve(shim, client))
}
/// What discovery found: ACP `availableCommands`, `configOptions` for the configured model,
/// and each model's effort option by model value (`None`: none).
pub(crate) struct Discovered {
pub commands: Vec<Value>,
pub options: Value,
pub efforts: Vec<(String, Option<Value>)>,
}
/// Discover the slash commands and the model, mode and effort options without a prompt,
/// ACP session, or pending session file.
pub(crate) async fn discover(spec: ProviderSpec, cwd: PathBuf) -> Result<Discovered, String> {
pub(crate) async fn discover(spec: ProviderSpec, cwd: PathBuf) -> Result<(Vec<Value>, Value), String> {
tokio::time::timeout(Duration::from_secs(5), async move {
let spec = tokio::task::spawn_blocking(move || (spec.resolve_for_launch(&cwd), cwd));
let (spec, cwd) = spec.await.map_err(|e| e.to_string())?;
@@ -108,11 +100,7 @@ pub(crate) async fn discover(spec: ProviderSpec, cwd: PathBuf) -> Result<Discove
.flatten()
.unwrap_or_default();
let settings = Settings::from_cli(&init, &applied);
Ok(Discovered {
commands: settings.commands.clone(),
options: settings.options(),
efforts: settings.efforts_by_model(),
})
Ok((settings.commands.clone(), settings.options()))
})
.await
.map_err(|_| "Claude Code command discovery timed out".to_owned())?
+9 -38
View File
@@ -132,18 +132,19 @@ impl Settings {
}).collect::<Vec<_>>(),
}));
}
if let Some(effort) = self.current_model().and_then(|m| effort_option(m, &self.effort)) {
out.push(effort);
let efforts = self.current_model().map(|m| m.efforts.as_slice()).unwrap_or_default();
if !efforts.is_empty() {
let mut values = vec![json!({ "value": "default", "name": "Default" })];
values.extend(efforts.iter().map(|e| json!({ "value": e, "name": capitalize(e) })));
let current = if efforts.contains(&self.effort) { self.effort.as_str() } else { "default" };
out.push(json!({
"id": EFFORT, "name": "Effort", "category": "thought_level", "type": "select",
"currentValue": current, "options": values,
}));
}
Value::Array(out)
}
/// Each model's effort option by model value (`None`: it has no effort levels): what a new
/// task offers once that model is picked, before any session has run it.
pub fn efforts_by_model(&self) -> Vec<(String, Option<Value>)> {
self.models.iter().map(|m| (m.value.clone(), effort_option(m, &self.effort))).collect()
}
pub fn valid(&self, option: &str, value: &str) -> bool {
self.options()
.as_array()
@@ -155,20 +156,6 @@ impl Settings {
}
}
/// `model`'s effort option, with `effort` selected when the model offers it; `None` without efforts.
fn effort_option(model: &Model, effort: &str) -> Option<Value> {
if model.efforts.is_empty() {
return None;
}
let mut values = vec![json!({ "value": "default", "name": "Default" })];
values.extend(model.efforts.iter().map(|e| json!({ "value": e, "name": capitalize(e) })));
let current = if model.efforts.iter().any(|e| e == effort) { effort } else { "default" };
Some(json!({
"id": EFFORT, "name": "Effort", "category": "thought_level", "type": "select",
"currentValue": current, "options": values,
}))
}
/// Slash commands as ACP `availableCommands`, without those that need Claude Code's terminal.
pub(super) fn commands(list: Option<&Value>, terminal_only: &[String]) -> Vec<Value> {
list.and_then(Value::as_array)
@@ -244,20 +231,4 @@ mod tests {
let cmds = commands(init().get("commands"), &["doctor".into()]);
assert_eq!(cmds, vec![json!({"name": "review", "description": "Review", "input": {"hint": "[pr]"}})]);
}
#[test]
fn every_model_offers_its_own_efforts() {
let s = Settings::from_cli(&init(), &json!({"applied": {"model": "claude-sonnet-5-5", "effort": "high"}}));
let efforts = s.efforts_by_model();
let values = |model: &str| {
let (_, o) = efforts.iter().find(|(m, _)| m == model).unwrap();
let o = o.as_ref().unwrap();
let values: Vec<_> = o["options"].as_array().unwrap().iter().map(|v| v["value"].clone()).collect();
(values, o["currentValue"].clone())
};
assert_eq!(values("default"), (vec![json!("default"), json!("low"), json!("high")], json!("high")));
// The configured effort only where the model offers it.
assert_eq!(values("opus"), (vec![json!("default"), json!("low"), json!("max")], json!("default")));
assert!(efforts.contains(&("claude-haiku-4-5".into(), None)), "no effort option without levels");
}
}
+7 -8
View File
@@ -869,14 +869,13 @@ impl CodexShim {
"item/commandExecution/requestApproval" => {
let choices = translate::decisions(params, true);
let title = translate::command_approval_title(params, item.as_ref());
let mut input = json!({
"command": translate::approval_command(params, item.as_ref()),
"cwd": params.get("cwd"),
});
if let Some(network) = params.get("networkApprovalContext").filter(|c| !c.is_null()) {
input["networkApprovalContext"] = network.clone();
}
let call = tool_call_of(&item_id, &title, "execute", reason, Some(input));
let call = tool_call_of(
&item_id,
&title,
"execute",
reason,
Some(json!({ "command": params.get("command"), "cwd": params.get("cwd") })),
);
Ok(self.ask(&thread, call, &choices, decline))
}
"item/fileChange/requestApproval" => {
+8 -12
View File
@@ -757,23 +757,19 @@ pub(super) fn answer_of(choices: &[Choice], outcome: &Value, decline: Value) ->
choices.iter().find(|c| selected && c.option["optionId"].as_str() == chosen).map_or(decline, |c| c.answer.clone())
}
/// The command a command approval is about: the request's own, else its item's.
pub(super) fn approval_command<'a>(params: &'a Value, item: Option<&'a Value>) -> Option<&'a str> {
// Approval payloads can describe a different command from the item's earlier summary.
let item = item.filter(|i| i.get("id") == params.get("itemId"));
params
.get("command")
.and_then(Value::as_str)
.filter(|c| !c.is_empty())
.or_else(|| item.and_then(|i| i.get("command")).and_then(Value::as_str).filter(|c| !c.is_empty()))
}
/// The ACP title of a command approval.
pub(super) fn command_approval_title(params: &Value, item: Option<&Value>) -> String {
if let Some(host) = params.pointer("/networkApprovalContext/host").and_then(Value::as_str) {
return format!("Allow network access to {host}");
}
match approval_command(params, item) {
// Approval payloads can describe a different command from the item's earlier summary.
let item = item.filter(|i| i.get("id") == params.get("itemId"));
let command = params
.get("command")
.and_then(Value::as_str)
.filter(|c| !c.is_empty())
.or_else(|| item.and_then(|i| i.get("command")).and_then(Value::as_str).filter(|c| !c.is_empty()));
match command {
Some(command) => format!("Run {}", crate::shell_script(command).unwrap_or_else(|| command.to_owned())),
None => "Run a command".to_owned(),
}
+35 -442
View File
@@ -2,7 +2,7 @@
//! `<config>/agents/<id>.toml` (written by hand, or by installing one from the ACP
//! Registry, see `registry`).
use crate::{AcpImportOutcome, AgentCommandReadOnlyReason, AgentProvider, AgentSource, CustomAgentInput, EnvSecrets};
use crate::{AgentCommandReadOnlyReason, AgentProvider, AgentSource};
use ide_core::{IdeError, IdeResult};
use serde::{Deserialize, Serialize};
use serde_json::Value;
@@ -59,14 +59,11 @@ const BUILTIN: &[(&str, &str, Driver, &[&str], &str)] = &[
("cursor", "Cursor", Driver::Acp, &["cursor-agent", "acp"], "Run `cursor-agent login` in a terminal."),
("opencode", "OpenCode", Driver::Acp, &["opencode", "acp"], "Run `opencode auth login` in a terminal."),
("qwen", "Qwen Code", Driver::Acp, &["qwen", "--experimental-acp"], "Run `qwen` in a terminal and sign in."),
("qoder", "Qoder", Driver::Acp, &["qodercli", "--acp"], "Run `qodercli login` in a terminal."),
("qodercn", "Qoder CN", Driver::Acp, &["qoderclicn", "--acp"], "Run `qoderclicn login` in a terminal."),
("kimi", "Kimi CLI", Driver::Acp, &["kimi", "acp"], "Run `kimi` in a terminal and sign in."),
("goose", "Goose", Driver::Acp, &["goose", "acp"], "Run `goose configure` in a terminal."),
("auggie", "Auggie", Driver::Acp, &["auggie", "--acp"], "Run `auggie login` in a terminal."),
("kiro", "Kiro", Driver::Acp, &["kiro-cli", "acp"], "Run `kiro-cli login` in a terminal."),
("vibe", "Mistral Vibe", Driver::Acp, &["vibe-acp"], "Run `vibe` in a terminal and sign in."),
("omp", "OMP", Driver::Acp, &["omp", "acp"], "Run `omp` in a terminal and sign in."),
];
fn builtins() -> Vec<ProviderSpec> {
@@ -187,14 +184,6 @@ pub(crate) fn builtin_spec(id: &str) -> Option<ProviderSpec> {
builtins().into_iter().find(|s| s.id == id)
}
/// The ACP adapters of Claude Code and Codex, which the native drivers replace: they still
/// launch, but are listed only once customized through an agent file or used by a task.
const UNLISTED: &[&str] = &["claude-acp", "codex-acp"];
pub(crate) fn is_unlisted(spec: &ProviderSpec) -> bool {
UNLISTED.contains(&spec.id.as_str()) && builtin_spec(&spec.id).as_ref() == Some(spec)
}
/// Public launch metadata, keeping the built-in defaults separate from agent-file overrides.
pub(crate) fn describe(spec: &ProviderSpec) -> AgentProvider {
let builtin = builtin_spec(&spec.id);
@@ -231,40 +220,23 @@ pub(crate) fn describe(spec: &ProviderSpec) -> AgentProvider {
/// Overrides built-in `id`'s command through its agent file, preserving other
/// customizations. `None` restores the built-in command and removes a redundant file.
pub(crate) fn set_builtin_command(
custom_dir: &Path,
secrets: Option<&dyn EnvSecrets>,
id: &str,
command: Option<&str>,
) -> IdeResult<()> {
update_builtin(custom_dir, secrets, id, |agent, builtin| {
agent.command = command.unwrap_or(&builtin.command).to_owned()
})
pub(crate) fn set_builtin_command(custom_dir: &Path, id: &str, command: Option<&str>) -> IdeResult<()> {
update_builtin(custom_dir, id, |agent, builtin| agent.command = command.unwrap_or(&builtin.command).to_owned())
}
/// The environment built-in `id` starts with, through its agent file (empty clears it).
pub(crate) fn set_builtin_env(
custom_dir: &Path,
secrets: Option<&dyn EnvSecrets>,
id: &str,
env: BTreeMap<String, String>,
) -> IdeResult<()> {
update_builtin(custom_dir, secrets, id, |agent, _| agent.env = env)
pub(crate) fn set_builtin_env(custom_dir: &Path, id: &str, env: BTreeMap<String, String>) -> IdeResult<()> {
update_builtin(custom_dir, id, |agent, _| agent.env = env)
}
/// Changes built-in `id`'s agent file; a file left with only the built-in's values is removed.
fn update_builtin(
custom_dir: &Path,
secrets: Option<&dyn EnvSecrets>,
id: &str,
change: impl FnOnce(&mut CustomAgent, &ProviderSpec),
) -> IdeResult<()> {
fn update_builtin(custom_dir: &Path, id: &str, change: impl FnOnce(&mut CustomAgent, &ProviderSpec)) -> IdeResult<()> {
let Some(builtin) = builtin_spec(id) else {
return Err(IdeError::invalid(format!("{id} is not a built-in agent")));
};
let file = custom_dir.join(format!("{id}.toml"));
let mut agent = if file.try_exists()? {
CustomAgent::read_with(&file, id, secrets).map_err(IdeError::invalid)?
CustomAgent::read(&file).map_err(IdeError::invalid)?
} else {
CustomAgent {
name: Some(builtin.name.clone()),
@@ -289,145 +261,14 @@ fn update_builtin(
Err(e) if e.kind() != std::io::ErrorKind::NotFound => return Err(e.into()),
_ => {}
}
forget_env(secrets, id, &[]);
return Ok(());
}
agent.write_with(&file, id, secrets)
}
/// Saves a new custom agent; refuses built-in and ACP Registry ids, taken ids
/// and an empty command. The id is also the agent file's name, so it must be
/// a safe file stem.
pub(crate) fn save_custom(
custom_dir: &Path,
secrets: Option<&dyn EnvSecrets>,
input: &CustomAgentInput,
) -> IdeResult<()> {
let id = validated_custom_id(&input.id)?;
if builtin_spec(&id).is_some() {
return Err(IdeError::invalid(format!("{id} is a built-in agent; change its command instead")));
}
if input.command.trim().is_empty() {
return Err(IdeError::invalid("the agent command must not be empty"));
}
let file = custom_dir.join(format!("{id}.toml"));
if file.try_exists()? {
if CustomAgent::read(&file).is_ok_and(|a| a.registry.is_some()) {
return Err(IdeError::invalid(format!(
"{id} is installed from the ACP Registry; uninstall it there to replace it"
)));
}
return Err(IdeError::invalid(format!("there is already a custom agent '{id}'")));
}
CustomAgent {
name: Some(input.name.clone()),
driver: Driver::Acp,
command: input.command.clone(),
args: input.args.clone(),
env: input.env.clone(),
registry: None,
}
.write_with(&file, &id, secrets)
}
/// Removes a custom agent's file; refuses built-in, ACP Registry and missing ids.
pub(crate) fn remove_custom(custom_dir: &Path, secrets: Option<&dyn EnvSecrets>, id: &str) -> IdeResult<()> {
let id = validated_custom_id(id)?;
let id = id.as_str();
if builtin_spec(id).is_some() {
return Err(IdeError::invalid(format!("{id} is a built-in agent and cannot be removed")));
}
let file = custom_dir.join(format!("{id}.toml"));
let agent =
CustomAgent::read(&file).map_err(|_| IdeError::not_found(format!("there is no custom agent '{id}'")))?;
if agent.registry.is_some() {
return Err(IdeError::invalid(format!("{id} is installed from the ACP Registry; uninstall it there")));
}
match std::fs::remove_file(&file) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => Err(e.into()),
_ => {
forget_env(secrets, id, &[]);
Ok(())
}
}
}
/// Imports a JetBrains `acp.json` file's `agent_servers` entries as custom
/// agents, one outcome per entry; entries that collide, lack a command or
/// have no usable name are skipped with a reason.
pub(crate) fn import_acp(
custom_dir: &Path,
secrets: Option<&dyn EnvSecrets>,
text: &str,
) -> IdeResult<Vec<AcpImportOutcome>> {
#[derive(Deserialize)]
struct AcpJson {
#[serde(default)]
agent_servers: BTreeMap<String, AcpServer>,
}
#[derive(Deserialize)]
struct AcpServer {
#[serde(default)]
command: String,
#[serde(default)]
args: Vec<String>,
#[serde(default)]
env: BTreeMap<String, String>,
}
let parsed: AcpJson =
serde_json::from_str(text).map_err(|e| IdeError::invalid(format!("not an acp.json file: {e}")))?;
let mut outcomes = Vec::with_capacity(parsed.agent_servers.len());
for (name, server) in parsed.agent_servers {
let id = slugify_id(&name);
let mut outcome = AcpImportOutcome { name, id: id.clone(), saved: false, reason: String::new() };
if id.is_empty() {
outcome.reason = "the name has no usable characters for an id".to_owned();
} else if builtin_spec(&id).is_some() {
outcome.reason = format!("{id} is a built-in agent");
} else if server.command.trim().is_empty() {
outcome.reason = "the entry has no command".to_owned();
} else if custom_dir.join(format!("{id}.toml")).try_exists()? {
outcome.reason = format!("there is already a custom agent '{id}'");
} else {
CustomAgent {
name: Some(outcome.name.clone()),
driver: Driver::Acp,
command: server.command,
args: server.args,
env: server.env,
registry: None,
}
.write_with(&custom_dir.join(format!("{id}.toml")), &id, secrets)?;
outcome.saved = true;
}
outcomes.push(outcome);
}
Ok(outcomes)
}
/// A custom agent id doubles as a file stem: letters, digits, `-` and `_` only.
fn validated_custom_id(id: &str) -> IdeResult<String> {
if !id.is_empty() && id.chars().all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_') {
return Ok(id.to_owned());
}
Err(IdeError::invalid(format!("'{id}' is not a usable agent id (letters, digits, '-' and '_' only)")))
}
/// Turns an `acp.json` server name into a file-stem id (`"My Agent!"` → `"my-agent"`).
fn slugify_id(name: &str) -> String {
let mut id = String::with_capacity(name.len());
for c in name.chars() {
if c.is_ascii_alphanumeric() {
id.push(c.to_ascii_lowercase());
} else if !id.ends_with('-') {
id.push('-');
}
}
id.trim_matches('-').to_owned()
agent.write(&file)?;
Ok(())
}
/// An agent file, `<config>/agents/<id>.toml`.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[derive(Debug, PartialEq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub(crate) struct CustomAgent {
#[serde(default, skip_serializing_if = "Option::is_none")]
@@ -474,67 +315,11 @@ impl CustomAgent {
std::fs::write(&tmp, text)?;
std::fs::rename(&tmp, path)
}
/// Reads agent `id`'s file with its environment values filled in from `secrets`; values
/// still written in the file (by hand, by an older version) move there first. A registry
/// install keeps the environment its registry entry gives it in the file.
pub(crate) fn read_with(path: &Path, id: &str, secrets: Option<&dyn EnvSecrets>) -> Result<Self, String> {
let mut agent = Self::read(path)?;
let Some(secrets) = secrets.filter(|_| agent.registry.is_none()) else {
return Ok(agent);
};
let written = agent.env.values().any(|v| !v.is_empty());
for (name, value) in agent.env.iter_mut() {
if value.is_empty()
&& let Some(saved) = secrets.get(&env_key(id, name))
{
*value = saved;
}
}
if written && let Err(e) = agent.write_with(path, id, Some(secrets)) {
tracing::warn!("agent {id}: could not move environment values out of {}: {}", path.display(), e.message);
}
Ok(agent)
}
/// Writes agent `id`'s file with the environment's names only, its values saved in `secrets`.
pub(crate) fn write_with(&self, path: &Path, id: &str, secrets: Option<&dyn EnvSecrets>) -> IdeResult<()> {
let Some(secrets) = secrets.filter(|_| self.registry.is_none()) else {
return Ok(self.write(path)?);
};
let mut keys = Vec::with_capacity(self.env.len());
for (name, value) in &self.env {
let key = env_key(id, name);
if value.is_empty() {
secrets.delete(&key);
} else if secrets.get(&key).as_deref() != Some(value.as_str()) {
secrets.set(&key, value)?;
}
keys.push(key);
}
forget_env(Some(secrets), id, &keys);
let names = self.env.keys().map(|name| (name.clone(), String::new())).collect();
Ok(Self { env: names, ..self.clone() }.write(path)?)
}
}
fn env_key(id: &str, name: &str) -> String {
format!("agent/{id}/env/{}", name.replace('%', "%25").replace('/', "%2F"))
}
/// Forgets agent `id`'s saved environment values, except `keep`.
fn forget_env(secrets: Option<&dyn EnvSecrets>, id: &str, keep: &[String]) {
let Some(secrets) = secrets else { return };
for key in secrets.keys(&format!("agent/{id}/env/")) {
if !keep.contains(&key) {
secrets.delete(&key);
}
}
}
/// Built-in agents, then the `*.toml` agents of `custom_dir`; a custom agent with a
/// built-in id replaces it.
pub(crate) fn load(custom_dir: Option<&Path>, secrets: Option<&dyn EnvSecrets>) -> Vec<ProviderSpec> {
pub(crate) fn load(custom_dir: Option<&Path>) -> Vec<ProviderSpec> {
let mut specs = builtins();
let mut files: Vec<PathBuf> = custom_dir
.and_then(|d| std::fs::read_dir(d).ok())
@@ -547,7 +332,7 @@ pub(crate) fn load(custom_dir: Option<&Path>, secrets: Option<&dyn EnvSecrets>)
files.sort();
for path in files {
let Some(id) = path.file_stem().and_then(|s| s.to_str()).map(str::to_owned) else { continue };
let agent = match CustomAgent::read_with(&path, &id, secrets) {
let agent = match CustomAgent::read(&path) {
Ok(a) => a,
Err(e) => {
tracing::warn!("skipping agent {}: {e}", path.display());
@@ -615,45 +400,6 @@ mod tests {
use super::*;
use serde_json::json;
#[test]
fn qoder_global_and_cn_editions_are_both_built_in() {
let info = |id: &str| describe(&load(None, None).into_iter().find(|s| s.id == id).unwrap());
let (global, cn) = (info("qoder"), info("qodercn"));
assert_eq!((global.name.as_str(), global.command.as_str()), ("Qoder", "qodercli --acp"));
assert_eq!((cn.name.as_str(), cn.command.as_str()), ("Qoder CN", "qoderclicn --acp"));
assert_eq!(global.login_hint.as_deref(), Some("Run `qodercli login` in a terminal."));
}
#[test]
fn qodercn_is_available_without_a_file_and_preserves_existing_customizations() {
let dir = tempfile::tempdir().unwrap();
let qodercn = load(None, None).into_iter().find(|s| s.id == "qodercn").unwrap();
assert_eq!(qodercn.driver, Driver::Acp);
assert!(!is_unlisted(&qodercn));
let info = describe(&qodercn);
assert_eq!(info.name, "Qoder CN");
assert_eq!(info.command, "qoderclicn --acp");
assert_eq!(info.source, AgentSource::Builtin);
assert!(info.command_editable);
assert_eq!(info.login_hint.as_deref(), Some("Run `qoderclicn login` in a terminal."));
std::fs::write(
dir.path().join("qodercn.toml"),
"command = \"/custom/qoderclicn\"\nargs = [\"--acp\"]\n[env]\nK = \"v\"\n",
)
.unwrap();
let specs = load(Some(dir.path()), None);
assert_eq!(specs.iter().filter(|s| s.id == "qodercn").count(), 1);
let custom = specs.iter().find(|s| s.id == "qodercn").unwrap();
assert_eq!(custom.command, "/custom/qoderclicn");
assert_eq!(custom.env, [("K".into(), "v".into())]);
set_builtin_command(dir.path(), None, "qodercn", None).unwrap();
let reset = load(Some(dir.path()), None).into_iter().find(|s| s.id == "qodercn").unwrap();
assert_eq!(reset.command, "qoderclicn");
assert_eq!(reset.args, ["--acp"]);
assert_eq!(reset.env, custom.env);
}
#[test]
fn login_hints_cover_all_builtin_commands_and_preserve_separate_login_tools() {
for spec in builtins() {
@@ -763,7 +509,7 @@ mod tests {
#[test]
fn launch_metadata_distinguishes_defaults_overrides_and_read_only_agents() {
let dir = tempfile::tempdir().unwrap();
let list = || load(Some(dir.path()), None).iter().map(describe).collect::<Vec<_>>();
let list = || load(Some(dir.path())).iter().map(describe).collect::<Vec<_>>();
let defaults = list();
let codex = defaults.iter().find(|p| p.id == "codex").unwrap();
assert_eq!(codex.program, "codex");
@@ -783,14 +529,14 @@ mod tests {
p.id.ends_with("-acp").then_some(AgentCommandReadOnlyReason::PackageRunner)
);
}
set_builtin_command(dir.path(), None, "codex", Some("/path with spaces/codex")).unwrap();
set_builtin_command(dir.path(), "codex", Some("/path with spaces/codex")).unwrap();
let overridden = list().into_iter().find(|p| p.id == "codex").unwrap();
assert_eq!(overridden.program, "/path with spaces/codex");
assert_eq!(overridden.args, codex.args);
assert_eq!(overridden.default_program, codex.default_program);
assert!(overridden.command_overridden && overridden.command_editable);
assert_eq!(overridden.source, AgentSource::Builtin);
set_builtin_command(dir.path(), None, "codex", Some("codex")).unwrap();
set_builtin_command(dir.path(), "codex", Some("codex")).unwrap();
assert!(!list().iter().find(|p| p.id == "codex").unwrap().command_overridden);
std::fs::write(
dir.path().join("gemini.toml"),
@@ -815,7 +561,7 @@ mod tests {
write("mine.toml", "name = \"Mine\"\ncommand = \"/opt/mine\"\nargs = [\"acp\"]\n[env]\nK = \"v\"\n");
write("gemini.toml", "command = \"gemini\"\nargs = [\"--acp\"]\n");
write("broken.toml", "nope = 1\n");
let specs = load(Some(dir.path()), None);
let specs = load(Some(dir.path()));
let mine = specs.iter().find(|s| s.id == "mine").unwrap();
assert_eq!((mine.name.as_str(), mine.command.as_str()), ("Mine", "/opt/mine"));
assert_eq!(mine.env, vec![("K".to_owned(), "v".to_owned())]);
@@ -831,35 +577,25 @@ mod tests {
fn set_builtin_env_writes_and_clears_the_environment() {
let dir = tempfile::tempdir().unwrap();
let env = BTreeMap::from([("CLAUDE_CODE_ENTRYPOINT".to_owned(), "openidea".to_owned())]);
set_builtin_env(dir.path(), None, "claude", env.clone()).unwrap();
let claude = load(Some(dir.path()), None).into_iter().find(|s| s.id == "claude").unwrap();
set_builtin_env(dir.path(), "claude", env.clone()).unwrap();
let claude = load(Some(dir.path())).into_iter().find(|s| s.id == "claude").unwrap();
assert_eq!(claude.env, vec![("CLAUDE_CODE_ENTRYPOINT".to_owned(), "openidea".to_owned())]);
assert_eq!(describe(&claude).env, env);
// The command override and the environment are kept independently.
set_builtin_command(dir.path(), None, "claude", Some("/opt/claude")).unwrap();
set_builtin_command(dir.path(), None, "claude", None).unwrap();
set_builtin_command(dir.path(), "claude", Some("/opt/claude")).unwrap();
set_builtin_command(dir.path(), "claude", None).unwrap();
assert!(dir.path().join("claude.toml").is_file());
set_builtin_env(dir.path(), None, "claude", BTreeMap::new()).unwrap();
set_builtin_env(dir.path(), "claude", BTreeMap::new()).unwrap();
assert!(!dir.path().join("claude.toml").exists());
set_builtin_env(dir.path(), None, "not-a-builtin", env).unwrap_err();
}
#[test]
fn acp_adapters_of_native_agents_are_unlisted_until_customized() {
let dir = tempfile::tempdir().unwrap();
let unlisted =
|dir: &Path| load(Some(dir), None).into_iter().filter(is_unlisted).map(|s| s.id).collect::<Vec<_>>();
assert_eq!(unlisted(dir.path()), ["claude-acp", "codex-acp"]);
set_builtin_env(dir.path(), None, "codex-acp", BTreeMap::from([("A".to_owned(), "1".to_owned())])).unwrap();
assert_eq!(unlisted(dir.path()), ["claude-acp"]);
set_builtin_env(dir.path(), "not-a-builtin", env).unwrap_err();
}
#[test]
fn set_builtin_command_overrides_and_clears_a_builtin() {
let dir = tempfile::tempdir().unwrap();
set_builtin_command(dir.path(), None, "not-a-builtin", Some("/x")).unwrap_err();
set_builtin_command(dir.path(), None, "claude", Some("/opt/claude")).unwrap();
let claude = load(Some(dir.path()), None).into_iter().find(|s| s.id == "claude").unwrap();
set_builtin_command(dir.path(), "not-a-builtin", Some("/x")).unwrap_err();
set_builtin_command(dir.path(), "claude", Some("/opt/claude")).unwrap();
let claude = load(Some(dir.path())).into_iter().find(|s| s.id == "claude").unwrap();
assert_eq!((claude.command.as_str(), claude.driver), ("/opt/claude", Driver::Claude));
assert!(claude.args.is_empty());
// A registry install is neither overwritten nor removed by a command override.
@@ -874,14 +610,14 @@ mod tests {
}
.write(&path)
.unwrap();
set_builtin_command(dir.path(), None, "codex", Some("/x")).unwrap_err();
set_builtin_command(dir.path(), None, "codex", None).unwrap_err();
set_builtin_command(dir.path(), "codex", Some("/x")).unwrap_err();
set_builtin_command(dir.path(), "codex", None).unwrap_err();
assert!(path.exists());
// Clearing (twice) returns the built-in to auto-detection.
set_builtin_command(dir.path(), None, "claude", None).unwrap();
set_builtin_command(dir.path(), None, "claude", None).unwrap();
set_builtin_command(dir.path(), "claude", None).unwrap();
set_builtin_command(dir.path(), "claude", None).unwrap();
assert!(!dir.path().join("claude.toml").exists());
assert_eq!(load(Some(dir.path()), None).iter().find(|s| s.id == "claude").unwrap().command, "claude");
assert_eq!(load(Some(dir.path())).iter().find(|s| s.id == "claude").unwrap().command, "claude");
}
#[test]
@@ -909,11 +645,11 @@ mod tests {
}
agent.write(&path).unwrap();
set_builtin_command(dir.path(), None, "claude", Some("/new/claude")).unwrap();
set_builtin_command(dir.path(), "claude", Some("/new/claude")).unwrap();
agent.command = "/new/claude".into();
assert_eq!(CustomAgent::read(&path).unwrap(), agent, "{field}");
set_builtin_command(dir.path(), None, "claude", None).unwrap();
set_builtin_command(dir.path(), "claude", None).unwrap();
agent.command = "claude".into();
assert_eq!(CustomAgent::read(&path).unwrap(), agent, "{field}");
}
@@ -929,11 +665,11 @@ mod tests {
)
.unwrap();
set_builtin_command(dir.path(), None, "codex", None).unwrap();
set_builtin_command(dir.path(), "codex", None).unwrap();
assert!(!path.exists());
assert_eq!(
load(Some(dir.path()), None).into_iter().find(|s| s.id == "codex").unwrap(),
load(Some(dir.path())).into_iter().find(|s| s.id == "codex").unwrap(),
builtin_spec("codex").unwrap()
);
}
@@ -954,151 +690,8 @@ mod tests {
let text = std::fs::read_to_string(&path).unwrap();
assert!(text.contains("[registry]") && !text.contains("driver"), "{text}");
assert_eq!(CustomAgent::read(&path).unwrap(), agent);
let spec = load(Some(path.parent().unwrap()), None).into_iter().find(|s| s.id == "amp-acp").unwrap();
let spec = load(Some(path.parent().unwrap())).into_iter().find(|s| s.id == "amp-acp").unwrap();
assert_eq!(spec.source, AgentSource::Registry { version: "0.10.0".into(), update: None });
assert!(toml::from_str::<CustomAgent>("command = \"x\"\n[registry]\nid = \"x\"\n").is_err());
}
fn custom_input(id: &str) -> CustomAgentInput {
CustomAgentInput {
id: id.into(),
name: "Mine".into(),
command: "/opt/mine".into(),
args: vec!["acp".into()],
env: BTreeMap::from([("K".to_owned(), "v".to_owned())]),
}
}
#[test]
fn custom_agents_save_list_and_remove() {
let dir = tempfile::tempdir().unwrap();
save_custom(dir.path(), None, &custom_input("mine")).unwrap();
let mine = load(Some(dir.path()), None).into_iter().find(|s| s.id == "mine").unwrap();
assert_eq!((mine.name.as_str(), mine.command.as_str()), ("Mine", "/opt/mine"));
assert_eq!(mine.env, vec![("K".to_owned(), "v".to_owned())]);
assert_eq!(mine.source, AgentSource::Custom);
remove_custom(dir.path(), None, "mine").unwrap();
assert!(!load(Some(dir.path()), None).iter().any(|s| s.id == "mine"));
remove_custom(dir.path(), None, "mine").unwrap_err();
}
#[test]
fn custom_agents_refuse_builtin_registry_and_bad_ids() {
let dir = tempfile::tempdir().unwrap();
save_custom(dir.path(), None, &custom_input("claude")).unwrap_err();
save_custom(dir.path(), None, &custom_input("../evil")).unwrap_err();
save_custom(dir.path(), None, &custom_input("")).unwrap_err();
let mut empty_command = custom_input("mine");
empty_command.command = " ".into();
save_custom(dir.path(), None, &empty_command).unwrap_err();
save_custom(dir.path(), None, &custom_input("mine")).unwrap();
save_custom(dir.path(), None, &custom_input("mine")).unwrap_err();
remove_custom(dir.path(), None, "claude").unwrap_err();
// An id that isn't a plain file stem never reaches outside the agents directory.
let outside = dir.path().join("outside.toml");
std::fs::write(&outside, "command = \"/bin/sh\"\n").unwrap();
let agents = dir.path().join("agents");
std::fs::create_dir(&agents).unwrap();
remove_custom(&agents, None, "../outside").unwrap_err();
assert!(outside.exists());
std::fs::write(
dir.path().join("amp-acp.toml"),
"command = \"/registry/amp\"\n[registry]\nid = \"amp-acp\"\nversion = \"1.0\"\n",
)
.unwrap();
save_custom(dir.path(), None, &custom_input("amp-acp")).unwrap_err();
remove_custom(dir.path(), None, "amp-acp").unwrap_err();
}
#[derive(Default)]
struct MemorySecrets(parking_lot::Mutex<BTreeMap<String, String>>);
impl EnvSecrets for MemorySecrets {
fn get(&self, key: &str) -> Option<String> {
self.0.lock().get(key).cloned()
}
fn set(&self, key: &str, value: &str) -> IdeResult<()> {
self.0.lock().insert(key.to_owned(), value.to_owned());
Ok(())
}
fn delete(&self, key: &str) {
self.0.lock().remove(key);
}
fn keys(&self, prefix: &str) -> Vec<String> {
self.0.lock().keys().filter(|k| k.starts_with(prefix)).cloned().collect()
}
}
#[test]
fn environment_values_go_to_the_secrets_and_the_files_keep_the_names() {
let dir = tempfile::tempdir().unwrap();
let secrets = MemorySecrets::default();
let s = Some(&secrets as &dyn EnvSecrets);
let env = BTreeMap::from([("API_KEY".to_owned(), "v-91f".to_owned()), ("A/B".to_owned(), String::new())]);
set_builtin_env(dir.path(), s, "claude", env.clone()).unwrap();
save_custom(dir.path(), s, &CustomAgentInput { env: env.clone(), ..custom_input("mine") }).unwrap();
for id in ["claude", "mine"] {
let text = std::fs::read_to_string(dir.path().join(format!("{id}.toml"))).unwrap();
assert!(text.contains("API_KEY") && !text.contains("v-91f"), "{text}");
let spec = load(Some(dir.path()), s).into_iter().find(|p| p.id == id).unwrap();
assert_eq!(describe(&spec).env, env);
}
assert_eq!(secrets.keys("agent/"), ["agent/claude/env/API_KEY", "agent/mine/env/API_KEY"]);
set_builtin_command(dir.path(), s, "claude", Some("/opt/claude")).unwrap();
assert_eq!(secrets.get("agent/claude/env/API_KEY").as_deref(), Some("v-91f"));
set_builtin_env(dir.path(), s, "claude", BTreeMap::from([("OTHER".to_owned(), "x".to_owned())])).unwrap();
assert_eq!(secrets.keys("agent/claude/"), ["agent/claude/env/OTHER"]);
set_builtin_command(dir.path(), s, "claude", None).unwrap();
set_builtin_env(dir.path(), s, "claude", BTreeMap::new()).unwrap();
remove_custom(dir.path(), s, "mine").unwrap();
assert_eq!(secrets.keys("agent/"), Vec::<String>::new());
}
#[test]
fn values_written_in_agent_files_move_to_the_secrets_unless_installed_from_the_registry() {
let dir = tempfile::tempdir().unwrap();
let secrets = MemorySecrets::default();
let s = Some(&secrets as &dyn EnvSecrets);
std::fs::write(dir.path().join("mine.toml"), "command = \"/opt/mine\"\n[env]\nK = \"v-91f\"\nS = \"\"\n")
.unwrap();
secrets.set("agent/mine/env/S", "saved").unwrap();
let registry =
"command = \"/registry/amp\"\n[env]\nR = \"1\"\n[registry]\nid = \"amp-acp\"\nversion = \"1.0\"\n";
std::fs::write(dir.path().join("amp-acp.toml"), registry).unwrap();
let specs = load(Some(dir.path()), s);
assert_eq!(
specs.iter().find(|p| p.id == "mine").unwrap().env,
[("K".to_owned(), "v-91f".to_owned()), ("S".to_owned(), "saved".to_owned())]
);
assert!(!std::fs::read_to_string(dir.path().join("mine.toml")).unwrap().contains("v-91f"));
assert_eq!(secrets.keys("agent/"), ["agent/mine/env/K", "agent/mine/env/S"]);
assert_eq!(std::fs::read_to_string(dir.path().join("amp-acp.toml")).unwrap(), registry);
}
#[test]
fn acp_json_imports_servers_and_skips_collisions() {
let dir = tempfile::tempdir().unwrap();
save_custom(dir.path(), None, &custom_input("taken")).unwrap();
let outcomes = import_acp(
dir.path(),
None,
r#"{"agent_servers": {
"My Agent!": {"command": "/opt/agent", "args": ["--acp"], "env": {"K": "v"}},
"taken": {"command": "/opt/taken"},
"claude": {"command": "/opt/claude"},
"!!!": {"command": "/opt/noname"},
"silent": {"args": []}
}}"#,
)
.unwrap();
let saved: Vec<&str> = outcomes.iter().filter(|o| o.saved).map(|o| o.id.as_str()).collect();
assert_eq!(saved, ["my-agent"]);
assert!(outcomes.iter().filter(|o| !o.saved).all(|o| !o.reason.is_empty()));
let mine = load(Some(dir.path()), None).into_iter().find(|s| s.id == "my-agent").unwrap();
assert_eq!((mine.name.as_str(), mine.command.as_str()), ("My Agent!", "/opt/agent"));
assert_eq!(mine.source, AgentSource::Custom);
import_acp(dir.path(), None, "not json").unwrap_err();
assert!(import_acp(dir.path(), None, "{}").unwrap().is_empty());
}
}
+4 -4
View File
@@ -826,7 +826,7 @@ mod tests {
assert!(!shown.iter().any(|a| a.id == "cline"));
write_atomic(&p.cache_file, INDEX.as_bytes()).unwrap();
let mut providers: Vec<AgentProvider> = provider::load(Some(&p.agents_dir), None)
let mut providers: Vec<AgentProvider> = provider::load(Some(&p.agents_dir))
.into_iter()
.filter(|s| s.id == "amp-acp")
.map(|s| provider::describe(&s))
@@ -918,7 +918,7 @@ mod tests {
assert_eq!(agent.command, v1.to_string_lossy());
assert_eq!((agent.args.as_slice(), agent.env["A"].as_str()), (&["acp".to_owned()][..], "1"));
assert_eq!(agent.registry, Some(RegistryRef { id: "demo".into(), version: "1.0.0".into() }));
let spec = provider::load(Some(&p.agents_dir), None).into_iter().find(|s| s.id == "demo").unwrap();
let spec = provider::load(Some(&p.agents_dir)).into_iter().find(|s| s.id == "demo").unwrap();
assert_eq!(spec.source, AgentSource::Registry { version: "1.0.0".into(), update: None });
assert!(provider::installed(&spec.command));
@@ -991,7 +991,7 @@ mod tests {
assert_eq!((shown[0].installed.as_deref(), shown[0].update), (Some("1.0.104"), true));
write_atomic(&p.cache_file, index.as_bytes()).unwrap();
let spec = provider::load(Some(&p.agents_dir), None).into_iter().find(|s| s.id == "copilot").unwrap();
let spec = provider::load(Some(&p.agents_dir)).into_iter().find(|s| s.id == "copilot").unwrap();
assert_eq!(spec.name, "GitHub Copilot");
assert_eq!(spec.login_hint, "Run `copilot` in a terminal and sign in.");
assert_eq!(spec.source, AgentSource::Registry { version: "1.0.104".into(), update: None });
@@ -1004,7 +1004,7 @@ mod tests {
);
remove(&p, "github-copilot-cli").unwrap();
let fallback = provider::load(Some(&p.agents_dir), None).into_iter().find(|s| s.id == "copilot").unwrap();
let fallback = provider::load(Some(&p.agents_dir)).into_iter().find(|s| s.id == "copilot").unwrap();
assert_eq!(fallback.source, AgentSource::Builtin);
}
}
-1
View File
@@ -24,7 +24,6 @@ fn paths(dir: &Path) -> AgentPaths {
models_file: dir.join("models.json"),
custom_agents_dir: None,
home_dir: None,
env_secrets: None,
}
}
-600
View File
@@ -1,600 +0,0 @@
//! Command execution security policy (Whitelist / Blacklist) for coding agents.
use ide_core::settings::AgentSecuritySettings;
use serde_json::Value;
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum PolicyDecision {
/// Command matched a blacklist pattern (`pattern`).
Blocked(String),
/// Command matched the whitelist and no blacklist pattern.
Allowed,
/// Neither matched; prompt the user as usual.
AskUser,
}
/// The shell command an ACP `session/request_permission` asks to run: only a tool call of kind
/// `execute`, and only its structured `rawInput` command. A title is free text the agent
/// writes, so it never names the command.
pub fn extract_command(params: &Value) -> Option<String> {
let tool = params.get("toolCall").or_else(|| params.pointer("/subject/toolCall"))?;
if tool.get("kind").and_then(Value::as_str) != Some("execute") {
return None;
}
let raw_cmd = tool.pointer("/rawInput/command").or_else(|| tool.pointer("/rawInput/cmd"));
raw_cmd.and_then(command_from_value)
}
/// Whether allowing the request grants more than running its command, such as network access
/// for it: the command policy may block such a request but never approves it.
pub fn widens_access(params: &Value) -> bool {
let tool = params.get("toolCall").or_else(|| params.pointer("/subject/toolCall"));
tool.and_then(|t| t.pointer("/rawInput/networkApprovalContext")).is_some_and(|c| !c.is_null())
}
fn command_from_value(val: &Value) -> Option<String> {
match val {
Value::String(s) => {
let trimmed = s.trim();
(!trimmed.is_empty()).then(|| trimmed.to_owned())
}
Value::Array(items) => {
let words: Vec<&str> = items.iter().filter_map(Value::as_str).collect();
if words.is_empty() {
return None;
}
let cmd = match words.as_slice() {
[_, flag, script] if flag.starts_with('-') && flag.ends_with('c') => (*script).trim().to_owned(),
_ => words.join(" ").trim().to_owned(),
};
(!cmd.is_empty()).then_some(cmd)
}
_ => None,
}
}
/// Checks whether a single pattern matches `command` using prefix + glob rules:
/// - If `pattern` contains `*` or `?`, matches using glob wildcards (where `*` matches
/// any sequence of characters). A trailing ` *` in the pattern also matches the bare
/// command without arguments (e.g. `git status *` matches `git status`).
/// - Otherwise, matches when `command` equals `pattern` or starts with `pattern` followed
/// by whitespace (prefix match at word boundary).
pub fn matches_command(pattern: &str, command: &str) -> bool {
let pat = pattern.trim();
let cmd = command.trim();
if pat.is_empty() || cmd.is_empty() {
return false;
}
if pat.contains('*') || pat.contains('?') {
if let Some(stem) = pat.strip_suffix(" *")
&& cmd == stem.trim()
{
return true;
}
wildcard_match(pat.as_bytes(), cmd.as_bytes())
} else {
cmd == pat || cmd.strip_prefix(pat).is_some_and(|rest| rest.starts_with(char::is_whitespace))
}
}
fn wildcard_match(pat: &[u8], text: &[u8]) -> bool {
let (mut p, mut t) = (0, 0);
let (mut star_p, mut star_t) = (None, 0);
while t < text.len() {
if p < pat.len() && (pat[p] == b'?' || pat[p] == text[t]) {
p += 1;
t += 1;
} else if p < pat.len() && pat[p] == b'*' {
star_p = Some(p);
p += 1;
star_t = t;
} else if let Some(sp) = star_p {
p = sp + 1;
star_t += 1;
t = star_t;
} else {
return false;
}
}
while p < pat.len() && pat[p] == b'*' {
p += 1;
}
p == pat.len()
}
/// Matches an allowed command entry against a sub-command.
/// An allow entry is a whole command or a prefix only up to a word boundary:
/// e.g. `git status` matches `git status` and `git status -s`, but NOT `git statusx`.
/// Any accidental trailing `*` in the allow entry is stripped so `git status*` behaves
/// cleanly as a prefix at word boundaries instead of a glob wildcard.
pub fn matches_allow_entry(entry: &str, command: &str) -> bool {
let mut entry = entry.trim();
if let Some(stripped) = entry.strip_suffix('*') {
entry = stripped.trim_end();
}
let cmd = command.trim();
if entry.is_empty() || cmd.is_empty() {
return false;
}
cmd == entry || cmd.strip_prefix(entry).is_some_and(|rest| rest.starts_with(char::is_whitespace))
}
/// Checks whether a word is an environment variable assignment (`IDENTIFIER=...`).
fn is_env_assignment_word(word: &str) -> bool {
let Some((name, _)) = word.split_once('=') else {
return false;
};
if name.is_empty() {
return false;
}
let mut chars = name.chars();
match chars.next() {
Some(c) if c.is_ascii_alphabetic() || c == '_' => {}
_ => return false,
}
chars.all(|c| c.is_ascii_alphanumeric() || c == '_')
}
/// Returns true if the sub-command starts with an environment variable assignment,
/// e.g. `FOO=bar cmd` or `PORT=3000`.
fn has_leading_env_assignment(cmd: &str) -> bool {
let trimmed = cmd.trim_start();
let bytes = trimmed.as_bytes();
let mut i = 0;
let mut quote: Option<u8> = None;
while i < bytes.len() {
let b = bytes[i];
if b == b'\\' && quote != Some(b'\'') && i + 1 < bytes.len() {
i += 2;
continue;
}
if let Some(q) = quote {
if b == q {
quote = None;
}
i += 1;
continue;
}
if b == b'\'' || b == b'"' {
quote = Some(b);
i += 1;
continue;
}
if b.is_ascii_whitespace() {
break;
}
i += 1;
}
let first_word = &trimmed[..i];
is_env_assignment_word(first_word)
}
/// Checks whether `command` contains shell constructs that must never be auto-approved:
/// - Command substitution: `$(...)` or backticks
/// - Redirection: `>`, `>>`, `<`
/// - Here-docs: `<<`
/// - Subshells: `(...)`
/// - Leading environment assignments: `VAR=val cmd`
pub fn has_unsafe_shell_syntax(command: &str) -> bool {
let cmd = command.trim();
if cmd.is_empty() {
return false;
}
if has_leading_env_assignment(cmd) {
return true;
}
let bytes = cmd.as_bytes();
let mut i = 0;
let mut quote: Option<u8> = None;
while i < bytes.len() {
let b = bytes[i];
if b == b'\\' && quote != Some(b'\'') && i + 1 < bytes.len() {
i += 2;
continue;
}
if let Some(q) = quote {
if b == q {
quote = None;
i += 1;
continue;
}
// Inside double quotes: $(...) and `...` are evaluated by the shell.
if q == b'"' {
if b == b'`' {
return true;
}
if b == b'$' && i + 1 < bytes.len() && bytes[i + 1] == b'(' {
return true;
}
}
i += 1;
continue;
}
match b {
b'\'' | b'"' => {
quote = Some(b);
i += 1;
}
b'`' => return true,
b'$' if i + 1 < bytes.len() && bytes[i + 1] == b'(' => return true,
b'<' | b'>' => return true,
b'(' | b')' => return true,
_ => i += 1,
}
}
false
}
/// Splits a shell command into sub-commands separated by `;`, `&&`, `||`, `|`, or `\n`
/// (outside single/double quotes) so compound commands cannot bypass policy checks.
fn split_subcommands(command: &str) -> Vec<&str> {
let bytes = command.as_bytes();
let mut out = Vec::new();
let mut start = 0;
let mut i = 0;
let mut quote: Option<u8> = None;
while i < bytes.len() {
let b = bytes[i];
if let Some(q) = quote {
if b == b'\\' && q == b'"' && i + 1 < bytes.len() {
i += 2;
continue;
}
if b == q {
quote = None;
}
i += 1;
continue;
}
match b {
b'\'' | b'"' => {
quote = Some(b);
i += 1;
}
b'\\' if i + 1 < bytes.len() => {
i += 2;
}
b';' | b'\n' => {
let seg = command[start..i].trim();
if !seg.is_empty() {
out.push(seg);
}
i += 1;
start = i;
}
b'&' | b'|' => {
let seg = command[start..i].trim();
if !seg.is_empty() {
out.push(seg);
}
if i + 1 < bytes.len() && bytes[i + 1] == b {
i += 2;
} else {
i += 1;
}
start = i;
}
_ => i += 1,
}
}
let tail = command[start..].trim();
if !tail.is_empty() {
out.push(tail);
}
out
}
/// Whether every shell an agent may run `command` in (a POSIX shell, PowerShell, cmd.exe) reads
/// it as the one command the policy sees. Outside double quotes no separator or redirection may
/// remain after splitting (cmd.exe takes single quotes literally); `\` may only stand in a
/// path (it escapes only in POSIX shells); no `^` may escape a quote (cmd.exe); no `%` (cmd.exe
/// expands it before parsing); no carriage return; no typographic quotes (PowerShell ends
/// strings at them).
fn reads_alike_in_every_shell(command: &str) -> bool {
let chars: Vec<char> = command.chars().collect();
let mut in_double = false;
for (i, &c) in chars.iter().enumerate() {
let next = chars.get(i + 1).copied();
match c {
'"' => in_double = !in_double,
'\\' if !next.is_none_or(|n| n.is_ascii_alphanumeric() || matches!(n, '\\' | '.' | '_' | '-')) => {
return false;
}
'^' if matches!(next, Some('"' | '\'' | '\n' | '\r')) => return false,
'%' | '\r' | '\u{2018}'..='\u{201f}' => return false,
';' | '&' | '|' | '<' | '>' | '\n' if !in_double => return false,
_ => {}
}
}
true
}
/// A command as blocked patterns are compared with it: lower case, single spaces, and the
/// program without its folder or `.exe` (`/bin/rm` and `RM.EXE` are `rm`).
fn normalized_for_blocking(command: &str) -> String {
let mut words = command.split_whitespace().map(str::to_lowercase);
let Some(program) = words.next() else { return String::new() };
let program = program.rsplit(['/', '\\']).next().unwrap_or(&program);
let program = program.strip_suffix(".exe").unwrap_or(program);
std::iter::once(program.to_owned()).chain(words).collect::<Vec<_>>().join(" ")
}
/// Evaluates `command` against `settings`:
/// 1. Blacklist takes precedence: if the full command or any sub-command matches a
/// `blocked_commands` entry, returns [`PolicyDecision::Blocked`].
/// 2. Whitelist: allowed_commands cannot auto-approve when empty.
/// 3. Every sub-command in a chain must be safe (no command substitution, redirection,
/// subshell, or env assignments) and match at least one `allowed_commands` entry
/// exactly or up to word boundary; otherwise falls back to [`PolicyDecision::AskUser`].
pub fn evaluate_command(command: &str, settings: &AgentSecuritySettings) -> PolicyDecision {
let cmd = command.trim();
if cmd.is_empty() {
return PolicyDecision::AskUser;
}
let subcommands = split_subcommands(cmd);
let normalized: Vec<String> =
std::iter::once(cmd).chain(subcommands.iter().copied()).map(normalized_for_blocking).collect();
for pat in &settings.blocked_commands {
let pat = pat.trim();
if pat.is_empty() {
continue;
}
let normalized_pat = normalized_for_blocking(pat);
if matches_command(pat, cmd)
|| subcommands.iter().any(|sub| matches_command(pat, sub))
|| normalized.iter().any(|c| matches_command(&normalized_pat, c))
{
return PolicyDecision::Blocked(pat.to_owned());
}
}
if settings.allowed_commands.is_empty() || subcommands.is_empty() {
return PolicyDecision::AskUser;
}
let all_allowed = subcommands.iter().all(|sub| {
let s = sub.trim();
if s.is_empty() || has_unsafe_shell_syntax(s) || !reads_alike_in_every_shell(s) {
return false;
}
settings.allowed_commands.iter().any(|entry| matches_allow_entry(entry, s))
});
if all_allowed {
return PolicyDecision::Allowed;
}
PolicyDecision::AskUser
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn prefix_and_glob_matching() {
assert!(matches_command("git status", "git status"));
assert!(matches_command("git status", "git status -s"));
assert!(!matches_command("git status", "git statusbar"));
assert!(matches_command("git status*", "git status"));
assert!(matches_command("git status*", "git status --short"));
assert!(matches_command("npm test *", "npm test"));
assert!(matches_command("npm test *", "npm test unit"));
assert!(matches_command("rm -rf *", "rm -rf /tmp/foo"));
assert!(matches_command("rm -rf *", "rm -rf /"));
}
#[test]
fn blacklist_takes_precedence_and_catches_chained_commands() {
let settings = AgentSecuritySettings::default();
assert_eq!(evaluate_command("rm -rf /tmp/build", &settings), PolicyDecision::Blocked("rm -rf *".into()));
assert_eq!(evaluate_command("git status && rm -rf /", &settings), PolicyDecision::Blocked("rm -rf *".into()));
assert_eq!(evaluate_command(":(){ :|:& };:", &settings), PolicyDecision::Blocked(":(){ :|:& };:".into()));
}
#[test]
fn allow_entry_matching_word_boundary() {
assert!(matches_allow_entry("git status", "git status"));
assert!(matches_allow_entry("git status", "git status -s"));
assert!(!matches_allow_entry("git status", "git statusbar"));
assert!(!matches_allow_entry("git status", "git statusx"));
// Trailing asterisk in allow entry normalizes cleanly to word boundary prefix
assert!(matches_allow_entry("git status*", "git status"));
assert!(matches_allow_entry("git status*", "git status --short"));
assert!(!matches_allow_entry("git status*", "git statusbar"));
assert!(matches_allow_entry("ls", "ls"));
assert!(matches_allow_entry("ls", "ls -la"));
assert!(!matches_allow_entry("ls", "lsof"));
assert!(matches_allow_entry("cargo check", "cargo check"));
assert!(matches_allow_entry("cargo check", "cargo check --workspace"));
assert!(!matches_allow_entry("cargo check", "cargo checkmate"));
}
#[test]
fn unsafe_shell_syntax_detection() {
// Command substitutions: $(...) and `...`
assert!(has_unsafe_shell_syntax("echo $(whoami)"));
assert!(has_unsafe_shell_syntax("echo \"user: $(whoami)\""));
assert!(has_unsafe_shell_syntax("echo `whoami`"));
assert!(has_unsafe_shell_syntax("echo \"`whoami`\""));
// Redirections: >, >>, <
assert!(has_unsafe_shell_syntax("git status > /tmp/out"));
assert!(has_unsafe_shell_syntax("git status >> /tmp/out"));
assert!(has_unsafe_shell_syntax("cat < /tmp/in"));
assert!(has_unsafe_shell_syntax("cat << 'EOF'"));
// Subshells: (...)
assert!(has_unsafe_shell_syntax("(cd /tmp && ls)"));
assert!(has_unsafe_shell_syntax("git status && (exit 1)"));
// Environment assignments: VAR=val ...
assert!(has_unsafe_shell_syntax("FOO=bar git status"));
assert!(has_unsafe_shell_syntax("PORT=3000 npm test"));
assert!(has_unsafe_shell_syntax("_SECRET=123 cargo test"));
assert!(has_unsafe_shell_syntax("A=\"b c\" git status"));
// Safe constructs with characters inside quotes or command flags
assert!(!has_unsafe_shell_syntax("git commit -m \"fix <bug> and (issue)\""));
assert!(!has_unsafe_shell_syntax("git commit -m 'fix <bug>'"));
assert!(!has_unsafe_shell_syntax("cargo test --test=unit"));
assert!(!has_unsafe_shell_syntax("git status -s"));
}
#[test]
fn default_settings_has_no_auto_approval_and_blocks_dangerous_defaults() {
let settings = AgentSecuritySettings::default();
assert!(settings.allowed_commands.is_empty(), "allowed_commands must start empty");
// Safe commands ask user by default
assert_eq!(evaluate_command("git status", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("ls", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("cargo check", &settings), PolicyDecision::AskUser);
// Blacklist patterns are still blocked by default
assert_eq!(evaluate_command("rm -rf /", &settings), PolicyDecision::Blocked("rm -rf *".into()));
assert_eq!(evaluate_command("rm -rf /tmp/test", &settings), PolicyDecision::Blocked("rm -rf *".into()));
}
#[test]
fn whitelist_allows_safe_single_and_chained_commands_only() {
let settings = AgentSecuritySettings {
allowed_commands: vec!["git status".into(), "git diff".into(), "cargo test".into()],
blocked_commands: vec!["rm -rf *".into()],
};
assert_eq!(evaluate_command("git status -s", &settings), PolicyDecision::Allowed);
assert_eq!(evaluate_command("git diff HEAD~1", &settings), PolicyDecision::Allowed);
assert_eq!(evaluate_command("git status && cargo test", &settings), PolicyDecision::Allowed);
// Word boundary: git statusx must not match git status
assert_eq!(evaluate_command("git statusx", &settings), PolicyDecision::AskUser);
// Unsafe syntax (redirection, command substitution, env assignment) falls back to AskUser
assert_eq!(evaluate_command("git status > out.txt", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("git status $(whoami)", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("FOO=bar git status", &settings), PolicyDecision::AskUser);
// Chaining an unapproved command must still ask the user.
assert_eq!(evaluate_command("git status && curl https://example.com", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("cargo publish", &settings), PolicyDecision::AskUser);
// Blacklist still takes precedence over whitelist
assert_eq!(evaluate_command("git status && rm -rf /", &settings), PolicyDecision::Blocked("rm -rf *".into()));
}
/// The command may run in PowerShell or cmd.exe as well as a POSIX shell: what any of them
/// would split or expand differently from the policy's reading is never auto-approved.
#[test]
fn what_another_shell_reads_as_two_commands_is_not_auto_approved() {
let settings = AgentSecuritySettings {
allowed_commands: vec!["git status".into(), "git commit".into()],
blocked_commands: vec![],
};
for cmd in [
// `\` escapes only in POSIX shells.
"git status \\; calc.exe",
"git status \\& calc.exe",
"git status \\| calc.exe",
"git status \\\\\\; calc.exe",
"git status \"\\\" ; calc.exe ; \\\"\"",
// cmd.exe: single quotes don't quote, `^` escapes a quote, `%` expands first.
"git status '&' calc.exe",
"git status ';' calc.exe",
"git status '>' out.txt",
"git status ^\" & calc.exe ^\"",
"git status %COMSPEC%",
// PowerShell also closes a string at a typographic quote.
"git status \"a \u{201d} ; calc.exe ; \u{201c} b\"",
"git status \u{2018}x\u{2019}",
// A carriage return ends the command line too.
"git status\r calc.exe",
] {
assert_eq!(evaluate_command(cmd, &settings), PolicyDecision::AskUser, "{cmd:?}");
}
// Text every shell reads alike stays approvable.
for cmd in [
"git status -s",
"git status && git status",
"git commit -m \"fix <bug> & (issue); done\"",
"git status src\\main.rs C:\\work\\repo",
"git status HEAD^",
] {
assert_eq!(evaluate_command(cmd, &settings), PolicyDecision::Allowed, "{cmd:?}");
}
}
#[test]
fn blocked_patterns_ignore_case_spacing_and_the_programs_folder() {
let settings = AgentSecuritySettings::default();
for cmd in
["rm -rf /", "RM -RF /", "/bin/rm -rf /", "Rm -Rf /tmp/x", "C:\\Windows\\System32\\shutdown.exe /s"]
{
assert!(matches!(evaluate_command(cmd, &settings), PolicyDecision::Blocked(_)), "{cmd:?}");
}
assert_eq!(evaluate_command("git status", &settings), PolicyDecision::AskUser);
}
#[test]
fn extracts_command_from_v1_v2_and_codex_payloads() {
let claude = json!({
"toolCall": {
"title": "git status",
"kind": "execute",
"rawInput": { "command": "git status --short" }
}
});
assert_eq!(extract_command(&claude).as_deref(), Some("git status --short"));
let codex = json!({
"toolCall": {
"title": "Run cargo test",
"kind": "execute",
"rawInput": { "command": ["/bin/zsh", "-lc", "cargo test --workspace"] }
}
});
assert_eq!(extract_command(&codex).as_deref(), Some("cargo test --workspace"));
let write = json!({
"toolCall": {
"title": "Write foo.txt",
"kind": "edit",
"rawInput": { "file_path": "foo.txt" }
}
});
assert_eq!(extract_command(&write), None);
}
/// Only a tool call of kind `execute` runs a command, and its command is in `rawInput`:
/// a title says what the agent wants it to say.
#[test]
fn a_request_is_a_command_only_by_its_kind_and_input() {
let escalation = json!({ "sessionId": "t", "toolCall": {
"toolCallId": "i1", "title": "Run git status", "kind": "other", "status": "pending",
"rawInput": { "fileSystem": { "write": ["C:\\"] }, "network": { "enabled": true } } },
"options": [ { "optionId": "turn", "name": "Allow for this turn", "kind": "allow_once" },
{ "optionId": "decline", "name": "Decline", "kind": "reject_once" } ] });
assert_eq!(extract_command(&escalation), None);
let titled_only = json!({ "toolCall": { "title": "Run git status", "kind": "execute" } });
assert_eq!(extract_command(&titled_only), None);
let named_only = json!({ "toolCall": { "title": "git status", "name": "Bash" } });
assert_eq!(extract_command(&named_only), None);
let mislabelled = json!({ "toolCall": { "title": "Run git status", "kind": "edit",
"rawInput": { "command": "git status" } } });
assert_eq!(extract_command(&mislabelled), None);
let network = json!({ "toolCall": { "title": "Allow network access to example.com", "kind": "execute",
"rawInput": { "command": "curl https://example.com", "networkApprovalContext": { "host": "example.com" } } } });
assert_eq!(extract_command(&network).as_deref(), Some("curl https://example.com"));
assert!(widens_access(&network));
let plain = json!({ "toolCall": { "title": "Run git status", "kind": "execute",
"rawInput": { "command": "git status" } } });
assert!(!widens_access(&plain));
}
}
+9 -799
View File
@@ -35,19 +35,11 @@ struct FakeAgent {
/// Without `loadSession` (a session lives only as long as its process).
no_load: bool,
auth_on_new: bool,
auth_on_load: bool,
authenticate_method: Option<String>,
authenticated: AtomicBool,
request_error: Option<(i64, String)>,
auth_methods: Vec<Value>,
/// Hold the late mode reply until the prompt's permission request is already waiting.
hold_late_mode: bool,
empty_config: AtomicBool,
/// Blocks `session/new` until discovery times out (see `acp::discover` test timeout).
hold_session_new: AtomicBool,
/// Sleeps this long at the start of `session/set_config_option`, after noting the time.
delay_config: Duration,
config_entered: Mutex<Option<std::time::Instant>>,
/// Config options `session/new` reports after the model one (per-model efforts, say).
extra_options: Mutex<Vec<Value>>,
model_options: Mutex<HashMap<String, Vec<Value>>>,
@@ -135,9 +127,9 @@ impl Handler for FakeAgent {
fn on_request(&self, method: &str, params: Value) -> Result<Value, (i64, String)> {
self.calls.lock().push(method.to_owned());
let needs_auth = !self.authenticated.load(Ordering::Acquire)
&& ((method == "session/new" && self.auth_on_new) || (method == "session/load" && self.auth_on_load));
if needs_auth || (method == "session/prompt" && params.pointer("/prompt/0/text") == Some(&json!("auth"))) {
if (method == "session/new" && self.auth_on_new)
|| (method == "session/prompt" && params.pointer("/prompt/0/text") == Some(&json!("auth")))
{
return Err(self
.request_error
.clone()
@@ -162,9 +154,6 @@ impl Handler for FakeAgent {
)
}
"session/new" => {
if self.hold_session_new.load(Ordering::Acquire) {
std::thread::sleep(Duration::from_secs(5));
}
if self.empty_config.load(Ordering::Acquire) {
return Ok(json!({"sessionId": "s1"}));
}
@@ -187,21 +176,7 @@ impl Handler for FakeAgent {
self.say(&session, "replayed history");
Ok(json!({}))
}
"authenticate" if self.authenticate_method.is_some() => {
let id = params.get("methodId").and_then(Value::as_str).unwrap_or_default();
self.calls.lock().push(format!("authenticate {id}"));
if Some(id) == self.authenticate_method.as_deref() {
self.authenticated.store(true, Ordering::Release);
Ok(json!({}))
} else {
Err((codes::INVALID_PARAMS, format!("unexpected methodId {id}")))
}
}
"session/set_config_option" | "session/set_mode" => {
if method == "session/set_config_option" && !self.delay_config.is_zero() {
*self.config_entered.lock() = Some(std::time::Instant::now());
std::thread::sleep(self.delay_config);
}
self.calls.lock().push(format!("{method} {}", params.get("value").or(params.get("modeId")).unwrap()));
if params["configId"] == "model" && !self.model_options.lock().is_empty() {
return Ok(json!({"configOptions": self.options_for(params["value"].as_str().unwrap())}));
@@ -264,23 +239,6 @@ impl Handler for FakeAgent {
self.say(&session, &format!("late chose {chosen}"));
return Ok(json!({"stopReason": "end_turn"}));
}
if text == "escalate" {
// A request for wider access, titled with text the model wrote.
let peer = self.latest();
let answer: Value = tokio::runtime::Handle::current()
.block_on(peer.request(
"session/request_permission",
json!({"sessionId": session, "toolCall": {"toolCallId": "e1", "title": "Run cargo test",
"kind": "other", "rawInput": {"network": {"enabled": true}}},
"options": [{"optionId": "yes", "name": "Allow for this turn", "kind": "allow_once"},
{"optionId": "no", "name": "Decline", "kind": "reject_once"}]}),
None,
))
.unwrap();
let chosen = answer.pointer("/outcome/optionId").and_then(Value::as_str).unwrap_or("none");
self.say(&session, &format!("escalation chose {chosen}"));
return Ok(json!({"stopReason": "end_turn"}));
}
if text == "ask" {
let peer = self.latest();
let answer: Value = tokio::runtime::Handle::current()
@@ -361,8 +319,7 @@ impl Handler for FakeAgent {
let answer: Value = tokio::runtime::Handle::current()
.block_on(peer.request(
"session/request_permission",
json!({"sessionId": session, "toolCall": {"toolCallId": "t1", "title": "Run cargo test",
"kind": "execute", "rawInput": {"command": "cargo test"}},
json!({"sessionId": session, "toolCall": {"toolCallId": "t1", "title": "Run cargo test"},
"options": [{"optionId": "yes", "name": "Allow", "kind": "allow_once"},
{"optionId": "no", "name": "Reject", "kind": "reject_once"}]}),
None,
@@ -384,8 +341,7 @@ fn manager(dir: &Path, agent: Arc<FakeAgent>) -> AgentManager {
fn manager_at(dir: &Path, models_file: &Path, agent: Arc<FakeAgent>) -> AgentManager {
// These tests speak to FakeAgent, never an installed CLI, including after a provider reload.
let agents = dir.join("agents");
provider::set_builtin_command(&agents, None, "claude", Some(&dir.join("missing-claude").to_string_lossy()))
.unwrap();
provider::set_builtin_command(&agents, "claude", Some(&dir.join("missing-claude").to_string_lossy())).unwrap();
let connector: Connector = Arc::new(move |_spec, _cwd, client, _stderr| {
let (a, b) = duplex(1 << 20);
let (ar, aw) = tokio::io::split(a);
@@ -404,42 +360,12 @@ fn manager_at(dir: &Path, models_file: &Path, agent: Arc<FakeAgent>) -> AgentMan
// The project is the boundary of the skill walk: nothing above it (such as the
// user's profile, when temp lives under it) leaks into the test.
home_dir: Some(dir.to_path_buf()),
env_secrets: None,
},
tokio::runtime::Handle::current(),
connector,
)
}
#[tokio::test(flavor = "multi_thread")]
async fn saved_conversation_reads_do_not_start_or_load_the_agent() {
let tmp = tempfile::tempdir().unwrap();
let agent = Arc::new(FakeAgent::default());
let mgr = manager(tmp.path(), agent.clone());
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("Review the saved conversation"),
provider: "claude".into(),
config: Vec::new(),
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::NeedsInput).await;
mgr.shutdown();
let restored_agent = Arc::new(FakeAgent::default());
let restored = manager(tmp.path(), restored_agent.clone());
assert!(!restored.entries(&task.id, None).unwrap().is_empty());
assert!(restored_agent.calls.lock().is_empty());
assert!(restored.entries("missing-task", None).is_err());
assert!(restored.entries(&task.id, Some("missing-subagent".into())).is_err());
restored.task(&task.id).unwrap().state.lock().conversation = Conversation::default();
assert!(restored.entries(&task.id, None).unwrap().is_empty());
restored.shutdown();
}
#[cfg(unix)]
#[tokio::test(flavor = "multi_thread")]
async fn claude_commands_before_session_are_fetched_once_and_saved() {
@@ -512,7 +438,7 @@ for line in sys.stdin:
}
let concurrent = async {
if outcome == "session_report" {
tokio::time::timeout(Duration::from_secs(10), async {
tokio::time::timeout(Duration::from_secs(2), async {
while !root.join("initialized").exists() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
@@ -584,7 +510,7 @@ for line in sys.stdin:
};
assert_eq!(subtypes, expected, "{outcome}");
let pid = std::fs::read_to_string(root.join("pid")).unwrap();
tokio::time::timeout(Duration::from_secs(10), async {
tokio::time::timeout(Duration::from_secs(2), async {
while std::process::Command::new("kill").args(["-0", pid.trim()]).output().unwrap().status.success() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
@@ -741,91 +667,6 @@ async fn registry_sign_in_errors_use_the_advertised_terminal_login() {
}
}
#[tokio::test(flavor = "multi_thread")]
async fn authenticates_and_retries_session_setup_once_on_auth_error() {
let tmp = tempfile::tempdir().unwrap();
let methods = vec![
json!({"id": "cli_login", "_meta": {"terminal-auth": {"command": "copilot", "args": ["login"]}}}),
json!({"id": "github_oauth", "name": "GitHub OAuth"}),
];
let agent = Arc::new(FakeAgent {
auth_on_new: true,
authenticate_method: Some("github_oauth".into()),
request_error: Some((-32000, "Authentication required".into())),
auth_methods: methods.clone(),
..Default::default()
});
let mgr = manager(tmp.path(), agent.clone());
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("again"),
provider: "copilot".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
assert_eq!(
agent
.calls
.lock()
.iter()
.filter(|c| c.starts_with("session/") || c.starts_with("authenticate"))
.collect::<Vec<_>>(),
["session/new", "authenticate", "authenticate github_oauth", "session/new", "session/prompt"],
);
// Resuming after an IDE restart also authenticates once and retries `session/load`.
mgr.shutdown();
let agent2 = Arc::new(FakeAgent {
auth_on_load: true,
auth_on_new: true,
authenticate_method: Some("github_oauth".into()),
request_error: Some((-32000, "Authentication required".into())),
auth_methods: methods.clone(),
..Default::default()
});
let mgr2 = manager(tmp.path(), agent2.clone());
mgr2.prompt(&task.id, text("again"), false).await.unwrap();
tokio::time::sleep(Duration::from_millis(50)).await;
wait_for(&mgr2, &task.id, AgentStatus::Ready).await;
assert_eq!(
agent2
.calls
.lock()
.iter()
.filter(|c| c.starts_with("session/") || c.starts_with("authenticate"))
.collect::<Vec<_>>(),
["session/load", "authenticate", "authenticate github_oauth", "session/load", "session/prompt"],
);
// If `authenticate` fails during `session/load`, fallback to `session/new` does not try it again.
mgr2.shutdown();
let agent3 = Arc::new(FakeAgent {
auth_on_load: true,
auth_on_new: true,
request_error: Some((-32000, "Authentication required".into())),
auth_methods: methods,
..Default::default()
});
let mgr3 = manager(tmp.path(), agent3.clone());
mgr3.prompt(&task.id, text("again"), false).await.unwrap();
wait_for(&mgr3, &task.id, AgentStatus::Failed).await;
assert_eq!(
agent3
.calls
.lock()
.iter()
.filter(|c| c.starts_with("session/") || c.starts_with("authenticate"))
.collect::<Vec<_>>(),
["session/load", "authenticate", "session/new"],
);
mgr3.shutdown();
}
#[tokio::test(flavor = "multi_thread")]
async fn commands_are_remembered_per_project_and_merged_with_skills_on_disk() {
let tmp = tempfile::tempdir().unwrap();
@@ -1430,7 +1271,6 @@ async fn real_claude_task_in_a_worktree() {
models_file: tmp.path().join("models.json"),
custom_agents_dir: None,
home_dir: None,
env_secrets: None,
},
tokio::runtime::Handle::current(),
);
@@ -2264,7 +2104,6 @@ async fn reload_providers_updates_registry_install_clears_stale_probes_and_notif
models_file: tmp.path().join("models.json"),
custom_agents_dir: Some(agents_dir.clone()),
home_dir: None,
env_secrets: None,
},
tokio::runtime::Handle::current(),
connector,
@@ -2307,49 +2146,6 @@ async fn reload_providers_updates_registry_install_clears_stale_probes_and_notif
assert!(probe.error.is_none());
}
/// A session report that changes the cached provider config notifies subscribers,
/// so the composer re-lists models without waiting for the next task (#1490).
/// Reports that change nothing stay silent.
#[tokio::test(flavor = "multi_thread")]
async fn remembered_config_changes_notify_subscribers_once() {
let tmp = tempfile::tempdir().unwrap();
let mgr = manager(tmp.path(), Arc::new(FakeAgent::default()));
let events = Arc::new(Mutex::new(Vec::new()));
let sink = events.clone();
mgr.subscribe(Arc::new(move |e| sink.lock().push(e)));
events.lock().clear();
let reported = config::options_of(&json!({ "configOptions": [
{ "id": "model", "category": "model", "type": "select", "currentValue": "default",
"options": [{ "value": "default" }, { "value": "custom-opus" }] },
] }));
mgr.shared.remember("claude", Some(&reported), None, Some("default"));
assert!(
events.lock().iter().any(|ev| matches!(ev, AgentEvent::Providers)),
"a first report changing the cache notifies subscribers"
);
events.lock().clear();
// The same report again: the cache is unchanged, so subscribers hear nothing.
mgr.shared.remember("claude", Some(&reported), None, Some("default"));
assert!(events.lock().is_empty(), "an unchanged report must stay silent");
// A changed model list notifies again, and providers() serves the new list.
let updated = config::options_of(&json!({ "configOptions": [
{ "id": "model", "category": "model", "type": "select", "currentValue": "default",
"options": [{ "value": "default" }, { "value": "custom-opus" }, { "value": "custom-sonnet" }] },
] }));
mgr.shared.remember("claude", Some(&updated), None, Some("default"));
assert!(
events.lock().iter().any(|ev| matches!(ev, AgentEvent::Providers)),
"a changed model list notifies subscribers"
);
let claude = mgr.providers().into_iter().find(|p| p.id == "claude").unwrap();
let model = claude.config.iter().find(|o| o.category.as_deref() == Some("model")).unwrap();
let values: Vec<_> = model.values.iter().map(|v| v.value.as_str()).collect();
assert_eq!(values, ["default", "custom-opus", "custom-sonnet"]);
}
#[tokio::test(flavor = "multi_thread")]
async fn idle_agents_stop_and_resume_their_session_on_the_next_message() {
let new = || NewAgentTask {
@@ -2575,7 +2371,6 @@ async fn delete_running_agent(no_load: bool) {
models_file: root.join("models.json"),
custom_agents_dir: None,
home_dir: None,
env_secrets: None,
},
tokio::runtime::Handle::current(),
connector,
@@ -2654,10 +2449,10 @@ async fn session_settings_win_over_discovery_in_memory_and_on_disk() {
let saved = options("saved", "low");
let discovered = options("discovered", "high");
if discovery_first {
mgr.shared.remember_discovered("claude", &discovered, &[], &[]);
mgr.shared.remember_discovered("claude", &discovered, &[]);
}
mgr.shared.remember("claude", Some(&saved), None, Some("saved"));
mgr.shared.remember_discovered("claude", &discovered, &[], &[]);
mgr.shared.remember_discovered("claude", &discovered, &[]);
let restored = manager(tmp.path(), Arc::new(FakeAgent::default()));
for mgr in [&mgr, &restored] {
assert_eq!(model_of(&mgr.shared.known_config("claude")).as_deref(), Some("saved"));
@@ -2665,588 +2460,3 @@ async fn session_settings_win_over_discovery_in_memory_and_on_disk() {
}
}
}
#[tokio::test(flavor = "multi_thread")]
async fn provider_reload_does_not_start_acp_discovery() {
let agent = Arc::new(FakeAgent::default());
let mgr = manager(tempfile::tempdir().unwrap().path(), agent.clone());
mgr.reload_providers();
tokio::time::sleep(Duration::from_millis(100)).await;
assert!(agent.calls.lock().is_empty(), "reload must not spawn cursor-agent");
}
#[tokio::test(flavor = "multi_thread")]
async fn composer_commands_discover_cursor_effort_and_fast() {
let agent = Arc::new(FakeAgent {
extra_options: Mutex::new(vec![
json!({"id": "effort", "type": "select", "currentValue": "medium",
"options": [{"value": "low"}, {"value": "medium"}, {"value": "high"}]}),
json!({"id": "fast", "type": "select", "currentValue": "false",
"options": [{"value": "false", "name": "Off"}, {"value": "true", "name": "On"}]}),
]),
..Default::default()
});
let mgr = manager(tempfile::tempdir().unwrap().path(), agent.clone());
mgr.commands("cursor", None).await.unwrap();
let calls = agent.calls.lock();
assert!(calls.iter().any(|m| m.as_str() == "session/new"));
assert_eq!(calls.iter().filter(|m| m.as_str() == "initialize").count(), 1);
let cursor = mgr.providers().into_iter().find(|p| p.id == "cursor").unwrap();
assert_eq!(cursor.config.iter().find(|o| o.id == "fast").unwrap().current.as_deref(), Some("false"));
assert!(
cursor
.thought_levels
.values()
.next()
.is_some_and(|levels| { levels.iter().any(|o| o.category.as_deref() == Some("thought_level")) })
);
}
#[tokio::test(flavor = "multi_thread")]
async fn config_choice_shows_before_the_agent_answers() {
let agent = Arc::new(FakeAgent {
extra_options: Mutex::new(vec![json!({
"id": "fast", "category": "model_config", "type": "select", "currentValue": "true",
"options": [{"value": "true", "name": "On"}, {"value": "false", "name": "Off"}]
})]),
delay_config: Duration::from_millis(300),
..Default::default()
});
let mgr = manager(tempfile::tempdir().unwrap().path(), agent.clone());
let shown: Arc<Mutex<Option<std::time::Instant>>> = Arc::default();
let mark = shown.clone();
mgr.subscribe(Arc::new(move |e| {
if let AgentEvent::Task { task } = e
&& task.config.iter().any(|o| o.id == "fast" && o.current.as_deref() == Some("false"))
&& mark.lock().is_none()
{
*mark.lock() = Some(std::time::Instant::now());
}
}));
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("again"),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
mgr.set_config(&task.id, "fast", "false".into()).await.unwrap();
let shown = (*shown.lock()).expect("the switch moved");
let entered = (*agent.config_entered.lock()).expect("the agent received the change");
assert!(shown < entered, "the composer updates before the agent answers");
}
#[tokio::test(flavor = "multi_thread")]
async fn discovery_times_out_when_session_new_hangs() {
let agent = Arc::new(FakeAgent { hold_session_new: AtomicBool::new(true), ..Default::default() });
let mgr = manager(tempfile::tempdir().unwrap().path(), agent.clone());
let started = std::time::Instant::now();
mgr.commands("cursor", None).await.unwrap();
assert!(started.elapsed() < Duration::from_secs(10), "discovery should time out quickly in tests");
let cursor = mgr.providers().into_iter().find(|p| p.id == "cursor").unwrap();
assert!(!cursor.config.iter().any(|o| o.category.as_deref() == Some("model")));
}
#[tokio::test(flavor = "multi_thread")]
async fn discovered_effort_levels_fill_models_no_session_has_reported() {
let tmp = tempfile::tempdir().unwrap();
let mgr = manager(tmp.path(), Arc::new(FakeAgent::default()));
let effort = |values: &[&str], current: &str| {
config::options_of(&json!({ "configOptions": [
{ "id": "effort", "category": "thought_level", "type": "select", "currentValue": current,
"options": values.iter().map(|v| json!({ "value": v })).collect::<Vec<_>>() }
] }))
};
let model = config::options_of(&json!({ "configOptions": [
{ "id": "model", "category": "model", "type": "select", "currentValue": "sonnet",
"options": [{ "value": "sonnet" }, { "value": "opus" }] }
] }));
// A session already reported sonnet's levels; discovery adds opus's.
let reported = [model.clone(), effort(&["low", "high"], "high")].concat();
mgr.shared.remember("claude", Some(&reported), None, Some("sonnet"));
let levels =
[("sonnet".to_owned(), effort(&["low"], "low")), ("opus".to_owned(), effort(&["low", "max"], "default"))];
mgr.shared.remember_discovered("claude", &model, &[], &levels);
let known = mgr.shared.known.lock();
assert_eq!(known["claude"].thought_levels["sonnet"], effort(&["low", "high"], "high"), "the session's report wins");
assert_eq!(known["claude"].thought_levels["opus"], effort(&["low", "max"], "default"));
}
#[test]
fn streamed_text_is_sent_as_appends() {
let mut streamed = HashMap::new();
let agent = |t: &str| AgentEntry::Agent { text: t.into() };
let first = entry_event("t", &mut streamed, None, 3, &agent("Hé"));
assert!(matches!(first, AgentEvent::Entry { index: 3, .. }), "the first sight of a message is whole");
let next = entry_event("t", &mut streamed, None, 3, &agent("Hé 👋 there"));
assert!(
matches!(&next, AgentEvent::Append { subagent: None, index: 3, from: 2, text, .. } if text == " 👋 there"),
"{next:?}"
);
let sub = entry_event("t", &mut streamed, Some("s".into()), 3, &agent("Hé 👋 there!"));
assert!(matches!(sub, AgentEvent::Entry { .. }), "each session streams on its own");
let after = entry_event("t", &mut streamed, None, 3, &agent("Hé 👋 there."));
assert!(
matches!(&after, AgentEvent::Append { from: 11, text, .. } if text == "."),
"from counts UTF-16 units: {after:?}"
);
let replaced = entry_event("t", &mut streamed, None, 3, &agent("Other"));
assert!(matches!(replaced, AgentEvent::Entry { .. }), "text that doesn't extend the sent text is sent whole");
let thought = entry_event("t", &mut streamed, None, 4, &AgentEntry::Thought { text: "hmm…".into() });
assert!(matches!(thought, AgentEvent::Entry { index: 4, .. }), "a new message starts whole");
let tool = AgentEntry::Error { message: "x".into() };
entry_event("t", &mut streamed, None, 4, &tool);
let again = entry_event("t", &mut streamed, None, 4, &AgentEntry::Thought { text: "hmm… ok".into() });
assert!(matches!(again, AgentEvent::Entry { .. }), "an entry replaced by another kind forgets its text");
let long = "é".repeat(5000);
entry_event("t", &mut streamed, None, 5, &agent(&long));
assert!(streamed[&None].tail.len() <= 64, "only the end of what was sent is kept");
let on = entry_event("t", &mut streamed, None, 5, &agent(&format!("{long}!")));
assert!(matches!(&on, AgentEvent::Append { from: 5000, text, .. } if text == "!"), "{on:?}");
let rewritten = entry_event("t", &mut streamed, None, 5, &agent(&format!("{}x!??", "é".repeat(4999))));
assert!(matches!(rewritten, AgentEvent::Entry { .. }), "a rewritten end is sent whole");
}
#[test]
fn error_messages_keep_the_agents_details() {
let details = json!({ "details": "spawn claude ENOENT" });
assert_eq!(with_details("Internal error".into(), Some(&details)), "Internal error: spawn claude ENOENT");
assert_eq!(
with_details("Internal error".into(), Some(&json!("model not found"))),
"Internal error: model not found"
);
assert_eq!(with_details(String::new(), Some(&details)), "spawn claude ENOENT");
assert_eq!(with_details("spawn claude ENOENT".into(), Some(&details)), "spawn claude ENOENT");
assert_eq!(with_details("Internal error".into(), Some(&json!({ "other": 1 }))), "Internal error");
assert_eq!(with_details("Internal error".into(), None), "Internal error");
}
#[tokio::test(flavor = "multi_thread")]
async fn rolling_back_restores_the_start_of_the_task_and_keeps_the_users_own_edits() {
let tmp = tempfile::tempdir().unwrap();
let dir = dunce::canonicalize(tmp.path()).unwrap();
let git = |args: &[&str]| {
let ok = std::process::Command::new("git")
.args(["-c", "user.name=t", "-c", "user.email=t@t", "-c", "commit.gpgsign=false"])
.args(args)
.current_dir(&dir)
.status()
.unwrap()
.success();
assert!(ok, "{args:?}");
};
let write = |rel: &str, text: &str| std::fs::write(dir.join(rel), text).unwrap();
let read = |rel: &str| std::fs::read_to_string(dir.join(rel)).unwrap();
git(&["init", "-q", "-b", "main"]);
write(".gitignore", "store/\nworktrees/\nagents/\nmodels.json\n");
write("mine.txt", "base\n");
write("other.txt", "base\n");
git(&["add", "."]);
git(&["commit", "-q", "-m", "init"]);
// The user's own uncommitted edit, before the task starts.
write("mine.txt", "my edit\n");
let mgr = manager(&dir, Arc::new(FakeAgent::default()));
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("Hello"),
provider: "claude".into(),
config: Vec::new(),
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::NeedsInput).await;
mgr.cancel(&task.id, None).unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
// What the agent did.
write("mine.txt", "my edit\nand the agent's\n");
write("other.txt", "agent\n");
write("new.txt", "agent\n");
async fn changed(mgr: &AgentManager, id: &str) -> Vec<String> {
mgr.changes(id).await.unwrap().into_iter().map(|c| c.rel_path).collect()
}
assert_eq!(changed(&mgr, &task.id).await, ["mine.txt", "new.txt", "other.txt"]);
mgr.rollback_changes(&task.id, Some(vec!["other.txt".into()]), AgentChangesView::SinceStart).await.unwrap();
assert_eq!(read("other.txt"), "base\n");
assert_eq!(changed(&mgr, &task.id).await, ["mine.txt", "new.txt"]);
// After a restart, from the saved copy.
mgr.shutdown();
let mgr = manager(&dir, Arc::new(FakeAgent::default()));
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert_eq!(read("mine.txt"), "my edit\n", "the user's edit stays");
assert!(!dir.join("new.txt").exists());
assert!(changed(&mgr, &task.id).await.is_empty());
mgr.delete(&task.id, false).await.unwrap();
assert!(!dir.join("store").join(format!("{}.before", task.id)).exists());
}
/// A git repository in a temporary folder, for local tasks whose checkout moves under them.
struct Checkout {
_tmp: tempfile::TempDir,
dir: PathBuf,
eol: Eol,
}
/// How the checkout keeps line endings: as committed, or CRLF on disk with `core.autocrlf`
/// (Git for Windows' default), where the files' bytes never equal the committed blobs.
#[derive(Debug, Clone, Copy)]
enum Eol {
Lf,
Crlf,
}
const EOLS: [Eol; 2] = [Eol::Lf, Eol::Crlf];
impl Checkout {
fn new(eol: Eol) -> Self {
let tmp = tempfile::tempdir().unwrap();
let dir = dunce::canonicalize(tmp.path()).unwrap();
let checkout = Self { _tmp: tmp, dir, eol };
checkout.git(&["init", "-q", "-b", "main"]);
let autocrlf = match eol {
Eol::Lf => "false",
Eol::Crlf => "true",
};
checkout.git(&["config", "core.autocrlf", autocrlf]);
checkout.write(".gitignore", "store/\nworktrees/\nagents/\nmodels.json\n");
checkout
}
fn git(&self, args: &[&str]) {
let ok = std::process::Command::new("git")
.args(["-c", "user.name=t", "-c", "user.email=t@t", "-c", "commit.gpgsign=false"])
.args(args)
.current_dir(&self.dir)
.status()
.unwrap()
.success();
assert!(ok, "{args:?}");
}
fn commit(&self, message: &str) {
self.git(&["add", "."]);
self.git(&["commit", "-q", "-m", message]);
}
fn write(&self, rel: &str, text: &str) {
let text = match self.eol {
Eol::Lf => text.to_owned(),
Eol::Crlf => text.replace('\n', "\r\n"),
};
std::fs::write(self.dir.join(rel), text).unwrap();
}
/// The file's text with LF line endings (a rollback writes the committed bytes back).
fn read(&self, rel: &str) -> String {
std::fs::read_to_string(self.dir.join(rel)).unwrap().replace("\r\n", "\n")
}
/// A local task, stopped so it can be rolled back.
async fn task(&self) -> (AgentManager, AgentTask) {
let mgr = manager(&self.dir, Arc::new(FakeAgent::default()));
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("Hello"),
provider: "claude".into(),
config: Vec::new(),
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::NeedsInput).await;
mgr.cancel(&task.id, None).unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
(mgr, task)
}
}
async fn changed(mgr: &AgentManager, id: &str) -> Vec<String> {
mgr.changes(id).await.unwrap().into_iter().map(|c| c.rel_path).collect()
}
#[tokio::test(flavor = "multi_thread")]
async fn work_uncommitted_by_a_reset_during_a_local_task_is_not_rolled_back() {
for eol in EOLS {
let repo = Checkout::new(eol);
repo.write("a.txt", "base\n");
repo.write("mine.txt", "base\n");
repo.commit("init");
repo.write("mine.txt", "my work\n");
repo.commit("my work");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
// The user takes the last commit back into the working tree (`git reset --soft`, an
// undone commit): its changes are the user's, not the agent's.
repo.git(&["reset", "-q", "--soft", "HEAD~1"]);
assert_eq!(changed(&mgr, &task.id).await, ["a.txt"], "{eol:?}");
let base = mgr.base_text(&task.id, repo.dir.join("a.txt")).await.unwrap();
assert_eq!(base.map(|b| b.replace("\r\n", "\n")).as_deref(), Some("base\n"), "{eol:?}");
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert_eq!(repo.read("a.txt"), "base\n", "{eol:?}");
assert_eq!(repo.read("mine.txt"), "my work\n", "{eol:?}: the user's work survives the rollback");
assert!(changed(&mgr, &task.id).await.is_empty(), "{eol:?}");
}
}
#[tokio::test(flavor = "multi_thread")]
async fn files_a_mixed_reset_during_a_local_task_leaves_behind_are_not_rolled_back() {
for eol in EOLS {
let repo = Checkout::new(eol);
repo.write("a.txt", "base\n");
repo.write("mine.txt", "base\n");
repo.commit("init");
repo.write("mine.txt", "my work\n");
repo.write("mine-new.txt", "my new file\n");
repo.commit("my work");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
// `git reset HEAD~1`: the commit's new file is untracked now, its edit unstaged.
repo.git(&["reset", "-q", "HEAD~1"]);
assert_eq!(changed(&mgr, &task.id).await, ["a.txt"], "{eol:?}");
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert_eq!(repo.read("a.txt"), "base\n", "{eol:?}");
assert_eq!(repo.read("mine.txt"), "my work\n", "{eol:?}");
assert_eq!(repo.read("mine-new.txt"), "my new file\n", "{eol:?}");
}
}
#[tokio::test(flavor = "multi_thread")]
async fn a_branch_switched_to_during_a_local_task_is_left_alone() {
for eol in EOLS {
let repo = Checkout::new(eol);
repo.write("a.txt", "base\n");
repo.write("other.txt", "base\n");
repo.commit("init");
repo.git(&["checkout", "-q", "-b", "feature"]);
repo.write("other.txt", "feature\n");
repo.write("only-feature.txt", "feature\n");
repo.commit("feature");
repo.git(&["checkout", "-q", "main"]);
repo.write("only-main.txt", "main\n");
repo.commit("main");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
// The agent's uncommitted edit comes along to a branch that doesn't contain the task's start.
repo.git(&["checkout", "-q", "feature"]);
assert_eq!(changed(&mgr, &task.id).await, ["a.txt"], "{eol:?}");
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert_eq!(repo.read("a.txt"), "base\n", "{eol:?}");
assert_eq!(repo.read("other.txt"), "feature\n", "{eol:?}");
assert_eq!(repo.read("only-feature.txt"), "feature\n", "{eol:?}");
assert!(!repo.dir.join("only-main.txt").exists(), "{eol:?}");
assert!(changed(&mgr, &task.id).await.is_empty(), "{eol:?}");
}
}
#[tokio::test(flavor = "multi_thread")]
async fn commits_made_during_a_local_task_are_kept_by_its_rollback() {
for eol in EOLS {
let repo = Checkout::new(eol);
repo.write("a.txt", "base\n");
repo.commit("init");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
repo.write("b.txt", "agent\n");
// The user commits what the agent did to `a.txt` so far: it drops out of the task's changes.
repo.git(&["add", "a.txt"]);
repo.git(&["commit", "-q", "-m", "keep"]);
assert_eq!(changed(&mgr, &task.id).await, ["b.txt"], "{eol:?}");
repo.write("a.txt", "agent\nagain\n");
assert_eq!(changed(&mgr, &task.id).await, ["a.txt", "b.txt"], "{eol:?}");
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert_eq!(repo.read("a.txt"), "agent\n", "{eol:?}: back to the commit, not to before it");
assert!(!repo.dir.join("b.txt").exists(), "{eol:?}");
assert!(changed(&mgr, &task.id).await.is_empty(), "{eol:?}");
}
}
#[tokio::test(flavor = "multi_thread")]
async fn rolling_back_a_rename_brings_the_old_file_back() {
for eol in EOLS {
let repo = Checkout::new(eol);
repo.write("Old.java", "class Old {}\n");
repo.write("Case.java", "class Case {}\n");
repo.commit("init");
let (mgr, task) = repo.task().await;
repo.git(&["mv", "Old.java", "New.java"]);
repo.git(&["mv", "Case.java", "CASE.java"]);
// Renames list their new path only.
let listed = changed(&mgr, &task.id).await;
assert_eq!(listed, ["CASE.java", "New.java"], "{eol:?}");
mgr.rollback_changes(&task.id, Some(listed), AgentChangesView::SinceStart).await.unwrap();
assert_eq!(repo.read("Old.java"), "class Old {}\n", "{eol:?}");
assert!(!repo.dir.join("New.java").exists(), "{eol:?}");
assert_eq!(repo.read("Case.java"), "class Case {}\n", "{eol:?}");
let names: Vec<String> = std::fs::read_dir(&repo.dir)
.unwrap()
.map(|e| e.unwrap().file_name().to_string_lossy().into_owned())
.filter(|n| n.ends_with(".java"))
.collect();
assert!(!names.contains(&"CASE.java".to_owned()), "{eol:?}: {names:?}");
}
}
#[tokio::test(flavor = "multi_thread")]
async fn the_group_view_rolls_back_only_uncommitted_files() {
for eol in EOLS {
let repo = Checkout::new(eol);
repo.write("a.txt", "base\n");
repo.commit("init");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
repo.commit("agent work");
repo.write("a.txt", "agent\nagain\n");
let grouped = mgr.grouped_changes(&task.id).await.unwrap();
assert_eq!(grouped.uncommitted.iter().map(|c| c.rel_path.as_str()).collect::<Vec<_>>(), ["a.txt"], "{eol:?}");
assert!(grouped.commits.is_empty(), "{eol:?}: local tasks have no commits to group");
mgr.rollback_changes(&task.id, None, AgentChangesView::Group).await.unwrap();
assert_eq!(repo.read("a.txt"), "agent\n", "{eol:?}: back to the commit, not to the start");
assert!(mgr.grouped_changes(&task.id).await.unwrap().uncommitted.is_empty(), "{eol:?}");
}
}
#[tokio::test(flavor = "multi_thread")]
async fn a_local_task_in_a_repository_without_commits_lists_and_rolls_back_its_changes() {
for eol in EOLS {
let repo = Checkout::new(eol);
repo.write("mine.txt", "mine\n");
let (mgr, task) = repo.task().await;
repo.write("new.txt", "agent\n");
assert_eq!(changed(&mgr, &task.id).await, ["new.txt"], "{eol:?}");
assert_eq!(mgr.base_text(&task.id, repo.dir.join("new.txt")).await.unwrap(), None, "{eol:?}");
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert!(!repo.dir.join("new.txt").exists(), "{eol:?}");
assert_eq!(repo.read("mine.txt"), "mine\n", "{eol:?}");
assert!(changed(&mgr, &task.id).await.is_empty(), "{eol:?}");
}
}
#[tokio::test(flavor = "multi_thread")]
async fn security_policy_auto_approves_whitelist_and_rejects_blacklist() {
use ide_core::settings::AgentSecuritySettings;
// 1. Whitelist auto-approves "cargo test" without waiting for manual respond().
let tmp = tempfile::tempdir().unwrap();
let agent = Arc::new(FakeAgent::default());
let mgr = manager(tmp.path(), agent);
mgr.set_security_hook(Arc::new(|| AgentSecuritySettings {
allowed_commands: vec!["cargo test".into()],
blocked_commands: vec!["rm -rf *".into()],
}));
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("run tests"),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
let entries = mgr.entries(&task.id, None).unwrap();
assert!(
entries.iter().any(|e| matches!(e, AgentEntry::Agent { text } if text.contains("chose yes"))),
"{entries:?}"
);
assert!(
entries.iter().any(|e| matches!(
e,
AgentEntry::Permission { chosen: Some(c), .. } if c.contains("whitelisted")
)),
"{entries:?}"
);
// 2. Blacklist auto-rejects "cargo test" when matched by blocked_commands.
mgr.set_security_hook(Arc::new(|| AgentSecuritySettings {
allowed_commands: vec!["cargo test".into()],
blocked_commands: vec!["cargo test*".into()],
}));
let blocked_task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("run blocked"),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &blocked_task.id, AgentStatus::Ready).await;
let blocked_entries = mgr.entries(&blocked_task.id, None).unwrap();
assert!(
blocked_entries.iter().any(|e| matches!(e, AgentEntry::Agent { text } if text.contains("chose no"))),
"{blocked_entries:?}"
);
assert!(
blocked_entries.iter().any(|e| matches!(
e,
AgentEntry::Permission { chosen: Some(c), .. } if c.contains("Blocked by policy")
)),
"{blocked_entries:?}"
);
}
#[tokio::test(flavor = "multi_thread")]
async fn security_policy_never_auto_approves_a_request_that_only_reads_like_a_command() {
use ide_core::settings::AgentSecuritySettings;
let tmp = tempfile::tempdir().unwrap();
let agent = Arc::new(FakeAgent::default());
let mgr = manager(tmp.path(), agent);
mgr.set_security_hook(Arc::new(|| AgentSecuritySettings {
allowed_commands: vec!["cargo test".into()],
blocked_commands: vec![],
}));
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("escalate"),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::NeedsInput).await;
let entries = mgr.entries(&task.id, None).unwrap();
let pending = entries
.iter()
.find_map(|e| match e {
AgentEntry::Permission { id, chosen: None, .. } => Some(*id),
_ => None,
})
.unwrap_or_else(|| panic!("the request was answered without the user: {entries:?}"));
mgr.respond(&task.id, pending, Some("no".into())).unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
let entries = mgr.entries(&task.id, None).unwrap();
assert!(
entries.iter().any(|e| matches!(e, AgentEntry::Agent { text } if text == "escalation chose no")),
"{entries:?}"
);
}
-1
View File
@@ -438,7 +438,6 @@ mod tests {
t.push(AgentEntry::Permission {
id: 1,
title: "Run?".into(),
command: None,
options: vec![],
questions: vec![],
chosen: None,
+14 -823
View File
@@ -1,8 +1,8 @@
//! Task worktrees and the changes made in them.
use crate::model::{AgentChange, AgentChanges, AgentCommit};
use crate::model::AgentChange;
use ide_core::{IdeError, IdeResult, IpcPath};
use ide_vcs::{ChangeKind, LogQuery, Repo};
use ide_vcs::{ChangeKind, Repo};
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use std::time::{Duration, UNIX_EPOCH};
@@ -14,18 +14,8 @@ pub(crate) struct Created {
pub base_commit: String,
}
/// The repository of `path`, free to execute: the host runs every agent command that
/// reaches a repository only in a trusted project.
fn discover(path: &Path) -> IdeResult<Option<Repo>> {
let repo = Repo::discover(path)?;
if let Some(repo) = &repo {
repo.set_execution_allowed(true);
}
Ok(repo)
}
fn repo_at(path: &Path) -> IdeResult<Repo> {
discover(path)?.ok_or_else(|| IdeError::unavailable(format!("{} is not in a git repository", path.display())))
Repo::discover(path)?.ok_or_else(|| IdeError::unavailable(format!("{} is not in a git repository", path.display())))
}
/// The current branch (or commit when detached) and its commit.
@@ -35,14 +25,6 @@ pub(crate) fn head_of(path: &Path) -> IdeResult<(String, String)> {
Ok((info.head.unwrap_or_else(|| commit.clone()), commit))
}
/// `HEAD`'s commit in `cwd`, `None` outside a repository or before its first commit.
pub(crate) fn head_commit(cwd: &Path) -> IdeResult<Option<String>> {
match discover(cwd)? {
Some(repo) => Ok(repo.head()?.1),
None => Ok(None),
}
}
/// `git worktree add -b agent/<slug> <dir>/<slug> <base>`.
pub(crate) fn create(project_root: &Path, worktrees_dir: &Path, title: &str, base: Option<&str>) -> IdeResult<Created> {
let repo = repo_at(project_root)?;
@@ -109,412 +91,32 @@ fn retrying(attempts: u32, pause: Duration, mut op: impl FnMut() -> std::io::Res
/// Files under `cwd` that differ from `base` (untracked files included), with line counts.
/// Outside a repository there is nothing to compare against: no changes.
pub(crate) fn changes(cwd: &Path, base: &str) -> IdeResult<Vec<AgentChange>> {
let Some(repo) = discover(cwd)? else { return Ok(Vec::new()) };
let Some(repo) = Repo::discover(cwd)? else { return Ok(Vec::new()) };
let root = repo.root().to_path_buf();
let stats: std::collections::HashMap<String, (u32, u32)> =
repo.numstat_with_worktree(base)?.into_iter().map(|(p, a, r)| (p, (a, r))).collect();
let stat = |rel: &str| stats.get(rel).copied().unwrap_or((0, 0));
let stats = repo.numstat_with_worktree(base)?;
let stat = |rel: &str| stats.iter().find(|(p, ..)| p == rel).map_or((0, 0), |&(_, a, r)| (a, r));
let mut out: Vec<AgentChange> = repo
.diff_with_worktree(base)?
.into_iter()
.map(|f| {
let (added, removed) = stat(&f.path);
AgentChange {
path: IpcPath(root.join(&f.path)),
rel_path: f.path,
kind: f.kind,
added,
removed,
old_path: f.old_path,
}
AgentChange { path: IpcPath(root.join(&f.path)), rel_path: f.path, kind: f.kind, added, removed }
})
.collect();
let tracked: std::collections::HashSet<String> = out.iter().map(|c| c.rel_path.clone()).collect();
for s in repo.status()? {
if s.unstaged == Some(ChangeKind::Untracked) && !tracked.contains(&s.rel_path) {
out.push(added(s.path, s.rel_path));
if s.unstaged == Some(ChangeKind::Untracked) && !out.iter().any(|c| c.rel_path == s.rel_path) {
let added = std::fs::read_to_string(&*s.path).map_or(0, |t| t.lines().count() as u32);
out.push(AgentChange { path: s.path, rel_path: s.rel_path, kind: ChangeKind::Added, added, removed: 0 });
}
}
out.sort_by(|a, b| a.rel_path.cmp(&b.rel_path));
Ok(out)
}
/// Files renamed since `rev`, by their new repository-relative path: the old path, as a deletion
/// to roll back along with the rename.
pub(crate) fn rename_sources(cwd: &Path, rev: &str) -> IdeResult<BTreeMap<String, AgentChange>> {
let Some(repo) = discover(cwd)? else { return Ok(BTreeMap::new()) };
let root = repo.root().to_path_buf();
Ok(repo
.diff_with_worktree(rev)?
.into_iter()
.filter_map(|f| {
let old = f.old_path.filter(|old| *old != f.path)?;
let source = AgentChange {
path: IpcPath(root.join(&old)),
rel_path: old,
kind: ChangeKind::Deleted,
added: 0,
removed: 0,
old_path: None,
};
Some((f.path, source))
})
.collect())
}
fn added(path: IpcPath, rel_path: String) -> AgentChange {
let added = std::fs::read_to_string(&*path).map_or(0, |t| t.lines().count() as u32);
AgentChange { path, rel_path, kind: ChangeKind::Added, added, removed: 0, old_path: None }
}
/// The commits from `base` (exclusive) to `head` (inclusive) in `cwd`, newest first, each
/// with the files it changed against its first parent and their line counts.
pub(crate) fn commits(cwd: &Path, base: &str, head: &str) -> IdeResult<Vec<AgentCommit>> {
if base == head {
return Ok(Vec::new());
}
let Some(repo) = discover(cwd)? else { return Ok(Vec::new()) };
let root = repo.root().to_path_buf();
let query = LogQuery { range: Some(format!("{base}..{head}")), ..LogQuery::default() };
let mut out = Vec::new();
for commit in repo.log(&query)? {
let parent = commit.parents.first().cloned();
// A root commit has no parent to count lines against; its files still show.
let stats: std::collections::HashMap<String, (u32, u32)> = match &parent {
Some(parent) => repo
.diff_numstat(parent, &commit.id, &[])?
.into_iter()
.map(|s| (s.path, (s.added, s.removed)))
.collect(),
None => std::collections::HashMap::new(),
};
let mut files: Vec<AgentChange> = repo
.commit_files(&commit.id)?
.into_iter()
.map(|f| {
let (added, removed) = stats.get(&f.path).copied().unwrap_or((0, 0));
AgentChange {
path: IpcPath(root.join(&f.path)),
rel_path: f.path,
kind: f.kind,
added,
removed,
old_path: f.old_path,
}
})
.collect();
files.sort_by(|a, b| a.rel_path.cmp(&b.rel_path));
out.push(AgentCommit {
id: commit.id,
short_id: commit.short_id,
summary: commit.summary,
message: commit.message,
time_ms: commit.time_ms,
parent,
files,
});
}
Ok(out)
}
/// Whether two paths name one file (a case-insensitive disk resolving both spellings).
fn same_file(a: &Path, b: &Path) -> bool {
match (std::fs::canonicalize(a), std::fs::canonicalize(b)) {
(Ok(a), Ok(b)) => a == b,
_ => false,
}
}
fn crlf_to_lf(bytes: &[u8]) -> Vec<u8> {
let mut out = Vec::with_capacity(bytes.len());
for (i, &b) in bytes.iter().enumerate() {
if !(b == b'\r' && bytes.get(i + 1) == Some(&b'\n')) {
out.push(b);
}
}
out
}
/// Where a task works and what its changes are measured against.
pub(crate) enum TaskBase {
/// A worktree of its own: everything since the commit it was created from, the agent's
/// own commits included.
Worktree { cwd: PathBuf, base: String },
/// The project's checkout, where the user works and commits too. `start` is the commit it
/// started from (`None` in a repository without commits, or an imported session).
Local { cwd: PathBuf, start: Option<String> },
}
impl TaskBase {
pub(crate) fn cwd(&self) -> &Path {
match self {
Self::Worktree { cwd, .. } | Self::Local { cwd, .. } => cwd,
}
}
/// The files the task changed, less those changed before it started and untouched since.
pub(crate) fn changes(&self, before: &BTreeMap<String, Stamp>) -> IdeResult<Vec<AgentChange>> {
let (cwd, start) = match self {
Self::Worktree { cwd, base } => return Ok(without_untouched(changes(cwd, base)?, before)),
Self::Local { cwd, start } => (cwd, start.as_deref()),
};
let Some(repo) = discover(cwd)? else { return Ok(Vec::new()) };
// What is uncommitted, as the Commit tool window lists it: commits made since the task
// started drop out.
let head = repo.branch_info()?.head_commit;
let listed = match &head {
Some(head) => changes(cwd, head)?,
None => {
let mut all: Vec<AgentChange> = repo
.status()?
.into_iter()
.filter(|s| s.path.is_file())
.map(|s| added(s.path, s.rel_path))
.collect();
all.sort_by(|a, b| a.rel_path.cmp(&b.rel_path));
all
}
};
let listed = without_untouched(listed, before);
let Some(start) = start.filter(|s| head.as_deref() != Some(*s)) else { return Ok(listed) };
// HEAD moved (a reset, a checkout, a pull): a file that is as it was when the task
// started is the user's doing, not the task's. Git compares tracked files, with its
// line-ending and other filters: the bytes on disk and in the commit differ under
// `core.autocrlf`.
let since_start: std::collections::HashSet<String> = repo
.diff_with_worktree(start)?
.into_iter()
.flat_map(|f| [Some(f.path), f.old_path].into_iter().flatten())
.collect();
let untracked: std::collections::HashSet<String> = repo
.status()?
.into_iter()
.filter(|s| s.unstaged == Some(ChangeKind::Untracked))
.map(|s| s.rel_path)
.collect();
Ok(listed
.into_iter()
.filter(|c| {
if before.contains_key(&c.rel_path) {
return true;
}
let Ok(Some(at_start)) = repo.file_bytes_at(&c.path.0, Some(start)) else { return true };
if !untracked.contains(&c.rel_path) {
return since_start.contains(&c.rel_path);
}
// Untracked since (a mixed reset): git doesn't diff it, so compare the bytes,
// CRLF on disk matching LF committed.
!std::fs::read(&c.path.0).is_ok_and(|now| now == at_start || crlf_to_lf(&now) == at_start)
})
.collect())
}
/// The task's uncommitted changes and, for a worktree task, the commits made since it
/// started (newest first). Both read the same `HEAD`, so a commit landing between the two
/// reads cannot put a file in both lists.
pub(crate) fn grouped_changes(&self, before: &BTreeMap<String, Stamp>) -> IdeResult<AgentChanges> {
let cwd = self.cwd();
let Some(repo) = discover(cwd)? else { return Ok(AgentChanges::default()) };
let head = repo.head()?.1;
let uncommitted = match &head {
Some(head) => without_untouched(changes(cwd, head)?, before),
None => {
// No commits yet: every file is new, as in `changes` for a local task.
let mut all: Vec<AgentChange> = repo
.status()?
.into_iter()
.filter(|s| s.path.is_file())
.map(|s| added(s.path, s.rel_path))
.collect();
all.sort_by(|a, b| a.rel_path.cmp(&b.rel_path));
without_untouched(all, before)
}
};
let commits = match (self, &head) {
(Self::Worktree { cwd, base }, Some(head)) => commits(cwd, base, head)?,
_ => Vec::new(),
};
Ok(AgentChanges { uncommitted, commits })
}
/// The revision the task's changed files go back to on rollback, and are compared with:
/// the commit the task started from, or the current `HEAD` once the user committed on top
/// of it, so those commits stay. `None` while the repository has no commits.
pub(crate) fn rev(&self) -> IdeResult<Option<String>> {
let (cwd, start) = match self {
Self::Worktree { base, .. } => return Ok(Some(base.clone())),
Self::Local { cwd, start } => (cwd, start.as_deref()),
};
let Some(repo) = discover(cwd)? else { return Ok(None) };
let head = repo.branch_info()?.head_commit;
// A start commit git no longer knows (garbage-collected) can't be gone back to.
let start = start.filter(|s| repo.merge_base(s, s).ok().flatten().is_some());
Ok(match (start, head) {
(Some(start), Some(head)) if start != head && repo.merge_base(start, &head)?.as_deref() != Some(start) => {
Some(start.to_owned())
}
(start, head) => head.or_else(|| start.map(str::to_owned)),
})
}
}
/// One side of an Accept (the task's worktree or the checkout): where its files really are, and
/// git's own folders for it.
struct Confined {
root: PathBuf,
metadata: Vec<PathBuf>,
}
impl Confined {
fn new(root: &Path, repo: Option<&Repo>) -> IdeResult<Self> {
let metadata = match repo {
Some(repo) => repo.metadata_dirs()?.iter().map(|d| ide_core::real_path::resolve_real(d)).collect(),
None => Vec::new(),
};
Ok(Self { root: dunce::canonicalize(root)?, metadata })
}
/// Whether `rel` may be read or written here: it doesn't name git's metadata under any
/// spelling (case, trailing dots or spaces, stream suffixes, short names), stays in the tree
/// once links are followed, and isn't a link itself.
fn allows(&self, rel: &str) -> bool {
use ide_core::real_path::{names_git_dir, reaches_git_dir, resolve_real};
let path = self.root.join(rel);
if names_git_dir(Path::new(rel)) || path.symlink_metadata().is_ok_and(|m| m.file_type().is_symlink()) {
return false;
}
let real = resolve_real(&path);
real.starts_with(&self.root)
&& !reaches_git_dir(&self.root, &real)
&& !self.metadata.iter().any(|dir| real.starts_with(dir))
}
}
/// Copies the task's `changes` (against `base`) from the worktree `cwd` into the
/// local checkout `root`, uncommitted: the user reviews and commits them with the
/// VCS tools. Files the checkout changed itself are left alone and reported, so
/// accepting never destroys the user's own edits. Returns accepted and skipped paths.
pub(crate) fn accept(
root: &Path,
cwd: &Path,
base: &str,
changes: &[AgentChange],
) -> IdeResult<crate::AgentAcceptOutcome> {
let repo = repo_at(cwd)?;
// Change paths are repository-relative, and the task's project may be a folder inside it.
let local = discover(root)?;
let local_root = local.as_ref().map_or_else(|| root.to_path_buf(), |r| r.root().to_path_buf());
// Paths come from what the agent left in its worktree: neither side may lead into git's own
// files or out through a link.
let task_side = Confined::new(repo.root(), Some(&repo))?;
let checkout_side = Confined::new(&local_root, local.as_ref())?;
let copyable = |rel: &str| task_side.allows(rel) && checkout_side.allows(rel);
let renamed_from: BTreeMap<String, String> =
repo.diff_with_worktree(base)?.into_iter().filter_map(|f| Some((f.path, f.old_path?))).collect();
// Brings one path of the checkout to the task's version, unless the checkout changed it itself.
let apply = |rel: &str| -> IdeResult<bool> {
let local = local_root.join(rel);
let in_task = repo.root().join(rel);
let base_bytes = repo.file_bytes_at(&in_task, Some(base))?;
let current = std::fs::read(&in_task).ok();
let local_bytes = std::fs::read(&local).ok();
if local_bytes == current {
return Ok(true);
}
// The checkout may hold the committed LF text with CRLF line ends (`core.autocrlf`).
let unchanged = match (&local_bytes, &base_bytes) {
(Some(now), Some(at_base)) => now == at_base || crlf_to_lf(now) == *at_base,
(now, at_base) => now == at_base,
};
if !unchanged {
return Ok(false);
}
match current {
Some(bytes) => {
if let Some(parent) = local.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(&local, bytes)?;
}
None => match std::fs::remove_file(&local) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => return Err(e.into()),
_ => {}
},
}
Ok(true)
};
let mut accepted = Vec::with_capacity(changes.len());
let mut skipped = Vec::new();
let skip = |path: &str| crate::AgentAcceptSkipped {
path: path.to_owned(),
reason: "the checkout has changes of its own to it; accept it by hand".to_owned(),
};
let refuse = |path: &str| crate::AgentAcceptSkipped {
path: path.to_owned(),
reason: "it leads into git's own files or through a link, so it is not copied".to_owned(),
};
for change in changes {
let old = renamed_from.get(&change.rel_path);
if !copyable(&change.rel_path) {
skipped.push(refuse(&change.rel_path));
continue;
}
if let Some(old) = old.filter(|old| !copyable(old)) {
skipped.push(refuse(old));
continue;
}
// A case-only rename (`a.txt` → `A.txt`). Where the disk resolves both
// spellings to one file it is renamed in place and `apply` then refreshes
// the content. On a case-sensitive disk the two names are different files:
// the old one's unchanged-ness (it is the file the checkout had) decides —
// then the task's version lands at the new name and the old one goes.
let case_only =
old.is_some_and(|o| o != &change.rel_path && o.to_lowercase() == change.rel_path.to_lowercase());
if case_only {
let old = old.expect("case_only implies a rename source");
let (old_path, new_path) = (local_root.join(old), local_root.join(&change.rel_path));
if same_file(&old_path, &new_path) {
std::fs::rename(&old_path, &new_path)?;
} else {
if !apply(old)? {
skipped.push(skip(old));
continue;
}
// `apply(old)` removed the old file (the task's tree has it only at
// the new name); now write that version under the new name. `apply`
// can't do it — it reads the new name's own base/absence — so the
// write is open-coded from the same task bytes `apply` would use.
if let Ok(bytes) = std::fs::read(repo.root().join(&change.rel_path)) {
if let Some(parent) = new_path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(&new_path, bytes)?;
}
accepted.push(change.rel_path.clone());
continue;
}
}
if !apply(&change.rel_path)? {
skipped.push(skip(&change.rel_path));
continue;
}
accepted.push(change.rel_path.clone());
// A rename also takes the file away from its old path (a case-only rename
// is handled above).
if !case_only
&& let Some(old) = old
&& !apply(old)?
{
skipped.push(skip(old));
}
}
Ok(crate::AgentAcceptOutcome { accepted, skipped })
}
/// Size and modification time of a file, `None` once it is gone.
pub(crate) type Stamp = Option<(u64, u128)>;
pub(crate) fn stamp(path: &Path) -> Stamp {
fn stamp(path: &Path) -> Stamp {
let meta = std::fs::metadata(path).ok()?;
let modified = meta.modified().ok()?.duration_since(UNIX_EPOCH).ok()?.as_nanos();
Some((meta.len(), modified))
@@ -522,10 +124,7 @@ pub(crate) fn stamp(path: &Path) -> Stamp {
/// The files already changed in `cwd` against `HEAD`, stamped so later edits to them show.
pub(crate) fn snapshot(cwd: &Path) -> IdeResult<BTreeMap<String, Stamp>> {
// Paths and stamps only: no diff and no line counts, which read every untracked file
// and held up starting a task in a checkout with many of them.
let Some(repo) = Repo::discover(cwd)? else { return Ok(BTreeMap::new()) };
Ok(repo.status()?.into_iter().map(|s| (s.rel_path, stamp(&s.path))).collect())
Ok(changes(cwd, "HEAD")?.into_iter().map(|c| (c.rel_path, stamp(&c.path))).collect())
}
/// Drops the changes that were there before the task and haven't been touched since.
@@ -533,82 +132,8 @@ pub(crate) fn without_untouched(changes: Vec<AgentChange>, before: &BTreeMap<Str
changes.into_iter().filter(|c| before.get(&c.rel_path).is_none_or(|s| *s != stamp(&c.path))).collect()
}
/// Files larger than this that were changed before the task keep no copy: they can't be rolled back.
const SAVED_MAX: u64 = 8 * 1024 * 1024;
fn saved_name(rel: &str) -> String {
use std::hash::{DefaultHasher, Hash, Hasher};
let mut hash = DefaultHasher::new();
rel.hash(&mut hash);
format!("{:016x}", hash.finish())
}
/// Copies the files in `before` (already changed in the checkout at `cwd` when the task started)
/// into `dir`, so rolling back the task's changes to them restores the user's own edits.
/// Whether anything was saved.
pub(crate) fn save_before(cwd: &Path, before: &BTreeMap<String, Stamp>, dir: &Path) -> IdeResult<bool> {
let Some(repo) = Repo::discover(cwd)? else { return Ok(false) };
let mut saved = false;
for (rel, stamp) in before {
let path = repo.root().join(rel);
if stamp.is_none() || !std::fs::metadata(&path).is_ok_and(|m| m.is_file() && m.len() <= SAVED_MAX) {
continue;
}
std::fs::create_dir_all(dir)?;
std::fs::copy(&path, dir.join(saved_name(rel)))?;
saved = true;
}
Ok(saved)
}
/// Rolls `change` back: to the copy saved when the task started when the file had changes
/// then (`before`, `saved` holding the copies), otherwise to `rev` (removed when `rev` lacks
/// it, or when there is no `rev`: the repository had no commits).
pub(crate) fn restore(
cwd: &Path,
rev: Option<&str>,
change: &AgentChange,
before: &BTreeMap<String, Stamp>,
saved: Option<&Path>,
) -> IdeResult<()> {
let path = &change.path.0;
let content = match before.get(&change.rel_path) {
Some(None) => None,
Some(Some(_)) => {
let copy =
saved.map(|d| d.join(saved_name(&change.rel_path))).filter(|p| p.is_file()).ok_or_else(|| {
IdeError::invalid(format!(
"{} had changes of yours when the task started that weren't saved; roll it back in Git instead",
change.rel_path
))
})?;
Some(std::fs::read(copy)?)
}
None => match rev {
Some(rev) => {
let repo = Repo::discover(cwd)?.ok_or_else(|| IdeError::invalid("not in a git repository"))?;
repo.file_bytes_at(path, Some(rev))?
}
None => None,
},
};
match content {
Some(bytes) => {
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(path, bytes)?;
}
None => match std::fs::remove_file(path) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => return Err(e.into()),
_ => {}
},
}
Ok(())
}
pub(crate) fn file_at(cwd: &Path, path: &Path, rev: &str) -> IdeResult<Option<String>> {
let Some(repo) = discover(cwd)? else { return Ok(None) };
let Some(repo) = Repo::discover(cwd)? else { return Ok(None) };
let encoding = match ide_vfs::load_text(path) {
Ok(loaded) => encoding_rs::Encoding::for_label(loaded.encoding.as_bytes()).unwrap_or(encoding_rs::UTF_8),
Err(_) => ide_vfs::detect_encoding(&repo.file_bytes_at(path, Some(rev))?.unwrap_or_default(), None),
@@ -717,18 +242,7 @@ mod tests {
fn git(dir: &Path, args: &[&str]) {
let ok = std::process::Command::new("git")
.args([
"-c",
"user.name=t",
"-c",
"user.email=t@t",
"-c",
"commit.gpgsign=false",
// Hermetic line endings: a developer or CI runner with
// `core.autocrlf=true` would otherwise check the fixtures out as CRLF.
"-c",
"core.autocrlf=false",
])
.args(["-c", "user.name=t", "-c", "user.email=t@t", "-c", "commit.gpgsign=false"])
.args(args)
.current_dir(dir)
.status()
@@ -776,88 +290,6 @@ mod tests {
assert!(!created.path.exists());
}
#[test]
fn commits_since_the_base_are_listed_with_their_files() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
std::fs::write(repo.join("a.txt"), "one\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
let created = create(&repo, &tmp.path().join("wt"), "Edit A", None).unwrap();
assert!(commits(&created.path, &created.base_commit, &created.base_commit).unwrap().is_empty());
std::fs::write(created.path.join("a.txt"), "one\ntwo\n").unwrap();
git(&created.path, &["add", "."]);
git(&created.path, &["commit", "-q", "-m", "change a"]);
std::fs::write(created.path.join("b.txt"), "b\n").unwrap();
git(&created.path, &["add", "."]);
git(&created.path, &["commit", "-q", "-m", "add b"]);
let head = head_of(&created.path).unwrap().1;
let log = commits(&created.path, &created.base_commit, &head).unwrap();
assert_eq!(log.iter().map(|c| c.summary.as_str()).collect::<Vec<_>>(), ["add b", "change a"]);
assert_eq!(log[0].files.iter().map(|f| f.rel_path.as_str()).collect::<Vec<_>>(), ["b.txt"]);
assert_eq!((log[0].files[0].added, log[0].files[0].removed), (1, 0));
assert_eq!(log[1].files.iter().map(|f| f.rel_path.as_str()).collect::<Vec<_>>(), ["a.txt"]);
assert_eq!((log[1].files[0].added, log[1].files[0].removed), (1, 0));
assert!(log.iter().all(|c| c.parent.is_some() && !c.message.is_empty()));
// Nothing is uncommitted after the commits; editing a committed file brings it back.
let at = TaskBase::Worktree { cwd: created.path.clone(), base: created.base_commit.clone() };
let grouped = at.grouped_changes(&BTreeMap::new()).unwrap();
assert!(grouped.uncommitted.is_empty());
assert_eq!(grouped.commits.len(), 2);
std::fs::write(created.path.join("a.txt"), "one\ntwo\nthree\n").unwrap();
let grouped = at.grouped_changes(&BTreeMap::new()).unwrap();
assert_eq!(grouped.uncommitted.iter().map(|c| c.rel_path.as_str()).collect::<Vec<_>>(), ["a.txt"]);
assert_eq!(grouped.commits.len(), 2);
}
#[test]
fn a_rename_in_a_commit_keeps_its_old_path() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
std::fs::write(repo.join("a.txt"), "one\ntwo\nthree\nfour\nfive\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
let base = head_of(&repo).unwrap().1;
std::fs::rename(repo.join("a.txt"), repo.join("b.txt")).unwrap();
std::fs::write(repo.join("b.txt"), "one\ntwo\nthree\nfour\nFIVE\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "rename a"]);
let head = head_of(&repo).unwrap().1;
let log = commits(&repo, &base, &head).unwrap();
assert_eq!(log.len(), 1);
assert_eq!(log[0].files.len(), 1);
assert_eq!(log[0].files[0].rel_path, "b.txt");
assert_eq!(log[0].files[0].old_path.as_deref(), Some("a.txt"));
assert_eq!(log[0].files[0].kind, ChangeKind::Renamed);
}
#[test]
fn a_local_task_has_no_commits() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
std::fs::write(repo.join("a.txt"), "one\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
std::fs::write(repo.join("a.txt"), "one\ntwo\n").unwrap();
let at = TaskBase::Local { cwd: repo.clone(), start: None };
let grouped = at.grouped_changes(&BTreeMap::new()).unwrap();
assert!(grouped.commits.is_empty());
assert_eq!(grouped.uncommitted.iter().map(|c| c.rel_path.as_str()).collect::<Vec<_>>(), ["a.txt"]);
}
#[test]
fn a_broken_worktree_registration_still_leaves_a_deletable_folder() {
let tmp = tempfile::tempdir().unwrap();
@@ -888,245 +320,4 @@ mod tests {
assert!(remove(&repo, &tmp.path().join("wt"), &elsewhere).is_err());
assert!(elsewhere.join("keep.txt").exists());
}
fn accept_repo() -> (tempfile::TempDir, PathBuf, Created) {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
// Hermetic line endings: a machine with `core.autocrlf=true` would check
// the fixtures and the worktree out as CRLF.
git(&repo, &["config", "core.autocrlf", "false"]);
std::fs::write(repo.join("a.txt"), "one\ntwo\n").unwrap();
std::fs::write(repo.join("gone.txt"), "bye\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
let created = create(&repo, &tmp.path().join("wt"), "Edit A", None).unwrap();
(tmp, repo, created)
}
fn listed(cwd: &Path, base: &str) -> Vec<AgentChange> {
without_untouched(changes(cwd, base).unwrap(), &BTreeMap::new())
}
#[test]
fn accept_moves_a_renamed_file_and_maps_paths_from_a_project_folder() {
let (_tmp, repo, created) = accept_repo();
std::fs::create_dir(created.path.join("app")).unwrap();
git(&created.path, &["mv", "a.txt", "app/moved.txt"]);
std::fs::write(created.path.join("app/new.txt"), "n\n").unwrap();
git(&created.path, &["add", "."]);
git(&created.path, &["-c", "user.name=t", "-c", "user.email=t@t", "commit", "-q", "-m", "move"]);
// The task's project is the repository's `app` folder.
let project = repo.join("app");
std::fs::create_dir(&project).unwrap();
let changes = listed(&created.path, &created.base_commit);
let outcome = accept(&project, &created.path, &created.base_commit, &changes).unwrap();
assert!(outcome.skipped.is_empty(), "{:?}", outcome.skipped);
assert_eq!(std::fs::read_to_string(repo.join("app/moved.txt")).unwrap(), "one\ntwo\n");
assert_eq!(std::fs::read_to_string(repo.join("app/new.txt")).unwrap(), "n\n");
assert!(!repo.join("a.txt").exists(), "the rename's old path is gone");
assert!(!repo.join("app/app").exists());
}
#[test]
fn accept_treats_a_crlf_checkout_of_the_base_as_unchanged() {
let (_tmp, repo, created) = accept_repo();
// The checkout holds the committed text with CRLF line ends, as core.autocrlf leaves it.
std::fs::write(repo.join("a.txt"), "one\r\ntwo\r\n").unwrap();
std::fs::write(created.path.join("a.txt"), "one\nTWO\n").unwrap();
git(&created.path, &["mv", "gone.txt", "kept.txt"]);
std::fs::write(repo.join("gone.txt"), "bye\r\n").unwrap();
let changes = listed(&created.path, &created.base_commit);
let outcome = accept(&repo, &created.path, &created.base_commit, &changes).unwrap();
assert!(outcome.skipped.is_empty(), "{:?}", outcome.skipped);
assert_eq!(std::fs::read_to_string(repo.join("a.txt")).unwrap(), "one\nTWO\n");
assert!(repo.join("kept.txt").exists() && !repo.join("gone.txt").exists());
}
#[test]
fn accept_applies_a_case_only_rename() {
let (_tmp, repo, created) = accept_repo();
git(&created.path, &["mv", "a.txt", "A.txt"]);
let changes = listed(&created.path, &created.base_commit);
let outcome = accept(&repo, &created.path, &created.base_commit, &changes).unwrap();
assert!(outcome.skipped.is_empty(), "{:?}", outcome.skipped);
let names: Vec<_> = std::fs::read_dir(&repo)
.unwrap()
.map(|e| e.unwrap().file_name().to_string_lossy().into_owned())
.filter(|n| n.eq_ignore_ascii_case("a.txt"))
.collect();
assert_eq!(names, ["A.txt"]);
}
#[cfg(windows)]
#[test]
fn accept_does_not_write_into_the_checkouts_git_folder_through_its_short_name() {
let (_tmp, repo, created) = accept_repo();
if !repo.join("GIT~1").join("HEAD").is_file() {
eprintln!("8.3 short names are disabled on this volume; nothing to check");
return;
}
// Written in the worktree under the name the checkout's `.git` folder also has.
let wt = &created.path;
std::fs::rename(wt.join(".git"), wt.join("gitfile")).unwrap();
std::fs::create_dir_all(wt.join("GIT~1").join("hooks")).unwrap();
std::fs::write(wt.join("GIT~1").join("hooks").join("pre-commit"), "#!/bin/sh\nexit 0\n").unwrap();
std::fs::rename(wt.join("gitfile"), wt.join(".git")).unwrap();
std::fs::write(wt.join("a.txt"), "one\nTWO\n").unwrap();
let changes = listed(wt, &created.base_commit);
assert!(changes.iter().any(|c| c.rel_path == "GIT~1/hooks/pre-commit"), "control: git lists it");
let outcome = accept(&repo, wt, &created.base_commit, &changes).unwrap();
assert!(!repo.join(".git").join("hooks").join("pre-commit").exists(), "{outcome:?}");
assert_eq!(outcome.accepted, ["a.txt"]);
assert_eq!(outcome.skipped.iter().map(|s| s.path.as_str()).collect::<Vec<_>>(), ["GIT~1/hooks/pre-commit"]);
}
#[test]
fn accept_refuses_paths_that_lead_into_git_metadata_under_any_spelling() {
let (tmp, repo, _created) = accept_repo();
let checkout = Confined::new(&repo, Repo::discover(&repo).unwrap().as_ref()).unwrap();
for rel in [
".git/hooks/pre-commit",
".GIT/hooks/pre-commit",
".git./hooks/pre-commit",
".git /hooks/pre-commit",
".git::$INDEX_ALLOCATION/hooks/pre-commit",
"sub/.git/config",
"../outside.txt",
] {
assert!(!checkout.allows(rel), "{rel}");
}
assert!(checkout.allows("src/new.txt"));
assert!(checkout.allows(".github/workflows/ci.yml"));
// git's folder under another name inside the tree (`--separate-git-dir`).
let separate = tmp.path().join("separate");
std::fs::create_dir(&separate).unwrap();
git(&separate, &["init", "-q", "--separate-git-dir", "meta", "-b", "main"]);
let side = Confined::new(&separate, Repo::discover(&separate).unwrap().as_ref()).unwrap();
assert!(!side.allows("meta/hooks/pre-commit"));
assert!(side.allows("metadata.txt"));
}
/// A link the task left in its worktree, or a linked folder in the checkout: neither is followed.
#[test]
fn accept_does_not_follow_links_on_either_side() {
let (tmp, repo, created) = accept_repo();
let outside = tmp.path().join("outside");
std::fs::create_dir(&outside).unwrap();
std::fs::write(outside.join("secret.txt"), "secret\n").unwrap();
// A folder link in the checkout (a junction on Windows, which needs no privilege).
#[cfg(unix)]
std::os::unix::fs::symlink(&outside, repo.join("linked")).unwrap();
#[cfg(windows)]
assert!(
std::process::Command::new("cmd")
.args(["/c", "mklink", "/J"])
.arg(repo.join("linked"))
.arg(&outside)
.output()
.unwrap()
.status
.success()
);
std::fs::create_dir(created.path.join("linked")).unwrap();
std::fs::write(created.path.join("linked/new.txt"), "x\n").unwrap();
std::fs::write(created.path.join("a.txt"), "one\nTWO\n").unwrap();
// A file link in the worktree, where the platform lets this user create one.
#[cfg(unix)]
let file_link = std::os::unix::fs::symlink(outside.join("secret.txt"), created.path.join("copied.txt"));
#[cfg(windows)]
let file_link = std::os::windows::fs::symlink_file(outside.join("secret.txt"), created.path.join("copied.txt"));
if let Err(e) = &file_link {
eprintln!("no file link ({e}); checking the folder link only");
}
let changes: Vec<_> = listed(&created.path, &created.base_commit);
let outcome = accept(&repo, &created.path, &created.base_commit, &changes).unwrap();
assert!(!outside.join("new.txt").exists(), "{outcome:?}");
assert!(!repo.join("copied.txt").exists(), "{outcome:?}");
assert!(outcome.accepted.contains(&"a.txt".to_owned()));
let refused: Vec<_> = outcome.skipped.iter().map(|s| s.path.as_str()).collect();
if file_link.is_ok() {
assert_eq!(refused, ["copied.txt", "linked/new.txt"]);
} else {
assert_eq!(refused, ["linked/new.txt"]);
}
}
#[test]
fn accept_copies_new_modified_and_deleted_files_into_the_checkout() {
let (_tmp, repo, created) = accept_repo();
std::fs::write(created.path.join("a.txt"), "one\nTWO\n").unwrap();
std::fs::write(created.path.join("new.txt"), "x\n").unwrap();
std::fs::remove_file(created.path.join("gone.txt")).unwrap();
let outcome =
accept(&repo, &created.path, &created.base_commit, &listed(&created.path, &created.base_commit)).unwrap();
assert!(outcome.skipped.is_empty());
assert_eq!(outcome.accepted, ["a.txt", "gone.txt", "new.txt"]);
assert_eq!(std::fs::read_to_string(repo.join("a.txt")).unwrap(), "one\nTWO\n");
assert_eq!(std::fs::read_to_string(repo.join("new.txt")).unwrap(), "x\n");
assert!(!repo.join("gone.txt").exists());
// Accepting again changes nothing: the checkout already matches.
let again =
accept(&repo, &created.path, &created.base_commit, &listed(&created.path, &created.base_commit)).unwrap();
assert!(again.skipped.is_empty());
assert_eq!(again.accepted, ["a.txt", "gone.txt", "new.txt"]);
}
#[test]
fn a_local_task_goes_back_to_its_start_unless_the_user_committed_on_top_of_it() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
let local = |start: Option<&str>| TaskBase::Local { cwd: repo.clone(), start: start.map(str::to_owned) };
let head = || head_of(&repo).unwrap().1;
let commit = |name: &str| {
std::fs::write(repo.join(name), "x\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", name]);
head_of(&repo).unwrap().1
};
assert_eq!(local(None).rev().unwrap(), None, "no commits yet");
let first = commit("a.txt");
assert_eq!(local(None).rev().unwrap(), Some(first.clone()), "an imported session: HEAD");
let start = commit("b.txt");
assert_eq!(local(Some(&start)).rev().unwrap(), Some(start.clone()), "HEAD where it started");
let ahead = commit("c.txt");
assert_eq!(local(Some(&start)).rev().unwrap(), Some(ahead), "committed on top: HEAD");
git(&repo, &["reset", "-q", "--soft", &first]);
assert_eq!(local(Some(&start)).rev().unwrap(), Some(start.clone()), "moved back: the start");
git(&repo, &["checkout", "-q", "-b", "side"]);
commit("d.txt");
assert_eq!(local(Some(&start)).rev().unwrap(), Some(start.clone()), "another branch: the start");
let gone = "0123456789abcdef0123456789abcdef01234567";
assert_eq!(local(Some(gone)).rev().unwrap(), Some(head()), "an unknown start: HEAD");
let worktree = TaskBase::Worktree { cwd: repo.clone(), base: start.clone() };
assert_eq!(worktree.rev().unwrap(), Some(start));
}
#[test]
fn accept_leaves_files_the_checkout_changed_itself() {
let (_tmp, repo, created) = accept_repo();
std::fs::write(created.path.join("a.txt"), "one\nTWO\n").unwrap();
std::fs::write(created.path.join("gone.txt"), "bye\nbye\n").unwrap();
std::fs::write(repo.join("a.txt"), "one\nMINE\n").unwrap();
let outcome =
accept(&repo, &created.path, &created.base_commit, &listed(&created.path, &created.base_commit)).unwrap();
assert_eq!(outcome.accepted, ["gone.txt"]);
assert_eq!(outcome.skipped.len(), 1);
assert_eq!(outcome.skipped[0].path, "a.txt");
assert!(!outcome.skipped[0].reason.is_empty());
// The user's own edit survives; the other file comes over.
assert_eq!(std::fs::read_to_string(repo.join("a.txt")).unwrap(), "one\nMINE\n");
assert_eq!(std::fs::read_to_string(repo.join("gone.txt")).unwrap(), "bye\nbye\n");
}
}
-2
View File
@@ -7,9 +7,7 @@ license.workspace = true
[dependencies]
blake3.workspace = true
chardetng.workspace = true
dunce.workspace = true
encoding_rs.workspace = true
ide-core.workspace = true
zip.workspace = true
serde.workspace = true
+22 -188
View File
@@ -79,26 +79,23 @@ fn scan(
Ok(())
}
/// One planned step as fingerprinted: resolved program, args, cwd and env.
type StepFingerprint = (PathBuf, Vec<String>, PathBuf, BTreeMap<String, String>);
/// Environment, planned steps and input paths shared by every Maven build
/// fingerprint: the filtered process environment plus the wrapper, settings
/// and JDK files the planned invocation reads.
struct Collected {
inputs: Vec<PathBuf>,
environment: BTreeMap<String, String>,
steps: Vec<StepFingerprint>,
inherited_hash: u64,
}
fn collect(
/// `None` means an input cannot be resolved safely: build normally without reuse.
/// All filesystem work belongs on the caller's blocking thread.
pub fn launch_fingerprint(
root: &Path,
projects: &[&maven::MavenProject],
plan: &BuildPlan,
model: &BuildModel,
module: &Path,
test: bool,
local_repo: Option<PathBuf>,
) -> IdeResult<Option<Collected>> {
let Some(mut inputs) = maven::launch_inputs(root, projects, local_repo) else { return Ok(None) };
cancel: &CancellationToken,
) -> IdeResult<Option<LaunchFingerprint>> {
let Some(m) = &model.maven else { return Ok(None) };
let projects = maven_launch_projects(root, m, module, test);
if projects.is_empty() {
return Ok(None);
}
let Some(mut inputs) = maven::launch_inputs(root, &projects, local_repo) else { return Ok(None) };
let plan = launch_plan(root, model, module, test, BuildKind::Build)?;
let mut environment: BTreeMap<String, String> = std::env::vars()
.map(|(key, value)| (key.to_ascii_uppercase(), value))
.filter(|(key, _)| {
@@ -168,132 +165,23 @@ fn collect(
Path::new(home).join(if cfg!(windows) { "bin/java.exe" } else { "bin/java" }),
]);
}
let steps = plan
.steps
.iter()
.map(|s| (process::program_path(&s.program, &s.env), s.args.clone(), s.cwd.clone(), s.env.clone()))
.collect();
let steps: Vec<_> =
plan.steps.iter().map(|s| (process::program_path(&s.program, &s.env), &s.args, &s.cwd, &s.env)).collect();
// Profiles and plugins can read arbitrary environment variables. Keep a signature
// of the inherited environment without persisting unrelated credentials.
let inherited: BTreeMap<_, _> = std::env::vars_os().collect();
let mut inherited_hash = std::hash::DefaultHasher::new();
inherited.hash(&mut inherited_hash);
Ok(Some(Collected { inputs, environment, steps, inherited_hash: inherited_hash.finish() }))
}
/// Scans every input, storing project inputs relative to the project.
fn scan_all(
root: &Path,
inputs: Vec<PathBuf>,
cancel: &CancellationToken,
) -> IdeResult<BTreeMap<PathBuf, Option<Stamp>>> {
let parameters =
serde_json::to_string(&(root, module, test, steps, environment, inherited_hash.finish(), projects))
.map_err(|e| IdeError::internal(e.to_string()))?;
let mut files = BTreeMap::new();
for path in inputs {
scan(root, &path, &mut files, &mut HashSet::new(), cancel)?;
}
// Store project inputs relative to the project, including each file's relative path.
Ok(files.into_iter().map(|(p, stamp)| (p.strip_prefix(root).unwrap_or(&p).to_path_buf(), stamp)).collect())
}
/// `None` means an input cannot be resolved safely: build normally without reuse.
/// All filesystem work belongs on the caller's blocking thread.
pub fn launch_fingerprint(
root: &Path,
model: &BuildModel,
module: &Path,
test: bool,
local_repo: Option<PathBuf>,
cancel: &CancellationToken,
) -> IdeResult<Option<LaunchFingerprint>> {
let Some(m) = &model.maven else { return Ok(None) };
let projects = maven_launch_projects(root, m, module, test);
if projects.is_empty() {
return Ok(None);
}
let plan = launch_plan(root, model, module, test, BuildKind::Build)?;
let Some(collected) = collect(root, &projects, &plan, local_repo)? else { return Ok(None) };
let Collected { inputs, environment, steps, inherited_hash } = collected;
let parameters = serde_json::to_string(&(root, module, test, steps, environment, inherited_hash, projects))
.map_err(|e| IdeError::internal(e.to_string()))?;
Ok(Some(LaunchFingerprint { version: 2, parameters, files: scan_all(root, inputs, cancel)? }))
}
/// Inputs fingerprint of a menu build (`Build Project` / `Build Module`): the
/// built modules' sources, resources and POM chain plus the exact planned
/// steps, so an unchanged project skips Maven entirely (#2476). Only `Build`
/// callers reuse it, never `Rebuild` or `Artifacts`; `None` disables reuse.
/// All filesystem work belongs on the caller's blocking thread.
pub fn menu_fingerprint(
root: &Path,
model: &BuildModel,
file: Option<&Path>,
cancel: &CancellationToken,
) -> IdeResult<Option<LaunchFingerprint>> {
let plan = plan(root, model, BuildKind::Build, file, None)?;
if plan.system != BuildSystem::Maven {
return Ok(None);
}
let Some(m) = &model.maven else { return Ok(None) };
let projects: Vec<&maven::MavenProject> = match file {
Some(file) => {
// The innermost linked project owning the file, as in single-file compilation.
let rel = rel_str(root, file);
let module = m
.projects
.iter()
.filter(|p| rel == p.dir || rel.starts_with(&format!("{}/", p.dir)) || p.dir.is_empty())
.max_by_key(|p| p.dir.len());
match module {
Some(project) if project.packaging != "pom" => {
maven_launch_projects(root, m, &subdir(root, &project.dir), true)
}
_ => return Ok(None),
}
}
// A project build walks every linked project.
None => m.projects.iter().collect(),
};
if projects.is_empty() {
return Ok(None);
}
let Some(collected) = collect(root, &projects, &plan, None)? else { return Ok(None) };
let Collected { inputs, environment, steps, inherited_hash } = collected;
let scope = file.map(|f| rel_str(root, f));
let parameters = serde_json::to_string(&("menu-build", scope, steps, environment, inherited_hash))
.map_err(|e| IdeError::internal(e.to_string()))?;
Ok(Some(LaunchFingerprint { version: 2, parameters, files: scan_all(root, inputs, cancel)? }))
}
impl LaunchFingerprint {
/// Folders (relative to the project) of the Maven projects whose POM is an input.
pub(super) fn project_dirs(&self) -> impl Iterator<Item = &Path> {
self.files
.keys()
.filter(|p| p.is_relative() && p.file_name().is_some_and(|n| n == "pom.xml"))
.map(|pom| pom.parent().unwrap_or(Path::new("")))
}
/// Relative paths whose content changed while `(length, mtime)` stayed the same
/// compared to `previous` (timestamp-preserving external edits, coarse-granularity
/// filesystems). Timestamp-incremental builders (Maven, the Kotlin plugin) would
/// skip these sources as "unchanged", so the rebuild the fingerprint triggers
/// compiles nothing and the stale outputs get re-recorded as fresh. Callers
/// re-stamp the returned paths before building. Inputs without a content hash
/// (directories, files outside the project) are never reported.
pub fn mtime_preserved_edits(&self, current: &Self) -> Vec<PathBuf> {
let mut edits = Vec::new();
for (path, current_stamp) in &current.files {
let Some(current_stamp) = current_stamp else { continue };
let Some(previous_stamp) = self.files.get(path).and_then(|stamp| stamp.as_ref()) else { continue };
let (len, mtime, Some(current_hash)) = current_stamp else { continue };
let (prev_len, prev_mtime, Some(prev_hash)) = previous_stamp else { continue };
if current_hash != prev_hash && len == prev_len && mtime == prev_mtime {
edits.push(path.clone());
}
}
edits.sort();
edits
}
let files = files.into_iter().map(|(p, stamp)| (p.strip_prefix(root).unwrap_or(&p).to_path_buf(), stamp)).collect();
Ok(Some(LaunchFingerprint { version: 2, parameters, files }))
}
#[cfg(test)]
@@ -394,60 +282,6 @@ mod tests {
assert_ne!(before, after, "a same-length, same-mtime content change must invalidate the fingerprint");
}
#[test]
fn mtime_preserved_edits_only_reports_lying_timestamps() {
fn stamp(len: u64, hash: &str) -> Option<Stamp> {
Some((len, SystemTime::UNIX_EPOCH + std::time::Duration::from_secs(42), Some(hash.into())))
}
fn prints(
previous: BTreeMap<PathBuf, Option<Stamp>>,
current: BTreeMap<PathBuf, Option<Stamp>>,
) -> Vec<PathBuf> {
let previous = LaunchFingerprint { version: 2, parameters: String::new(), files: previous };
let current = LaunchFingerprint { version: 2, parameters: String::new(), files: current };
previous.mtime_preserved_edits(&current)
}
// Same length and mtime, different content: the builder would skip it.
assert_eq!(
prints([("a.java".into(), stamp(5, "old"))].into(), [("a.java".into(), stamp(5, "new"))].into()),
vec![PathBuf::from("a.java")]
);
// A normal edit bumps the mtime: the builder already sees it.
let mut touched = stamp(5, "new").unwrap();
touched.1 += std::time::Duration::from_secs(1);
assert!(
prints([("a.java".into(), stamp(5, "old"))].into(), [("a.java".into(), Some(touched))].into()).is_empty()
);
// Added, deleted, and hash-less inputs (directories, files outside the
// project) carry no comparable content: never reported.
assert!(
prints(
[("gone.java".into(), stamp(5, "old")), ("dir".into(), None)].into(),
[("new.java".into(), stamp(5, "new")), ("dir".into(), None)].into()
)
.is_empty()
);
}
#[test]
fn lying_source_is_found_against_a_real_fingerprint() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let source = root.join("lib/src/main/kotlin/Channel.kt");
let before = fingerprint(root, &model).unwrap();
let mtime = fs::metadata(&source).unwrap().modified().unwrap();
// Same 5-byte length, different content, preserved mtime.
write(root, "lib/src/main/kotlin/Channel.kt", "valve");
fs::File::options().write(true).open(&source).unwrap().set_modified(mtime).unwrap();
let after = fingerprint(root, &model).unwrap();
assert_eq!(before.mtime_preserved_edits(&after), [PathBuf::from("lib/src/main/kotlin/Channel.kt")]);
// A normal edit (mtime advances) needs no repair.
write(root, "lib/src/main/kotlin/Channel.kt", "valve!");
let normal = fingerprint(root, &model).unwrap();
assert!(before.mtime_preserved_edits(&normal).is_empty());
}
#[test]
fn profile_options_jdk_goal_and_wrapper_invalidate_without_using_class_outputs() {
let dir = tempfile::tempdir().unwrap();
@@ -1,253 +0,0 @@
//! Resources a Gradle build changes while copying them (filters, `expand`, renames, extra
//! sources). The Java language server copies them as they are into Buildship's
//! `bin/<source set>` folders, which launches run from; IntelliJ IDEA runs Gradle's copy.
//! Before a launch, Gradle processes them and its output replaces those plain copies, so
//! each resource is on the classpath once, as the build produces it.
use super::{BuildPlan, BuildStep, BuildSystem, Format, run, subdir};
use crate::BuildModel;
use crate::gradle::{self, GradleModel, ReportedProject};
use ide_core::{CancellationToken, IdeError, IdeResult, Severity};
use std::fs;
use std::path::{Path, PathBuf};
/// A source set whose resources Gradle processes, with the folder the launch reads them from.
#[derive(Debug, PartialEq)]
struct Target {
/// Task path from the root build (`:app:processResources`).
task: String,
/// Where the task writes the resources.
output: PathBuf,
/// The Java language server's output folder of the source set, on the launch classpath.
jdtls: PathBuf,
}
fn targets(reported: &[ReportedProject], class_paths: &[String]) -> Vec<Target> {
// buildSrc is built before the build, never addressable as `:buildSrc:<task>`.
let addressable = |p: &&ReportedProject| !p.build.as_deref().is_some_and(|b| b.ends_with(":buildSrc"));
let sets: Vec<_> = reported
.iter()
.filter(addressable)
.flat_map(|p| p.source_sets.iter().filter_map(move |s| Some((p, s, s.processed.as_ref()?))))
.collect();
if sets.is_empty() {
return Vec::new();
}
// Canonical, as Gradle reports its folders: jdtls may spell them differently.
let entries: Vec<PathBuf> = class_paths.iter().filter_map(|p| dunce::canonicalize(p).ok()).collect();
sets.into_iter()
.filter_map(|(p, set, resources)| {
// Buildship's output folder of the source set.
let jdtls = dunce::canonicalize(Path::new(&p.dir).join("bin").join(&set.name)).ok()?;
let project = if p.path == ":" { "" } else { p.path.as_str() };
entries.contains(&jdtls).then(|| Target {
task: format!("{project}:{}", resources.task),
output: PathBuf::from(&resources.dir),
jdtls,
})
})
.collect()
}
/// Before a JVM launch from the Java language server's output folders on `class_paths`:
/// runs the Gradle tasks that process resources of those folders' source sets (see the
/// module docs) and copies their output over the plain copies. `cache_dir` holds the
/// linked builds' last syncs. Returns the tasks run, none for builds that copy their
/// resources as they are.
pub fn process_launch_resources(
root: &Path,
model: &BuildModel,
cache_dir: &Path,
class_paths: &[String],
cancel: &CancellationToken,
) -> IdeResult<Vec<String>> {
// A composite also reports the builds it includes: the deepest build owning a folder runs its task.
let mut builds: Vec<&GradleModel> = model.gradle.iter().collect();
builds.sort_by_key(|g| std::cmp::Reverse(subdir(root, &g.dir).components().count()));
let mut done: Vec<PathBuf> = Vec::new();
let mut tasks = Vec::new();
for g in builds {
let reported = crate::sync::reported_gradle_projects(&gradle::cache_dir(cache_dir, &g.dir));
let targets: Vec<Target> =
targets(&reported, class_paths).into_iter().filter(|t| !done.contains(&t.jdtls)).collect();
if targets.is_empty() {
continue;
}
let mut step = BuildStep::new(&g.command, &["--console=plain"], &subdir(root, &g.dir), Format::Gradle);
step.args.extend(targets.iter().map(|t| t.task.clone()));
step.env = g.env.clone();
let plan = BuildPlan { system: BuildSystem::Gradle, title: "Process resources".into(), steps: vec![step] };
let result = run(root, &plan, cancel, &|_| {})?;
if !result.success {
let errors: Vec<&str> = result
.messages
.iter()
.filter(|m| m.severity == Severity::Error)
.take(5)
.map(|m| m.text.as_str())
.collect();
return Err(IdeError::internal(errors.join("\n")));
}
for t in targets {
if cancel.is_cancelled() {
return Err(IdeError::cancelled());
}
overlay(&t.output, &t.output, &t.jdtls)?;
done.push(t.jdtls);
tasks.push(t.task);
}
}
Ok(tasks)
}
/// Copies the files under `dir` to the same place under `to` (relative to `base`) where
/// they differ, so unchanged resources are not rewritten on every launch.
fn overlay(base: &Path, dir: &Path, to: &Path) -> IdeResult<()> {
let entries = match fs::read_dir(dir) {
// No resources: the task had nothing to copy.
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(()),
entries => entries?,
};
for entry in entries {
let entry = entry?;
let path = entry.path();
let kind = entry.file_type()?;
if kind.is_dir() {
overlay(base, &path, to)?;
} else if kind.is_file() {
let dest = to.join(path.strip_prefix(base).map_err(|e| IdeError::internal(e.to_string()))?);
if !same_content(&path, &dest) {
if let Some(parent) = dest.parent() {
fs::create_dir_all(parent)?;
}
fs::copy(&path, &dest)?;
}
}
}
Ok(())
}
fn same_content(a: &Path, b: &Path) -> bool {
match (fs::metadata(a), fs::metadata(b)) {
(Ok(m), Ok(n)) if m.len() == n.len() => matches!((fs::read(a), fs::read(b)), (Ok(x), Ok(y)) if x == y),
_ => false,
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::gradle::{ProcessedResources, ReportedSourceSet};
fn write(path: &Path, text: &str) {
fs::create_dir_all(path.parent().unwrap()).unwrap();
fs::write(path, text).unwrap();
}
fn set(name: &str, processed: Option<&Path>) -> ReportedSourceSet {
ReportedSourceSet {
name: name.into(),
test: name == "test",
processed: processed.map(|dir| ProcessedResources {
task: if name == "main" { "processResources".into() } else { "processTestResources".into() },
dir: dir.to_string_lossy().into_owned(),
}),
..Default::default()
}
}
fn project(path: &str, dir: &Path, sets: Vec<ReportedSourceSet>) -> ReportedProject {
ReportedProject {
path: path.into(),
dir: dir.to_string_lossy().into_owned(),
source_sets: sets,
..Default::default()
}
}
/// Only processed source sets whose jdtls output is on the classpath: the launched
/// module, upstream modules of the build, tests only for a test classpath; never buildSrc.
#[test]
fn targets_are_processed_sets_on_the_classpath() {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
let (app, lib, plain, src) = (root.join("app"), root.join("lib"), root.join("plain"), root.join("buildSrc"));
for dir in [&root, &app, &lib, &plain, &src] {
for set in ["main", "test"] {
fs::create_dir_all(dir.join("bin").join(set)).unwrap();
}
}
let out = |dir: &Path, set: &str| dir.join("build/resources").join(set);
let mut build_src = project(":buildSrc", &src, vec![set("main", Some(&out(&src, "main")))]);
build_src.build = Some(":buildSrc".into());
let reported = [
project(":", &root, vec![set("main", Some(&out(&root, "main")))]),
project(":app", &app, vec![set("main", Some(&out(&app, "main"))), set("test", Some(&out(&app, "test")))]),
project(":lib", &lib, vec![set("main", Some(&out(&lib, "main")))]),
project(":plain", &plain, vec![set("main", None)]),
build_src,
];
let entry = |dir: &Path, set: &str| dir.join("bin").join(set).to_string_lossy().into_owned();
let runtime = [
entry(&app, "main"),
// Spelled differently from Gradle's canonical folders.
format!("{}/", entry(&lib, "main")),
entry(&plain, "main"),
entry(&src, "main"),
"/m2/spring-core.jar".into(),
];
let tasks = |cp: &[String]| targets(&reported, cp).into_iter().map(|t| t.task).collect::<Vec<_>>();
assert_eq!(tasks(&runtime), [":app:processResources", ":lib:processResources"]);
let mut test = runtime.to_vec();
test.insert(0, entry(&app, "test"));
assert_eq!(tasks(&test), [":app:processResources", ":app:processTestResources", ":lib:processResources"]);
assert_eq!(tasks(&[entry(&root, "main")]), [":processResources"], "the root project");
let app_targets = targets(&reported, &runtime);
let t = &app_targets[0];
assert_eq!(
(t.output.as_path(), t.jdtls.as_path()),
(out(&app, "main").as_path(), app.join("bin/main").as_path())
);
let unprocessed = [project(":app", &app, vec![set("main", None)])];
assert!(targets(&unprocessed, &runtime).is_empty());
}
/// The processed copy replaces jdtls's plain one (the reported `@version@` case), new
/// files are added, identical ones are left alone and files Gradle doesn't produce stay.
#[test]
fn overlay_replaces_plain_copies_with_gradle_output() {
let tmp = tempfile::tempdir().unwrap();
let (output, jdtls) = (tmp.path().join("build/resources/main"), tmp.path().join("bin/main"));
write(&output.join("application.yaml"), "spring:\n application:\n version: 0.0.1\n");
write(&output.join("static/extra.txt"), "from an extra source");
write(&output.join("migrations/V1__init.sql"), "create table t (id int);");
write(&jdtls.join("application.yaml"), "spring:\n application:\n version: @version@\n");
write(&jdtls.join("migrations/V1__init.sql"), "create table t (id int);");
write(&jdtls.join("com/example/App.class"), "bytecode");
let unchanged = fs::metadata(jdtls.join("migrations/V1__init.sql")).unwrap().modified().unwrap();
std::thread::sleep(std::time::Duration::from_millis(20));
overlay(&output, &output, &jdtls).unwrap();
let read = |rel: &str| fs::read_to_string(jdtls.join(rel)).unwrap();
assert_eq!(read("application.yaml"), "spring:\n application:\n version: 0.0.1\n");
assert_eq!(read("static/extra.txt"), "from an extra source");
assert_eq!(read("com/example/App.class"), "bytecode");
let modified = fs::metadata(jdtls.join("migrations/V1__init.sql")).unwrap().modified().unwrap();
assert_eq!(modified, unchanged, "identical files are not rewritten");
overlay(&tmp.path().join("missing"), &tmp.path().join("missing"), &jdtls).unwrap();
}
#[test]
fn builds_without_processed_resources_run_nothing() {
let tmp = tempfile::tempdir().unwrap();
// Running this command would fail the call.
let missing = tmp.path().join("missing-gradle").to_string_lossy().into_owned();
let model =
BuildModel { gradle: vec![GradleModel { command: missing, ..Default::default() }], ..Default::default() };
let class_paths = [tmp.path().join("bin/main").to_string_lossy().into_owned()];
let tasks =
process_launch_resources(tmp.path(), &model, tmp.path(), &class_paths, &CancellationToken::new()).unwrap();
assert!(tasks.is_empty());
}
}
+6 -85
View File
@@ -4,17 +4,9 @@
//! for the Build tool window, the Problems view and the editor.
mod fingerprint;
mod launch_resources;
mod parse;
mod single;
mod up_to_date;
pub use fingerprint::{LaunchFingerprint, launch_fingerprint, menu_fingerprint};
pub use launch_resources::process_launch_resources;
pub use single::{
SingleFileTarget, compile_file_plan, find_javac, javac_step, single_file_classpath, single_file_target,
};
pub use up_to_date::{menu_up_to_date, record_menu_build};
pub use fingerprint::{LaunchFingerprint, launch_fingerprint};
use crate::process::{self, Stream};
use crate::{BuildModel, maven};
@@ -266,26 +258,13 @@ pub(super) fn maven_steps(
BuildKind::Rebuild => &["clean", "test-compile"],
BuildKind::Artifacts => &["package", "-DskipTests"],
};
// The IDE's compile builds are not coverage runs. JaCoCo's offline `instrument`
// rewrites target/classes in `process-classes`; `jacoco.skip` prevents that in this
// build. It does not restore classes instrumented by an earlier build: Rebuild's
// `clean` removes those, while an incremental Build can still reuse them.
let skip_jacoco = matches!(kind, BuildKind::Build | BuildKind::Rebuild);
let tops: Vec<&maven::MavenProject> = m.projects.iter().filter(|p| !p.aggregated).collect();
if tops.is_empty() {
return Err(IdeError::not_found("no Maven project found"));
}
let step = |p: &maven::MavenProject, extra: &[String]| {
let mut s = BuildStep::new(
&p.command,
&["-B", "-Dstyle.color=never", maven::LOCAL_REPO_TRACKING_ARG],
&subdir(root, &p.dir),
Format::Maven,
);
let mut s = BuildStep::new(&p.command, &["-B", "-Dstyle.color=never"], &subdir(root, &p.dir), Format::Maven);
s.args.extend(p.options.iter().cloned());
if skip_jacoco {
s.args.push("-Djacoco.skip=true".into());
}
s.args.extend(extra.iter().cloned());
s.args.extend(goals.iter().map(|g| (*g).to_owned()));
s.env = p.env.clone();
@@ -1049,22 +1028,6 @@ mod tests {
std::fs::write(p, text).unwrap();
}
#[test]
fn maven_compile_steps_skip_jacoco_but_artifacts_do_not() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path().join("demo");
write(&root, "pom.xml", "<project><artifactId>demo</artifactId></project>");
let model = crate::load(&root, &tmp.path().join("cache"), None);
for (kind, expected) in [(BuildKind::Build, true), (BuildKind::Rebuild, true), (BuildKind::Artifacts, false)] {
let plan = plan(&root, &model, kind, None, None).unwrap();
assert_eq!(
plan.steps[0].args.iter().any(|arg| arg == "-Djacoco.skip=true"),
expected,
"{kind:?}: {plan:?}"
);
}
}
#[test]
fn plans_per_build_system() {
let tmp = tempfile::tempdir().unwrap();
@@ -1081,25 +1044,9 @@ mod tests {
assert_eq!(p.system, BuildSystem::Maven);
assert_eq!(p.title, "demo");
assert_eq!(p.steps.len(), 1);
assert_eq!(
p.steps[0].args,
["-B", "-Dstyle.color=never", maven::LOCAL_REPO_TRACKING_ARG, "-Djacoco.skip=true", "test-compile"]
);
assert_eq!(p.steps[0].args, ["-B", "-Dstyle.color=never", "test-compile"]);
let p = plan(&root, &model, BuildKind::Rebuild, Some(&root.join("core/src/main/java/A.java")), None).unwrap();
assert_eq!(
p.steps[0].args,
[
"-B",
"-Dstyle.color=never",
maven::LOCAL_REPO_TRACKING_ARG,
"-Djacoco.skip=true",
"-pl",
"core",
"-am",
"clean",
"test-compile"
]
);
assert_eq!(p.steps[0].args, ["-B", "-Dstyle.color=never", "-pl", "core", "-am", "clean", "test-compile"]);
assert_eq!(p.steps[0].cwd, root);
assert_eq!(p.title, "demo: build module 'core'");
@@ -1373,38 +1320,12 @@ mod tests {
assert_eq!(step.cwd, root);
assert_eq!(step.program, "mvnw");
assert_eq!(step.env["JAVA_HOME"], "/jdk");
assert_eq!(
step.args,
[
"-B",
"-Dstyle.color=never",
maven::LOCAL_REPO_TRACKING_ARG,
"-s",
"settings.xml",
"-Djacoco.skip=true",
"-pl",
"app",
"-am",
goal
]
);
assert_eq!(step.args, ["-B", "-Dstyle.color=never", "-s", "settings.xml", "-pl", "app", "-am", goal]);
// Rebuild cleans first, as IDEA's Rebuild does.
let plan = launch_plan(root, &model, &root.join("app"), test, BuildKind::Rebuild).unwrap();
assert_eq!(
plan.steps[0].args,
[
"-B",
"-Dstyle.color=never",
maven::LOCAL_REPO_TRACKING_ARG,
"-s",
"settings.xml",
"-Djacoco.skip=true",
"-pl",
"app",
"-am",
"clean",
goal
]
["-B", "-Dstyle.color=never", "-s", "settings.xml", "-pl", "app", "-am", "clean", goal]
);
}
}
+26 -1
View File
@@ -7,7 +7,6 @@
//! [`super::resolve`] turns them into IDE coordinates.
use ide_core::Severity;
pub(crate) use ide_core::output_encoding::strip_ansi;
use serde_json::Value;
/// Output dialect of a build step.
@@ -99,6 +98,32 @@ pub(crate) struct OutputParser {
pending: Option<Pending>,
}
/// Removes ANSI escape sequences (colour, cursor movement).
pub(crate) fn strip_ansi(s: &str) -> String {
if !s.contains('\u{1b}') {
return s.to_owned();
}
let mut out = String::with_capacity(s.len());
let mut chars = s.chars().peekable();
while let Some(c) = chars.next() {
if c != '\u{1b}' {
out.push(c);
continue;
}
if chars.peek() == Some(&'[') {
chars.next();
for c in chars.by_ref() {
if c.is_ascii_alphabetic() || c == '~' {
break;
}
}
} else {
chars.next();
}
}
out
}
fn num(s: &str) -> Option<u32> {
s.trim().parse().ok().filter(|n| *n > 0)
}
-331
View File
@@ -1,331 +0,0 @@
//! Single-file Java compilation with `javac` for the Compile File action (#2476).
//!
//! A module build (`-pl … -am`) still walks every upstream module, and packaging
//! walks the whole reactor; compiling one file only needs that file's module
//! outputs plus its dependency classpath, which is much faster to assemble.
use super::*;
use std::fs;
use std::process::Command as ProcCommand;
/// Output file of the `dependency:build-classpath` probe, relative to each module.
const CP_FILE: &str = "target/ide-compile-cp.txt";
/// A `.java` source and the module build that compiles it.
pub struct SingleFileTarget {
/// Absolute module directory.
pub module: PathBuf,
/// The source is under `src/test`: compile to test classes with the test classpath.
pub test: bool,
/// Absolute `target/classes` (or `target/test-classes`).
pub out_dir: PathBuf,
/// Reactor root building the module (owns the Maven command and options).
pub top: maven::MavenProject,
/// `-pl` value of the module relative to the reactor root; empty when the
/// module is the root itself.
pub pl: String,
/// `maven.compiler.release` (or equivalent) of the module's project.
pub java_version: Option<u32>,
}
/// Locates the linked Maven module owning a `.java` source file.
pub fn single_file_target(root: &Path, model: &BuildModel, file: &Path) -> IdeResult<SingleFileTarget> {
if file.extension().is_none_or(|ext| ext != "java") {
return Err(IdeError::invalid("only Java sources can be compiled as a single file"));
}
if !file.is_file() {
return Err(IdeError::not_found("no Java source found for single-file compilation"));
}
let Some(m) = &model.maven else {
return Err(IdeError::not_found("no linked Maven project found for module"));
};
// The innermost project whose directory contains the file owns it; the
// aggregator root (`""`) is a prefix of everything, so take the longest match.
let rel = rel_str(root, file);
let project = m
.projects
.iter()
.filter(|p| rel == p.dir || rel.starts_with(&format!("{}/", p.dir)) || p.dir.is_empty())
.max_by_key(|p| p.dir.len())
.ok_or_else(|| IdeError::not_found("no linked Maven project found for module"))?;
if project.packaging == "pom" {
// An aggregator owns no sources itself: a file that only matches it
// belongs to no linked module.
return Err(IdeError::not_found("no linked Maven project found for module"));
}
let module = subdir(root, &project.dir);
let top = maven_reactor(m, &project.dir)
.ok_or_else(|| IdeError::not_found("no linked Maven project found for module"))?;
let top_dir = subdir(root, &top.dir);
let pl = module.strip_prefix(&top_dir).map(|r| r.to_string_lossy().replace('\\', "/")).unwrap_or_default();
// Test sources see the test classpath and compile into test classes.
let parts: Vec<&str> = rel.split('/').collect();
let test = parts.windows(2).any(|w| w == ["src", "test"]);
let out_dir = module.join("target").join(if test { "test-classes" } else { "classes" });
Ok(SingleFileTarget { module, test, out_dir, top: top.clone(), pl, java_version: project.java_version })
}
/// `javac` for the module: its `JAVA_HOME`, else `PATH`.
pub fn find_javac(env: &BTreeMap<String, String>) -> String {
let home = env.get("JAVA_HOME").map(Path::new);
let exe = if cfg!(windows) { "javac.exe" } else { "javac" };
if let Some(javac) = home.map(|h| h.join("bin").join(exe)).filter(|p| p.is_file()) {
return javac.to_string_lossy().into_owned();
}
exe.to_owned()
}
/// Dependency classpath of the module: existing reactor class directories plus
/// `mvn dependency:build-classpath` (which resolves without compiling anything,
/// so `-am` stays cheap). Entries are platform-separated (`;` on Windows).
pub fn single_file_classpath(root: &Path, model: &BuildModel, target: &SingleFileTarget) -> IdeResult<String> {
let mut entries: Vec<String> = Vec::new();
// The module's own classes first: the file almost always uses classes next to it.
let own: &[&str] = if target.test { &["target/test-classes", "target/classes"] } else { &["target/classes"] };
for out in own {
let path = target.module.join(out);
if path.is_dir() {
entries.push(path.to_string_lossy().into_owned());
}
}
if let Some(m) = &model.maven {
// Upstream reactor modules first: their freshly built classes shadow
// installed snapshots of the same artifacts.
for project in maven_launch_projects(root, m, &target.module, target.test) {
let dir = subdir(root, &project.dir);
if dir == target.module {
continue;
}
for classes in ["target/classes", "target/test-classes"] {
let path = dir.join(classes);
if path.is_dir() {
entries.push(path.to_string_lossy().into_owned());
}
}
}
}
let probe = CP_FILE;
let mut args = vec!["-B", "-Dstyle.color=never", maven::LOCAL_REPO_TRACKING_ARG];
args.extend(target.top.options.iter().map(String::as_str));
if !target.pl.is_empty() {
args.extend(["-pl", target.pl.as_str(), "-am"]);
}
let output_arg = format!("-Dmdep.outputFile={probe}");
args.extend(["dependency:build-classpath", output_arg.as_str(), "-Dmdep.includeScope=test", "-q"]);
let top_dir = subdir(root, &target.top.dir);
let status = ProcCommand::new(&target.top.command)
.args(&args)
.envs(&target.top.env)
.current_dir(&top_dir)
.output()
.map_err(|e| IdeError::internal(format!("cannot resolve the compile classpath: {e}")))?;
if !status.status.success() {
let stderr = String::from_utf8_lossy(&status.stderr);
let tail: Vec<&str> = stderr.lines().filter(|l| !l.trim().is_empty()).collect();
let tail = tail.iter().rev().take(5).rev().cloned().collect::<Vec<_>>().join("\n");
return Err(IdeError::internal(format!("cannot resolve the compile classpath:\n{tail}")));
}
// With `-am` every visited module writes its own file; only this module's counts.
let cp_file = target.module.join(probe);
let content = fs::read_to_string(&cp_file).map_err(|e| {
IdeError::internal(format!("cannot resolve the compile classpath: {} ({e})", cp_file.display()))
})?;
let separator = if cfg!(windows) { ';' } else { ':' };
entries.extend(content.split([separator, '\n', '\r']).map(str::trim).filter(|e| !e.is_empty()).map(str::to_owned));
entries.dedup();
Ok(entries.join(&separator.to_string()))
}
/// The `javac` step compiling `file` into `target`'s output directory.
pub fn javac_step(target: &SingleFileTarget, file: &Path, classpath: &str, javac: &str) -> IdeResult<BuildStep> {
fs::create_dir_all(&target.out_dir)
.map_err(|e| IdeError::internal(format!("cannot create {} ({e})", target.out_dir.display())))?;
let mut step = BuildStep::new(javac, &[], &target.module, Format::Gradle);
// Raw `javac` prints `path:line: error: …`, the same form Gradle forwards.
step.args.push("-encoding".into());
step.args.push("UTF-8".into());
step.args.push("-d".into());
step.args.push(target.out_dir.to_string_lossy().into_owned());
step.args.push("-cp".into());
step.args.push(classpath.to_owned());
// Classes of the module that were never built (or are older than their source) are read
// from its sources; only `file` is written, so a test compile never puts main classes
// into test-classes.
let sources = source_roots(target);
if !sources.is_empty() {
let separator = if cfg!(windows) { ";" } else { ":" };
step.args.push("-sourcepath".into());
step.args.push(sources.iter().map(|p| p.to_string_lossy()).collect::<Vec<_>>().join(separator));
step.args.push("-implicit:none".into());
}
if let Some(version) = target.java_version {
step.args.push("--release".into());
step.args.push(version.to_string());
}
step.args.push(file.to_string_lossy().into_owned());
step.env = target.top.env.clone();
Ok(step)
}
/// The module's Java source folders that exist, generated ones included; test sources see
/// the main ones too.
fn source_roots(target: &SingleFileTarget) -> Vec<PathBuf> {
let scopes: &[(&str, &str)] = if target.test {
&[("src/test/java", "target/generated-test-sources"), ("src/main/java", "target/generated-sources")]
} else {
&[("src/main/java", "target/generated-sources")]
};
let mut roots = Vec::new();
for (src, generated) in scopes {
roots.push(target.module.join(src));
let generated = target.module.join(generated);
let mut dirs: Vec<PathBuf> =
fs::read_dir(&generated).into_iter().flatten().flatten().map(|d| d.path()).filter(|p| p.is_dir()).collect();
dirs.sort();
roots.extend(dirs);
}
roots.retain(|p| p.is_dir());
roots
}
/// Resolves the classpath and plans the `javac` invocation for one Java source.
pub fn compile_file_plan(root: &Path, model: &BuildModel, file: &Path) -> IdeResult<BuildPlan> {
let target = single_file_target(root, model, file)?;
let classpath = single_file_classpath(root, model, &target)?;
let javac = find_javac(&target.top.env);
let name = file.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
Ok(BuildPlan {
system: BuildSystem::Maven,
title: format!("Compile {name}"),
steps: vec![javac_step(&target, file, &classpath, &javac)?],
})
}
#[cfg(test)]
mod tests {
use super::*;
fn write(root: &Path, rel: &str, value: &str) {
let path = root.join(rel);
fs::create_dir_all(path.parent().unwrap()).unwrap();
fs::write(path, value).unwrap();
}
fn fixture(root: &Path) -> BuildModel {
write(
root,
"pom.xml",
r#"<project><groupId>demo</groupId><artifactId>parent</artifactId><version>1</version>
<packaging>pom</packaging><modules><module>lib</module><module>app</module></modules></project>"#,
);
write(
root,
"lib/pom.xml",
r#"<project><parent><groupId>demo</groupId><artifactId>parent</artifactId><version>1</version></parent>
<artifactId>lib</artifactId></project>"#,
);
write(
root,
"app/pom.xml",
r#"<project><parent><groupId>demo</groupId><artifactId>parent</artifactId><version>1</version></parent>
<artifactId>app</artifactId><dependencies><dependency><groupId>demo</groupId><artifactId>lib</artifactId><version>1</version></dependency></dependencies></project>"#,
);
write(root, "app/src/main/java/demo/Main.java", "class Main {}");
write(root, "app/src/test/java/demo/MainTest.java", "class MainTest {}");
write(root, "lib/target/classes/Lib.class", "built");
crate::load(root, &root.join("cache"), None)
}
#[test]
fn target_finds_the_owning_module_and_the_test_scope() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let main = single_file_target(root, &model, &root.join("app/src/main/java/demo/Main.java")).unwrap();
assert_eq!(main.module, root.join("app"));
assert!(!main.test);
assert_eq!(main.out_dir, root.join("app/target/classes"));
let test = single_file_target(root, &model, &root.join("app/src/test/java/demo/MainTest.java")).unwrap();
assert!(test.test);
assert_eq!(test.out_dir, root.join("app/target/test-classes"));
}
#[test]
fn target_rejects_non_java_missing_and_unlinked_files() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
write(root, "app/pom.xml", "<project/>");
assert!(single_file_target(root, &model, &root.join("app/pom.xml")).is_err());
assert!(single_file_target(root, &model, &root.join("app/src/main/java/demo/Missing.java")).is_err());
write(root, "other/Main.java", "class Main {}");
assert!(single_file_target(root, &model, &root.join("other/Main.java")).is_err());
}
#[test]
fn javac_prefers_the_module_jdk_and_scopes_test_sources() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let file = root.join("app/src/test/java/demo/MainTest.java");
let target = single_file_target(root, &model, &file).unwrap();
let jdk = root.join("jdk");
fs::create_dir_all(jdk.join("bin")).unwrap();
let exe = if cfg!(windows) { "javac.exe" } else { "javac" };
fs::write(jdk.join("bin").join(exe), "fake").unwrap();
let mut target = target;
target.top.env.insert("JAVA_HOME".into(), jdk.to_string_lossy().into_owned());
let step = javac_step(&target, &file, "/repo/a.jar", &find_javac(&target.top.env)).unwrap();
assert!(step.program.ends_with(exe));
assert!(target.out_dir.is_dir());
let args = step.args.join(" ");
assert!(args.contains("-encoding UTF-8"), "{args}");
assert!(args.contains("-d ") && args.contains("test-classes"), "{args}");
assert!(args.contains("-cp /repo/a.jar"), "{args}");
assert!(args.contains("MainTest.java"), "{args}");
// Test sources resolve their own and the main sources, without writing them.
let sourcepath = step.args.iter().position(|a| a == "-sourcepath").map(|i| step.args[i + 1].clone());
let roots = sourcepath.expect("a -sourcepath");
assert!(roots.contains("src/test/java") && roots.contains("src/main/java"), "{roots}");
assert!(step.args.iter().any(|a| a == "-implicit:none"), "{args}");
// No `JAVA_HOME`: plain `javac` from `PATH`.
assert_eq!(find_javac(&BTreeMap::new()), exe);
}
/// The classpath probe runs the project's Maven command with `-pl … -am`
/// `dependency:build-classpath`; the fake `mvnw` below only records its
/// arguments and writes a canned classpath file for the `-pl` module.
#[cfg(unix)]
#[test]
fn classpath_probe_writes_one_file_per_module_and_reads_the_compiled_one() {
use std::os::unix::fs::PermissionsExt;
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
fixture(root);
let script = root.join("mvnw");
fs::write(
&script,
"#!/bin/sh\npl='.'\nwhile [ $# -gt 0 ]; do\nif [ \"$1\" = \"-pl\" ]; then pl=\"$2\"; shift 2; else shift; fi\ndone\nmkdir -p \"$pl/target\"\nprintf '/repo/a.jar:/repo/b.jar' > \"$pl/target/ide-compile-cp.txt\"\n",
)
.unwrap();
fs::set_permissions(&script, fs::Permissions::from_mode(0o755)).unwrap();
let model = crate::load(root, &root.join("cache"), None);
let file = root.join("app/src/main/java/demo/Main.java");
let target = single_file_target(root, &model, &file).unwrap();
assert!(target.top.command.ends_with("mvnw"));
assert_eq!(target.pl, "app");
let cp = single_file_classpath(root, &model, &target).unwrap();
let separator = ':';
// Reactor class directories first, so fresh outputs shadow installed snapshots.
let lib = root.join("lib/target/classes").to_string_lossy().into_owned();
assert!(cp.starts_with(&lib), "{cp}");
// The module's own classes come first once they exist.
fs::create_dir_all(root.join("app/target/classes")).unwrap();
let cp = single_file_classpath(root, &model, &target).unwrap();
let own = root.join("app/target/classes").to_string_lossy().into_owned();
assert!(cp.starts_with(&format!("{own}:{lib}")), "{cp}");
assert!(cp.contains("/repo/a.jar"), "{cp}");
assert!(cp.contains("/repo/b.jar"), "{cp}");
assert!(cp.split(separator).count() >= 3, "{cp}");
}
}
-253
View File
@@ -1,253 +0,0 @@
//! Up-to-date checks for menu builds (`Build Project` / `Build Module`, #2476).
//!
//! Menu builds have no fingerprint record on the launch path, so a second
//! click always pays a full offline Maven run. This module stores one JSON
//! record per scope (project, or one module file's build) after a successful
//! build and compares it before the next build; on a match the build is
//! skipped entirely. Like launch reuse, timestamp-preserving external edits
//! are re-stamped first so Maven actually recompiles them on the next run.
use super::*;
use std::fs;
/// Record of the last successful menu build of one scope.
#[derive(serde::Serialize, serde::Deserialize)]
struct Record {
fingerprint: LaunchFingerprint,
/// Output folders of the built projects that existed after the build, relative to the
/// project. Deleting one outside the IDE (`mvn clean` in a terminal) leaves every input
/// unchanged, so a build is skipped only while they are all still there.
#[serde(default)]
outputs: Vec<PathBuf>,
}
/// The build's output folders that exist now, relative to `root`.
fn existing_outputs(root: &Path, fingerprint: &LaunchFingerprint) -> Vec<PathBuf> {
fingerprint
.project_dirs()
.flat_map(|dir| ["target/classes", "target/test-classes"].map(|out| dir.join(out)))
.filter(|rel| root.join(rel).is_dir())
.collect()
}
fn record_path(cache: &Path, root: &Path, file: Option<&Path>) -> PathBuf {
let base = cache.join("menu-build").join("records");
match file {
Some(f) => base.join(f.strip_prefix(root).unwrap_or(f)).with_extension("json"),
None => base.join("project.json"),
}
}
/// Re-stamps files whose content changed without advancing mtime (git
/// checkout, external sync): timestamp-incremental Maven would otherwise skip
/// them as "unchanged" and the rebuild would compile nothing.
fn restamp(root: &Path, edits: &[PathBuf]) {
let now = std::time::SystemTime::now();
for rel in edits {
let path = root.join(rel);
let _ = fs::File::options().write(true).open(&path).and_then(|f| f.set_modified(now)).or_else(|_| {
fs::OpenOptions::new()
.create(true)
.truncate(false)
.write(true)
.open(&path)
.and_then(|f| f.set_modified(now))
});
}
}
/// Whether the last successful menu build of this scope is still fresh: its
/// record exists and the fingerprint matches. On a mismatch caused by
/// timestamp-preserving external edits the files are re-stamped first, so the
/// rebuild Maven runs actually recompiles them.
pub fn menu_up_to_date(
cache: &Path,
root: &Path,
model: &BuildModel,
file: Option<&Path>,
cancel: &CancellationToken,
) -> bool {
let current = match menu_fingerprint(root, model, file, cancel) {
Ok(Some(current)) => current,
_ => return false,
};
let bytes = match fs::read(record_path(cache, root, file)) {
Ok(bytes) => bytes,
_ => return false,
};
let record: Record = match serde_json::from_slice(&bytes) {
Ok(record) => record,
_ => return false,
};
if record.fingerprint != current {
restamp(root, &record.fingerprint.mtime_preserved_edits(&current));
return false;
}
record.outputs.iter().all(|out| root.join(out).is_dir())
}
/// Records a successful menu build as the baseline for future up-to-date
/// checks. Failures are swallowed: a missing record only costs one Maven run.
pub fn record_menu_build(
cache: &Path,
root: &Path,
model: &BuildModel,
file: Option<&Path>,
cancel: &CancellationToken,
) {
let current = match menu_fingerprint(root, model, file, cancel) {
Ok(Some(current)) => current,
_ => return,
};
// A torn write fails to parse later and is treated as "no record", so a
// plain write is enough.
let path = record_path(cache, root, file);
if path.parent().is_some_and(|p| fs::create_dir_all(p).is_err()) {
return;
}
let outputs = existing_outputs(root, &current);
let bytes = match serde_json::to_vec(&Record { fingerprint: current, outputs }) {
Ok(bytes) => bytes,
_ => return,
};
let _ = fs::write(&path, bytes);
}
#[cfg(test)]
mod tests {
use super::*;
fn write(root: &Path, rel: &str, value: &str) {
let path = root.join(rel);
fs::create_dir_all(path.parent().unwrap()).unwrap();
fs::write(path, value).unwrap();
}
fn fixture(root: &Path) -> BuildModel {
maven_two_module(root, "int x;");
BuildModel { maven: Some(maven::load(root)), ..Default::default() }
}
fn maven_two_module(root: &Path, extra_app: &str) {
write(
root,
"pom.xml",
r#"<project><modelVersion>4.0.0</modelVersion><groupId>t</groupId><artifactId>root</artifactId><version>1</version><packaging>pom</packaging><modules><module>app</module><module>lib</module></modules></project>"#,
);
write(
root,
"app/pom.xml",
r#"<project><modelVersion>4.0.0</modelVersion><parent><groupId>t</groupId><artifactId>root</artifactId><version>1</version></parent><artifactId>app</artifactId><dependencies><dependency><groupId>t</groupId><artifactId>lib</artifactId><version>1</version></dependency></dependencies></project>"#,
);
write(
root,
"lib/pom.xml",
r#"<project><modelVersion>4.0.0</modelVersion><parent><groupId>t</groupId><artifactId>root</artifactId><version>1</version></parent><artifactId>lib</artifactId></project>"#,
);
write(root, "app/src/main/java/t/A.java", &format!("package t; public class A {{ {extra_app} }}"));
write(root, "lib/src/main/java/t/L.java", "package t; public class L {}");
}
#[test]
fn record_then_skip_without_changes() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let file = root.join("app/src/main/java/t/A.java");
let cancel = CancellationToken::new();
assert!(!menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
record_menu_build(&cache, root, &model, Some(&file), &cancel);
assert!(menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
// The project scope is independent of the module scope.
assert!(!menu_up_to_date(&cache, root, &model, None, &cancel));
record_menu_build(&cache, root, &model, None, &cancel);
assert!(menu_up_to_date(&cache, root, &model, None, &cancel));
}
#[test]
fn deleted_outputs_invalidate_the_record() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let cancel = CancellationToken::new();
for out in ["app/target/classes/t", "lib/target/classes/t", "app/target/test-classes"] {
fs::create_dir_all(root.join(out)).unwrap();
}
record_menu_build(&cache, root, &model, None, &cancel);
assert!(menu_up_to_date(&cache, root, &model, None, &cancel));
// `mvn clean` outside the IDE: no input changed, but the classes are gone.
fs::remove_dir_all(root.join("lib/target")).unwrap();
assert!(!menu_up_to_date(&cache, root, &model, None, &cancel));
fs::create_dir_all(root.join("lib/target/classes")).unwrap();
assert!(menu_up_to_date(&cache, root, &model, None, &cancel));
}
#[test]
fn edit_invalidates_and_rerecord_restores_skip() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let file = root.join("app/src/main/java/t/A.java");
let cancel = CancellationToken::new();
record_menu_build(&cache, root, &model, Some(&file), &cancel);
assert!(menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
write(root, "app/src/main/java/t/A.java", "package t; public class A { int y; }");
assert!(!menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
record_menu_build(&cache, root, &model, Some(&file), &cancel);
assert!(menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
}
#[test]
fn mtime_preserved_edit_forces_rebuild_and_restamps() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let rel = "app/src/main/java/t/A.java";
let file = root.join(rel);
let cancel = CancellationToken::new();
record_menu_build(&cache, root, &model, Some(&file), &cancel);
let before = fs::metadata(&file).unwrap().modified().unwrap();
// Same length, same mtime, different content: an external edit that
// preserved the timestamp.
write(root, rel, "package t; public class A { int z; }");
fs::File::options().write(true).open(&file).and_then(|f| f.set_modified(before)).unwrap();
assert!(!menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
assert!(fs::metadata(&file).unwrap().modified().unwrap() > before);
}
#[test]
fn corrupt_record_never_skips() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let file = root.join("app/src/main/java/t/A.java");
let cancel = CancellationToken::new();
fs::create_dir_all(cache.join("menu-build").join("records").join("app/src/main/java/t")).unwrap();
fs::write(cache.join("menu-build").join("records").join("app/src/main/java/t/A.json"), b"{not json").unwrap();
assert!(!menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
}
#[test]
fn unsupported_builds_never_skip() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
write(root, "other/Main.java", "public class Main {}");
let cache = root.join("cache");
let cancel = CancellationToken::new();
// A file outside any non-aggregator module has no fingerprint.
let stray = root.join("other/Main.java");
record_menu_build(&cache, root, &model, Some(&stray), &cancel);
assert!(!menu_up_to_date(&cache, root, &model, Some(&stray), &cancel));
}
}
+1 -22
View File
@@ -8,24 +8,6 @@ def canon = { File f -> f.canonicalPath }
// the identity path (`:lib:core`, `:buildSrc`) tells them apart and addresses their tasks
// from the root build. Internal API, hence the fallback.
def identity = { Project p -> try { p.identityPath.toString() } catch (ignored) { p.path } }
// The task copying a source set's resources and its output folder, when it changes them on the
// way (filters, `expand`, renames, includes / excludes, extra sources): jdtls copies resources as
// they are, so launches take them from Gradle. Internal API, hence the fallback.
def processedResources = { Project p, s ->
try {
def task = p.tasks.findByName(s.processResourcesTaskName)
if (task == null) return null
def specs = 0
def changes = task.mainSpec.sourcePaths.size() > 1
task.rootSpec.buildRootResolver().walk { r ->
specs++
changes |= !r.allCopyActions.isEmpty() || !r.allIncludes.isEmpty() || !r.allExcludes.isEmpty() ||
!r.allIncludeSpecs.isEmpty() || !r.allExcludeSpecs.isEmpty() || r.destPath.segments.length > 0
}
// The root and main specs exist for every task; more come from `from(…) { … }`.
(changes || specs > 2) ? [task: task.name, dir: canon(task.destinationDir)] : null
} catch (ignored) { null }
}
def describe = { Project p ->
def m = [path: identity(p), dir: canon(p.projectDir), buildDir: canon(p.layout.buildDirectory.get().asFile)]
@@ -44,10 +26,7 @@ def describe = { Project p ->
def test = s.name == 'test' || s.name == 'testFixtures' || suites.contains(s.name) ||
s.name.endsWith('Test') || s.name.endsWith('Tests') ||
(!src.isEmpty() && src.every { ideaTests.contains(it) })
def set = [name: s.name, test: test, src: src, res: res]
def processed = processedResources(p, s)
if (processed != null) set.processed = processed
set
[name: s.name, test: test, src: src, res: res]
}
}
try {
+10 -655
View File
@@ -109,11 +109,11 @@ fn strip_comments(text: &str) -> String {
/// Quoted string literals in `s`.
fn literals(s: &str) -> Vec<String> {
let mut out = Vec::new();
let mut rest = s;
while let Some(start) = rest.find(['"', '\'']) {
let Some((value, used)) = literal(&rest[start..]) else { break };
out.push(value);
rest = &rest[start + used..];
let mut chars = s.chars();
while let Some(c) = chars.next() {
if c == '"' || c == '\'' {
out.push(chars.by_ref().take_while(|&n| n != c).collect());
}
}
out
}
@@ -204,7 +204,7 @@ pub fn parse_settings(text: &str) -> Settings {
}
/// `a/b/../c` → `a/c`, without touching the filesystem.
pub(crate) fn normalize(p: &Path) -> PathBuf {
fn normalize(p: &Path) -> PathBuf {
let mut out = PathBuf::new();
for c in p.components() {
match c {
@@ -335,19 +335,6 @@ pub struct ReportedSourceSet {
/// Absolute resource folders.
#[serde(default)]
pub res: Vec<String>,
/// Set when Gradle changes the resources while copying them (filters, `expand`, renames,
/// includes / excludes, extra sources), which the Java language server's copy does not.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub processed: Option<ProcessedResources>,
}
/// The task that copies a source set's resources, and where it writes them.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct ProcessedResources {
/// Task name in its project (`processResources`).
pub task: String,
/// Absolute output folder.
pub dir: String,
}
const MODEL_LINE: &str = "@openidea-model\t";
@@ -578,12 +565,11 @@ fn is_dependency_configuration(name: &str) -> bool {
/// `(group, artifact, version)` of the external dependencies a build script declares:
/// `implementation 'g:a:v'`, `api("g:a:v:classifier@jar")`, and the map notation
/// `implementation group: 'g', name: 'a', version: 'v'` (Groovy) / `group = "g", …` (Kotlin).
/// Versions written as `${gsonVersion}` are filled in from `properties` (`ext` values, see
/// [`script_properties`]); ranges, unknown names and catalog aliases (`libs.x`) are skipped.
pub fn script_dependencies(text: &str, properties: &BTreeMap<String, String>) -> Vec<(String, String, String)> {
/// Versions with interpolations, ranges and catalog aliases (`libs.x`) are skipped.
pub fn script_dependencies(text: &str) -> Vec<(String, String, String)> {
let text = strip_comments(text);
let mut out = Vec::new();
for line in text.lines().flat_map(statements) {
for line in text.lines().flat_map(|l| l.split(['{', ';'])) {
let line = line.trim_start();
let name: String = line.chars().take_while(|c| c.is_ascii_alphanumeric() || *c == '_').collect();
if name.is_empty() || !is_dependency_configuration(&name) {
@@ -605,7 +591,6 @@ pub fn script_dependencies(text: &str, properties: &BTreeMap<String, String>) ->
}),
};
if let Some((g, a, v)) = coordinate {
let v = interpolate(&v, properties);
let plain = |s: &str| {
!s.is_empty() && s.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '-' | '_' | '+'))
};
@@ -617,354 +602,11 @@ pub fn script_dependencies(text: &str, properties: &BTreeMap<String, String>) ->
out
}
/// `line` split at `{` and `;` outside quoted strings, so a `${version}` stays in one piece.
fn statements(line: &str) -> Vec<&str> {
let mut out = Vec::new();
let mut start = 0;
let mut index = 0;
while index < line.len() {
let c = line[index..].chars().next().unwrap();
if c == '"' || c == '\'' {
index += literal(&line[index..]).map_or(line.len() - index, |(_, used)| used);
continue;
}
if matches!(c, '{' | '}' | ';' | '\n') {
out.push(&line[start..index]);
start = index + c.len_utf8();
}
index += c.len_utf8();
}
out.push(&line[start..]);
out
}
/// `apply from: 'x.gradle'` (Groovy) and `apply(from = "x.gradle.kts")` (Kotlin DSL)
/// targets, as written in the script.
pub fn applied_scripts(text: &str) -> Vec<String> {
let text = strip_comments(text);
let mut out = Vec::new();
for (index, word) in code_words(&text) {
if word != "apply" || !starts_word(&text, index) {
continue;
}
let mut rest = text[index + "apply".len()..].trim_start();
if let Some(inner) = rest.strip_prefix('(') {
rest = inner.trim_start();
}
let Some(rest) = rest.strip_prefix("from") else { continue };
let Some(rest) = rest.trim_start().strip_prefix(':').or_else(|| rest.trim_start().strip_prefix('=')) else {
continue;
};
let mut rest = rest.trim_start();
if let Some(inner) = rest.strip_prefix("file") {
let inner = inner.trim_start();
if let Some(inner) = inner.strip_prefix('(') {
rest = inner.trim_start();
}
}
if let Some((path, _)) = literal(rest) {
out.push(path);
}
}
out
}
/// Literal extra properties declared by Groovy's `ext` or Kotlin's `extra`.
pub fn script_properties(text: &str) -> BTreeMap<String, String> {
let text = strip_comments(text);
let mut out = BTreeMap::new();
for (index, word) in code_words(&text) {
if !starts_word(&text, index) {
continue;
}
let rest = text[index + word.len()..].trim_start();
if word == "val" || word == "var" {
let (name, rest) = identifier(rest);
let Some(rest) = rest.trim_start().strip_prefix("by").map(str::trim_start) else { continue };
let Some(rest) = rest.strip_prefix("extra").map(str::trim_start) else { continue };
if let Some(rest) = rest.strip_prefix('(')
&& let Some((value, used)) = literal(rest.trim_start())
&& rest.trim_start()[used..].trim_start().starts_with(')')
&& !name.is_empty()
{
out.insert(name.to_owned(), value);
}
continue;
}
if word != "ext" && word != "extra" {
continue;
}
if let Some(block) = rest.strip_prefix('{') {
if let Some(end) = matching_brace(block) {
out.extend(assignments(&block[..end]));
}
} else if let Some(rest) = rest.strip_prefix('.') {
let (name, rest) = identifier(rest);
if let Some(value) = assigned_literal(rest)
&& !name.is_empty()
{
out.insert(name.to_owned(), value);
}
} else if let Some(rest) = rest.strip_prefix('[')
&& let Some((name, used)) = literal(rest.trim_start())
&& let Some(rest) = rest.trim_start()[used..].trim_start().strip_prefix(']')
&& let Some(value) = assigned_literal(rest)
{
out.insert(name, value);
}
}
out
}
pub(crate) fn file_properties(build_dir: &Path) -> BTreeMap<String, String> {
let text = std::fs::read_to_string(build_dir.join("gradle.properties")).unwrap_or_default();
text.lines()
.filter_map(|line| {
let line = line.trim();
if line.starts_with(['#', '!']) {
return None;
}
let separator = line.find(|c: char| c == '=' || c == ':' || c.is_whitespace())?;
let (key, rest) = line.split_at(separator);
let value = rest.trim_start().trim_start_matches(['=', ':']).trim_start();
(!key.is_empty()).then(|| (key.to_owned(), value.to_owned()))
})
.collect()
}
/// The build script of a project and, recursively, the scripts it applies with `apply from`,
/// read from disk: absolute path and text. Targets that do not exist are skipped.
pub fn scripts_of(root: &Path, project_dir: &Path, build_dir: &Path, build_file: &Path) -> Vec<(PathBuf, String)> {
const MAX_SCRIPTS: usize = 256;
const MAX_DEPTH: usize = 32;
let mut out = Vec::new();
let mut seen = std::collections::HashSet::new();
let first = normalize(&root.join(build_file));
let Ok(identity) = dunce::canonicalize(&first) else { return out };
seen.insert(identity);
let mut stack = vec![(first, 0)];
while let Some((path, depth)) = stack.pop() {
let Ok(text) = std::fs::read_to_string(&path) else { continue };
if depth < MAX_DEPTH {
let mut targets = Vec::new();
for spec in applied_scripts(&text) {
if seen.len() >= MAX_SCRIPTS {
break;
}
if let Some(target) = resolve_script(&spec, &path, project_dir, build_dir)
&& let Ok(identity) = dunce::canonicalize(&target)
&& seen.insert(identity)
{
targets.push((target, depth + 1));
}
}
stack.extend(targets.into_iter().rev());
}
out.push((path, text));
}
out
}
/// Relative targets are project-scoped, with a lenient fallback beside the applying script.
fn resolve_script(spec: &str, script: &Path, project_dir: &Path, build_dir: &Path) -> Option<PathBuf> {
let spec = spec.trim();
if spec.contains("://") {
return None;
}
let properties = BTreeMap::from([
("rootDir".into(), build_dir.to_string_lossy().into_owned()),
("rootProject.projectDir".into(), build_dir.to_string_lossy().into_owned()),
("projectDir".into(), project_dir.to_string_lossy().into_owned()),
]);
let spec = interpolate(spec, &properties);
if spec.contains('$') {
return None;
}
let path = Path::new(&spec);
if path.is_absolute() {
return path.is_file().then(|| normalize(path));
}
let project_path = normalize(&project_dir.join(path));
if project_path.is_file() {
return Some(project_path);
}
let path = normalize(&script.parent()?.join(path));
path.is_file().then_some(path)
}
pub(crate) fn project_build_dir(root: &Path, model: &GradleModel, project: &GradleProject) -> PathBuf {
let dir = project.build.as_ref().and_then(|build| model.projects.iter().find(|p| &p.path == build));
root.join(dir.map_or(&model.dir, |p| &p.dir))
}
/// `2.8.9` for `${gsonVersion}` / `$gsonVersion` / `${ext.gsonVersion}`; unknown names and
/// expressions are left as written, so ranges and catalogs stay skipped.
fn interpolate(value: &str, properties: &BTreeMap<String, String>) -> String {
if !value.contains('$') {
return value.to_owned();
}
let mut out = String::with_capacity(value.len());
let mut chars = value.chars().peekable();
while let Some(c) = chars.next() {
if c != '$' {
out.push(c);
continue;
}
let braced = chars.peek() == Some(&'{');
if braced {
chars.next();
}
let mut name = String::new();
if braced {
for n in chars.by_ref() {
if n == '}' {
break;
}
name.push(n);
}
} else {
while let Some(&n) = chars.peek() {
if n.is_alphanumeric() || n == '_' || n == '.' {
name.push(n);
chars.next();
} else {
break;
}
}
}
let key = name.strip_prefix("ext.").or_else(|| name.strip_prefix("extra.")).unwrap_or(&name);
let bracket = name.strip_prefix("ext[").or_else(|| name.strip_prefix("extra[")).and_then(|rest| {
let (key, used) = literal(rest.trim_start())?;
(rest.trim_start()[used..].trim() == "]").then_some(key)
});
match properties.get(bracket.as_deref().unwrap_or(key)) {
Some(v) if !name.is_empty() => out.push_str(v),
_ => {
out.push('$');
if braced {
out.push('{');
}
out.push_str(&name);
if braced {
out.push('}');
}
}
}
}
out
}
/// Whether `index` starts a whole word in `text` (not `reapply`, `my.apply`).
fn starts_word(text: &str, index: usize) -> bool {
text[..index].chars().next_back().is_none_or(|c| !(c.is_alphanumeric() || c == '_' || c == '.'))
}
/// Identifiers outside quoted strings, with their byte positions.
fn code_words(text: &str) -> Vec<(usize, &str)> {
let mut out = Vec::new();
let mut index = 0;
while index < text.len() {
let c = text[index..].chars().next().unwrap();
if c == '"' || c == '\'' {
index += literal(&text[index..]).map_or(text.len() - index, |(_, used)| used);
} else if c.is_alphabetic() || c == '_' {
let (word, _) = identifier(&text[index..]);
out.push((index, word));
index += word.len();
} else {
index += c.len_utf8();
}
}
out
}
fn identifier(text: &str) -> (&str, &str) {
let end = text.find(|c: char| !(c.is_alphanumeric() || c == '_')).unwrap_or(text.len());
text.split_at(end)
}
fn assigned_literal(text: &str) -> Option<String> {
let rest = text.trim_start().strip_prefix('=')?.trim_start();
let (value, used) = literal(rest)?;
// Do not turn a computed value (`'1.' + suffix`) into a concrete version.
let tail = rest[used..].trim_start_matches([' ', '\t', '\r']);
(tail.is_empty() || tail.starts_with(['\n', ';', '}'])).then_some(value)
}
/// The quoted string at the start of `s`, and how many bytes it spans.
fn literal(s: &str) -> Option<(String, usize)> {
let quote = s.chars().next().filter(|c| *c == '"' || *c == '\'')?;
let rest = &s[quote.len_utf8()..];
let mut index = 0;
while index < rest.len() {
let c = rest[index..].chars().next()?;
if c == quote {
return Some((rest[..index].to_owned(), quote.len_utf8() + index + c.len_utf8()));
}
if c == '\\' {
index += c.len_utf8();
index += rest[index..].chars().next()?.len_utf8();
} else if quote == '"' && rest[index..].starts_with("${") {
index += 2;
index += matching_brace(&rest[index..])? + 1;
} else {
index += c.len_utf8();
}
}
None
}
/// Byte index of the `}` closing a `{` that was already consumed, ignoring quoted text.
fn matching_brace(text: &str) -> Option<usize> {
let mut depth = 1usize;
let mut index = 0;
while index < text.len() {
let c = text[index..].chars().next()?;
if c == '"' || c == '\'' {
index += literal(&text[index..])?.1;
continue;
}
if c == '{' {
depth += 1;
} else if c == '}' {
depth -= 1;
if depth == 0 {
return Some(index);
}
}
index += c.len_utf8();
}
None
}
/// `name = 'value'` pairs in `text`.
fn assignments(text: &str) -> Vec<(String, String)> {
statements(text)
.into_iter()
.filter_map(|statement| {
let (name, rest) = identifier(statement.trim_start());
let value = assigned_literal(rest)?;
(!name.is_empty()).then(|| (name.to_owned(), value))
})
.collect()
}
/// Every settings and build script of the model (for change detection).
pub fn build_files(root: &Path, model: &GradleModel) -> Vec<PathBuf> {
let dir = root.join(&model.dir);
let mut out: Vec<PathBuf> = SETTINGS_FILES.iter().map(|f| dir.join(f)).filter(|p| p.is_file()).collect();
out.extend(model.projects.iter().filter_map(|p| p.build_file.as_ref().map(|f| root.join(f))));
// Scripts a build script applies are build files too: editing one can change the model.
for project in &model.projects {
let Some(file) = project.build_file.as_deref() else { continue };
let build_dir = project_build_dir(root, model, project);
let properties = build_dir.join("gradle.properties");
if properties.is_file() {
out.push(properties);
}
let applied = scripts_of(root, &root.join(&project.dir), &build_dir, Path::new(file));
out.extend(applied.into_iter().skip(1).map(|(path, _)| path));
}
// The settings of each nested build (its root project's folder).
for p in model.projects.iter().filter(|p| p.build.as_deref() == Some(p.path.as_str())) {
let nested = root.join(&p.dir);
@@ -982,8 +624,6 @@ pub fn build_files(root: &Path, model: &GradleModel) -> Vec<PathBuf> {
catalogs.sort();
out.extend(catalogs);
}
out.sort();
out.dedup();
out
}
@@ -1168,7 +808,7 @@ dependencies {
classpath 'not:a:library'
}
"#;
let deps = script_dependencies(script, &BTreeMap::new());
let deps = script_dependencies(script);
let names: Vec<String> = deps.iter().map(|(g, a, v)| format!("{g}:{a}:{v}")).collect();
assert_eq!(
names,
@@ -1182,291 +822,6 @@ dependencies {
);
}
#[test]
fn applied_scripts_read_groovy_and_kotlin_forms() {
let script = r#"
apply plugin: 'java'
apply from: 'gradle/dependencies.gradle'
apply from: file('gradle/extra.gradle')
apply(from = "gradle/kotlin.gradle.kts")
apply(from = file("gradle/kotlin-file.gradle.kts"))
// apply from: 'commented.gradle'
reapply from: 'not-a-call.gradle'
"#;
assert_eq!(
applied_scripts(script),
[
"gradle/dependencies.gradle",
"gradle/extra.gradle",
"gradle/kotlin.gradle.kts",
"gradle/kotlin-file.gradle.kts"
]
);
}
#[test]
fn ext_values_fill_in_dependency_versions() {
let script = r#"
ext {
gsonVersion = '2.8.9'
junit = "4.13.2"
}
ext.logback = '1.4.14'
dependencies {
implementation "com.google.code.gson:gson:${gsonVersion}"
testImplementation "junit:junit:$junit"
implementation "ch.qos.logback:logback-classic:${ext.logback}"
implementation 'org.plain:literal:1.0'
implementation "org.x:unknown:${missing}"
}
"#;
let names: Vec<String> = script_dependencies(script, &script_properties(script))
.into_iter()
.map(|(g, a, v)| format!("{g}:{a}:{v}"))
.collect();
assert_eq!(
names,
[
"com.google.code.gson:gson:2.8.9",
"junit:junit:4.13.2",
"ch.qos.logback:logback-classic:1.4.14",
"org.plain:literal:1.0",
]
);
}
#[test]
fn extra_properties_support_brackets_and_kotlin_delegates() {
for declaration in [
"ext.x = '1.2'",
"ext { x = '1.2' }",
"ext[\"x\"] = '1.2'",
"extra[\"x\"] = \"1.2\"",
"val x by extra(\"1.2\")",
] {
let script = format!("{declaration}\ndependencies {{ implementation \"g:a:${{x}}\" }}");
assert_eq!(
script_dependencies(&script, &script_properties(&script)),
[("g".into(), "a".into(), "1.2".into())],
"{declaration}"
);
}
}
#[test]
fn parsers_ignore_apply_and_ext_inside_strings() {
let script = r#"
println "apply from: 'fake.gradle'"
println "ext.x = 'bogus'"
ext.real = '1'
"#;
assert!(applied_scripts(script).is_empty());
assert_eq!(script_properties(script), BTreeMap::from([("real".into(), "1".into())]));
}
#[test]
fn dependency_statements_preserve_interpolation_and_escaped_quotes() {
let script = r#"dependencies { implementation "g:a:${v}"; api(group = "g", name = "b", version = "${v}-RC1") }; dependencies { runtimeOnly 'g:c:2' }"#;
let properties = BTreeMap::from([("v".into(), "1.2".into())]);
assert_eq!(
script_dependencies(script, &properties),
[
("g".into(), "a".into(), "1.2".into()),
("g".into(), "b".into(), "1.2-RC1".into()),
("g".into(), "c".into(), "2".into())
]
);
assert_eq!(
statements(r#"println "escaped\"; {still quoted}"; implementation 'g:a:1'"#),
[r#"println "escaped\"; {still quoted}""#, " implementation 'g:a:1'"]
);
}
#[test]
fn dependency_versions_skip_unresolved_names_and_expressions() {
let script = r#"dependencies {
implementation "g:missing:${unknown}"
implementation "g:expression:${1 + 2}"
implementation "g:cyclic:${cycle}"
implementation 'g:range:[1,2)'
implementation 'g:dynamic:1.+'
}"#;
let properties = BTreeMap::from([("cycle".into(), "${cycle}".into())]);
assert!(script_dependencies(script, &properties).is_empty());
}
#[test]
fn extra_property_references_support_qualified_and_bracket_access() {
let script = r#"dependencies {
implementation "g:a:${ext.x}"
implementation "g:b:${extra.x}"
implementation "g:c:${ext['x']}"
implementation "g:d:${extra["x"]}"
}"#;
let properties = BTreeMap::from([("x".into(), "1".into())]);
assert_eq!(script_dependencies(script, &properties).len(), 4);
}
#[test]
fn applied_scripts_are_read_recursively_and_survive_cycles() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
std::fs::create_dir_all(root.join("gradle")).unwrap();
std::fs::write(root.join("build.gradle"), "apply from: 'gradle/deps.gradle'\n").unwrap();
std::fs::write(root.join("gradle/deps.gradle"), "apply from: '$rootDir/gradle/more.gradle'\n").unwrap();
std::fs::write(root.join("gradle/more.gradle"), "apply from: 'deps.gradle'\n").unwrap();
let scripts: Vec<String> = scripts_of(root, root, root, Path::new("build.gradle"))
.into_iter()
.map(|(path, _)| path.strip_prefix(root).unwrap().to_string_lossy().replace('\\', "/"))
.collect();
assert_eq!(scripts, ["build.gradle", "gradle/deps.gradle", "gradle/more.gradle"]);
assert_eq!(build_files(root, &load(root).unwrap()).len(), 3);
}
#[test]
fn applied_paths_prefer_the_project_directory() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
std::fs::create_dir_all(root.join("gradle")).unwrap();
std::fs::write(root.join("build.gradle"), "apply from: 'gradle/deps.gradle'").unwrap();
std::fs::write(root.join("gradle/deps.gradle"), "apply from: 'versions.gradle'").unwrap();
std::fs::write(root.join("versions.gradle"), "right").unwrap();
std::fs::write(root.join("gradle/versions.gradle"), "wrong").unwrap();
let scripts = scripts_of(root, root, root, Path::new("build.gradle"));
assert_eq!(scripts.len(), 3);
assert_eq!(scripts[2].1, "right");
}
#[test]
fn nested_applied_paths_are_relative_to_the_project_directory() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
std::fs::create_dir_all(root.join("gradle")).unwrap();
std::fs::write(root.join("build.gradle"), "apply from: 'gradle/a.gradle'").unwrap();
std::fs::write(root.join("gradle/a.gradle"), "apply from: 'gradle/b.gradle'").unwrap();
std::fs::write(root.join("gradle/b.gradle"), "").unwrap();
let scripts = scripts_of(root, root, root, Path::new("build.gradle"));
assert_eq!(scripts.len(), 3);
assert_eq!(scripts[2].0, root.join("gradle/b.gradle"));
}
#[test]
fn applied_paths_fall_back_to_the_applying_script_directory() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
std::fs::create_dir_all(root.join("gradle")).unwrap();
std::fs::write(root.join("build.gradle"), "apply from: 'gradle/a.gradle'").unwrap();
std::fs::write(root.join("gradle/a.gradle"), "apply from: 'c.gradle'").unwrap();
std::fs::write(root.join("gradle/c.gradle"), "").unwrap();
let scripts = scripts_of(root, root, root, Path::new("build.gradle"));
assert_eq!(scripts.len(), 3);
assert_eq!(scripts[2].0, root.join("gradle/c.gradle"));
}
#[test]
fn applied_paths_resolve_gradle_directory_variables() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
let project = root.join("app");
let script = project.join("gradle/deps.gradle");
std::fs::create_dir_all(script.parent().unwrap()).unwrap();
std::fs::write(root.join("shared.gradle"), "").unwrap();
std::fs::write(project.join("local.gradle"), "").unwrap();
for prefix in ["$rootDir", "${rootDir}", "$rootProject.projectDir", "${rootProject.projectDir}"] {
assert_eq!(
resolve_script(&format!("{prefix}/shared.gradle"), &script, &project, root),
Some(root.join("shared.gradle")),
"{prefix}"
);
}
for prefix in ["$projectDir", "${projectDir}"] {
assert_eq!(
resolve_script(&format!("{prefix}/local.gradle"), &script, &project, root),
Some(project.join("local.gradle")),
"{prefix}"
);
}
assert_eq!(
resolve_script(root.join("shared.gradle").to_str().unwrap(), &script, &project, root),
Some(root.join("shared.gradle"))
);
}
#[test]
fn applied_scripts_skip_urls_and_missing_files() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
// Even a local path resembling a URL must not be read.
if !cfg!(windows) {
for scheme in ["http", "https"] {
let local = root.join(format!("{scheme}:/example.com/deps.gradle"));
std::fs::create_dir_all(local.parent().unwrap()).unwrap();
std::fs::write(local, "implementation 'g:remote:1'").unwrap();
}
}
std::fs::write(root.join("build.gradle"), "apply from: 'http://example.com/deps.gradle'\napply(from = \"https://example.com/deps.gradle\")\napply from: 'missing.gradle'").unwrap();
assert_eq!(scripts_of(root, root, root, Path::new("build.gradle")).len(), 1);
}
#[test]
fn applied_scripts_deduplicate_self_application_and_path_aliases() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
std::fs::create_dir_all(root.join("gradle")).unwrap();
std::fs::write(
root.join("build.gradle"),
"apply from: 'build.gradle'\napply from: 'gradle/../build.gradle'\napply from: 'gradle/a.gradle'",
)
.unwrap();
std::fs::write(root.join("gradle/a.gradle"), "apply from: '../build.gradle'").unwrap();
assert_eq!(scripts_of(root, root, root, Path::new("build.gradle")).len(), 2);
}
#[cfg(unix)]
#[test]
fn applied_scripts_deduplicate_symlink_cycles() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
std::fs::write(root.join("build.gradle"), "apply from: 'alias/build.gradle'").unwrap();
std::os::unix::fs::symlink(root, root.join("alias")).unwrap();
assert_eq!(scripts_of(root, root, root, Path::new("build.gradle")).len(), 1);
}
#[test]
fn applied_script_traversal_bounds_depth_and_total_files() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
for i in 0..300 {
std::fs::write(root.join(format!("{i}.gradle")), format!("apply from: '{}.gradle'", i + 1)).unwrap();
}
let chain = scripts_of(root, root, root, Path::new("0.gradle"));
assert_eq!(chain.len(), 33);
let fanout: String = (0..300).map(|i| format!("apply from: '{i}.gradle'\n")).collect();
std::fs::write(root.join("build.gradle"), fanout).unwrap();
assert_eq!(scripts_of(root, root, root, Path::new("build.gradle")).len(), 256);
}
#[test]
fn applied_scripts_use_their_composite_build_root() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
std::fs::create_dir_all(root.join("included/app")).unwrap();
std::fs::write(root.join("build.gradle"), "").unwrap();
std::fs::write(root.join("included/build.gradle"), "").unwrap();
std::fs::write(root.join("included/app/build.gradle"), "apply from: \"${rootDir}/deps.gradle\"").unwrap();
std::fs::write(root.join("included/deps.gradle"), "implementation 'g:included:1'").unwrap();
let mut model = load(root).unwrap();
for (path, dir) in [(":included", "included"), (":included:app", "included/app")] {
let mut project = project_at(root, path.into(), path.into(), &root.join(dir));
project.build = Some(":included".into());
model.projects.push(project);
}
assert!(build_files(root, &model).contains(&root.join("included/deps.gradle")));
let model = crate::BuildModel { maven: None, gradle: vec![model] };
assert_eq!(crate::dependency_coordinates(root, &model)[0].artifact_id, "included");
}
use super::*;
#[test]
+1 -12
View File
@@ -528,18 +528,7 @@ mod tests {
for (step, name) in plan.steps.iter().zip(["java", "kotlin"]) {
assert_eq!(step.cwd, root.join(name));
assert_eq!(step.program, format!("{name}-mvnw"));
assert_eq!(
step.args,
[
"-B",
"-Dstyle.color=never",
crate::maven::LOCAL_REPO_TRACKING_ARG,
"-o",
"-Peditor",
"-Djacoco.skip=true",
"test-compile"
]
);
assert_eq!(step.args, ["-B", "-Dstyle.color=never", "-o", "-Peditor", "test-compile"]);
assert_eq!(step.env.get("JAVA_HOME").map(String::as_str), Some("/jdk"));
}
}
+6 -65
View File
@@ -122,14 +122,6 @@ pub fn gradle_external_roots(
gradle::external_roots(root, dirs.iter().map(String::as_str))
}
/// Directories of Maven POMs linked from outside `root`, outermost only.
/// Reads just those explicit paths (no POM parse and no directory walk).
pub fn maven_external_roots(root: &Path, linked: &[String]) -> Vec<PathBuf> {
let dirs = maven::external_pom_dirs(root, linked);
let owned: Vec<String> = dirs.iter().map(|dir| dir.to_string_lossy().into_owned()).collect();
gradle::external_roots(root, owned.iter().map(String::as_str))
}
/// Source, resource and build folders of the last Gradle sync, absolute.
pub fn gradle_source_roots(
root: &Path,
@@ -155,8 +147,7 @@ pub struct DependencyCoordinate {
/// The external libraries of `model`, sorted and deduplicated: Maven dependencies
/// (with the transitive ones the last sync read), except the build's own modules,
/// `pom` imports and `system` scope; dependencies declared in the Gradle build scripts,
/// including the scripts they pull in with `apply from`.
/// `pom` imports and `system` scope; dependencies declared in the Gradle build scripts.
pub fn dependency_coordinates(root: &Path, model: &BuildModel) -> Vec<DependencyCoordinate> {
let mut out = Vec::new();
if let Some(m) = &model.maven {
@@ -186,26 +177,11 @@ pub fn dependency_coordinates(root: &Path, model: &BuildModel) -> Vec<Dependency
}
}
for g in &model.gradle {
for p in &g.projects {
let Some(build_file) = p.build_file.as_deref() else { continue };
let build_dir = gradle::project_build_dir(root, g, p);
let scripts = gradle::scripts_of(root, &root.join(&p.dir), &build_dir, Path::new(build_file));
// The script's own `ext` values win over the scripts it applies.
let mut properties = std::collections::BTreeMap::new();
for (_, text) in scripts.iter().rev() {
properties.extend(gradle::script_properties(text));
}
properties.extend(gradle::file_properties(&build_dir));
for (_, text) in &scripts {
out.extend(gradle::script_dependencies(text, &properties).into_iter().map(
|(group_id, artifact_id, version)| DependencyCoordinate {
gradle: true,
group_id,
artifact_id,
version,
},
));
}
for script in g.projects.iter().filter_map(|p| p.build_file.as_ref()) {
let Ok(text) = std::fs::read_to_string(root.join(script)) else { continue };
out.extend(gradle::script_dependencies(&text).into_iter().map(|(group_id, artifact_id, version)| {
DependencyCoordinate { gradle: true, group_id, artifact_id, version }
}));
}
}
out.sort();
@@ -336,39 +312,4 @@ mod tests {
.collect();
assert_eq!(names, ["M org.b:y:2", "M org.c:z:3", "G org.a:x:1.0"]);
}
#[test]
fn dependency_coordinates_read_scripts_brought_in_by_apply_from() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
std::fs::create_dir_all(root.join("gradle")).unwrap();
std::fs::write(root.join("build.gradle"), "apply from: 'gradle/dependencies.gradle'\n").unwrap();
std::fs::write(
root.join("gradle/dependencies.gradle"),
"ext { gson = '2.8.9' }\ndependencies { implementation \"com.google.code.gson:gson:${gson}\" }\n",
)
.unwrap();
let model = BuildModel { maven: None, gradle: gradle::load(root).into_iter().collect() };
let names: Vec<String> = dependency_coordinates(root, &model)
.into_iter()
.map(|c| format!("{}:{}:{}", c.group_id, c.artifact_id, c.version))
.collect();
assert_eq!(names, ["com.google.code.gson:gson:2.8.9"]);
}
#[test]
fn gradle_properties_override_script_extra_values() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
std::fs::write(root.join("build.gradle"), "ext.v = 'wrong'\napply from: 'deps.gradle'").unwrap();
std::fs::write(
root.join("deps.gradle"),
"ext.v = 'alsoWrong'\nimplementation \"g:a:${v}\"\nimplementation \"g:b:${onlyInProperties}\"",
)
.unwrap();
std::fs::write(root.join("gradle.properties"), "# versions\nv = 1.2\nonlyInProperties: 2.3\n").unwrap();
let model = BuildModel { maven: None, gradle: gradle::load(root).into_iter().collect() };
let versions: Vec<_> = dependency_coordinates(root, &model).into_iter().map(|c| c.version).collect();
assert_eq!(versions, ["1.2", "2.3"]);
}
}
+47 -512
View File
@@ -16,17 +16,6 @@ use std::io::Read;
use std::path::{Path, PathBuf};
use std::sync::{Mutex, OnceLock};
pub mod completion;
pub mod sources;
/// Lets the IDE's own Maven runs (Reload, Build) use what the local repository already
/// has. Maven 3.9 re-resolves an artifact whose recorded origin repository (`_remote.repositories`)
/// isn't declared in the build, such as a vendor jar downloaded through another project or
/// installed by hand, and fails when no declared repository has it. Tracking under another
/// file name makes such artifacts count as locally installed. Maven goals the user runs keep
/// plain Maven's behaviour.
pub const LOCAL_REPO_TRACKING_ARG: &str = "-Daether.enhancedLocalRepository.trackingFilename=openidea.repositories";
/// Phases shown under "Lifecycle" (IDEA's default list).
pub const LIFECYCLE: &[&str] =
&["clean", "validate", "compile", "test", "package", "verify", "install", "site", "deploy"];
@@ -107,9 +96,7 @@ pub struct MavenProject {
pub profiles: Vec<String>,
/// Java feature version the project compiles for (`maven.compiler.release`, `java.version`…).
pub java_version: Option<u32>,
/// Maven launcher: the nearest `mvnw` (absolute), else `mvn`. Inside the opened
/// project the search stops at its root; outside it, the search stops before
/// that root so a linked project keeps its own wrapper.
/// Maven launcher: the nearest `mvnw` up to the project root (absolute), else `mvn`.
pub command: String,
/// Options before the goals (`-s`, `-Dmaven.repo.local=`); filled in by the shell from settings.
pub options: Vec<String>,
@@ -300,7 +287,7 @@ fn read_plugin(e: &Element) -> Option<RawPlugin> {
})
}
pub(crate) fn read_properties(e: Option<&Element>) -> Vec<(String, String)> {
fn read_properties(e: Option<&Element>) -> Vec<(String, String)> {
e.map(|p| p.children.iter().map(|c| (c.name.clone(), c.text.trim().to_owned())).collect()).unwrap_or_default()
}
@@ -510,7 +497,7 @@ pub fn user_settings_file(root: &Path, settings: &ide_core::settings::MavenSetti
/// IDEA lists them): checked by default when `<activeProfiles>` names them or they are
/// `activeByDefault`.
pub fn settings_profiles(settings: &Path) -> Vec<MavenProfile> {
let Some(root) = xml::read_text(settings).as_deref().and_then(xml::parse) else {
let Some(root) = std::fs::read_to_string(settings).ok().and_then(|text| xml::parse(&text)) else {
return Vec::new();
};
let active: Vec<String> =
@@ -530,7 +517,7 @@ pub fn settings_profiles(settings: &Path) -> Vec<MavenProfile> {
/// `<localRepository>` of a settings file, when it names one: the repository a build
/// with `-s <file>` resolves into, which is not necessarily the user's default.
pub fn local_repository_of(settings: &Path) -> Option<PathBuf> {
let text = xml::read_text(settings)?;
let text = std::fs::read_to_string(settings).ok()?;
let root = xml::parse(&text)?;
let repo = root.text_of("localRepository")?.trim().to_owned();
if repo.is_empty() {
@@ -556,7 +543,7 @@ impl Resolver {
fn pom(&mut self, path: &Path) -> Option<RawPom> {
self.poms
.entry(path.to_path_buf())
.or_insert_with(|| xml::read_text(path).as_deref().and_then(parse_pom))
.or_insert_with(|| std::fs::read_to_string(path).ok().and_then(|t| parse_pom(&t)))
.clone()
}
@@ -747,7 +734,7 @@ pub(crate) fn launch_inputs(
return None;
}
out.push(path.clone());
let text = xml::read_text(&path)?;
let text = std::fs::read_to_string(&path).ok()?;
roots(&xml::parse(&text)?, "", &dir, &mut out)?;
let raw = resolver.pom(&path)?;
let Some(parent) = raw.parent else { break };
@@ -1035,6 +1022,10 @@ pub(crate) fn plugin(
const MAX_PROJECTS: usize = 1000;
fn rel_path(root: &Path, p: &Path) -> String {
p.strip_prefix(root).unwrap_or(p).to_string_lossy().replace('\\', "/")
}
/// The POMs that are not modules of another one: the root `pom.xml`, else the
/// shallowest `pom.xml` of each branch (a folder holding several builds). An
/// aggregate POM directly inside `src/` counts too (issue #1001: `src/pom.xml`
@@ -1048,29 +1039,18 @@ pub fn is_maven_project(root: &Path) -> bool {
!top_level_poms(root).is_empty()
}
/// The `mvnw` nearest to `dir`. Inside `root` the walk stops at `root`. Outside it,
/// the walk stops before entering `root` or any ancestor of `root`: a sibling project
/// keeps its own wrapper, and the search does not continue to the filesystem root.
/// The `mvnw` nearest to `dir`, looking up to `root`.
fn wrapper(root: &Path, dir: &Path) -> String {
let name = if cfg!(windows) { "mvnw.cmd" } else { "mvnw" };
let root = dunce::canonicalize(root).unwrap_or_else(|_| root.to_path_buf());
let dir = dunce::canonicalize(dir).unwrap_or_else(|_| dir.to_path_buf());
let external = !dir.starts_with(&root);
let mut cur = Some(dir.as_path());
let mut cur = Some(dir);
while let Some(d) = cur {
if d.join(name).is_file() {
return d.join(name).to_string_lossy().into_owned();
}
if !external && d == root {
if d == root {
break;
}
let Some(parent) = d.parent() else { break };
// `parent` inside the opened project, or an ancestor of it (the common parent,
// then the filesystem root). Stop before inspecting those.
if external && (parent.starts_with(&root) || root.starts_with(parent)) {
break;
}
cur = Some(parent);
cur = d.parent();
}
if cfg!(windows) { "mvn.cmd".into() } else { "mvn".into() }
}
@@ -1114,118 +1094,24 @@ pub(crate) fn load_with(root: &Path, local_repo: Option<PathBuf>, profiles: &[St
load_with_linked(root, local_repo, profiles, &[])
}
/// The `pom.xml` files `linked` names. Each entry is used as written (project-relative,
/// `../sibling/pom.xml`, or absolute) and canonicalized; a path that is not an existing
/// `pom.xml` is skipped. Nothing here lists a directory or walks parents looking for builds.
/// The POMs of `linked` (project-relative, from Settings) that exist inside `root`.
fn linked_poms(root: &Path, linked: &[String]) -> Vec<PathBuf> {
let base = dunce::canonicalize(root).unwrap_or_else(|_| root.to_path_buf());
let mut out: Vec<PathBuf> = Vec::new();
for pom in linked {
let Some(path) = dunce::canonicalize(base.join(pom.trim())).ok() else { continue };
if is_pom_file(&path) && !out.contains(&path) {
let pom = normalized_pom(pom);
if pom.is_empty() || pom.starts_with("../") || pom == ".." {
continue;
}
let path = root.join(&pom);
let inside = dunce::canonicalize(&path).is_ok_and(|p| p.starts_with(&base));
if path.is_file() && inside && !out.contains(&path) {
out.push(path);
}
}
out
}
fn is_pom_file(path: &Path) -> bool {
path.is_file()
&& path
.file_name()
.and_then(|name| name.to_str())
.is_some_and(|name| if cfg!(windows) { name.eq_ignore_ascii_case("pom.xml") } else { name == "pom.xml" })
}
/// Identity of `path` (both canonical) within `root`: project-relative inside it,
/// otherwise the absolute path with forward slashes.
fn project_rel(root: &Path, path: &Path) -> String {
path.strip_prefix(root).unwrap_or(path).to_string_lossy().replace('\\', "/")
}
/// Canonical `pom.xml` paths named by `linked`, keyed by the spelling to store and to
/// show so unlink removes the same entry settings hold. The first spelling wins.
fn linked_identities(root: &Path, linked: &[String]) -> HashMap<PathBuf, String> {
let mut out = HashMap::new();
for entry in linked {
let entry = entry.trim();
if entry.is_empty() {
continue;
}
let Some(path) = dunce::canonicalize(root.join(entry)).ok() else { continue };
if is_pom_file(&path) {
out.entry(path).or_insert_with(|| entry.replace('\\', "/"));
}
}
out
}
/// Stable, deduplicated form of `poms` for `languages.java.maven.linkedProjects`.
/// An existing `pom.xml` inside `root` is project-relative; one outside it is the
/// canonical absolute path. Spellings of the same file (`../B/pom.xml`, an absolute
/// path, a symlink) collapse to one entry. A path that exists and is not a `pom.xml`
/// is dropped. A `pom.xml` that cannot be resolved yet is kept, so a link is not
/// discarded while its volume is offline. No directory is scanned.
pub fn normalize_linked_poms(root: &Path, poms: &[String]) -> Vec<String> {
let Some(root) = dunce::canonicalize(root).ok() else {
let mut out = Vec::new();
for pom in poms {
let pom = normalized_pom(pom);
if !pom.is_empty() && pom_name_ok(&pom) && !out.contains(&pom) {
out.push(pom);
}
}
return out;
};
let mut out = Vec::new();
let mut seen = HashSet::new();
for pom in poms {
let pom = pom.trim();
if pom.is_empty() {
continue;
}
let key = match dunce::canonicalize(root.join(pom)) {
Ok(path) if is_pom_file(&path) => project_rel(&root, &path),
Ok(_) => continue,
Err(_) => {
let fallback = normalized_pom(pom);
let joined = root.join(pom);
if !pom_name_ok(&fallback) || (joined.exists() && !is_pom_file(&joined)) {
continue;
}
fallback
}
};
if key.is_empty() || !seen.insert(key.clone()) {
continue;
}
out.push(key);
}
out
}
fn pom_name_ok(path: &str) -> bool {
let name = path.rsplit('/').next().unwrap_or(path);
if cfg!(windows) { name.eq_ignore_ascii_case("pom.xml") } else { name == "pom.xml" }
}
/// Directories of linked `pom.xml` files that sit outside `root` (the opened project
/// stays the root). Callers keep the outermost directory only. Modules a POM declares
/// outside its own directory are not included: a linked file cannot widen the indexed
/// tree to an arbitrary path.
pub fn external_pom_dirs(root: &Path, linked: &[String]) -> Vec<PathBuf> {
let base = dunce::canonicalize(root).unwrap_or_else(|_| root.to_path_buf());
let mut out = Vec::new();
for pom in linked_poms(root, linked) {
let Some(dir) = pom.parent() else { continue };
if dir.starts_with(&base) || out.iter().any(|have| have == dir) {
continue;
}
out.push(dir.to_path_buf());
}
out
}
/// Whether `root` holds a Maven build or the settings link one inside it.
pub fn has_projects(root: &Path, settings: &ide_core::settings::MavenSettings) -> bool {
is_maven_project(root) || !linked_poms(root, &settings.linked_projects).is_empty()
@@ -1234,9 +1120,7 @@ pub fn has_projects(root: &Path, settings: &ide_core::settings::MavenSettings) -
/// The model with the chosen `profiles` (`-P` entries; `!id` deactivates: their modules,
/// dependencies and properties are part of the model, those turned off are not) and the
/// `linked` POMs (Add Maven Projects): builds that no reactor lists, shown as independent
/// top-level projects, including a `pom.xml` outside the opened root. A linked POM that
/// turns out to be a module of the root reactor stays that module. The same canonical file
/// is loaded once, so a cycle or a second spelling does not duplicate it or edit either POM.
/// top-level projects. A linked POM that turns out to be a module of the root reactor stays that module.
pub fn load_with_linked(
root: &Path,
local_repo: Option<PathBuf>,
@@ -1245,10 +1129,7 @@ pub fn load_with_linked(
) -> MavenModel {
let mut resolver = Resolver::new(local_repo, profiles);
let mut model = MavenModel::default();
// Canonical POM → its project, for a module listed again by a later aggregator.
let mut seen: HashMap<PathBuf, Option<usize>> = HashMap::new();
let canonical_root = dunce::canonicalize(root).unwrap_or_else(|_| root.to_path_buf());
let linked_ids = linked_identities(&canonical_root, linked);
let mut seen: HashSet<PathBuf> = HashSet::new();
let mut tops = top_level_poms(root);
for pom in linked_poms(root, linked) {
if !tops.contains(&pom) {
@@ -1256,31 +1137,17 @@ pub fn load_with_linked(
}
}
// (pom, aggregated); the root reactor and its modules come before linked POMs.
// (pom, listed by): a module carries the top-level project its chain started from and
// whether the aggregator listing it lies inside the opened root.
let mut stack: Vec<(PathBuf, Option<(usize, bool)>)> = tops.into_iter().rev().map(|p| (p, None)).collect();
while let Some((path, listed_by)) = stack.pop() {
let aggregated = listed_by.is_some();
let mut stack: Vec<(PathBuf, bool)> = tops.into_iter().rev().map(|p| (p, false)).collect();
while let Some((path, aggregated)) = stack.pop() {
if model.projects.len() >= MAX_PROJECTS {
break;
}
let canonical = dunce::canonicalize(&path).unwrap_or_else(|_| path.clone());
if let Some(known) = seen.get(&canonical) {
// Found as a top-level POM first, then listed by an aggregator elsewhere
// (`<module>../</module>` of `parent/pom.xml`): it is that aggregator's module.
// Not when the chain leads back to where it started (a cycle), and not for an
// aggregator outside the opened root: a linked sibling does not take it over.
if let (Some(known), Some((origin, true))) = (*known, listed_by)
&& known != origin
{
model.projects[known].aggregated = true;
}
if !seen.insert(canonical) {
continue;
}
seen.insert(canonical.clone(), None);
let Some(pom) = resolver.pom(&path) else { continue };
let dir = path.parent().unwrap_or(root).to_path_buf();
let dir_canon = dunce::canonicalize(&dir).unwrap_or_else(|_| dir.clone());
let ctx = resolver.context(&path, 0);
let props = resolver.properties_for(&pom, &ctx, &dir);
let i = |s: &str| interpolate(s, &props);
@@ -1291,7 +1158,7 @@ pub fn load_with_linked(
}
let mut modules: Vec<PathBuf> = Vec::new();
for m in module_names {
let mut mp = crate::gradle::normalize(&dir.join(i(&m)));
let mut mp = dir.join(i(&m));
if mp.is_dir() || mp.extension().is_none_or(|e| e != "xml") {
mp.push("pom.xml");
}
@@ -1355,39 +1222,28 @@ pub fn load_with_linked(
}
}
let artifact_id = i(&pom.artifact_id);
let module_ids: Vec<String> = modules
.iter()
.map(|m| {
let canonical = dunce::canonicalize(m).unwrap_or_else(|_| m.clone());
linked_ids.get(&canonical).cloned().unwrap_or_else(|| project_rel(&canonical_root, &canonical))
})
.collect();
model.projects.push(MavenProject {
project_root: None,
pom: linked_ids.get(&canonical).cloned().unwrap_or_else(|| project_rel(&canonical_root, &canonical)),
dir: project_rel(&canonical_root, &dir_canon),
pom: rel_path(root, &path),
dir: rel_path(root, &dir),
group_id: i(props.get("project.groupId").map(String::as_str).unwrap_or_default()),
version: i(props.get("project.version").map(String::as_str).unwrap_or_default()),
display_name: pom.name.as_deref().map(i).filter(|n| !n.is_empty()).unwrap_or_else(|| artifact_id.clone()),
artifact_id,
packaging,
parent: pom.parent.as_ref().map(|p| format!("{}:{}:{}", p.group_id, p.artifact_id, i(&p.version))),
modules: module_ids,
modules: modules.iter().map(|m| rel_path(root, m)).collect(),
aggregated,
plugins,
dependencies,
profiles: ctx.profiles.iter().map(|(id, _)| id.clone()).collect(),
java_version: jv,
command: wrapper(&canonical_root, &dir_canon),
command: wrapper(root, &dir),
options: Vec::new(),
env: BTreeMap::new(),
});
let index = model.projects.len() - 1;
seen.insert(canonical, Some(index));
let origin = listed_by.map_or(index, |(origin, _)| origin);
let inside = dir_canon.starts_with(&canonical_root);
for m in modules.into_iter().rev() {
stack.push((m, Some((origin, inside))));
stack.push((m, true));
}
}
model
@@ -1475,27 +1331,17 @@ impl UnlinkedProjects {
fn normalized_pom(pom: &str) -> String {
let pom = pom.replace('\\', "/");
let unc = pom.starts_with("//");
let absolute = pom.starts_with('/');
let mut parts: Vec<&str> = Vec::new();
let mut parts = Vec::new();
for part in pom.split('/') {
match part {
"" | "." => {}
// Keep a drive letter (`C:`) and a leading `..` that has nothing to climb.
".." if parts.last().is_some_and(|p| *p != ".." && !p.ends_with(':')) => {
".." if parts.last().is_some_and(|p| *p != "..") => {
parts.pop();
}
part => parts.push(part),
}
}
let joined = parts.join("/");
if unc {
format!("//{joined}")
} else if absolute {
format!("/{joined}")
} else {
joined
}
parts.join("/")
}
/// Every POM of the model (for change detection).
@@ -1680,15 +1526,6 @@ pub enum PomGotoKind {
ManagedDependency,
/// A `<pluginManagement>` entry providing the plugin's version.
ManagedPlugin,
/// The POM of the artifact itself: a project module's, else the local repository's.
ArtifactPom,
}
/// A Maven project of the workspace, for going to a dependency on it.
pub struct ModulePom {
pub group_id: String,
pub artifact_id: String,
pub pom: PathBuf,
}
/// Byte offset of `pos` (0-based line / UTF-16 column), clamped to `text`.
@@ -1739,7 +1576,7 @@ fn property_ref_at(text: &str, at: usize) -> Option<(&str, usize, usize)> {
}
/// The elements containing `at`, from the root down to the innermost one.
pub(crate) fn element_path(root: &Element, at: usize) -> Vec<&Element> {
fn element_path(root: &Element, at: usize) -> Vec<&Element> {
let mut path = vec![root];
loop {
let Some(next) = path.last().unwrap().children.iter().find(|c| c.start <= at && at < c.end) else {
@@ -1768,20 +1605,6 @@ fn coordinates(e: &Element, props: &HashMap<String, String>) -> Option<(String,
/// the local repository when it is not in the project). `None` when there is no
/// resolvable declaration here.
pub fn goto_declaration(text: &str, path: &Path, pos: Pos, local_repo: Option<PathBuf>) -> Option<PomGoto> {
goto_declaration_in(text, path, pos, local_repo, Vec::new)
}
/// [`goto_declaration`], where a `<parent>` and a dependency or plugin whose version no
/// management entry provides (or a management entry itself) go to the artifact's own
/// POM: the project module among `modules` with its coordinates, else the local
/// repository's POM of its version.
pub fn goto_declaration_in(
text: &str,
path: &Path,
pos: Pos,
local_repo: Option<PathBuf>,
modules: impl FnOnce() -> Vec<ModulePom>,
) -> Option<PomGoto> {
let root = xml::parse(text).filter(|e| e.name == "project")?;
let at = offset_at(text, pos);
if let Some((name, _, _)) = property_ref_at(text, at)
@@ -1792,56 +1615,15 @@ pub fn goto_declaration_in(
// A `<dependency>` / `<plugin>` in a declaration position: its version may come
// from a management entry of this POM or an ancestor's.
let elements = element_path(&root, at);
if elements.get(1).is_some_and(|e| e.name == "parent") {
let chain = parent_chain(path, text, local_repo);
let (pom, parent_text) = chain.get(1)?;
let label = elements[1].text_of("artifactId").unwrap_or_default();
return Some(pom_goto(pom.clone(), parent_text, label));
}
let e = elements.iter().rev().find(|e| e.name == "dependency" || e.name == "plugin")?;
// A management entry declares the version: it goes to the artifact (a BOM import, say).
if !elements.iter().any(|a| a.name == "dependencyManagement" || a.name == "pluginManagement") {
let kind = match e.name.as_str() {
"dependency" => PomGotoKind::ManagedDependency,
_ => PomGotoKind::ManagedPlugin,
};
if let Some(found) = managed_target(text, path, e, kind, local_repo.clone()) {
return Some(found);
}
if elements.iter().any(|a| a.name == "dependencyManagement" || a.name == "pluginManagement") {
return None; // Management entries are declarations; there is nothing to go to.
}
artifact_target(text, path, e, local_repo, modules)
}
/// The artifact's own POM for a dependency or plugin element `e`.
fn artifact_target(
text: &str,
path: &Path,
e: &Element,
local_repo: Option<PathBuf>,
modules: impl FnOnce() -> Vec<ModulePom>,
) -> Option<PomGoto> {
let mut resolver = Resolver::new(local_repo.clone(), &[]);
let chain = parent_chain(path, text, local_repo);
let props = properties_at(&mut resolver, &chain, 0);
let (group, artifact) = coordinates(e, &props)?;
let label = format!("{group}:{artifact}");
if let Some(module) = modules().into_iter().find(|m| m.group_id == group && m.artifact_id == artifact) {
let module_text = xml::read_text(&module.pom)?;
return Some(pom_goto(module.pom, &module_text, label));
}
let version = interpolate(&e.text_of("version")?, &props);
let pom = resolver.repo_pom(&group, &artifact, &version)?;
let pom_text = xml::read_text(&pom)?;
Some(pom_goto(pom, &pom_text, label))
}
/// The POM at `path` as a target, at its `<artifactId>`.
fn pom_goto(path: PathBuf, text: &str, label: String) -> PomGoto {
let range = xml::parse(text)
.filter(|e| e.name == "project")
.and_then(|root| root.child("artifactId").map(|a| range_of(text, a)))
.unwrap_or_default();
PomGoto { path, range, kind: PomGotoKind::ArtifactPom, label }
let kind = match e.name.as_str() {
"dependency" => PomGotoKind::ManagedDependency,
_ => PomGotoKind::ManagedPlugin,
};
managed_target(text, path, e, kind, local_repo)
}
/// The parent POMs of the POM in `text` at `path`, child-first (its `<parent>` refs,
@@ -1856,7 +1638,7 @@ fn parent_chain(path: &Path, text: &str, local_repo: Option<PathBuf>) -> Vec<(Pa
let Some(parent) = pom.and_then(|p| p.parent) else { return chain };
let Some(pp) = resolver.parent_path(&dir, &parent) else { return chain };
let pp = dunce::canonicalize(&pp).unwrap_or(pp);
let Some(next) = xml::read_text(&pp) else { return chain };
let Ok(next) = std::fs::read_to_string(&pp) else { return chain };
pom = resolver.pom(&pp);
dir = pp.parent().unwrap_or(Path::new("")).to_path_buf();
chain.push((pp, next));
@@ -2355,169 +2137,6 @@ mod tests {
assert_eq!(top(&load_linked(deep.path(), &settings)), [("a/b/c/tool".into(), false)]);
}
/// A sibling Maven project is linked beside the opened root. Its reactor, including a
/// module that points back at the opened project, is read once. Neither POM is rewritten,
/// and only the linked project's directory is an extra content root — not the common
/// parent and not a module that lives outside that directory.
#[test]
fn external_sibling_keeps_its_directory_and_does_not_edit_poms() {
let tmp = tempfile::tempdir().unwrap();
let base = dunce::canonicalize(tmp.path()).unwrap();
let a = base.join("A");
let b = base.join("B");
let c = base.join("C");
std::fs::create_dir_all(a.join("core")).unwrap();
std::fs::create_dir_all(b.join("mod")).unwrap();
std::fs::create_dir_all(&c).unwrap();
let a_pom = "<project><groupId>demo</groupId><artifactId>a</artifactId><version>1</version><packaging>pom</packaging><modules><module>core</module></modules></project>";
let b_pom = "<project><groupId>demo</groupId><artifactId>b</artifactId><version>1</version><packaging>pom</packaging><modules><module>mod</module><module>../A</module><module>../C</module><module>.</module></modules></project>";
let c_pom = "<project><groupId>demo</groupId><artifactId>c</artifactId><version>1</version><modules><module>../B</module></modules></project>";
std::fs::write(a.join("pom.xml"), a_pom).unwrap();
std::fs::write(a.join("core/pom.xml"), "<project><artifactId>core</artifactId></project>").unwrap();
std::fs::write(b.join("pom.xml"), b_pom).unwrap();
std::fs::write(b.join("mod/pom.xml"), "<project><artifactId>bmod</artifactId></project>").unwrap();
std::fs::write(c.join("pom.xml"), c_pom).unwrap();
let wrapper = if cfg!(windows) { "mvnw.cmd" } else { "mvnw" };
std::fs::write(a.join(wrapper), "#!/bin/sh\n").unwrap();
std::fs::write(b.join(wrapper), "#!/bin/sh\n").unwrap();
std::fs::write(base.join(wrapper), "#!/bin/sh\n").unwrap();
let discovered = load_with_linked(&a, None, &[], &[]);
assert_eq!(discovered.projects.iter().map(|p| p.artifact_id.as_str()).collect::<Vec<_>>(), ["a", "core"]);
let stored = vec!["../B/pom.xml".into()];
let linked = load_with_linked(&a, None, &[], &stored);
let b_project = linked.projects.iter().find(|p| p.artifact_id == "b").unwrap();
assert!(!b_project.aggregated);
assert_eq!(b_project.pom, "../B/pom.xml");
assert_eq!(dunce::canonicalize(a.join(&b_project.dir)).unwrap(), b);
assert_eq!(dunce::canonicalize(&b_project.command).unwrap(), dunce::canonicalize(b.join(wrapper)).unwrap());
let a_project = linked.projects.iter().find(|p| p.artifact_id == "a").unwrap();
assert!(!a_project.aggregated);
assert_eq!(a_project.modules, ["core/pom.xml"]);
assert_eq!(dunce::canonicalize(&a_project.command).unwrap(), dunce::canonicalize(a.join(wrapper)).unwrap());
assert_eq!(linked.projects.iter().filter(|p| p.artifact_id == "a").count(), 1);
assert_eq!(linked.projects.iter().filter(|p| p.artifact_id == "b").count(), 1);
assert!(linked.projects.iter().any(|p| p.artifact_id == "c" && p.aggregated));
assert!(linked.projects.iter().any(|p| p.artifact_id == "bmod" && p.aggregated));
let again = load_with_linked(&a, None, &[], &stored);
assert_eq!(
again.projects.iter().map(|p| &p.pom).collect::<Vec<_>>(),
linked.projects.iter().map(|p| &p.pom).collect::<Vec<_>>()
);
let absolute = b.join("pom.xml").to_string_lossy().replace('\\', "/");
let via_a = a.join("../B/pom.xml").to_string_lossy().replace('\\', "/");
let normalized = normalize_linked_poms(
&a,
&[
"../B/pom.xml".into(),
absolute,
via_a,
a.join("pom.xml").to_string_lossy().into_owned(),
b.to_string_lossy().into_owned(),
"stub/pom.xml".into(),
"notes.xml".into(),
],
);
assert_eq!(normalized.len(), 3, "{normalized:?}");
assert_eq!(normalized[0], dunce::canonicalize(b.join("pom.xml")).unwrap().to_string_lossy().replace('\\', "/"));
assert_eq!(normalized[1], "pom.xml");
assert_eq!(normalized[2], "stub/pom.xml");
assert_eq!(crate::maven_external_roots(&a, &normalized), vec![b.clone()]);
assert_eq!(crate::maven_external_roots(&a, &stored), vec![b.clone()]);
let removed = load_with_linked(&a, None, &[], &[]);
assert!(removed.projects.iter().all(|p| p.artifact_id != "b"));
assert_eq!(std::fs::read_to_string(a.join("pom.xml")).unwrap(), a_pom);
assert_eq!(std::fs::read_to_string(b.join("pom.xml")).unwrap(), b_pom);
assert_eq!(std::fs::read_to_string(c.join("pom.xml")).unwrap(), c_pom);
assert!(b.join("mod/pom.xml").is_file());
}
/// The launcher recorded on an external project is that project's wrapper, and
/// running it uses the external directory as the working directory.
#[cfg(unix)]
#[test]
fn external_project_wrapper_runs_in_its_own_directory() {
let tmp = tempfile::tempdir().unwrap();
let base = dunce::canonicalize(tmp.path()).unwrap();
let a = base.join("A");
let b = base.join("B");
std::fs::create_dir_all(b.join("src/main/java")).unwrap();
std::fs::create_dir_all(&a).unwrap();
std::fs::write(a.join("pom.xml"), "<project><artifactId>a</artifactId></project>").unwrap();
std::fs::write(
b.join("pom.xml"),
"<project><artifactId>b</artifactId><build><sourceDirectory>src/main/java</sourceDirectory></build></project>",
)
.unwrap();
std::fs::write(b.join("src/main/java/B.java"), "class B {}").unwrap();
let script = b.join("mvnw");
std::fs::write(&script, "#!/bin/sh\npwd > \"$1\"\nprintf 'BUILD SUCCESS\\n'\n").unwrap();
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(&script, std::fs::Permissions::from_mode(0o755)).unwrap();
}
let model = load_with_linked(&a, None, &[], &["../B/pom.xml".into()]);
let project = model.projects.iter().find(|p| p.artifact_id == "b").unwrap();
let cwd = a.join(&project.dir);
assert_eq!(dunce::canonicalize(&cwd).unwrap(), b);
let marker = tmp.path().join("cwd");
let out = crate::process::run(
&project.command,
&[marker.to_string_lossy().into_owned()],
&cwd,
&std::collections::BTreeMap::new(),
&ide_core::CancellationToken::new(),
|_, _| {},
)
.unwrap();
assert!(out.success, "{}", out.text);
let recorded = dunce::canonicalize(std::fs::read_to_string(&marker).unwrap().trim()).unwrap();
assert_eq!(recorded, b);
}
/// #2877: an aggregator in a subfolder lists the root POM as `../`; linked, it holds every
/// module, the root one included, under the same POM paths the projects have.
#[test]
fn an_aggregator_in_a_subfolder_holds_the_root_pom() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
let child = |id: &str, rel: &str| {
format!(
"<project><parent><groupId>demo</groupId><artifactId>parent</artifactId><version>1.0</version><relativePath>{rel}</relativePath></parent><artifactId>{id}</artifactId></project>"
)
};
write(root, "pom.xml", &child("app", "parent/pom.xml"));
write(
root,
"parent/pom.xml",
"<project><groupId>demo</groupId><artifactId>parent</artifactId><version>1.0</version><packaging>pom</packaging><modules><module>../</module><module>../infra</module></modules></project>",
);
write(root, "infra/pom.xml", &child("infra", "../parent/pom.xml"));
let model = load_with_linked(root, None, &[], &["parent/pom.xml".into()]);
let tops: Vec<&str> = model.projects.iter().filter(|p| !p.aggregated).map(|p| p.artifact_id.as_str()).collect();
assert_eq!(tops, ["parent"]);
let parent = model.projects.iter().find(|p| p.artifact_id == "parent").unwrap();
assert_eq!(parent.modules, ["pom.xml", "infra/pom.xml"]);
for pom in &parent.modules {
assert!(model.projects.iter().any(|p| &p.pom == pom), "{pom}");
}
// A cycle (the root lists the aggregator, which lists the root) keeps the root on top.
write(
root,
"pom.xml",
"<project><groupId>demo</groupId><artifactId>app</artifactId><version>1.0</version><packaging>pom</packaging><modules><module>parent</module></modules></project>",
);
let model = load_with_linked(root, None, &[], &[]);
let tops: Vec<&str> = model.projects.iter().filter(|p| !p.aggregated).map(|p| p.artifact_id.as_str()).collect();
assert_eq!(tops, ["app"]);
}
#[test]
fn chosen_profiles_bring_their_modules_and_dependencies() {
let tmp = tempfile::tempdir().unwrap();
@@ -2676,24 +2295,6 @@ mod tests {
assert!(!is_maven_project(&root.join("c")));
}
#[test]
fn non_utf8_pom_with_declared_encoding_is_a_build() {
// Issue #2439: a GBK `pom.xml` is not valid UTF-8; it must still load
// instead of vanishing as "No maven project".
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
let (bytes, _, _) = encoding_rs::GBK.encode(
"<?xml version=\"1.0\" encoding=\"GBK\"?>\n<project><groupId>g</groupId>\
<artifactId>应用</artifactId><version>1</version><name>演示</name></project>",
);
std::fs::write(root.join("pom.xml"), bytes.as_ref()).unwrap();
let model = load_with_repo(root, None);
assert_eq!(model.projects.len(), 1);
assert_eq!(model.projects[0].artifact_id, "应用");
assert_eq!(model.projects[0].display_name, "演示");
assert!(is_maven_project(root));
}
#[test]
fn aggregate_pom_in_src_is_a_build() {
// Issue #1001: `src/pom.xml` aggregating modules under `src/` is recognized,
@@ -3151,9 +2752,9 @@ mod tests {
let text = std::fs::read_to_string(&path).unwrap();
// The property is declared nowhere along the chain.
assert_eq!(goto_declaration(&text, &path, at(&text, "${missing.key}"), None), None);
// The dependency names its own version and no local repository has its POM.
// The dependency names its own version: it is the declaration.
assert_eq!(goto_declaration(&text, &path, at(&text, "<artifactId>other-lib"), None), None);
// A management entry declares the version; its artifact's POM is not at hand either.
// A management entry is a declaration itself, not a reference.
let parent_path = tmp.path().join("pom.xml");
let parent_text = std::fs::read_to_string(&parent_path).unwrap();
assert_eq!(
@@ -3165,72 +2766,6 @@ mod tests {
assert_eq!(goto_declaration("<project></project>", &path, Pos::new(0, 3), None), None);
}
#[test]
fn artifacts_go_to_their_own_pom_in_the_project_or_the_local_repository() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
let bom_dir = repo.join("com/alibaba/cloud/spring-cloud-alibaba-dependencies/2.2.6.RELEASE");
std::fs::create_dir_all(&bom_dir).unwrap();
let bom = "<project>\n <groupId>com.alibaba.cloud</groupId>\n <artifactId>spring-cloud-alibaba-dependencies</artifactId>\n</project>";
std::fs::write(bom_dir.join("spring-cloud-alibaba-dependencies-2.2.6.RELEASE.pom"), bom).unwrap();
let parent = format!(
r#"{PARENT_HEADER}<properties><alibaba.version>2.2.6.RELEASE</alibaba.version></properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-alibaba-dependencies</artifactId>
<version>${{alibaba.version}}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
</project>"#
);
let child = format!(
r#"{CHILD_HEADER}<dependencies>
<dependency>
<groupId>com.example</groupId>
<artifactId>core</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
</project>"#
);
write_parent_child(tmp.path(), &parent, &child);
let core = tmp.path().join("core/pom.xml");
std::fs::create_dir_all(core.parent().unwrap()).unwrap();
std::fs::write(
&core,
"<project>\n <groupId>com.example</groupId>\n <artifactId>core</artifactId>\n</project>",
)
.unwrap();
let modules =
|| vec![ModulePom { group_id: "com.example".into(), artifact_id: "core".into(), pom: core.clone() }];
// A BOM import in <dependencyManagement> opens the BOM from the local repository.
let parent_path = tmp.path().join("pom.xml");
let parent_text = std::fs::read_to_string(&parent_path).unwrap();
let at_bom = at(&parent_text, "spring-cloud-alibaba-dependencies<");
let goto = goto_declaration_in(&parent_text, &parent_path, at_bom, Some(repo.clone()), modules).unwrap();
assert_eq!(goto.path, bom_dir.join("spring-cloud-alibaba-dependencies-2.2.6.RELEASE.pom"));
assert_eq!(goto.kind, PomGotoKind::ArtifactPom);
assert_eq!(goto.range.start, pos_at(bom, bom.find("<artifactId>").unwrap()));
// A dependency on a project module opens the module's POM.
let path = tmp.path().join("app/pom.xml");
let text = std::fs::read_to_string(&path).unwrap();
let goto =
goto_declaration_in(&text, &path, at(&text, "core</artifactId>"), Some(repo.clone()), modules).unwrap();
assert_eq!((goto.path.as_path(), goto.label.as_str()), (core.as_path(), "com.example:core"));
// The <parent> section opens the parent POM.
let goto = goto_declaration_in(&text, &path, at(&text, "parent</artifactId>"), Some(repo), modules).unwrap();
assert_eq!(goto.path, canon(&parent_path));
assert_eq!(goto.kind, PomGotoKind::ArtifactPom);
}
#[test]
fn a_property_reference_after_non_ascii_text_is_still_found() {
let text = format!(
-677
View File
@@ -1,677 +0,0 @@
//! Completion of Maven coordinates in a POM: inside a `<dependency>`, the
//! `<groupId>`, `<artifactId>` and `<version>` values are completed from the
//! local repository layout (`<repo>/<group path>/<artifactId>/<version>/…`).
//! The Maven Central search endpoint tops the lists up remotely; the host op
//! fetches those (cached) and merges them behind the local ones, so this side
//! stays pure text and directory logic and tests build their own repositories.
use super::xml::{self, Element};
use super::{element_path, interpolate, read_properties};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::cmp::Ordering;
use std::collections::HashMap;
use std::path::{Path, PathBuf};
/// The coordinate element a POM completion fills.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum PomCompletionKind {
/// The dependency's `<groupId>`.
GroupId,
/// The dependency's `<artifactId>`.
ArtifactId,
/// The dependency's `<version>`.
Version,
}
/// One suggestion: the coordinate as inserted, and an optional note shown in
/// the lookup (the owning group of an artifactId, its newest installed version).
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PomCompletionOption {
/// The text the completion inserts (`org.example.lib`, `example-lib`, `1.2.0`).
pub label: String,
pub detail: Option<String>,
}
/// The completion offered at a caret of a POM: the replaced value span (UTF-16
/// offsets) and the suggestions, local ones first.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PomCompletion {
/// UTF-16 offset where the completed text starts: the value's first character.
pub from: u32,
/// UTF-16 offset where the replaced value ends: past the caret when the caret
/// sits inside an existing value, which is then replaced whole.
pub to: u32,
/// Which coordinate element the caret is in.
pub kind: PomCompletionKind,
/// The suggestions, local repository first.
pub options: Vec<PomCompletionOption>,
}
/// Where the caret sits in a `<dependency>` coordinate element: what is being
/// completed, the typed prefix, and the sibling coordinates the rest need.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct PomCompletionTarget {
/// UTF-16 offset of the value's first character (the caret in an empty value).
pub from: u32,
/// UTF-16 offset past the value's last character (the caret in an empty value).
pub to: u32,
/// The value typed so far: the text between `from` and the caret, trimmed.
pub typed: String,
pub kind: PomCompletionKind,
/// The interpolated `<groupId>` sibling, when completing the artifactId and
/// the group is written out plain (no unresolved `${property}`).
pub group: Option<String>,
/// The interpolated `<artifactId>` sibling, when completing the version.
pub artifact: Option<String>,
}
/// A coordinate element's text, with the POM's own `${properties}` replaced;
/// `None` when the element is missing or a property stays unresolved (the
/// coordinates behind it are not known without the parent chain, which the
/// completion does not follow).
fn coordinate(e: Option<&Element>, props: &HashMap<String, String>) -> Option<String> {
let text = interpolate(e?.text.trim(), props);
(!text.is_empty() && !text.contains("${")).then_some(text)
}
fn own_properties(root: &Element) -> HashMap<String, String> {
read_properties(root.child("properties")).into_iter().collect()
}
/// UTF-16 offset -> byte offset, clamped to `text`.
fn utf16_to_byte(text: &str, offset: u32) -> usize {
let mut seen = 0u32;
for (byte, c) in text.char_indices() {
if seen >= offset {
return byte;
}
seen += c.len_utf16() as u32;
}
text.len()
}
/// Byte offset -> UTF-16 offset.
fn utf16_at(text: &str, byte: usize) -> u32 {
text[..byte].encode_utf16().count() as u32
}
/// Byte offset just past the start tag's `>` of the element at `start`.
fn start_tag_end(text: &str, start: usize) -> Option<usize> {
let bytes = text.as_bytes();
let mut i = start.checked_add(1)?;
let mut quote = None;
while i < bytes.len() {
match (quote, bytes[i]) {
(Some(q), b) if b == q => quote = None,
(Some(_), _) => {}
(None, b'"' | b'\'') => quote = Some(bytes[i]),
(None, b'>') => return Some(i + 1),
_ => {}
}
i += 1;
}
None
}
/// What to complete at the caret (`offset`, UTF-16) of the POM `text`: the
/// `<groupId>` / `<artifactId>` / `<version>` element of a `<dependency>` the
/// caret sits in the value of. `None` everywhere else — other elements, carets
/// in an end tag or a `${property}` reference, dependencies without the
/// `<groupId>` (and, for versions, the `<artifactId>`) their completions need.
pub fn completion_target(text: &str, offset: u32) -> Option<PomCompletionTarget> {
let root = xml::parse(text).filter(|e| e.name == "project")?;
let at = utf16_to_byte(text, offset);
// A caret at the very end of the input sits in the element still being
// typed, whose range reaches to the input's end; step back one character
// so `element_path` sees it (the caret's own offset stays `at`).
let probe = if at == text.len() && at > 0 {
let mut p = at - 1;
while p > 0 && !text.is_char_boundary(p) {
p -= 1;
}
p
} else {
at
};
let elements = element_path(&root, probe);
let e = *elements.last()?;
let parent = elements.get(elements.len().checked_sub(2)?)?;
let kind = match e.name.as_str() {
"groupId" => PomCompletionKind::GroupId,
"artifactId" => PomCompletionKind::ArtifactId,
"version" => PomCompletionKind::Version,
_ => return None,
};
if parent.name != "dependency" || at > e.inner_end {
return None;
}
let inner_start = start_tag_end(text, e.start)?;
let value = &text[inner_start..e.inner_end.max(inner_start)];
if value.contains("${") {
return None; // Property references are goto-declaration territory, not completion.
}
let lead = value.len() - value.trim_start().len();
let trail = value.len() - value.trim_end().len();
let (mut from, mut to) = (inner_start + lead, e.inner_end.max(inner_start) - trail);
if from > to {
// A value of whitespace only: complete at the caret.
from = at.min(to);
to = to.max(from);
}
from = from.min(at);
to = to.max(from);
let props = own_properties(&root);
let group = (kind != PomCompletionKind::GroupId).then(|| coordinate(parent.child("groupId"), &props)).flatten();
let artifact =
(kind == PomCompletionKind::Version).then(|| coordinate(parent.child("artifactId"), &props)).flatten();
let typed = text[from..at].trim().to_owned();
Some(PomCompletionTarget { from: utf16_at(text, from), to: utf16_at(text, to), typed, kind, group, artifact })
}
/// The suggestions the local repository at `repo` offers for `target`: group
/// ids installed under it, the artifacts of the target's group, or the versions
/// of the target's artifact. Every scan is bounded (see `SCAN_BUDGET`) and
/// reads only directory listings of `repo` — nothing outside it, no jar
/// contents.
pub fn local_options(repo: &Path, target: &PomCompletionTarget) -> Vec<PomCompletionOption> {
let mut budget = SCAN_BUDGET;
match target.kind {
// `org.spr` completes the `spr` part under `org/`, `org.` the next
// segment under `org/`; every typed segment is validated so POM text
// cannot walk out of the repository.
PomCompletionKind::GroupId => {
let segments: Vec<&str> = target.typed.split('.').collect();
let (partial, base) = segments.split_last().unwrap_or((&"", &[]));
if base.iter().all(|s| segment(s)) {
let dir = base.iter().fold(repo.to_path_buf(), |p, s| p.join(s));
scan_groups(&dir, &base.join("."), partial, &mut budget)
} else {
Vec::new()
}
}
PomCompletionKind::ArtifactId => target
.group
.as_deref()
.and_then(|g| repo_dir(repo, &[g]))
.map_or_else(Vec::new, |dir| scan_artifacts(&dir, &target.typed, &mut budget)),
PomCompletionKind::Version => target
.group
.as_deref()
.zip(target.artifact.as_deref())
.and_then(|(g, a)| repo_dir(repo, &[g, a]))
.map_or_else(Vec::new, |dir| scan_versions(&dir, &target.typed, &mut budget)),
}
}
/// Directory listings one completion may read: the scans are interactive, so
/// they stop before they could be felt, and a huge repository simply yields
/// fewer suggestions (its shallowest first — the walk is breadth-preferring).
const SCAN_BUDGET: u32 = 2048;
/// Suggestion caps per kind: group ids and artifact ids fill a lookup page,
/// versions list an artifact's releases.
const MAX_GROUP_OPTIONS: usize = 60;
const MAX_ARTIFACT_OPTIONS: usize = 60;
const MAX_VERSION_OPTIONS: usize = 100;
/// The cap the merged list (local plus Maven Central's) is truncated to.
pub const MAX_OPTIONS: usize = 100;
/// A directory entry whose name passes the repository layout's alphabet
/// (mirrors `sources::artifact`'s): letters, digits, `._-`. Path separators
/// and `..` never pass, so a POM's `${…}`-free text cannot traverse out of
/// the repository.
fn segment(name: &str) -> bool {
!name.is_empty()
&& name != "."
&& name != ".."
&& name.chars().all(|c| c.is_ascii_alphanumeric() || "._-".contains(c))
}
/// The subdirectories of `dir` with repository-layout names, spending `budget`
/// on the listing; `None` when the budget ran out before `dir` was listed.
fn subdirs(dir: &Path, budget: &mut u32) -> Option<Vec<String>> {
if *budget == 0 {
return None;
}
*budget -= 1;
let Ok(entries) = std::fs::read_dir(dir) else {
return Some(Vec::new());
};
let mut names = Vec::new();
for entry in entries.flatten() {
// The budget bounds file-type checks too: a plain listing answers.
if entry.file_type().map(|t| t.is_dir()).unwrap_or(false)
&& let Ok(name) = entry.file_name().into_string()
&& segment(&name)
{
names.push(name);
}
}
Some(names)
}
/// Whether `dir` holds files — a version directory does (jar, pom, the
/// tracking files), a group path segment never does.
fn holds_files(dir: &Path, budget: &mut u32) -> bool {
if *budget == 0 {
return false;
}
*budget -= 1;
let Ok(entries) = std::fs::read_dir(dir) else {
return false;
};
entries.flatten().any(|e| e.file_type().map(|t| t.is_file()).unwrap_or(false))
}
/// Whether the version directory `dir` holds the version: its `.pom` or a `.jar`. A failed
/// download leaves only `*.lastUpdated` tracking files behind, for a version that may not
/// even exist (one typed while the POM was being edited).
fn installed(dir: &Path, budget: &mut u32) -> bool {
if *budget == 0 {
return false;
}
*budget -= 1;
let Ok(entries) = std::fs::read_dir(dir) else {
return false;
};
entries.flatten().any(|e| {
e.file_type().map(|t| t.is_file()).unwrap_or(false)
&& e.file_name().to_str().is_some_and(|n| n.ends_with(".pom") || n.ends_with(".jar"))
})
}
/// Whether `dir` is an artifact directory: it holds versions of one artifact,
/// recognizable by a `maven-metadata*.xml` beside them or a child directory
/// holding files. A group segment's children are directories without files.
fn artifact_dir(dir: &Path, budget: &mut u32) -> bool {
if *budget == 0 {
return false;
}
*budget -= 1;
let Ok(entries) = std::fs::read_dir(dir) else {
return false;
};
for entry in entries.flatten() {
let Ok(name) = entry.file_name().into_string() else { continue };
if name.starts_with("maven-metadata") {
return true;
}
}
let Some(children) = subdirs(dir, budget) else { return false };
children.iter().any(|c| holds_files(&dir.join(c), budget))
}
/// Whether `dir` is a group directory (holds artifact directories) and whether
/// it also holds deeper group segments worth descending into.
fn group_dir(dir: &Path, budget: &mut u32) -> Option<(bool, bool)> {
let children = subdirs(dir, budget)?;
let (mut group, mut descend) = (false, false);
for child in &children {
if artifact_dir(&dir.join(child), budget) {
group = true;
} else {
descend = true;
}
}
Some((group, descend))
}
/// The group ids installed under `dir` (which the typed prefix `prefix` —
/// "" for the repository root — already names), completing the partial last
/// segment `partial`, shallower groups first.
fn scan_groups(dir: &Path, prefix: &str, partial: &str, budget: &mut u32) -> Vec<PomCompletionOption> {
let mut out = Vec::new();
scan_groups_into(dir, prefix, partial, &mut out, budget);
out.sort_by(|a, b| a.label.cmp(&b.label));
out
}
fn scan_groups_into(dir: &Path, prefix: &str, partial: &str, out: &mut Vec<PomCompletionOption>, budget: &mut u32) {
if out.len() >= MAX_GROUP_OPTIONS {
return;
}
let Some(children) = subdirs(dir, budget) else { return };
for name in children {
if !name.starts_with(partial) {
continue;
}
let path = dir.join(&name);
let label = if prefix.is_empty() { name.clone() } else { format!("{prefix}.{name}") };
let Some((group, descend)) = group_dir(&path, budget) else { return };
if group {
out.push(PomCompletionOption { label: label.clone(), detail: None });
if out.len() >= MAX_GROUP_OPTIONS {
return;
}
}
// Deeper groups (org.example under org.example's own parent) follow the
// shallower ones, so a truncated scan of a huge repository still offers
// every first-level segment.
if descend {
scan_groups_into(&path, &label, "", out, budget);
}
}
}
/// The artifact ids installed under the group directory `dir`, completing the
/// typed prefix, each with its newest installed version as the detail.
fn scan_artifacts(dir: &Path, typed: &str, budget: &mut u32) -> Vec<PomCompletionOption> {
let Some(children) = subdirs(dir, budget) else { return Vec::new() };
let mut out = Vec::new();
for name in children {
if !name.starts_with(typed) || !artifact_dir(&dir.join(&name), budget) {
continue;
}
let detail = newest_version(&dir.join(&name), budget);
out.push(PomCompletionOption { label: name, detail });
}
out.sort_by(|a, b| a.label.cmp(&b.label));
out.truncate(MAX_ARTIFACT_OPTIONS);
out
}
/// The newest version directory of the artifact at `dir`, by Maven's ordering.
fn newest_version(dir: &Path, budget: &mut u32) -> Option<String> {
let children = subdirs(dir, budget)?;
children.into_iter().filter(|name| installed(&dir.join(name), budget)).max_by(|a, b| compare_versions(a, b))
}
/// The versions installed of the artifact at `dir`, completing the typed
/// prefix, newest first.
fn scan_versions(dir: &Path, typed: &str, budget: &mut u32) -> Vec<PomCompletionOption> {
let Some(children) = subdirs(dir, budget) else { return Vec::new() };
let mut out: Vec<_> = children
.into_iter()
.filter(|name| name.starts_with(typed) && installed(&dir.join(name), budget))
.map(|name| PomCompletionOption { label: name, detail: None })
.collect();
out.sort_by(|a, b| compare_versions(&b.label, &a.label));
out.truncate(MAX_VERSION_OPTIONS);
out
}
/// The repository directory of the `/`-separated `coordinates` (a group id, or
/// a group and an artifact id), each segment validated against the layout's
/// alphabet so POM text cannot walk outside `repo`.
fn repo_dir(repo: &Path, coordinates: &[&str]) -> Option<PathBuf> {
let segments: Vec<&str> = coordinates.iter().flat_map(|c| c.split('.')).collect();
segments.iter().all(|s| segment(s)).then(|| segments.iter().fold(repo.to_path_buf(), |p, s| p.join(s)))
}
/// Maven-ish version order, best-effort (a full `ComparableVersion` is out of
/// proportion for ordering a lookup): numeric runs compare numerically, letter
/// runs by their qualifier rank (`alpha < beta < m < rc < snapshot < release <
/// sp`, unknown ones with releases lexically); missing parts rank as zero.
fn compare_versions(a: &str, b: &str) -> Ordering {
fn tokens(v: &str) -> Vec<String> {
v.split(|c: char| !c.is_ascii_alphanumeric()).filter(|t| !t.is_empty()).map(str::to_ascii_lowercase).collect()
}
fn rank(t: &str) -> Option<u8> {
match t {
"a" | "alpha" => Some(0),
"b" | "beta" => Some(1),
"m" | "milestone" => Some(2),
"rc" | "cr" => Some(3),
"snapshot" => Some(4),
"" | "ga" | "final" => Some(5),
"sp" => Some(6),
_ => None,
}
}
let (a, b) = (tokens(a), tokens(b));
for i in 0..a.len().max(b.len()) {
let (x, y) = (a.get(i).map(String::as_str), b.get(i).map(String::as_str));
let (xn, yn) = (x.and_then(|s| s.parse::<u64>().ok()), y.and_then(|s| s.parse::<u64>().ok()));
let order = match (xn, yn) {
// Numbers compare numerically; a number outranks a qualifier.
(Some(xn), Some(yn)) => xn.cmp(&yn),
(Some(_), None) if y.is_some() => return Ordering::Greater,
(None, Some(_)) if x.is_some() => return Ordering::Less,
_ => {
let xr = x.and_then(rank).unwrap_or(5);
let yr = y.and_then(rank).unwrap_or(5);
if xr != yr { xr.cmp(&yr) } else { x.unwrap_or("").cmp(y.unwrap_or("")) }
}
};
if order != Ordering::Equal {
return order;
}
}
Ordering::Equal
}
#[cfg(test)]
mod tests {
use super::*;
/// A small local repository: every file is laid out as Maven installs it,
/// so the scans recognize group, artifact and version directories.
fn repo(dir: &Path) -> PathBuf {
let repo = dir.join("repository");
for file in [
"org/apache/maven/maven-core/4.0.0-rc-5/maven-core-4.0.0-rc-5.jar",
"org/apache/maven/plugins/maven-compiler-plugin/3.8.1/maven-compiler-plugin-3.8.1.jar",
"org/example/lib/1.0/lib-1.0.jar",
"org/example/lib/1.9.1/lib-1.9.1.jar",
"org/example/lib/1.10/lib-1.10.jar",
"org/example/lib/2.0-beta-1/lib-2.0-beta-1.jar",
"org/example/lib/2.0/lib-2.0.pom",
"net/nu/netty/4.1.100/netty-4.1.100.jar",
// Failed downloads (a version typed while editing): tracking files only, not installed.
"org/example/lib/2/lib-2.jar.lastUpdated",
"org/example/lib/3.0/lib-3.0.pom.lastUpdated",
] {
let path = repo.join(file);
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
std::fs::write(&path, b"").unwrap();
}
repo
}
fn target(text: &str, needle: &str) -> Option<PomCompletionTarget> {
completion_target(text, (text.find(needle).unwrap() + needle.len()) as u32)
}
#[test]
fn completes_each_coordinate_element_of_a_dependency() {
let text = "<project><dependencies><dependency>\
<groupId>org.example</groupId><artifactId>lib</artifactId><version>1.0</version>\
</dependency></dependencies></project>";
let t = target(text, "<groupId>org.example").unwrap();
assert_eq!(t.kind, PomCompletionKind::GroupId);
assert_eq!(t.typed, "org.example");
assert_eq!((t.from, t.to), (44, 55));
assert!(t.group.is_none() && t.artifact.is_none());
let t = target(text, "<artifactId>lib").unwrap();
assert_eq!(t.kind, PomCompletionKind::ArtifactId);
assert_eq!(t.typed, "lib");
assert_eq!(t.group.as_deref(), Some("org.example"));
assert!(t.artifact.is_none());
let t = target(text, "<version>1.0").unwrap();
assert_eq!(t.kind, PomCompletionKind::Version);
assert_eq!(t.typed, "1.0");
assert_eq!(t.group.as_deref(), Some("org.example"));
assert_eq!(t.artifact.as_deref(), Some("lib"));
}
#[test]
fn completes_while_the_element_is_being_typed_and_the_value_is_empty() {
let empty = "<project><dependencies><dependency><groupId></groupId></dependency></dependencies></project>";
let t = target(empty, "<groupId>").unwrap();
assert_eq!(t.kind, PomCompletionKind::GroupId);
assert_eq!(t.typed, "");
assert_eq!(t.from, t.to);
let open = "<project><dependencies><dependency><artifactId>li";
let t = target(open, "li").unwrap();
assert_eq!(t.kind, PomCompletionKind::ArtifactId);
assert_eq!(t.typed, "li");
// The value under construction ends with the input and is replaced whole.
assert_eq!(t.to, open.len() as u32);
}
#[test]
fn completes_only_dependency_coordinates() {
// The same elements outside a <dependency> get nothing: a <plugin> is
// completed by the repository, not coordinates.
let plugin =
"<project><build><plugins><plugin><groupId>org.apache.maven</groupId></plugin></plugins></build></project>";
assert_eq!(target(plugin, "<groupId>org.apache.maven"), None);
// A caret in the end tag, or on a `${property}` value, is not a completion.
let text =
"<project><dependencies><dependency><groupId>org.example</groupId></dependency></dependencies></project>";
assert_eq!(target(text, "</groupId>"), None);
let property = "<project><dependencies><dependency><groupId>${example.group}</groupId></dependency></dependencies></project>";
assert_eq!(target(property, "${example.group}"), None);
// A dependency's groupId is required before its artifacts can be listed.
let no_group =
"<project><dependencies><dependency><artifactId>lib</artifactId></dependency></dependencies></project>";
let t = target(no_group, "<artifactId>lib").unwrap();
assert!(t.group.is_none());
}
#[test]
fn interpolates_the_poms_own_properties_for_siblings() {
let text = "<project>\
<properties><example.group>org.example</example.group><example.version>2.0</example.version></properties>\
<dependencies><dependency><groupId>${example.group}</groupId><artifactId>lib</artifactId><version>${example.version}</version></dependency></dependencies>\
</project>";
// The groupId the caret is typing stays a completion; the siblings resolve.
let t = target(text, "<artifactId>lib").unwrap();
assert_eq!(t.group.as_deref(), Some("org.example"));
// A `${property}` under the caret itself gets nothing to complete.
assert_eq!(target(text, "<version>${example.version}"), None);
}
#[test]
fn local_scan_lists_installed_groups_shallowest_first() {
let tmp = tempfile::tempdir().unwrap();
let repo = repo(tmp.path());
let options = |typed: &str| {
local_options(
&repo,
&PomCompletionTarget {
from: 0,
to: 0,
typed: typed.into(),
kind: PomCompletionKind::GroupId,
group: None,
artifact: None,
},
)
.into_iter()
.map(|o| o.label)
.collect::<Vec<_>>()
};
// Only directories holding artifacts are group ids: `net` and `org`
// alone never receive an install, their deeper levels do.
assert_eq!(options(""), ["net.nu", "org.apache.maven", "org.apache.maven.plugins", "org.example"]);
assert_eq!(options("org"), ["org.apache.maven", "org.apache.maven.plugins", "org.example"]);
assert_eq!(options("org."), ["org.apache.maven", "org.apache.maven.plugins", "org.example"]);
assert_eq!(options("org.spr"), Vec::<String>::new());
assert_eq!(options("net."), ["net.nu"]);
}
#[test]
fn local_scan_lists_artifacts_of_the_group_with_their_newest_version() {
let tmp = tempfile::tempdir().unwrap();
let repo = repo(tmp.path());
let options = |typed: &str| {
local_options(
&repo,
&PomCompletionTarget {
from: 0,
to: 0,
typed: typed.into(),
kind: PomCompletionKind::ArtifactId,
group: Some("org.example".into()),
artifact: None,
},
)
.into_iter()
.map(|o| (o.label, o.detail))
.collect::<Vec<_>>()
};
assert_eq!(options(""), [("lib".to_owned(), Some("2.0".to_owned()))]);
assert_eq!(options("net"), Vec::<(String, Option<String>)>::new());
}
#[test]
fn local_scan_lists_versions_of_the_artifact_newest_first() {
let tmp = tempfile::tempdir().unwrap();
let repo = repo(tmp.path());
let options = |typed: &str| {
local_options(
&repo,
&PomCompletionTarget {
from: 0,
to: 0,
typed: typed.into(),
kind: PomCompletionKind::Version,
group: Some("org.example".into()),
artifact: Some("lib".into()),
},
)
.into_iter()
.map(|o| o.label)
.collect::<Vec<_>>()
};
assert_eq!(options(""), ["2.0", "2.0-beta-1", "1.10", "1.9.1", "1.0"]);
assert_eq!(options("1."), ["1.10", "1.9.1", "1.0"]);
}
#[test]
fn a_poms_text_cannot_walk_out_of_the_repository() {
let tmp = tempfile::tempdir().unwrap();
let repo = repo(tmp.path());
for group in ["../../etc", "..", "org/../.."] {
let options = local_options(
&repo,
&PomCompletionTarget {
from: 0,
to: 0,
typed: "".into(),
kind: PomCompletionKind::ArtifactId,
group: Some(group.into()),
artifact: None,
},
);
assert!(options.is_empty(), "{group}");
}
}
#[test]
fn the_scan_stops_at_the_option_cap() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repository");
for i in 0..MAX_GROUP_OPTIONS + 10 {
let group = repo.join(format!("com/example/g{i}"));
std::fs::create_dir_all(group.join("art/1.0")).unwrap();
std::fs::write(group.join("art/1.0/art-1.0.jar"), b"").unwrap();
}
let options = local_options(
&repo,
&PomCompletionTarget {
from: 0,
to: 0,
typed: "".into(),
kind: PomCompletionKind::GroupId,
group: None,
artifact: None,
},
);
assert_eq!(options.len(), MAX_GROUP_OPTIONS);
}
#[test]
fn maven_version_order() {
let mut versions = ["2.0", "1.0", "2.0-beta-1", "2.0-alpha-2", "1.10", "1.9.1", "4.0.0-rc-5", "4.0.0-rc-13"];
versions.sort_by(|a, b| compare_versions(a, b));
assert_eq!(versions, ["1.0", "1.9.1", "1.10", "2.0-alpha-2", "2.0-beta-1", "2.0", "4.0.0-rc-5", "4.0.0-rc-13"]);
}
}
-124
View File
@@ -1,124 +0,0 @@
//! On-demand sources resolution using the project's Maven launcher and repositories.
use super::MavenProject;
use crate::process;
use ide_core::{CancellationToken, IdeError, IdeResult};
use std::path::{Component, Path, PathBuf};
#[derive(Debug, PartialEq, Eq)]
pub struct Artifact {
pub group: String,
pub name: String,
pub version: String,
pub sources: PathBuf,
}
/// Resolve only jars inside the selected local repository, including custom repositories.
pub fn artifact(jar: &Path, repository: &Path) -> Option<Artifact> {
let relative = jar.strip_prefix(repository).ok()?;
let parts: Vec<&str> = relative
.components()
.map(|c| match c {
Component::Normal(s) => s.to_str(),
_ => None,
})
.collect::<Option<_>>()?;
if parts.len() < 4 {
return None;
}
let n = parts.len();
let (name, version, file) = (parts[n - 3], parts[n - 2], parts[n - 1]);
let valid = |s: &str| {
!s.is_empty() && s != "." && s != ".." && s.chars().all(|c| c.is_ascii_alphanumeric() || "_.-".contains(c))
};
if !parts[..n - 1].iter().all(|s| valid(s)) || file != format!("{name}-{version}.jar") {
return None;
}
Some(Artifact {
group: parts[..n - 3].join("."),
name: name.into(),
version: version.into(),
sources: jar.with_file_name(format!("{name}-{version}-sources.jar")),
})
}
pub fn download(
root: &Path,
project: &MavenProject,
artifact: &Artifact,
repository: &Path,
cancel: &CancellationToken,
) -> IdeResult<Option<PathBuf>> {
if cancel.is_cancelled() {
return Err(IdeError::cancelled());
}
if artifact.sources.is_file() {
return Ok(Some(artifact.sources.clone()));
}
let mut args = vec!["-B".into(), "-N".into(), "-Dstyle.color=never".into()];
args.extend(project.options.clone());
args.extend([
format!("-Dmaven.repo.local={}", repository.display()),
"org.apache.maven.plugins:maven-dependency-plugin:3.8.1:get".into(),
format!("-Dartifact={}:{}:{}:jar:sources", artifact.group, artifact.name, artifact.version),
"-Dtransitive=false".into(),
]);
let output = process::run(&project.command, &args, &root.join(&project.dir), &project.env, cancel, |_, _| {})?;
if cancel.is_cancelled() {
return Err(IdeError::cancelled());
}
if artifact.sources.is_file() {
return Ok(Some(artifact.sources.clone()));
}
let coordinate = format!("{}:{}:jar:sources:{}", artifact.group, artifact.name, artifact.version);
if output.success
|| output.text.lines().any(|line| {
line.contains(&coordinate) && (line.contains("Could not find artifact") || line.contains("was not found"))
})
{
return Ok(None);
}
Err(IdeError::unavailable(
output.text.lines().rev().take(20).collect::<Vec<_>>().into_iter().rev().collect::<Vec<_>>().join("\n"),
))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn resolves_sources_coordinates_from_maven_layouts() {
for repository in ["/home/me/.m2/repository", "/tmp/project repo", "C:/Users/me/.m2/repository"] {
for (group, name, version) in [
("org/apache/commons", "commons-lang3", "3.17.0"),
("com/example", "example-lib", "1.0-SNAPSHOT"),
("single", "lib", "1.0.0-rc.1"),
] {
let repository = Path::new(repository);
let dir = repository.join(group).join(name).join(version);
let found = artifact(&dir.join(format!("{name}-{version}.jar")), repository).unwrap();
assert_eq!(found.group, group.replace('/', "."));
assert_eq!(found.name, name);
assert_eq!(found.version, version);
assert_eq!(found.sources, dir.join(format!("{name}-{version}-sources.jar")));
}
}
}
#[test]
fn rejects_non_artifacts_and_paths_outside_the_repository() {
let repository = Path::new("/m2/repository");
for path in [
"/m2/repository-other/g/a/1/a-1.jar",
"/m2/repository/g/a/1/a-1-sources.jar",
"/m2/repository/g/a/1/other.jar",
"/m2/repository/g/a/../a-1.jar",
"/m2/repository/g/a/1/a-1.pom",
"/m2/repository/g/a-1.jar",
"/gradle/caches/modules-2/files-2.1/g/a/1/hash/a-1.jar",
] {
assert_eq!(artifact(Path::new(path), repository), None, "{path}");
}
}
}
+4 -66
View File
@@ -73,9 +73,6 @@ struct GradleCache {
/// The projects as Gradle evaluated them; empty in caches from before they were read.
#[serde(default)]
projects: Vec<ReportedProject>,
/// The projects report how their resources are processed; older caches are stale.
#[serde(default)]
resources: bool,
}
/// Init script printing every project, its source sets and Java version once configured.
@@ -247,12 +244,11 @@ pub fn apply_gradle_cache(root: &Path, cache_dir: &Path, model: &mut GradleModel
}
}
}
// A successful sync from before projects were cached has not seen computed includes,
// nor, before resources were read, which launches need Gradle to process them.
let outdated = cache.error.is_none() && (cache.projects.is_empty() || !cache.resources);
// A successful sync from before projects were cached has not seen computed includes.
let without_projects = cache.error.is_none() && cache.projects.is_empty();
model.synced_at = Some(cache.at_ms);
model.sync_error = cache.error;
model.stale = outdated || cache.hash != hash_gradle_files(&gradle::build_files(root, model));
model.stale = without_projects || cache.hash != hash_gradle_files(&gradle::build_files(root, model));
}
// ---------------------------------------------------------------- processes
@@ -345,13 +341,7 @@ pub fn sync_maven(
// `-T1` keeps mvnd serial: its default parallel build interleaves the
// `dependency:tree` output of concurrent modules, which the sync parses
// as one stream. mvn is serial anyway, so this only pins the daemon.
let mut args: Vec<String> = vec![
"-B".into(),
"-fae".into(),
"-Dstyle.color=never".into(),
"-T1".into(),
maven::LOCAL_REPO_TRACKING_ARG.into(),
];
let mut args: Vec<String> = vec!["-B".into(), "-fae".into(), "-Dstyle.color=never".into(), "-T1".into()];
args.extend(sync_maven_options(&options.maven_options));
if options.offline {
args.push("-o".into());
@@ -466,7 +456,6 @@ pub fn sync_gradle_model(
error: (!out.success).then(|| error_summary(&out.text)),
tasks: if out.success { gradle::parse_tasks(&out.text) } else { Vec::new() },
projects,
resources: true,
};
write_cache(&cache_dir, GRADLE_CACHE, &cache)
}
@@ -718,7 +707,6 @@ mod tests {
test,
src: src.iter().map(|p| core_src(p)).collect(),
res: res.iter().map(|p| core_src(p)).collect(),
processed: None,
};
// A settings script's own println comes first; task listing lines follow.
let out = format!(
@@ -743,15 +731,6 @@ mod tests {
);
let projects = parse_reported_projects(&out);
assert_eq!(projects.len(), 3, "the malformed line is skipped");
// As the init script prints a set whose resources Gradle filters or renames.
let processed = parse_reported_projects(&format!(
"@openidea-model\t{}",
r#"{"path":":","dir":"/p","sourceSets":[{"name":"main","test":false,"src":[],"res":[],"processed":{"task":"processResources","dir":"/p/build/resources/main"}}]}"#
));
assert_eq!(
processed[0].source_sets[0].processed,
Some(gradle::ProcessedResources { task: "processResources".into(), dir: "/p/build/resources/main".into() })
);
let src = |p: &str| PathBuf::from(core_src(p));
assert_eq!(
gradle::reported_source_roots(&projects),
@@ -817,18 +796,12 @@ mod tests {
..Default::default()
})
.collect(),
resources: true,
};
write_cache(&cache_dir, GRADLE_CACHE, &cache).unwrap();
apply_gradle_cache(&root, &cache_dir, &mut model);
assert_eq!(model.projects[1].tasks.len(), 1);
assert!(model.projects[0].tasks.iter().any(|t| t.name == "wrapper"), "root keeps its defaults");
assert!(!model.stale);
write_cache(&cache_dir, GRADLE_CACHE, &GradleCache { resources: false, ..cache.clone() }).unwrap();
let mut m = gradle::load(&root).unwrap();
apply_gradle_cache(&root, &cache_dir, &mut m);
assert!(m.stale, "synced before resource processing was read");
write_cache(&cache_dir, GRADLE_CACHE, &cache).unwrap();
let stale_after = |script: &str| {
std::fs::write(root.join("app/build.gradle"), script).unwrap();
@@ -845,41 +818,6 @@ mod tests {
assert!(m.stale, "gradle.properties is watched");
}
#[test]
fn applied_script_edits_invalidate_the_cached_gradle_model() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path().join("project");
let cache_dir = tmp.path().join("cache");
std::fs::create_dir_all(root.join("gradle")).unwrap();
std::fs::write(root.join("build.gradle"), "apply from: 'gradle/dependencies.gradle'").unwrap();
let applied = root.join("gradle/dependencies.gradle");
std::fs::write(&applied, "implementation 'g:a:1'").unwrap();
let model = gradle::load(&root).unwrap();
let cache = GradleCache {
hash: hash_gradle_files(&gradle::build_files(&root, &model)),
projects: vec![ReportedProject {
path: ":".into(),
dir: root.to_string_lossy().into_owned(),
..Default::default()
}],
resources: true,
..Default::default()
};
write_cache(&cache_dir, GRADLE_CACHE, &cache).unwrap();
let stale = || {
let mut model = gradle::load(&root).unwrap();
apply_gradle_cache(&root, &cache_dir, &mut model);
model.stale
};
assert!(!stale());
std::fs::write(&applied, "// comment\nimplementation 'g:a:1'\n").unwrap();
assert!(!stale());
std::fs::write(&applied, "implementation 'g:a:2'").unwrap();
assert!(stale());
std::fs::remove_file(&applied).unwrap();
assert!(stale());
}
#[test]
fn sync_reads_verbose_trees_with_a_pinned_plugin_unless_offline() {
let online = maven_sync_goals(Path::new("e.xml"), false);
-66
View File
@@ -3,57 +3,6 @@
//! skipped. Namespace prefixes are dropped (`<x:project>` → `project`).
//! Elements keep their byte offsets in the source, for edits ([`crate::pom_edit`]).
use encoding_rs::Encoding;
use std::path::Path;
/// Reads an XML file, honoring its `encoding="…"` declaration: a GBK POM
/// (issue #2439) is not valid UTF-8, and `read_to_string` would drop the
/// whole project as "No maven project" while CLI Maven reads it fine.
pub fn read_text(path: &Path) -> Option<String> {
std::fs::read(path).ok().map(|bytes| decode_bytes(&bytes))
}
/// Decodes XML bytes: BOM first, then the `encoding` declaration (ASCII in
/// every single-byte encoding, so the head scans before decoding), then
/// UTF-8, then a chardetng guess.
pub fn decode_bytes(bytes: &[u8]) -> String {
if let Some((enc, len)) = Encoding::for_bom(bytes) {
let (text, _) = enc.decode_without_bom_handling(&bytes[len..]);
return text.into_owned();
}
if let Some(enc) = declared_encoding(bytes) {
let (text, _) = enc.decode_without_bom_handling(bytes);
return text.into_owned();
}
if let Ok(text) = std::str::from_utf8(bytes) {
return text.to_owned();
}
let mut detector = chardetng::EncodingDetector::new();
detector.feed(bytes, true);
let (text, _) = detector.guess(None, true).decode_without_bom_handling(bytes);
text.into_owned()
}
fn declared_encoding(bytes: &[u8]) -> Option<&'static Encoding> {
let head = std::str::from_utf8(bytes.get(..bytes.len().min(1024))?).ok()?;
let mut rest = head.split("?>").next().unwrap_or(head);
while let Some(i) = rest.find("encoding") {
rest = &rest[i + "encoding".len()..];
let Some(value) = rest.trim_start().strip_prefix('=') else { continue };
let value = value.trim_start();
let (&quote, after) = value.as_bytes().split_first()?;
if quote != b'"' && quote != b'\'' {
continue;
}
let after = std::str::from_utf8(after).ok()?;
let end = after.find(quote as char)?;
if let Some(enc) = Encoding::for_label(after[..end].trim().as_bytes()) {
return Some(enc);
}
}
None
}
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct Element {
pub name: String,
@@ -261,19 +210,4 @@ mod tests {
assert!(parse("").is_none());
assert!(parse("just text").is_none());
}
#[test]
fn decodes_the_declared_encoding() {
// Issue #2439: a GBK POM is not valid UTF-8; decoding must honor the declaration.
let (bytes, _, _) =
encoding_rs::GBK.encode("<?xml version=\"1.0\" encoding=\"GBK\"?>\n<project><name>中文</name></project>");
let text = decode_bytes(&bytes);
assert_eq!(parse(&text).unwrap().text_of("name").as_deref(), Some("中文"));
// An unknown label falls back to UTF-8 rather than mis-decoding.
let text = decode_bytes(b"<?xml version=\"1.0\" encoding=\"X-UNKNOWN\"?>\n<project><name>ok</name></project>");
assert_eq!(parse(&text).unwrap().text_of("name").as_deref(), Some("ok"));
// No declaration: plain UTF-8 passes through untouched.
let text = decode_bytes("<project><name>plain</name></project>".as_bytes());
assert_eq!(parse(&text).unwrap().text_of("name").as_deref(), Some("plain"));
}
}
-3
View File
@@ -11,6 +11,3 @@ license.workspace = true
objc2 = { version = "0.6", default-features = false, features = ["std"] }
objc2-foundation = { version = "0.3", default-features = false, features = ["std", "NSArray", "NSDictionary", "NSEnumerator", "NSObject", "NSString", "NSURL", "NSValue"] }
objc2-app-kit = { version = "0.3", default-features = false, features = ["std", "NSPasteboard"] }
[target.'cfg(target_os = "linux")'.dependencies]
gtk = "0.18"
+1 -60
View File
@@ -21,12 +21,6 @@
use std::path::PathBuf;
/// Replaces every clipboard flavor with plain text. `owner` is the calling window
/// handle on Windows, ignored elsewhere. Call on the main thread on macOS and Linux.
pub fn set_text(text: &str, owner: usize) -> std::io::Result<()> {
imp::set_text(text, owner)
}
/// Size of `DROPFILES`: `pFiles`, `pt` (two `LONG`s), `fNC`, `fWide`.
const HEADER: usize = 20;
@@ -171,12 +165,8 @@ mod imp {
/// Opens the clipboard as its owner: the list is written in full, so no
/// owning window is needed.
fn open() -> bool {
open_for(ptr::null_mut())
}
fn open_for(owner: Handle) -> bool {
for _ in 0..OPEN_ATTEMPTS {
if unsafe { OpenClipboard(owner) } != 0 {
if unsafe { OpenClipboard(ptr::null_mut()) } != 0 {
return true;
}
std::thread::sleep(OPEN_RETRY);
@@ -240,29 +230,6 @@ mod imp {
}
}
pub fn set_text(text: &str, owner: usize) -> std::io::Result<()> {
let (memory, _) = global_text(text);
if memory.is_null() {
return Err(std::io::Error::last_os_error());
}
// SAFETY: ownership transfers only on success; otherwise we free the allocation.
unsafe {
if !open_for(owner as Handle) {
GlobalFree(memory);
return Err(std::io::Error::last_os_error());
}
let taken = EmptyClipboard() != 0 && !SetClipboardData(CF_UNICODETEXT, memory).is_null();
let error = std::io::Error::last_os_error();
CloseClipboard();
if taken {
Ok(())
} else {
GlobalFree(memory);
Err(error)
}
}
}
pub fn set_files(paths: &[PathBuf]) -> bool {
let bytes = encode(paths);
if bytes.len() <= HEADER {
@@ -373,19 +340,6 @@ mod imp {
})
}
pub fn set_text(text: &str, _owner: usize) -> std::io::Result<()> {
debug_assert!(MainThreadMarker::new().is_some(), "NSPasteboard access requires the main thread");
autoreleasepool(|_| {
let board = NSPasteboard::generalPasteboard();
board.clearContents();
if board.setString_forType(&NSString::from_str(text), unsafe { NSPasteboardTypeString }) {
Ok(())
} else {
Err(std::io::Error::other("Cannot write clipboard text"))
}
})
}
pub fn set_files(paths: &[PathBuf]) -> bool {
debug_assert!(MainThreadMarker::new().is_some(), "NSPasteboard access requires the main thread");
autoreleasepool(|_| {
@@ -416,19 +370,6 @@ mod imp {
mod imp {
use std::path::PathBuf;
pub fn set_text(text: &str, _owner: usize) -> std::io::Result<()> {
#[cfg(target_os = "linux")]
{
gtk::Clipboard::get(&gtk::gdk::SELECTION_CLIPBOARD).set_text(text);
Ok(())
}
#[cfg(not(target_os = "linux"))]
{
let _ = text;
Err(std::io::Error::other("Clipboard text is unavailable on this platform"))
}
}
pub fn has_files() -> bool {
false
}
+1 -1
View File
@@ -22,4 +22,4 @@ tempfile.workspace = true
libc = "0.2"
[target.'cfg(windows)'.dependencies]
windows-sys = { version = "0.61", features = ["Wdk_System_SystemServices", "Win32_Foundation", "Win32_Globalization", "Win32_Security", "Win32_System_Diagnostics_ToolHelp", "Win32_System_IO", "Win32_System_JobObjects", "Win32_System_Registry", "Win32_System_SystemInformation", "Win32_System_Threading"] }
windows-sys = { version = "0.61", features = ["Wdk_System_SystemServices", "Win32_Foundation", "Win32_Globalization", "Win32_Security", "Win32_System_Diagnostics_ToolHelp", "Win32_System_IO", "Win32_System_Registry", "Win32_System_SystemInformation", "Win32_System_Threading"] }
-31
View File
@@ -67,23 +67,15 @@ pub struct EditorAppearanceSettings {
/// Smooth caret movement curve: `snappy` | `gliding`.
pub caret_movement_style: String,
pub line_numbers: bool,
/// Gutter numbering: `absolute` | `relative` | `hybrid` (absolute on the caret line, distances elsewhere).
pub line_number_mode: String,
pub method_separators: bool,
/// Sub-options of `editor.showWhitespace`.
pub whitespace_leading: bool,
pub whitespace_inner: bool,
pub whitespace_trailing: bool,
pub indent_guides: bool,
/// Colour indent guides by depth (`--syn-rainbow-*`), while `indent_guides` is on.
pub rainbow_indent_guides: bool,
/// Right margin and visual guides from Code Style.
pub hard_wrap_guides: bool,
pub intention_bulb: bool,
/// Draw doc comments (Javadoc, KDoc, JSDoc, rustdoc) as rendered documentation
/// instead of their source, as IDEA's option of the same name. View › Active
/// Editor › Render Documentation Comments overrides this for one editor.
pub render_doc_comments: bool,
/// Distraction Free Mode: width of the centred text column, in characters.
pub distraction_free_width: u32,
}
@@ -98,16 +90,13 @@ impl Default for EditorAppearanceSettings {
smooth_caret_movement: false,
caret_movement_style: "snappy".into(),
line_numbers: true,
line_number_mode: "absolute".into(),
method_separators: false,
whitespace_leading: true,
whitespace_inner: true,
whitespace_trailing: true,
indent_guides: true,
rainbow_indent_guides: false,
hard_wrap_guides: true,
intention_bulb: true,
render_doc_comments: false,
distraction_free_width: 80,
}
}
@@ -341,23 +330,6 @@ pub struct InlayHintsSettings {
pub code_vision_usages: bool,
/// Code vision: last author of a declaration (VCS blame) beside its usages.
pub code_vision_author: bool,
/// Code vision: "no usages" on declarations nothing uses (IDEA shows it; clicking searches everywhere).
pub code_vision_no_usages: bool,
/// Code vision: where the usages go, above the declaration or at the end of its line.
pub code_vision_usages_position: CodeVisionPosition,
/// Code vision: where the author goes, above the declaration or at the end of its line.
pub code_vision_author_position: CodeVisionPosition,
}
/// Editor › Inlay Hints › Code Vision position (IDEA's "Default position for metrics").
#[derive(Clone, Copy, Debug, Default, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum CodeVisionPosition {
/// A row of its own above the declaration.
#[default]
Top,
/// After the end of the declaration's first line.
Right,
}
impl Default for InlayHintsSettings {
@@ -369,9 +341,6 @@ impl Default for InlayHintsSettings {
disabled_languages: Vec::new(),
code_vision_usages: true,
code_vision_author: true,
code_vision_no_usages: true,
code_vision_usages_position: CodeVisionPosition::Top,
code_vision_author_position: CodeVisionPosition::Top,
}
}
}
-60
View File
@@ -42,10 +42,6 @@ pub enum IdeEvent {
created: Vec<crate::IpcPath>,
modified: Vec<crate::IpcPath>,
removed: Vec<crate::IpcPath>,
/// Subtrees whose ignore rules changed, after the matchers were rebuilt.
#[serde(default)]
#[specta(optional)]
ignore_rules: Vec<crate::IpcPath>,
},
/// Excluded or ignored folders whose contents changed; they stay out of `FsChanged`.
ExcludedDirsChanged {
@@ -152,12 +148,6 @@ pub enum IdeEvent {
SettingsChanged {
key: String,
},
UpdateStatusChanged {
status: crate::updates::UpdateStatus,
},
UpdateAvailable {
update: crate::updates::UpdateInfo,
},
/// The project's source / test / resource / excluded roots changed (build files
/// changed or folders were marked); the UI re-reads them with `project_roots`.
ProjectRootsChanged {
@@ -273,32 +263,6 @@ pub enum DebugEvent {
change_type: String,
message: String,
},
/// The session's capabilities changed after it started: fetch them again.
Capabilities,
/// The adapter's `invalidated` event: what it shows for `areas` (`all`, `stacks`, `threads`,
/// `variables`; empty means all) is stale and has to be fetched again.
Invalidated {
areas: Vec<String>,
},
JvmBackend {
state: JvmBackendState,
},
}
/// The backend that reads a debugged JVM's objects (instances, fields, referrers).
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmBackendState {
Ready,
/// Turned off for the rest of the session: the debug adapter lacks what it needs.
Unsupported {
detail: String,
},
/// The Kotlin JDWP link only forwards now; inspecting stops.
Degraded {
detail: String,
},
Closed,
}
/// Where backend crates publish [`IdeEvent`]s. Cheap to clone via `Arc`.
@@ -320,27 +284,3 @@ pub struct NullSink;
impl EventSink for NullSink {
fn emit(&self, _: IdeEvent) {}
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn jvm_backend_events_json_shape() {
let degraded = DebugEvent::JvmBackend { state: JvmBackendState::Degraded { detail: "x".into() } };
assert_eq!(
serde_json::to_value(&degraded).unwrap(),
json!({"kind": "jvmBackend", "state": {"kind": "degraded", "detail": "x"}})
);
assert_eq!(serde_json::to_value(DebugEvent::Capabilities).unwrap(), json!({"kind": "capabilities"}));
}
#[test]
fn invalidated_event_json_shape() {
assert_eq!(
serde_json::to_value(DebugEvent::Invalidated { areas: vec!["variables".into()] }).unwrap(),
json!({"kind": "invalidated", "areas": ["variables"]})
);
}
}
+2 -10
View File
@@ -12,16 +12,11 @@ use std::path::{Path, PathBuf};
use serde::{Deserialize, Deserializer, Serialize, Serializer};
/// `p` as the UI sees it: `C:\a\b`, `c:\a\b` and `\\?\C:\a\b` become `C:/a/b`.
/// `p` as the UI sees it: `C:\a\b` and `\\?\C:\a\b` become `C:/a/b`.
pub fn to_ipc(p: &Path) -> String {
#[cfg(windows)]
{
let mut s = dunce::simplified(p).to_string_lossy().replace('\\', "/");
// The UI compares path strings; a drive letter is case-insensitive.
if s.as_bytes().get(1) == Some(&b':') {
s[..1].make_ascii_uppercase();
}
s
dunce::simplified(p).to_string_lossy().replace('\\', "/")
}
#[cfg(not(windows))]
{
@@ -149,8 +144,5 @@ mod tests {
assert_eq!(to_ipc(Path::new(r"\\?\C:\a\b")), "C:/a/b");
assert_eq!(to_ipc(&Path::new("C:/a").join("b")), "C:/a/b");
assert_eq!(serde_json::to_string(&IpcPath::from(r"C:\a\b")).unwrap(), r#""C:/a/b""#);
assert_eq!(to_ipc(Path::new(r"c:\a\b")), "C:/a/b");
assert_eq!(to_ipc(Path::new(r"\\?\d:\a")), "D:/a");
assert_eq!(to_ipc(Path::new(r"\\server\share\a")), "//server/share/a");
}
}
-2
View File
@@ -21,13 +21,11 @@ pub mod process;
pub mod project_dir;
pub mod project_roots;
pub mod proxy_settings;
pub mod real_path;
pub mod remote_settings;
pub mod settings;
pub mod task;
pub mod text;
pub mod tool_path;
pub mod updates;
pub mod version_managers;
pub mod workflow_settings;
pub mod wsl;
-26
View File
@@ -21,32 +21,6 @@
use std::collections::BTreeMap;
/// Removes ANSI escape sequences (colour, cursor movement).
pub fn strip_ansi(s: &str) -> String {
if !s.contains('\u{1b}') {
return s.to_owned();
}
let mut out = String::with_capacity(s.len());
let mut chars = s.chars().peekable();
while let Some(c) = chars.next() {
if c != '\u{1b}' {
out.push(c);
continue;
}
if chars.peek() == Some(&'[') {
chars.next();
for c in chars.by_ref() {
if c.is_ascii_alphabetic() || c == '~' {
break;
}
}
} else {
chars.next();
}
}
out
}
/// Decodes complete lines or collected output from IDE-managed tools, preferring UTF-8.
/// Windows launcher/native diagnostics fall back to the system ANSI code page.
/// Callers must buffer incomplete lines so multibyte characters are never split.
+3 -383
View File
@@ -72,75 +72,6 @@ pub fn kill_tree_of(pid: u32) {
let _ = pid;
}
/// Whether a process with this id exists. PID reuse makes this conservative: a reused id keeps
/// whatever was tied to the old process alive, a live process is never reported dead.
pub fn process_alive(pid: u32) -> bool {
// `kill(0, 0)` would signal the caller's own process group.
if pid == 0 {
return false;
}
#[cfg(unix)]
{
// Above `i32::MAX` the id would wrap to a negative value, which `kill` reads as a group.
let Ok(pid) = libc::pid_t::try_from(pid) else { return false };
// SAFETY: signal 0 delivers nothing; it only checks that the process exists and is reachable.
unsafe { libc::kill(pid, 0) == 0 || std::io::Error::last_os_error().raw_os_error() == Some(libc::EPERM) }
}
#[cfg(windows)]
{
use std::os::windows::io::{AsRawHandle, FromRawHandle, OwnedHandle};
use windows_sys::Win32::Foundation::{ERROR_ACCESS_DENIED, GetLastError, STILL_ACTIVE};
use windows_sys::Win32::System::Threading::{
GetExitCodeProcess, OpenProcess, PROCESS_QUERY_LIMITED_INFORMATION,
};
// SAFETY: plain call; a non-null result is a process handle owned by this function.
let handle = unsafe { OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, 0, pid) };
if handle.is_null() {
// A process we may not open still exists.
// SAFETY: reads the thread's last error, set by the call above.
return unsafe { GetLastError() } == ERROR_ACCESS_DENIED;
}
// SAFETY: `handle` is valid and nobody else owns it.
let handle = unsafe { OwnedHandle::from_raw_handle(handle) };
let mut code = 0u32;
// SAFETY: a live process handle opened with query access; one out-parameter.
let ok = unsafe { GetExitCodeProcess(handle.as_raw_handle(), &mut code) };
// A handle that cannot be queried is not evidence that the process is gone.
ok == 0 || code == STILL_ACTIVE as u32
}
#[cfg(not(any(unix, windows)))]
{
true
}
}
/// The program and everything it started: an adapter may report the launcher shell, whose
/// child is the debugged program, so stopping only the reported pid leaves the program alive.
#[cfg(unix)]
pub fn process_tree(pid: i32) -> Vec<i32> {
use std::collections::HashMap;
let Ok(out) = std::process::Command::new("ps").args(["-A", "-o", "pid=,ppid="]).output() else {
return vec![pid];
};
let mut children: HashMap<i32, Vec<i32>> = HashMap::new();
for line in String::from_utf8_lossy(&out.stdout).lines() {
let mut it = line.split_whitespace();
let (Some(p), Some(parent)) = (it.next(), it.next()) else { continue };
let (Ok(p), Ok(parent)) = (p.parse::<i32>(), parent.parse::<i32>()) else { continue };
children.entry(parent).or_default().push(p);
}
let mut tree = vec![pid];
let mut at = 0;
while at < tree.len() {
if let Some(kids) = children.get(&tree[at]) {
tree.extend(kids.iter().copied());
}
at += 1;
}
tree
}
/// [`kill_tree`] that first asks the group to exit: `SIGTERM`, then `grace`, then `SIGKILL`.
/// Git removes its `index.lock` on `SIGTERM`, where `SIGKILL` leaves it behind and the next
/// git command fails. Windows has no such signal to send from outside the child's console,
@@ -173,11 +104,6 @@ fn signal_group(child: &Child, signal: libc::c_int) {
#[cfg(windows)]
const TREE_EXIT_WAIT: Duration = Duration::from_secs(5);
#[cfg(windows)]
pub use windows_job::Job;
#[cfg(windows)]
pub use windows_tree::ProcessTree;
/// `taskkill /T /F` from `pid`, then waits (up to [`TREE_EXIT_WAIT`]) until what the child
/// started is gone: termination is asynchronous, and until a process is gone the files it
/// holds open can't be opened or deleted. The child itself is left to its owner to reap.
@@ -210,48 +136,13 @@ mod windows_tree {
CreateToolhelp32Snapshot, PROCESSENTRY32W, Process32FirstW, Process32NextW, TH32CS_SNAPPROCESS,
};
use windows_sys::Win32::System::Threading::{
GetProcessTimes, OpenProcess, PROCESS_QUERY_LIMITED_INFORMATION, PROCESS_SYNCHRONIZE, PROCESS_TERMINATE,
TerminateProcess, WaitForSingleObject,
GetProcessTimes, OpenProcess, PROCESS_QUERY_LIMITED_INFORMATION, PROCESS_SYNCHRONIZE, WaitForSingleObject,
};
/// Owned handles to a program and its current descendants, retained even if its launcher exits.
pub struct ProcessTree {
processes: Vec<OwnedHandle>,
}
impl ProcessTree {
pub fn capture(pid: u32) -> Self {
let Some(root) = open_with(pid, PROCESS_TERMINATE) else { return Self { processes: Vec::new() } };
let children = descendants_with(pid, PROCESS_TERMINATE);
Self { processes: std::iter::once(root).chain(children).collect() }
}
pub fn is_running(&self) -> bool {
self.processes.iter().any(|process| {
// SAFETY: an owned process handle opened with SYNCHRONIZE access.
unsafe {
WaitForSingleObject(process.as_raw_handle(), 0) == windows_sys::Win32::Foundation::WAIT_TIMEOUT
}
})
}
/// Requests termination children first; poll `is_running` until termination completes.
pub fn terminate(&self) {
for process in self.processes.iter().rev() {
// SAFETY: an owned process handle opened with TERMINATE access, never a reused pid.
unsafe { TerminateProcess(process.as_raw_handle(), 1) };
}
}
}
/// Every process `root` started, directly or not, opened so it can be waited for. A
/// parent's id is reused once it is gone, so a process counts only if it started after
/// its parent.
pub(super) fn descendants(root: u32) -> Vec<OwnedHandle> {
descendants_with(root, 0)
}
fn descendants_with(root: u32, access: u32) -> Vec<OwnedHandle> {
let Some(root_started) = open(root).as_ref().and_then(started) else { return Vec::new() };
let processes = processes();
let mut seen = HashSet::from([root]);
@@ -262,7 +153,7 @@ mod windows_tree {
if ppid != parent || !seen.insert(pid) {
continue;
}
let Some(handle) = open_with(pid, access) else { continue };
let Some(handle) = open(pid) else { continue };
if let Some(at) = started(&handle).filter(|&at| at >= parent_started) {
parents.push((pid, at));
found.push(handle);
@@ -302,12 +193,8 @@ mod windows_tree {
}
fn open(pid: u32) -> Option<OwnedHandle> {
open_with(pid, 0)
}
fn open_with(pid: u32, access: u32) -> Option<OwnedHandle> {
// SAFETY: plain call; a non-null result is a process handle owned by the caller.
let handle = unsafe { OpenProcess(PROCESS_SYNCHRONIZE | PROCESS_QUERY_LIMITED_INFORMATION | access, 0, pid) };
let handle = unsafe { OpenProcess(PROCESS_SYNCHRONIZE | PROCESS_QUERY_LIMITED_INFORMATION, 0, pid) };
// SAFETY: as above.
(!handle.is_null()).then(|| unsafe { OwnedHandle::from_raw_handle(handle) })
}
@@ -322,167 +209,6 @@ mod windows_tree {
}
}
/// A kernel job object over one launched process tree. Everything a launch starts joins
/// the job by inheritance, so [`Job::terminate`] ends the whole tree — also processes
/// started after the launch's snapshot was taken, orphans of a launcher that already
/// exited (where a `taskkill /T /F` from the tracked pid no longer finds them), and
/// processes born in the window a process-enumeration kill would miss. Spawns assign
/// their root right after it starts; when the assignment fails (an enclosing job that
/// disallows nesting, an already-exited pid), callers fall back to `taskkill /T /F`.
#[cfg(windows)]
mod windows_job {
use std::mem::size_of;
use std::os::windows::io::{AsRawHandle, FromRawHandle, OwnedHandle, RawHandle};
use windows_sys::Win32::Foundation::{ERROR_MORE_DATA, INVALID_HANDLE_VALUE};
use windows_sys::Win32::System::JobObjects::{
AssignProcessToJobObject, CreateJobObjectW, JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE,
JOBOBJECT_BASIC_LIMIT_INFORMATION, JOBOBJECT_BASIC_PROCESS_ID_LIST, JOBOBJECT_EXTENDED_LIMIT_INFORMATION,
JobObjectBasicProcessIdList, JobObjectExtendedLimitInformation, QueryInformationJobObject,
SetInformationJobObject, TerminateJobObject,
};
use windows_sys::Win32::System::Threading::{IO_COUNTERS, OpenProcess, PROCESS_SET_QUOTA, PROCESS_TERMINATE};
/// A job object holding one launch's processes. The handle is kept open for the
/// launch's lifetime: `KILL_ON_JOB_CLOSE` ends anything still in the job when the
/// launch record (a run entry, a debug session) is finally dropped.
pub struct Job(OwnedHandle);
impl Job {
/// Takes the just-started `child` into a new job, so everything it starts is in
/// there too. `None` when the job could not be made or the process rejected the
/// assignment (callers fall back to `taskkill /T /F` from the child's pid).
pub fn for_child(child: &std::process::Child) -> Option<Job> {
// SAFETY: the child's own handle is a live process handle.
Self::from_handle(child.as_raw_handle())
}
/// [`Job::for_child`] for a process known by its id (a PTY child, a follow-up run,
/// a `tokio` child). `None` when the id no longer names an openable process — a
/// caller that holds the child cannot have had it reaped, so this is a dead or
/// already-gone launch.
pub fn for_pid(pid: u32) -> Option<Job> {
// SAFETY: plain call; a non-null result is a process handle owned by the caller.
// Assignment needs `SET_QUOTA | TERMINATE` on the process.
let handle = unsafe { OpenProcess(PROCESS_SET_QUOTA | PROCESS_TERMINATE, 0, pid) };
if handle.is_null() {
return None;
}
// SAFETY: the handle was just opened and is owned here.
let handle = unsafe { OwnedHandle::from_raw_handle(handle) };
Self::from_handle(handle.as_raw_handle())
}
fn from_handle(handle: RawHandle) -> Option<Job> {
// SAFETY: plain calls; a non-null result is a job handle owned by the caller.
let job = unsafe { CreateJobObjectW(std::ptr::null(), std::ptr::null()) };
if job.is_null() || job == INVALID_HANDLE_VALUE {
return None;
}
// SAFETY: the job handle is owned here and `limits` is sized as the API requires.
let job = unsafe { OwnedHandle::from_raw_handle(job) };
let mut limits = JOBOBJECT_EXTENDED_LIMIT_INFORMATION {
BasicLimitInformation: JOBOBJECT_BASIC_LIMIT_INFORMATION {
PerProcessUserTimeLimit: 0,
PerJobUserTimeLimit: 0,
LimitFlags: JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE,
MinimumWorkingSetSize: 0,
MaximumWorkingSetSize: 0,
ActiveProcessLimit: 0,
Affinity: 0,
PriorityClass: 0,
SchedulingClass: 0,
},
IoInfo: IO_COUNTERS {
ReadOperationCount: 0,
WriteOperationCount: 0,
OtherOperationCount: 0,
ReadTransferCount: 0,
WriteTransferCount: 0,
OtherTransferCount: 0,
},
ProcessMemoryLimit: 0,
JobMemoryLimit: 0,
PeakProcessMemoryUsed: 0,
PeakJobMemoryUsed: 0,
};
// SAFETY: the job handle is owned here and `limits` is sized as the API requires.
let ok = unsafe {
SetInformationJobObject(
job.as_raw_handle(),
JobObjectExtendedLimitInformation,
&mut limits as *mut _ as *const _,
size_of::<JOBOBJECT_EXTENDED_LIMIT_INFORMATION>() as u32,
)
};
if ok == 0 {
return None;
}
// SAFETY: the process handle belongs to the just-spawned child and outlives
// this call; the job handle is owned here.
let ok = unsafe { AssignProcessToJobObject(job.as_raw_handle(), handle) };
if ok == 0 {
return None;
}
Some(Job(job))
}
/// Adds another just-started root of the same launch to the job (a Go build's
/// follow-up run, a JVM a debug session starts itself). `false` when it did not
/// take; the process then relies on its own kill.
pub fn assign_pid(&self, pid: u32) -> bool {
// SAFETY: plain call; the handle is opened only to assign it, and the job
// handle is owned and valid.
let handle = unsafe { OpenProcess(PROCESS_SET_QUOTA | PROCESS_TERMINATE, 0, pid) };
if handle.is_null() {
return false;
}
// SAFETY: both handles are valid for the duration of the call.
let ok = unsafe { AssignProcessToJobObject(self.0.as_raw_handle(), handle) };
// SAFETY: the process handle was opened above and is owned here.
unsafe { drop(OwnedHandle::from_raw_handle(handle)) };
ok != 0
}
/// Terminates every process in the job. `false` when the call failed (the job
/// handle is invalid): callers then fall back to `taskkill /T /F`. Termination is
/// asynchronous; poll [`Job::is_running`] (bounded by [`TREE_EXIT_WAIT`]) until
/// the tree is gone.
pub fn terminate(&self) -> bool {
// SAFETY: a job handle owned here.
unsafe { TerminateJobObject(self.0.as_raw_handle(), 1) != 0 }
}
/// Whether any process of the launch is still in the job. When the job's state
/// cannot be read, reports gone: waits on this must not wedge, and a kill via
/// [`Job::terminate`] does not depend on the answer.
pub fn is_running(&self) -> bool {
// A list of one is the common case; `ERROR_MORE_DATA` says the real list is
// longer, which means there are processes.
let mut list = JOBOBJECT_BASIC_PROCESS_ID_LIST {
NumberOfAssignedProcesses: 0,
NumberOfProcessIdsInList: 0,
ProcessIdList: [0],
};
let mut returned = 0u32;
// SAFETY: the job handle is owned here and `list` is sized as the API requires.
let ok = unsafe {
QueryInformationJobObject(
self.0.as_raw_handle(),
JobObjectBasicProcessIdList,
&mut list as *mut _ as *mut _,
size_of::<JOBOBJECT_BASIC_PROCESS_ID_LIST>() as u32,
&mut returned,
)
};
if ok != 0 {
list.NumberOfProcessIdsInList > 0
} else {
std::io::Error::last_os_error().raw_os_error() == Some(ERROR_MORE_DATA as i32)
}
}
}
}
/// Gives `cmd` the host's environment back when the IDE runs from an AppImage. `AppRun` and
/// its hooks point `LD_LIBRARY_PATH`, `PATH`, `XDG_DATA_DIRS`, `GIO_MODULE_DIR`, `GTK_PATH`,
/// `GST_PLUGIN_*`, ... into the mounted image for the IDE itself; a host program that
@@ -799,30 +525,6 @@ fn append_path_entries(path: &OsStr, extra: &OsStr) -> OsString {
mod tests {
use super::*;
#[test]
fn process_alive_tells_live_from_dead_pids() {
assert!(process_alive(std::process::id()));
assert!(!process_alive(0));
let mut cmd = if cfg!(windows) {
let mut cmd = Command::new("cmd");
cmd.args(["/C", "exit 0"]);
cmd
} else {
Command::new("true")
};
let mut child = cmd.spawn().unwrap();
let pid = child.id();
child.wait().unwrap();
drop(child);
assert!(!process_alive(pid));
}
#[cfg(unix)]
#[test]
fn process_alive_rejects_ids_that_are_not_a_single_process() {
assert!(!process_alive(u32::MAX));
}
#[cfg(unix)]
#[test]
fn kill_tree_of_reaches_what_the_child_started() {
@@ -1143,25 +845,6 @@ mod tests {
}
assert_eq!(find_executable("nope", &search), None);
}
#[cfg(unix)]
#[test]
fn process_tree_lists_the_child_of_a_shell() {
let mut cmd = Command::new("sh");
own_group(&mut cmd).args(["-c", "sleep 5 & wait"]);
let mut child = cmd.spawn().unwrap();
let sh_pid = child.id() as i32;
// The shell forks `sleep` after it starts, so the tree grows a moment later.
let deadline = std::time::Instant::now() + Duration::from_secs(2);
let mut tree = process_tree(sh_pid);
while tree.len() < 2 && std::time::Instant::now() < deadline {
std::thread::sleep(Duration::from_millis(20));
tree = process_tree(sh_pid);
}
kill_tree(&mut child);
assert_eq!(tree[0], sh_pid, "the pid itself comes first");
assert!(tree.len() >= 2 && tree[1..].iter().all(|p| *p != sh_pid), "the shell's child follows: {tree:?}");
}
}
#[cfg(all(test, unix))]
@@ -1231,66 +914,3 @@ mod kill_tests {
assert!(started.elapsed() < Duration::from_secs(5));
}
}
#[cfg(all(test, windows))]
mod job_tests {
use super::*;
use std::os::windows::process::CommandExt;
use std::process::Stdio;
use std::time::Instant;
/// `cmd` that keeps a `ping` grandchild for a minute, like a launcher that started an
/// application: the job must reach both.
fn launch_tree() -> (std::process::Child, Job, ProcessTree) {
let mut cmd = Command::new("cmd");
cmd.args(["/D", "/S", "/C", "ping -n 60 127.0.0.1 >NUL"]).stdin(Stdio::null()).creation_flags(CREATE_NO_WINDOW);
let child = cmd.spawn().unwrap();
let job = Job::for_child(&child).expect("the job takes the launch");
let tree = ProcessTree::capture(child.id());
assert!(tree.is_running(), "cmd started a grandchild the job inherits");
(child, job, tree)
}
/// Termination is asynchronous: the job reports its processes gone while their
/// handles still wait, so wait on the handles (the reused-pid-proof oracle).
fn assert_gone(tree: &ProcessTree) {
let until = Instant::now() + Duration::from_secs(10);
while tree.is_running() {
assert!(Instant::now() < until, "the tree survived the job");
std::thread::sleep(Duration::from_millis(20));
}
}
#[test]
fn job_terminates_the_tree_the_root_started() {
let (mut child, job, tree) = launch_tree();
assert!(job.is_running());
assert!(job.terminate(), "TerminateJobObject succeeds");
// The captured tree's handles survive the root's death, so the wait below is the
// grandchild's, not a reused pid's.
assert_gone(&tree);
assert!(!job.is_running());
let _ = child.wait();
}
#[test]
fn a_later_spawned_root_joins_the_job() {
let (mut first, job, _) = launch_tree();
let mut cmd = Command::new("cmd");
cmd.args(["/D", "/S", "/C", "ping -n 60 127.0.0.1 >NUL"]).stdin(Stdio::null()).creation_flags(CREATE_NO_WINDOW);
let mut second = cmd.spawn().unwrap();
let second_tree = ProcessTree::capture(second.id());
assert!(job.assign_pid(second.id()), "the follow-up run joins the job");
assert!(job.terminate());
assert_gone(&second_tree);
let _ = first.wait();
let _ = second.wait();
}
#[test]
fn a_dead_pid_makes_no_job() {
// An id of a process this test cannot open (or that does not exist) gives no job,
// which is the callers' cue for the `taskkill /T /F` fallback.
assert!(Job::for_pid(u32::MAX).is_none());
}
}
-53
View File
@@ -1,53 +0,0 @@
//! Where a path really lands, and whether it reaches git's own metadata, for writes that must
//! stay inside a project and out of its `.git`.
use std::path::{Path, PathBuf};
/// `joined` after `..`/symlink resolution as far as the file system allows:
/// the canonical path when it exists, else the canonicalized deepest existing
/// ancestor with the still-missing tail resolved textually — so the location
/// of a missing path is still judged where it would really land.
pub fn resolve_real(joined: &Path) -> PathBuf {
let mut tail = Vec::new();
let mut candidate = joined.to_path_buf();
let mut real = loop {
match dunce::canonicalize(&candidate) {
Ok(real) => break real,
Err(_) => {
let Some(last) = candidate.components().next_back() else {
return joined.to_path_buf();
};
tail.push(last.as_os_str().to_owned());
if !candidate.pop() {
return joined.to_path_buf();
}
}
}
};
for part in tail.iter().rev() {
if part == ".." {
real.pop();
} else if part != "." {
real.push(part);
}
}
real
}
/// Whether a path (relative to the project root) goes through git's own `.git` folder or file.
/// git runs what it finds in `.git` (hooks, `gpg.program`, filters), so a content write must
/// never reach it. Names are normalized for trailing dots and spaces and NTFS stream suffixes;
/// the caller also checks the short-name-resolved path via [`resolve_real`].
pub fn names_git_dir(rel: &Path) -> bool {
rel.components().any(|part| {
let name = part.as_os_str().to_string_lossy();
let name = name.split(':').next().unwrap_or_default().trim_end_matches(['.', ' ']);
name.eq_ignore_ascii_case(".git")
})
}
/// A resolved write destination under `root` that reaches git's own `.git` metadata, judged by
/// the name given and by the name the file system resolves it to.
pub fn reaches_git_dir(root: &Path, real: &Path) -> bool {
real.strip_prefix(root).is_ok_and(names_git_dir) || resolve_real(real).strip_prefix(root).is_ok_and(names_git_dir)
}
+3 -42
View File
@@ -108,32 +108,9 @@ impl Default for DeploymentSettings {
pub enum DeploymentType {
#[default]
Sftp,
Ftp,
/// FTP secured with `AUTH TLS` (explicit FTPS).
Ftps,
Local,
}
/// Host and login of an FTP / FTPS deployment server. The password is kept with
/// the saved passwords, under the server's id.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct FtpConnection {
pub host: String,
/// `None`: 21.
pub port: Option<u16>,
/// Empty: anonymous login.
pub user: String,
/// Passive mode (the client opens data connections); off: active mode.
pub passive: bool,
}
impl Default for FtpConnection {
fn default() -> Self {
Self { host: String::new(), port: None, user: String::new(), passive: true }
}
}
/// A deployment filesystem and how project folders map onto it.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
@@ -143,8 +120,6 @@ pub struct DeploymentServer {
pub name: String,
/// [`SshConfiguration::id`].
pub ssh_config: String,
/// FTP / FTPS servers.
pub ftp: FtpConnection,
/// Absolute remote folder the mappings are relative to; empty = the login directory.
pub root_path: String,
/// Web server URL of the root (Open in Browser on the deployed file).
@@ -226,18 +201,7 @@ pub struct PluginRunTarget {
pub args: Vec<String>,
pub workdir_args: Vec<String>,
pub container_root: Option<String>,
pub commands: Vec<PluginCommandOverride>,
}
/// Per-command argv prefix and optional replacement working-directory arguments.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PluginCommandOverride {
pub name: String,
pub args: Vec<String>,
/// None inherits the target's list; Some([]) suppresses it.
#[serde(default)]
pub workdir_args: Option<Vec<String>>,
pub commands: Vec<(String, Vec<String>)>,
}
#[cfg(test)]
@@ -250,11 +214,8 @@ mod deployment_tests {
serde_json::from_str(r#"{"id":"existing","sshConfig":"ssh","rootPath":"/srv"}"#).unwrap();
assert_eq!(legacy.kind, DeploymentType::Sftp);
assert_eq!(legacy.ssh_config, "ssh");
assert!(legacy.ftp.passive);
let ftp =
FtpConnection { host: "ftp.example.com".into(), port: Some(2121), user: "web".into(), passive: false };
for kind in [DeploymentType::Sftp, DeploymentType::Ftp, DeploymentType::Ftps, DeploymentType::Local] {
let server = DeploymentServer { kind, ftp: ftp.clone(), ..legacy.clone() };
for kind in [DeploymentType::Sftp, DeploymentType::Local] {
let server = DeploymentServer { kind, ..legacy.clone() };
assert_eq!(
serde_json::from_str::<DeploymentServer>(&serde_json::to_string(&server).unwrap()).unwrap(),
server
+9 -425
View File
@@ -59,18 +59,16 @@ pub struct Settings {
/// Build, Execution, Deployment › Run Targets.
pub run_targets: Vec<RunTarget>,
pub cmake: CMakeSettings,
/// Settings › Agents.
/// Tools › Agents › Chat.
pub agents: AgentSettings,
/// Agents › MCP Server (global only).
/// Tools › MCP Server (global only).
pub mcp_server: McpServerSettings,
/// Tools › Agents: Command Execution Policy.
pub agent_security: AgentSecuritySettings,
/// Plugin settings (`[contributes.settings]`): plugin id → its values by key.
#[specta(type = BTreeMap<String, BTreeMap<String, specta_typescript::Unknown>>)]
pub plugins: BTreeMap<String, serde_json::Map<String, serde_json::Value>>,
}
/// Settings › Agents: what the conversation shows and how messages are sent.
/// What the meters in an agent task's header show.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct AgentSettings {
@@ -82,220 +80,20 @@ pub struct AgentSettings {
pub show_usage_limits: bool,
/// What the session has cost so far, as the agent reports it.
pub show_cost: bool,
/// A finished turn's work folds into one line under its message, leaving the reply.
pub fold_turns: bool,
/// Streaming text is revealed at a steady pace rather than chunk by chunk.
pub smooth_text: bool,
#[serde(deserialize_with = "or_default")]
#[specta(type = ThinkingDisplay)]
pub thinking: ThinkingDisplay,
#[serde(deserialize_with = "or_default")]
#[specta(type = ToolCallDisplay)]
pub tool_calls: ToolCallDisplay,
/// A tool call's output (a command's, a diff) is shown without opening it.
pub expand_tool_output: bool,
#[serde(deserialize_with = "or_default")]
#[specta(type = SendKey)]
pub send_with: SendKey,
/// What sending does while the agent works: queue the message for after the turn, or
/// send it into the turn.
#[serde(deserialize_with = "or_default")]
#[specta(type = WhileRunning)]
pub while_running: WhileRunning,
/// Whether new tasks start in the agent's mode that approves everything (Claude Code's
/// bypass permissions, Codex's full access) or in one that asks. User settings only.
#[serde(deserialize_with = "or_default")]
#[specta(type = NewTaskApprovals)]
pub new_task_approvals: NewTaskApprovals,
/// Settings new tasks of an agent start with: agent id → option id → value.
#[serde(deserialize_with = "agent_defaults")]
#[specta(type = BTreeMap<String, BTreeMap<String, String>>)]
pub defaults: BTreeMap<String, BTreeMap<String, String>>,
/// MCP servers passed to coding agents in `session/new`, `session/load` and `session/fork`,
/// keyed by server name (`[agents.mcpServers.<name>]`).
#[serde(deserialize_with = "or_default")]
#[specta(type = BTreeMap<String, AgentMcpServer>)]
pub mcp_servers: BTreeMap<String, AgentMcpServer>,
}
impl Default for AgentSettings {
fn default() -> Self {
Self {
show_context_usage: true,
show_work_time: true,
show_usage_limits: true,
show_cost: false,
fold_turns: true,
smooth_text: true,
thinking: ThinkingDisplay::Auto,
tool_calls: ToolCallDisplay::Grouped,
expand_tool_output: false,
send_with: SendKey::Enter,
while_running: WhileRunning::Queue,
new_task_approvals: NewTaskApprovals::Bypass,
defaults: BTreeMap::new(),
mcp_servers: BTreeMap::new(),
}
Self { show_context_usage: true, show_work_time: true, show_usage_limits: true, show_cost: false }
}
}
/// Transport of an MCP server configured in Settings › Agents › MCP Servers.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum McpTransport {
#[default]
Stdio,
Http,
Sse,
}
/// An MCP server passed to coding agents in `session/new`, `session/load` and `session/fork`.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct AgentMcpServer {
#[serde(deserialize_with = "or_default")]
#[specta(type = McpTransport)]
pub transport: McpTransport,
pub enabled: bool,
#[serde(skip_serializing_if = "Option::is_none")]
#[specta(optional)]
pub command: Option<String>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub args: Vec<String>,
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub env: BTreeMap<String, String>,
#[serde(skip_serializing_if = "Option::is_none")]
#[specta(optional)]
pub url: Option<String>,
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub headers: BTreeMap<String, String>,
/// Agent provider ids this server is enabled for (e.g. `["claude", "codex"]`);
/// empty means enabled for all agents.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub agents: Vec<String>,
}
impl Default for AgentMcpServer {
fn default() -> Self {
Self {
transport: McpTransport::Stdio,
enabled: true,
command: None,
args: Vec::new(),
env: BTreeMap::new(),
url: None,
headers: BTreeMap::new(),
agents: Vec::new(),
}
}
}
/// A hand-edited value these settings don't know reads as the default, rather than
/// failing the whole settings file.
fn or_default<'de, D, T>(d: D) -> Result<T, D::Error>
where
D: serde::Deserializer<'de>,
T: serde::de::DeserializeOwned + Default,
{
Ok(serde_json::from_value(serde_json::Value::deserialize(d)?).unwrap_or_default())
}
/// A theme id, with the built-in themes that were removed read as the Islands theme of their kind.
fn theme_id<'de, D>(d: D) -> Result<String, D::Error>
where
D: serde::Deserializer<'de>,
{
let id = String::deserialize(d)?;
Ok(match id.as_str() {
"one-light" => "light".into(),
"one-dark" => "dark".into(),
_ => id,
})
}
/// [`AgentSettings::defaults`], keeping only the text values of each agent's table.
fn agent_defaults<'de, D>(d: D) -> Result<BTreeMap<String, BTreeMap<String, String>>, D::Error>
where
D: serde::Deserializer<'de>,
{
let serde_json::Value::Object(agents) = serde_json::Value::deserialize(d)? else { return Ok(BTreeMap::new()) };
Ok(agents
.into_iter()
.filter_map(|(agent, options)| match options {
serde_json::Value::Object(options) => Some((
agent,
options
.into_iter()
.filter_map(|(k, v)| match v {
serde_json::Value::String(v) => Some((k, v)),
_ => None,
})
.collect(),
)),
_ => None,
})
.collect())
}
/// How the agent's reasoning shows.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum ThinkingDisplay {
/// Open while it streams, folded once it ends.
#[default]
Auto,
Expanded,
Collapsed,
}
/// How tool calls between messages show.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum ToolCallDisplay {
/// A run of calls is one line saying what they did ("Read 4 files · Edited 2 files").
#[default]
Grouped,
/// Each call on its own line.
Each,
}
/// The key that sends a message; the other one starts a new line.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum SendKey {
#[default]
Enter,
ModEnter,
}
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum WhileRunning {
#[default]
Queue,
Steer,
}
/// The approvals new agent tasks start with.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum NewTaskApprovals {
/// The agent's mode that approves everything without asking, when it has one.
#[default]
Bypass,
/// A mode in which the agent asks before edits and commands.
Ask,
}
/// Agents › MCP Server: agents use the IDE's built-in tools.
/// Tools › MCP Server: agents use the IDE's built-in tools.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct McpServerSettings {
/// Serve the tools on 127.0.0.1 while the IDE runs.
pub enabled: bool,
/// Run the tools that change the project or start processes without asking first
/// (IDEA's brave mode). Read from the IDE's own settings only, never a project's.
pub run_without_confirmation: bool,
}
/// Absent profiles retain automatic discovery for existing projects.
@@ -443,7 +241,6 @@ impl Default for Settings {
cmake: CMakeSettings::default(),
agents: AgentSettings::default(),
mcp_server: McpServerSettings::default(),
agent_security: AgentSecuritySettings::default(),
plugins: BTreeMap::new(),
}
}
@@ -469,10 +266,6 @@ pub struct CustomKeymap {
pub struct GeneralSettings {
pub reopen_last_project: bool,
pub confirm_exit: bool,
/// When on, the last window closes instead of showing the Welcome screen.
pub close_without_welcome: bool,
/// When on, closing a window drops its saved editor tabs, so they are not restored next time.
pub close_tabs_with_window: bool,
/// Where Open… puts a project while one is open: `ask` | `new` | `same` | `attach`.
pub open_project_in: String,
/// Initial folder of the Open / Clone dialogs; empty = the OS default.
@@ -488,11 +281,6 @@ pub struct GeneralSettings {
pub project_name: String,
/// Where updates come from: `auto` (the CNB mirror for the Chinese UI, else GitHub) | `github` | `cnb`.
pub update_source: String,
pub auto_check_updates: bool,
/// System Settings › Updates: open the What's New page after the app starts a new version.
pub show_whats_new_after_update: bool,
/// The version the app last started with, so the first start after an update is recognisable.
pub last_run_version: String,
}
impl Default for GeneralSettings {
@@ -500,8 +288,6 @@ impl Default for GeneralSettings {
Self {
reopen_last_project: true,
confirm_exit: true,
close_without_welcome: false,
close_tabs_with_window: false,
open_project_in: "ask".into(),
default_project_directory: String::new(),
save_on_idle: false,
@@ -510,9 +296,6 @@ impl Default for GeneralSettings {
power_save_mode: false,
project_name: String::new(),
update_source: "auto".into(),
auto_check_updates: true,
show_whats_new_after_update: true,
last_run_version: String::new(),
}
}
}
@@ -567,39 +350,18 @@ impl Default for NotificationGroupSettings {
}
}
/// Where the Navigation Bar (the file path breadcrumbs) is shown. Mirrors
/// IDEA's `View | Appearance | Navigation Bar` (Top / Status Bar / Don't show).
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum NavigationBarPosition {
/// Above the editor, below the main menu / toolbar.
Top,
/// In the footer, next to the status bar widgets — the default, matching
/// where the bar lived before the three-state choice existed.
#[default]
StatusBar,
/// Not shown at all.
Hidden,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct AppearanceSettings {
/// A built-in theme id (`light`, `dark`, `dark-hc`, …), `system` (legacy
/// "Sync with OS"), or a plugin's theme: `plugin:<plugin-id>/<theme-id>`.
#[serde(deserialize_with = "theme_id")]
#[specta(type = String)]
pub theme: String,
pub accent: String,
pub ui_font_size: f32,
pub locale: String,
/// Follow the OS light/dark mode with the preferred themes below.
pub sync_with_os: bool,
#[serde(deserialize_with = "theme_id")]
#[specta(type = String)]
pub preferred_light_theme: String,
#[serde(deserialize_with = "theme_id")]
#[specta(type = String)]
pub preferred_dark_theme: String,
/// A plugin's file icon theme (`<plugin-id>/<theme-id>`); empty = the built-in icons.
pub file_icon_theme: String,
@@ -622,7 +384,7 @@ pub struct AppearanceSettings {
pub project_colors_in_toolbar: bool,
pub dnd_with_alt_only: bool,
pub menu_icons: bool,
/// `hamburger` | `merged` | `toolbar` (a separate row): main menu presentation in the frameless title bar.
/// `hamburger` | `merged`: main menu presentation in the frameless title bar.
pub main_menu_style: String,
/// A lone Alt press focuses or opens the main menu (IDEA `ide.windows.main.menu.focus.on.alt`).
pub main_menu_alt: bool,
@@ -636,9 +398,6 @@ pub struct AppearanceSettings {
/// View › Appearance › Toolbar; hidden leaves IDEA's compact window header.
pub show_main_toolbar: bool,
pub show_status_bar: bool,
/// View › Appearance › Navigation Bar: the file path breadcrumbs, shown in
/// the status bar by default, above the editor, or not at all.
pub navigation_bar: NavigationBarPosition,
/// Status bar widgets shown (`true`) or hidden (`false`) by the user
/// (status bar context menu); widgets not listed use their default.
pub status_bar_widgets: BTreeMap<String, bool>,
@@ -711,7 +470,6 @@ impl Default for AppearanceSettings {
widescreen_tool_windows: false,
show_main_toolbar: true,
show_status_bar: true,
navigation_bar: NavigationBarPosition::StatusBar,
status_bar_widgets: BTreeMap::new(),
main_toolbar: None,
}
@@ -749,9 +507,6 @@ pub struct EditorSettings {
/// Color Scheme › Language Defaults › Semantic highlighting: local variables and
/// parameters get a colour derived from their name (IDEA's rainbow identifiers).
pub semantic_highlighting: bool,
/// Docstring markup format whose tags and keywords the Python editor highlights
/// (`plain`, `rest`, `epytext`, `google`, `numpy`).
pub docstring_format: String,
/// Color Scheme › Language Defaults › Rainbow brackets: `()`, `[]`, `{}` coloured by depth.
pub rainbow_brackets: bool,
pub ligatures: bool,
@@ -796,7 +551,6 @@ impl Default for EditorSettings {
color_overrides: BTreeMap::new(),
custom_schemes: BTreeMap::new(),
semantic_highlighting: false,
docstring_format: "rest".into(),
rainbow_brackets: false,
ligatures: false,
fallback_font: String::new(),
@@ -841,18 +595,6 @@ pub struct FileSettings {
pub file_types: BTreeMap<String, FileTypeAssociation>,
/// Override File Type: file (project-relative and `/`-separated, or absolute) → language id.
pub file_type_overrides: BTreeMap<String, String>,
/// File Types › Files opened in associated applications: file-name patterns
/// (`*.xlsx`) opened with the system application instead of an editor.
pub associated_applications: Vec<String>,
/// Compose detection choices, keyed by project-relative file path.
pub compose_file_overrides: BTreeMap<String, ComposeFileOverride>,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum ComposeFileOverride {
Kept,
Excluded,
}
/// One language's file name and hashbang patterns on top of its definition.
@@ -880,16 +622,6 @@ impl Default for FileSettings {
local_history_max_mb: 512,
file_types: BTreeMap::new(),
file_type_overrides: BTreeMap::new(),
associated_applications: vec![
"*.xlsx".into(),
"*.xls".into(),
"*.docx".into(),
"*.doc".into(),
"*.pptx".into(),
"*.ppt".into(),
"*.pdf".into(),
],
compose_file_overrides: BTreeMap::new(),
}
}
}
@@ -911,7 +643,6 @@ pub struct LanguagesSettings {
pub java: JavaSettings,
pub ruby: RubySettings,
pub markdown: MarkdownSettings,
pub arduino: ArduinoSettings,
}
impl Default for LanguagesSettings {
@@ -933,22 +664,10 @@ impl Default for LanguagesSettings {
java: JavaSettings::default(),
ruby: RubySettings::default(),
markdown: MarkdownSettings::default(),
arduino: ArduinoSettings::default(),
}
}
}
/// Arduino tool discovery paths and default board FQBN, configured in `settings.toml`.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct ArduinoSettings {
pub cli: String,
pub clangd: String,
pub language_server: String,
pub cli_config: String,
pub fqbn: String,
}
/// Languages & Frameworks › Markdown. Like IDEA's, it is application-level only.
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
@@ -1038,28 +757,11 @@ pub struct PythonSettings {
pub interpreter: String,
/// Activate the interpreter's virtualenv in new terminal sessions.
pub activate_virtualenv: bool,
/// Interpreters the user added to Settings › Python › All Interpreters (executable paths).
/// Detection contributes the rest; `hidden_interpreters` removes some of them.
pub interpreters: Vec<String>,
/// Interpreters the user removed from that list, as executable paths: detection does
/// not list them again.
pub hidden_interpreters: Vec<String>,
/// Hide interpreters that belong to another project (a virtualenv inside its folder).
pub hide_other_project_interpreters: bool,
/// Show each interpreter's path under its name on the interpreters page.
pub show_interpreter_paths: bool,
}
impl Default for PythonSettings {
fn default() -> Self {
Self {
interpreter: String::new(),
activate_virtualenv: true,
interpreters: Vec::new(),
hidden_interpreters: Vec::new(),
hide_other_project_interpreters: false,
show_interpreter_paths: true,
}
Self { interpreter: String::new(), activate_virtualenv: true }
}
}
@@ -1183,8 +885,6 @@ pub struct JavaSettings {
/// Extra JVM options for the Java language server (jdtls), space-separated
/// (VS Code's `java.jdt.ls.vmargs`).
pub jdtls_vm_args: String,
/// Share one JDT LS across builds in an ordinary project window; other windows remain separate.
pub jdtls_shared: bool,
/// Build Tools › Maven.
pub maven: MavenSettings,
/// Build Tools › Gradle.
@@ -1235,9 +935,7 @@ pub struct MavenSettings {
pub runner_jre: String,
/// Project-relative root POMs removed from the Maven project model.
pub unlinked_projects: Vec<String>,
/// POMs added as independent top-level projects. Inside the opened project the path is
/// project-relative (`tools/pom.xml`); outside it, the canonical absolute path. Linking
/// does not edit either POM.
/// Project-relative POMs outside the root reactor added as independent top-level projects.
pub linked_projects: Vec<String>,
/// Maven home (or its `mvn` executable); empty = the project's `mvnw`, else `mvn` on `PATH`.
pub home: String,
@@ -1258,8 +956,6 @@ pub struct MavenSettings {
pub threads: String,
/// Active Maven profiles (`-P`), comma-separated; empty leaves profiles to the POM/settings.
pub active_profiles: String,
/// Delegate IDE build/run actions to Maven: launches build with Maven (`javac`) instead of jdtls (ECJ).
pub delegate_build: bool,
}
impl MavenSettings {
@@ -1411,11 +1107,6 @@ pub struct ProjectStructureSettings {
pub unmarked: Vec<String>,
/// Content roots besides the project folder (absolute paths), shown and searched with it.
pub content_roots: Vec<String>,
/// Other projects (absolute paths) this project depends on; indexed and navigable like content roots.
pub dependencies: Vec<String>,
/// Per-workspace-member dependencies: member root (IPC path) → the roots it depends on.
/// A member not listed here (or with an empty list) navigates only into itself.
pub member_dependencies: BTreeMap<String, Vec<String>>,
/// "Exclude files": `;`-separated name patterns (`*.log;tmp*`) excluded at any depth.
pub exclude_files: String,
/// Project compiler output directory (relative to project root or absolute; empty = default).
@@ -1477,39 +1168,6 @@ impl Settings {
mod tests {
use super::*;
#[test]
fn removed_one_themes_read_as_islands_themes() {
let settings: Settings = serde_json::from_str(
r#"{"appearance":{"theme":"one-dark","preferredLightTheme":"one-light","preferredDarkTheme":"one-dark"}}"#,
)
.unwrap();
let a = settings.appearance;
assert_eq!((a.theme.as_str(), a.preferred_light_theme.as_str()), ("dark", "light"));
assert_eq!(a.preferred_dark_theme, "dark");
let kept: Settings = serde_json::from_str(r#"{"appearance":{"theme":"solarized-dark"}}"#).unwrap();
assert_eq!(kept.appearance.theme, "solarized-dark");
}
#[test]
fn window_close_settings_default_off_and_round_trip() {
for old in [r#"{}"#, r#"{"general":{"confirmExit":false}}"#] {
let settings: Settings = serde_json::from_str(old).unwrap();
assert!(!settings.general.close_without_welcome);
assert!(!settings.general.close_tabs_with_window);
}
for close_window in [false, true] {
for close_tabs in [false, true] {
let mut settings = Settings::default();
settings.general.close_without_welcome = close_window;
settings.general.close_tabs_with_window = close_tabs;
let json = serde_json::to_value(&settings).unwrap();
assert_eq!(json["general"]["closeWithoutWelcome"], close_window);
assert_eq!(json["general"]["closeTabsWithWindow"], close_tabs);
assert_eq!(serde_json::from_value::<Settings>(json).unwrap().general, settings.general);
}
}
}
#[test]
fn cmake_profile_argument_arrays_round_trip() {
let mut settings = Settings::default();
@@ -1572,21 +1230,6 @@ mod tests {
}
}
#[test]
fn docstring_format_defaults_and_round_trips() {
let old: Settings = serde_json::from_str(r#"{"editor":{"semanticHighlighting":true}}"#).unwrap();
assert_eq!(old.editor.docstring_format, "rest");
assert_eq!(Settings::default().editor.docstring_format, "rest");
for format in ["plain", "rest", "epytext", "google", "numpy"] {
let mut settings = Settings::default();
settings.editor.docstring_format = format.into();
let json = serde_json::to_value(&settings).unwrap();
assert_eq!(json["editor"]["docstringFormat"], format);
let restored: Settings = serde_json::from_value(json).unwrap();
assert_eq!(restored.editor.docstring_format, format);
}
}
#[test]
fn tool_window_badges_default_on_and_round_trip() {
let old: Settings = serde_json::from_str(r#"{"appearance":{"showToolWindowNames":true}}"#).unwrap();
@@ -1602,27 +1245,6 @@ mod tests {
}
}
#[test]
fn double_click_opens_diff_is_unset_by_default_and_round_trips() {
let old: Settings = serde_json::from_str(r#"{"vcs":{"commit":{"nonModal":true}}}"#).unwrap();
assert_eq!(old.vcs.commit.double_click_opens_diff, None);
// The unset value shows up as `null` in the defaults: `SettingsStore::set` only
// accepts a key path it finds there.
let json = serde_json::to_value(Settings::default()).unwrap();
assert_eq!(json["vcs"]["commit"]["doubleClickOpensDiff"], serde_json::Value::Null);
for on in [true, false] {
let mut settings = Settings::default();
settings.vcs.commit.double_click_opens_diff = Some(on);
let json = serde_json::to_value(&settings).unwrap();
assert_eq!(json["vcs"]["commit"]["doubleClickOpensDiff"], on);
let restored: Settings = serde_json::from_value(json).unwrap();
assert_eq!(restored.vcs.commit.double_click_opens_diff, Some(on));
}
// Explicit `null` reads back as unset.
let cleared: Settings = serde_json::from_str(r#"{"vcs":{"commit":{"doubleClickOpensDiff":null}}}"#).unwrap();
assert_eq!(cleared.vcs.commit.double_click_opens_diff, None);
}
#[test]
fn agent_meters_default_and_round_trip() {
let old: Settings = serde_json::from_str(r#"{"agents":{"showCost":true}}"#).unwrap();
@@ -1637,22 +1259,6 @@ mod tests {
assert_eq!(restored.agents, settings.agents);
}
#[test]
fn hand_edited_agent_values_it_does_not_know_keep_the_rest() {
let settings: Settings = serde_json::from_str(
r#"{"editor":{"fontSize":17},"agents":{"thinking":"open","sendWith":"modEnter",
"defaults":{"claude":{"model":"opus","effort":3},"codex":"x"}}}"#,
)
.unwrap();
assert_eq!(settings.editor.font_size, 17.0);
assert_eq!(settings.agents.thinking, ThinkingDisplay::Auto);
assert_eq!(settings.agents.send_with, SendKey::ModEnter);
assert_eq!(
settings.agents.defaults,
BTreeMap::from([("claude".into(), BTreeMap::from([("model".into(), "opus".into())]))])
);
}
#[test]
fn main_menu_style_defaults_and_round_trips() {
let old: Settings = serde_json::from_str(r#"{"appearance":{"compactMode":true}}"#).unwrap();
@@ -1777,12 +1383,10 @@ mod maven_linkage_tests {
fn maven_runner_jre_defaults_and_round_trips() {
let old: MavenSettings = serde_json::from_str(r#"{"home":"/opt/maven"}"#).unwrap();
assert!(old.runner_jre.is_empty());
assert!(!old.delegate_build);
assert!(MavenSettings::default().runner_jre.is_empty());
let settings = MavenSettings { runner_jre: "/opt/jdk-21".into(), delegate_build: true, ..old };
let settings = MavenSettings { runner_jre: "/opt/jdk-21".into(), ..old };
let encoded = serde_json::to_string(&settings).unwrap();
assert!(encoded.contains(r#""runnerJre":"/opt/jdk-21""#));
assert!(encoded.contains(r#""delegateBuild":true"#));
assert_eq!(serde_json::from_str::<MavenSettings>(&encoded).unwrap(), settings);
}
@@ -1862,26 +1466,6 @@ mod maven_linkage_tests {
}
}
#[cfg(test)]
mod java_settings_tests {
use super::*;
#[test]
fn shared_jdtls_defaults_off_and_round_trips() {
for old in [r#"{}"#, r#"{"jdk":"/jdk"}"#] {
let java: JavaSettings = serde_json::from_str(old).unwrap();
assert!(!java.jdtls_shared);
}
for shared in [false, true] {
let java = JavaSettings { jdtls_shared: shared, ..JavaSettings::default() };
let json = serde_json::to_value(&java).unwrap();
assert_eq!(json["jdtlsShared"], serde_json::json!(shared));
assert_eq!(serde_json::from_value::<JavaSettings>(json).unwrap(), java);
}
assert!(serde_json::from_str::<JavaSettings>(r#"{"jdtlsShared":"true"}"#).is_err());
}
}
#[cfg(test)]
mod gradle_settings_tests {
use super::*;
+2 -35
View File
@@ -138,11 +138,8 @@ impl WorkspaceEdit {
continue;
}
*expected = expected.or(version);
// Repeats within one entry are intended (LSP inserts at one position
// apply in order); only an alias entry repeats what is already merged.
let merged = own.len();
for edit in edits {
if !own[..merged].contains(&edit) {
if !own.contains(&edit) {
own.push(edit);
}
}
@@ -280,40 +277,10 @@ pub struct OutlineItem {
/// Fuzzy-match highlight: indices (in chars of the displayed string) that matched.
pub type MatchIndices = Vec<u32>;
#[cfg(test)]
#[cfg(all(test, unix))]
mod workspace_edit_tests {
use super::*;
#[test]
fn repeated_edits_within_one_entry_survive_but_repeats_across_entries_do_not() {
let path = PathBuf::from("missing-dir/main.ts");
let at = |line, text: &str| TextEdit {
range: Range::new(Pos::new(line, 0), Pos::new(line, 0)),
new_text: text.into(),
};
let entry = |edits: Vec<TextEdit>| FileEdit::Edit { path: path.clone().into(), edits, version: None };
let (nl, brace, other) = (at(0, "\n"), at(1, "}"), at(2, "x"));
for (entries, expected) in [
(vec![vec![nl.clone(), nl.clone()]], vec![nl.clone(), nl.clone()]),
(vec![vec![brace.clone(), nl.clone(), brace.clone()]], vec![brace.clone(), nl.clone(), brace.clone()]),
(vec![vec![nl.clone(), nl.clone()], vec![nl.clone(), nl.clone()]], vec![nl.clone(), nl.clone()]),
(vec![vec![nl.clone()], vec![nl.clone(), other.clone()]], vec![nl.clone(), other.clone()]),
(
vec![vec![nl.clone()], vec![other.clone(), other.clone()]],
vec![nl.clone(), other.clone(), other.clone()],
),
] {
let mut workspace =
WorkspaceEdit { files: entries.clone().into_iter().map(entry).collect(), ..Default::default() };
workspace.deduplicate();
assert_eq!(workspace.files, [entry(expected.clone())], "{entries:?}");
let once = workspace.clone();
workspace.deduplicate();
assert_eq!(workspace, once, "{entries:?}");
}
}
#[cfg(unix)]
#[test]
fn aliases_are_one_edit_but_resource_operations_preserve_order() {
let dir = tempfile::tempdir().unwrap();
+5 -2
View File
@@ -61,13 +61,16 @@ pub fn find_command(command: &str, search: &OsStr, root: &Path) -> Option<PathBu
fn is_executable(path: &Path) -> bool {
let Ok(metadata) = path.metadata() else { return false };
if !metadata.is_file() {
return false;
}
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
metadata.is_file() && metadata.permissions().mode() & 0o111 != 0
metadata.permissions().mode() & 0o111 != 0
}
#[cfg(not(unix))]
metadata.is_file()
true
}
/// [`search_path`]'s directories, in order and once each. Only absolute ones: a
-151
View File
@@ -1,151 +0,0 @@
use serde::{Deserialize, Serialize};
use specta::Type;
use std::collections::HashSet;
use std::time::{Duration, Instant};
pub const CHECK_INTERVAL: Duration = Duration::from_secs(4 * 60 * 60);
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum UpdateSource {
#[default]
Github,
Cnb,
}
impl UpdateSource {
pub fn resolve(preference: &str, locale: &str, system_locale: &str) -> Self {
match preference {
"github" => return Self::Github,
"cnb" => return Self::Cnb,
_ => {}
}
let locale = locale.trim();
let locale = if matches!(locale, "" | "auto" | "system") { system_locale } else { locale };
if locale.split(['-', '_']).next().is_some_and(|base| base.eq_ignore_ascii_case("zh")) {
Self::Cnb
} else {
Self::Github
}
}
}
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct UpdateInfo {
pub version: String,
pub current_version: String,
pub notes: Option<String>,
pub date: Option<String>,
}
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct UpdateCheck {
pub enabled: bool,
pub update: Option<UpdateInfo>,
}
#[derive(Clone, Debug, Default, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct UpdateStatus {
/// Unix milliseconds of the last successful check in this process.
pub last_checked: Option<f64>,
pub checking: bool,
}
/// One schedule and announcement history for the desktop process, independent of its windows.
#[derive(Default)]
pub struct UpdatePolicy {
attempted: Option<Instant>,
pub status: UpdateStatus,
announced: HashSet<String>,
}
impl UpdatePolicy {
pub fn due_in(&self, now: Instant) -> Duration {
self.attempted.map_or(Duration::ZERO, |at| CHECK_INTERVAL.saturating_sub(now.saturating_duration_since(at)))
}
pub fn begin(&mut self, now: Instant) {
self.attempted = Some(now);
self.status.checking = true;
}
pub fn finish(&mut self, checked_at: Option<f64>) {
self.status.checking = false;
if let Some(at) = checked_at {
self.status.last_checked = Some(at);
}
}
pub fn announced(&self, version: &str) -> bool {
self.announced.contains(version)
}
/// Manual checks always show their result, but also count as an announcement.
pub fn announce(&mut self, version: &str) -> bool {
self.announced.insert(version.to_owned())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn automatic_and_manual_checks_resolve_the_same_preferred_source() {
for (preference, locale, system, expected) in [
("auto", "zh-CN", "en-US", UpdateSource::Cnb),
("auto", " ZH_CN ", "en-US", UpdateSource::Cnb),
("auto", "system", "zh-CN", UpdateSource::Cnb),
("auto", "auto", "zh-TW", UpdateSource::Cnb),
("auto", "", "zh", UpdateSource::Cnb),
("auto", "en", "zh-CN", UpdateSource::Github),
("github", "zh", "zh-CN", UpdateSource::Github),
("cnb", "en", "en-US", UpdateSource::Cnb),
] {
assert_eq!(UpdateSource::resolve(preference, locale, system), expected);
}
}
#[test]
fn checks_at_startup_and_every_four_hours_even_after_failure() {
let mut policy = UpdatePolicy::default();
let now = Instant::now();
assert_eq!(policy.due_in(now), Duration::ZERO);
policy.begin(now);
assert!(policy.status.checking);
policy.finish(None);
assert_eq!(policy.status.last_checked, None);
assert_eq!(policy.due_in(now + CHECK_INTERVAL - Duration::from_secs(1)), Duration::from_secs(1));
assert_eq!(policy.due_in(now + CHECK_INTERVAL), Duration::ZERO);
policy.begin(now + CHECK_INTERVAL);
policy.finish(Some(1234.0));
assert_eq!(policy.status.last_checked, Some(1234.0));
assert!(!policy.status.checking);
assert_eq!(policy.due_in(now + CHECK_INTERVAL), CHECK_INTERVAL);
}
#[test]
fn manual_checks_reset_the_schedule_without_losing_last_success_on_failure() {
let mut policy = UpdatePolicy::default();
let now = Instant::now();
policy.begin(now);
policy.finish(Some(1000.0));
policy.begin(now + Duration::from_secs(60));
policy.finish(None);
assert_eq!(policy.status.last_checked, Some(1000.0));
assert_eq!(policy.due_in(now + CHECK_INTERVAL), Duration::from_secs(60));
}
#[test]
fn announcements_are_per_version_across_windows_sources_and_manual_checks() {
let mut policy = UpdatePolicy::default();
assert!(policy.announce("0.1.40"));
assert!(!policy.announce("0.1.40"));
assert!(policy.announce("0.1.41"));
assert!(!policy.announce("0.1.40"));
assert!(UpdatePolicy::default().announce("0.1.40"));
}
}
-69
View File
@@ -43,7 +43,6 @@ impl Default for VcsSettings {
pub struct GitSettings {
/// Show blame after the current editor line.
pub inline_blame: bool,
pub show_branch_in_project_view: bool,
/// Path to the git executable; empty = auto-detect.
pub executable: String,
/// Default of Update Project: `merge` | `rebase`.
@@ -75,7 +74,6 @@ impl Default for GitSettings {
fn default() -> Self {
Self {
inline_blame: false,
show_branch_in_project_view: false,
executable: String::new(),
update_method: "merge".into(),
auto_update_on_push_rejected: false,
@@ -117,13 +115,6 @@ pub struct CommitSettings {
pub wrap_at_margin: bool,
/// Plugin before-commit checks turned off (`<plugin id>/<check id>`); the rest run.
pub disabled_plugin_checks: Vec<String>,
/// Double-click / Enter in the changes view: `None` follows the interface (the
/// Commit tool window opens the diff, Local Changes the source file).
///
/// No `skip_serializing_if`: an unset value must still appear in
/// `Settings::default()`, or `SettingsStore::set` rejects the key as unknown.
#[specta(optional)]
pub double_click_opens_diff: Option<bool>,
}
impl Default for CommitSettings {
@@ -144,7 +135,6 @@ impl Default for CommitSettings {
show_right_margin: true,
wrap_at_margin: false,
disabled_plugin_checks: Vec::new(),
double_click_opens_diff: None,
}
}
}
@@ -192,33 +182,6 @@ pub struct IssueLink {
pub link: String,
}
/// Per-level colours for console log lines: a line whose level is detected
/// (`ERROR`, `WARN`, …) takes the level's colour; ANSI colours the program
/// prints itself win over it. Values are CSS colours, usually scheme tokens.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct LogLevelColors {
pub enabled: bool,
pub error: String,
pub warn: String,
pub info: String,
pub debug: String,
pub trace: String,
}
impl Default for LogLevelColors {
fn default() -> Self {
Self {
enabled: false,
error: "var(--ansi-red)".into(),
warn: "var(--ansi-yellow)".into(),
info: "var(--ansi-green)".into(),
debug: "var(--ansi-cyan)".into(),
trace: "var(--ansi-magenta)".into(),
}
}
}
/// Build, Execution, Deployment › Console.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
@@ -230,8 +193,6 @@ pub struct ConsoleSettings {
pub encoding: String,
/// Consecutive lines containing any of these are folded.
pub fold_patterns: Vec<String>,
/// Log lines take the colour of their level (Run/Debug consoles).
pub log_colors: LogLevelColors,
}
impl Default for ConsoleSettings {
@@ -248,7 +209,6 @@ impl Default for ConsoleSettings {
"at node:internal/".into(),
"<frozen importlib._bootstrap".into(),
],
log_colors: LogLevelColors::default(),
}
}
}
@@ -352,7 +312,6 @@ pub struct TerminalSettings {
pub override_ide_shortcuts: bool,
pub shell_integration: bool,
pub highlight_hyperlinks: bool,
pub highlight_selection: bool,
pub option_as_meta: bool,
/// Escape moves focus to the editor instead of reaching the shell.
pub escape_to_editor: bool,
@@ -385,7 +344,6 @@ impl Default for TerminalSettings {
override_ide_shortcuts: true,
shell_integration: true,
highlight_hyperlinks: true,
highlight_selection: true,
option_as_meta: false,
escape_to_editor: false,
cursor_shape: "block".into(),
@@ -442,30 +400,3 @@ impl Default for ExternalTool {
}
}
}
/// Tools › Agents: Command Execution Policy (whitelist and blacklist patterns).
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct AgentSecuritySettings {
/// Commands or patterns (`git status*`, `pwd`) automatically approved without prompting.
pub allowed_commands: Vec<String>,
/// Commands or patterns (`rm -rf *`, `mkfs*`) automatically rejected.
pub blocked_commands: Vec<String>,
}
impl Default for AgentSecuritySettings {
fn default() -> Self {
Self {
allowed_commands: Vec::new(),
blocked_commands: vec![
"rm -rf *".into(),
"rm -rf /".into(),
"mkfs*".into(),
"dd *".into(),
":(){ :|:& };:".into(),
"shutdown*".into(),
"reboot*".into(),
],
}
}
}
-5
View File
@@ -11,17 +11,12 @@ tempfile.workspace = true
ide-core.workspace = true
ide-language.workspace = true
ide-jsonrpc.workspace = true
ide-jvm-attach.workspace = true
ide-jvm-inspect.workspace = true
ide-jdwp.workspace = true
ide-heap.workspace = true
serde.workspace = true
serde_json.workspace = true
specta.workspace = true
tokio = { workspace = true, features = ["net"] }
parking_lot.workspace = true
memchr = "2"
percent-encoding = "2"
portable-pty.workspace = true
tracing.workspace = true
+85 -361
View File
@@ -72,13 +72,6 @@ fn builtin(adapter: &str) -> Option<DebuggerConfig> {
"stdio",
"Install Kotlin Debug Adapter in Settings › Languages & Frameworks › Language Servers",
),
// Inside the language server: reached through `debugServer`, never started.
"kotlin-lsp" => cfg(
"kotlin-lsp",
&[],
"tcp",
"Install JetBrains Kotlin LSP in Settings › Languages & Frameworks › Language Servers",
),
// Windows installs `python.exe`; `python3.exe` there is at most a Store alias.
"debugpy" => cfg(
if cfg!(windows) { "python" } else { "python3" },
@@ -105,20 +98,9 @@ fn builtin(adapter: &str) -> Option<DebuggerConfig> {
"stdio",
"Install netcoredbg (.NET debugger) in Settings › Languages & Frameworks › Language Servers",
),
"lldb-dap" => cfg(
"lldb-dap",
&[],
"stdio",
"Install LLVM's lldb-dap (brew install llvm), or on macOS use the one found by xcrun -f lldb-dap",
),
// GDB's own DAP interpreter, stdio transport. The DAP support it needs shipped in GDB 14
// (`prepare_gdb` refuses older ones with the required version in the message).
"gdb" => cfg(
"gdb",
&["-i", "dap"],
"stdio",
"Install GDB 14 or newer (apt install gdb, dnf install gdb, brew install gdb)",
),
"lldb-dap" => {
cfg("lldb-dap", &[], "stdio", "Install LLVM's lldb-dap (brew install llvm; on macOS `xcrun -f lldb-dap`)")
}
"rdbg" => cfg("rdbg", &["-O", "--port", PORT_PLACEHOLDER, "-c"], "tcp", "gem install debug"),
// A jdtls bundle: the session is started through jdtls and reached via `debugServer`.
"java-debug" => cfg(
@@ -127,12 +109,6 @@ fn builtin(adapter: &str) -> Option<DebuggerConfig> {
"tcp",
"Install Microsoft java-debug (Settings › Languages & Frameworks › Language Servers)",
),
"bun-dap" => cfg(
"bun-dap",
&["${port}", "127.0.0.1"],
"tcp",
"To install bun-dap: git clone https://github.com/morf-bls/bun-dap, run bun install && bun run build, then place dist/bun-dap in your PATH",
),
_ => return None,
})
}
@@ -151,7 +127,6 @@ pub fn default_type(adapter: &str) -> Option<&'static str> {
"delve" => "go",
"netcoredbg" => "coreclr",
"lldb-dap" => "lldb-dap",
"gdb" => "gdb",
"java-debug" => "java",
_ => return None,
})
@@ -191,11 +166,6 @@ pub fn parse_address(addr: &str) -> Option<(String, u16)> {
}
/// A port that was free a moment ago on the loopback interface.
///
/// Inherently racy: another process can bind the port between the listener being
/// dropped here and the adapter binding it. Callers that hand the port to a child
/// retry on a bind conflict (see [`ProcessLauncher`]), or — better — let the child
/// pick `:0` and report the actual port back (as the Kotlin JDWP debuggee does).
pub fn free_port() -> std::io::Result<u16> {
let listener = std::net::TcpListener::bind(("127.0.0.1", 0))?;
Ok(listener.local_addr()?.port())
@@ -357,89 +327,6 @@ pub async fn prepare_debugpy(spec: &mut AdapterSpec) -> IdeResult<()> {
)))
}
/// GDB's DAP interpreter (`-i dap`) shipped in GDB 14; older ones exit with
/// "Interpreter `dap' unrecognized".
const GDB_MIN_MAJOR: u32 = 14;
/// `(major, minor)` of the first version number in the output of `gdb --version`
/// (`GNU gdb (GDB) 14.2`, `GNU gdb (Ubuntu 12.1-0ubuntu1~22.04.2) 12.1`, `GNU gdb
/// (GDB) 15.0.50.20240403-0ubuntu1`). `None` when no standalone version number is there.
pub fn parse_gdb_version(output: &str) -> Option<(u32, u32)> {
let line = output.lines().next()?;
// Distribution names can contain versions, e.g. `(GDB; SUSE Linux Enterprise 15) 14.2`.
let line = line.rsplit_once(')').map_or(line, |(_, version)| version);
let b = line.as_bytes();
let mut i = 0;
while i < b.len() {
if !b[i].is_ascii_digit() {
i += 1;
continue;
}
let start = i;
while i < b.len() && b[i].is_ascii_digit() {
i += 1;
}
let major: u32 = line[start..i].parse().ok()?;
let mut minor = 0;
if i + 1 < b.len() && b[i] == b'.' && b[i + 1].is_ascii_digit() {
i += 1;
let ms = i;
while i < b.len() && b[i].is_ascii_digit() {
i += 1;
}
minor = line[ms..i].parse().unwrap_or(0);
}
// A version stands alone: `mingw32-gdb13.exe`'s embedded digits are part of a word.
if start > 0 && (b[start - 1].is_ascii_alphabetic() || b[start - 1] == b'_') {
continue;
}
return Some((major, minor));
}
None
}
/// Probe the already resolved and verified executable with its launch environment.
async fn gdb_version(binary: &Path, spec: &AdapterSpec) -> IdeResult<Option<(u32, u32)>> {
let mut probe = Command::new(binary);
if let Some(path) = adapter_path(&spec.env) {
probe.env("PATH", path);
}
ide_core::process::prepare(probe.as_std_mut());
let probe = probe
.arg("--version")
.current_dir(&spec.cwd)
.envs(spec.env.iter().cloned())
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::null())
.kill_on_drop(true)
.output();
let Ok(Ok(out)) = tokio::time::timeout(Duration::from_secs(15), probe).await else {
return Ok(None);
};
Ok(parse_gdb_version(&String::from_utf8_lossy(&out.stdout)))
}
/// Refuse old GDB versions and report whether launch must follow configurationDone.
/// GDB 15 starts the inferior in launch; 14 and 16+ start it at configurationDone.
async fn prepare_gdb(spec: &AdapterSpec, binary: &Path) -> IdeResult<bool> {
let Some((major, minor)) = gdb_version(binary, spec).await? else {
return Ok(false);
};
if major >= GDB_MIN_MAJOR {
return Ok(major == 15);
}
let mut msg = format!(
"debugging with GDB needs GDB {GDB_MIN_MAJOR} or newer (found {major}.{minor}): \
this one has no DAP interpreter (`gdb -i dap`)"
);
if let Some(hint) = &spec.install_hint {
msg.push_str(". ");
msg.push_str(hint);
}
Err(IdeError::unavailable(msg))
}
/// An adapter that is a plugin of another program (java-debug, a jdtls bundle) rather
/// than a program: it has no process of its own and is reached through `debugServer`.
pub fn is_plugin_bundle(command: &str) -> bool {
@@ -473,8 +360,6 @@ pub type BoxWriter = Box<dyn AsyncWrite + Unpin + Send>;
/// A connected adapter: byte streams plus the process (if we spawned one).
pub struct AdapterIo {
/// GDB 15 starts running inside `launch`; configure breakpoints before sending it.
pub launch_after_configuration: bool,
pub reader: BoxReader,
pub writer: BoxWriter,
pub child: Option<Child>,
@@ -509,9 +394,6 @@ const CONNECT_TIMEOUT: Duration = Duration::from_secs(15);
const EMBEDDED_START_TIMEOUT: Duration = Duration::from_secs(120);
/// Probe of a `connect` address before starting the command.
const PROBE_TIMEOUT: Duration = Duration::from_millis(500);
/// Attempts at a fresh port for a TCP adapter: the one `free_port` hands out can be
/// claimed by another process before the adapter binds it, so a new one usually wins.
const PORT_ATTEMPTS: u32 = 3;
impl AdapterLauncher for ProcessLauncher {
fn launch(&self, mut spec: AdapterSpec, reuse_port: Option<u16>) -> LaunchFuture {
@@ -526,7 +408,6 @@ impl AdapterLauncher for ProcessLauncher {
let stream = connect_with_retries(host, port, None, timeout).await?;
let (r, w) = stream.into_split();
return Ok(AdapterIo {
launch_after_configuration: false,
reader: Box::new(r),
writer: Box::new(w),
child: None,
@@ -550,7 +431,6 @@ impl AdapterLauncher for ProcessLauncher {
let _ = stream.set_nodelay(true);
let (r, w) = stream.into_split();
return Ok(AdapterIo {
launch_after_configuration: false,
reader: Box::new(r),
writer: Box::new(w),
child: None,
@@ -579,152 +459,95 @@ impl AdapterLauncher for ProcessLauncher {
.ok_or_else(|| missing(&spec))?
}
};
let launch_after_configuration =
if spec.adapter == "gdb" { prepare_gdb(&spec, &binary).await? } else { false };
let base_args = substitute_root(&spec.args, &spec.cwd);
match spec.transport {
Transport::Stdio => {
let mut io = spawn_adapter(&spec, &binary, &base_args, None, CONNECT_TIMEOUT).await?;
io.launch_after_configuration = launch_after_configuration;
Ok(io)
let port = match spec.transport {
// An embedded adapter started without `${port}` listens on its usual address.
Transport::Tcp => match &embedded {
Some((_, port)) if !spec.args.iter().any(|a| a.contains(PORT_PLACEHOLDER)) => Some(*port),
_ => Some(free_port()?),
},
Transport::Stdio => None,
};
let args = substitute_root(&spec.args, &spec.cwd);
let args = match port {
Some(port) => substitute_port(&args, port),
None => args,
};
let mut cmd = Command::new(&binary);
if let Some(path) = adapter_path(&spec.env) {
cmd.env("PATH", path);
}
ide_core::process::prepare(cmd.as_std_mut());
if spec.adapter == "rdbg" {
ide_core::process::own_group(cmd.as_std_mut());
}
cmd.args(&args)
.envs(spec.env.iter().map(|(k, v)| (k, v)))
.current_dir(&spec.cwd)
.stdin(if port.is_some() { Stdio::null() } else { Stdio::piped() })
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.kill_on_drop(true);
let mut child =
cmd.spawn().map_err(|e| IdeError::unavailable(format!("failed to start {}: {e}", binary.display())))?;
let (program_sink, program_output) = if SHARES_STDIO.contains(&spec.adapter.as_str()) {
let (tx, rx) = tokio::sync::mpsc::unbounded_channel();
(Some(tx), Some(rx))
} else {
(None, None)
};
if let Some(err) = child.stderr.take() {
pump_lines(err, spec.adapter.clone(), stderr.clone(), program_sink.clone().map(|tx| (tx, "stderr")));
}
match port {
None => {
let stdin = child.stdin.take().ok_or_else(|| IdeError::internal("adapter stdin unavailable"))?;
let stdout = child.stdout.take().ok_or_else(|| IdeError::internal("adapter stdout unavailable"))?;
Ok(AdapterIo {
reader: Box::new(stdout),
writer: Box::new(stdin),
child: Some(child),
port: None,
stderr,
program_output,
})
}
Transport::Tcp => {
// An embedded adapter started without `${port}` listens on its usual
// address; every other TCP adapter gets a port chosen here so it can be
// retried if another process takes it before the adapter binds.
let fixed = match &embedded {
Some((_, port)) if !spec.args.iter().any(|a| a.contains(PORT_PLACEHOLDER)) => Some(*port),
_ => None,
};
let start_timeout = if embedded.is_some() { EMBEDDED_START_TIMEOUT } else { CONNECT_TIMEOUT };
for attempt in 1..=PORT_ATTEMPTS {
let port = match fixed {
Some(port) => port,
None => free_port()?,
};
let args = substitute_port(&base_args, port);
match spawn_adapter(&spec, &binary, &args, Some(port), start_timeout).await {
Ok(mut io) => {
io.launch_after_configuration = launch_after_configuration;
return Ok(io);
}
Err(e) => {
if fixed.is_none() && attempt < PORT_ATTEMPTS && is_port_race(&e) {
tracing::warn!(
target: "ide_dap::adapter",
"debug adapter failed to start on port {port}: {}; retrying",
e.message
);
continue;
}
return Err(e);
}
}
Some(port) => {
if let Some(out) = child.stdout.take() {
pump_lines(out, spec.adapter.clone(), stderr.clone(), program_sink.map(|tx| (tx, "stdout")));
}
unreachable!("the retry loop returns on its last attempt")
let timeout = if embedded.is_some() { EMBEDDED_START_TIMEOUT } else { CONNECT_TIMEOUT };
let stream = match connect_with_retries(LOOPBACK, port, Some(&mut child), timeout).await {
Ok(s) => s,
Err(e) => {
if spec.adapter == "rdbg"
&& let Some(pid) = child.id()
{
ide_core::process::kill_tree_of(pid);
}
let _ = child.start_kill();
let tail = stderr.lock().join("\n");
return Err(if tail.is_empty() {
e
} else {
IdeError::unavailable(format!("{}\n{tail}", e.message))
});
}
};
let (r, w) = stream.into_split();
Ok(AdapterIo {
reader: Box::new(r),
writer: Box::new(w),
child: Some(child),
port: Some(port),
stderr,
program_output,
})
}
}
})
}
}
/// Starts `binary` with `args`, then connects to it when `port` is set (a TCP adapter)
/// or hands back its stdio when it is not. The child is stopped if it never accepts.
async fn spawn_adapter(
spec: &AdapterSpec,
binary: &Path,
args: &[String],
port: Option<u16>,
start_timeout: Duration,
) -> IdeResult<AdapterIo> {
let stderr = Arc::new(parking_lot::Mutex::new(Vec::new()));
let mut cmd = Command::new(binary);
if let Some(path) = adapter_path(&spec.env) {
cmd.env("PATH", path);
}
ide_core::process::prepare(cmd.as_std_mut());
// Every adapter leads its own process group: the program an adapter launches itself
// (java-debug's JVM) joins the group, so a forced stop reaches the whole tree with
// one `kill(-pgid)`. On Windows the same coverage comes from the session's job object.
ide_core::process::own_group(cmd.as_std_mut());
cmd.args(args)
.envs(spec.env.iter().map(|(k, v)| (k, v)))
.current_dir(&spec.cwd)
.stdin(if port.is_some() { Stdio::null() } else { Stdio::piped() })
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.kill_on_drop(true);
let mut child =
cmd.spawn().map_err(|e| IdeError::unavailable(format!("failed to start {}: {e}", binary.display())))?;
let (program_sink, program_output) = if SHARES_STDIO.contains(&spec.adapter.as_str()) {
let (tx, rx) = tokio::sync::mpsc::unbounded_channel();
(Some(tx), Some(rx))
} else {
(None, None)
};
if let Some(err) = child.stderr.take() {
pump_lines(err, spec.adapter.clone(), stderr.clone(), program_sink.clone().map(|tx| (tx, "stderr")));
}
match port {
None => {
let stdin = child.stdin.take().ok_or_else(|| IdeError::internal("adapter stdin unavailable"))?;
let stdout = child.stdout.take().ok_or_else(|| IdeError::internal("adapter stdout unavailable"))?;
Ok(AdapterIo {
launch_after_configuration: false,
reader: Box::new(stdout),
writer: Box::new(stdin),
child: Some(child),
port: None,
stderr,
program_output,
})
}
Some(port) => {
if let Some(out) = child.stdout.take() {
pump_lines(out, spec.adapter.clone(), stderr.clone(), program_sink.map(|tx| (tx, "stdout")));
}
let stream = match connect_with_retries(LOOPBACK, port, Some(&mut child), start_timeout).await {
Ok(s) => s,
Err(e) => {
if spec.adapter == "rdbg"
&& let Some(pid) = child.id()
{
ide_core::process::kill_tree_of(pid);
}
let _ = child.start_kill();
let tail = stderr.lock().join("\n");
return Err(if tail.is_empty() {
e
} else {
IdeError::unavailable(format!("{}\n{tail}", e.message))
});
}
};
let (r, w) = stream.into_split();
Ok(AdapterIo {
launch_after_configuration: false,
reader: Box::new(r),
writer: Box::new(w),
child: Some(child),
port: Some(port),
stderr,
program_output,
})
}
}
}
/// A failed adapter start that another process grabbing the free port can explain
/// (POSIX and Windows "address already in use" texts, or the adapter exiting because
/// it could not bind). Anything else is a real start failure and is not retried.
fn is_port_race(err: &IdeError) -> bool {
let message = err.message.to_ascii_lowercase();
message.contains("address already in use")
|| message.contains("addrinuse")
|| message.contains("only one usage of each socket address")
|| message.contains("exited before accepting connections")
}
fn pump_lines<R: AsyncRead + Unpin + Send + 'static>(
stream: R,
name: String,
@@ -860,102 +683,15 @@ mod tests {
#[test]
fn builtins_cover_common_adapters() {
for name in ["codelldb", "debugpy", "js-debug", "delve", "netcoredbg", "java-debug", "gdb"] {
for name in ["codelldb", "debugpy", "js-debug", "delve", "netcoredbg", "java-debug"] {
let cfg = builtin(name).unwrap();
assert!(cfg.install_hint.is_some());
}
let delve = resolve(&builtin("delve").unwrap(), PathBuf::from("/"));
assert_eq!(delve.transport, Transport::Tcp);
// GDB's own DAP interpreter over stdio, with the required version in its hint.
let gdb = builtin("gdb").unwrap();
assert_eq!(gdb.command, "gdb");
assert_eq!(gdb.args, ["-i", "dap"]);
assert_eq!(gdb.transport, "stdio");
assert!(gdb.install_hint.as_deref().is_some_and(|h| h.contains("14")), "{gdb:?}");
assert_eq!(default_type("gdb"), Some("gdb"));
assert!(builtin("nope").is_none());
}
/// The first version number of `gdb --version`'s first line, however the distro decorates it.
#[test]
fn parses_gdb_versions() {
for (out, want) in [
("GNU gdb (GDB) 14.2\nCopyright (C) 2023 Free Software Foundation", Some((14, 2))),
("GNU gdb (GDB) 15.0.50.20240403-0ubuntu1\n", Some((15, 0))),
("GNU gdb (Ubuntu 12.1-0ubuntu1~22.04.2) 12.1", Some((12, 1))),
("GNU gdb (Debian 13.1-3) 13.1", Some((13, 1))),
("GNU gdb (GDB) Fedora Linux 16.3-1.fc41", Some((16, 3))),
("GNU gdb (GDB; SUSE Linux Enterprise 15) 14.2", Some((14, 2))),
("GNU gdb (GDB; SUSE Linux Enterprise 15) 11.1", Some((11, 1))),
("mingw32-gdb13.exe (GDB) 13.2", Some((13, 2))),
("GNU gdb 14", Some((14, 0))),
("no version here", None),
("", None),
] {
assert_eq!(parse_gdb_version(out), want, "{out}");
}
}
#[tokio::test]
async fn gdb_probe_never_executes_a_changed_pinned_program() {
let dir = tempfile::tempdir().unwrap();
let fake = dir.path().join(if cfg!(windows) { "gdb.cmd" } else { "gdb" });
std::fs::write(&fake, "approved").unwrap();
let pin = ProgramPin::approve(&fake).unwrap();
#[cfg(windows)]
std::fs::write(&fake, "@echo off\r\necho ran>probe-ran\r\necho GNU gdb (GDB) 15.1\r\n").unwrap();
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::write(&fake, "#!/bin/sh\ntouch probe-ran\necho 'GNU gdb (GDB) 15.1'\n").unwrap();
std::fs::set_permissions(&fake, std::fs::Permissions::from_mode(0o755)).unwrap();
}
let mut spec = resolve(&builtin("gdb").unwrap(), dir.path().to_owned());
spec.command = fake.to_string_lossy().into_owned();
spec.pinned = Some(pin);
let error = ProcessLauncher.launch(spec, None).await.err().expect("changed program is refused");
assert_eq!(error.kind, ide_core::error::ErrorKind::PermissionDenied);
assert!(!dir.path().join("probe-ran").exists(), "the version probe executed an unapproved program");
}
#[tokio::test]
async fn gdb_probe_uses_the_verified_program_and_launch_environment_once() {
use tokio::io::AsyncReadExt;
let dir = tempfile::tempdir().unwrap();
let fake = dir.path().join(if cfg!(windows) { "gdb.cmd" } else { "gdb" });
#[cfg(windows)]
std::fs::write(&fake, "@echo off\r\nif \"%~1\"==\"--version\" (\r\n echo probe>>probes\r\n echo GNU gdb ^(GDB^) %GDB_TEST_VERSION%\r\n) else (\r\n echo adapter\r\n)\r\n").unwrap();
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::write(&fake, "#!/bin/sh\nif [ \"$1\" = --version ]; then\n echo probe >> probes\n echo \"GNU gdb (GDB) $GDB_TEST_VERSION\"\nelse\n echo adapter\nfi\n").unwrap();
std::fs::set_permissions(&fake, std::fs::Permissions::from_mode(0o755)).unwrap();
}
let pin = ProgramPin::approve(&fake).unwrap();
for (version, deferred) in [("12.1", false), ("14.2", false), ("15.1", true), ("16.3", false)] {
let mut spec = resolve(&builtin("gdb").unwrap(), dir.path().to_owned());
// Only the approved path is executable; the symbolic command must never be probed.
spec.command = "definitely-not-a-real-binary-openidea".into();
spec.pinned = Some(pin.clone());
spec.env.push(("GDB_TEST_VERSION".into(), version.into()));
let result = ProcessLauncher.launch(spec, None).await;
if version == "12.1" {
let error = result.err().expect("GDB 12 has no DAP interpreter");
assert_eq!(error.kind, ide_core::error::ErrorKind::Unavailable);
assert!(error.message.contains("GDB 14 or newer"), "{}", error.message);
assert!(error.message.contains("12.1"), "{}", error.message);
} else {
let mut io = result.unwrap();
assert_eq!(io.launch_after_configuration, deferred, "{version}");
let mut output = String::new();
io.reader.read_to_string(&mut output).await.unwrap();
assert_eq!(output.trim(), "adapter");
io.child.as_mut().unwrap().wait().await.unwrap();
}
}
assert_eq!(std::fs::read_to_string(dir.path().join("probes")).unwrap().lines().count(), 4);
}
#[test]
fn free_port_is_nonzero() {
assert_ne!(free_port().unwrap(), 0);
@@ -1039,16 +775,4 @@ mod tests {
assert_eq!(err.kind, ide_core::error::ErrorKind::Unavailable);
assert!(err.message.contains("runs inside the Java language server"), "{}", err.message);
}
#[test]
fn port_conflicts_are_recognised_for_retry() {
let in_use = IdeError::unavailable(
"debug adapter exited before accepting connections (exit status: 1)\nbind: address already in use",
);
assert!(is_port_race(&in_use));
assert!(is_port_race(&IdeError::unavailable(
"Only one usage of each socket address (protocol/network address/port) is normally permitted."
)));
assert!(!is_port_race(&IdeError::unavailable("failed to start codelldb: file not found")));
}
}
+7 -48
View File
@@ -246,7 +246,13 @@ impl Connection {
fn dispatch(inner: &Inner, incoming: &mpsc::UnboundedSender<Incoming>, body: &[u8]) {
tracing::trace!(target: "ide_dap::wire", "<-- {}", String::from_utf8_lossy(body));
let Some(msg) = parse_message(body) else { return };
let msg: Value = match serde_json::from_slice(body) {
Ok(v) => v,
Err(e) => {
tracing::warn!("dap: unparsable message: {e}");
return;
}
};
match msg.get("type").and_then(Value::as_str) {
Some("response") => {
let Some(request_seq) = msg.get("request_seq").and_then(Value::as_i64) else {
@@ -279,25 +285,6 @@ fn dispatch(inner: &Inner, incoming: &mpsc::UnboundedSender<Incoming>, body: &[u
}
}
/// A message body as JSON. Some adapters pass the debuggee's output through as it was
/// written: netcoredbg on Windows embeds console output in the ANSI code page (GBK on a
/// Chinese system), which isn't valid UTF-8 and made the whole `output` event unparsable,
/// so every line with non-ASCII text went missing. Such a body is decoded like other tool
/// output (UTF-8, else the system code page) before parsing.
fn parse_message(body: &[u8]) -> Option<Value> {
let error = match serde_json::from_slice(body) {
Ok(v) => return Some(v),
Err(e) => e,
};
if std::str::from_utf8(body).is_err()
&& let Ok(v) = serde_json::from_str(&ide_core::output_encoding::decode_tool_output(body))
{
return Some(v);
}
tracing::warn!("dap: unparsable message: {error}");
None
}
/// `message`, or the formatted `body.error` (`{format, variables}`) when present.
fn error_message(msg: &Value) -> String {
if let Some(err) = msg.pointer("/body/error")
@@ -319,22 +306,6 @@ fn error_message(msg: &Value) -> String {
mod tests {
use super::*;
#[test]
fn parses_output_events_with_non_utf8_text() {
// "中文" in GBK inside an output event, as netcoredbg sends it on a Chinese Windows.
let mut body = br#"{"seq":5,"type":"event","event":"output","body":{"category":"stdout","output":""#.to_vec();
body.extend_from_slice(&[0xd6, 0xd0, 0xce, 0xc4]);
body.extend_from_slice(br#"\n"}}"#);
let msg = parse_message(&body).expect("parsed");
assert_eq!(msg["event"], "output");
let output = msg["body"]["output"].as_str().unwrap();
assert!(output.ends_with('\n'));
// Off Windows (UTF-8 code page) the bytes decode lossily; the line still arrives.
#[cfg(not(windows))]
assert!(output.starts_with(char::REPLACEMENT_CHARACTER));
assert!(parse_message(b"{not json").is_none());
}
#[test]
fn formats_structured_errors() {
let msg =
@@ -342,16 +313,4 @@ mod tests {
assert_eq!(error_message(&msg), "cannot find a.py");
assert_eq!(error_message(&json!({"message": "nope"})), "nope");
}
#[test]
fn a_failed_launch_response_keeps_delves_too_old_error_verbatim() {
// The UI's repair offer keys off this exact wording (`missingAdapter`), so the
// conversion to the IPC error must not fold it into a generic message.
let message = "Failed to launch /p/api: Version of Delve is too old for Go version go1.27.0 \
(maximum supported version 1.26, suppress this error with --check-go-version=false)";
let e = ide_core::IdeError::from(DapError::Failed { command: "launch".into(), message: message.into() });
assert_eq!(e.kind, ide_core::error::ErrorKind::InvalidInput);
assert_eq!(e.message, format!("launch failed: {message}"));
assert!(e.message.contains("Version of Delve is too old for Go version go1.27.0"));
}
}
+12 -99
View File
@@ -12,35 +12,6 @@ use tokio::net::{TcpListener, TcpStream};
/// How long the relay waits for java-debug to attach once the JVM is connected.
const ATTACH_TIMEOUT: Duration = Duration::from_secs(60);
/// Loopback ports relaying a connected JVM, while their relay runs.
static RELAY_PORTS: std::sync::Mutex<Vec<u16>> = std::sync::Mutex::new(Vec::new());
fn relay_ports() -> std::sync::MutexGuard<'static, Vec<u16>> {
RELAY_PORTS.lock().unwrap_or_else(std::sync::PoisonError::into_inner)
}
/// Whether `port` is a relay of [`listen_for_jvm`]: a session attaching to it debugs a JVM that
/// connected to the IDE, so the JVM has no JDWP listener the memory view could find it by.
pub(crate) fn is_listen_relay(port: u16) -> bool {
relay_ports().contains(&port)
}
/// Registers a relay port for as long as it lives.
struct RelayPort(u16);
impl RelayPort {
fn new(port: u16) -> Self {
relay_ports().push(port);
Self(port)
}
}
impl Drop for RelayPort {
fn drop(&mut self) {
relay_ports().retain(|&p| p != self.0);
}
}
/// The address to listen on for `host`: the interface it names, or every interface when
/// it is not one of this machine's (a DNS name or a NAT address the JVM connects to).
async fn bind(host: &str, port: u16) -> IdeResult<TcpListener> {
@@ -69,11 +40,11 @@ pub async fn listen_for_jvm(host: &str, port: u16, cancel: CancellationToken) ->
let _ = jvm.set_nodelay(true);
let relay = TcpListener::bind(("127.0.0.1", 0)).await?;
let local = relay.local_addr()?.port();
tokio::spawn(relay_one(relay, jvm, RelayPort::new(local)));
tokio::spawn(relay_one(relay, jvm));
Ok(local)
}
async fn relay_one(relay: TcpListener, mut jvm: TcpStream, _registered: RelayPort) {
async fn relay_one(relay: TcpListener, mut jvm: TcpStream) {
let debugger = match tokio::time::timeout(ATTACH_TIMEOUT, relay.accept()).await {
Ok(Ok((s, _))) => s,
_ => return,
@@ -87,64 +58,22 @@ async fn relay_one(relay: TcpListener, mut jvm: TcpStream, _registered: RelayPor
}
#[cfg(test)]
pub(crate) mod tests {
mod tests {
use super::*;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
/// A remote JVM (`server=n`) connected to [`listen_for_jvm`], and the relay port. A port
/// picked by binding to 0 can be taken by a parallel test before `listen_for_jvm` binds it,
/// so a failed listen is retried on a fresh one.
pub(crate) async fn connected_jvm() -> (TcpStream, u16) {
for _ in 0..20 {
let port = TcpListener::bind(("127.0.0.1", 0)).await.unwrap().local_addr().unwrap().port();
if let Some(connected) = jvm_on(port).await {
return connected;
}
}
panic!("listen_for_jvm never accepted a JVM");
}
/// One attempt of [`connected_jvm`]; `None` when the port turned out to be taken.
async fn jvm_on(port: u16) -> Option<(TcpStream, u16)> {
let mut waiting = tokio::spawn(listen_for_jvm("127.0.0.1", port, CancellationToken::new()));
let mut jvm = None;
for _ in 0..100 {
// A listen that failed to bind leaves nothing to connect to.
if waiting.is_finished() {
break;
}
// macOS does not refuse a connect to a port a socket holds without listening: the
// SYN goes unanswered until the connect times out.
let connect = TcpStream::connect(("127.0.0.1", port));
if let Ok(Ok(s)) = tokio::time::timeout(Duration::from_secs(1), connect).await {
jvm = Some(s);
break;
}
tokio::time::sleep(Duration::from_millis(20)).await;
}
// A connection to another test's listener never reaches `listen_for_jvm`.
let relay = tokio::time::timeout(Duration::from_secs(2), &mut waiting).await;
if let (Some(jvm), Ok(Ok(Ok(relay)))) = (jvm, relay) {
return Some((jvm, relay));
}
waiting.abort();
None
}
#[tokio::test]
async fn a_port_another_socket_holds_is_given_up() {
// Bound but not listening, as a parallel test's outgoing connection holds its local port.
let held = tokio::net::TcpSocket::new_v4().unwrap();
held.bind(([127, 0, 0, 1], 0).into()).unwrap();
let port = held.local_addr().unwrap().port();
let attempt = tokio::time::timeout(Duration::from_secs(5), jvm_on(port)).await;
assert!(matches!(attempt, Ok(None)), "an attempt on a taken port gives up instead of waiting forever");
}
#[tokio::test]
async fn relays_the_jdwp_handshake_both_ways() {
let port = TcpListener::bind(("127.0.0.1", 0)).await.unwrap().local_addr().unwrap().port();
let waiting = tokio::spawn(listen_for_jvm("127.0.0.1", port, CancellationToken::new()));
// The remote JVM (`server=n`) connects to the IDE.
let (mut jvm, local) = connected_jvm().await;
let mut jvm = loop {
if let Ok(s) = TcpStream::connect(("127.0.0.1", port)).await {
break s;
}
tokio::time::sleep(Duration::from_millis(20)).await;
};
let local = waiting.await.unwrap().unwrap();
// java-debug attaches to the relay and starts the handshake.
let mut debugger = TcpStream::connect(("127.0.0.1", local)).await.unwrap();
debugger.write_all(b"JDWP-Handshake").await.unwrap();
@@ -156,22 +85,6 @@ pub(crate) mod tests {
assert_eq!(&buf, b"JDWP-Handshake");
}
#[tokio::test]
async fn relay_port_is_known_while_the_relay_runs() {
let (jvm, local) = connected_jvm().await;
assert!(is_listen_relay(local));
let debugger = TcpStream::connect(("127.0.0.1", local)).await.unwrap();
drop((jvm, debugger));
for _ in 0..200 {
if !is_listen_relay(local) {
return;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
panic!("relay port {local} still registered after the relay ended");
}
#[tokio::test]
async fn waiting_is_cancellable() {
let cancel = CancellationToken::new();
-92
View File
@@ -1,92 +0,0 @@
//! The class table counted by the debugger (JDWP `InstanceCounts`) where no histogram can be
//! taken: JDK 8-16 while suspended, Windows, a remote JVM. Only while suspended, and slow on a
//! large heap (seconds on JDK 17+).
use super::inspect::Call;
use super::snapshot::diff_rows;
use super::{ClassCounts, JvmMemoryResult, JvmMemorySnapshot, JvmMemorySource, JvmMemoryState, JvmMemoryUnavailable};
use crate::session::Session;
use ide_core::IdeResult;
use ide_jvm_attach::HistogramRow;
use ide_jvm_inspect::ClassCount;
/// Reasons the debugger may be able to work around; the others (the program exited, the JVM
/// did not answer) would fail for it too.
pub(super) fn falls_back(reason: &JvmMemoryUnavailable) -> bool {
use JvmMemoryUnavailable as U;
matches!(
reason,
U::UnsafeWhileSuspended { .. }
| U::UnsupportedPlatform
| U::AttachedJvmNotFound
| U::ListenerNotRunning
| U::ListenModeNotSupported
| U::NoJvmProcess
)
}
/// `histogram` unless it is unavailable for a reason `session`'s inspector can work around.
pub(super) async fn or_debugger_counts(
state: &mut JvmMemoryState,
histogram: Result<JvmMemorySnapshot, JvmMemoryUnavailable>,
session: Option<&Session>,
) -> IdeResult<JvmMemoryResult> {
let reason = match histogram {
Ok(snapshot) => return Ok(JvmMemoryResult::Ready { snapshot }),
Err(reason) => reason,
};
let unavailable = |can_count_when_paused| Ok(JvmMemoryResult::Unavailable { reason, can_count_when_paused });
let Some(session) = session.filter(|_| falls_back(&reason)) else {
return unavailable(false);
};
let call = match Call::begin(session, None).await {
Ok(call) => call,
Err(super::JvmInspectUnavailable::NotSuspended) => return unavailable(true),
Err(_) => return unavailable(false),
};
let started = std::time::Instant::now();
let result = call.run(call.inspector.class_counts(&call.target)).await;
match call.finish(result) {
Ok(counts) => {
let elapsed_ms = u32::try_from(started.elapsed().as_millis()).unwrap_or(u32::MAX);
let previous = state.baseline(JvmMemorySource::Debugger);
let (snapshot, counts) = debugger_snapshot(counts, previous.as_deref(), elapsed_ms);
state.remember(JvmMemorySource::Debugger, counts);
Ok(JvmMemoryResult::Ready { snapshot })
}
Err(e) => {
tracing::debug!("debugger class counts: {e}");
let running = !session.suspended.get();
unavailable(running && session.jvm_inspector.lock().is_some())
}
}
}
/// Classes without instances are left out; most instances first.
pub(super) fn debugger_snapshot(
counts: Vec<ClassCount>,
previous: Option<&ClassCounts>,
elapsed_ms: u32,
) -> (JvmMemorySnapshot, ClassCounts) {
let rows = counts
.into_iter()
.filter(|c| c.instances > 0)
.map(|c| HistogramRow { name: c.name, module: None, instances: c.instances, bytes: 0 })
.collect();
let (mut rows, counts) = diff_rows(rows, previous);
for row in &mut rows {
row.delta_bytes = None;
}
rows.sort_by(|a, b| b.instances.cmp(&a.instances).then_with(|| a.name.cmp(&b.name)));
let snapshot = JvmMemorySnapshot {
total_instances: rows.iter().map(|r| r.instances).sum(),
total_bytes: 0,
rows,
java_version: String::new(),
taken_while_suspended: true,
has_baseline: previous.is_some(),
elapsed_ms,
source: JvmMemorySource::Debugger,
};
(snapshot, counts)
}
-293
View File
@@ -1,293 +0,0 @@
//! A heap dump of a debugged JVM, written over its attach listener into a snapshot directory.
use super::snapshot::{AttachProbe, JdkProbe, attach_failure, gone, pick_jvm_pid};
use super::{
JvmMemoryState, JvmMemoryUnavailable, JvmTarget, histogram_pid, keep_found, lookup, resolve_jvm,
safe_while_suspended, session_target,
};
use crate::session::Session;
use ide_core::{CancellationToken, IdeError, IdeResult};
use ide_jvm_attach::{AttachError, HeapDumpReply};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::future::Future;
use std::path::{Path, PathBuf};
use std::sync::Arc;
const DUMP_FILE: &str = "dump.hprof";
/// For a refusal whose reply holds no reason: empty, or only the line announcing the dump.
const NO_REASON: &str = "the JVM wrote no heap dump and gave no reason";
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmHeapUnavailable {
/// The reason a class table could not be read applies to the dump too.
Memory {
reason: JvmMemoryUnavailable,
},
/// Bytes.
NotEnoughDiskSpace {
needed: u64,
available: u64,
},
DumpFailed {
message: String,
},
DumpTimedOut {
seconds: u32,
},
}
#[derive(Clone, Debug, PartialEq)]
pub struct JvmHeapDump {
pub path: PathBuf,
/// The class histogram taken just before the dump.
pub histogram: Vec<ide_heap::HistogramEntry>,
/// Raw `VM.flags`; empty when the JVM did not answer.
pub vm_flags: String,
/// 0 when unknown.
pub java_major: u32,
/// Empty when unknown.
pub java_version: String,
pub pid: u32,
pub taken_while_suspended: bool,
pub dump_ms: u32,
}
#[derive(Clone, Debug, PartialEq)]
pub enum JvmHeapOutcome {
Ready(JvmHeapDump),
/// The snapshot directory has been deleted.
Unavailable(JvmHeapUnavailable),
/// The snapshot directory has been deleted.
Cancelled,
}
pub(crate) async fn session_heap_dump(
session: Arc<Session>,
dir: PathBuf,
cancel: CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
let launched = histogram_pid(&session);
let capture = {
let (dir, cancel) = (dir.clone(), cancel.clone());
async move {
let target = session_target(&session, launched);
let probe = |suspended| -> Arc<dyn AttachProbe> { Arc::new(JdkProbe { suspended }) };
let stopped = session.clone();
let suspended = move || capture_suspended(&stopped);
heap_dump_with(&session.jvm_memory, suspended, probe, target, dir, cancel).await
}
};
detached(capture, dir, cancel).await
}
/// The JVM can be suspended before the adapter reports the stop, so a session with a JDWP link
/// also counts the link's view.
pub(super) fn capture_suspended(session: &Session) -> bool {
session.suspended.get()
|| session
.jdwp_link
.lock()
.as_ref()
.is_some_and(|link| matches!(link.state(), ide_jdwp::LinkState::Suspended { .. }))
}
/// Runs `capture` in a task of its own, so a caller that stops waiting cannot release the
/// session's lock while the JVM is still writing the dump and let a class-table request run
/// next to it. On `cancel` returns at once and deletes `dir`.
pub(super) async fn detached(
capture: impl Future<Output = IdeResult<JvmHeapOutcome>> + Send + 'static,
dir: PathBuf,
cancel: CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
let task = tokio::spawn(capture);
tokio::select! {
joined = task => joined.map_err(|e| IdeError::internal(format!("JVM heap dump failed: {e}")))?,
() = cancel.cancelled() => {
discard_later(dir).await;
Ok(JvmHeapOutcome::Cancelled)
}
}
}
/// Holds `state`'s lock throughout, as a class-table request does; `suspended` and then `probe`
/// are read once it is held, and `suspended` again by [`take_heap_dump`]. Anything but `Ready`
/// deletes `dir`.
pub(super) async fn heap_dump_with(
state: &tokio::sync::Mutex<JvmMemoryState>,
suspended: impl Fn() -> bool + Send + Sync + 'static,
probe: impl FnOnce(bool) -> Arc<dyn AttachProbe>,
target: JvmTarget<'_>,
dir: PathBuf,
cancel: CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
// The capture's histogram never becomes the Memory view's baseline, so the state is unused.
let _turn = state.lock().await;
let outcome = capture(suspended, probe, target, dir.clone(), cancel).await;
if !matches!(outcome, Ok(JvmHeapOutcome::Ready(_))) {
discard_later(dir).await;
}
outcome
}
async fn capture(
suspended_now: impl Fn() -> bool + Send + Sync + 'static,
probe: impl FnOnce(bool) -> Arc<dyn AttachProbe>,
target: JvmTarget<'_>,
dir: PathBuf,
cancel: CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
if cancel.is_cancelled() {
return Ok(JvmHeapOutcome::Cancelled);
}
let suspended = suspended_now();
let lookup = match lookup(target) {
Ok(lookup) => lookup,
Err(reason) => return Ok(memory(reason)),
};
let probe = probe(suspended);
let (found, outcome) = tokio::task::spawn_blocking(move || match resolve_jvm(lookup, &*probe, suspended) {
Ok((tree, found)) => (found, take_heap_dump(&*probe, &tree, suspended, &suspended_now, &dir, &cancel)),
Err(reason) => (None, Ok(memory(reason))),
})
.await
.map_err(|e| IdeError::internal(format!("JVM heap dump failed: {e}")))?;
keep_found(target, found);
outcome
}
/// `tree` starts with the session's reported pid, followed by its descendants. `suspended` is
/// the stop read when the capture began; `suspended_now` reads it again. Anything but `Ready`
/// deletes `dir`.
pub(crate) fn take_heap_dump(
probe: &dyn AttachProbe,
tree: &[i32],
suspended: bool,
suspended_now: &dyn Fn() -> bool,
dir: &Path,
cancel: &CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
let outcome = dump(probe, tree, suspended, suspended_now, dir, cancel);
if !matches!(outcome, Ok(JvmHeapOutcome::Ready(_))) {
discard(dir);
}
outcome
}
fn dump(
probe: &dyn AttachProbe,
tree: &[i32],
suspended: bool,
suspended_now: &dyn Fn() -> bool,
dir: &Path,
cancel: &CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
use JvmHeapUnavailable as H;
use JvmMemoryUnavailable as U;
let Some(root) = tree.first().and_then(|&p| u32::try_from(p).ok()) else {
return Ok(memory(U::NoJvmProcess));
};
let Some(jvm) = pick_jvm_pid(tree, |p| probe.has_listener(p)) else {
return Ok(memory(gone(probe, root)));
};
let version = probe.version(jvm);
let java_major = version.as_ref().map_or(0, |v| v.major);
let safe = safe_while_suspended(version.as_ref());
let unsafe_stop = || Ok(memory(U::UnsafeWhileSuspended { java_major }));
if suspended && !safe {
return unsafe_stop();
}
// The program may stop during the histogram, and a stopped JDK below 17 hangs on the next
// request until it resumes.
let stopped_since = || !safe && suspended_now();
let text = match probe.histogram(jvm) {
Ok(text) => text,
Err(e) => return attach_failure(probe, jvm, e, "class histogram").map(memory),
};
let histogram: Vec<ide_heap::HistogramEntry> = ide_jvm_attach::parse_class_histogram(&text)
.rows
.into_iter()
.map(|r| ide_heap::HistogramEntry { name: r.name, instances: r.instances, bytes: r.bytes })
.collect();
let (instances, bytes) =
histogram.iter().fold((0u64, 0u64), |(i, b), e| (i.saturating_add(e.instances), b.saturating_add(e.bytes)));
let needed = ide_heap::capture_need(bytes, instances);
if let Some(available) = probe.free_bytes(dir)
&& available < needed
{
return Ok(JvmHeapOutcome::Unavailable(H::NotEnoughDiskSpace { needed, available }));
}
if stopped_since() {
return unsafe_stop();
}
// Without flags the index assumes the default object layout.
let vm_flags = probe.vm_flags(jvm).unwrap_or_else(|e| {
tracing::debug!("VM.flags of JVM {jvm}: {e}");
String::new()
});
if cancel.is_cancelled() {
return Ok(JvmHeapOutcome::Cancelled);
}
if stopped_since() {
return unsafe_stop();
}
let path = dir.join(DUMP_FILE);
let timeout = ide_jvm_attach::dump_timeout(bytes);
let started = std::time::Instant::now();
let reply = probe.heap_dump(jvm, &path, timeout);
let dump_ms = u32::try_from(started.elapsed().as_millis()).unwrap_or(u32::MAX);
if cancel.is_cancelled() {
return Ok(JvmHeapOutcome::Cancelled);
}
let failed = |message: String| Ok(JvmHeapOutcome::Unavailable(H::DumpFailed { message }));
match reply {
Ok(HeapDumpReply::Created { .. }) => {
if !std::fs::metadata(&path).is_ok_and(|m| m.is_file() && m.len() > 0) {
return failed(format!("the JVM reported a heap dump, but {} is missing or empty", path.display()));
}
Ok(JvmHeapOutcome::Ready(JvmHeapDump {
path,
histogram,
vm_flags,
java_major,
java_version: version.map(|v| v.text).unwrap_or_default(),
pid: jvm,
taken_while_suspended: suspended,
dump_ms,
}))
}
Ok(HeapDumpReply::Refused { message }) if message.is_empty() || message.starts_with("Dumping heap to") => {
failed(NO_REASON.into())
}
Ok(HeapDumpReply::Refused { message }) => failed(message),
Err(AttachError::Timeout) => {
let seconds = u32::try_from(timeout.as_secs()).unwrap_or(u32::MAX);
Ok(JvmHeapOutcome::Unavailable(H::DumpTimedOut { seconds }))
}
// A running JDK rejects an argument with a non-zero result code.
Err(AttachError::Command { message, .. }) => failed(message),
Err(e) => attach_failure(probe, jvm, e, "heap dump").map(memory),
}
}
fn memory(reason: JvmMemoryUnavailable) -> JvmHeapOutcome {
JvmHeapOutcome::Unavailable(JvmHeapUnavailable::Memory { reason })
}
fn discard(dir: &Path) {
if let Err(e) = std::fs::remove_dir_all(dir)
&& e.kind() != std::io::ErrorKind::NotFound
{
tracing::warn!("could not delete heap snapshot directory {}: {e}", dir.display());
}
}
/// A dump may be large, so it is deleted off the async threads.
async fn discard_later(dir: PathBuf) {
let _ = tokio::task::spawn_blocking(move || discard(&dir)).await;
}
-251
View File
@@ -1,251 +0,0 @@
//! A suspended JVM's instances, fields and referrers, read through the session's
//! [`JvmInspector`]. An object is only valid within the stop it was read in.
use super::launch_pid;
use crate::session::Session;
use ide_core::event::{DebugEvent, JvmBackendState};
use ide_core::{IdeError, IdeResult};
use ide_jvm_inspect::{
DEFAULT_REFERRERS, InspectError, InspectFuture, JvmField, JvmInspector, JvmReferrer, JvmValue, MAX_INSTANCES,
MAX_REFERRERS, ObjectId, VmTarget,
};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::sync::Arc;
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct JvmObjectRef {
/// The `generation` of the [`JvmInstancesResult`] the object came from.
pub generation: u64,
pub id: ObjectId,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmInspectUnavailable {
/// The session has no inspector: not a JVM session, not ready yet, or turned off.
NoBackend,
/// The debug adapter lacks what the inspector needs; turned off for the session.
Unsupported {
detail: String,
},
NotSuspended,
/// The object is from an earlier stop, or the program stopped or resumed meanwhile.
Stale,
Collected,
ClassNotLoaded,
VmNotFound,
/// Transient: ask again.
TimedOut,
BackendClosed,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "status", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmInstancesResult {
Ready {
/// Goes into the [`JvmObjectRef`] of each instance.
generation: u64,
instances: Vec<JvmValue>,
/// The limit was reached; there may be more.
truncated: bool,
},
Unavailable {
reason: JvmInspectUnavailable,
},
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "status", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmFieldsResult {
Ready { fields: Vec<JvmField> },
Unavailable { reason: JvmInspectUnavailable },
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "status", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmReferrersResult {
Ready { referrers: Vec<JvmReferrer>, truncated: bool },
Unavailable { reason: JvmInspectUnavailable },
}
/// The session's JVM for the inspector: java-debug's VM is found by the launched program's pid
/// or by the port of an attach (the thread lists of two JVMs can be identical).
pub(crate) fn vm_target(s: &Session) -> VmTarget {
let (pid, attach) = if s.request == "launch" {
(u32::try_from(launch_pid(s)).ok().filter(|&p| p > 0), None)
} else {
(None, s.attach_target.get())
};
VmTarget {
pid,
attach_host: attach.map(|(host, _)| host.clone()).filter(|h| !h.is_empty()),
attach_port: attach.map(|&(_, port)| port),
generation: s.suspended.ticket(),
}
}
pub(crate) fn set_inspector(s: &Session, inspector: Arc<dyn JvmInspector>) {
*s.jvm_inspector.lock() = Some(inspector);
s.emit(DebugEvent::Capabilities);
}
pub(crate) async fn instances(s: &Session, class_name: &str, limit: u32) -> IdeResult<JvmInstancesResult> {
let limit = limit.clamp(1, MAX_INSTANCES);
let call = match Call::begin(s, None).await {
Ok(call) => call,
Err(reason) => return Ok(JvmInstancesResult::Unavailable { reason }),
};
let generation = call.target.generation;
let result = call.run(call.inspector.instances(&call.target, class_name, limit)).await;
Ok(match settle(call.finish(result))? {
Ok(instances) => {
JvmInstancesResult::Ready { generation, truncated: instances.len() >= limit as usize, instances }
}
Err(reason) => JvmInstancesResult::Unavailable { reason },
})
}
pub(crate) async fn fields(s: &Session, object: JvmObjectRef) -> IdeResult<JvmFieldsResult> {
let call = match Call::begin(s, Some(object.generation)).await {
Ok(call) => call,
Err(reason) => return Ok(JvmFieldsResult::Unavailable { reason }),
};
let result = call.run(call.inspector.fields(&call.target, object.id)).await;
Ok(match settle(call.finish(result))? {
Ok(fields) => JvmFieldsResult::Ready { fields },
Err(reason) => JvmFieldsResult::Unavailable { reason },
})
}
pub(crate) async fn referrers(s: &Session, object: JvmObjectRef, limit: u32) -> IdeResult<JvmReferrersResult> {
let limit = if limit == 0 { DEFAULT_REFERRERS } else { limit.min(MAX_REFERRERS) };
let call = match Call::begin(s, Some(object.generation)).await {
Ok(call) => call,
Err(reason) => return Ok(JvmReferrersResult::Unavailable { reason }),
};
let result = call.run(call.inspector.referrers(&call.target, object.id, limit)).await;
Ok(match settle(call.finish(result))? {
Ok(referrers) => JvmReferrersResult::Ready { truncated: referrers.len() >= limit as usize, referrers },
Err(reason) => JvmReferrersResult::Unavailable { reason },
})
}
/// One inspector request. Holds the session's inspect lock and a clone of its inspector, which
/// [`Call::run`] lets go of when the session ends.
pub(super) struct Call<'s> {
session: &'s Session,
pub(super) inspector: Arc<dyn JvmInspector>,
pub(super) target: VmTarget,
_turn: tokio::sync::MutexGuard<'s, ()>,
}
impl<'s> Call<'s> {
/// `object_generation`: the stop the request's object was read in.
pub(super) async fn begin(
s: &'s Session,
object_generation: Option<u64>,
) -> Result<Call<'s>, JvmInspectUnavailable> {
// Checked under the lock: a request queued behind another must see the state it runs in.
let turn = s.inspect_lock.lock().await;
let Some(inspector) = s.jvm_inspector.lock().clone() else {
return Err(JvmInspectUnavailable::NoBackend);
};
if !s.suspended.get() {
return Err(JvmInspectUnavailable::NotSuspended);
}
let target = vm_target(s);
if object_generation.is_some_and(|g| g != target.generation) {
return Err(JvmInspectUnavailable::Stale);
}
Ok(Call { session: s, inspector, target, _turn: turn })
}
pub(super) async fn run<T>(&self, request: InspectFuture<'_, T>) -> Result<T, InspectError> {
tokio::select! {
result = request => result,
() = self.session.ended.cancelled() => Err(InspectError::Closed),
}
}
/// The result unless the program stopped or resumed while it was read: a request of several
/// commands can straddle a stop. `Unsupported` turns the backend off.
pub(super) fn finish<T>(self, result: Result<T, InspectError>) -> Result<T, InspectError> {
let s = self.session;
match result {
Err(InspectError::Unsupported(detail)) => {
turn_off(s, &self.inspector, &detail);
Err(InspectError::Unsupported(detail))
}
_ if !s.suspended.get() || s.suspended.ticket() != self.target.generation => Err(InspectError::Stale),
result => result,
}
}
}
/// An adapter that lacks what the inspector needs will not gain it during the session.
fn turn_off(s: &Session, inspector: &Arc<dyn JvmInspector>, detail: &str) {
let removed = {
let mut slot = s.jvm_inspector.lock();
// Not one installed since this request began.
let current = slot.as_ref().is_some_and(|i| std::ptr::addr_eq(Arc::as_ptr(i), Arc::as_ptr(inspector)));
current && slot.take().is_some()
};
if removed {
s.emit(DebugEvent::JvmBackend { state: JvmBackendState::Unsupported { detail: detail.to_owned() } });
s.emit(DebugEvent::Capabilities);
}
}
/// A failure the inspector could not classify is an error, not a reason shown to the user.
fn settle<T>(result: Result<T, InspectError>) -> IdeResult<Result<T, JvmInspectUnavailable>> {
use JvmInspectUnavailable as U;
let error = match result {
Ok(value) => return Ok(Ok(value)),
Err(error) => error,
};
Ok(Err(match error {
InspectError::NotSuspended => U::NotSuspended,
InspectError::Stale => U::Stale,
InspectError::Collected => U::Collected,
InspectError::ClassNotLoaded => U::ClassNotLoaded,
InspectError::VmNotFound => U::VmNotFound,
InspectError::Unsupported(detail) => U::Unsupported { detail },
// Also the Kotlin link's `Busy`: the backend stays.
InspectError::TimedOut => U::TimedOut,
InspectError::Closed => U::BackendClosed,
InspectError::Failed(message) => return Err(IdeError::internal(format!("JVM inspector: {message}"))),
}))
}
/// Reports a Kotlin link that stops inspecting or closes. Holds neither the link nor the
/// session: it ends once the session drops the link, and never drops it itself (after
/// `VirtualMachine.Dispose` the link is closed but still forwarding the reply).
pub(crate) fn watch_link(s: &Arc<Session>, link: &ide_jdwp::JdwpLink) {
let mut states = link.subscribe();
let session = Arc::downgrade(s);
tokio::spawn(async move {
let mut reported = None;
loop {
let state = match &*states.borrow_and_update() {
ide_jdwp::LinkState::Degraded(detail) => Some(JvmBackendState::Degraded { detail: detail.clone() }),
ide_jdwp::LinkState::Closed => Some(JvmBackendState::Closed),
_ => None,
};
if let Some(state) = state
&& reported.as_ref() != Some(&state)
{
let Some(s) = session.upgrade() else { return };
if s.ended.is_cancelled() || s.terminated.load(std::sync::atomic::Ordering::Acquire) {
return;
}
s.emit(DebugEvent::JvmBackend { state: state.clone() });
reported = Some(state);
}
if states.changed().await.is_err() {
return;
}
}
});
}
-338
View File
@@ -1,338 +0,0 @@
//! Class-table snapshots of a debugged JVM, read over the HotSpot attach listener or, where
//! that is unavailable, counted by the debugger; and the JVM's objects through the debugger.
mod fallback;
mod heap;
mod inspect;
mod scan;
mod snapshot;
#[cfg(test)]
pub(crate) mod test_support;
#[cfg(test)]
mod tests;
use crate::session::Session;
use crate::session::Suspension;
use ide_core::IdeResult;
use serde::{Deserialize, Serialize};
use specta::Type;
use std::collections::HashMap;
use std::sync::Arc;
use std::sync::atomic::{AtomicI64, Ordering};
use std::time::Duration;
pub use heap::*;
pub use inspect::{JvmFieldsResult, JvmInspectUnavailable, JvmInstancesResult, JvmObjectRef, JvmReferrersResult};
pub(crate) use inspect::{fields, instances, referrers, set_inspector, watch_link};
#[cfg(all(test, unix))]
pub(crate) use scan::watch_for_attached_jvm;
pub(crate) use scan::{JvmFinder, attach_target, find_attached_jvm, find_local_jvm, local_jdwp_port};
use snapshot::{AttachProbe, JdkProbe, take_snapshot};
pub const JVM_MEMORY_ADAPTERS: &[&str] = &["java-debug", crate::kotlin::ADAPTER, crate::kotlin_lsp::ADAPTER];
pub const HISTOGRAM_TIMEOUT: Duration = Duration::from_secs(10);
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct JvmClassRow {
pub name: String,
pub module: Option<String>,
pub instances: u64,
pub bytes: u64,
/// Change since the previous snapshot; `None` on the first one and for ambiguous rows.
pub delta_instances: Option<i64>,
pub delta_bytes: Option<i64>,
/// Several classes share this name (different class loaders), so it has no delta.
pub ambiguous: bool,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct JvmMemorySnapshot {
/// Largest `bytes` first.
pub rows: Vec<JvmClassRow>,
pub total_instances: u64,
pub total_bytes: u64,
/// Empty when unknown.
pub java_version: String,
pub taken_while_suspended: bool,
/// A previous snapshot from the same source exists; deltas are against it.
pub has_baseline: bool,
/// Wall time of the histogram or count request.
pub elapsed_ms: u32,
pub source: JvmMemorySource,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum JvmMemorySource {
/// `GC.class_histogram -all`: sizes, and objects not yet collected.
Histogram,
/// JDWP `InstanceCounts`: reachable objects only, no sizes (`bytes` is 0).
Debugger,
}
#[derive(Clone, Copy, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmMemoryUnavailable {
/// An attach session whose JVM is remote, or local but not found by its JDWP port.
AttachedJvmNotFound,
/// An attach through the IDE's listen-mode relay: the JVM connected to the IDE, so it has
/// no JDWP listener to be found by.
ListenModeNotSupported,
UnsupportedPlatform,
NoJvmProcess,
ListenerNotRunning,
/// `java_major` is 0 when the version is unknown.
UnsafeWhileSuspended {
java_major: u32,
},
ProgramExited,
TimedOut,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(tag = "status", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmMemoryResult {
Ready {
snapshot: JvmMemorySnapshot,
},
Unavailable {
reason: JvmMemoryUnavailable,
/// The debugger could count the classes if the program were paused.
can_count_when_paused: bool,
},
}
/// Instances and bytes per class name, for the next snapshot's deltas; `None` for a name
/// several classes shared, so it gets no delta next time either.
type ClassCounts = HashMap<String, Option<(u64, u64)>>;
/// Per-session baseline; its mutex also keeps one class-table request per session at a time.
#[derive(Default)]
pub struct JvmMemoryState {
/// Shared with the blocking task, so a task that panics leaves it in place.
previous: Option<(JvmMemorySource, Arc<ClassCounts>)>,
}
impl JvmMemoryState {
/// The previous snapshot's counts if it came from `source`: counts from the other source
/// are not comparable (only reachable objects versus all of them).
pub(crate) fn baseline(&self, source: JvmMemorySource) -> Option<Arc<ClassCounts>> {
self.previous.as_ref().filter(|(s, _)| *s == source).map(|(_, counts)| counts.clone())
}
pub(crate) fn remember(&mut self, source: JvmMemorySource, counts: ClassCounts) {
self.previous = Some((source, Arc::new(counts)));
}
}
pub(crate) async fn session_classes(session: Arc<Session>) -> IdeResult<JvmMemoryResult> {
until_located(&session).await;
let launched = histogram_pid(&session);
// Detached, so a caller that gives up cannot release the lock while the JVM is still
// answering and let a second histogram request run next to it.
tokio::spawn(async move {
let target = session_target(&session, launched);
let probe = |suspended| -> Arc<dyn AttachProbe> { Arc::new(JdkProbe { suspended }) };
snapshot(&session.jvm_memory, &session.suspended, probe, target, Some(&session)).await
})
.await
.map_err(|e| ide_core::IdeError::internal(format!("JVM memory snapshot failed: {e}")))?
}
/// Bounds the wait of [`until_located`]; `configure` has its own timeouts, this only guards them.
const LOCATE_WAIT: Duration = Duration::from_secs(30);
/// Until `configure` has recorded where `session`'s JVM is. The Memory view asks as soon as the
/// session appears; looked up earlier, an attached JVM is reported as missing.
async fn until_located(session: &Session) {
let mut located = session.jvm_located.subscribe();
tokio::select! {
_ = located.wait_for(|v| *v) => {}
() = session.ended.cancelled() => {}
() = tokio::time::sleep(LOCATE_WAIT) => {}
}
}
/// `launched`: [`histogram_pid`] of `session`.
fn session_target(session: &Session, launched: Result<Option<i32>, JvmMemoryUnavailable>) -> JvmTarget<'_> {
match launched {
Ok(Some(pid)) => JvmTarget::Launched(pid),
Ok(None) => JvmTarget::Attached { pid: &session.jvm_pid, jdwp_port: session.jdwp_port.get().copied() },
Err(reason) => JvmTarget::Unavailable(reason),
}
}
/// The pid of a launch session's program (`None` for an attach session), or why no histogram
/// can be taken.
fn histogram_pid(session: &Session) -> Result<Option<i32>, JvmMemoryUnavailable> {
if !JVM_MEMORY_ADAPTERS.contains(&session.adapter.as_str()) {
return Err(JvmMemoryUnavailable::NoJvmProcess);
}
if !cfg!(unix) {
return Err(JvmMemoryUnavailable::UnsupportedPlatform);
}
if session.request == "launch" {
return match i32::try_from(launch_pid(session)).ok().filter(|&p| p > 0) {
Some(pid) => Ok(Some(pid)),
None => Err(JvmMemoryUnavailable::NoJvmProcess),
};
}
if session.jvm_pid.load(Ordering::Acquire) <= 0 {
if session.terminated.load(Ordering::Acquire) {
return Err(JvmMemoryUnavailable::ProgramExited);
}
if session.listen_relay.load(Ordering::Acquire) {
return Err(JvmMemoryUnavailable::ListenModeNotSupported);
}
}
Ok(None)
}
/// The JVM a launch session started: the one the IDE started for an app-launched (Kotlin)
/// session, else the program the adapter reported; 0 when neither is known.
pub(crate) fn launch_pid(session: &Session) -> i64 {
match session.jvm_pid.load(Ordering::Acquire) {
p if p > 0 => p,
_ => session.debuggee_pid.load(Ordering::Acquire),
}
}
#[derive(Clone, Copy)]
pub(crate) enum JvmTarget<'a> {
/// The program a launch session started: its pid and everything it started.
Launched(i32),
/// The JVM of an attach session. While `pid` is 0 it is looked up by `jdwp_port`, which is
/// `None` for a remote JVM.
Attached { pid: &'a AtomicI64, jdwp_port: Option<u16> },
/// Known without asking any JVM: no histogram can be taken, for this reason.
Unavailable(JvmMemoryUnavailable),
}
enum Lookup {
Tree(i32),
Known(i32),
Scan(u16),
}
/// `probe` is built from the suspension read once the lock is held: a request queued behind
/// another must not decide on a state from before the program hit a breakpoint. Where no
/// histogram can be taken, `debugger` (the session) may count the classes instead.
pub(crate) async fn snapshot(
state: &tokio::sync::Mutex<JvmMemoryState>,
suspended: &Suspension,
probe: impl FnOnce(bool) -> Arc<dyn AttachProbe>,
target: JvmTarget<'_>,
debugger: Option<&Session>,
) -> IdeResult<JvmMemoryResult> {
let mut state = state.lock().await;
let outcome = histogram(&mut state, suspended, probe, target).await?;
fallback::or_debugger_counts(&mut state, outcome, debugger).await
}
async fn histogram(
state: &mut JvmMemoryState,
suspended: &Suspension,
probe: impl FnOnce(bool) -> Arc<dyn AttachProbe>,
target: JvmTarget<'_>,
) -> IdeResult<Result<JvmMemorySnapshot, JvmMemoryUnavailable>> {
let suspended = suspended.get();
let lookup = match lookup(target) {
Ok(lookup) => lookup,
Err(reason) => return Ok(Err(reason)),
};
let probe = probe(suspended);
let previous = state.baseline(JvmMemorySource::Histogram);
let (found, outcome) = tokio::task::spawn_blocking(move || match resolve_jvm(lookup, &*probe, suspended) {
Ok((tree, found)) => (found, take_snapshot(&*probe, &tree, suspended, previous.as_deref())),
Err(reason) => (None, Ok(Err(reason))),
})
.await
.map_err(|e| ide_core::IdeError::internal(format!("JVM memory snapshot failed: {e}")))?;
keep_found(target, found);
Ok(outcome?.map(|(snapshot, counts)| {
state.remember(JvmMemorySource::Histogram, counts);
snapshot
}))
}
/// Where to look for `target`'s JVM, or why there is none to ask.
fn lookup(target: JvmTarget<'_>) -> Result<Lookup, JvmMemoryUnavailable> {
match target {
JvmTarget::Launched(pid) => Ok(Lookup::Tree(pid)),
JvmTarget::Attached { pid, jdwp_port } => match i32::try_from(pid.load(Ordering::Acquire)) {
Ok(p) if p > 0 => Ok(Lookup::Known(p)),
_ => jdwp_port.map(Lookup::Scan).ok_or(JvmMemoryUnavailable::AttachedJvmNotFound),
},
JvmTarget::Unavailable(reason) => Err(reason),
}
}
/// Blocking. The pids to pick the JVM from, and the pid a scan found (for [`keep_found`]).
fn resolve_jvm(
lookup: Lookup,
probe: &dyn AttachProbe,
suspended: bool,
) -> Result<(Vec<i32>, Option<i32>), JvmMemoryUnavailable> {
match lookup {
Lookup::Tree(pid) => Ok((pid_tree(pid), None)),
Lookup::Known(pid) => Ok((vec![pid], None)),
Lookup::Scan(port) => scan_for_jvm(probe, port, suspended).map(|pid| (vec![pid], Some(pid))),
}
}
fn keep_found(target: JvmTarget<'_>, found: Option<i32>) {
if let (Some(found), JvmTarget::Attached { pid, .. }) = (found, target) {
// The attach-time scan may have found it meanwhile; either answer is the same JVM.
let _ = pid.compare_exchange(0, i64::from(found), Ordering::AcqRel, Ordering::Acquire);
}
}
/// A suspended JDK below 17 hangs on an attach request, so while suspended only a JVM known to
/// be 17 or later is asked anything. Pass a version read without attaching (the `release` file).
fn safe_while_suspended(version: Option<&ide_jvm_attach::JavaVersion>) -> bool {
version.is_some_and(|v| ide_jvm_attach::histogram_safe_while_suspended(v.major))
}
/// The JVM listening on JDWP `port`. While suspended the scan reaches the session's own JVM
/// too, so JVMs unsafe to ask are skipped, and when one of them may be the session's the
/// answer is a refusal rather than "not found".
fn scan_for_jvm(probe: &dyn AttachProbe, port: u16, suspended: bool) -> Result<i32, JvmMemoryUnavailable> {
// Majors of the JVMs left out; 0 for an unknown version.
let skipped = std::cell::RefCell::new(std::collections::BTreeSet::new());
let may_query = |pid: u32| {
if !suspended {
return true;
}
let version = probe.version(pid);
let safe = safe_while_suspended(version.as_ref());
if !safe {
skipped.borrow_mut().insert(version.map_or(0, |v| v.major));
}
safe
};
match probe.find_by_jdwp_port(port, &may_query).and_then(|p| i32::try_from(p).ok()) {
Some(pid) if pid > 0 => Ok(pid),
_ => {
let skipped = skipped.into_inner();
Err(match skipped.first() {
None => JvmMemoryUnavailable::AttachedJvmNotFound,
Some(&major) => JvmMemoryUnavailable::UnsafeWhileSuspended {
java_major: if skipped.len() == 1 { major } else { 0 },
},
})
}
}
}
fn pid_tree(pid: i32) -> Vec<i32> {
#[cfg(unix)]
{
ide_core::process::process_tree(pid)
}
#[cfg(not(unix))]
{
vec![pid]
}
}
-146
View File
@@ -1,146 +0,0 @@
//! Finding the JVM behind an attach session by its JDWP port.
use super::safe_while_suspended;
use crate::session::Session;
use serde_json::Value;
use std::net::IpAddr;
use std::sync::Arc;
use std::sync::atomic::Ordering;
use std::time::Duration;
/// One try of the scan at attach time, and the limit per socket within it: a silent socket's
/// query thread lives that long, so a longer per-socket limit piles threads up across tries.
const ATTACH_SCAN_TRY: Duration = Duration::from_millis(500);
const ATTACH_SCAN_INTERVAL: Duration = Duration::from_millis(150);
/// The JVM's attach socket can appear a little after the JDWP "Listening" banner, which is
/// when the IDE attaches.
const ATTACH_SCAN_WINDOW: Duration = Duration::from_secs(3);
/// The single scan a refresh makes while the attach-time one found nothing.
pub(super) const RESCAN_PER_SOCKET: Duration = Duration::from_secs(1);
pub(super) const RESCAN_BUDGET: Duration = Duration::from_secs(2);
/// JDWP port of an attach to this machine (`hostName` loopback or absent): only such a JVM has
/// an attach socket the IDE can find.
pub(crate) fn local_jdwp_port(arguments: &Value) -> Option<u16> {
let local = match arguments.get("hostName") {
None | Some(Value::Null) => true,
Some(Value::String(host)) => {
let host = host.trim();
let ip = host.strip_prefix('[').and_then(|h| h.strip_suffix(']')).unwrap_or(host);
host.eq_ignore_ascii_case("localhost") || ip.parse::<IpAddr>().is_ok_and(|ip| ip.is_loopback())
}
Some(_) => false,
};
jdwp_port(arguments).filter(|_| local)
}
/// `port` of an attach request: a number or a numeric string, never 0.
fn jdwp_port(arguments: &Value) -> Option<u16> {
let port = match arguments.get("port")? {
Value::Number(n) => n.as_u64().and_then(|p| u16::try_from(p).ok()),
Value::String(s) => s.trim().parse().ok(),
_ => None,
};
port.filter(|&p| p != 0)
}
/// `hostName` (empty when absent) and `port` of an attach request, local or not.
pub(crate) fn attach_target(arguments: &Value) -> Option<(String, u16)> {
let host = arguments.get("hostName").and_then(Value::as_str).unwrap_or_default();
Some((host.to_owned(), jdwp_port(arguments)?))
}
/// Pid of the local JVM listening on a JDWP port (first argument), given whether the session is
/// suspended (second). Replaceable so tests never send attach requests to real JVMs.
pub(crate) type JvmFinder = Arc<dyn Fn(u16, bool) -> Option<u32> + Send + Sync>;
/// The [`JvmFinder`] of the attach-time scan.
pub(crate) fn find_local_jvm(port: u16, suspended: bool) -> Option<u32> {
let may_query = |pid| !suspended || safe_while_suspended(ide_jvm_attach::java_version_of(pid).as_ref());
ide_jvm_attach::find_jvm_by_jdwp_port(port, ATTACH_SCAN_TRY, ATTACH_SCAN_TRY, may_query)
}
/// Looks for the JVM behind a local attach: one try now, the rest in the background, so a JVM
/// that never opens an attach socket holds the attach up by one try at most. Call it before the
/// debugger connects: the JVM clears its JDWP listener address then, and a scan can no longer
/// tell it by its port.
pub(crate) async fn find_attached_jvm(session: &Arc<Session>, port: u16, finder: JvmFinder) {
let _ = session.jdwp_port.set(port);
let find = move |suspended: bool| finder(port, suspended);
let first = find.clone();
if let Ok(Some(pid)) = tokio::task::spawn_blocking(move || first(false)).await {
let _ = session.jvm_pid.compare_exchange(0, i64::from(pid), Ordering::AcqRel, Ordering::Acquire);
return;
}
drop(watch_for_attached_jvm(session, port, find, ATTACH_SCAN_INTERVAL, ATTACH_SCAN_WINDOW));
}
/// Retries `find` for the session's `jvm_pid`, holding the session weakly so an ended session
/// is not kept alive. `find` is told whether the session is suspended.
pub(crate) fn watch_for_attached_jvm<F>(
session: &Arc<Session>,
port: u16,
find: F,
interval: Duration,
window: Duration,
) -> tokio::task::JoinHandle<()>
where
F: Fn(bool) -> Option<u32> + Clone + Send + 'static,
{
let weak = Arc::downgrade(session);
tokio::spawn(async move {
let gate = || match weak.upgrade() {
Some(s) if !s.terminated.load(Ordering::Acquire) && !s.conn.is_closed() => {
if s.jvm_pid.load(Ordering::Acquire) > 0 {
ScanGate::Ended
} else {
ScanGate::Scan { suspended: s.suspended.get() }
}
}
_ => ScanGate::Ended,
};
let Some(pid) = retry_find(find, gate, interval, window).await else {
tracing::debug!("no local JVM found for JDWP port {port} yet");
return;
};
if let Some(s) = weak.upgrade() {
let _ = s.jvm_pid.compare_exchange(0, i64::from(pid), Ordering::AcqRel, Ordering::Acquire);
}
})
}
#[derive(Clone, Copy, Debug, PartialEq)]
pub(crate) enum ScanGate {
Scan { suspended: bool },
Ended,
}
/// Calls `find` on a blocking thread, on each try `gate` allows, until it returns a pid, `gate`
/// reports the end or `window` has passed.
pub(crate) async fn retry_find<F>(
find: F,
gate: impl Fn() -> ScanGate,
interval: Duration,
window: Duration,
) -> Option<u32>
where
F: Fn(bool) -> Option<u32> + Clone + Send + 'static,
{
let deadline = tokio::time::Instant::now() + window;
loop {
match gate() {
ScanGate::Ended => break,
ScanGate::Scan { suspended } => {
let find = find.clone();
if let Ok(Some(pid)) = tokio::task::spawn_blocking(move || find(suspended)).await {
return Some(pid);
}
}
}
if tokio::time::Instant::now() + interval >= deadline {
break;
}
tokio::time::sleep(interval).await;
}
None
}
-214
View File
@@ -1,214 +0,0 @@
//! One class histogram of a JVM and its deltas against the previous one.
use super::scan::{RESCAN_BUDGET, RESCAN_PER_SOCKET};
use super::{
ClassCounts, HISTOGRAM_TIMEOUT, JvmClassRow, JvmMemorySnapshot, JvmMemorySource, JvmMemoryUnavailable,
safe_while_suspended,
};
use ide_core::IdeResult;
use ide_jvm_attach::{AttachError, HeapDumpReply, HistogramRow, JavaVersion};
use std::collections::HashMap;
use std::path::Path;
use std::time::Duration;
/// Version probe for a JVM whose `release` file could not be read.
const VERSION_TIMEOUT: Duration = Duration::from_secs(2);
const FLAGS_TIMEOUT: Duration = Duration::from_secs(2);
/// The attach calls a snapshot or a heap dump makes, so tests can stand in for a JVM.
pub(crate) trait AttachProbe: Send + Sync {
fn has_listener(&self, pid: u32) -> bool;
/// Asks only the JVMs whose pid `may_query` accepts.
fn find_by_jdwp_port(&self, port: u16, may_query: &dyn Fn(u32) -> bool) -> Option<u32>;
fn version(&self, pid: u32) -> Option<JavaVersion>;
fn histogram(&self, pid: u32) -> Result<String, AttachError>;
fn alive(&self, pid: u32) -> bool;
fn vm_flags(&self, pid: u32) -> Result<String, AttachError>;
fn heap_dump(&self, pid: u32, path: &Path, timeout: Duration) -> Result<HeapDumpReply, AttachError>;
/// `None` where the free space is not known.
fn free_bytes(&self, dir: &Path) -> Option<u64>;
}
pub(super) struct JdkProbe {
pub(super) suspended: bool,
}
impl AttachProbe for JdkProbe {
fn has_listener(&self, pid: u32) -> bool {
ide_jvm_attach::has_listener(pid)
}
fn find_by_jdwp_port(&self, port: u16, may_query: &dyn Fn(u32) -> bool) -> Option<u32> {
ide_jvm_attach::find_jvm_by_jdwp_port(port, RESCAN_PER_SOCKET, RESCAN_BUDGET, may_query)
}
fn version(&self, pid: u32) -> Option<JavaVersion> {
ide_jvm_attach::java_version_of(pid).or_else(|| {
// The version decides whether attaching to a suspended VM hangs it, so never
// attach to learn it while suspended.
if self.suspended {
return None;
}
let out = ide_jvm_attach::jcmd(pid, "VM.version", VERSION_TIMEOUT).ok()?;
ide_jvm_attach::parse_vm_version(&out)
})
}
fn histogram(&self, pid: u32) -> Result<String, AttachError> {
ide_jvm_attach::jcmd(pid, "GC.class_histogram -all", HISTOGRAM_TIMEOUT)
}
fn alive(&self, pid: u32) -> bool {
alive(pid)
}
fn vm_flags(&self, pid: u32) -> Result<String, AttachError> {
ide_jvm_attach::vm_flags(pid, FLAGS_TIMEOUT)
}
fn heap_dump(&self, pid: u32, path: &Path, timeout: Duration) -> Result<HeapDumpReply, AttachError> {
ide_jvm_attach::heap_dump(pid, path, timeout)
}
fn free_bytes(&self, dir: &Path) -> Option<u64> {
ide_heap::free_bytes(dir)
}
}
fn alive(pid: u32) -> bool {
#[cfg(unix)]
{
// 0 and negative pids address process groups.
let Ok(pid) = i32::try_from(pid) else { return false };
// SAFETY: signal 0 only checks that the process exists and may be signalled.
pid > 0
&& (unsafe { libc::kill(pid, 0) } == 0
|| std::io::Error::last_os_error().raw_os_error() == Some(libc::EPERM))
}
#[cfg(not(unix))]
{
let _ = pid;
false
}
}
/// Why `pid`'s attach listener cannot answer: it stopped listening or the program ended.
pub(super) fn gone(probe: &dyn AttachProbe, pid: u32) -> JvmMemoryUnavailable {
if probe.alive(pid) { JvmMemoryUnavailable::ListenerNotRunning } else { JvmMemoryUnavailable::ProgramExited }
}
/// The reason for an attach request (`what`) to `jvm` that failed, or an error where no reason
/// fits.
pub(super) fn attach_failure(
probe: &dyn AttachProbe,
jvm: u32,
error: AttachError,
what: &str,
) -> IdeResult<JvmMemoryUnavailable> {
use JvmMemoryUnavailable as U;
match error {
AttachError::Timeout => Ok(U::TimedOut),
AttachError::Refused | AttachError::NoListener => Ok(gone(probe, jvm)),
AttachError::ForeignListener => {
tracing::warn!("attach socket of JVM {jvm} is no longer private to this user; not using it");
Ok(U::ListenerNotRunning)
}
AttachError::Unsupported => Ok(U::UnsupportedPlatform),
e => Err(ide_core::IdeError::internal(format!("{what} of JVM {jvm}: {e}"))),
}
}
pub(crate) fn pick_jvm_pid(tree: &[i32], has_listener: impl Fn(u32) -> bool) -> Option<u32> {
tree.iter().filter_map(|&p| u32::try_from(p).ok()).filter(|&p| p > 0).find(|&p| has_listener(p))
}
pub(crate) fn diff_rows(rows: Vec<HistogramRow>, previous: Option<&ClassCounts>) -> (Vec<JvmClassRow>, ClassCounts) {
let mut seen: HashMap<&str, u32> = HashMap::with_capacity(rows.len());
for r in &rows {
*seen.entry(r.name.as_str()).or_default() += 1;
}
let ambiguous: Vec<bool> = rows.iter().map(|r| seen[r.name.as_str()] > 1).collect();
drop(seen);
let mut counts = ClassCounts::with_capacity(rows.len());
let mut out: Vec<JvmClassRow> = rows
.into_iter()
.zip(ambiguous)
.map(|(r, ambiguous)| {
counts.insert(r.name.clone(), (!ambiguous).then_some((r.instances, r.bytes)));
let delta = match previous {
_ if ambiguous => None,
None => None,
Some(p) => match p.get(&r.name) {
Some(None) => None,
// Absent: new since the previous snapshot, so it grew from nothing.
before => {
let (instances, bytes) = before.copied().flatten().unwrap_or_default();
Some((delta(r.instances, instances), delta(r.bytes, bytes)))
}
},
};
JvmClassRow {
name: r.name,
module: r.module,
instances: r.instances,
bytes: r.bytes,
delta_instances: delta.map(|d| d.0),
delta_bytes: delta.map(|d| d.1),
ambiguous,
}
})
.collect();
out.sort_by(|a, b| b.bytes.cmp(&a.bytes).then_with(|| a.name.cmp(&b.name)));
(out, counts)
}
/// Exact while the difference fits in an i64 (two's complement).
fn delta(now: u64, before: u64) -> i64 {
now.wrapping_sub(before) as i64
}
/// `pid_tree` starts with the session's reported pid, followed by its descendants.
pub(crate) fn take_snapshot(
probe: &dyn AttachProbe,
pid_tree: &[i32],
suspended: bool,
previous: Option<&ClassCounts>,
) -> IdeResult<Result<(JvmMemorySnapshot, ClassCounts), JvmMemoryUnavailable>> {
use JvmMemoryUnavailable as U;
let Some(root) = pid_tree.first().and_then(|&p| u32::try_from(p).ok()) else {
return Ok(Err(U::NoJvmProcess));
};
let Some(jvm) = pick_jvm_pid(pid_tree, |p| probe.has_listener(p)) else {
return Ok(Err(gone(probe, root)));
};
let version = probe.version(jvm);
if suspended && !safe_while_suspended(version.as_ref()) {
return Ok(Err(U::UnsafeWhileSuspended { java_major: version.map_or(0, |v| v.major) }));
}
let started = std::time::Instant::now();
let text = match probe.histogram(jvm) {
Ok(text) => text,
Err(e) => return attach_failure(probe, jvm, e, "class histogram").map(Err),
};
let elapsed_ms = u32::try_from(started.elapsed().as_millis()).unwrap_or(u32::MAX);
let histogram = ide_jvm_attach::parse_class_histogram(&text);
if histogram.skipped_lines > 0 {
tracing::debug!("class histogram of JVM {jvm}: {} unreadable lines", histogram.skipped_lines);
}
let (rows, counts) = diff_rows(histogram.rows, previous);
let snapshot = JvmMemorySnapshot {
total_instances: rows.iter().map(|r| r.instances).sum(),
total_bytes: rows.iter().map(|r| r.bytes).sum(),
rows,
java_version: version.map(|v| v.text).unwrap_or_default(),
taken_while_suspended: suspended,
has_baseline: previous.is_some(),
elapsed_ms,
source: JvmMemorySource::Histogram,
};
Ok(Ok((snapshot, counts)))
}
@@ -1,119 +0,0 @@
//! A scripted [`JvmInspector`] and a session without an adapter behind it.
use crate::SessionOwner;
use crate::adapter::{AdapterSpec, Transport};
use crate::connection::Connection;
use crate::session::Session;
use ide_core::{DebugSessionId, IdeEvent, SharedSink};
use ide_jvm_inspect::{
ClassCount, InspectError, InspectFuture, JvmField, JvmInspector, JvmReferrer, JvmValue, ObjectId, VmTarget,
};
use parking_lot::Mutex;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use tokio::sync::Notify;
#[derive(Default)]
pub(crate) struct FakeInspector {
pub targets: Mutex<Vec<VmTarget>>,
/// The `limit` of each `instances` / `referrers` call.
pub limits: Mutex<Vec<u32>>,
pub calls: AtomicUsize,
/// Returned by every call when set.
pub error: Mutex<Option<InspectError>>,
pub counts: Mutex<Vec<ClassCount>>,
pub instances: Mutex<Vec<JvmValue>>,
/// Every call signals `entered`, then waits for `release`.
pub hold: bool,
pub entered: Notify,
pub release: Notify,
}
impl FakeInspector {
pub fn failing(error: InspectError) -> Self {
Self { error: Mutex::new(Some(error)), ..Self::default() }
}
pub fn calls(&self) -> usize {
self.calls.load(Ordering::SeqCst)
}
async fn answer<T: Send>(&self, target: &VmTarget, limit: Option<u32>, ok: T) -> Result<T, InspectError> {
self.calls.fetch_add(1, Ordering::SeqCst);
self.targets.lock().push(target.clone());
self.limits.lock().extend(limit);
if self.hold {
self.entered.notify_one();
self.release.notified().await;
}
match self.error.lock().clone() {
Some(e) => Err(e),
None => Ok(ok),
}
}
}
impl JvmInspector for FakeInspector {
fn class_counts<'a>(&'a self, target: &'a VmTarget) -> InspectFuture<'a, Vec<ClassCount>> {
Box::pin(self.answer(target, None, self.counts.lock().clone()))
}
fn instances<'a>(&'a self, target: &'a VmTarget, _: &'a str, limit: u32) -> InspectFuture<'a, Vec<JvmValue>> {
Box::pin(self.answer(target, Some(limit), self.instances.lock().clone()))
}
fn fields<'a>(&'a self, target: &'a VmTarget, _: ObjectId) -> InspectFuture<'a, Vec<JvmField>> {
Box::pin(self.answer(target, None, Vec::new()))
}
fn referrers<'a>(&'a self, target: &'a VmTarget, _: ObjectId, limit: u32) -> InspectFuture<'a, Vec<JvmReferrer>> {
Box::pin(self.answer(target, Some(limit), Vec::new()))
}
}
pub(crate) struct BareSession {
pub session: Arc<Session>,
pub events: Arc<Mutex<Vec<IdeEvent>>>,
/// The adapter's end of the connection, kept open.
_adapter: tokio::io::DuplexStream,
}
/// A java-debug session that no adapter answers; must be called inside a Tokio runtime.
pub(crate) fn bare_session(request: &str) -> BareSession {
bare_session_of("java-debug", request)
}
/// A session of adapter `name` that no adapter answers; must be called inside a Tokio runtime.
pub(crate) fn bare_session_of(name: &str, request: &str) -> BareSession {
let events: Arc<Mutex<Vec<IdeEvent>>> = Arc::default();
let sink_events = events.clone();
let sink: SharedSink = Arc::new(move |e: IdeEvent| sink_events.lock().push(e));
let (client, adapter) = tokio::io::duplex(1 << 10);
let (reader, writer) = tokio::io::split(client);
let (conn, _) = Connection::start(reader, writer);
let spec = AdapterSpec {
adapter: name.into(),
command: String::new(),
args: Vec::new(),
transport: Transport::Stdio,
install_hint: None,
cwd: std::env::temp_dir(),
env: Vec::new(),
server_host: None,
connect: None,
pinned: None,
};
let session = Session::new(
DebugSessionId(1),
SessionOwner::new("test", sink),
"test".into(),
name.into(),
None,
request.into(),
conn,
spec,
None,
None,
);
BareSession { session: Arc::new(session), events, _adapter: adapter }
}
File diff suppressed because it is too large. Load diff
-11
View File
@@ -9,17 +9,6 @@ use std::{
pub const ADAPTER: &str = "kotlin-debug-adapter";
/// fwcd's adapter or kotlin-lsp's: both debug a JVM the IDE starts, through the JDWP link.
pub fn is_kotlin(adapter: &str) -> bool {
adapter == ADAPTER || adapter == crate::kotlin_lsp::ADAPTER
}
/// fwcd's adapter resumes the VM while constructing its debuggee, before its breakpoint hooks
/// exist, so only its link holds resumes back until the attach is done.
pub(crate) fn link_options(adapter: &str) -> ide_jdwp::LinkOptions {
ide_jdwp::LinkOptions { hold_resumes: adapter == ADAPTER }
}
pub struct SourceMirror {
dir: tempfile::TempDir,
paths: HashMap<PathBuf, PathBuf>,
+166
View File
@@ -0,0 +1,166 @@
//! Keep a suspended JVM stopped until Kotlin's adapter has installed its listeners.
//! The official 0.4.4 adapter resumes VMStart while constructing its debuggee, before
//! installing breakpoint hooks. DAP attach completes only after those hooks exist.
use ide_core::IdeResult;
use std::{io, time::Duration};
use tokio::{
io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt},
net::{TcpListener, TcpStream},
sync::oneshot,
task::JoinHandle,
};
pub(crate) struct StartupRelay {
release: Option<oneshot::Sender<()>>,
task: JoinHandle<()>,
}
impl StartupRelay {
pub async fn start(host: String, port: u16) -> IdeResult<(Self, u16)> {
let listener = TcpListener::bind(("127.0.0.1", 0)).await?;
let local = listener.local_addr()?.port();
let (release, ready) = oneshot::channel();
let task = tokio::spawn(async move {
let connect = async {
let (adapter, _) = listener.accept().await?;
let jvm = TcpStream::connect((host.as_str(), port)).await?;
adapter.set_nodelay(true)?;
jvm.set_nodelay(true)?;
Ok::<_, io::Error>((adapter, jvm))
};
if let Ok(Ok((adapter, jvm))) = tokio::time::timeout(Duration::from_secs(60), connect).await {
let (mut ar, mut aw) = adapter.into_split();
let (mut jr, mut jw) = jvm.into_split();
let upstream = forward(&mut ar, &mut jw, ready);
let downstream = tokio::io::copy(&mut jr, &mut aw);
// Close both sockets as soon as either side disconnects.
tokio::select! {
r = upstream => if let Err(e) = r { tracing::debug!("Kotlin JDWP relay: {e}"); },
_ = downstream => {},
}
}
});
Ok((Self { release: Some(release), task }, local))
}
pub fn release(&mut self) {
if let Some(ready) = self.release.take() {
let _ = ready.send(());
}
}
}
impl Drop for StartupRelay {
fn drop(&mut self) {
self.task.abort();
}
}
async fn packet(reader: &mut (impl AsyncRead + Unpin)) -> io::Result<Vec<u8>> {
let mut header = [0; 11];
reader.read_exact(&mut header).await?;
let len = u32::from_be_bytes(header[..4].try_into().unwrap()) as usize;
if !(11..=16 * 1024 * 1024).contains(&len) {
return Err(io::Error::new(io::ErrorKind::InvalidData, "Invalid JDWP packet length"));
}
let mut packet = vec![0; len];
packet[..11].copy_from_slice(&header);
reader.read_exact(&mut packet[11..]).await?;
Ok(packet)
}
async fn forward(
reader: &mut (impl AsyncRead + Unpin),
writer: &mut (impl AsyncWrite + Unpin),
mut ready: oneshot::Receiver<()>,
) -> io::Result<()> {
let mut handshake = [0; 14];
reader.read_exact(&mut handshake).await?;
if &handshake != b"JDWP-Handshake" {
return Err(io::Error::new(io::ErrorKind::InvalidData, "Invalid JDWP handshake"));
}
writer.write_all(&handshake).await?;
let mut released = false;
let mut held: Vec<Vec<u8>> = Vec::new();
loop {
// Retain the in-flight read when attach completes, including partial packets.
let read = packet(reader);
tokio::pin!(read);
let packet = tokio::select! {
result = &mut read => result?,
result = &mut ready, if !released => {
result.map_err(|_| io::Error::new(io::ErrorKind::ConnectionAborted, "Kotlin attach cancelled"))?;
released = true;
for packet in held.drain(..) {
writer.write_all(&packet).await?;
}
read.await?
}
};
// JDWP VirtualMachine.Resume (1/9), ThreadReference.Resume (11/3).
// Other commands must proceed: attach itself queries threads and installs requests.
if !released && packet[8] == 0 && matches!((packet[9], packet[10]), (1, 9) | (11, 3)) {
if held.len() >= 1024 {
return Err(io::Error::new(io::ErrorKind::InvalidData, "Too many pending JDWP resumes"));
}
held.push(packet);
} else {
writer.write_all(&packet).await?;
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn command(id: u8, set: u8, cmd: u8) -> Vec<u8> {
vec![0, 0, 0, 11, 0, 0, 0, id, 0, set, cmd]
}
#[tokio::test]
async fn holds_resumes_but_allows_setup_and_preserves_partial_reads() {
let (mut adapter, mut reader) = tokio::io::duplex(4096);
let (mut writer, mut jvm) = tokio::io::duplex(4096);
let (release, ready) = oneshot::channel();
let task = tokio::spawn(async move { forward(&mut reader, &mut writer, ready).await });
adapter.write_all(b"JDWP-Handshake").await.unwrap();
let mut handshake = [0; 14];
jvm.read_exact(&mut handshake).await.unwrap();
assert_eq!(&handshake, b"JDWP-Handshake");
let resume = command(1, 1, 9);
let thread_resume = command(2, 11, 3);
let setup = command(3, 1, 4);
adapter.write_all(&resume).await.unwrap();
adapter.write_all(&thread_resume).await.unwrap();
adapter.write_all(&setup).await.unwrap();
assert_eq!(packet(&mut jvm).await.unwrap(), setup);
let next = command(4, 1, 9);
adapter.write_all(&next[..6]).await.unwrap();
release.send(()).unwrap();
assert_eq!(packet(&mut jvm).await.unwrap(), resume);
assert_eq!(packet(&mut jvm).await.unwrap(), thread_resume);
adapter.write_all(&next[6..]).await.unwrap();
assert_eq!(packet(&mut jvm).await.unwrap(), next);
drop(adapter);
assert!(task.await.unwrap().is_err());
}
#[tokio::test]
async fn dropping_session_closes_relay() {
let (relay, port) = StartupRelay::start("127.0.0.1".into(), 1).await.unwrap();
let mut adapter = TcpStream::connect(("127.0.0.1", port)).await.unwrap();
drop(relay);
let result = tokio::time::timeout(Duration::from_secs(1), adapter.read_u8()).await.unwrap();
assert!(result.is_err());
}
#[tokio::test]
async fn rejects_invalid_packet_lengths() {
for len in [0_u32, 10, 16 * 1024 * 1024 + 1] {
let mut bytes = command(1, 1, 9);
bytes[..4].copy_from_slice(&len.to_be_bytes());
assert_eq!(packet(&mut bytes.as_slice()).await.unwrap_err().kind(), io::ErrorKind::InvalidData);
}
}
}
-147
View File
@@ -1,147 +0,0 @@
//! JetBrains kotlin-lsp's debugger: a DAP server inside the language server, reached through
//! the `debugServer` port it opens once the project is imported.
//!
//! Known limits, with no guard here: an evaluate that calls a function with a breakpoint, right
//! after a step over a suspension point, loses every later stop at that breakpoint; while
//! stepping, breakpoints hit by other coroutines are skipped.
use ide_core::{IdeError, IdeResult};
use parking_lot::Mutex;
use std::collections::HashSet;
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
pub const ADAPTER: &str = "kotlin-lsp";
/// The `adapterID` its debug server expects in `initialize`.
pub const DAP_ADAPTER_ID: &str = "intellij_jvm";
/// The path of a `file:` URI, as kotlin-lsp reports some sources (percent-encoded); `None` for
/// anything else, including a URI naming another host.
pub(crate) fn source_path(path: &str) -> Option<PathBuf> {
let rest = path.strip_prefix("file:")?;
let path = match rest.strip_prefix("//") {
Some(authority) => {
let slash = authority.find('/')?;
if !matches!(&authority[..slash], "" | "localhost") {
return None;
}
&authority[slash..]
}
None => rest,
};
let decoded = percent_encoding::percent_decode_str(path).decode_utf8().ok()?;
// `/C:/p/App.kt` on Windows.
let decoded = match decoded.strip_prefix('/') {
Some(drive) if cfg!(windows) && drive.as_bytes().get(1) == Some(&b':') => drive,
_ => &decoded,
};
Some(PathBuf::from(decoded))
}
pub(crate) fn frame_name(name: &str) -> &str {
name.strip_prefix("at ").unwrap_or(name)
}
/// What kotlin-lsp's DAP server does unlike other adapters.
///
/// It never answers a request that carries a thread id from an earlier stop (K1 G4), so such a
/// request would wait out `REQUEST_TIMEOUT`: [`Self::check_thread`] refuses it first. Only thread
/// ids were measured; a stale `frameId` or `variablesReference` may wait the same way. A step
/// ends with `reason: "breakpoint"`; a real breakpoint hit during a step is then reported as a
/// step too, since the stop carries neither `hitBreakpointIds` nor a description.
#[derive(Default)]
pub(crate) struct Quirks {
step_pending: AtomicBool,
/// The stop (a `Suspension` ticket) and the thread ids seen in it.
threads: Mutex<(u64, HashSet<i64>)>,
}
impl Quirks {
/// Before `next`, `stepIn`, `stepOut` or a forced step.
pub fn step_sent(&self) {
self.step_pending.store(true, Ordering::Release);
}
/// Before `continue` or run to cursor, or after a step request failed.
pub fn resume_sent(&self) {
self.step_pending.store(false, Ordering::Release);
}
/// The reason to report for a `stopped` event.
pub fn stop_reason(&self, reason: String) -> String {
let stepped = self.step_pending.swap(false, Ordering::AcqRel);
if stepped && reason == "breakpoint" { "step".into() } else { reason }
}
/// Thread ids seen in stop `stop`: a `stopped` event's, or a `threads` reply to a request sent
/// during that stop. A reply from an older stop changes nothing.
pub fn saw_threads(&self, stop: u64, ids: impl IntoIterator<Item = i64>) {
let mut threads = self.threads.lock();
if stop > threads.0 {
*threads = (stop, ids.into_iter().collect());
} else if stop == threads.0 {
threads.1.extend(ids);
}
}
/// Refuses a thread id that is not one of current stop `stop`'s, once some are known.
pub fn check_thread(&self, stop: u64, suspended: bool, id: i64) -> IdeResult<()> {
let threads = self.threads.lock();
if suspended && threads.0 == stop && !threads.1.is_empty() && !threads.1.contains(&id) {
return Err(IdeError::invalid(format!("thread {id} belongs to an earlier stop")));
}
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::Path;
#[test]
fn file_uris_become_paths() {
assert_eq!(source_path("file:///p/a%20b/App.kt").as_deref(), Some(Path::new("/p/a b/App.kt")));
assert_eq!(source_path("file://localhost/p/x.kt").as_deref(), Some(Path::new("/p/x.kt")));
assert_eq!(source_path("/p/x.kt"), None);
}
#[test]
fn frame_names_lose_the_at_prefix() {
assert_eq!(frame_name("at com.x.AppKt.main(App.kt:13)"), "com.x.AppKt.main(App.kt:13)");
assert_eq!(frame_name("main"), "main");
}
#[test]
fn only_a_pending_step_turns_a_breakpoint_into_a_step() {
let quirks = Quirks::default();
quirks.step_sent();
assert_eq!(quirks.stop_reason("breakpoint".into()), "step");
assert_eq!(quirks.stop_reason("breakpoint".into()), "breakpoint");
quirks.step_sent();
quirks.resume_sent();
assert_eq!(quirks.stop_reason("breakpoint".into()), "breakpoint");
}
#[test]
fn threads_are_checked_against_the_current_stop() {
let quirks = Quirks::default();
quirks.saw_threads(3, [18]);
assert!(quirks.check_thread(3, true, 18).is_ok());
assert!(quirks.check_thread(3, true, 6).is_err());
assert!(quirks.check_thread(4, true, 6).is_ok(), "nothing known about stop 4 yet");
assert!(quirks.check_thread(3, false, 6).is_ok());
}
#[test]
fn an_older_threads_reply_does_not_change_a_newer_stop() {
let quirks = Quirks::default();
quirks.saw_threads(4, [2]);
quirks.saw_threads(3, [1]);
assert!(quirks.check_thread(4, true, 1).is_err());
quirks.saw_threads(4, [7]);
assert!(quirks.check_thread(4, true, 2).is_ok() && quirks.check_thread(4, true, 7).is_ok());
quirks.saw_threads(5, [9]);
assert!(quirks.check_thread(5, true, 2).is_err());
}
}
+48 -253
View File
@@ -13,9 +13,8 @@ pub mod attach;
mod cargo;
pub mod connection;
pub mod jdwp;
mod jvm_memory;
pub mod kotlin;
pub mod kotlin_lsp;
mod kotlin_jdwp;
mod session;
use adapter::{AdapterLauncher, AdapterSpec, ProcessLauncher};
@@ -23,7 +22,6 @@ use connection::{Connection, DapError};
use futures::future::join_all;
use ide_core::{DebugSessionId, IdeError, IdeResult, SharedSink};
use ide_language::LanguageRegistry;
pub use jvm_memory::*;
use parking_lot::Mutex;
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
@@ -128,14 +126,6 @@ pub struct DebugCapabilities {
pub log_points: bool,
/// HotSwap ("Reload Changed Classes"): java-debug's `redefineClasses` request.
pub hot_code_replace: bool,
/// The session debugs a JVM, so it has a Memory view; [`DebugManager::jvm_memory_classes`]
/// says why when it has no data (platform, remote JVM, ...).
pub jvm_memory: bool,
/// The session has a JVM inspector: instances, fields and referrers while suspended. Can
/// change during the session ([`ide_core::event::DebugEvent::Capabilities`]).
pub jvm_instances: bool,
/// The session can be asked for a heap dump ([`DebugManager::jvm_heap_dump`]).
pub jvm_heap: bool,
}
/// Result of [`DebugManager::redefine_classes`].
@@ -312,7 +302,6 @@ pub struct DebugManager {
data_breakpoints: Mutex<Vec<DataBreakpoint>>,
/// In-flight stops, by session: a second Stop sets `stop_force`, the waiting one kills.
stopping: Mutex<HashMap<DebugSessionId, Arc<Session>>>,
jvm_finder: Mutex<jvm_memory::JvmFinder>,
}
/// rdbg (Ruby's debug gem) starts the debuggee itself and ignores `launch`'s program,
@@ -577,21 +566,9 @@ pub(crate) fn capabilities_from(caps: &Value) -> DebugCapabilities {
hit_conditional_breakpoints: has("supportsHitConditionalBreakpoints"),
log_points: has("supportsLogPoints"),
hot_code_replace: false,
jvm_memory: false,
jvm_instances: false,
jvm_heap: false,
}
}
pub(crate) fn session_capabilities(session: &Session) -> DebugCapabilities {
let mut caps = capabilities_from(&session.caps.lock());
caps.hot_code_replace = HOT_CODE_REPLACE_ADAPTERS.contains(&session.adapter.as_str());
caps.jvm_memory = JVM_MEMORY_ADAPTERS.contains(&session.adapter.as_str());
caps.jvm_instances = session.jvm_inspector.lock().is_some();
caps.jvm_heap = JVM_MEMORY_ADAPTERS.contains(&session.adapter.as_str());
caps
}
/// States of a file's line breakpoints that no session has been sent.
fn unsent(breakpoints: &[Breakpoint]) -> Vec<BreakpointState> {
breakpoints
@@ -611,19 +588,9 @@ fn unverified(enabled: impl Iterator<Item = bool>) -> Vec<BreakpointState> {
.collect()
}
/// A step request's outcome; a failed one leaves no step pending, so the next stop keeps its reason.
fn step_settled(session: &Session, sent: Result<Value, DapError>) -> Result<Value, DapError> {
if sent.is_err()
&& let Some(quirks) = session.kotlin_lsp_quirks()
{
quirks.resume_sent();
}
sent
}
/// Delve compiles the debuggee to `__debug_bin` in the session's cwd (in DAP mode a
/// temp-file name, `__debug_bin<digits>`; on Windows with `.exe` before the digits) and
/// removes it on a graceful exit — a shutdown it doesn't always reach, so the files pile up. Sweep
/// Delve compiles the debuggee to `__debug_bin` in the session's cwd (on Windows
/// `__debug_bin.exe` plus `__debug_bin.exe<digits>` in DAP mode) and removes it on a
/// graceful exit — a shutdown it doesn't always reach, so the files pile up. Sweep
/// the leftovers once the session is down. Runs after `session_closed`, so Delve
/// has already exited or been killed and holds no file locks; a leftover of a
/// still-building run is removed on that run's own shutdown.
@@ -632,26 +599,19 @@ fn remove_delve_binaries(cwd: &Path) {
for entry in entries.flatten() {
let name = entry.file_name();
let Some(name) = name.to_str() else { continue };
// `__debug_bin`, then `.exe` on Windows, then DAP mode's temp-file digits.
// Suffixes like `.go` are user files.
let suffix = name.strip_prefix("__debug_bin").map(|s| s.strip_prefix(".exe").unwrap_or(s));
let leftover = suffix.is_some_and(|s| s.chars().all(|c| c.is_ascii_digit()));
if !leftover || entry.file_type().is_ok_and(|t| !t.is_file()) {
continue;
}
// Windows: a just-killed dlv / debuggee can still hold this session's
// binary while the previous one's lock is already gone, so one try
// leaves exactly the current leftover behind. Retry on a short back-off
// to outlast the handle release; a truly busy file is retried into the
// next run's sweep anyway.
for attempt in 0..5 {
match std::fs::remove_file(entry.path()) {
Ok(()) => break,
Err(e) if attempt == 4 => {
tracing::debug!("could not remove Delve leftover {}: {e}", entry.path().display());
}
Err(_) => std::thread::sleep(std::time::Duration::from_millis(50 * (attempt as u64 + 1))),
}
// Windows writes `__debug_bin.exe` (and `.exe<digits>` in DAP mode); other
// platforms keep the bare name. Suffixes like `.go` are user files.
let suffix = name.strip_prefix("__debug_bin");
let leftover = suffix.is_some_and(|s| {
s.is_empty()
|| s == ".exe"
|| (s.starts_with(".exe") && s[".exe".len()..].chars().all(|c| c.is_ascii_digit()))
});
if leftover
&& entry.file_type().is_ok_and(|t| t.is_file())
&& let Err(e) = std::fs::remove_file(entry.path())
{
tracing::debug!("could not remove Delve leftover {}: {e}", entry.path().display());
}
}
}
@@ -686,15 +646,9 @@ impl DebugManager {
function_breakpoints: Mutex::default(),
data_breakpoints: Mutex::default(),
stopping: Mutex::default(),
jvm_finder: Mutex::new(Arc::new(jvm_memory::find_local_jvm)),
})
}
#[cfg(test)]
pub(crate) fn set_jvm_finder(&self, finder: jvm_memory::JvmFinder) {
*self.jvm_finder.lock() = finder;
}
fn session(&self, s: DebugSessionId) -> IdeResult<Arc<Session>> {
self.sessions.lock().get(&s).cloned().ok_or_else(|| no_session(s))
}
@@ -753,10 +707,6 @@ impl DebugManager {
{
spec.env.push(("JAVA_HOME".into(), home));
}
// Delve looks up Go before applying the debuggee's launch environment.
if config.adapter == "delve" {
spec.env.extend(config.env.iter().map(|(key, value)| (key.clone(), value.clone())));
}
if config.adapter == "rdbg" {
rdbg_command_line(&mut spec, &config, &cwd)?;
}
@@ -789,18 +739,6 @@ impl DebugManager {
if let Some(server) = &debug_server {
spec.server_host = server.host.clone();
}
if config.adapter == "gdb" && config.request == "launch" && debug_server.is_none() {
// GDB's DAP env replaces the entire inferior environment. Ordinary run overrides
// instead belong on the adapter process, which the inferior inherits. Explicit
// launch JSON keeps GDB's replacement semantics.
let explicit_env = config.extra_json.as_deref().is_some_and(|extra| {
serde_json::from_str::<Value>(extra).is_ok_and(|extra| extra.get("env").is_some())
});
if !explicit_env {
spec.env.extend(config.env.iter().map(|(k, v)| (k.clone(), v.clone())));
arguments.as_object_mut().unwrap().remove("env");
}
}
if let Some(cargo) = arguments.as_object_mut().and_then(|o| o.remove("cargo")) {
let cargo_args: Vec<String> = cargo
.get("args")
@@ -910,7 +848,6 @@ impl DebugManager {
return Err(IdeError::invalid(format!("request must be `launch` or `attach`, got `{request}`")));
}
let io = self.launcher.launch(spec.clone(), reuse_port).await?;
let launch_after_configuration = io.launch_after_configuration && request == "launch";
let stderr = io.stderr.clone();
let (conn, incoming) = {
let stderr = stderr.clone();
@@ -948,7 +885,7 @@ impl DebugManager {
let configured = tokio::select! {
biased;
_ = session.startup_cancel.cancelled() => return Err(IdeError::cancelled()),
result = self.configure(&session, arguments, launch_after_configuration) => result,
result = self.configure(&session, arguments) => result,
};
match configured {
Ok(()) => Ok(id),
@@ -975,7 +912,7 @@ impl DebugManager {
}
}
async fn configure(&self, s: &Arc<Session>, mut arguments: Value, launch_in_config: bool) -> IdeResult<()> {
async fn configure(&self, s: &Arc<Session>, mut arguments: Value) -> IdeResult<()> {
if s.adapter == kotlin::ADAPTER {
let project = arguments
.get("projectRoot")
@@ -988,7 +925,6 @@ impl DebugManager {
arguments["projectRoot"] = json!(mirror.root());
*s.kotlin_mirror.lock() = Some(mirror);
}
let is_kotlin_lsp = s.adapter == kotlin_lsp::ADAPTER;
let caps = s
.conn
.request_timeout(
@@ -996,7 +932,7 @@ impl DebugManager {
json!({
"clientID": "openidea",
"clientName": "OpenIDEA",
"adapterID": if is_kotlin_lsp { kotlin_lsp::DAP_ADAPTER_ID } else { s.adapter.as_str() },
"adapterID": s.adapter,
"locale": "en",
"linesStartAt1": true,
"columnsStartAt1": true,
@@ -1006,8 +942,7 @@ impl DebugManager {
"supportsRunInTerminalRequest": true,
"supportsStartDebuggingRequest": true,
"supportsProgressReporting": true,
// kotlin-lsp sends `invalidated` either way.
"supportsInvalidatedEvent": is_kotlin_lsp,
"supportsInvalidatedEvent": false,
"supportsMemoryReferences": false,
"supportsArgsCanBeInterpretedByShell": false,
}),
@@ -1016,32 +951,14 @@ impl DebugManager {
.await?;
*s.caps.lock() = if caps.is_object() { caps } else { json!({}) };
// The Kotlin adapters ignore the resolved classpath and project runtime on launch.
let request = if kotlin::is_kotlin(&s.adapter) && s.request == "launch" {
// The Kotlin adapter ignores the resolved classpath and project runtime on launch.
let request = if s.adapter == kotlin::ADAPTER && s.request == "launch" {
session::start_kotlin_debuggee(s, &mut arguments).await?;
"attach"
} else {
&s.request
};
// Both blocks below run before the Kotlin link replaces the JVM's address with its own.
if request == "attach"
&& let Some(target) = jvm_memory::attach_target(&arguments)
{
let _ = s.attach_target.set(target);
}
if cfg!(unix)
&& s.request == "attach"
&& JVM_MEMORY_ADAPTERS.contains(&s.adapter.as_str())
&& let Some(port) = jvm_memory::local_jdwp_port(&arguments)
{
if jdwp::is_listen_relay(port) {
s.listen_relay.store(true, Ordering::Release);
} else {
let finder = self.jvm_finder.lock().clone();
jvm_memory::find_attached_jvm(s, port, finder).await;
}
}
if kotlin::is_kotlin(&s.adapter) {
if s.adapter == kotlin::ADAPTER {
let host = arguments
.get("hostName")
.and_then(Value::as_str)
@@ -1053,39 +970,24 @@ impl DebugManager {
.and_then(|p| u16::try_from(p).ok())
.filter(|p| *p != 0)
.ok_or_else(|| IdeError::invalid("Invalid Kotlin JDWP port"))?;
let (link, port) = ide_jdwp::JdwpLink::start(host, port, kotlin::link_options(&s.adapter)).await?;
*s.jvm_inspector.lock() = Some(Arc::new(ide_jdwp::LinkInspector::new(link.clone())));
jvm_memory::watch_link(s, &link);
*s.jdwp_link.lock() = Some(link);
let (relay, port) = kotlin_jdwp::StartupRelay::start(host, port).await?;
*s.kotlin_relay.lock() = Some(relay);
arguments["hostName"] = json!("127.0.0.1");
arguments["port"] = json!(port);
}
s.jvm_located.send_replace(true);
// Launch/attach may wait for a slow runtime or a remote target. Let the adapter report
// failure; Stop cancels configuration and a closed connection fails the pending reply.
// Many adapters answer only after `configurationDone`, so don't wait for the reply yet.
// GDB 15 is the exception: its `launch` starts the program itself (14 runs at
// `configurationDone`, 16+ defers the launch there), so its launch waits below.
let mut launch = if launch_in_config {
None
} else {
let reply = s.conn.send_request(request, arguments.clone())?;
Some(Box::pin(async move { reply.await.unwrap_or(Err(DapError::Closed)) }))
};
let launch = s.conn.send_request(request, arguments)?;
let mut launch = Box::pin(async move { launch.await.unwrap_or(Err(DapError::Closed)) });
let mut initialized = s.initialized.subscribe();
let mut launch_result = None;
let got_initialized = match launch.as_mut() {
Some(launch) => tokio::select! {
r = initialized.wait_for(|v| *v) => { r.map_err(|_| DapError::Closed)?; true }
r = launch => {
r?;
launch_result = Some(());
false
}
},
None => {
initialized.wait_for(|v| *v).await.map_err(|_| DapError::Closed)?;
true
let got_initialized = tokio::select! {
r = initialized.wait_for(|v| *v) => { r.map_err(|_| DapError::Closed)?; true }
r = &mut launch => {
r?;
launch_result = Some(());
false
}
};
if !got_initialized {
@@ -1143,20 +1045,11 @@ impl DebugManager {
}
}
s.configured.store(true, Ordering::Release);
match launch.as_mut() {
Some(launch) => {
if launch_result.is_none() {
launch.await?;
}
}
None => {
// GDB 15: the breakpoints are in, so its `run` cannot miss them.
let reply = s.conn.send_request(request, arguments)?;
reply.await.unwrap_or(Err(DapError::Closed))?;
}
if launch_result.is_none() {
launch.await?;
}
if let Some(link) = s.jdwp_link.lock().as_ref() {
link.release();
if let Some(relay) = s.kotlin_relay.lock().as_mut() {
relay.release();
}
Ok(())
}
@@ -1173,9 +1066,6 @@ impl DebugManager {
/// Ends a session: `terminate` (when asked and supported) → `disconnect` → kill.
pub(crate) async fn shutdown(&self, s: &Arc<Session>, terminate: bool) {
if terminate {
crate::session::remember_debuggee(s).await;
}
// java-debug runs the program itself and pipes its stdout over DAP. A `disconnect`
// ends that DAP session, so every line the program writes while shutting down is lost.
// Ask the program to exit ourselves and keep the adapter attached until it has drained
@@ -1203,7 +1093,7 @@ impl DebugManager {
if own_terminate && !graceful {
// Signal before anything else: waiting on a request the adapter may not answer
// would delay the shutdown the console is supposed to show.
crate::session::signal_debuggee(s, false).await;
crate::session::signal_debuggee(s, false);
}
if graceful {
// java-debug sends `terminated` only after it has drained the program's console
@@ -1389,11 +1279,8 @@ impl DebugManager {
}
pub async fn threads(&self, s: DebugSessionId) -> IdeResult<Vec<ThreadInfo>> {
let session = self.session(s)?;
// Before sending: a stop that overtakes the reply must not take the reply's ids as its own.
let stop = session.suspended.ticket();
let body = session.conn.request_timeout("threads", Value::Null, REQUEST_TIMEOUT).await?;
let threads: Vec<ThreadInfo> = body
let body = self.session(s)?.conn.request_timeout("threads", Value::Null, REQUEST_TIMEOUT).await?;
Ok(body
.get("threads")
.and_then(Value::as_array)
.map(|t| {
@@ -1404,16 +1291,11 @@ impl DebugManager {
})
.collect()
})
.unwrap_or_default();
if let Some(quirks) = session.kotlin_lsp_quirks() {
quirks.saw_threads(stop, threads.iter().map(|t| t.id));
}
Ok(threads)
.unwrap_or_default())
}
pub async fn stack_trace(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<Vec<StackFrame>> {
let session = self.session(s)?;
session.check_thread(thread_id)?;
let body = session
.conn
.request_timeout(
@@ -1422,15 +1304,11 @@ impl DebugManager {
REQUEST_TIMEOUT,
)
.await?;
let kotlin_lsp = session.kotlin_lsp_quirks().is_some();
let frame = |f: &Value| {
let mut frame = frame_from(f);
if let Some(client) = frame.path.as_deref().and_then(|p| session.client_path(&p.to_string_lossy())) {
frame.path = Some(client.into());
}
if kotlin_lsp {
frame.name = kotlin_lsp::frame_name(&frame.name).to_owned();
}
frame
};
Ok(body.get("stackFrames").and_then(Value::as_array).map(|f| f.iter().map(frame).collect()).unwrap_or_default())
@@ -1542,15 +1420,8 @@ impl DebugManager {
pub async fn resume(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
session.check_thread(thread_id)?;
if let Some(quirks) = session.kotlin_lsp_quirks() {
quirks.resume_sent();
}
let ticket = session.suspended.ticket();
let body = session.conn.request_timeout("continue", json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await?;
// DAP: a response without the flag continued every thread.
let all = body.get("allThreadsContinued").and_then(Value::as_bool).unwrap_or(true);
session.suspended.resumed(ticket, thread_id, all);
session.emit(ide_core::event::DebugEvent::Continued { thread_id, all_threads_continued: all });
Ok(())
}
@@ -1590,15 +1461,7 @@ impl DebugManager {
StepKind::Out => "stepOut",
};
let session = self.session(s)?;
session.check_thread(thread_id)?;
if let Some(quirks) = session.kotlin_lsp_quirks() {
quirks.step_sent();
}
let ticket = session.suspended.ticket();
let sent = session.conn.request_timeout(command, json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await;
step_settled(&session, sent)?;
// A step runs its thread; other threads stopped by their own events stay stopped.
session.suspended.resumed(ticket, thread_id, false);
session.conn.request_timeout(command, json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await?;
session.emit(ide_core::event::DebugEvent::Continued { thread_id, all_threads_continued: true });
Ok(())
}
@@ -1615,8 +1478,6 @@ impl DebugManager {
force: bool,
) -> IdeResult<()> {
let session = self.session(s)?;
// Before touching breakpoints: a refused `continue` would leave them replaced.
session.check_thread(thread_id)?;
if force {
self.suspend_breakpoints(&session, Some(&path)).await?;
}
@@ -1691,7 +1552,6 @@ impl DebugManager {
/// Force Step Over: `next` with every breakpoint suspended until the step ends.
pub async fn force_step_over(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
session.check_thread(thread_id)?;
self.suspend_breakpoints(&session, None).await?;
self.step(s, thread_id, StepKind::Over).await
}
@@ -1699,17 +1559,10 @@ impl DebugManager {
/// Step into a specific call of the current line (Smart Step Into).
pub async fn step_in_target(&self, s: DebugSessionId, thread_id: i64, target_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
session.check_thread(thread_id)?;
if let Some(quirks) = session.kotlin_lsp_quirks() {
quirks.step_sent();
}
let ticket = session.suspended.ticket();
let sent = session
session
.conn
.request_timeout("stepIn", json!({ "threadId": thread_id, "targetId": target_id }), REQUEST_TIMEOUT)
.await;
step_settled(&session, sent)?;
session.suspended.resumed(ticket, thread_id, false);
.await?;
session.emit(ide_core::event::DebugEvent::Continued { thread_id, all_threads_continued: true });
Ok(())
}
@@ -1743,68 +1596,10 @@ impl DebugManager {
}
pub fn capabilities(&self, s: DebugSessionId) -> IdeResult<DebugCapabilities> {
Ok(session_capabilities(&*self.session(s)?))
}
/// Class table of the session's JVM (`GC.class_histogram -all`, or counted by the debugger
/// while suspended where no histogram can be taken), with deltas against the session's
/// previous snapshot from the same source.
pub async fn jvm_memory_classes(&self, s: DebugSessionId) -> IdeResult<JvmMemoryResult> {
let session = self.session(s)?;
jvm_memory::session_classes(session).await
}
/// Writes a heap dump of the session's JVM to `dir/dump.hprof`, one class-table or dump
/// request per session at a time. Anything but `Ready` deletes `dir`, unless `s` is not a
/// session. On `cancel` the result is `Cancelled` at once, while a dump the JVM is writing
/// still holds the session's turn until the JVM answers.
pub async fn jvm_heap_dump(
&self,
s: DebugSessionId,
dir: PathBuf,
cancel: ide_core::CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
let session = self.session(s)?;
jvm_memory::session_heap_dump(session, dir, cancel).await
}
/// Gives the session the backend for [`Self::jvm_memory_instances`] and the debugger's class
/// counts; the UI is told to reload the session's capabilities.
pub fn set_jvm_inspector(
&self,
s: DebugSessionId,
inspector: Arc<dyn ide_jvm_inspect::JvmInspector>,
) -> IdeResult<()> {
let session = self.session(s)?;
if session.ended.is_cancelled() {
return Err(no_session(s));
}
jvm_memory::set_inspector(&session, inspector);
Ok(())
}
/// Up to `limit` (clamped to `1..=MAX_INSTANCES`) instances of `class_name` while suspended.
pub async fn jvm_memory_instances(
&self,
s: DebugSessionId,
class_name: String,
limit: u32,
) -> IdeResult<JvmInstancesResult> {
jvm_memory::instances(&*self.session(s)?, &class_name, limit).await
}
pub async fn jvm_memory_fields(&self, s: DebugSessionId, object: JvmObjectRef) -> IdeResult<JvmFieldsResult> {
jvm_memory::fields(&*self.session(s)?, object).await
}
/// Objects that point at `object`; `limit` 0 means `DEFAULT_REFERRERS`, at most `MAX_REFERRERS`.
pub async fn jvm_memory_referrers(
&self,
s: DebugSessionId,
object: JvmObjectRef,
limit: u32,
) -> IdeResult<JvmReferrersResult> {
jvm_memory::referrers(&*self.session(s)?, object, limit).await
let mut caps = capabilities_from(&session.caps.lock());
caps.hot_code_replace = HOT_CODE_REPLACE_ADAPTERS.contains(&session.adapter.as_str());
Ok(caps)
}
/// HotSwap: reloads the classes compiled since the session started (or since the
+54 -522
View File
@@ -3,11 +3,8 @@
use crate::adapter::AdapterSpec;
use crate::connection::{Connection, DapError, Incoming};
use crate::kotlin_lsp;
use crate::{Breakpoint, BreakpointState, DataBreakpoint, DebugManager, FunctionBreakpoint, SessionOwner};
use ide_core::event::DebugEvent;
#[cfg(unix)]
use ide_core::process::process_tree;
use ide_core::{DebugSessionId, IdeEvent, TaskId};
use parking_lot::Mutex;
use serde_json::{Value, json};
@@ -48,13 +45,6 @@ pub(crate) struct Session {
pub debuggees: Mutex<Vec<Child>>,
/// The terminal a `runInTerminal` program runs in: typed input goes to it.
pub program: Mutex<Option<ProgramTerminal>>,
/// The job object over the adapter and everything it started: the program an adapter
/// like java-debug launches is its child, so a forced stop (and the session's
/// cleanup) ends the whole tree even when the adapter exited first and the reported
/// pid no longer names it. `None` when the assignment failed; the kill then falls
/// back to the captured process tree.
#[cfg(windows)]
pub job: Mutex<Option<ide_core::process::Job>>,
/// Set once `configurationDone` was sent; breakpoint updates go out live after that.
pub configured: AtomicBool,
pub initialized: watch::Sender<bool>,
@@ -65,21 +55,10 @@ pub(crate) struct Session {
pub stop_force: AtomicBool,
/// `systemProcessId` from the adapter's `process` event: the program being debugged.
pub debuggee_pid: AtomicI64,
/// The JVM an app-launched (Kotlin) session started, or the local JVM an attach session
/// found by its JDWP port. Separate from `debuggee_pid`, which decides how Stop signals the
/// program and is 0 for Kotlin.
pub jvm_pid: AtomicI64,
/// JDWP port of an attach to a local JVM, to look the JVM up again; unset for a remote one.
pub jdwp_port: std::sync::OnceLock<u16>,
/// The attach goes through the listen-mode relay of [`crate::jdwp::listen_for_jvm`].
pub listen_relay: AtomicBool,
pub suspended: Suspension,
/// Pids signalled for the current stop: the reported pid and everything it started (an
/// adapter may report the launcher shell, whose child is the program).
#[cfg_attr(not(unix), allow(dead_code))]
pub stop_pids: parking_lot::Mutex<Vec<i32>>,
#[cfg(windows)]
pub stop_tree: Mutex<Option<ide_core::process::ProcessTree>>,
/// Breakpoints replaced for run to cursor / force stepping, restored on the next stop.
pub pending_restore: Mutex<PendingRestore>,
/// Adapter breakpoint id → (file, index in the stored list).
@@ -87,107 +66,8 @@ pub(crate) struct Session {
/// Resolved path → the path as the IDE knows it, for adapters given resolved paths.
resolved_paths: Mutex<HashMap<PathBuf, PathBuf>>,
progress: Mutex<HashMap<String, TaskId>>,
/// Relay between a Kotlin adapter and its JVM. Dropping it closes both sockets.
pub jdwp_link: parking_lot::Mutex<Option<ide_jdwp::JdwpLink>>,
pub jvm_inspector: parking_lot::Mutex<Option<Arc<dyn ide_jvm_inspect::JvmInspector>>>,
/// One inspector request per session at a time.
pub inspect_lock: tokio::sync::Mutex<()>,
/// Cancelled when the session ends, so a request still waiting on the inspector lets go of
/// it (and of the JDWP link it holds).
pub ended: ide_core::CancellationToken,
/// `hostName` (empty when absent) and `port` of the session's `attach` request, as the IDE
/// was given them: Kotlin then replaces them with the link's. Unlike `jdwp_port`, also set
/// for a remote JVM.
pub attach_target: std::sync::OnceLock<(String, u16)>,
/// Set once `configure` has recorded where the program's JVM is (the fields above, the
/// Kotlin link), just before the `launch`/`attach` request goes out.
pub jvm_located: watch::Sender<bool>,
pub kotlin_relay: Mutex<Option<crate::kotlin_jdwp::StartupRelay>>,
pub kotlin_mirror: Mutex<Option<crate::kotlin::SourceMirror>>,
pub jvm_memory: tokio::sync::Mutex<crate::JvmMemoryState>,
/// Used only by a kotlin-lsp session: see [`Session::kotlin_lsp_quirks`].
pub(crate) quirks: kotlin_lsp::Quirks,
}
/// Which threads of the program are suspended: each from its `stopped` event until a
/// `continued` event or a successful resume/step request for it. Never cleared elsewhere, since
/// a stale "suspended" only makes a later check refuse. fwcd and java-debug stop a breakpoint's
/// thread alone, so a thread can run while another stays stopped.
///
/// A resume reply ends only the stops taken before the request was sent: a thread's next
/// `stopped` event can overtake the reply. The check and the removal happen under one lock, so
/// no stop can land between them and be lost.
#[derive(Default)]
pub(crate) struct Suspension(Mutex<Stops>);
#[derive(Default)]
struct Stops {
/// Counts stops, and moves back to an earlier stopped thread: an object read under one
/// ticket is stale under another.
ticket: u64,
/// Threads stopped alone by their event, oldest first, with the ticket of that stop.
threads: Vec<(i64, u64)>,
/// The ticket of a stop of every thread (`allThreadsStopped`, or no thread id): any resume
/// ends it.
all: Option<u64>,
}
impl Stops {
fn latest(&self) -> Option<u64> {
self.threads.last().map(|&(_, ticket)| ticket).max(self.all)
}
/// Ends the stops taken by ticket `upto` of `thread`, or of every thread.
fn release(&mut self, thread: Option<i64>, every: bool, upto: u64) {
let latest = self.latest();
if every || thread.is_none() || self.all.is_some_and(|t| t <= upto) {
self.all = self.all.filter(|&t| t > upto);
self.threads.retain(|&(_, t)| t > upto);
} else {
self.threads.retain(|&(id, t)| Some(id) != thread || t > upto);
}
// Back on an earlier thread's stop, as the JDWP link moves to it with a new generation.
let now = self.latest();
if now.is_some() && now != latest {
self.ticket += 1;
}
}
}
impl Suspension {
/// Whether some thread is stopped.
pub fn get(&self) -> bool {
let stops = self.0.lock();
stops.all.is_some() || !stops.threads.is_empty()
}
/// A `stopped` event; `all`: its `allThreadsStopped`.
pub fn stopped(&self, thread: Option<i64>, all: bool) {
let mut stops = self.0.lock();
stops.ticket += 1;
let ticket = stops.ticket;
match thread.filter(|_| !all) {
Some(thread) => {
stops.threads.retain(|&(id, _)| id != thread);
stops.threads.push((thread, ticket));
}
None => stops.all = Some(ticket),
}
}
/// A `continued` event; `all`: its `allThreadsContinued`.
pub fn continued(&self, thread: Option<i64>, all: bool) {
self.0.lock().release(thread, all, u64::MAX);
}
/// Take before sending a resume-like request and pass to [`Self::resumed`].
pub fn ticket(&self) -> u64 {
self.0.lock().ticket
}
/// After a successful resume-like request for `thread`; `all`: every thread ran.
pub fn resumed(&self, ticket: u64, thread: i64, all: bool) {
self.0.lock().release(Some(thread), all, ticket);
}
}
/// Adapters that only match files by their resolved path: Godot maps a script to
@@ -209,10 +89,6 @@ impl Session {
port: Option<u16>,
child: Option<Child>,
) -> Self {
// Before `child` moves into the session: the adapter is the root of the job, so
// everything it starts (java-debug's JVM) is in there by inheritance.
#[cfg(windows)]
let job = child.as_ref().and_then(tokio::process::Child::id).and_then(ide_core::process::Job::for_pid);
Self {
id,
owner,
@@ -227,8 +103,6 @@ impl Session {
adapter_process: Mutex::new(child),
debuggees: Mutex::new(Vec::new()),
program: Mutex::new(None),
#[cfg(windows)]
job: Mutex::new(job),
configured: AtomicBool::new(false),
initialized: watch::channel(false).0,
startup_cancel: ide_core::CancellationToken::new(),
@@ -236,38 +110,13 @@ impl Session {
terminated: AtomicBool::new(false),
stop_force: AtomicBool::new(false),
debuggee_pid: AtomicI64::new(0),
jvm_pid: AtomicI64::new(0),
jdwp_port: std::sync::OnceLock::new(),
listen_relay: AtomicBool::new(false),
suspended: Suspension::default(),
stop_pids: parking_lot::Mutex::new(Vec::new()),
#[cfg(windows)]
stop_tree: Mutex::new(None),
pending_restore: Mutex::default(),
breakpoint_ids: Mutex::new(HashMap::new()),
resolved_paths: Mutex::new(HashMap::new()),
progress: Mutex::new(HashMap::new()),
jdwp_link: parking_lot::Mutex::new(None),
jvm_inspector: parking_lot::Mutex::new(None),
inspect_lock: tokio::sync::Mutex::default(),
ended: ide_core::CancellationToken::new(),
attach_target: std::sync::OnceLock::new(),
jvm_located: watch::channel(false).0,
kotlin_relay: Mutex::new(None),
kotlin_mirror: Mutex::new(None),
jvm_memory: tokio::sync::Mutex::default(),
quirks: kotlin_lsp::Quirks::default(),
}
}
pub fn kotlin_lsp_quirks(&self) -> Option<&kotlin_lsp::Quirks> {
(self.adapter == kotlin_lsp::ADAPTER).then_some(&self.quirks)
}
/// Call before sending a request for thread `id`.
pub fn check_thread(&self, id: i64) -> ide_core::IdeResult<()> {
match self.kotlin_lsp_quirks() {
Some(quirks) => quirks.check_thread(self.suspended.ticket(), self.suspended.get(), id),
None => Ok(()),
}
}
@@ -299,11 +148,6 @@ impl Session {
/// A path from the adapter as the IDE knows it (undoes [`Session::adapter_path`]).
pub fn client_path(&self, path: &str) -> Option<PathBuf> {
if self.adapter == kotlin_lsp::ADAPTER
&& let Some(path) = kotlin_lsp::source_path(path)
{
return Some(path);
}
if let Some(real) = self.kotlin_mirror.lock().as_ref().and_then(|m| m.client_path(Path::new(path))) {
return Some(real);
}
@@ -433,28 +277,7 @@ impl Session {
/// Kills the adapter and every debuggee we started.
pub fn kill_processes(&self) {
self.ended.cancel();
// The inspector holds a clone of the link, so both go.
self.jvm_inspector.lock().take();
self.jdwp_link.lock().take();
#[cfg(unix)]
if self.request == "launch" {
for group in launch_groups(self) {
ide_core::process::kill_tree_of(group as u32);
}
}
// The job first: it ends the adapter and everything it started (the program
// java-debug launched is its child), members included that the stop's captured
// tree never saw. An empty job is unaffected, and whatever survives this session
// dies when the handle closes (`KILL_ON_JOB_CLOSE`).
#[cfg(windows)]
if let Some(job) = self.job.lock().as_ref() {
job.terminate();
}
#[cfg(windows)]
if let Some(tree) = self.stop_tree.lock().as_ref() {
tree.terminate();
}
self.kotlin_relay.lock().take();
if let Some(mut c) = self.adapter_process.lock().take() {
if self.adapter == "rdbg"
&& let Some(pid) = c.id()
@@ -468,9 +291,6 @@ impl Session {
}
if let Some(mut program) = self.program.lock().take() {
let _ = program.child.kill();
let _ = std::thread::Builder::new().name("openidea-debug-reap".into()).spawn(move || {
let _ = program.child.wait();
});
}
}
}
@@ -478,54 +298,47 @@ impl Session {
/// How long Stop waits for a launched program to exit before reporting the session ended.
pub(crate) const DEBUGGEE_EXIT_WAIT: Duration = Duration::from_secs(10);
pub(crate) async fn remember_debuggee(s: &Arc<Session>) {
#[cfg(windows)]
{
let Ok(pid) = u32::try_from(s.debuggee_pid.load(Ordering::Acquire)) else { return };
if s.request != "launch" || pid == 0 || s.stop_tree.lock().is_some() {
return;
}
if let Ok(tree) = tokio::task::spawn_blocking(move || ide_core::process::ProcessTree::capture(pid)).await {
s.stop_tree.lock().get_or_insert(tree);
}
/// The program and everything it started: an adapter may report the launcher shell, whose
/// child is the debugged program, so stopping only the reported pid leaves the program alive.
#[cfg(unix)]
fn process_tree(pid: i32) -> Vec<i32> {
use std::collections::HashMap;
let Ok(out) = std::process::Command::new("ps").args(["-A", "-o", "pid=,ppid="]).output() else {
return vec![pid];
};
let mut children: HashMap<i32, Vec<i32>> = HashMap::new();
for line in String::from_utf8_lossy(&out.stdout).lines() {
let mut it = line.split_whitespace();
let (Some(p), Some(parent)) = (it.next(), it.next()) else { continue };
let (Ok(p), Ok(parent)) = (p.parse::<i32>(), parent.parse::<i32>()) else { continue };
children.entry(parent).or_default().push(p);
}
#[cfg(not(windows))]
let _ = s;
let mut tree = vec![pid];
let mut at = 0;
while at < tree.len() {
if let Some(kids) = children.get(&tree[at]) {
tree.extend(kids.iter().copied());
}
at += 1;
}
tree
}
/// Asks the program itself to exit (`force` = SIGKILL) without ending the adapter session:
/// its output keeps flowing over DAP while it shuts down. The first call remembers the pid
/// tree, because the reported pid may die before its program does.
pub(crate) async fn signal_debuggee(s: &Arc<Session>, force: bool) {
if s.request != "launch" {
pub(crate) fn signal_debuggee(s: &Arc<Session>, force: bool) {
let pid = s.debuggee_pid.load(Ordering::Acquire);
if pid <= 0 {
return;
}
#[cfg(not(windows))]
let pid = s.debuggee_pid.load(Ordering::Acquire);
#[cfg(unix)]
{
let needs_tree = pid > 0 && s.stop_pids.lock().is_empty();
if needs_tree {
let tree = tokio::task::spawn_blocking(move || process_tree(pid as i32)).await.unwrap_or_default();
*s.stop_pids.lock() = tree;
let mut known = s.stop_pids.lock();
if known.is_empty() {
*known = process_tree(pid as i32);
}
let known = s.stop_pids.lock();
let signal = if force { libc::SIGKILL } else { libc::SIGTERM };
if force {
// A forced stop kills the launch: every root the session spawned leads its own
// process group (`own_group` at spawn), so one signal reaches the program and
// everything it started, even processes born after the snapshot below was
// taken — the adapter's program is in the adapter's group. The session's own
// JVM roots (Kotlin) and `runInTerminal` programs lead groups of their own.
// The roots are unreaped until the session drops them, so their ids (and the
// groups named by them) cannot have been reused yet.
for group in launch_groups(s) {
// SAFETY: plain syscalls on the processes this session started.
if unsafe { libc::kill(-group, libc::SIGKILL) } != 0 {
unsafe { libc::kill(group, libc::SIGKILL) };
}
}
}
// Children first: the shell that started the program would otherwise reparent it.
for p in known.iter().rev() {
// SAFETY: plain syscall on the processes this session started.
@@ -536,53 +349,17 @@ pub(crate) async fn signal_debuggee(s: &Arc<Session>, force: bool) {
}
#[cfg(windows)]
{
// Console programs have no external graceful signal; the adapter gets its chance
// first. The job ends the program and everything it started, members included
// that the captured tree never saw.
let mut cmd = std::process::Command::new("taskkill");
cmd.args(["/PID", &pid.to_string(), "/T"]);
if force {
if let Some(job) = s.job.lock().as_ref() {
job.terminate();
}
// A separately spawned program may have rejected assignment to the adapter's job.
if let Some(tree) = s.stop_tree.lock().as_ref() {
tree.terminate();
}
cmd.arg("/F");
}
let _ = cmd.status();
}
#[cfg(not(any(unix, windows)))]
let _ = (pid, force);
}
/// The process groups the session spawned roots for: the adapter's (the program it
/// launched is in it), the JVMs the session started itself, the `runInTerminal` program.
#[cfg(unix)]
fn launch_groups(s: &Session) -> Vec<libc::pid_t> {
let mut groups = Vec::new();
let mut push = |id: Option<u32>| {
if let Ok(pid) = libc::pid_t::try_from(id.unwrap_or(0)) {
groups.push(pid);
}
};
push(s.adapter_process.lock().as_ref().and_then(tokio::process::Child::id));
for child in s.debuggees.lock().iter() {
push(child.id());
}
push(s.program.lock().as_ref().and_then(|p| p.child.process_id()));
groups.retain(|g| *g > 0);
groups
}
/// Whether the launch is still running: the stop's captured tree when there is one,
/// otherwise the session's job (the adapter and everything it started).
#[cfg(windows)]
fn launch_running(s: &Arc<Session>) -> bool {
if let Some(tree) = s.stop_tree.lock().as_ref() {
tree.is_running()
} else {
s.job.lock().as_ref().is_some_and(|job| job.is_running())
}
}
/// Waits until the program a `launch` session started is gone; a second Stop kills it.
pub(crate) async fn wait_for_debuggee(s: &Arc<Session>, terminate: bool) {
if !terminate || s.request != "launch" {
@@ -596,39 +373,25 @@ pub(crate) async fn wait_for_debuggee(s: &Arc<Session>, terminate: bool) {
#[cfg(unix)]
{
use std::time::Instant;
let mut targets = s.stop_pids.lock().clone();
if targets.is_empty() {
targets = tokio::task::spawn_blocking(move || process_tree(pid as i32)).await.unwrap_or_default();
}
let targets = {
let known = s.stop_pids.lock();
if known.is_empty() { process_tree(pid as i32) } else { known.clone() }
};
let alive = || targets.iter().any(|p| unsafe { libc::kill(*p, 0) } == 0);
let deadline = Instant::now() + DEBUGGEE_EXIT_WAIT;
while alive() {
if s.stop_force.swap(false, Ordering::AcqRel) {
signal_debuggee(s, true).await;
signal_debuggee(s, true);
}
if Instant::now() >= deadline {
// The program ignored the terminate: do not leave it running.
signal_debuggee(s, true).await;
signal_debuggee(s, true);
break;
}
tokio::time::sleep(Duration::from_millis(50)).await;
}
}
#[cfg(windows)]
{
remember_debuggee(s).await;
let deadline = std::time::Instant::now() + DEBUGGEE_EXIT_WAIT;
// The captured tree is the oracle when it was captured; otherwise the job (every
// process the launch started) is, so a failed capture can no longer end the wait
// while the program still holds its port.
while launch_running(s) {
if s.stop_force.swap(false, Ordering::AcqRel) || std::time::Instant::now() >= deadline {
signal_debuggee(s, true).await;
}
tokio::time::sleep(Duration::from_millis(50)).await;
}
}
#[cfg(not(any(unix, windows)))]
#[cfg(not(unix))]
{
let _ = pid;
}
@@ -716,31 +479,18 @@ async fn on_event(mgr: &Arc<DebugManager>, s: &Arc<Session>, event: &str, mut bo
s.emit(DebugEvent::Initialized);
}
"stopped" => {
let all = body.get("allThreadsStopped").and_then(Value::as_bool).unwrap_or(false);
s.suspended.stopped(int("threadId"), all);
let mut reason = str_("reason").unwrap_or_else(|| "pause".into());
if let Some(quirks) = s.kotlin_lsp_quirks() {
quirks.saw_threads(s.suspended.ticket(), int("threadId"));
reason = quirks.stop_reason(reason);
}
mgr.restore_breakpoints(s).await;
s.emit(DebugEvent::Stopped {
thread_id: int("threadId").unwrap_or(0),
reason,
reason: str_("reason").unwrap_or_else(|| "pause".into()),
description: str_("description").or_else(|| str_("text")),
all_threads_stopped: all,
});
}
"continued" => {
let all = body.get("allThreadsContinued").and_then(Value::as_bool);
// DAP: an event without the flag continues its thread alone. The UI keeps treating
// it as all threads; the suspension, which holds only threads that stopped, need not.
s.suspended.continued(int("threadId"), all.unwrap_or(false));
s.emit(DebugEvent::Continued {
thread_id: int("threadId").unwrap_or(0),
all_threads_continued: all.unwrap_or(true),
all_threads_stopped: body.get("allThreadsStopped").and_then(Value::as_bool).unwrap_or(false),
});
}
"continued" => s.emit(DebugEvent::Continued {
thread_id: int("threadId").unwrap_or(0),
all_threads_continued: body.get("allThreadsContinued").and_then(Value::as_bool).unwrap_or(true),
}),
"output" => {
let category = str_("category").unwrap_or_else(|| "console".into());
if category == "telemetry" {
@@ -824,14 +574,6 @@ async fn on_event(mgr: &Arc<DebugManager>, s: &Arc<Session>, event: &str, mut bo
change_type: crate::hot_code_replace_type(body.get("changeType")),
message: str_("message").unwrap_or_default(),
}),
// Sent only by adapters told `supportsInvalidatedEvent` (kotlin-lsp).
"invalidated" => s.emit(DebugEvent::Invalidated {
areas: body
.get("areas")
.and_then(Value::as_array)
.map(|a| a.iter().filter_map(Value::as_str).map(str::to_owned).collect())
.unwrap_or_default(),
}),
_ => tracing::trace!("dap: ignored event {event}"),
}
}
@@ -923,14 +665,6 @@ pub(crate) fn run_in_terminal(s: &Arc<Session>, args: &Value) -> Result<Value, S
}
}
let mut child = pair.slave.spawn_command(cmd).map_err(|e| format!("failed to start {program}: {e}"))?;
// The program joins the session's job, so Stop reaches what it starts (its children),
// though only the program itself is the session's child.
#[cfg(windows)]
if let Some(pid) = child.process_id()
&& let Some(job) = s.job.lock().as_ref()
{
job.assign_pid(pid);
}
drop(pair.slave);
let reader = pair.master.try_clone_reader().map_err(|e| format!("terminal of {program}: {e}"))?;
let writer = pair.master.take_writer().map_err(|e| format!("terminal of {program}: {e}"))?;
@@ -1054,22 +788,10 @@ fn forward_output<R: AsyncRead + Unpin + Send + 'static>(s: &Session, mut stream
});
}
/// The attach listener must be on the command line before the user's arguments, which may
/// override it; the memory view needs it because the JVM is started by the IDE, not attached to.
/// Not on Windows (`windows`), which the memory view does not support.
fn kotlin_jvm_args(user: Vec<String>, windows: bool) -> Vec<String> {
if windows || !ide_jvm_attach::wants_listener_flag(&user.join(" ")) {
return user;
}
let mut args = Vec::with_capacity(user.len() + 1);
args.push(ide_jvm_attach::START_LISTENER_FLAG.to_owned());
args.extend(user);
args
}
/// Launch with the IDE's resolved runtime and classpath; the adapter only attaches.
pub(crate) async fn start_kotlin_debuggee(s: &Arc<Session>, args: &mut Value) -> ide_core::IdeResult<()> {
use tokio::io::{AsyncBufReadExt, BufReader};
let port = crate::adapter::free_port()?;
let java = args.get("javaExec").and_then(Value::as_str).unwrap_or("java");
let strings = |key: &str| -> Vec<String> {
args.get(key)
@@ -1081,10 +803,8 @@ pub(crate) async fn start_kotlin_debuggee(s: &Arc<Session>, args: &mut Value) ->
std::env::join_paths(strings("classPaths")).map_err(|e| ide_core::IdeError::invalid(e.to_string()))?;
let mut cmd = tokio::process::Command::new(java);
ide_core::process::prepare(cmd.as_std_mut());
cmd.args(kotlin_jvm_args(strings("vmArguments"), cfg!(windows)))
// Let the JVM pick its own port (`:0`) and report it below: pre-picking a free
// port here races with other processes that may take it before the JVM binds.
.arg("-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=127.0.0.1:0")
cmd.args(strings("vmArguments"))
.arg(format!("-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=127.0.0.1:{port}"))
.arg("-cp")
.arg(classpath);
let modules = strings("modulePaths");
@@ -1111,34 +831,20 @@ pub(crate) async fn start_kotlin_debuggee(s: &Arc<Session>, args: &mut Value) ->
}
}
}
// A root the session spawns itself: its own process group on Unix (`kill(-pgid)` in a
// forced stop), the session's job on Windows, so a stop reaches what it forks.
ide_core::process::own_group(cmd.as_std_mut());
let mut child = cmd.spawn()?;
if let Some(pid) = child.id() {
s.jvm_pid.store(i64::from(pid), Ordering::Release);
#[cfg(windows)]
if let Some(job) = s.job.lock().as_ref() {
job.assign_pid(pid);
}
}
if let Some(err) = child.stderr.take() {
forward_output(s, err, "stderr");
}
let mut stdout =
BufReader::new(child.stdout.take().ok_or_else(|| ide_core::IdeError::internal("Missing JVM stdout"))?);
// The JVM chose the port (`:0` above) and reports it here; reading it back removes
// the free-port race window entirely.
let port = tokio::time::timeout(Duration::from_secs(15), async {
tokio::time::timeout(Duration::from_secs(15), async {
loop {
let mut line = String::new();
if stdout.read_line(&mut line).await? == 0 {
return Err(ide_core::IdeError::unavailable("JVM exited before starting its debug listener"));
}
if line.contains("Listening for transport dt_socket") {
return parse_jdwp_port(&line).ok_or_else(|| {
ide_core::IdeError::unavailable(format!("JVM reported an unreadable debug port: {}", line.trim()))
});
return Ok(());
}
s.emit(DebugEvent::Output { category: "stdout".into(), output: line });
}
@@ -1152,177 +858,3 @@ pub(crate) async fn start_kotlin_debuggee(s: &Arc<Session>, args: &mut Value) ->
args["timeout"] = json!(10000);
Ok(())
}
/// The port a JVM chose, from `Listening for transport dt_socket at address: <port>`
/// (some JVMs print `host:port` or `[::1]:port`, so take what follows the last colon).
fn parse_jdwp_port(line: &str) -> Option<u16> {
line.rsplit_once("address:")?.1.trim().rsplit(':').next()?.trim().parse().ok()
}
#[cfg(test)]
mod tests {
use super::{Suspension, kotlin_jvm_args, parse_jdwp_port};
#[test]
fn a_resume_reply_clears_the_stop_it_was_sent_for() {
let s = Suspension::default();
assert!(!s.get());
s.stopped(Some(1), true);
let ticket = s.ticket();
s.resumed(ticket, 1, true);
assert!(!s.get());
}
#[test]
fn a_stop_after_the_resume_request_keeps_the_session_suspended() {
let s = Suspension::default();
s.stopped(Some(1), true);
let ticket = s.ticket();
// The next breakpoint's `stopped` event overtakes the reply to `continue`.
s.continued(Some(1), true);
s.stopped(Some(1), true);
s.resumed(ticket, 1, true);
assert!(s.get());
// A reply for the new stop still clears it.
s.resumed(s.ticket(), 1, true);
assert!(!s.get());
}
#[test]
fn a_reply_after_a_continued_event_changes_nothing() {
let s = Suspension::default();
s.stopped(Some(1), true);
let ticket = s.ticket();
s.continued(Some(1), true);
s.resumed(ticket, 1, true);
assert!(!s.get());
s.stopped(Some(1), true);
assert!(s.get());
}
#[test]
fn concurrent_stops_are_never_lost() {
let s = std::sync::Arc::new(Suspension::default());
s.stopped(Some(1), true);
let ticket = s.ticket();
let stopping = {
let s = s.clone();
std::thread::spawn(move || (0..10_000).for_each(|_| s.stopped(Some(1), true)))
};
for _ in 0..10_000 {
s.resumed(ticket, 1, true);
}
stopping.join().unwrap();
// However they interleave, a resume for the old ticket never clears a later stop.
assert!(s.get());
assert_eq!(s.ticket(), ticket + 10_000);
}
/// fwcd and java-debug stop a breakpoint's thread alone (`SUSPEND_EVENT_THREAD`): 7, then 8.
fn two_threads_stopped() -> Suspension {
let s = Suspension::default();
s.stopped(Some(7), false);
s.stopped(Some(8), false);
s
}
#[test]
fn continuing_the_latest_stopped_thread_keeps_the_other_as_a_new_stop() {
let s = two_threads_stopped();
let ticket = s.ticket();
s.continued(Some(8), false);
assert!(s.get());
// Objects read on thread 8's stop are stale.
assert_eq!(s.ticket(), ticket + 1);
s.continued(Some(7), false);
assert!(!s.get());
}
#[test]
fn continuing_an_earlier_stopped_thread_keeps_the_latest_stop() {
let s = two_threads_stopped();
let ticket = s.ticket();
s.continued(Some(7), false);
assert!(s.get());
assert_eq!(s.ticket(), ticket);
}
#[test]
fn continuing_all_threads_ends_every_stop() {
let s = two_threads_stopped();
s.continued(Some(7), true);
assert!(!s.get());
let s = two_threads_stopped();
s.continued(None, false);
assert!(!s.get());
}
#[test]
fn continuing_a_thread_that_did_not_stop_changes_nothing() {
let s = two_threads_stopped();
let ticket = s.ticket();
s.continued(Some(5), false);
assert!(s.get());
assert_eq!(s.ticket(), ticket);
}
#[test]
fn a_step_reply_resumes_its_thread_alone() {
let s = two_threads_stopped();
let ticket = s.ticket();
s.resumed(ticket, 8, false);
assert!(s.get());
assert_eq!(s.ticket(), ticket + 1);
s.resumed(s.ticket(), 7, true);
assert!(!s.get());
}
#[test]
fn a_thread_stopping_again_before_its_resume_reply_stays_stopped() {
let s = two_threads_stopped();
let ticket = s.ticket();
// Thread 7's step ends, and its `stopped` event overtakes the reply to `next`.
s.stopped(Some(7), false);
s.resumed(ticket, 7, false);
s.continued(Some(8), false);
assert!(s.get());
}
#[test]
fn a_stop_of_every_thread_ends_with_any_resume() {
let s = Suspension::default();
s.stopped(Some(7), false);
s.stopped(Some(8), true);
s.continued(Some(8), false);
assert!(!s.get());
// A stop without a thread.
s.stopped(None, false);
s.resumed(s.ticket(), 5, false);
assert!(!s.get());
}
#[test]
fn kotlin_launch_starts_attach_listener_first() {
assert_eq!(
kotlin_jvm_args(vec!["-Xmx1g".into()], false),
vec!["-XX:+StartAttachListener".to_string(), "-Xmx1g".into()]
);
assert_eq!(
kotlin_jvm_args(vec!["-XX:+DisableAttachMechanism".into()], false),
vec!["-XX:+DisableAttachMechanism".to_string()]
);
}
#[test]
fn kotlin_launch_on_windows_gets_no_attach_listener() {
assert_eq!(kotlin_jvm_args(vec!["-Xmx1g".into()], true), vec!["-Xmx1g".to_string()]);
}
#[test]
fn reads_the_port_the_jvm_chose() {
assert_eq!(parse_jdwp_port("Listening for transport dt_socket at address: 51234"), Some(51234));
assert_eq!(parse_jdwp_port("Listening for transport dt_socket at address: 127.0.0.1:51234"), Some(51234));
assert_eq!(parse_jdwp_port("Listening for transport dt_socket at address: [::1]:51234"), Some(51234));
assert_eq!(parse_jdwp_port("unrelated output"), None);
}
}
File diff suppressed because it is too large. Load diff
-401
View File
@@ -1,401 +0,0 @@
//! Instances, fields and referrers through a session's JVM inspector.
use super::*;
use crate::jvm_memory::test_support::FakeInspector;
use ide_core::event::JvmBackendState;
use ide_jvm_inspect::{InspectError, JvmValue, VmTarget};
use tokio::io::AsyncReadExt;
fn debug_events(h: &Harness) -> Vec<DebugEvent> {
h.events
.lock()
.iter()
.filter_map(|e| match e {
IdeEvent::Debug { event, .. } => Some(event.clone()),
_ => None,
})
.collect()
}
fn backend_events(h: &Harness) -> Vec<JvmBackendState> {
debug_events(h)
.into_iter()
.filter_map(|e| match e {
DebugEvent::JvmBackend { state } => Some(state),
_ => None,
})
.collect()
}
/// A java-debug session stopped at its breakpoint, inspected by `fake`.
async fn stopped_with(h: &Harness, config: LaunchConfig, fake: &Arc<FakeInspector>) -> DebugSessionId {
let id = h.start(config).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
h.manager.set_jvm_inspector(id, fake.clone()).unwrap();
id
}
fn object(id: u64) -> JvmValue {
JvmValue::Object { id, type_name: "A".into() }
}
const NOT_SUSPENDED: JvmInstancesResult =
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::NotSuspended };
#[tokio::test]
async fn instances_need_a_suspended_session() {
let h = harness(Script { continued_after_stop: true, ..Script::default() });
let id = h.start(java_config("launch")).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Continued { .. })).await;
let fake = Arc::new(FakeInspector::default());
h.manager.set_jvm_inspector(id, fake.clone()).unwrap();
assert_eq!(h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap(), NOT_SUSPENDED);
assert_eq!(fake.calls(), 0);
}
#[tokio::test]
async fn instances_without_an_inspector_have_no_backend() {
let h = harness(Script::default());
let id = h.start(java_config("launch")).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
assert_eq!(
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::NoBackend }
);
}
#[tokio::test]
async fn object_from_previous_stop_is_stale() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::default());
fake.instances.lock().push(object(7));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let JvmInstancesResult::Ready { generation, instances, truncated } =
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap()
else {
panic!("instances unavailable");
};
assert_eq!((generation, instances, truncated), (1, vec![object(7)], false));
h.manager.session(id).unwrap().suspended.stopped(None, true);
let stale = JvmObjectRef { generation, id: 7 };
assert_eq!(
h.manager.jvm_memory_fields(id, stale).await.unwrap(),
JvmFieldsResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
assert_eq!(
h.manager.jvm_memory_referrers(id, stale, 10).await.unwrap(),
JvmReferrersResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
assert_eq!(fake.calls(), 1, "a stale object never reaches the inspector");
let current = JvmObjectRef { generation: 2, id: 7 };
assert_eq!(h.manager.jvm_memory_fields(id, current).await.unwrap(), JvmFieldsResult::Ready { fields: Vec::new() });
}
#[tokio::test]
async fn instances_stay_readable_while_another_thread_is_stopped() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::default());
fake.instances.lock().push(object(7));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let session = h.manager.session(id).unwrap();
// Threads 20, 21, then 22 stop alone (SUSPEND_EVENT_THREAD).
session.suspended.continued(Some(1), true);
for thread in [20, 21, 22] {
session.suspended.stopped(Some(thread), false);
}
let instances = || h.manager.jvm_memory_instances(id, "A".into(), 10);
let result = instances().await.unwrap();
let JvmInstancesResult::Ready { generation, .. } = result else { panic!("{result:?}") };
let read_on_22 = JvmObjectRef { generation, id: 7 };
// An earlier thread resumes: the stop on 22 goes on, and so does what was read on it.
session.suspended.continued(Some(20), false);
assert_eq!(
h.manager.jvm_memory_fields(id, read_on_22).await.unwrap(),
JvmFieldsResult::Ready { fields: Vec::new() }
);
// The latest one resumes: 21 is still stopped, but what was read on 22 is stale.
session.suspended.continued(Some(22), false);
assert_eq!(
h.manager.jvm_memory_fields(id, read_on_22).await.unwrap(),
JvmFieldsResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
let result = instances().await.unwrap();
assert!(matches!(result, JvmInstancesResult::Ready { .. }), "{result:?}");
}
/// Starts `instances` on a held fake inspector; returns once the inspector was entered.
async fn held_request(
h: &Harness,
id: DebugSessionId,
fake: &Arc<FakeInspector>,
) -> tokio::task::JoinHandle<IdeResult<JvmInstancesResult>> {
let manager = h.manager.clone();
let request = tokio::spawn(async move { manager.jvm_memory_instances(id, "A".into(), 10).await });
tokio::time::timeout(Duration::from_secs(2), fake.entered.notified()).await.unwrap();
request
}
#[tokio::test]
async fn stop_during_request_is_stale() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector { hold: true, ..FakeInspector::default() });
fake.instances.lock().push(object(7));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let request = held_request(&h, id, &fake).await;
h.manager.session(id).unwrap().suspended.stopped(None, true);
fake.release.notify_one();
assert_eq!(
request.await.unwrap().unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
}
#[tokio::test]
async fn resume_during_request_is_stale() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector { hold: true, ..FakeInspector::default() });
let id = stopped_with(&h, java_config("launch"), &fake).await;
let request = held_request(&h, id, &fake).await;
h.manager.session(id).unwrap().suspended.continued(None, true);
fake.release.notify_one();
assert_eq!(
request.await.unwrap().unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
}
#[tokio::test]
async fn one_request_at_a_time_per_session() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector { hold: true, ..FakeInspector::default() });
let id = stopped_with(&h, java_config("launch"), &fake).await;
let first = held_request(&h, id, &fake).await;
let manager = h.manager.clone();
let second = tokio::spawn(async move { manager.jvm_memory_instances(id, "B".into(), 10).await });
tokio::time::sleep(Duration::from_millis(50)).await;
assert_eq!(fake.calls(), 1, "the second request waits for the first");
fake.release.notify_one();
first.await.unwrap().unwrap();
tokio::time::timeout(Duration::from_secs(2), fake.entered.notified()).await.unwrap();
fake.release.notify_one();
second.await.unwrap().unwrap();
assert_eq!(fake.calls(), 2);
}
#[tokio::test]
async fn session_end_lets_go_of_a_waiting_inspector() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector { hold: true, ..FakeInspector::default() });
let id = stopped_with(&h, java_config("launch"), &fake).await;
let request = held_request(&h, id, &fake).await;
h.manager.session(id).unwrap().kill_processes();
assert_eq!(
tokio::time::timeout(Duration::from_secs(2), request).await.unwrap().unwrap().unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::BackendClosed }
);
assert_eq!(Arc::strong_count(&fake), 1, "no clone of the inspector outlives the session");
}
#[tokio::test]
async fn target_carries_pid_for_launch() {
let h = harness(Script { debuggee_pid: Some(4242), java_process_id: true, ..Script::default() });
let fake = Arc::new(FakeInspector::default());
let id = stopped_with(&h, java_config("launch"), &fake).await;
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap();
assert_eq!(
*fake.targets.lock(),
[VmTarget { pid: Some(4242), attach_host: None, attach_port: None, generation: 1 }]
);
}
#[tokio::test]
async fn target_carries_attach_port_and_host() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let fake = Arc::new(FakeInspector::default());
let id = stopped_with(&h, remote_attach("java-debug", project.path()), &fake).await;
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap();
assert_eq!(
*fake.targets.lock(),
[VmTarget { pid: None, attach_host: Some("10.0.0.5".into()), attach_port: Some(5005), generation: 1 }]
);
}
#[tokio::test]
async fn unsupported_backend_turns_instances_off() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::failing(InspectError::Unsupported("x".into())));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let before = debug_events(&h).len();
assert_eq!(
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::Unsupported { detail: "x".into() } }
);
let after: Vec<_> = debug_events(&h).split_off(before);
assert!(
matches!(
after.as_slice(),
[DebugEvent::JvmBackend { state: JvmBackendState::Unsupported { detail } }, DebugEvent::Capabilities]
if detail == "x"
),
"{after:?}"
);
assert!(!h.manager.capabilities(id).unwrap().jvm_instances);
assert_eq!(
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::NoBackend }
);
assert_eq!(fake.calls(), 1);
}
/// `error` is reported but leaves the backend on.
async fn keeps_the_backend(error: InspectError, reason: JvmInspectUnavailable) {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::failing(error));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let object = JvmObjectRef { generation: 1, id: 7 };
assert_eq!(h.manager.jvm_memory_fields(id, object).await.unwrap(), JvmFieldsResult::Unavailable { reason });
assert!(h.manager.capabilities(id).unwrap().jvm_instances);
assert!(backend_events(&h).is_empty());
assert!(h.manager.session(id).unwrap().jvm_inspector.lock().is_some());
}
#[tokio::test]
async fn collected_keeps_the_backend() {
keeps_the_backend(InspectError::Collected, JvmInspectUnavailable::Collected).await;
}
#[tokio::test]
async fn timed_out_keeps_the_backend() {
keeps_the_backend(InspectError::TimedOut, JvmInspectUnavailable::TimedOut).await;
}
#[tokio::test]
async fn failed_inspection_is_an_error() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::failing(InspectError::Failed("boom".into())));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let err = h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap_err();
assert!(err.to_string().contains("boom"), "{err}");
assert!(h.manager.capabilities(id).unwrap().jvm_instances);
}
#[tokio::test]
async fn limits_are_clamped() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::default());
let id = stopped_with(&h, java_config("launch"), &fake).await;
for limit in [5000, 0] {
h.manager.jvm_memory_instances(id, "A".into(), limit).await.unwrap();
}
let object = JvmObjectRef { generation: 1, id: 7 };
for limit in [0, 900, 20] {
h.manager.jvm_memory_referrers(id, object, limit).await.unwrap();
}
assert_eq!(*fake.limits.lock(), [1000, 1, 50, 200, 20]);
}
#[tokio::test]
async fn a_full_page_is_truncated() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::default());
fake.instances.lock().extend([object(1), object(2)]);
let id = stopped_with(&h, java_config("launch"), &fake).await;
let truncated = |r: JvmInstancesResult| match r {
JvmInstancesResult::Ready { truncated, .. } => truncated,
other => panic!("{other:?}"),
};
assert!(truncated(h.manager.jvm_memory_instances(id, "A".into(), 2).await.unwrap()));
assert!(!truncated(h.manager.jvm_memory_instances(id, "A".into(), 3).await.unwrap()));
}
#[tokio::test]
async fn capability_follows_the_inspector() {
let h = harness(Script::default());
let id = h.start(java_config("launch")).await.unwrap();
assert!(!h.manager.capabilities(id).unwrap().jvm_instances);
let before = debug_events(&h).len();
h.manager.set_jvm_inspector(id, Arc::new(FakeInspector::default())).unwrap();
assert!(h.manager.capabilities(id).unwrap().jvm_instances);
assert!(matches!(debug_events(&h).split_off(before).as_slice(), [DebugEvent::Capabilities]));
}
#[tokio::test]
async fn remote_attach_counts_classes_through_the_debugger() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let fake = Arc::new(FakeInspector::default());
fake.counts.lock().push(ide_jvm_inspect::ClassCount { name: "A".into(), instances: 3 });
let id = stopped_with(&h, remote_attach("java-debug", project.path()), &fake).await;
let result = h.manager.jvm_memory_classes(id).await.unwrap();
assert!(
matches!(&result, JvmMemoryResult::Ready { snapshot }
if snapshot.source == JvmMemorySource::Debugger && snapshot.rows.len() == 1),
"{result:?}"
);
assert_eq!(fake.targets.lock()[0].attach_port, Some(5005));
}
const HANDSHAKE: &[u8; 14] = b"JDWP-Handshake";
/// A Kotlin attach whose link reaches a fake JVM: (adapter end, JVM end) after both handshakes.
async fn linked_kotlin(h: &Harness) -> (DebugSessionId, tokio::net::TcpStream, tokio::net::TcpStream) {
let jvm = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_attach(project.path(), "127.0.0.1", jvm.local_addr().unwrap().port())).await.unwrap();
let link_port = h.request("attach")["arguments"]["port"].as_u64().unwrap() as u16;
let mut adapter = tokio::net::TcpStream::connect(("127.0.0.1", link_port)).await.unwrap();
adapter.write_all(HANDSHAKE).await.unwrap();
let (mut jvm, _) = tokio::time::timeout(Duration::from_secs(2), jvm.accept()).await.unwrap().unwrap();
let mut buf = [0; 14];
jvm.read_exact(&mut buf).await.unwrap();
jvm.write_all(HANDSHAKE).await.unwrap();
adapter.read_exact(&mut buf).await.unwrap();
(id, adapter, jvm)
}
/// A command the adapter sends with an id from the link's own range: the link then only forwards.
async fn degrade(adapter: &mut tokio::net::TcpStream) {
let mut packet = 11u32.to_be_bytes().to_vec();
packet.extend(ide_jdwp::OWN_IDS.start.to_be_bytes());
packet.extend([0, 1, 1]);
adapter.write_all(&packet).await.unwrap();
}
async fn wait_backend(h: &Harness, want: impl Fn(&JvmBackendState) -> bool) {
h.wait_event(|e| matches!(e, DebugEvent::JvmBackend { state } if want(state))).await;
}
#[tokio::test]
async fn kotlin_link_reports_degraded_then_closed() {
let h = harness(Script::default());
let (id, mut adapter, jvm) = linked_kotlin(&h).await;
degrade(&mut adapter).await;
wait_backend(&h, |s| matches!(s, JvmBackendState::Degraded { .. })).await;
drop(jvm);
wait_backend(&h, |s| *s == JvmBackendState::Closed).await;
assert_eq!(backend_events(&h).len(), 2, "{:?}", backend_events(&h));
let session = h.manager.session(id).unwrap();
assert!(session.jdwp_link.lock().is_some(), "a closed link is kept until the session ends");
}
#[tokio::test]
async fn kotlin_link_is_quiet_once_the_program_ended() {
let h = harness(Script::default());
let (id, _adapter, jvm) = linked_kotlin(&h).await;
let session = h.manager.session(id).unwrap();
// The adapter reported the end; the session keeps its link until it shuts down.
session.terminated.store(true, Ordering::Release);
drop(jvm);
for _ in 0..100 {
if session.jdwp_link.lock().as_ref().is_some_and(|l| l.state() == ide_jdwp::LinkState::Closed) {
break;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
tokio::time::sleep(Duration::from_millis(50)).await;
assert!(backend_events(&h).is_empty(), "{:?}", backend_events(&h));
}
-20
View File
@@ -151,16 +151,6 @@ fn cases(dir: &Path) -> Vec<Case> {
field: None,
config: launch("lldb-dap", &c.join("main"), &c, serde_json::json!({"env": ["OPENIDEA_FIXTURE=1"]})),
},
// GDB's own DAP interpreter needs GDB 14+; older ones are skipped by `installed`.
Case {
adapter: "gdb",
source: c.join("main.c"),
loop_line: 13,
after_loop: 15,
function: "parse",
field: None,
config: launch("gdb", &c.join("main"), &c, serde_json::json!({})),
},
]);
out
}
@@ -240,16 +230,6 @@ fn installed(adapter: &str) -> bool {
"js-debug" => "js-debug-adapter",
"delve" => "dlv",
"java-debug" => return true,
// Only a GDB with the DAP interpreter (14+) counts as installed.
"gdb" => {
return std::process::Command::new("gdb").arg("--version").output().is_ok_and(|out| {
String::from_utf8_lossy(&out.stdout)
.lines()
.next()
.and_then(adapter::parse_gdb_version)
.is_some_and(|(major, _)| major >= 14)
});
}
other => other,
};
adapter::resolve_binary(cmd, Path::new("/")).is_some()
+2 -8
View File
@@ -17,11 +17,10 @@ blake3.workspace = true
tokio.workspace = true
tracing.workspace = true
parking_lot.workspace = true
notify.workspace = true
futures.workspace = true
sqlx.workspace = true
tiberius.workspace = true
oracledb.workspace = true
oracle.workspace = true
bb8.workspace = true
tokio-util = { workspace = true, features = ["compat"] }
dirs.workspace = true
@@ -30,13 +29,8 @@ ide-http.workspace = true
reqwest = { workspace = true, features = ["query"] }
rustls.workspace = true
uuid.workspace = true
base64.workspace = true
flate2 = "1"
rsa = { version = "0.10.0-rc.18", features = ["sha2", "pkcs5"] }
zeroize = "1"
open = "5"
[dev-dependencies]
tokio = { workspace = true, features = ["rt-multi-thread", "macros", "net"] }
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
tempfile.workspace = true
ide-ssh = { workspace = true, features = ["testing"] }
-188
View File
@@ -1,188 +0,0 @@
//! The introspected catalog of a data source, kept on disk: the schemas, tables and views of
//! its databases, so completion knows them from the start of the next run without reading the
//! server again. Names only — no data, no credentials. One file per project and data source in
//! the app data directory.
use crate::TableInfo;
use ide_core::{IdeError, IdeResult};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::path::{Path, PathBuf};
/// A schema of a database and its tables and views.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbCatalogSchema")]
pub struct CatalogSchema {
pub name: String,
pub tables: Vec<TableInfo>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbCatalogDatabase")]
pub struct CatalogDatabase {
pub name: String,
/// When this database was read, in milliseconds since the Unix epoch.
pub read_at: f64,
pub schemas: Vec<CatalogSchema>,
}
/// What introspection has read of a data source.
#[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase", default)]
#[serde(rename = "DbSourceCatalog")]
pub struct SourceCatalog {
pub databases: Vec<CatalogDatabase>,
}
pub(crate) struct CatalogStore {
dir: PathBuf,
project: String,
/// One writer at a time: an update reads the file, changes a database and writes it back.
writing: tokio::sync::Mutex<()>,
}
fn now_ms() -> f64 {
std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map_or(0.0, |d| d.as_millis() as f64)
}
impl CatalogStore {
pub(crate) fn new(dir: PathBuf, root: &Path) -> Self {
Self { dir, project: root.to_string_lossy().into_owned(), writing: tokio::sync::Mutex::new(()) }
}
fn file(&self, id: &str) -> PathBuf {
let name = blake3::hash(format!("{}\0{id}", self.project).as_bytes()).to_hex();
self.dir.join(format!("{}.json", &name.as_str()[..32]))
}
fn read(file: &Path) -> SourceCatalog {
std::fs::read(file).ok().and_then(|bytes| serde_json::from_slice(&bytes).ok()).unwrap_or_default()
}
fn write(file: &Path, catalog: &SourceCatalog) -> IdeResult<()> {
if let Some(dir) = file.parent() {
std::fs::create_dir_all(dir)?;
}
// Written beside the file and moved over it, so a crash leaves the old catalog whole.
let partial = file.with_extension("json.part");
let bytes = serde_json::to_vec(catalog).map_err(|e| IdeError::internal(e.to_string()))?;
std::fs::write(&partial, bytes)?;
std::fs::rename(&partial, file)?;
Ok(())
}
async fn blocking<T: Send + 'static>(task: impl FnOnce() -> T + Send + 'static) -> IdeResult<T> {
tokio::task::spawn_blocking(task).await.map_err(|e| IdeError::internal(e.to_string()))
}
/// The saved catalog of data source `id` (`None`: never introspected).
pub(crate) async fn load(&self, id: &str) -> IdeResult<Option<SourceCatalog>> {
let file = self.file(id);
Self::blocking(move || file.exists().then(|| Self::read(&file))).await
}
/// Saves what was just read of `database`, replacing what was known of it.
pub(crate) async fn put(&self, id: &str, database: &str, schemas: Vec<CatalogSchema>) -> IdeResult<()> {
let _writing = self.writing.lock().await;
let (file, name) = (self.file(id), database.to_owned());
Self::blocking(move || {
let mut catalog = Self::read(&file);
catalog.databases.retain(|d| d.name != name);
catalog.databases.push(CatalogDatabase { name, read_at: now_ms(), schemas });
catalog.databases.sort_by(|a, b| a.name.cmp(&b.name));
Self::write(&file, &catalog)
})
.await?
}
/// Forgets the databases not in `keep` (dropped on the server, or no longer shown).
pub(crate) async fn retain(&self, id: &str, keep: Vec<String>) -> IdeResult<()> {
let _writing = self.writing.lock().await;
let file = self.file(id);
Self::blocking(move || {
if !file.exists() {
return Ok(());
}
let mut catalog = Self::read(&file);
let before = catalog.databases.len();
catalog.databases.retain(|d| keep.contains(&d.name));
if catalog.databases.len() == before { Ok(()) } else { Self::write(&file, &catalog) }
})
.await?
}
/// Deletes the saved catalog of data source `id` (the data source was removed).
pub(crate) fn forget(&self, id: &str) {
let _ = std::fs::remove_file(self.file(id));
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::DbTableKind;
fn schema(name: &str, tables: &[&str]) -> CatalogSchema {
CatalogSchema {
name: name.into(),
tables: tables.iter().map(|t| TableInfo { name: (*t).into(), kind: DbTableKind::Table }).collect(),
}
}
#[tokio::test]
async fn a_catalog_survives_a_restart_per_project_and_data_source() {
let dir = tempfile::tempdir().unwrap();
let store = CatalogStore::new(dir.path().join("db-catalogs"), Path::new("/projects/shop"));
assert_eq!(store.load("sf").await.unwrap(), None);
store.put("sf", "RAW", vec![schema("PUBLIC", &["EVENTS"])]).await.unwrap();
store
.put("sf", "ANALYTICS", vec![schema("PUBLIC", &["ORDERS", "USERS"]), schema("STAGING", &[])])
.await
.unwrap();
// A new store over the same directory: what the next run of the app sees.
let reopened = CatalogStore::new(dir.path().join("db-catalogs"), Path::new("/projects/shop"));
let saved = reopened.load("sf").await.unwrap().unwrap();
assert_eq!(saved.databases.iter().map(|d| d.name.as_str()).collect::<Vec<_>>(), ["ANALYTICS", "RAW"]);
assert_eq!(saved.databases[0].schemas, vec![schema("PUBLIC", &["ORDERS", "USERS"]), schema("STAGING", &[])]);
assert!(saved.databases.iter().all(|d| d.read_at > 0.0));
// Another data source and another project keep their own.
assert_eq!(reopened.load("other").await.unwrap(), None);
let elsewhere = CatalogStore::new(dir.path().join("db-catalogs"), Path::new("/projects/blog"));
assert_eq!(elsewhere.load("sf").await.unwrap(), None);
}
#[tokio::test]
async fn rereading_replaces_a_database_and_dropped_ones_are_forgotten() {
let dir = tempfile::tempdir().unwrap();
let store = CatalogStore::new(dir.path().to_path_buf(), Path::new("/p"));
store.put("sf", "RAW", vec![schema("PUBLIC", &["OLD"])]).await.unwrap();
store.put("sf", "GONE", vec![schema("PUBLIC", &["T"])]).await.unwrap();
store.put("sf", "RAW", vec![schema("PUBLIC", &["NEW"])]).await.unwrap();
store.retain("sf", vec!["RAW".into()]).await.unwrap();
let saved = store.load("sf").await.unwrap().unwrap();
assert_eq!(saved.databases.len(), 1);
assert_eq!(saved.databases[0].schemas, vec![schema("PUBLIC", &["NEW"])]);
// Nothing half-written is left beside the file.
let files: Vec<_> = std::fs::read_dir(dir.path()).unwrap().map(|f| f.unwrap().file_name()).collect();
assert_eq!(files.len(), 1);
store.forget("sf");
assert_eq!(store.load("sf").await.unwrap(), None);
store.retain("sf", vec![]).await.unwrap();
assert_eq!(store.load("sf").await.unwrap(), None);
}
#[tokio::test]
async fn a_damaged_file_reads_as_an_empty_catalog() {
let dir = tempfile::tempdir().unwrap();
let store = CatalogStore::new(dir.path().to_path_buf(), Path::new("/p"));
std::fs::write(store.file("sf"), b"{ not json").unwrap();
assert_eq!(store.load("sf").await.unwrap(), Some(SourceCatalog::default()));
store.put("sf", "RAW", vec![schema("PUBLIC", &["T"])]).await.unwrap();
assert_eq!(store.load("sf").await.unwrap().unwrap().databases.len(), 1);
}
}
+14 -121
View File
@@ -238,13 +238,6 @@ pub enum Pool {
SqlServer(Bb8Pool<MssqlConnectionManager>),
Clickhouse(crate::clickhouse::ClickhousePool),
Oracle(crate::oracle::OraclePool),
Snowflake(crate::snowflake::SnowflakePool),
}
/// Some servers and proxies (e.g. Aliyun RDS for MySQL 5.6) advertise `CLIENT_DEPRECATE_EOF` but
/// still end result sets with legacy EOF packets, which sqlx then fails to read as OK packets.
fn legacy_eof(e: &sqlx::Error) -> bool {
matches!(e, sqlx::Error::Protocol(msg) if msg.starts_with("OK_Packet too short"))
}
pub(crate) fn db_err(e: sqlx::Error) -> IdeError {
@@ -290,16 +283,6 @@ fn mssql_pool_err(e: bb8::RunError<tiberius::error::Error>) -> IdeError {
}
}
/// Tiberius requires TLS when a TLS backend is enabled. `trust_cert` only skips
/// certificate checks, so Disable has to turn encryption off.
fn apply_sqlserver_ssl(config: &mut tiberius::Config, ssl: DbSslMode) {
match ssl {
DbSslMode::Disable => config.encryption(tiberius::EncryptionLevel::NotSupported),
DbSslMode::Prefer => config.trust_cert(),
DbSslMode::Require | DbSslMode::VerifyCa | DbSslMode::VerifyFull => {}
}
}
const DAYS_1900_01_01: i64 = 693_595;
/// Formats a proleptic Gregorian date given days since `0001-01-01`.
@@ -434,7 +417,7 @@ fn binary_type(name: &str, text: bool) -> bool {
}
/// Bytes as text: UTF-8, else hex (grids) or with invalid sequences replaced (catalog names).
pub(crate) fn bytes_text(bytes: Vec<u8>, mode: Cells) -> String {
fn bytes_text(bytes: Vec<u8>, mode: Cells) -> String {
match (String::from_utf8(bytes), mode) {
(Ok(s), _) => s,
(Err(e), Cells::Display) => hex(e.as_bytes()),
@@ -611,13 +594,6 @@ fn is_identifier(name: &str) -> bool {
&& chars.all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '.')
}
/// sqlx's handshake flags for `sql_mode`: with either enabled it sends
/// `SET sql_mode=(SELECT CONCAT(...))`, a non-constant subquery that MySQL-protocol
/// servers like StarRocks reject (#2452), so the StarRocks preset turns both off.
fn mysql_mode_flags(vendor: &str) -> (bool, bool) {
if vendor == "starrocks" { (false, false) } else { (true, true) }
}
/// A MySQL `SET` value: numbers and keywords as they are, anything else quoted.
fn mysql_value(v: &str) -> String {
let keyword = ["ON", "OFF", "TRUE", "FALSE", "DEFAULT"].iter().any(|k| k.eq_ignore_ascii_case(v));
@@ -710,9 +686,6 @@ impl Setup {
if source.driver == DbDriver::Clickhouse && !o.time_zone.trim().is_empty() {
s.params.push(("session_timezone".into(), o.time_zone.trim().into()));
}
if source.driver == DbDriver::Snowflake && !o.time_zone.trim().is_empty() {
s.params.push(("TIMEZONE".into(), o.time_zone.trim().into()));
}
let script = o.startup_script.trim();
if !script.is_empty() {
s.statements.extend(crate::split::split(script, source.driver).into_iter().map(|st| st.text));
@@ -730,7 +703,6 @@ impl Pool {
Pool::SqlServer(_) => DbDriver::SqlServer,
Pool::Clickhouse(_) => DbDriver::Clickhouse,
Pool::Oracle(_) => DbDriver::Oracle,
Pool::Snowflake(_) => DbDriver::Snowflake,
}
}
@@ -752,9 +724,6 @@ impl Pool {
if source.driver == DbDriver::Oracle {
return crate::oracle::OraclePool::connect(source, password, &setup).await.map(Pool::Oracle);
}
if source.driver == DbDriver::Snowflake {
return crate::snowflake::SnowflakePool::connect(source, password, setup).await.map(Pool::Snowflake);
}
let timeout = setup.connect_timeout.unwrap_or(Duration::from_secs(30));
let init = Arc::new(setup.statements);
// Runs the session statements (startup script…) on every new connection.
@@ -870,8 +839,6 @@ impl Pool {
if let Some(c) = &setup.collation {
opts = opts.collation(c);
}
let (pipes_as_concat, no_engine_substitution) = mysql_mode_flags(&source.vendor);
opts = opts.pipes_as_concat(pipes_as_concat).no_engine_substitution(no_engine_substitution);
if let Some(n) = setup.statement_cache {
opts = opts.statement_cache_capacity(n);
}
@@ -879,16 +846,9 @@ impl Pool {
if !tz.is_empty() {
opts = opts.timezone(Some(tz.to_owned()));
}
let mut manager = MysqlManager { options: opts, init: init.clone(), timeout };
let manager = MysqlManager { options: opts, init: init.clone(), timeout };
// Open once directly so authentication failures retain their database error.
let first = match manager.connect().await {
Err(e) if legacy_eof(&e) => {
manager.options = manager.options.clone().deprecate_eof(false);
manager.connect().await
}
first => first,
}
.map_err(db_err)?;
let first = manager.connect().await.map_err(db_err)?;
let pool = Bb8Pool::builder().max_size(MAX).connection_timeout(timeout).build_unchecked(manager);
let _ = pool.add(first);
Ok(Pool::Mysql(ServerPool { inner: pool, closed: AtomicBool::new(false) }))
@@ -907,7 +867,9 @@ impl Pool {
if !source.database.is_empty() {
config.database(&source.database);
}
apply_sqlserver_ssl(&mut config, source.ssl);
if matches!(source.ssl, DbSslMode::Disable | DbSslMode::Prefer) {
config.trust_cert();
}
let manager = MssqlConnectionManager(config);
let pool = bb8::Pool::builder()
.max_size(MAX)
@@ -920,7 +882,7 @@ impl Pool {
Ok(Pool::SqlServer(pool))
}
DbDriver::Clickhouse => unreachable!(),
DbDriver::Oracle | DbDriver::Snowflake => unreachable!(),
DbDriver::Oracle => unreachable!(),
}
}
@@ -928,7 +890,6 @@ impl Pool {
match self {
Pool::SqlServer(_) | Pool::Oracle(_) => {}
Pool::Clickhouse(p) => p.close(),
Pool::Snowflake(p) => p.close().await,
Pool::Sqlite(p) => p.close().await,
Pool::Postgres(p) => p.closed.store(true, Ordering::Relaxed),
Pool::Mysql(p) => p.closed.store(true, Ordering::Relaxed),
@@ -940,7 +901,6 @@ impl Pool {
Pool::SqlServer(_) => false,
Pool::Oracle(p) => p.is_closed(),
Pool::Clickhouse(p) => p.is_closed(),
Pool::Snowflake(p) => p.is_closed(),
Pool::Sqlite(p) => p.is_closed(),
Pool::Postgres(p) => p.closed.load(Ordering::Relaxed),
Pool::Mysql(p) => p.closed.load(Ordering::Relaxed),
@@ -952,9 +912,6 @@ impl Pool {
if let Pool::Clickhouse(p) = self {
return Ok(p.execute(sql, max, &CancellationToken::new()).await?.rows);
}
if let Pool::Snowflake(p) = self {
return Ok(p.metadata().execute(sql, max, &CancellationToken::new()).await?.rows);
}
let mut conn = Held::acquire(self).await?;
conn.apply_context(None).await?;
let fetched = conn.fetch_rows(sql, max).await;
@@ -979,9 +936,6 @@ impl Pool {
if let Pool::Clickhouse(p) = self {
return p.execute_from(sql, skip, max_rows, cancel).await;
}
if let Pool::Snowflake(p) = self {
return p.execute_from(sql, skip, max_rows, cancel).await;
}
let mut conn = Held::acquire(self).await?;
let r = match conn.apply_context(None).await {
Ok(()) => conn.execute_from(self, sql, skip, max_rows, cancel).await,
@@ -999,7 +953,6 @@ impl Pool {
pub async fn transaction(&self, statements: &[(String, bool)]) -> IdeResult<u64> {
match self {
Pool::Clickhouse(_) => Err(crate::clickhouse::transaction_error()),
Pool::Snowflake(p) => p.transaction(statements).await,
Pool::SqlServer(_) => {
let mut held = Held::acquire(self).await?;
held.apply_context(None).await?;
@@ -1080,9 +1033,7 @@ pub enum Held {
Mysql(Box<bb8::PooledConnection<'static, MysqlManager>>),
SqlServer(Box<bb8::PooledConnection<'static, MssqlConnectionManager>>),
Clickhouse(crate::clickhouse::ClickhousePool),
Oracle(Box<crate::oracle::OracleConnection>),
/// The shared session, and the schema this holder's statements run in.
Snowflake(crate::snowflake::SnowflakePool, Option<String>),
Oracle(crate::oracle::OracleConnection),
}
impl Held {
@@ -1098,13 +1049,7 @@ impl Held {
}
Held::Clickhouse(p.clone())
}
Pool::Oracle(p) => Held::Oracle(Box::new(p.get().await?)),
Pool::Snowflake(p) => {
if p.is_closed() {
return Err(IdeError::new(ErrorKind::Io, "Snowflake connection is closed"));
}
Held::Snowflake(p.clone(), None)
}
Pool::Oracle(p) => Held::Oracle(p.get().await?),
})
}
@@ -1149,11 +1094,6 @@ impl Held {
}
pub async fn apply_context(&mut self, schema: Option<&str>) -> IdeResult<()> {
if let Held::Snowflake(_, wanted) = self {
// Applied with each statement, while it holds the shared session.
*wanted = schema.filter(|s| !s.is_empty()).map(str::to_owned);
return Ok(());
}
let driver = match self {
Held::Postgres(_) => DbDriver::Postgres,
Held::Mysql(_) => DbDriver::Mysql,
@@ -1209,7 +1149,7 @@ impl Held {
Held::Mysql(c) => fetch!(&mut ****c, sql, max, Cells::Text),
Held::SqlServer(c) => mssql_fetch(c, sql, 0, max).await,
Held::Oracle(c) => crate::oracle::run(c, sql, 0, max, Cells::Text, &CancellationToken::new()).await,
Held::Clickhouse(_) | Held::Snowflake(..) => unreachable!(),
Held::Clickhouse(_) => unreachable!(),
};
if result.is_ok() {
self.after_statement(sql);
@@ -1231,12 +1171,6 @@ impl Held {
match self {
Held::SqlServer(c) => mssql_command(c, sql).await,
Held::Clickhouse(_) => Err(crate::clickhouse::transaction_error()),
Held::Snowflake(c, schema) => {
if sql.trim().is_empty() {
return Ok(());
}
c.execute_in(schema.as_deref(), sql, 0, 0, &CancellationToken::new()).await.map(drop)
}
Held::Oracle(c) => {
let sql = sql.trim();
if sql.is_empty() {
@@ -1250,9 +1184,7 @@ impl Held {
Held::Sqlite(c) => sqlx::raw_sql(sql).execute(&mut **c).await.map(drop),
Held::Postgres(c) => sqlx::raw_sql(sql).execute(&mut ****c).await.map(drop),
Held::Mysql(c) => sqlx::raw_sql(sql).execute(&mut ****c).await.map(drop),
Held::SqlServer(_) | Held::Clickhouse(_) | Held::Oracle(_) | Held::Snowflake(..) => {
unreachable!()
}
Held::SqlServer(_) | Held::Clickhouse(_) | Held::Oracle(_) => unreachable!(),
}
.map_err(db_err)
}
@@ -1277,7 +1209,7 @@ impl Held {
Held::SqlServer(mut c) => {
c.0.take();
}
Held::Clickhouse(_) | Held::Snowflake(..) => {}
Held::Clickhouse(_) => {}
Held::Oracle(mut c) => c.discard_inner(),
}
}
@@ -1307,9 +1239,6 @@ impl Held {
if let Held::Clickhouse(c) = self {
return c.execute_from(sql, skip, max_rows, cancel).await;
}
if let Held::Snowflake(c, schema) = self {
return c.execute_in(schema.as_deref(), sql, skip, max_rows, cancel).await;
}
self.before_statement(sql);
let started = Instant::now();
// `$id`: the query naming the server process, `$kill`: how another connection stops it.
@@ -1354,7 +1283,7 @@ impl Held {
}
}
Held::Oracle(c) => crate::oracle::run(c, sql, skip, max_rows, Cells::Display, cancel).await?,
Held::Clickhouse(_) | Held::Snowflake(..) => unreachable!(),
Held::Clickhouse(_) => unreachable!(),
};
self.after_statement(sql);
let (columns, rows, affected, has_more, first) = fetched;
@@ -1377,7 +1306,7 @@ pub fn begin_statement(driver: DbDriver) -> &'static str {
DbDriver::Mysql => "START TRANSACTION",
DbDriver::SqlServer => "BEGIN TRAN",
DbDriver::Oracle => "", // opens implicitly with autocommit off
DbDriver::Sqlite | DbDriver::Postgres | DbDriver::Clickhouse | DbDriver::Snowflake => "BEGIN",
DbDriver::Sqlite | DbDriver::Postgres | DbDriver::Clickhouse => "BEGIN",
}
}
@@ -1393,33 +1322,6 @@ mod tests {
use super::*;
use crate::source::{DbOptions, DbProperty};
fn sqlserver_debug(ssl: DbSslMode) -> String {
let mut config = tiberius::Config::new();
apply_sqlserver_ssl(&mut config, ssl);
format!("{config:?}")
}
#[test]
fn sqlserver_disable_does_not_require_tls() {
let debug = sqlserver_debug(DbSslMode::Disable);
assert!(debug.contains("encryption: NotSupported"), "{debug}");
assert!(debug.contains("bypass: false"), "{debug}");
}
#[test]
fn sqlserver_prefer_trusts_the_server_certificate() {
let debug = sqlserver_debug(DbSslMode::Prefer);
assert!(debug.contains("encryption: Required"), "{debug}");
assert!(debug.contains("bypass: true"), "{debug}");
}
#[test]
fn sqlserver_require_keeps_validated_tls() {
let debug = sqlserver_debug(DbSslMode::Require);
assert!(debug.contains("encryption: Required"), "{debug}");
assert!(debug.contains("bypass: false"), "{debug}");
}
#[test]
fn discarded_sqlserver_connections_are_broken() {
let manager = MssqlConnectionManager(tiberius::Config::new());
@@ -1563,15 +1465,6 @@ mod tests {
assert!(Setup::of(&DataSource { properties: vec![prop("connect_timeout", "soon")], ..s }).is_err());
}
#[test]
fn starrocks_disables_the_non_constant_sql_mode_handshake() {
// MySQL and its other wire-compatible presets keep sqlx's defaults.
for vendor in ["", "mysql", "mariadb", "tidb"] {
assert_eq!(mysql_mode_flags(vendor), (true, true), "{vendor}");
}
assert_eq!(mysql_mode_flags("starrocks"), (false, false));
}
#[test]
fn sqlite_properties_become_pragmas() {
let s = DataSource { properties: vec![prop("cache_size", "-2000")], ..Default::default() };
+17 -290
View File
@@ -2,7 +2,6 @@
//! execution ([`conn`]), introspection, table data paging / editing ([`sql`])
//! and statement splitting ([`split`]), for SQLite, PostgreSQL and MySQL/MariaDB.
pub mod catalog;
mod clickhouse;
pub mod conn;
pub mod history;
@@ -10,20 +9,15 @@ mod oracle;
pub mod plan;
pub mod plugin;
pub mod secrets;
mod snowflake;
pub mod source;
pub mod split;
pub mod sql;
mod watch;
pub use watch::SourcesWatch;
pub use catalog::{CatalogDatabase, CatalogSchema, SourceCatalog};
pub use conn::{Held, Pool, QueryResult, ResultColumn};
pub use history::HistoryEntry;
pub use plan::{PlanNode, PlanProperty};
pub use source::{
DataSource, DbColor, DbDriver, DbIntrospection, DbOptions, DbProperty, DbSnowflake, DbSnowflakeAuth, DbSsh,
DbSshAuth, DbSslMode, DbTxMode,
DataSource, DbColor, DbDriver, DbIntrospection, DbOptions, DbProperty, DbSsh, DbSshAuth, DbSslMode, DbTxMode,
};
pub use sql::{CellValue, DbRowEdit, SortKey};
@@ -63,7 +57,6 @@ impl DbTableKind {
pub enum DbObjectKind {
Function,
Procedure,
Package,
Trigger,
Sequence,
}
@@ -99,37 +92,13 @@ pub struct SequenceInfo {
pub data_type: String,
}
/// An Oracle package (its procedures and functions are not listed separately).
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbPackageInfo")]
pub struct PackageInfo {
pub name: String,
}
/// A schema's routines (procedures and functions), packages and sequences.
/// A schema's routines and sequences (the tree's `routines` / `sequences` folders).
#[derive(Debug, Clone, PartialEq, Eq, Default, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbSchemaObjects")]
pub struct SchemaObjects {
pub routines: Vec<RoutineInfo>,
pub packages: Vec<PackageInfo>,
pub sequences: Vec<SequenceInfo>,
/// Objects of kinds only some databases have (Snowflake stages, streams, tasks…).
#[serde(default)]
pub others: Vec<SchemaObject>,
}
/// A schema object the tree lists by name under a folder of its kind.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbSchemaObject")]
pub struct SchemaObject {
/// `stage`, `stream`, `task`, `pipe`, `fileFormat`.
pub kind: String,
pub name: String,
/// What the list says about it (a stage's type, a stream's table, a task's schedule).
pub detail: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
@@ -225,18 +194,6 @@ pub struct TablePage {
pub offset: f64,
}
/// What a Snowflake session computes with and acts as, and what it may switch to.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbSessionContext")]
pub struct SessionContext {
/// Empty: no warehouse is in use.
pub warehouse: String,
pub role: String,
pub warehouses: Vec<String>,
pub roles: Vec<String>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbDatabases")]
@@ -261,8 +218,6 @@ pub struct DbManager {
/// Directory of the global data sources and their saved passwords ([`DbManager::with_global_sources`]).
global: Option<(PathBuf, SecretStore)>,
history: Arc<HistoryStore>,
/// What introspection read of each data source, kept between runs.
catalogs: catalog::CatalogStore,
/// Records executed statements in the history off the async threads, in order.
recorder: std::sync::mpsc::Sender<(String, String, f64)>,
/// Passwords entered this session (not saved, or not yet), by credential key.
@@ -270,9 +225,6 @@ pub struct DbManager {
/// Manual-commit data sources, each with the connection of its open
/// transaction (taken by the first statement after a commit / rollback).
sessions: Mutex<HashMap<String, Session>>,
/// The data sources as last read from disk, for [`DbManager::reconcile_sources`]
/// (`None` until [`DbManager::watch_sources`]).
on_disk: Mutex<Option<Vec<DataSource>>>,
}
type Session = Arc<tokio::sync::Mutex<Option<Held>>>;
@@ -344,7 +296,7 @@ fn through_tunnel(e: IdeError, source: &DataSource, via: Option<&str>) -> IdeErr
/// Options › Run keep-alive query: pings the pool until it is closed or dropped
/// (a dropped SSH tunnel or an idle-killing proxy stays alive meanwhile).
pub(crate) fn keep_alive(pool: &Arc<Pool>, every: Duration) {
fn keep_alive(pool: &Arc<Pool>, every: Duration) {
let pool = Arc::downgrade(pool);
tokio::spawn(async move {
let mut tick = tokio::time::interval(every);
@@ -374,9 +326,6 @@ impl DbManager {
/// `passwords` / `history` are the app-wide files of saved passwords
/// ([`SecretStore`]) and query history ([`HistoryStore`]).
pub fn new(root: PathBuf, passwords: PathBuf, history: PathBuf, ssh: Arc<SshManager>) -> Self {
// Beside the query history, in the app data directory.
let catalogs = history.parent().unwrap_or(&root).join("db-catalogs");
let catalogs = catalog::CatalogStore::new(catalogs, &root);
let history = Arc::new(HistoryStore::new(history, &root));
let (recorder, statements) = std::sync::mpsc::channel::<(String, String, f64)>();
{
@@ -396,7 +345,6 @@ impl DbManager {
saved_passwords: SecretStore::new(passwords, &root),
global: None,
history,
catalogs,
recorder,
root,
pools: Mutex::default(),
@@ -404,7 +352,6 @@ impl DbManager {
tunnel_open: tokio::sync::Mutex::new(()),
ssh,
session_passwords: Mutex::default(),
on_disk: Mutex::default(),
sessions: Mutex::default(),
}
}
@@ -460,10 +407,6 @@ impl DbManager {
.ok_or_else(|| IdeError::not_found(format!("no data source {id}")))?
.resolved()?;
if let Some(db) = database {
// The configured schema belongs to the configured database.
if source.driver == DbDriver::Snowflake && !source.database.eq_ignore_ascii_case(db) {
source.options.default_schema.clear();
}
source.database = db.to_owned();
}
Ok(source)
@@ -498,10 +441,9 @@ impl DbManager {
match sources.iter().find(|s| s.id == o.id) {
Some(n) if *n == o => {}
gone_or_changed => {
let keys = [o.id.clone(), secrets::ssh_key(&o.id), Self::sso_credential(&o.id)];
let keys = [o.id.clone(), secrets::ssh_key(&o.id)];
match gone_or_changed {
None => {
self.catalogs.forget(&o.id);
for k in &keys {
self.session_passwords.lock().remove(k);
self.store_of(&o).delete(k);
@@ -522,45 +464,9 @@ impl DbManager {
}
}
}
// Our own write is not reported as an edit on disk.
if let Some(known) = self.on_disk.lock().as_mut() {
*known = self.sources().unwrap_or(sources);
}
Ok(())
}
/// Watches the data source files (the project's and the global one) for edits made
/// outside the IDE; `on_change` runs on a background thread once they settle, and
/// should call [`DbManager::reconcile_sources`]. Stops when the result is dropped.
pub fn watch_sources(&self, on_change: Box<dyn Fn() + Send>) -> IdeResult<SourcesWatch> {
*self.on_disk.lock() = Some(self.sources().unwrap_or_default());
SourcesWatch::start(&self.root, self.global.as_ref().map(|(dir, _)| dir.as_path()), on_change)
}
/// Re-reads the data sources after their files changed on disk: connections of
/// removed or changed sources close, as on [`DbManager::save_sources`] (saved
/// passwords stay, the source may come back). Returns whether the list the tree
/// shows changed, or the files no longer parse (the tree reports the error).
pub async fn reconcile_sources(&self) -> bool {
let now = match self.sources() {
Ok(now) => now,
Err(e) => {
tracing::warn!("data sources: {}", e.message);
return true;
}
};
let before = self.on_disk.lock().replace(now.clone()).unwrap_or_default();
if before == now {
return false;
}
for o in &before {
if now.iter().find(|s| s.id == o.id) != Some(o) {
self.disconnect_source(&o.id).await;
}
}
true
}
/// Remembers `password` for the session, and in the saved passwords when
/// the data source saves passwords (`None` forgets it).
pub async fn set_password(&self, id: &str, password: Option<String>, save: bool) -> IdeResult<()> {
@@ -655,9 +561,7 @@ impl DbManager {
ssh_secret: Option<String>,
shared: bool,
) -> IdeResult<(DataSource, Option<Arc<Tunnel>>)> {
// Snowflake is HTTPS to a named host: a tunnel's local end cannot pass its certificate check.
let tunneled = !matches!(source.driver, DbDriver::Sqlite | DbDriver::Snowflake);
let Some(ssh) = source.ssh.as_ref().filter(|s| s.enabled && tunneled) else {
let Some(ssh) = source.ssh.as_ref().filter(|s| s.enabled && source.driver != DbDriver::Sqlite) else {
return Ok((source.clone(), None));
};
let local = |t: &Tunnel| DataSource { host: "127.0.0.1".into(), port: Some(t.port()), ..source.clone() };
@@ -711,10 +615,7 @@ impl DbManager {
None => self.password(source).await,
};
let (target, tunnel) = self.endpoint(source, ssh_secret, false).await?;
self.restore_sso(source).await;
let connected = Pool::connect(&target, password.as_deref(), Some(&self.root)).await;
self.keep_sso(source).await;
let pool = connected.map_err(|e| {
let pool = Pool::connect(&target, password.as_deref(), Some(&self.root)).await.map_err(|e| {
through_tunnel(
e,
source,
@@ -727,16 +628,10 @@ impl DbManager {
DbDriver::Mysql => "SELECT CONCAT(@@version_comment, ' ', VERSION())",
DbDriver::SqlServer => "SELECT @@VERSION",
DbDriver::Oracle => "SELECT banner FROM v$version WHERE banner LIKE 'Oracle%' AND ROWNUM = 1",
DbDriver::Snowflake => "SELECT 'Snowflake ' || CURRENT_VERSION()",
};
let result = match &pool {
// On the session the login opened: metadata has its own, which would sign in again.
Pool::Snowflake(p) => p.execute(version, 1, &CancellationToken::new()).await.map(|r| r.rows),
_ => pool.rows(version, 1).await,
};
let result = pool.rows(version, 1).await;
pool.close().await;
let strict = matches!(source.driver, DbDriver::Clickhouse | DbDriver::Snowflake);
let rows = if strict { result? } else { result.unwrap_or_default() };
let rows = if source.driver == DbDriver::Clickhouse { result? } else { result.unwrap_or_default() };
let v = rows.into_iter().next().and_then(|r| r.into_iter().next().flatten());
Ok(v.unwrap_or_else(|| "Connected".into()))
}
@@ -763,10 +658,7 @@ impl DbManager {
let source = self.source(id)?;
let password = self.password(&source).await;
let (target, tunnel) = self.endpoint(&source, None, true).await?;
self.restore_sso(&source).await;
let connected = Pool::connect(&target, password.as_deref(), Some(&self.root)).await;
self.keep_sso(&source).await;
let pool = match connected {
let pool = match Pool::connect(&target, password.as_deref(), Some(&self.root)).await {
Ok(p) => Arc::new(p),
Err(e) => {
self.drop_unused_tunnel(source_id);
@@ -779,10 +671,9 @@ impl DbManager {
};
// Another caller may have connected meanwhile; keep theirs.
let kept = self.pools.lock().entry(id.to_owned()).or_insert_with(|| pool.clone()).clone();
if !Arc::ptr_eq(&kept, &pool) {
// Ours lost the race: close it rather than leave its session open on the server.
pool.close().await;
} else if let Some(secs) = source.options.keep_alive_secs.filter(|s| *s > 0) {
if Arc::ptr_eq(&kept, &pool)
&& let Some(secs) = source.options.keep_alive_secs.filter(|s| *s > 0)
{
keep_alive(&pool, Duration::from_secs(secs.into()));
}
Ok(kept)
@@ -832,23 +723,6 @@ impl DbManager {
let names = first(pool.rows(sql::sqlserver_databases(), MAX_META_ROWS).await?).collect();
return Ok(Databases { current, names });
}
if pool.driver() == DbDriver::Snowflake {
let current = pool
.rows("SELECT CURRENT_DATABASE()", 1)
.await?
.into_iter()
.next()
.and_then(|r| r.into_iter().next().flatten())
.unwrap_or_default();
// `SHOW` needs no running warehouse and lists what the role may see. Columns: created_on, name…
let names = pool
.rows("SHOW TERSE DATABASES", MAX_META_ROWS)
.await?
.into_iter()
.filter_map(|r| r.into_iter().nth(1).flatten())
.collect();
return Ok(Databases { current, names });
}
if pool.driver() != DbDriver::Postgres {
return Err(IdeError::invalid("this data source does not list databases"));
}
@@ -863,95 +737,17 @@ impl DbManager {
Ok(Databases { current, names })
}
/// The schemas of `database` of data source `id` with their tables and views (Snowflake):
/// what completion needs to follow `database.schema.table` without the tree being opened.
/// The result is also saved ([`DbManager::saved_catalog`]) for the next run.
pub async fn database_catalog(&self, id: &str, database: &str) -> IdeResult<Vec<CatalogSchema>> {
let id = split_target(id).0;
let schemas = match &*self.pool(id).await? {
Pool::Snowflake(p) => snowflake::meta::catalog(p, database).await?,
_ => return Err(IdeError::invalid("this data source does not list other databases")),
};
if let Err(e) = self.catalogs.put(id, database, schemas.clone()).await {
// Completion still has it for this run.
tracing::warn!("could not save the catalog of {database}: {}", e.message);
}
Ok(schemas)
}
/// What earlier introspection of data source `id` saved (`None`: never introspected).
/// Read from disk: the data source is not connected to.
pub async fn saved_catalog(&self, id: &str) -> IdeResult<Option<SourceCatalog>> {
self.catalogs.load(split_target(id).0).await
}
/// Drops the saved databases of data source `id` that are not in `databases` any more.
pub async fn retain_catalog(&self, id: &str, databases: Vec<String>) -> IdeResult<()> {
self.catalogs.retain(split_target(id).0, databases).await
}
/// Credential key of the single sign-on ID token of data source `id`.
fn sso_credential(id: &str) -> String {
format!("{id}#sso")
}
/// Hands the saved single sign-on token of `source` to the driver, so it signs in without the browser.
async fn restore_sso(&self, source: &DataSource) {
let Some(key) = snowflake::sso_key(source) else { return };
if snowflake::sso_token(&key).is_none()
&& let Some(saved) = self.store_of(source).get(&Self::sso_credential(&source.id))
{
snowflake::sso_seed(&key, &saved);
}
}
/// Keeps the token the last sign-in of `source` left in the credential store (or forgets a refused one).
async fn keep_sso(&self, source: &DataSource) {
let Some(key) = snowflake::sso_key(source) else { return };
if source.id.is_empty() {
return;
}
let credential = Self::sso_credential(&source.id);
let store = self.store_of(source);
let (now, saved) = (snowflake::sso_token(&key), store.get(&credential));
if now == saved {
return;
}
match now {
Some(token) => {
if let Err(e) = store.set(&credential, &token) {
tracing::warn!("could not save the single sign-on token: {}", e.message);
}
}
None => store.delete(&credential),
}
}
/// The warehouse and role of the session of data source `id` (Snowflake), and its choices.
pub async fn session_context(&self, id: &str) -> IdeResult<SessionContext> {
match &*self.pool(id).await? {
Pool::Snowflake(p) => snowflake::meta::session(p).await,
_ => Err(IdeError::invalid("this data source has no warehouse or role")),
}
}
/// Every schema (the Schemas tab picks the ones the tree shows); system schemas
/// only with [`DbIntrospection::show_system`].
pub async fn schemas(&self, id: &str) -> IdeResult<Vec<String>> {
let system = self.source(id)?.introspection.show_system;
let pool = self.pool(id).await?;
if let Pool::Snowflake(p) = &*pool {
return snowflake::meta::schemas(p, system).await;
}
let rows = pool.rows(sql::schemas(pool.driver(), system), MAX_META_ROWS).await?;
Ok(rows.into_iter().filter_map(|r| r.into_iter().next().flatten()).collect())
}
pub async fn tables(&self, id: &str, schema: &str) -> IdeResult<Vec<TableInfo>> {
let pool = self.pool(id).await?;
if let Pool::Snowflake(p) = &*pool {
return snowflake::meta::tables(p, schema).await;
}
let rows = pool.rows(&sql::tables(pool.driver(), schema), MAX_META_ROWS).await?;
Ok(rows
.into_iter()
@@ -965,9 +761,6 @@ impl DbManager {
pub async fn table_details(&self, id: &str, schema: &str, table: &str) -> IdeResult<TableDetails> {
let pool = self.pool(id).await?;
if let Pool::Snowflake(p) = &*pool {
return snowflake::meta::table_details(p, schema, table).await;
}
let driver = pool.driver();
let columns = pool
.rows(&sql::columns(driver, schema, table), MAX_META_ROWS)
@@ -995,7 +788,7 @@ impl DbManager {
_ => indexes.push(IndexInfo { name, unique: flag(&unique), columns: vec![col] }),
}
}
let fk_rows = if matches!(driver, DbDriver::Clickhouse | DbDriver::Snowflake) {
let fk_rows = if driver == DbDriver::Clickhouse {
Vec::new()
} else {
pool.rows(&sql::table_foreign_keys(driver, schema, table), MAX_META_ROWS).await.unwrap_or_default()
@@ -1012,7 +805,7 @@ impl DbManager {
.collect();
}
foreign_keys.retain(|k| k.columns.len() == k.ref_columns.len());
let trigger_rows = if matches!(driver, DbDriver::Clickhouse | DbDriver::Snowflake) {
let trigger_rows = if driver == DbDriver::Clickhouse {
Vec::new()
} else {
pool.rows(&sql::triggers(driver, schema, table), MAX_META_ROWS).await.unwrap_or_else(|e| {
@@ -1039,9 +832,6 @@ impl DbManager {
/// The routines and sequences of `schema`.
pub async fn schema_objects(&self, id: &str, schema: &str) -> IdeResult<SchemaObjects> {
let pool = self.pool(id).await?;
if let Pool::Snowflake(p) = &*pool {
return snowflake::meta::schema_objects(p, schema).await;
}
let driver = pool.driver();
let mut out = SchemaObjects::default();
if let Some(q) = sql::routines(driver, schema) {
@@ -1063,14 +853,6 @@ impl DbManager {
})
.collect();
}
if let Some(q) = sql::packages(driver, schema) {
out.packages = pool
.rows(&q, MAX_META_ROWS)
.await?
.into_iter()
.filter_map(|r| Some(PackageInfo { name: r.into_iter().next().flatten()? }))
.collect();
}
if let Some(q) = sql::sequences(driver, schema) {
out.sequences = pool
.rows(&q, MAX_META_ROWS)
@@ -1088,37 +870,9 @@ impl DbManager {
Ok(out)
}
/// The definition of a routine (every overload), package (spec and body), trigger or sequence.
/// The definition of a routine (every overload), trigger or sequence.
pub async fn object_ddl(&self, id: &str, schema: &str, kind: DbObjectKind, name: &str) -> IdeResult<String> {
let pool = self.pool(id).await?;
if let Pool::Snowflake(p) = &*pool {
let defs = snowflake::meta::object_ddl(p, schema, kind, name).await?;
if defs.is_empty() {
return Err(IdeError::not_found(format!("{schema}.{name}")));
}
let defs: Vec<String> = defs.iter().map(|d| format!("{};\n", d.trim_end().trim_end_matches(';'))).collect();
return Ok(defs.join("\n"));
}
if kind == DbObjectKind::Package {
let mut defs: Vec<String> = Vec::new();
for body in [false, true] {
let Some(q) = sql::package_ddl(pool.driver(), schema, name, body) else { continue };
match pool.rows(&q, MAX_META_ROWS).await {
Ok(rows) => defs.extend(rows.into_iter().filter_map(|r| r.into_iter().next().flatten())),
// A spec-only package has no body: the spec alone still opens.
Err(e) if body => tracing::warn!("package body {schema}.{name}: {}", e.message),
Err(e) => return Err(e),
}
}
if defs.is_empty() {
return Err(IdeError::not_found(format!("{schema}.{name}")));
}
return Ok(defs
.iter()
.map(|d| format!("{};\n", d.trim_end().trim_end_matches(';')))
.collect::<Vec<_>>()
.join("\n"));
}
let (q, column) = sql::object_ddl(pool.driver(), schema, kind, name)
.ok_or_else(|| IdeError::invalid(format!("no {kind:?} objects in this database")))?;
let defs: Vec<String> = pool
@@ -1199,26 +953,6 @@ impl DbManager {
.next();
ddl.map(|d| format!("{};\n", d.trim_end().trim_end_matches(';'))).ok_or_else(missing)
}
DbDriver::Snowflake => {
let path = match &*pool {
Pool::Snowflake(p) => p.database_ident().map(|db| format!("{db}.")).unwrap_or_default(),
_ => String::new(),
};
let target = sql::quote_literal(driver, &format!("{path}{}", sql::qualified(driver, schema, name)));
let mut ddl = None;
// Views answer to their own object type on some accounts.
for kind in ["TABLE", "VIEW"] {
match pool.rows(&format!("SELECT GET_DDL('{kind}', {target})"), 1).await {
Ok(rows) => {
ddl = cells(rows, 0).into_iter().next();
break;
}
Err(_) if kind == "TABLE" => {}
Err(e) => return Err(e),
}
}
ddl.map(|d| format!("{};\n", d.trim_end().trim_end_matches(';'))).ok_or_else(missing)
}
DbDriver::Postgres => {
let rel = pool.rows(&sql::postgres_relation(schema, name), 1).await?.into_iter().next();
let (oid, kind) = match rel.as_deref() {
@@ -1267,14 +1001,7 @@ impl DbManager {
let pool = self.pool(id).await?;
let driver = pool.driver();
let mut tables: Vec<CompletionTable> = Vec::new();
let rows = match &*pool {
Pool::Snowflake(p) => {
tables = snowflake::meta::completion(p).await?;
Vec::new()
}
_ => pool.rows(sql::completion_columns(driver), MAX_META_ROWS).await?,
};
for r in rows {
for r in pool.rows(sql::completion_columns(driver), MAX_META_ROWS).await? {
let mut it = r.into_iter().map(Option::unwrap_or_default);
let mut next = || it.next().unwrap_or_default();
let (schema, name, kind) = (next(), next(), next());
@@ -1299,7 +1026,7 @@ impl DbManager {
}
}
// Foreign keys are a nicety (JOIN completion): a server that refuses the catalog query still completes.
let fk_rows = if matches!(driver, DbDriver::Clickhouse | DbDriver::Snowflake) {
let fk_rows = if driver == DbDriver::Clickhouse {
Vec::new()
} else {
pool.rows(sql::foreign_keys(driver), MAX_META_ROWS).await.unwrap_or_else(|e| {
File diff suppressed because it is too large. Load diff
Loaded 100 of 2407 files, more files were not shown because too many files have changed in this diff. Show more