Compare commits

..
7 Commits
Author SHA1 Message Date
github-actions[bot] d597f0e095 chore(main): release 0.11.0 (#1146)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-16 11:28:23 +08:00
HazyLabandcl-ment ce2fcffaf1 perf(redaction): add fast-path byte checks to eliminate regex allocations on clean strings (#945)
* perf(redaction): add fast-path byte checks to eliminate regex allocations on clean strings

Evaluating cascading regular expressions across every log line, tool output,
and session event incurred ~23 µs and 51 heap allocations even on strings with
zero secrets.

This introduces fast-path substring checks before invoking expensive regex
substitutions, yielding a 70x speedup and 0 B/op heap allocation on clean
strings.

* test(redaction): lengthen ghp_ fixtures and cover conditional gates

GitHub classic tokens need 36 characters after ghp_. Benchmarks and
shape tests now use a matching fixture. A table-driven test checks each
fast-path gate, including near-misses and case-insensitive Authorization.

* perf(redaction): structure triggered patterns and avoid allocation in header case checks

* test(redaction): cover secret-header and URL-password gates

Keep the redaction marker literal after url.URL rewrites userinfo, so
callers still see [REDACTED] instead of a percent-encoded form.

* fix(redaction): unescape only the generated URL password

A path or query that already contains the encoded redaction marker
must stay encoded. Only the userinfo password is rewritten in the
clear [REDACTED] form.

* fix(redaction): re-escape URL username after parse

Username() is decoded. Re-encoding with url.User keeps control bytes
out of the redacted URL so a %0AAuthorization userinfo cannot inject
a header after the header pass.

* fix(redaction): fail closed on password URLs without a host

* test(redaction): isolate query-gate coverage from assign and text secrets

* fix(redaction): fold Unicode U+017F and U+212A in containsCaseInsensitive

Go (?i) regular expressions apply Unicode simple case folding, where U+017F (ſ) folds to 's' and U+212A (K) folds to 'k'. Teach containsCaseInsensitive to fold these runes to ASCII lowercase with zero allocations so case-insensitive header gates cannot be bypassed by Unicode-folded equivalents.

---------

Co-authored-by: cl-ment <[email protected]>
2026-10-15 21:15:42 +08:00
Kevin Codex 61b09364f0 feat(providers): add Concentrate provider preset (#1147)
* feat(providers): add Concentrate provider preset

Register Concentrate (https://api.concentrate.ai/v1) as an
OpenAI-compatible catalog provider. Concentrate is already listed in the
README Partners section; this adds the provider itself.

- models.dev has no Concentrate provider, so the id `concentrate` and the
  key name CONCENTRATE_API_KEY match OpenClaude's Concentrate
  integration. `Concentrate AI`, `concentrate.ai` and `concentrate-ai`
  resolve to it.
- Default model deepseek-v4-flash-0731: the dated id behind the
  deepseek-v4-flash alias that OpenClaude uses. Concentrate's /v1/models
  keeps the rolling name only in `aliases`, and onboarding keeps the
  default selected only when it matches a live id, so an alias default
  made setup land on the first model in the list (claude-fable-5).
- Curated models: deepseek-v4-flash-0731 (default), deepseek-v4-pro,
  kimi-k3 and glm-5.3. Context windows (1,048,576 each), tool calling and
  thinking support come from Concentrate's public /v1/models.
- Regression test: a Concentrate-shaped live list must contain the
  preset's default model.
- README and README_ZH: key line and setup command.

* test(providers): prove the Concentrate default check runs on the live list

The regression test passed whenever the default model was in the result,
and the curated fallback also contains it. Assert that the mock served
/v1/models and that the result includes claude-fable-5, a model only the
live response lists, so a silent fall back to the curated catalog fails.
2026-10-15 21:14:16 +08:00
fchengyan d1a0d048ee fix(cli): let providers subcommands run with no active provider (#1143)
* fix(cli): let providers subcommands run with no active provider

`zero providers list`, `use`, `remove` and `rename` exited 3 with
`no active provider configured: active provider "" not found` as soon as the
environment supplied more than one provider and config.json had no active
provider: normalizeProvidersWithOptions only elects an active provider by
itself when there is exactly one, and resolveCommandCenterConfig treated the
resulting error as fatal, so the command stopped before doing anything. A
base URL without a key counts as a second provider too (applyProviderEnv
builds a profile from a base URL alone).

The resolver already returns the successfully normalized list alongside
ErrNoActiveProvider, and the interactive TUI already falls back to a usable
entry from it in runInteractiveTUIWithSetup. Do the same here instead of
failing, so these subcommands can answer for providers that only come from
the environment.

Also pin the four env-derived provider tests to a fixed environment and a
temporary user config. They set OPENAI_API_KEY but resolved through the real
resolver against the developer's own config.json and ambient provider
variables, so they failed on any machine that also exported another provider
variable and whose user config had no active provider.

Adds regression coverage for the two-env-provider case: `providers list`
must report every detected provider and `use`/`remove`/`rename` must give
their usual env-derived answer. Verified against the fix (pass) and with the
fix reverted (exit 3, `active provider "" not found`).

Fixes #1130

* refactor(cli): report the configured active provider, not an elected one

Provider-management subcommands answer for the SAVED configuration, while
the TUI's recovery path answers "what can I run with". Adopting
firstUsableProvider in resolveCommandCenterConfig conflated the two: with
two env-derived providers it made `providers current`/`list` present a
provider that `zero exec` would still refuse to run with, and a
base-URL-only profile failed `providers list` because the election demands
a credential. Return the resolver's partial config untouched instead, so
nothing is reported as active when nothing is selected.

Also removes the OAuth store read this path picked up with firstUsableProvider.

Adds assertions that `current` shows nothing selected, `list` marks nothing
active, `use --json` leaves activeProvider empty, plus a base-URL-only case.

* fix(cli): scope the no-active-provider exception to provider management

Returning the resolver's partial config made `zero config` exit 0 with a
partial summary, contradicting the resolver's documented headless contract:
`zero config` / `zero exec` still fail with the actionable message and only
the interactive TUI recovers from a missing active provider. Split the
helper so only issue #1130's commands — `providers list`/`current` and the
unpersisted `use`/`remove`/`rename` reporters — tolerate the state; `zero
config`, `detect`, `models` and `check` keep failing as before.

Also pin the OAuth store in providerEnvSetupDeps: summarizeConfig reads
OAuth login state through the default store, so a developer's stored login
could flip these assertions. Point ZERO_OAUTH_TOKENS_PATH at a temp file
and force ZERO_OAUTH_STORAGE=file so an ambient keyring setting cannot pull
the OS keychain in either.
2026-10-15 15:40:43 +08:00
Fabio Tavares Leitão c88c77bf70 fix(workspaceindex): skip git-ignored and build-cache dirs in Scan only (#1117)
* fix(workspaceindex): skip git-ignored and build-cache dirs in Scan only

Scan has a fixed file budget, and build output (Cargo target/, Python
virtualenvs, caches) could exhaust it before real source was reached:
repo-map reported truncation and ranked fingerprint files above code.

Inside a git work tree, Scan now asks git for the ignored set
(`git status --porcelain=v1 -z --ignored=matching .`), so the repo's own
.gitignore, nested ignore files, info/exclude and core.excludesFile all
apply, including names no fixed list anticipates (.locust_env/). Outside
a work tree, or when git is missing, older than 2.16, fails, or exceeds
5s, a fixed list of common build/cache directory names stands in.

ShouldSkipDir is unchanged: glob, grep, list_directory and path
autocomplete share it and must keep seeing target/, .venv/ and friends.
The lookup runs with GIT_OPTIONAL_LOCKS=0 so a per-turn scan never takes
.git/index.lock next to a concurrent git command.

Fixes #1108

* fix(workspaceindex): skip a rename's original-path field when parsing status

In `git status --porcelain=v1 -z`, a rename or copy record is followed by one
more NUL field holding the original path, with no XY prefix. The parser
inspected every field, so an original name beginning with "!! " read as an
ignored entry: after `git mv "!! a" b`, a tracked file "a" disappeared from
Scan.

Move the parsing into parseIgnoredStatus, which consumes that field for R or C
in either status column, and test it directly and end to end.

* fix(workspaceindex): disable repo fsmonitor and force untracked-files=normal for status

Scan's git status lookup must not run a repository's core.fsmonitor hook or fail
when status.showUntrackedFiles=no would reject --ignored=matching.

* fix(workspaceindex): ignore inherited git repository location

Git exports GIT_DIR and GIT_WORK_TREE to hooks. With those pointing at
another repository, Scan kept .locust_env/lib/x.py, which root's own
.gitignore excludes.

* fix(workspaceindex): quote the fsmonitor hook path in the regression

On Windows, sh treats an unquoted backslash path as escapes, so the hook
ran and the marker the test stats was never created. Require a status
without the override to create that marker first.

* fix(workspaceindex): pass the fsmonitor hook path to git with forward slashes

Smoke (windows-latest) failed TestGitIgnoredPathsDisablesRepoFsmonitor
because core.fsmonitor was set to a backslash script path. Git for Windows
hands that value to sh, which consumes the separators, so the hook never
runs and the marker is never created. The test now stores a quoted forward-slash
path. The product code is unchanged.
2026-10-15 15:34:28 +08:00
Vasanth T 640d936bfb fix(tui,cli): stop a repository's git config from running programs in Zero's own git calls (#1133)
* fix(tui,cli): stop a repository's git config from running programs in Zero's own git calls

Zero runs git status and git diff on its own: the TUI's files sweep at
startup and after each command, its pull request diffstat, and the
scratch-file warning after a run. A repository's own config can name
programs those commands run (core.fsmonitor on every status, hooks when
the index is rewritten, clean filters and diff drivers assigned through
.gitattributes), so a workspace unpacked from an archive ran its own code
on the first turn, although Zero refuses the same workspace's project
hooks and plugins until the user trusts it.

internal/gitsafe builds those commands with core.fsmonitor and
core.hooksPath turned off and GIT_OPTIONAL_LOCKS=0, and drops the
repository-location variables git exports to hooks, using the same
environment for its config check and for the command. A repository whose
own config, or per-worktree config when extensions.worktreeConfig is on,
defines a filter or diff driver command is refused, and the callers
already treat that like a directory that is not a repository. The status
and diff calls also pass --ignore-submodules=dirty, because a submodule's
own config is not checked.

* test(gitsafe): keep the real-git setup off an inherited GIT_DIR

The setup helper in TestCommandAgainstRealGit ran git with the inherited
environment, so from a hook, or a shell that exports GIT_DIR or
GIT_WORK_TREE, git init and git config would act on the repository those
name instead of the temporary one. It now uses the same environment as
Command, and the test sets both variables to a decoy repository and checks
after each setup step that the decoy was not touched.
2026-10-15 15:33:32 +08:00
Kevin Codex 8e363fcaa5 fix(npm): wait for npm to apply each publish, and drop the one-time @gitlawb/zero steps (#1145)
* fix(npm): wait for npm to apply each publish, and drop the one-time @gitlawb/zero steps

npm applies a publish asynchronously: `npm publish` returns once the upload
is accepted, and the version appears later. In the 0.10.0 release the
@twigpine/zero wrapper appeared 9 seconds after publish-packages.sh stopped
checking for it, and the last platform payload about ten minutes after its
upload. Two things went wrong:

- The wrapper became visible before four of its five platform payloads, so
  installs in that window resolved aliases that 404 and fell back to the
  GitHub-release download. Ordering the uploads does not order what npm
  shows.
- The final `latest` check gave up after 25 seconds and failed the job even
  though every publish landed, which also skipped the @gitlawb/zero steps
  until a re-run.

publish-packages.sh now waits until npm shows every platform version before
it publishes the wrapper, then waits until `latest` is the wrapper version.
One deadline bounds the waits (ZERO_NPM_PUBLISH_WAIT_TIMEOUT, default 30
minutes). On timeout it fails before the wrapper goes out and says what npm
never showed. These lookups treat an error as "not yet", unlike the skip
decision, which still stops on anything but E404.

The test fake can now apply a publish only after a number of later lookups,
and records which aliased versions were missing when the wrapper was
published. Against the script as released in v0.10.0, the two new tests fail
the way the release did: the job exits 1 although everything landed, and the
wrapper goes out while its platform payload is not visible.

The two release steps that published 0.10.0 once more as @gitlawb/zero have
done their job and are removed; docs/NPM_PACKAGING.md and the comments that
pointed at them now describe what happened. The Go and wrapper handling of
the old name stays, since only a build installed under it can trigger it.

* fix(npm): validate the publish wait settings, and correct the legacy publishing claim

ZERO_NPM_PUBLISH_WAIT_TIMEOUT went into Bash arithmetic as typed. A leading
zero reads as octal (0600 waited 384 seconds), and 08 is an arithmetic error
that, in a script, does not stop it: Bash prints the error, leaves the
deadline unset and carries on, so the run only died at its first wait, after
the platform payloads were published. A negative timeout ended the wait
before it started, and a bad ZERO_NPM_PUBLISH_RETRY_DELAY only failed at the
first sleep, also after publishing began.

Both settings are now checked before anything is published: whole seconds
only, converted with 10# so a leading zero is decimal, bounded before the
conversion, and capped (timeout 86400, delay 600). A bad value fails with
the variable's name.

TestPublishPackagesReadsTheWaitTimeoutAsDecimal makes the script wait, which
is where the old code broke: against the reviewed script it publishes the
platform payload and then exits 1 on "08: value too great for base".
TestPublishPackagesRejectsAnInvalidWaitSetting covers -5, abc, 1e3, an
over-range and an overflowing timeout, and a non-numeric delay; the old
script published with five of them and crashed on the sixth with
"abc: unbound variable".

docs/NPM_PACKAGING.md said nothing could publish @gitlawb/zero again.
Removing the trusted publisher stops this workflow; a maintainer could
still publish by hand, and the doc now says that.
2026-10-15 13:30:11 +08:00
36 changed files with 2593 additions and 205 deletions

No files matched your search

+4 -49
View File
@@ -206,55 +206,10 @@ jobs:
# Platform payloads first under the `platform` dist-tag, then the wrapper,
# with the `latest` dist-tag checked before and after (platform versions
# are semver PRERELEASES of the wrapper version and must never become
# `latest`). Versions already on the registry are skipped, so "Re-run
# failed jobs" is safe after a partial publish.
# `latest`). npm applies publishes asynchronously, so the script waits
# until npm shows every platform version before the wrapper goes out,
# and until `latest` moves after it. Versions already on the registry
# are skipped, so "Re-run failed jobs" is safe after a partial publish.
- name: Publish npm packages
shell: bash
run: scripts/npm/publish-packages.sh dist/npm
# One final release under the pre-rename name @gitlawb/zero. Zero 0.9.0's
# `zero upgrade` runs `npm install -g @gitlawb/zero@latest`, and people
# reinstall the old name from shell history and old tutorials, so this
# release is published once more under it: the same verified artifacts,
# assembled as @gitlawb/zero, which the binary and wrapper recognise and
# answer with the switch steps (docs/NPM_PACKAGING.md). Runs after the
# @twigpine/zero publish, so the package it points at already exists.
#
# The gate keeps this to a single release: it is owed only while the old
# name's latest is still the last pre-rename version, or is this version
# (a re-run after it went out). After this release, `npm deprecate` the
# old package and delete these two steps.
- name: Check whether the final @gitlawb/zero release is owed
id: legacy-name
shell: bash
env:
LAST_PRE_RENAME_VERSION: 0.9.0
run: |
version="$(node -p "require('./package.json').version")"
latest=""
for _ in 1 2 3; do
latest="$(npm view @gitlawb/zero dist-tags.latest 2>/dev/null || true)"
if [ -n "$latest" ]; then
break
fi
sleep 10
done
if [ -z "$latest" ]; then
echo "::error::could not read the latest dist-tag of @gitlawb/zero from the registry" >&2
exit 1
fi
if [ "$latest" = "$LAST_PRE_RENAME_VERSION" ] || [ "$latest" = "$version" ]; then
owed=true
else
owed=false
fi
echo "@gitlawb/zero latest is ${latest}; final release under the old name owed: ${owed}"
echo "owed=${owed}" >> "$GITHUB_OUTPUT"
- name: Publish the final @gitlawb/zero release
if: ${{ steps.legacy-name.outputs.owed == 'true' }}
shell: bash
run: |
node scripts/npm/build-platform-packages.mjs \
--artifacts-dir dist/artifacts --out-dir dist/npm-legacy --package-name @gitlawb/zero
scripts/npm/publish-packages.sh dist/npm-legacy
+1 -1
View File
@@ -1 +1 @@
{".":"0.10.0"}
{".":"0.11.0"}
+20
View File
@@ -6,6 +6,26 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html) once the first release is
tagged. Until then, source builds report the version `dev`.
## [0.11.0](https://github.com/Twigpine/zero/compare/v0.10.0...v0.11.0) (2026-10-15)
### Features
* **providers:** add Concentrate provider preset ([#1147](https://github.com/Twigpine/zero/issues/1147)) ([61b0936](https://github.com/Twigpine/zero/commit/61b09364f0f45b0fa8aebaf0a6c3d1c55dbe01c3))
### Bug Fixes
* **cli:** let providers subcommands run with no active provider ([#1143](https://github.com/Twigpine/zero/issues/1143)) ([d1a0d04](https://github.com/Twigpine/zero/commit/d1a0d048ee2c1e7cbfb75160ec4c9f099f3c4472))
* **npm:** wait for npm to apply each publish, and drop the one-time @gitlawb/zero steps ([#1145](https://github.com/Twigpine/zero/issues/1145)) ([8e363fc](https://github.com/Twigpine/zero/commit/8e363fcaa5d51bf5b5b244692dc61bf8d4ae8799))
* **tui,cli:** stop a repository's git config from running programs in Zero's own git calls ([#1133](https://github.com/Twigpine/zero/issues/1133)) ([640d936](https://github.com/Twigpine/zero/commit/640d936bfbe00c4cd3017ca35fb7b5c15d91a1c5))
* **workspaceindex:** skip git-ignored and build-cache dirs in Scan only ([#1117](https://github.com/Twigpine/zero/issues/1117)) ([c88c77b](https://github.com/Twigpine/zero/commit/c88c77bf701039dca01d9786bf5648d779c949b3))
### Performance Improvements
* **redaction:** add fast-path byte checks to eliminate regex allocations on clean strings ([#945](https://github.com/Twigpine/zero/issues/945)) ([ce2fcff](https://github.com/Twigpine/zero/commit/ce2fcffaf10d8e8c8c5a13e06fd7809f09fc9867))
## [0.10.0](https://github.com/Twigpine/zero/compare/v0.9.0...v0.10.0) (2026-10-15)
+7
View File
@@ -216,6 +216,7 @@ export AIMLAPI_API_KEY=...
export LONGCAT_API_KEY=...
export FIREWORKS_API_KEY=...
export GMI_API_KEY=...
export CONCENTRATE_API_KEY=...
export MINIMAX_API_KEY=...
export MINIMAXI_API_KEY=...
```
@@ -244,6 +245,12 @@ To configure GMI Cloud directly, run:
zero providers setup gmicloud --set-active
```
To configure Concentrate directly, run:
```bash
zero providers setup concentrate --set-active
```
MiniMax presets use the Anthropic-compatible endpoints for the global and China
regions:
+7
View File
@@ -186,6 +186,7 @@ export ANTHROPIC_API_KEY=...
export GEMINI_API_KEY=...
export LONGCAT_API_KEY=...
export GMI_API_KEY=...
export CONCENTRATE_API_KEY=...
```
要直接配置美团 LongCat(LongCat-2.0),运行:
@@ -200,6 +201,12 @@ zero providers setup longcat --set-active
zero providers setup gmicloud --set-active
```
要直接配置 Concentrate,运行:
```bash
zero providers setup concentrate --set-active
```
对于本地模型,运行 Ollama、LM Studio 或 [Atomic Chat](https://atomic.chat) 桌面应用,
然后使用 `zero setup` 或 `zero providers detect`。使用 Atomic Chat 时,请先加载模型并启用
本地 OpenAI 兼容 API(默认地址为 `http://127.0.0.1:1337/v1`),再选择 `atomic-chat-local`。
+26 -44
View File
@@ -130,9 +130,14 @@ registry.
bootstrap publish) — a missing value must not abort the job between the
platform and wrapper publishes, which would strand a half-published
release.
2. **Platform versions publish before the wrapper.** The wrapper's
`optionalDependencies` pin exact suffixed versions; publishing the wrapper
first would create a window where installs resolve aliases that 404.
2. **Platform versions are on the registry before the wrapper.** The
wrapper's `optionalDependencies` pin exact suffixed versions; publishing
the wrapper first would create a window where installs resolve aliases
that 404. Ordering the uploads is not enough: npm applies a publish
asynchronously, and in 0.10.0 a platform version appeared ten minutes
after its upload returned, after the wrapper. The publish script waits
until npm shows every platform version before it publishes the wrapper,
and waits the same way for `latest` afterwards (up to 30 minutes in all).
3. **Exact-version pinning.** The wrapper at `X.Y.Z` references platform
versions `X.Y.Z-<platform>-<arch>` exactly — never ranges — so a wrapper
and its binaries can never skew. The publish script refuses to start when
@@ -161,54 +166,31 @@ bin, so npm refuses to install one over the other (`EEXIST`), and forcing it
lets a later uninstall of the old package delete the shared bin. The user
has to uninstall the old package and install the new one, in that order.
Deprecating the old package alone does not reach existing users. 0.9.0's
`zero upgrade` runs `npm install -g @gitlawb/zero@latest`; with nothing newer
under that name, npm reinstalls nothing, prints no deprecation warning (it
only warns when it installs something), and Zero reports the update as done.
So 0.10.0 is published once more under the old name, as a bridge:
Deprecating the old package alone would not have reached existing users.
0.9.0's `zero upgrade` runs `npm install -g @gitlawb/zero@latest`; with
nothing newer under that name, npm reinstalls nothing, prints no deprecation
warning (it only warns when it installs something), and Zero reports the
update as done. So 0.10.0 was published once more under the old name, as a
bridge:
- The publish job assembles the same verified artifacts a second time with
`build-platform-packages.mjs --package-name @gitlawb/zero` and publishes
them with the same script, after the `@twigpine/zero` publish. A gate runs
it only while the old name's `latest` is still 0.9.0 (or is this version,
on a re-run), so it cannot happen twice.
- The release job assembled the same verified artifacts a second time with
`build-platform-packages.mjs --package-name @gitlawb/zero` and published
them with the same script, after the `@twigpine/zero` publish. The steps
that did this ran once and have been removed from `release-artifacts.yml`.
- The bridge build knows its name from the `package.json` beside it. The Go
binary reports it as `InstallMethodNpmLegacy`: `zero upgrade` refuses
with the switch steps instead of reinstalling it, and `zero update --check`
adds them. `bin/zero.js` prints the same steps after `zero` exits, on a
terminal only.
- Updating from 0.9.0 now installs a real version, so npm also shows the
terminal only. This handling stays in the code, since only a build
installed under the old name ever triggers it.
- Updating from 0.9.0 installs that bridge build, so npm also shows the
deprecation message.
### Runbook
Before the first release under the new name:
1. Create the package, since npm only attaches a trusted publisher to a
package that exists: publish an empty `@twigpine/[email protected]` by hand with
`npm publish --access public --tag bootstrap`, then
`npm deprecate @twigpine/[email protected] "Placeholder; install @twigpine/zero@latest"`.
2. Trust this workflow for it:
`npm trust github @twigpine/zero --repo Twigpine/zero --file release-artifacts.yml --env npm-publish --allow-publish`.
3. Point the old package's trusted publisher at this repository too, for the
bridge publish: `npm trust list @gitlawb/zero`, revoke the existing entry
with `npm trust revoke @gitlawb/zero --id <id>`, then run the same
`npm trust github` command for `@gitlawb/zero`.
4. Confirm Settings → Environments → `npm-publish` has a required reviewer.
The environment exists either way; without a reviewer the job does not
wait for approval.
After the release, once both packages show 0.10.0 as `latest`:
1. `npm deprecate @gitlawb/zero "Zero moved to @twigpine/zero. Switch with: npm uninstall -g @gitlawb/zero && npm install -g @twigpine/zero"`
(no version range, so every version, including the platform payloads,
carries it). Never unpublish: existing installs resolve their platform
payload through these versions.
2. Revoke the old package's trusted publisher (`npm trust revoke`), so
nothing can publish under that name again.
3. Delete the two `@gitlawb/zero` steps from `release-artifacts.yml`. The Go
and wrapper handling of the old name can stay; it only activates in a
build installed under that name.
`@gitlawb/zero` is deprecated on every version and no longer has a trusted
publisher, so this release workflow cannot publish it again; another version
would take a maintainer publishing by hand. Its versions stay on the
registry: existing installs resolve their platform payload through them, so
they must never be unpublished.
## Runbook: bumping the vendored helpers
+35 -1
View File
@@ -1,6 +1,7 @@
package cli
import (
"errors"
"fmt"
"io"
"sort"
@@ -144,7 +145,7 @@ func runProviders(args []string, stdout io.Writer, stderr io.Writer, deps appDep
return exitSuccess
}
resolved, exitCode := resolveCommandCenterConfig(stderr, deps)
resolved, exitCode := resolveProviderManagementConfig(stderr, deps)
if exitCode != exitSuccess {
return exitCode
}
@@ -217,12 +218,45 @@ func runModels(args []string, stdout io.Writer, stderr io.Writer) int {
}
func resolveCommandCenterConfig(stderr io.Writer, deps appDeps) (config.ResolvedConfig, int) {
return resolveCommandCenterConfigFor(stderr, deps, false)
}
// resolveProviderManagementConfig resolves for the `providers` subcommands that
// only need the provider LIST (`list`, `current`, `use`, `remove`, `rename`).
// Those keep working when nothing is selected (issue #1130). It is deliberately
// NOT shared with the commands that REPORT the configuration — `zero config`
// above all: per the resolver's headless contract, `zero config` / `zero exec`
// still fail with the actionable resolve error, and only the interactive TUI
// recovers from a missing active provider.
func resolveProviderManagementConfig(stderr io.Writer, deps appDeps) (config.ResolvedConfig, int) {
return resolveCommandCenterConfigFor(stderr, deps, true)
}
func resolveCommandCenterConfigFor(stderr io.Writer, deps appDeps, allowMissingActiveProvider bool) (config.ResolvedConfig, int) {
workspaceRoot, err := resolveWorkspaceRoot("", deps)
if err != nil {
return config.ResolvedConfig{}, writeExecUsageError(stderr, err.Error())
}
resolved, err := deps.resolveConfig(workspaceRoot, config.Overrides{})
if err != nil {
// A missing active provider must not stop the `providers` subcommands
// that answer for profiles coming from the environment (OPENAI_API_KEY,
// GEMINI_API_KEY, ...), where an env-derived profile is never marked
// active in config.json. The resolver elects an active provider by
// itself only when there is exactly one, so two env providers (or one
// key plus one base URL) made every subcommand exit with `no active
// provider configured: active provider "" not found` (issue #1130). It
// still returns the successfully normalized list alongside the error,
// so hand that partial config back WITHOUT electing an active provider:
// these commands report what is configured, not what a run would use.
// Adopting firstUsableProvider here made `providers current`/`list`
// present a provider that `zero exec` would still refuse to run with,
// and made a base-URL-only profile (no API key) fail `providers list`
// because the election demands a credential. Electing a provider to RUN
// with stays in the interactive TUI's recovery path (app.go).
if allowMissingActiveProvider && errors.Is(err, config.ErrNoActiveProvider) {
return resolved, exitSuccess
}
return config.ResolvedConfig{}, writeAppError(stderr, err.Error(), exitProvider)
}
return resolved, exitSuccess
+34
View File
@@ -880,6 +880,40 @@ func providerSetupDeps(configPath string) appDeps {
}
}
// providerEnvSetupDeps is providerSetupDeps plus a REAL resolver pinned to a
// temporary user config and a fixed environment. providerSetupDeps only
// redirects WRITES, so without this the `providers` subcommands resolve against
// the developer's own config.json and ambient provider variables: a test that
// exports OPENAI_API_KEY passes on a machine whose real config has an active
// provider and fails on one that also has GEMINI_API_KEY exported (issue
// #1130). Passing Env makes resolution fully hermetic — config.envValue prefers
// the map over os.Getenv whenever it is non-nil.
func providerEnvSetupDeps(t *testing.T, configPath string, env map[string]string) appDeps {
t.Helper()
cwd := t.TempDir()
deps := providerSetupDeps(configPath)
deps.getwd = func() (string, error) { return cwd, nil }
deps.resolveConfig = func(workspaceRoot string, overrides config.Overrides) (config.ResolvedConfig, error) {
return config.Resolve(config.ResolveOptions{
UserConfigPath: configPath,
Env: env,
Overrides: overrides,
})
}
// summarizeConfig reads OAuth LOGIN state through the default store, which
// resolves the developer's own ZERO_OAUTH_TOKENS_PATH (or XDG keyring root)
// — a machine with a stored ChatGPT login would make `providers list`
// assertions flip. Pin the file and the backend per the coding guideline
// that no test may read the developer's real config, cache or state dirs:
// forcing "file" also prevents an ambient ZERO_OAUTH_STORAGE=keyring from
// pulling the OS keychain into these tests.
tokensPath := filepath.Join(t.TempDir(), "oauth-tokens.json")
t.Setenv("ZERO_OAUTH_TOKENS_PATH", tokensPath)
t.Setenv("ZERO_OAUTH_STORAGE", "file")
return deps
}
func readFileConfig(t *testing.T, path string) config.FileConfig {
t.Helper()
@@ -21,6 +21,7 @@ func TestProviderCatalogRuntimeProvidersShowOnboardingSetupHints(t *testing.T) {
{id: "longcat", name: "LongCat", transport: "openai-compatible", defaultModel: "LongCat-2.0"},
{id: "fireworks", name: "Fireworks AI", transport: "openai-compatible", defaultModel: "accounts/fireworks/models/kimi-k2p7-code"},
{id: "gmicloud", name: "GMI Cloud", transport: "openai-compatible", defaultModel: "Qwen/Qwen3.8-Max"},
{id: "concentrate", name: "Concentrate", transport: "openai-compatible", defaultModel: "deepseek-v4-flash-0731"},
{id: "ollama-cloud", name: "Ollama Cloud", transport: "openai-compatible", defaultModel: "qwen3-coder:480b"},
{id: "ollama", name: "Ollama Local", transport: "openai-compatible", defaultModel: "llama3.1"},
}
+4 -4
View File
@@ -542,9 +542,9 @@ func providerResolvedByName(providers []config.ProviderProfile, name string) boo
// reports the situation plainly instead of that confusing error (issue
// #707).
func reportUnpersistedProviderUse(stdout, stderr io.Writer, deps appDeps, options providerUseOptions, configPath string) (int, bool) {
resolved, exitCode := resolveCommandCenterConfig(stderr, deps)
resolved, exitCode := resolveProviderManagementConfig(stderr, deps)
if exitCode != exitSuccess {
// resolveCommandCenterConfig already wrote its own error to stderr;
// resolveProviderManagementConfig already wrote its own error to stderr;
// stop here instead of letting the caller try SetActiveProvider too.
return exitCode, true
}
@@ -578,7 +578,7 @@ func reportUnpersistedProviderUse(stdout, stderr io.Writer, deps appDeps, option
// name isn't resolvable at all, it returns handled=false so the caller falls
// through to RemoveProvider's real "not found" error.
func reportUnpersistedProviderRemove(stdout, stderr io.Writer, deps appDeps, name string, jsonOutput bool, configPath string) (int, bool) {
resolved, exitCode := resolveCommandCenterConfig(stderr, deps)
resolved, exitCode := resolveProviderManagementConfig(stderr, deps)
if exitCode != exitSuccess {
return exitCode, true
}
@@ -609,7 +609,7 @@ func reportUnpersistedProviderRemove(stdout, stderr io.Writer, deps appDeps, nam
}
func reportUnpersistedProviderRename(stdout, stderr io.Writer, deps appDeps, name string, jsonOutput bool, configPath string) (int, bool) {
resolved, exitCode := resolveCommandCenterConfig(stderr, deps)
resolved, exitCode := resolveProviderManagementConfig(stderr, deps)
if exitCode != exitSuccess {
return exitCode, true
}
+227 -8
View File
@@ -189,12 +189,11 @@ func TestRunProvidersUseSurfacesMalformedConfig(t *testing.T) {
}
func TestRunProvidersUseEnvDerivedJSONIncludesConfigPath(t *testing.T) {
t.Setenv("OPENAI_API_KEY", "sk-env")
var stdout, stderr bytes.Buffer
configPath := filepath.Join(t.TempDir(), "config.json")
writeProviderOnboardingConfig(t, configPath, config.FileConfig{})
exitCode := runWithDeps([]string{"providers", "use", "openai", "--json"}, &stdout, &stderr, providerSetupDeps(configPath))
exitCode := runWithDeps([]string{"providers", "use", "openai", "--json"}, &stdout, &stderr, providerEnvSetupDeps(t, configPath, map[string]string{"OPENAI_API_KEY": "sk-env"}))
if exitCode != exitSuccess {
t.Fatalf("exit code = %d, want %d: %s", exitCode, exitSuccess, stderr.String())
@@ -212,12 +211,11 @@ func TestRunProvidersUseEnvDerivedJSONIncludesConfigPath(t *testing.T) {
}
func TestRunProvidersRemoveEnvDerivedJSONKeepsSchema(t *testing.T) {
t.Setenv("OPENAI_API_KEY", "sk-env")
var stdout, stderr bytes.Buffer
configPath := filepath.Join(t.TempDir(), "config.json")
writeProviderOnboardingConfig(t, configPath, config.FileConfig{})
exitCode := runWithDeps([]string{"providers", "remove", "openai", "--json"}, &stdout, &stderr, providerSetupDeps(configPath))
exitCode := runWithDeps([]string{"providers", "remove", "openai", "--json"}, &stdout, &stderr, providerEnvSetupDeps(t, configPath, map[string]string{"OPENAI_API_KEY": "sk-env"}))
if exitCode != exitSuccess {
t.Fatalf("exit code = %d, want %d: %s", exitCode, exitSuccess, stderr.String())
@@ -237,12 +235,11 @@ func TestRunProvidersRemoveEnvDerivedJSONKeepsSchema(t *testing.T) {
}
func TestRunProvidersRenameEnvDerivedExplainsNoSavedProfile(t *testing.T) {
t.Setenv("OPENAI_API_KEY", "sk-env")
var stdout, stderr bytes.Buffer
configPath := filepath.Join(t.TempDir(), "config.json")
writeProviderOnboardingConfig(t, configPath, config.FileConfig{})
exitCode := runWithDeps([]string{"providers", "rename", "openai", "renamed"}, &stdout, &stderr, providerSetupDeps(configPath))
exitCode := runWithDeps([]string{"providers", "rename", "openai", "renamed"}, &stdout, &stderr, providerEnvSetupDeps(t, configPath, map[string]string{"OPENAI_API_KEY": "sk-env"}))
if exitCode != exitSuccess {
t.Fatalf("exit code = %d, want %d: %s", exitCode, exitSuccess, stderr.String())
@@ -253,12 +250,11 @@ func TestRunProvidersRenameEnvDerivedExplainsNoSavedProfile(t *testing.T) {
}
func TestRunProvidersRenameEnvDerivedJSONKeepsSchema(t *testing.T) {
t.Setenv("OPENAI_API_KEY", "sk-env")
var stdout, stderr bytes.Buffer
configPath := filepath.Join(t.TempDir(), "config.json")
writeProviderOnboardingConfig(t, configPath, config.FileConfig{})
exitCode := runWithDeps([]string{"providers", "rename", "openai", "renamed", "--json"}, &stdout, &stderr, providerSetupDeps(configPath))
exitCode := runWithDeps([]string{"providers", "rename", "openai", "renamed", "--json"}, &stdout, &stderr, providerEnvSetupDeps(t, configPath, map[string]string{"OPENAI_API_KEY": "sk-env"}))
if exitCode != exitSuccess {
t.Fatalf("exit code = %d, want %d: %s", exitCode, exitSuccess, stderr.String())
@@ -276,6 +272,229 @@ func TestRunProvidersRenameEnvDerivedJSONKeepsSchema(t *testing.T) {
}
}
// Two env-derived providers and no saved active provider is the case issue
// #1130 reports: the resolver only elects an active provider by itself when
// there is exactly one, so `providers list/use/remove/rename` used to exit 3
// with `active provider "" not found` before doing anything.
func TestRunProvidersSubcommandsWithMultipleEnvDerivedProviders(t *testing.T) {
env := map[string]string{
"OPENAI_API_KEY": "sk-env",
"GEMINI_API_KEY": "gem-env",
}
cases := []struct {
name string
args []string
want string
}{
{name: "list", args: []string{"providers", "list"}, want: "openai"},
{name: "list json", args: []string{"providers", "list", "--json"}, want: `"name": "openai"`},
{name: "use", args: []string{"providers", "use", "openai"}, want: "is not saved in config.json"},
{name: "remove", args: []string{"providers", "remove", "openai"}, want: "nothing to remove there"},
{name: "rename", args: []string{"providers", "rename", "openai", "renamed"}, want: "no saved profile to rename"},
}
for _, tt := range cases {
t.Run(tt.name, func(t *testing.T) {
configPath := filepath.Join(t.TempDir(), "config.json")
writeProviderOnboardingConfig(t, configPath, config.FileConfig{})
var stdout, stderr bytes.Buffer
exitCode := runWithDeps(tt.args, &stdout, &stderr, providerEnvSetupDeps(t, configPath, env))
if exitCode != exitSuccess {
t.Fatalf("exit code = %d, want %d: %s", exitCode, exitSuccess, stderr.String())
}
if !strings.Contains(stdout.String(), tt.want) {
t.Fatalf("stdout = %q, want it to contain %q", stdout.String(), tt.want)
}
})
}
}
func TestRunProvidersListShowsEveryEnvDerivedProvider(t *testing.T) {
env := map[string]string{
"OPENAI_API_KEY": "sk-env",
"GEMINI_API_KEY": "gem-env",
}
configPath := filepath.Join(t.TempDir(), "config.json")
writeProviderOnboardingConfig(t, configPath, config.FileConfig{})
var stdout, stderr bytes.Buffer
exitCode := runWithDeps([]string{"providers", "list", "--json"}, &stdout, &stderr, providerEnvSetupDeps(t, configPath, env))
if exitCode != exitSuccess {
t.Fatalf("exit code = %d, want %d: %s", exitCode, exitSuccess, stderr.String())
}
var payload struct {
Providers []struct {
Name string `json:"name"`
} `json:"providers"`
}
if err := json.Unmarshal(stdout.Bytes(), &payload); err != nil {
t.Fatalf("decode JSON: %v\n%s", err, stdout.String())
}
names := make(map[string]bool, len(payload.Providers))
for _, provider := range payload.Providers {
names[provider.Name] = true
}
for _, want := range []string{"openai", "google"} {
if !names[want] {
t.Fatalf("providers = %v, want %q listed", names, want)
}
}
}
// The provider-management commands report what is CONFIGURED, not what a run
// would use: with env-derived providers and no saved active provider they must
// not present any provider as selected. Electing firstUsableProvider in
// resolveCommandCenterConfig made `providers current` claim `openai` while
// `zero exec` in the same environment still exited 3 with `no active provider
// configured` — and it read the OAuth store on this path (maintainer review of
// this PR).
func TestRunProvidersSubcommandsElectNoActiveProvider(t *testing.T) {
env := map[string]string{
"OPENAI_API_KEY": "sk-env",
"GEMINI_API_KEY": "gem-env",
}
deps := func(t *testing.T) appDeps {
configPath := filepath.Join(t.TempDir(), "config.json")
writeProviderOnboardingConfig(t, configPath, config.FileConfig{})
return providerEnvSetupDeps(t, configPath, env)
}
t.Run("current shows nothing selected", func(t *testing.T) {
var stdout, stderr bytes.Buffer
exitCode := runWithDeps([]string{"providers", "current"}, &stdout, &stderr, deps(t))
if exitCode != exitSuccess {
t.Fatalf("exit code = %d, want %d: %s", exitCode, exitSuccess, stderr.String())
}
if !strings.Contains(stdout.String(), "(none)") {
t.Fatalf("stdout = %q, want an empty selection", stdout.String())
}
if strings.Contains(stdout.String(), "openai") {
t.Fatalf("stdout = %q, want no provider elected as current", stdout.String())
}
})
t.Run("list marks nothing active", func(t *testing.T) {
var stdout, stderr bytes.Buffer
exitCode := runWithDeps([]string{"providers", "list", "--json"}, &stdout, &stderr, deps(t))
if exitCode != exitSuccess {
t.Fatalf("exit code = %d, want %d: %s", exitCode, exitSuccess, stderr.String())
}
var payload struct {
Providers []struct {
Name string `json:"name"`
Active bool `json:"active"`
} `json:"providers"`
}
if err := json.Unmarshal(stdout.Bytes(), &payload); err != nil {
t.Fatalf("decode JSON: %v\n%s", err, stdout.String())
}
if len(payload.Providers) < 2 {
t.Fatalf("providers = %+v, want both env-derived profiles listed", payload.Providers)
}
for _, provider := range payload.Providers {
if provider.Active {
t.Fatalf("provider %q marked active, want nothing active", provider.Name)
}
}
})
t.Run("use json leaves activeProvider empty", func(t *testing.T) {
var stdout, stderr bytes.Buffer
exitCode := runWithDeps([]string{"providers", "use", "openai", "--json"}, &stdout, &stderr, deps(t))
if exitCode != exitSuccess {
t.Fatalf("exit code = %d, want %d: %s", exitCode, exitSuccess, stderr.String())
}
var payload struct {
ActiveProvider string `json:"activeProvider"`
}
if err := json.Unmarshal(stdout.Bytes(), &payload); err != nil {
t.Fatalf("decode JSON: %v\n%s", err, stdout.String())
}
if payload.ActiveProvider != "" {
t.Fatalf("activeProvider = %q, want it empty for an unpersisted selection", payload.ActiveProvider)
}
})
}
// A base-URL-only environment (OPENAI_BASE_URL / GEMINI_BASE_URL set, no API
// keys) is the other half of issue #1130: electing firstUsableProvider made
// `providers list` exit 3 because the election demands a credential. The list
// must succeed and still mark nothing active. The resolver itself keeps only
// the keyless openai-compatible profile here (a google profile without a
// credential is dropped during normalization), so `openai` is the expected
// entry — pinning that as-is keeps this test about the COMMAND, not the
// resolver.
func TestRunProvidersListWithBaseURLOnlyProviders(t *testing.T) {
env := map[string]string{
"OPENAI_BASE_URL": "https://openai.proxy.example.com/v1",
"GEMINI_BASE_URL": "https://gemini.proxy.example.com",
}
configPath := filepath.Join(t.TempDir(), "config.json")
writeProviderOnboardingConfig(t, configPath, config.FileConfig{})
var stdout, stderr bytes.Buffer
exitCode := runWithDeps([]string{"providers", "list", "--json"}, &stdout, &stderr, providerEnvSetupDeps(t, configPath, env))
if exitCode != exitSuccess {
t.Fatalf("exit code = %d, want %d: %s", exitCode, exitSuccess, stderr.String())
}
var payload struct {
Providers []struct {
Name string `json:"name"`
Active bool `json:"active"`
} `json:"providers"`
}
if err := json.Unmarshal(stdout.Bytes(), &payload); err != nil {
t.Fatalf("decode JSON: %v\n%s", err, stdout.String())
}
names := make(map[string]bool, len(payload.Providers))
for _, provider := range payload.Providers {
if provider.Active {
t.Fatalf("provider %q marked active, want nothing active", provider.Name)
}
names[provider.Name] = true
}
if !names["openai"] {
t.Fatalf("providers = %v, want the base-URL-only openai profile listed", names)
}
}
// `zero config` is NOT one of the provider-management commands: it reports the
// resolved configuration, so a missing active provider has to stay actionable
// (the headless contract the resolver documents for `zero config` / `zero
// exec`). Only the interactive TUI recovers from that state.
func TestRunConfigSummaryStillFailsWithNoActiveProvider(t *testing.T) {
env := map[string]string{
"OPENAI_API_KEY": "sk-env",
"GEMINI_API_KEY": "gem-env",
}
configPath := filepath.Join(t.TempDir(), "config.json")
writeProviderOnboardingConfig(t, configPath, config.FileConfig{})
var stdout, stderr bytes.Buffer
exitCode := runWithDeps([]string{"config"}, &stdout, &stderr, providerEnvSetupDeps(t, configPath, env))
if exitCode == exitSuccess {
t.Fatalf("exit code = %d, want the actionable resolve error: %s", exitCode, stdout.String())
}
message := stderr.String()
if message == "" {
message = stdout.String()
}
if !strings.Contains(message, "no active provider configured") {
t.Fatalf("output = %q / %q, want the actionable resolve error", stdout.String(), stderr.String())
}
}
func TestRunProvidersUseRejectsUsageErrors(t *testing.T) {
cases := []struct {
name string
+9 -1
View File
@@ -7,6 +7,8 @@ import (
"strconv"
"strings"
"time"
"github.com/Gitlawb/zero/internal/gitsafe"
)
type scratchFileBaseline map[string]bool
@@ -78,7 +80,13 @@ func gitUntrackedFiles(workspaceRoot string) ([]string, bool) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
output, err := exec.CommandContext(ctx, "git", "-C", workspaceRoot, "status", "--porcelain", "-z", "--untracked-files=all").Output()
// gitsafe keeps the workspace's own git config from running a program, and
// --ignore-submodules=dirty keeps git out of submodule work trees.
command, err := gitsafe.Command(ctx, workspaceRoot, "status", "--porcelain", "-z", "--untracked-files=all", "--ignore-submodules=dirty")
if err != nil {
return nil, false
}
output, err := command.Output()
if err != nil {
// Not a git repo (or git failed for some other reason) — nothing
// reliable to report against, so stay silent rather than guess.
+218
View File
@@ -0,0 +1,218 @@
package gitsafe
import (
"go/ast"
"go/parser"
"go/token"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
)
// The packages whose git calls Zero makes on its own. The CLI runs git for the
// scratch-file warning and the TUI for its files sweep and pull request
// diffstat; each call has to come through Command.
var automaticGitPackages = []string{
filepath.Join("..", "cli"),
filepath.Join("..", "tui"),
}
// parsePackage parses the non-test Go files in dir.
func parsePackage(t *testing.T, dir string) (*token.FileSet, []*ast.File) {
t.Helper()
entries, err := os.ReadDir(dir)
if err != nil {
t.Fatalf("read %s: %v", dir, err)
}
fset := token.NewFileSet()
var files []*ast.File
for _, entry := range entries {
name := entry.Name()
if entry.IsDir() || !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") {
continue
}
file, err := parser.ParseFile(fset, filepath.Join(dir, name), nil, 0)
if err != nil {
t.Fatalf("parse %s: %v", name, err)
}
files = append(files, file)
}
if len(files) == 0 {
t.Fatalf("no Go files in %s", dir)
}
return fset, files
}
// importName is the name file uses for the package at path, or "" when the
// file does not import it.
func importName(file *ast.File, path string) string {
for _, spec := range file.Imports {
if value, err := strconv.Unquote(spec.Path.Value); err != nil || value != path {
continue
}
if spec.Name != nil {
return spec.Name.Name
}
return filepath.Base(path)
}
return ""
}
// directGitExecs lists the exec.Command and exec.CommandContext calls in files
// whose program is the literal "git".
func directGitExecs(fset *token.FileSet, files []*ast.File) []string {
var found []string
for _, file := range files {
execName := importName(file, "os/exec")
if execName == "" {
continue
}
ast.Inspect(file, func(node ast.Node) bool {
call, ok := node.(*ast.CallExpr)
if !ok {
return true
}
selector, ok := call.Fun.(*ast.SelectorExpr)
if !ok {
return true
}
if pkg, ok := selector.X.(*ast.Ident); !ok || pkg.Name != execName {
return true
}
program := 0
switch selector.Sel.Name {
case "Command":
case "CommandContext":
program = 1
default:
return true
}
if len(call.Args) <= program {
return true
}
if literal, ok := call.Args[program].(*ast.BasicLit); ok && literal.Kind == token.STRING {
if value, err := strconv.Unquote(literal.Value); err == nil && value == "git" {
found = append(found, fset.Position(call.Pos()).String())
}
}
return true
})
}
return found
}
func TestAutomaticGitCallsDoNotExecGitDirectly(t *testing.T) {
for _, dir := range automaticGitPackages {
fset, files := parsePackage(t, dir)
for _, position := range directGitExecs(fset, files) {
t.Errorf("%s runs git directly; go through gitsafe.Command so the repository's config cannot make it run a program", position)
}
}
}
// gitArguments returns the git arguments of call when it is gitsafe.Command or a
// runner called with the program "git" (the TUI's runner(ctx, cwd, "git", ...)),
// as far as they are string literals.
func gitArguments(call *ast.CallExpr, gitsafeName string) ([]string, bool) {
start := -1
if selector, ok := call.Fun.(*ast.SelectorExpr); ok && gitsafeName != "" && selector.Sel.Name == "Command" {
if pkg, ok := selector.X.(*ast.Ident); ok && pkg.Name == gitsafeName {
start = 2
}
}
if start < 0 {
for index, argument := range call.Args {
if literal, ok := argument.(*ast.BasicLit); ok && literal.Kind == token.STRING && literal.Value == `"git"` {
start = index + 1
break
}
}
}
if start < 0 || start > len(call.Args) {
return nil, false
}
var arguments []string
for _, argument := range call.Args[start:] {
if literal, ok := argument.(*ast.BasicLit); ok && literal.Kind == token.STRING {
if value, err := strconv.Unquote(literal.Value); err == nil {
arguments = append(arguments, value)
continue
}
}
arguments = append(arguments, "")
}
return arguments, true
}
// A submodule has its own config, which Command does not check, so the
// automatic status and diff calls stay out of submodule work trees.
func TestAutomaticStatusAndDiffSkipSubmoduleWorkTrees(t *testing.T) {
checked := 0
for _, dir := range automaticGitPackages {
fset, files := parsePackage(t, dir)
for _, file := range files {
gitsafeName := importName(file, "github.com/Gitlawb/zero/internal/gitsafe")
ast.Inspect(file, func(node ast.Node) bool {
call, ok := node.(*ast.CallExpr)
if !ok {
return true
}
arguments, ok := gitArguments(call, gitsafeName)
if !ok || len(arguments) == 0 || (arguments[0] != "status" && arguments[0] != "diff") {
return true
}
checked++
for _, argument := range arguments {
if argument == "--ignore-submodules=dirty" {
return true
}
}
t.Errorf("%s runs git %s without --ignore-submodules=dirty", fset.Position(call.Pos()), arguments[0])
return true
})
}
}
// The scratch-file status, the files sweep's status and diff, and the pull
// request diffstat.
if checked != 4 {
t.Fatalf("checked %d automatic status or diff calls, want 4; update this test when they change", checked)
}
}
// The TUI runs every git command through one runner, whose program name is a
// parameter, so the check above cannot see it. The runner has to hand git to
// Command itself.
func TestTUICommandRunnerSendsGitThroughCommand(t *testing.T) {
_, files := parsePackage(t, filepath.Join("..", "tui"))
var runner *ast.FuncDecl
var runnerFile *ast.File
for _, file := range files {
for _, declaration := range file.Decls {
if function, ok := declaration.(*ast.FuncDecl); ok && function.Recv == nil && function.Name.Name == "defaultPRCommandRunner" {
runner, runnerFile = function, file
}
}
}
if runner == nil {
t.Fatal("defaultPRCommandRunner not found in internal/tui")
}
gitsafeName := importName(runnerFile, "github.com/Gitlawb/zero/internal/gitsafe")
calls := false
if gitsafeName != "" {
ast.Inspect(runner.Body, func(node ast.Node) bool {
if call, ok := node.(*ast.CallExpr); ok {
if selector, ok := call.Fun.(*ast.SelectorExpr); ok && selector.Sel.Name == "Command" {
if pkg, ok := selector.X.(*ast.Ident); ok && pkg.Name == gitsafeName {
calls = true
}
}
}
return !calls
})
}
if !calls {
t.Fatal("defaultPRCommandRunner does not call gitsafe.Command, so the TUI's git calls skip it")
}
}
+192
View File
@@ -0,0 +1,192 @@
// Package gitsafe builds the read-only git commands Zero runs on its own, as
// opposed to a git command the user or the model asked for: the status and diff
// calls behind the TUI's changed-files sweep, its pull request diffstat and the
// scratch-file warning, which run without anyone asking for git.
//
// A REPOSITORY CAN MAKE THOSE COMMANDS RUN A PROGRAM. git status and git diff read
// the repository's own .git/config, and several keys there name programs git then
// runs: core.fsmonitor on every status, a clean filter or diff driver assigned
// through .gitattributes whenever a file's content has to be read, and hooks such
// as post-index-change when the index is rewritten. A repository unpacked from an
// archive or handed over as a directory carries that config, and Zero would run it
// on the first turn, although it refuses the same workspace's project hooks and
// plugins until the user trusts it.
//
// The single-key cases are switched off on the command line. Filter and diff
// driver commands are named per driver, so they cannot be switched off without
// knowing the names; a repository whose own config defines any is skipped instead.
// Commands defined in the user's own global config, such as git-lfs, are the
// user's choice and are not affected.
package gitsafe
import (
"bytes"
"context"
"errors"
"os"
"os/exec"
"runtime"
"strings"
)
// ErrRepositoryDefinesCommands is returned for a repository whose own config
// defines a filter or diff driver command. The caller goes without the git
// information rather than run it.
var ErrRepositoryDefinesCommands = errors.New("the repository's own git config defines a filter or diff command, so Zero does not run git there on its own")
// commandKeys matches the config keys whose value is a program git can run while
// computing status or a diff, and which are named per driver. Git prints section
// and variable names in lower case, so the patterns do not need to fold case.
const commandKeys = `filter\..+\.(clean|smudge|process)|diff\..+\.(command|textconv)|diff\.external`
// worktreeConfigKey makes git also read per-worktree config (config.worktree),
// which a --local read leaves out. Without it git ignores that file.
const worktreeConfigKey = "extensions.worktreeconfig"
const (
localConfigPattern = `^(` + commandKeys + `|extensions\.worktreeconfig)$`
worktreeConfigPattern = `^(` + commandKeys + `)$`
)
// Command returns git with args, run in dir, with the programs a repository can
// configure turned off. It fails with ErrRepositoryDefinesCommands when the
// repository's own config defines a filter or diff driver command, and with git's
// own error when dir is not inside a repository.
//
// Callers that inspect a work tree that may contain submodules should also pass
// --ignore-submodules=dirty: a submodule has its own config, which this does not
// read, and git only runs commands there when it looks inside the submodule.
func Command(ctx context.Context, dir string, args ...string) (*exec.Cmd, error) {
env := environment()
defines, err := repositoryDefinesCommands(ctx, dir, env)
if err != nil {
return nil, err
}
if defines {
return nil, ErrRepositoryDefinesCommands
}
command := exec.CommandContext(ctx, "git", append(disabledPrograms(), args...)...)
command.Dir = dir
// No opportunistic index refresh: a read-only call never rewrites the index,
// so it never triggers the hooks that run when the index is written. Git
// before 2.15 does not know the variable and ignores it.
command.Env = append(env, "GIT_OPTIONAL_LOCKS=0")
return command, nil
}
// disabledPrograms turns off the single-key ways a repository runs a program
// during status or diff. Git ignores a key it does not know, so an older git
// without hooksPath (2.9) or fsmonitor (2.16) accepts these unchanged.
func disabledPrograms() []string {
return []string{"-c", "core.fsmonitor=false", "-c", "core.hooksPath=" + os.DevNull}
}
// repositoryLocationVariables make git use a given repository, work tree or
// index instead of discovering them from the working directory. Git exports
// them to hooks, so a Zero started from a hook inherits the hook's repository.
var repositoryLocationVariables = []string{
"GIT_DIR", "GIT_WORK_TREE", "GIT_INDEX_FILE", "GIT_COMMON_DIR",
"GIT_OBJECT_DIRECTORY", "GIT_ALTERNATE_OBJECT_DIRECTORIES", "GIT_NAMESPACE", "GIT_PREFIX",
}
// environment is Zero's environment without the repository-location variables,
// so git finds the repository from dir. The config check and the command share
// it: checking one repository's config and then running git in another would
// let the second one's commands through.
func environment() []string {
parent := os.Environ()
env := make([]string, 0, len(parent)+1)
for _, entry := range parent {
name, _, _ := strings.Cut(entry, "=")
if !isRepositoryLocationVariable(name) {
env = append(env, entry)
}
}
return env
}
func isRepositoryLocationVariable(name string) bool {
for _, variable := range repositoryLocationVariables {
// Windows environment names are case-insensitive.
if name == variable || (runtime.GOOS == "windows" && strings.EqualFold(name, variable)) {
return true
}
}
return false
}
// repositoryDefinesCommands reports whether the repository's own config, with
// includes followed, defines a filter or diff driver command. The local config is
// read once, together with the key that turns on per-worktree config, and that
// file is read only when it is on, so most repositories cost one git process.
// Reading config runs nothing.
func repositoryDefinesCommands(ctx context.Context, dir string, env []string) (bool, error) {
entries, err := readConfig(ctx, dir, env, "--local", localConfigPattern)
if err != nil {
return false, err
}
worktreeConfig := false
for _, entry := range entries {
if entry.key != worktreeConfigKey {
return true, nil
}
worktreeConfig = !configFalse(entry.value)
}
if !worktreeConfig {
return false, nil
}
// The extension needs git 2.20, which also has --worktree, so a failure here
// is real and the caller goes without git.
entries, err = readConfig(ctx, dir, env, "--worktree", worktreeConfigPattern)
if err != nil {
return false, err
}
return len(entries) > 0, nil
}
// configFalse reports whether a config value is one of git's spellings of false.
// Anything else, including a key with no value, counts as on, so an unexpected
// spelling costs one more read rather than skipping the file.
func configFalse(value string) bool {
switch strings.ToLower(strings.TrimSpace(value)) {
case "false", "no", "off", "0":
return true
}
return false
}
type configEntry struct {
key string
value string
}
// readConfig returns the entries in one config scope whose key matches pattern.
// It is a variable so tests can answer for git.
var readConfig = func(ctx context.Context, dir string, env []string, scope string, pattern string) ([]configEntry, error) {
command := exec.CommandContext(ctx, "git", "config", "-z", scope, "--includes", "--get-regexp", pattern)
command.Dir = dir
command.Env = env
output, err := command.Output()
if err != nil {
var exit *exec.ExitError
if errors.As(err, &exit) && exit.ExitCode() == 1 {
return nil, nil // no key matched
}
return nil, err
}
return parseConfigEntries(output), nil
}
// parseConfigEntries splits git config -z output: each entry ends with a NUL,
// and a newline separates the key from its value when there is one.
func parseConfigEntries(output []byte) []configEntry {
var entries []configEntry
for _, record := range bytes.Split(output, []byte{0}) {
if len(record) == 0 {
continue
}
key, value, _ := bytes.Cut(record, []byte{'\n'})
entries = append(entries, configEntry{key: string(key), value: string(value)})
}
return entries
}
+327
View File
@@ -0,0 +1,327 @@
package gitsafe
import (
"context"
"errors"
"fmt"
"os"
"os/exec"
"path/filepath"
"reflect"
"regexp"
"runtime"
"strings"
"testing"
)
type configCall struct {
scope string
env []string
}
// answerConfig stands in for git's config read and records each call.
func answerConfig(t *testing.T, answer func(scope string) ([]configEntry, error)) *[]configCall {
t.Helper()
var calls []configCall
previous := readConfig
readConfig = func(_ context.Context, _ string, env []string, scope string, _ string) ([]configEntry, error) {
calls = append(calls, configCall{scope: scope, env: env})
return answer(scope)
}
t.Cleanup(func() { readConfig = previous })
return &calls
}
func noEntries(string) ([]configEntry, error) { return nil, nil }
func scopes(calls *[]configCall) []string {
var read []string
for _, call := range *calls {
read = append(read, call.scope)
}
return read
}
// effectiveEnv returns the value a child process gets for name: os/exec keeps
// the last entry for a name, compared without case on Windows.
func effectiveEnv(env []string, name string) (string, bool) {
value, found := "", false
for _, entry := range env {
key, rest, ok := strings.Cut(entry, "=")
if ok && (key == name || (runtime.GOOS == "windows" && strings.EqualFold(key, name))) {
value, found = rest, true
}
}
return value, found
}
func TestCommandTurnsOffRepositoryPrograms(t *testing.T) {
answerConfig(t, noEntries)
t.Setenv("GIT_OPTIONAL_LOCKS", "1")
dir := t.TempDir()
command, err := Command(context.Background(), dir, "status", "--porcelain")
if err != nil {
t.Fatalf("Command: %v", err)
}
want := []string{"git", "-c", "core.fsmonitor=false", "-c", "core.hooksPath=" + os.DevNull, "status", "--porcelain"}
if !reflect.DeepEqual(command.Args, want) {
t.Fatalf("args = %q, want %q", command.Args, want)
}
if command.Dir != dir {
t.Fatalf("dir = %q, want %q", command.Dir, dir)
}
// An inherited GIT_OPTIONAL_LOCKS must not turn the index refresh back on.
if value, _ := effectiveEnv(command.Env, "GIT_OPTIONAL_LOCKS"); value != "0" {
t.Fatalf("effective GIT_OPTIONAL_LOCKS = %q, want 0", value)
}
}
func TestCommandDropsRepositoryLocationVariables(t *testing.T) {
// Turn per-worktree config on so both reads happen and both are checked.
calls := answerConfig(t, func(scope string) ([]configEntry, error) {
if scope == "--local" {
return []configEntry{{key: worktreeConfigKey, value: "true"}}, nil
}
return nil, nil
})
elsewhere := filepath.Join(t.TempDir(), "elsewhere")
for _, name := range repositoryLocationVariables {
t.Setenv(name, elsewhere)
}
t.Setenv("ZERO_GITSAFE_TEST_KEPT", "kept")
command, err := Command(context.Background(), t.TempDir(), "status")
if err != nil {
t.Fatalf("Command: %v", err)
}
for _, name := range repositoryLocationVariables {
if value, found := effectiveEnv(command.Env, name); found {
t.Fatalf("%s=%q reached git", name, value)
}
}
if value, _ := effectiveEnv(command.Env, "ZERO_GITSAFE_TEST_KEPT"); value != "kept" {
t.Fatalf("an unrelated variable was dropped: %q", value)
}
// The config check has to look at the repository the command runs in, so it
// gets the same environment, minus only the lock variable added afterwards.
if got := scopes(calls); !reflect.DeepEqual(got, []string{"--local", "--worktree"}) {
t.Fatalf("config reads = %q, want --local then --worktree", got)
}
for _, call := range *calls {
if !reflect.DeepEqual(call.env, command.Env[:len(command.Env)-1]) {
t.Fatalf("the %s config read ran with a different environment than the command", call.scope)
}
}
}
func TestLocationVariableNamesFoldCaseOnlyOnWindows(t *testing.T) {
if !isRepositoryLocationVariable("GIT_DIR") {
t.Fatal("GIT_DIR not recognised")
}
if got, want := isRepositoryLocationVariable("Git_Dir"), runtime.GOOS == "windows"; got != want {
t.Fatalf("Git_Dir recognised = %v, want %v on %s", got, want, runtime.GOOS)
}
for _, name := range []string{"GIT_DIRECTORY", "MY_GIT_DIR", "GIT_CONFIG_GLOBAL", ""} {
if isRepositoryLocationVariable(name) {
t.Fatalf("%q treated as a repository-location variable", name)
}
}
}
func TestCommandRefusesRepositoryThatDefinesCommands(t *testing.T) {
enabled := configEntry{key: worktreeConfigKey, value: "true"}
command := configEntry{key: "filter.x.clean", value: "x"}
cases := map[string]map[string][]configEntry{
"local": {"--local": {command}},
"local after the extension": {"--local": {enabled, command}},
"per-worktree": {"--local": {enabled}, "--worktree": {{key: "diff.x.textconv", value: "x"}}},
}
for name, answers := range cases {
t.Run(name, func(t *testing.T) {
answerConfig(t, func(scope string) ([]configEntry, error) { return answers[scope], nil })
got, err := Command(context.Background(), t.TempDir(), "status")
if !errors.Is(err, ErrRepositoryDefinesCommands) {
t.Fatalf("err = %v, want ErrRepositoryDefinesCommands", err)
}
if got != nil {
t.Fatal("a command came back for a repository that defines commands")
}
})
}
}
func TestCommandReadsPerWorktreeConfigOnlyWhenItIsOn(t *testing.T) {
cases := map[string]struct {
local []configEntry
want []string
}{
"no extension": {nil, []string{"--local"}},
"extension off": {[]configEntry{{key: worktreeConfigKey, value: "false"}}, []string{"--local"}},
"extension off as no": {[]configEntry{{key: worktreeConfigKey, value: "No"}}, []string{"--local"}},
"extension on": {[]configEntry{{key: worktreeConfigKey, value: "true"}}, []string{"--local", "--worktree"}},
"extension with no value": {[]configEntry{{key: worktreeConfigKey}}, []string{"--local", "--worktree"}},
"turned off again": {[]configEntry{{key: worktreeConfigKey, value: "true"}, {key: worktreeConfigKey, value: "off"}}, []string{"--local"}},
}
for name, tc := range cases {
t.Run(name, func(t *testing.T) {
calls := answerConfig(t, func(scope string) ([]configEntry, error) {
if scope == "--local" {
return tc.local, nil
}
return nil, nil
})
if _, err := Command(context.Background(), t.TempDir(), "status"); err != nil {
t.Fatalf("Command: %v", err)
}
if got := scopes(calls); !reflect.DeepEqual(got, tc.want) {
t.Fatalf("config reads = %q, want %q", got, tc.want)
}
})
}
}
func TestCommandFailsWhenConfigCannotBeRead(t *testing.T) {
unreadable := errors.New("not a git repository")
for _, failing := range []string{"--local", "--worktree"} {
t.Run(failing, func(t *testing.T) {
answerConfig(t, func(scope string) ([]configEntry, error) {
if scope == failing {
return nil, unreadable
}
return []configEntry{{key: worktreeConfigKey, value: "true"}}, nil
})
if _, err := Command(context.Background(), t.TempDir(), "status"); !errors.Is(err, unreadable) {
t.Fatalf("err = %v, want the config read error", err)
}
})
}
}
// Go's regexp stands in for git's extended regex here; the patterns use nothing
// the two treat differently.
func TestConfigPatterns(t *testing.T) {
local := regexp.MustCompile(localConfigPattern)
worktree := regexp.MustCompile(worktreeConfigPattern)
for _, key := range []string{
"filter.lfs.clean", "filter.lfs.smudge", "filter.lfs.process", "filter.a.b.clean",
"diff.word.textconv", "diff.tool.command", "diff.external",
} {
if !local.MatchString(key) || !worktree.MatchString(key) {
t.Errorf("%s is not matched in both scopes", key)
}
}
if !local.MatchString(worktreeConfigKey) || worktree.MatchString(worktreeConfigKey) {
t.Errorf("%s belongs to the local read only", worktreeConfigKey)
}
for _, key := range []string{
"filter.lfs.required", "diff.word.cachetextconv", "diff.word.binary",
"core.fsmonitor", "diff.external.x", "diff.renames", "extensions.worktreeconfigx",
} {
if local.MatchString(key) || worktree.MatchString(key) {
t.Errorf("%s is matched", key)
}
}
}
func TestParseConfigEntries(t *testing.T) {
got := parseConfigEntries([]byte("filter.x.clean\nrun this\nand that\x00extensions.worktreeconfig\x00"))
want := []configEntry{
{key: "filter.x.clean", value: "run this\nand that"},
{key: "extensions.worktreeconfig"},
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("entries = %+v, want %+v", got, want)
}
}
// Real git: a fresh repository with nothing configured, the same repository with
// per-worktree config turned on, and a directory that is not a repository. This
// pins the output format and exit codes readConfig relies on; the command keys
// themselves are covered above.
func TestCommandAgainstRealGit(t *testing.T) {
if _, err := exec.LookPath("git"); err != nil {
t.Skip("git not available")
}
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home)
t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))
t.Setenv("GIT_CONFIG_GLOBAL", os.DevNull)
t.Setenv("GIT_CONFIG_NOSYSTEM", "1")
repo := t.TempDir()
outside := t.TempDir()
// Both share a parent; stop git from finding an enclosing repository there.
t.Setenv("GIT_CEILING_DIRECTORIES", filepath.Dir(repo))
// Run as if from a hook in another repository. The setup below has to reach
// repo, as Command does, and never the repository these name.
decoy := t.TempDir()
t.Setenv("GIT_DIR", filepath.Join(decoy, ".git"))
t.Setenv("GIT_WORK_TREE", decoy)
decoyUntouched := func(step string) {
t.Helper()
if _, err := os.Stat(filepath.Join(decoy, ".git")); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("%s reached the repository GIT_DIR names instead of repo: %v", step, err)
}
}
git := func(args ...string) {
t.Helper()
command := exec.Command("git", append([]string{"-C", repo}, args...)...)
command.Env = environment()
if output, err := command.CombinedOutput(); err != nil {
t.Fatalf("git %v: %v: %s", args, err, output)
}
}
git("init", "-q")
decoyUntouched("git init")
gitRead := readConfig
var read []string
readConfig = func(ctx context.Context, dir string, env []string, scope string, pattern string) ([]configEntry, error) {
read = append(read, scope)
return gitRead(ctx, dir, env, scope, pattern)
}
t.Cleanup(func() { readConfig = gitRead })
command, err := Command(context.Background(), repo, "status", "--porcelain")
if err != nil {
t.Fatalf("Command in a plain repository: %v", err)
}
if output, err := command.CombinedOutput(); err != nil {
t.Fatalf("git status: %v: %s", err, output)
}
if !reflect.DeepEqual(read, []string{"--local"}) {
t.Fatalf("config reads in a plain repository = %q, want --local only", read)
}
if !gitAtLeast(t, 2, 20) {
t.Log("skipping the per-worktree case: it needs git 2.20 for extensions.worktreeConfig")
} else {
git("config", "extensions.worktreeConfig", "true")
decoyUntouched("git config")
read = nil
if _, err := Command(context.Background(), repo, "status"); err != nil {
t.Fatalf("Command with per-worktree config on: %v", err)
}
if !reflect.DeepEqual(read, []string{"--local", "--worktree"}) {
t.Fatalf("config reads with per-worktree config on = %q, want --local then --worktree", read)
}
}
if _, err := Command(context.Background(), outside, "status"); err == nil {
t.Fatal("Command outside a repository succeeded")
}
}
// gitAtLeast reports whether the git on PATH is at least major.minor.
func gitAtLeast(t *testing.T, major, minor int) bool {
t.Helper()
output, err := exec.Command("git", "version").Output()
if err != nil {
t.Fatalf("git version: %v", err)
}
var gotMajor, gotMinor int
if _, err := fmt.Sscanf(strings.TrimPrefix(strings.TrimSpace(string(output)), "git version "), "%d.%d", &gotMajor, &gotMinor); err != nil {
t.Fatalf("parse %q: %v", output, err)
}
return gotMajor > major || (gotMajor == major && gotMinor >= minor)
}
+110 -4
View File
@@ -23,6 +23,10 @@ import (
// FAKE_NPM_CLOBBER_LATEST=1 makes every publish take `latest`, the failure the
// script must stop on. FAKE_NPM_VERSION_LOOKUP_ERRORS=<n> fails the first n
// version lookups with a network error instead of answering them.
// FAKE_NPM_APPLY_AFTER=<n> applies each publish only after n further lookups,
// the way npm accepts an upload and shows the version minutes later; until
// then the version and its dist-tag are invisible. When a wrapper is published,
// the aliased versions not yet visible are recorded in missingAtWrapperPublish.
const fakeNpm = `#!/usr/bin/env node
const fs = require('fs');
const stateFile = process.env.FAKE_NPM_STATE;
@@ -31,7 +35,17 @@ const args = process.argv.slice(2);
fs.appendFileSync(stateFile + '.log', JSON.stringify({ args, cwd: process.cwd() }) + '\n');
const save = () => fs.writeFileSync(stateFile, JSON.stringify(state));
const notFound = (detail) => { process.stderr.write('npm error code E404\nnpm error 404 ' + detail + '\n'); process.exit(1); };
state.pending = state.pending || [];
const apply = (publish) => {
state.versions.push(publish.version);
state.distTags[publish.tag] = publish.version;
if (process.env.FAKE_NPM_CLOBBER_LATEST === '1') state.distTags.latest = publish.version;
};
if (args[0] === 'view') {
for (const publish of state.pending) publish.wait -= 1;
state.pending.filter((publish) => publish.wait <= 0).forEach(apply);
state.pending = state.pending.filter((publish) => publish.wait > 0);
save();
const [spec, field] = args.slice(1);
const at = spec.lastIndexOf('@');
const name = at > 0 ? spec.slice(0, at) : spec;
@@ -62,14 +76,19 @@ if (args[0] === 'publish') {
process.stderr.write('fake npm: refusing to publish ' + pkg.name + '\n');
process.exit(1);
}
if (state.versions.includes(pkg.version)) {
if (state.versions.includes(pkg.version) || state.pending.some((publish) => publish.version === pkg.version)) {
process.stderr.write('npm error code EPUBLISHCONFLICT\n');
process.exit(1);
}
if (pkg.optionalDependencies) {
state.missingAtWrapperPublish = Object.values(pkg.optionalDependencies)
.map((spec) => spec.slice(spec.lastIndexOf('@') + 1))
.filter((version) => !state.versions.includes(version));
}
const tagIndex = args.indexOf('--tag');
state.versions.push(pkg.version);
state.distTags[tagIndex === -1 ? 'latest' : args[tagIndex + 1]] = pkg.version;
if (process.env.FAKE_NPM_CLOBBER_LATEST === '1') state.distTags.latest = pkg.version;
const publish = { version: pkg.version, tag: tagIndex === -1 ? 'latest' : args[tagIndex + 1], wait: Number(process.env.FAKE_NPM_APPLY_AFTER || 0) };
if (publish.wait > 0) state.pending.push(publish);
else apply(publish);
save();
process.exit(0);
}
@@ -88,6 +107,9 @@ type fakeRegistryState struct {
DistTags map[string]string `json:"distTags"`
// VersionLookups counts `npm view <name>@<version> version` calls.
VersionLookups int `json:"versionLookups,omitempty"`
// MissingAtWrapperPublish lists the platform versions the wrapper aliased
// that npm did not show yet when the wrapper was published.
MissingAtWrapperPublish []string `json:"missingAtWrapperPublish,omitempty"`
}
type npmCall struct {
@@ -325,3 +347,87 @@ func TestPublishPackagesFailsWhenTheRegistryCannotBeReached(t *testing.T) {
t.Fatalf("stderr does not explain the failure or show npm's error: %s", run.stderr)
}
}
// npm accepts an upload and shows the version minutes later (ten minutes for a
// 0.10.0 platform payload). Publishing the wrapper as soon as the platform
// uploads return would leave installs resolving aliases that 404, and checking
// `latest` right after the wrapper upload reports a failure that is not one.
func TestPublishPackagesWaitsForNpmToApplyEachPublish(t *testing.T) {
platforms := []string{"darwin-arm64", "linux-x64"}
npmDir := writePublishFixture(t, platforms, platforms)
run := runPublishPackages(t, npmDir, fakeRegistryState{Name: publishFixtureName}, "FAKE_NPM_APPLY_AFTER=8")
if run.exitCode != 0 {
t.Fatalf("exit = %d while npm applied the publishes late; stdout=%s stderr=%s", run.exitCode, run.stdout, run.stderr)
}
if len(run.state.MissingAtWrapperPublish) != 0 {
t.Fatalf("the wrapper went out while npm did not show %v yet; installs would resolve aliases that 404", run.state.MissingAtWrapperPublish)
}
if latest := run.state.DistTags["latest"]; latest != publishFixtureVersion {
t.Fatalf("latest = %q, want %q", latest, publishFixtureVersion)
}
}
// A publish npm never applies must fail the job at the deadline, before the
// wrapper goes out, instead of waiting forever.
func TestPublishPackagesGivesUpWhenNpmNeverAppliesAPublish(t *testing.T) {
platforms := []string{"linux-x64"}
npmDir := writePublishFixture(t, platforms, platforms)
run := runPublishPackages(t, npmDir, fakeRegistryState{Name: publishFixtureName},
"FAKE_NPM_APPLY_AFTER=1000000", "ZERO_NPM_PUBLISH_WAIT_TIMEOUT=1")
if run.exitCode == 0 {
t.Fatalf("exit = 0 although npm never showed the platform version; stdout=%s", run.stdout)
}
if got := strings.Join(run.publishedDirs(), ","); got != "zero-linux-x64" {
t.Fatalf("published %s, want only the platform payload (the wrapper must wait for it)", got)
}
if !strings.Contains(run.stderr, "still does not show "+publishFixtureName+"@"+publishFixtureVersion+"-linux-x64") {
t.Fatalf("stderr does not say what npm never showed: %s", run.stderr)
}
}
// Bash arithmetic reads a leading zero as octal and rejects 08 outright, so
// the timeout has to be converted explicitly rather than used as typed. The
// rejection does not stop a script: it prints an error, leaves the deadline
// unset and carries on, so the failure only shows once the script has to wait,
// after the platform payloads are out. npm applying late makes it wait here.
func TestPublishPackagesReadsTheWaitTimeoutAsDecimal(t *testing.T) {
platforms := []string{"linux-x64"}
npmDir := writePublishFixture(t, platforms, platforms)
run := runPublishPackages(t, npmDir, fakeRegistryState{Name: publishFixtureName},
"ZERO_NPM_PUBLISH_WAIT_TIMEOUT=08", "FAKE_NPM_APPLY_AFTER=3")
if run.exitCode != 0 {
t.Fatalf("exit = %d with ZERO_NPM_PUBLISH_WAIT_TIMEOUT=08; stdout=%s stderr=%s", run.exitCode, run.stdout, run.stderr)
}
if got := strings.Join(run.publishedDirs(), ","); got != "zero-linux-x64,wrapper" {
t.Fatalf("published %s, want zero-linux-x64,wrapper", got)
}
}
// A setting the script cannot use must stop it before anything is published:
// a negative timeout would otherwise end the wait before it starts, and a bad
// delay would only fail at the first sleep, after the platform payloads.
func TestPublishPackagesRejectsAnInvalidWaitSetting(t *testing.T) {
for _, tc := range []struct{ variable, value string }{
{"ZERO_NPM_PUBLISH_WAIT_TIMEOUT", "-5"},
{"ZERO_NPM_PUBLISH_WAIT_TIMEOUT", "abc"},
{"ZERO_NPM_PUBLISH_WAIT_TIMEOUT", "1e3"},
{"ZERO_NPM_PUBLISH_WAIT_TIMEOUT", "86401"},
{"ZERO_NPM_PUBLISH_WAIT_TIMEOUT", "9999999999999999999999"},
{"ZERO_NPM_PUBLISH_RETRY_DELAY", "soon"},
} {
t.Run(tc.variable+"="+tc.value, func(t *testing.T) {
platforms := []string{"linux-x64"}
npmDir := writePublishFixture(t, platforms, platforms)
run := runPublishPackages(t, npmDir, fakeRegistryState{Name: publishFixtureName}, tc.variable+"="+tc.value)
if run.exitCode == 0 {
t.Fatalf("exit = 0 with %s=%s; stdout=%s", tc.variable, tc.value, run.stdout)
}
if len(run.publishes) != 0 {
t.Fatalf("published %v before rejecting %s=%s, want nothing", run.publishedDirs(), tc.variable, tc.value)
}
if !strings.Contains(run.stderr, tc.variable) {
t.Fatalf("stderr does not name %s: %s", tc.variable, run.stderr)
}
})
}
}
+8
View File
@@ -147,6 +147,14 @@ var descriptors = []Descriptor{
// provider id so the remote catalog resolves without a mapping; GMI's own
// docs name the key GMI_API_KEY, models.dev names it GMICLOUD_API_KEY.
openAICompat("gmicloud", "GMI Cloud", "https://api.gmi-serving.com/v1", "Qwen/Qwen3.8-Max", []string{"GMI_API_KEY", "GMICLOUD_API_KEY"}, "gmi cloud", "gmi"),
// Concentrate AI — OpenAI-compatible gateway to models from many labs.
// models.dev has no Concentrate provider, so the id and key name match
// OpenClaude's Concentrate integration. The default is the dated id
// deepseek-v4-flash-0731: Concentrate's /v1/models keeps the rolling name
// deepseek-v4-flash only as an alias, and onboarding matches the default
// against live ids. "concentrate ai" also covers "concentrate.ai" and
// "concentrate-ai" once normalized.
openAICompat("concentrate", "Concentrate", "https://api.concentrate.ai/v1", "deepseek-v4-flash-0731", []string{"CONCENTRATE_API_KEY"}, "concentrate ai"),
openAICompat("longcat", "LongCat", "https://api.longcat.chat/openai", "LongCat-2.0", []string{"LONGCAT_API_KEY"}, "meituan longcat", "meituan", "longcat-2.0"),
openAICompat("nvidia-nim", "NVIDIA NIM", "https://integrate.api.nvidia.com/v1", "nvidia/llama-3.1-nemotron-70b-instruct", []string{"NVIDIA_API_KEY"}, "nvidia nim"),
anthropicCompat("minimax", "MiniMax", "https://api.minimax.io/anthropic", "MiniMax-M3", []string{"MINIMAX_API_KEY"}, "mini-max", "mini_max"),
+31 -1
View File
@@ -28,6 +28,7 @@ var expectedCatalogIDs = []string{
"moonshot",
"atlascloud",
"gmicloud",
"concentrate",
"longcat",
"nvidia-nim",
"minimax",
@@ -227,6 +228,31 @@ func TestGMICloudDescriptor(t *testing.T) {
}
}
func TestConcentrateDescriptor(t *testing.T) {
descriptor, err := Require("concentrate")
if err != nil {
t.Fatalf("Require(concentrate) error = %v", err)
}
if descriptor.Name != "Concentrate" {
t.Fatalf("Name = %q, want Concentrate", descriptor.Name)
}
if descriptor.DefaultBaseURL != "https://api.concentrate.ai/v1" {
t.Fatalf("DefaultBaseURL = %q, want Concentrate OpenAI-compatible endpoint", descriptor.DefaultBaseURL)
}
if descriptor.DefaultModel != "deepseek-v4-flash-0731" {
t.Fatalf("DefaultModel = %q, want deepseek-v4-flash-0731", descriptor.DefaultModel)
}
if descriptor.Transport != TransportOpenAICompatible {
t.Fatalf("Transport = %q, want %q", descriptor.Transport, TransportOpenAICompatible)
}
if !descriptor.RequiresAuth {
t.Fatal("RequiresAuth = false, want true")
}
if !reflect.DeepEqual(descriptor.AuthEnvVars, []string{"CONCENTRATE_API_KEY"}) {
t.Fatalf("AuthEnvVars = %#v, want CONCENTRATE_API_KEY", descriptor.AuthEnvVars)
}
}
func TestCatalogDescriptorsExposeRequiredDefaults(t *testing.T) {
for _, descriptor := range All() {
if descriptor.ID == "" {
@@ -350,6 +376,10 @@ func TestLookupNormalizesIDsAndAliases(t *testing.T) {
"GMI Cloud": "gmicloud",
"gmi-cloud": "gmicloud",
"gmi": "gmicloud",
"Concentrate": "concentrate",
"concentrate ai": "concentrate",
"concentrate.ai": "concentrate",
"concentrate-ai": "concentrate",
"nvidia nim": "nvidia-nim",
"xiaomi mimo": "xiaomi-mimo",
"custom_openai_compatible": "custom-openai-compatible",
@@ -400,7 +430,7 @@ func TestListByTransportPreservesCatalogOrder(t *testing.T) {
TransportBedrock: {"bedrock"},
TransportVertex: {"vertex"},
TransportAnthropicCompat: {"minimax", "minimaxi-cn", "opencode-go-anthropic-compatible", "custom-anthropic-compatible"},
TransportOpenAICompat: {"gitlawb-opengateway", "aimlapi", "ollama-cloud", "ollama", "lmstudio", "atomic-chat-local", "openrouter", "huggingface", "chatgpt", "groq", "deepseek", "together", "fireworks", "dashscope", "moonshot", "atlascloud", "gmicloud", "longcat", "nvidia-nim", "mistral", "github", "xai", "venice", "xiaomi-mimo", "bankr", "zai", "zai-cn", "kilocode", "opencode", "opencode-go", "atomic-chat", "chatgpt-proxy", "custom-openai-compatible"},
TransportOpenAICompat: {"gitlawb-opengateway", "aimlapi", "ollama-cloud", "ollama", "lmstudio", "atomic-chat-local", "openrouter", "huggingface", "chatgpt", "groq", "deepseek", "together", "fireworks", "dashscope", "moonshot", "atlascloud", "gmicloud", "concentrate", "longcat", "nvidia-nim", "mistral", "github", "xai", "venice", "xiaomi-mimo", "bankr", "zai", "zai-cn", "kilocode", "opencode", "opencode-go", "atomic-chat", "chatgpt-proxy", "custom-openai-compatible"},
}
for transport, wantIDs := range cases {
+14
View File
@@ -27,6 +27,8 @@ type Model struct {
const minimaxModelSource = "https://platform.minimax.io/docs/api-reference/api-overview"
const concentrateModelSource = "https://api.concentrate.ai/v1/models"
// Shared by both "minimax" and "minimaxi-cn"; Models() rebuilds a fresh slice
// per call, so the shared backing slice cannot be mutated by callers.
// Flat cost fields stay unset because MiniMax-M3 pricing is tiered and this
@@ -155,6 +157,18 @@ var curatedModels = map[string][]Model{
{ID: "deepseek-ai/DeepSeek-V4-Pro", Description: "reasoning model"},
{ID: "deepseek-ai/DeepSeek-V4-Flash", Description: "fast coding model"},
},
// Context windows, tool calling and thinking support come from
// concentrateModelSource (max_input_tokens, function_calling, thinking).
// Max output: 393,216 for both DeepSeek V4 models, 262,144 for Kimi K3 and
// GLM-5.3. The default is the dated id deepseek-v4-flash-0731, the model
// behind the deepseek-v4-flash alias (OpenClaude's Concentrate default): the
// live list carries ids only, so an alias would not survive discovery.
"concentrate": {
{ID: "deepseek-v4-flash-0731", Description: "catalog default", ContextWindow: 1_048_576, ToolCall: true, Reasoning: true, Source: concentrateModelSource},
{ID: "deepseek-v4-pro", Description: "reasoning model", ContextWindow: 1_048_576, ToolCall: true, Reasoning: true, Source: concentrateModelSource},
{ID: "kimi-k3", Description: "coding model", ContextWindow: 1_048_576, ToolCall: true, Reasoning: true, Source: concentrateModelSource},
{ID: "glm-5.3", Description: "coding model", ContextWindow: 1_048_576, ToolCall: true, Reasoning: true, Source: concentrateModelSource},
},
"nvidia-nim": {
{ID: "nvidia/llama-3.1-nemotron-70b-instruct", Description: "catalog default"},
{ID: "meta/llama-3.1-70b-instruct", Description: "general model"},
@@ -53,6 +53,11 @@ func TestModelsAreProviderScoped(t *testing.T) {
want: []string{"Qwen/Qwen3.8-Max", "Qwen/Qwen3.8-Flash", "deepseek-ai/DeepSeek-V4-Pro", "deepseek-ai/DeepSeek-V4-Flash"},
notWant: []string{"gpt-4.1", "claude-sonnet-4.5", "deepseek-ai/deepseek-v4-pro"},
},
{
provider: "concentrate",
want: []string{"deepseek-v4-flash-0731", "deepseek-v4-pro", "kimi-k3", "glm-5.3"},
notWant: []string{"gpt-4.1", "claude-sonnet-4.5", "deepseek-ai/DeepSeek-V4-Pro"},
},
{
provider: "mistral",
want: []string{"mistral-large-latest", "codestral-latest"},
@@ -183,6 +188,28 @@ func TestMiniMaxModelsExposeCurrentCapabilities(t *testing.T) {
}
}
func TestConcentrateModelsExposeCatalogLimits(t *testing.T) {
descriptor, ok := providercatalog.Get("concentrate")
if !ok {
t.Fatal("provider concentrate missing from catalog")
}
models := Models(descriptor)
if got, want := modelIDs(models), []string{"deepseek-v4-flash-0731", "deepseek-v4-pro", "kimi-k3", "glm-5.3"}; !reflect.DeepEqual(got, want) {
t.Fatalf("Concentrate models = %#v, want %#v", got, want)
}
if models[0].Description != "catalog default" {
t.Fatalf("first Concentrate model description = %q, want catalog default", models[0].Description)
}
for _, model := range models {
if model.ContextWindow != 1_048_576 || !model.ToolCall || !model.Reasoning {
t.Fatalf("%s capabilities = %#v, want 1,048,576 context with tools and reasoning", model.ID, model)
}
if model.Source != concentrateModelSource {
t.Fatalf("%s source = %q, want %q", model.ID, model.Source, concentrateModelSource)
}
}
}
// Pins the contract that MiniMax CN exposes exactly the same model IDs as the
// international provider; a divergence here is almost always a mistake.
func TestMiniMaxCNModelsMirrorInternational(t *testing.T) {
@@ -5,6 +5,7 @@ import (
"net/http"
"net/http/httptest"
"strings"
"sync/atomic"
"testing"
"github.com/Gitlawb/zero/internal/config"
@@ -38,3 +39,54 @@ func TestDiscoverCatalogFallsBackWhenLiveIDsMismatch(t *testing.T) {
t.Fatalf("models = %q, want curated catalog fallback (gpt-4.1) instead of an empty list", got)
}
}
// Concentrate's /v1/models lists dated snapshots as ids and keeps rolling names
// such as deepseek-v4-flash only in "aliases", which discovery does not read.
// Onboarding keeps the default selected only when it matches a live id, so the
// preset's default must be an id, or setup lands on whatever model sorts first.
func TestDiscoverCatalogConcentrateKeepsDefaultModelInLiveList(t *testing.T) {
var liveHits atomic.Int32
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api.json":
_, _ = w.Write([]byte(`{}`))
case "/v1/models":
liveHits.Add(1)
_, _ = w.Write([]byte(`{"object":"list","data":[
{"id":"claude-fable-5","slug":"claude-fable-5","max_input_tokens":1000000},
{"id":"deepseek-v4-flash-0731","slug":"deepseek-v4-flash-0731","aliases":["deepseek-v4-flash"],"max_input_tokens":1048576},
{"id":"deepseek-v4-pro","slug":"deepseek-v4-pro","max_input_tokens":1048576},
{"id":"kimi-k3","slug":"kimi-k3","max_input_tokens":1048576},
{"id":"glm-5.3","slug":"glm-5.3","max_input_tokens":1048576}
]}`))
default:
t.Fatalf("unexpected request path %q", r.URL.Path)
}
}))
defer server.Close()
provider, ok := providercatalog.Get("concentrate")
if !ok {
t.Fatal("provider concentrate missing from catalog")
}
provider.DefaultBaseURL = server.URL + "/v1"
models, err := DiscoverCatalog(context.Background(), provider, config.ProviderProfile{
CatalogID: "concentrate", ProviderKind: config.ProviderKindOpenAICompatible, BaseURL: server.URL + "/v1", APIKey: "sk-test",
}, Options{HTTPClient: server.Client(), ModelsDevURL: server.URL + "/api.json"})
if err != nil {
t.Fatalf("DiscoverCatalog returned error: %v", err)
}
if liveHits.Load() == 0 {
t.Fatal("DiscoverCatalog never requested /v1/models")
}
// claude-fable-5 is only in the live response, never in the curated list:
// finding it proves the result is the live list, not the curated fallback
// (which also contains the default and would pass on its own).
got := "," + strings.Join(modelIDs(models), ",") + ","
if !strings.Contains(got, ",claude-fable-5,") {
t.Fatalf("models = %q, want the live Concentrate list (with live-only claude-fable-5), not the curated fallback", strings.Trim(got, ","))
}
if !strings.Contains(got, ","+provider.DefaultModel+",") {
t.Fatalf("default model %q missing from discovered models %q", provider.DefaultModel, strings.Trim(got, ","))
}
}
+157 -47
View File
@@ -9,6 +9,7 @@ import (
"sort"
"strings"
"unicode"
"unicode/utf8"
)
const (
@@ -96,6 +97,21 @@ var textSecretPatterns = []*regexp.Regexp{
regexp.MustCompile(`\beyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}`),
}
// textSecretPrefixes lists the mandatory literal prefixes for each entry in
// textSecretPatterns so RedactString can skip the regex scan when none of the
// trigger prefixes appear in the input.
var textSecretPrefixes = [][]string{
{"sk-ant-"},
{"github_pat_"},
{"ghp_", "gho_", "ghu_", "ghs_", "ghr_"},
{"glpat-"},
{"AIza"},
{"xox"},
{"AKIA", "ASIA"},
{"eyJ"},
{"eyJ"},
}
// anchoredSecretPatterns mirrors textSecretPatterns with the leading \b
// replaced by \A, so each shape can be tested at an exact byte offset. A
// credential that starts exactly where a redacted span ends (two AWS keys
@@ -248,55 +264,71 @@ func RedactString(value string, options Options) string {
}
}
redacted = privateKeyPattern.ReplaceAllString(redacted, replacement)
redacted = jsonStringPattern.ReplaceAllStringFunc(redacted, func(match string) string {
parts := jsonStringPattern.FindStringSubmatch(match)
if len(parts) < 3 || !IsSensitiveKey(parts[2], options) {
return match
if strings.Contains(redacted, "-----BEGIN ") && strings.Contains(redacted, "PRIVATE KEY") {
redacted = privateKeyPattern.ReplaceAllString(redacted, replacement)
}
if strings.Contains(redacted, ":") && strings.Contains(redacted, "\"") {
redacted = jsonStringPattern.ReplaceAllStringFunc(redacted, func(match string) string {
parts := jsonStringPattern.FindStringSubmatch(match)
if len(parts) < 3 || !IsSensitiveKey(parts[2], options) {
return match
}
return parts[1] + `"` + replacement + `"`
})
}
if strings.Contains(redacted, "=") {
redacted = assignPattern.ReplaceAllStringFunc(redacted, func(match string) string {
parts := assignPattern.FindStringSubmatch(match)
if len(parts) < 6 || !IsSensitiveKey(parts[1], options) {
return match
}
if parts[3] != "" {
return parts[1] + parts[2] + `"` + replacement + `"`
}
if parts[4] != "" {
return parts[1] + parts[2] + `'` + replacement + `'`
}
return parts[1] + parts[2] + replacement
})
}
if strings.Contains(redacted, ":") {
if containsCaseInsensitive(redacted, "authorization") {
redacted = headerPattern.ReplaceAllStringFunc(redacted, func(match string) string {
groups := headerPattern.FindStringSubmatch(match)
if groups[2] != "" {
return groups[1] + ": " + groups[2] + " " + replacement
}
return groups[1] + ": " + replacement
})
}
return parts[1] + `"` + replacement + `"`
})
redacted = assignPattern.ReplaceAllStringFunc(redacted, func(match string) string {
parts := assignPattern.FindStringSubmatch(match)
if len(parts) < 6 || !IsSensitiveKey(parts[1], options) {
return match
}
if parts[3] != "" {
return parts[1] + parts[2] + `"` + replacement + `"`
}
if parts[4] != "" {
return parts[1] + parts[2] + `'` + replacement + `'`
}
return parts[1] + parts[2] + replacement
})
redacted = headerPattern.ReplaceAllStringFunc(redacted, func(match string) string {
groups := headerPattern.FindStringSubmatch(match)
// groups[2] is the known scheme (kept for readability) or "" for an opaque /
// custom-scheme credential — in which case the whole value is redacted (M12).
if groups[2] != "" {
return groups[1] + ": " + groups[2] + " " + replacement
}
return groups[1] + ": " + replacement
})
redacted = secretHeader.ReplaceAllString(redacted, "$1: "+replacement)
redacted = redactURLPasswords(redacted, replacement)
redacted = queryPattern.ReplaceAllStringFunc(redacted, func(match string) string {
parts := queryPattern.FindStringSubmatch(match)
if len(parts) < 4 || !IsSensitiveKey(parts[2], options) {
return match
}
return parts[1] + parts[2] + "=" + replacement
})
redacted = secretHeader.ReplaceAllString(redacted, "$1: "+replacement)
}
if strings.Contains(redacted, "://") && strings.Contains(redacted, "@") {
redacted = redactURLPasswords(redacted, replacement)
}
if strings.Contains(redacted, "?") || strings.Contains(redacted, "&") {
redacted = queryPattern.ReplaceAllStringFunc(redacted, func(match string) string {
parts := queryPattern.FindStringSubmatch(match)
if len(parts) < 4 || !IsSensitiveKey(parts[2], options) {
return match
}
return parts[1] + parts[2] + "=" + replacement
})
}
// openai keys first so the filter can drop kebab-case false positives
// before any other pattern rewrites nearby text.
redacted = openaiKeyPattern.ReplaceAllStringFunc(redacted, func(match string) string {
if !openAIShapeIsSecret(match) {
return match
}
return replacement
})
if strings.Contains(redacted, "sk-") {
redacted = openaiKeyPattern.ReplaceAllStringFunc(redacted, func(match string) string {
if !openAIShapeIsSecret(match) {
return match
}
return replacement
})
}
for i, pattern := range textSecretPatterns {
redacted = redactAdjacent(redacted, pattern, anchoredSecretPatterns[i], replacement)
if hasAnyPrefix(redacted, textSecretPrefixes[i]) {
redacted = redactAdjacent(redacted, pattern, anchoredSecretPatterns[i], replacement)
}
}
// Credentials broken up by invisible characters last, over what the
// contiguous matchers left: a whole credential is already claimed by its own
@@ -306,6 +338,76 @@ func RedactString(value string, options Options) string {
return redacted
}
func hasAnyPrefix(s string, prefixes []string) bool {
for _, p := range prefixes {
if strings.Contains(s, p) {
return true
}
}
return false
}
func containsCaseInsensitive(s, substr string) bool {
if len(substr) == 0 {
return true
}
if len(s) < len(substr) {
return false
}
for i := 0; i <= len(s)-len(substr); i++ {
if hasFoldedASCIIPrefix(s[i:], substr) {
return true
}
}
return false
}
// hasFoldedASCIIPrefix reports whether s starts with substr under Unicode
// simple case folding (assuming substr is lowercase ASCII). This ensures that
// non-ASCII runes that fold to ASCII in (?i) regular expressions (U+017F 'ſ' ->
// 's' and U+212A 'K' -> 'k') pass the fast-path gate with zero allocations.
func hasFoldedASCIIPrefix(s, substr string) bool {
si := 0
for j := 0; j < len(substr); j++ {
if si >= len(s) {
return false
}
c2 := toLowerASCII(substr[j])
c1 := s[si]
if c1 < utf8.RuneSelf {
if c1 != c2 && toLowerASCII(c1) != c2 {
return false
}
si++
continue
}
r, width := utf8.DecodeRuneInString(s[si:])
if foldRuneToLowerASCII(r) != c2 {
return false
}
si += width
}
return true
}
func foldRuneToLowerASCII(r rune) byte {
switch r {
case '\u017f': // LATIN SMALL LETTER LONG S
return 's'
case '\u212a': // KELVIN SIGN
return 'k'
default:
return 0
}
}
func toLowerASCII(c byte) byte {
if c >= 'A' && c <= 'Z' {
return c + ('a' - 'A')
}
return c
}
// knownOpenAIKeyPrefix is the redaction-side twin of secrets.knownOpenAIKeyPrefix:
// known OpenAI-issued forms redact even with an alphabet-only body.
func knownOpenAIKeyPrefix(match string) bool {
@@ -557,7 +659,15 @@ func redactURLPasswords(value string, replacement string) string {
if _, hasPassword := parsed.User.Password(); !hasPassword {
return candidate
}
parsed.User = url.UserPassword(parsed.User.Username(), replacement)
return parsed.String()
username := parsed.User.Username()
userinfo := url.UserPassword(username, replacement)
parsed.User = userinfo
out := parsed.String()
encodedUserinfo := userinfo.String()
literalUserinfo := url.User(username).String() + ":" + replacement
if encodedUserinfo != literalUserinfo {
out = strings.Replace(out, encodedUserinfo, literalUserinfo, 1)
}
return out
})
}
+418 -3
View File
@@ -33,7 +33,7 @@ func TestRedactStringRedactsAdjacentAWSKeys(t *testing.T) {
func TestRedactStringCoversCommonSecretShapes(t *testing.T) {
input := strings.Join([]string{
`{"apiKey":"sk-proj-abcdefghijklmnopqrstuvwxyz"}`,
"authorization: Bearer ghp_abcdefghijklmnopqrstuvwxyz123456",
"authorization: Bearer ghp_abcdefghijklmnopqrstuvwxyz1234567890",
"https://zero:[email protected]/path?token=glpat-abcdefghijklmnopqrstuvwxyz",
"-----BEGIN PRIVATE KEY-----\nabc123\n-----END PRIVATE KEY-----",
}, "\n")
@@ -42,7 +42,7 @@ func TestRedactStringCoversCommonSecretShapes(t *testing.T) {
for _, leaked := range []string{
"sk-proj-abcdefghijklmnopqrstuvwxyz",
"ghp_abcdefghijklmnopqrstuvwxyz123456",
"ghp_abcdefghijklmnopqrstuvwxyz1234567890",
"super-secret",
"glpat-abcdefghijklmnopqrstuvwxyz",
"abc123",
@@ -81,7 +81,7 @@ func TestRedactValueHandlesSensitiveKeysAndCycles(t *testing.T) {
func TestRedactErrorRedactsMessageStackAndFields(t *testing.T) {
err := withFieldsError{
err: errors.New("request failed with api_key=sk-test-secret1234567890"),
Token: "ghp_abcdefghijklmnopqrstuvwxyz123456",
Token: "ghp_abcdefghijklmnopqrstuvwxyz1234567890",
Detail: "safe",
}
@@ -185,3 +185,418 @@ func TestRedactStringLongAdjacentKeyRunCompletesPromptly(t *testing.T) {
t.Fatalf("expected %d redaction markers, got %d", count, strings.Count(got, RedactedSecret))
}
}
func BenchmarkRedactString(b *testing.B) {
sample := strings.Join([]string{
`{"apiKey":"sk-proj-abcdefghijklmnopqrstuvwxyz1234567890"}`,
"authorization: Bearer ghp_abcdefghijklmnopqrstuvwxyz1234567890",
"https://zero:[email protected]/path?token=glpat-abcdefghijklmnopqrstuvwxyz",
"export AWS_SECRET_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE",
"normal line with no secrets just log messages and numbers 123456789",
"another normal line about git commit status and file diff output",
}, "\n")
opts := Options{ExtraSecretValues: []string{"super-secret"}}
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
_ = RedactString(sample, opts)
}
}
func BenchmarkRedactStringClean(b *testing.B) {
cleanText := "normal line with no secrets just log messages and numbers 123456789\nanother line without secrets"
opts := Options{}
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
_ = RedactString(cleanText, opts)
}
}
func TestRedactStringConditionalGates(t *testing.T) {
const ghp = "ghp_abcdefghijklmnopqrstuvwxyz1234567890"
const gho = "gho_abcdefghijklmnopqrstuvwxyz1234567890"
const pat = "github_pat_abcdefghijklmnopqrstuv"
const ant = "sk-ant-api03-abcdefghijklmnopqrst"
const glpat = "glpat-abcdefghijkl"
const aiza = "AIza" + "01234567890123456789012345678901234"
const slack = "xoxb-1234567890-abcdefghij"
const akia = "AKIAIOSFODNN7EXAMPLE"
const jwt = "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.signaturexx"
tests := []struct {
name string
in string
leaked []string
keep []string
wantHit bool
}{
{
name: "private key match",
in: "-----BEGIN RSA PRIVATE KEY-----\nabc123\n-----END RSA PRIVATE KEY-----",
leaked: []string{"abc123"},
wantHit: true,
},
{
name: "private key near-miss",
in: "-----BEGIN PUBLIC KEY-----\nabc123\n-----END PUBLIC KEY-----",
keep: []string{"abc123"},
wantHit: false,
},
{
name: "json sensitive key",
in: `{"apiKey":"hunter2secret"}`,
leaked: []string{"hunter2secret"},
wantHit: true,
},
{
name: "json non-sensitive key",
in: `{"name":"hunter2secret"}`,
keep: []string{"hunter2secret"},
wantHit: false,
},
{
name: "assign sensitive",
in: "AWS_SECRET_ACCESS_KEY=supersecretvalue",
leaked: []string{"supersecretvalue"},
wantHit: true,
},
{
name: "assign non-sensitive",
in: "PATH=/usr/bin",
keep: []string{"/usr/bin"},
wantHit: false,
},
{
name: "authorization header case-insensitive",
in: "AUTHORIZATION: Bearer " + ghp,
leaked: []string{ghp},
wantHit: true,
},
{
name: "authorization near-miss not a header name",
in: "X-Request-Id: Bearer not-a-token-value",
keep: []string{"not-a-token-value"},
wantHit: false,
},
{
name: "query token",
in: "https://example.test/x?token=" + glpat,
leaked: []string{glpat},
wantHit: true,
},
{
name: "query near-miss",
in: "https://example.test/x?page=42",
keep: []string{"page=42"},
wantHit: false,
},
{
name: "openai sk-proj",
in: "key=" + "sk-proj-abcdefghijklmnopqrstuvwxyz1234",
leaked: []string{"sk-proj-abcdefghijklmnopqrstuvwxyz1234"},
wantHit: true,
},
{
name: "sk-ant",
in: ant,
leaked: []string{ant},
wantHit: true,
},
{
name: "github_pat",
in: pat,
leaked: []string{pat},
wantHit: true,
},
{
name: "ghp classic",
in: ghp,
leaked: []string{ghp},
wantHit: true,
},
{
name: "gho oauth",
in: gho,
leaked: []string{gho},
wantHit: true,
},
{
name: "ghp too short unchanged",
in: "ghp_shorttoken",
keep: []string{"ghp_shorttoken"},
wantHit: false,
},
{
name: "unsupported prefix ghx",
in: "ghx_abcdefghijklmnopqrstuvwxyz1234567890",
keep: []string{"ghx_abcdefghijklmnopqrstuvwxyz1234567890"},
wantHit: false,
},
{
name: "glpat",
in: glpat,
leaked: []string{glpat},
wantHit: true,
},
{
name: "google aiza",
in: aiza,
leaked: []string{aiza},
wantHit: true,
},
{
name: "slack xoxb",
in: slack,
leaked: []string{slack},
wantHit: true,
},
{
name: "aws akia",
in: akia,
leaked: []string{akia},
wantHit: true,
},
{
name: "jwt",
in: jwt,
leaked: []string{jwt},
wantHit: true,
},
{
name: "secret header x-api-key",
in: "X-API-Key: hunter2secret",
leaked: []string{"hunter2secret"},
wantHit: true,
},
{
name: "non-secret header unchanged",
in: "X-Request-Id: hunter2secret",
keep: []string{"hunter2secret"},
wantHit: false,
},
{
name: "url password",
in: "https://user:[email protected]/x",
leaked: []string{"hunter2secret"},
wantHit: true,
},
{
name: "url user without password unchanged",
in: "https://[email protected]/x",
keep: []string{"https://[email protected]/x"},
wantHit: false,
},
{
name: "url password does not unescape path marker",
in: "https://user:[email protected]/%5BREDACTED%5D",
leaked: []string{"hunter2secret"},
keep: []string{"/%5BREDACTED%5D"},
wantHit: true,
},
{
name: "url encoded username does not inject header",
in: "https://%0AAuthorization%3A%20Bearer%20opaque-token:[email protected]/x",
leaked: []string{
"hunter2secret",
"Authorization: Bearer",
"\nAuthorization",
},
keep: []string{"Authorization"},
wantHit: true,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
got := RedactString(tc.in, Options{})
for _, leak := range tc.leaked {
if strings.Contains(got, leak) {
t.Fatalf("leaked %q in %q", leak, got)
}
}
for _, keep := range tc.keep {
if !strings.Contains(got, keep) {
t.Fatalf("near-miss changed: want %q still in %q", keep, got)
}
}
if tc.wantHit && !strings.Contains(got, RedactedSecret) {
t.Fatalf("expected %q, got %q", RedactedSecret, got)
}
if !tc.wantHit && strings.Contains(got, RedactedSecret) {
t.Fatalf("unexpected redaction: %q", got)
}
})
}
}
func TestRedactURLPasswords_HostlessFailClosed(t *testing.T) {
cases := []struct {
in string
leaked string
keep []string
}{
{in: "http://admin:secret@", leaked: "secret", keep: []string{"admin"}},
{in: "http://admin:secret@?q=1", leaked: "secret", keep: []string{"?q=1"}},
{in: "http://admin:secret@#fragment", leaked: "secret", keep: []string{"#fragment"}},
}
for _, tc := range cases {
t.Run(tc.in, func(t *testing.T) {
got := redactURLPasswords(tc.in, RedactedSecret)
if strings.Contains(got, tc.leaked) {
t.Fatalf("leaked %q in %q", tc.leaked, got)
}
if !strings.Contains(got, RedactedSecret) {
t.Fatalf("expected literal %q in %q", RedactedSecret, got)
}
if strings.Contains(got, "%5BREDACTED%5D") {
t.Fatalf("marker was percent-encoded in %q", got)
}
for _, keep := range tc.keep {
if !strings.Contains(got, keep) {
t.Fatalf("want %q still in %q", keep, got)
}
}
})
}
}
func TestRedactString_URLPasswordHostlessFailClosed(t *testing.T) {
tests := []struct {
name string
in string
leaked []string
keep []string
}{
{
name: "hostless trailing at",
in: "http://admin:secret@",
leaked: []string{"secret"},
keep: []string{RedactedSecret, "admin"},
},
{
name: "hostless query",
in: "http://admin:secret@?q=1",
leaked: []string{"secret"},
keep: []string{RedactedSecret, "?q=1"},
},
{
name: "hostless fragment",
in: "http://admin:secret@#fragment",
leaked: []string{"secret"},
keep: []string{RedactedSecret, "#fragment"},
},
{
name: "embedded hostless",
in: "connecting to http://admin:hunter2@ now",
leaked: []string{"hunter2"},
keep: []string{RedactedSecret, "connecting to ", " now"},
},
{
name: "https proxy env hostless",
in: "HTTPS_PROXY=http://proxyuser:pr0xyp4ss@",
leaked: []string{"pr0xyp4ss"},
keep: []string{RedactedSecret, "HTTPS_PROXY=", "proxyuser"},
},
{
name: "host present keeps encoded path and query",
in: "https://user:[email protected]/%5BREDACTED%5D?q=a%20b",
leaked: []string{"secret"},
keep: []string{RedactedSecret, "/%5BREDACTED%5D", "q=a%20b"},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
got := RedactString(tc.in, Options{})
if strings.Contains(got, "%5BREDACTED%5D") && !strings.Contains(tc.in, "%5BREDACTED%5D") {
t.Fatalf("marker was percent-encoded: %q", got)
}
for _, leak := range tc.leaked {
if strings.Contains(got, leak) {
t.Fatalf("leaked %q in %q", leak, got)
}
}
for _, keep := range tc.keep {
if !strings.Contains(got, keep) {
t.Fatalf("want %q still in %q", keep, got)
}
}
})
}
}
func TestRedactString_QueryGateIsolatedFromAssignAndTextSecrets(t *testing.T) {
const opaque = "opaque-query-fixture-value"
in := "https://example.test/x?[password]=" + opaque
if !IsSensitiveKey("[password]", Options{}) {
t.Fatal("fixture key [password] must normalize to a sensitive key")
}
if assignPattern.MatchString(in) {
t.Fatal("fixture must not satisfy assignPattern; otherwise the query gate is not isolated")
}
if kept := RedactString(opaque, Options{}); kept != opaque {
t.Fatalf("opaque value must not match text-secret patterns, got %q", kept)
}
got := RedactString(in, Options{})
if strings.Contains(got, opaque) {
t.Fatalf("query gate missed the value: %q", got)
}
if !strings.Contains(got, RedactedSecret) {
t.Fatalf("query gate did not insert marker: %q", got)
}
}
func TestRedactString_UnicodeFoldedHeaderNames(t *testing.T) {
tests := []struct {
name string
in string
leaked string
}{
{
name: "x-api-key with kelvin sign U+212A",
in: "X-API-\u212aey: opaque-header-secret-1",
leaked: "opaque-header-secret-1",
},
{
name: "set-cookie with long s U+017F",
in: "\u017fet-cookie: session=opaque-cookie-secret-2",
leaked: "opaque-cookie-secret-2",
},
{
name: "containsCaseInsensitive kelvin sign U+212A",
in: "Authori\u212aation",
leaked: "",
},
{
name: "containsCaseInsensitive long s U+017F",
in: "\u017fet-cookie",
leaked: "",
},
}
if !containsCaseInsensitive("Authori\u212aation", "authorikation") {
t.Fatal("containsCaseInsensitive failed to fold U+212A KELVIN SIGN to 'k'")
}
if !containsCaseInsensitive("\u017fet-cookie", "set-cookie") {
t.Fatal("containsCaseInsensitive failed to fold U+017F LONG S to 's'")
}
for _, tc := range tests {
if tc.leaked == "" {
continue
}
t.Run(tc.name, func(t *testing.T) {
got := RedactString(tc.in, Options{})
if strings.Contains(got, tc.leaked) {
t.Fatalf("leaked %q in %q", tc.leaked, got)
}
if !strings.Contains(got, RedactedSecret) {
t.Fatalf("expected %q in %q", RedactedSecret, got)
}
})
}
}
+4 -2
View File
@@ -90,7 +90,9 @@ func TestScanBuildsDeterministicSnapshot(t *testing.T) {
func TestScanDoesNotFollowSymlinkedDirectories(t *testing.T) {
root := t.TempDir()
writeFile(t, root, "main.go", "package main\n")
target := filepath.Join(root, "target")
// Not named "target": outside a git work tree Scan skips that name as Cargo build
// output, and the real directory must be scanned for this test to exercise the link.
target := filepath.Join(root, "real-dir")
writeFile(t, target, "hidden.go", "package hidden\n")
link := filepath.Join(root, "linked")
if err := os.Symlink(target, link); err != nil {
@@ -101,7 +103,7 @@ func TestScanDoesNotFollowSymlinkedDirectories(t *testing.T) {
if err != nil {
t.Fatalf("Scan: %v", err)
}
want := []string{"main.go", "target/hidden.go"}
want := []string{"main.go", "real-dir/hidden.go"}
if !reflect.DeepEqual(pathsOf(got.Files), want) {
t.Fatalf("Files=%v want %v", pathsOf(got.Files), want)
}
+4 -2
View File
@@ -54,7 +54,9 @@ func gitSweepCmd(parent context.Context, cwd string, baseline bool) tea.Cmd {
defer cancel()
// --untracked-files=all enumerates files inside new directories; plain
// porcelain collapses them to "dir/", which is useless as a FILES row.
status, err := defaultPRCommandRunner(ctx, cwd, "git", "status", "--porcelain", "--untracked-files=all")
// --ignore-submodules=dirty keeps git out of submodule work trees, whose
// own config gitsafe does not check.
status, err := defaultPRCommandRunner(ctx, cwd, "git", "status", "--porcelain", "--untracked-files=all", "--ignore-submodules=dirty")
if err != nil {
return gitSweepMsg{baseline: baseline, ok: false}
}
@@ -66,7 +68,7 @@ func gitSweepCmd(parent context.Context, cwd string, baseline bool) tea.Cmd {
// would never match a porcelain path). Best-effort: a failure (e.g. unborn
// HEAD in a fresh repo) keeps the file list with zero counts rather than
// dropping the sweep.
if numstat, err := defaultPRCommandRunner(ctx, cwd, "git", "diff", "HEAD", "--numstat", "-z"); err == nil {
if numstat, err := defaultPRCommandRunner(ctx, cwd, "git", "diff", "HEAD", "--numstat", "-z", "--ignore-submodules=dirty"); err == nil {
stats := parseGitNumstat(numstat)
for i := range files {
if counts, ok := stats[files[i].path]; ok {
+18 -4
View File
@@ -13,6 +13,8 @@ import (
"time"
"charm.land/lipgloss/v2"
"github.com/Gitlawb/zero/internal/gitsafe"
)
type PrStatus string
@@ -243,7 +245,7 @@ func getLocalDiffStats(ctx context.Context, cwd string, baseBranch string, run p
if err != nil {
mergeBase = originBase
}
output, err := run(ctx, cwd, "git", "diff", "--numstat", strings.TrimSpace(mergeBase))
output, err := run(ctx, cwd, "git", "diff", "--numstat", "--ignore-submodules=dirty", strings.TrimSpace(mergeBase))
if err != nil {
return 0, 0, err
}
@@ -290,9 +292,21 @@ func jsonScalarString(raw json.RawMessage) string {
}
func defaultPRCommandRunner(ctx context.Context, cwd string, name string, args ...string) (string, error) {
command := exec.CommandContext(ctx, name, args...)
if strings.TrimSpace(cwd) != "" {
command.Dir = cwd
var command *exec.Cmd
if name == "git" {
// Every git call the TUI makes on its own comes through here (the files
// sweep and the pull request diffstat), so the repository being shown
// cannot make them run a program.
safe, err := gitsafe.Command(ctx, cwd, args...)
if err != nil {
return "", fmt.Errorf("%s: %w", name, err)
}
command = safe
} else {
command = exec.CommandContext(ctx, name, args...)
if strings.TrimSpace(cwd) != "" {
command.Dir = cwd
}
}
var stdout bytes.Buffer
var stderr bytes.Buffer
+3 -3
View File
@@ -15,7 +15,7 @@ func TestPrServiceDetectsGitHubPRAndDiffStats(t *testing.T) {
"git merge-base origin/main HEAD": {
output: "abc123\n",
},
"git diff --numstat abc123": {
"git diff --numstat --ignore-submodules=dirty abc123": {
output: "42\t7\tinternal/tui/view.go\n",
},
}}
@@ -49,7 +49,7 @@ func TestPrServiceFallsBackToGitLabMR(t *testing.T) {
"git merge-base origin/develop HEAD": {
output: "def456\n",
},
"git diff --numstat def456": {
"git diff --numstat --ignore-submodules=dirty def456": {
output: "4\t5\ta.go\n5\t6\tb.go\n",
},
}}
@@ -71,7 +71,7 @@ func TestGetLocalDiffStatsFallsBackToOriginBase(t *testing.T) {
"git merge-base origin/main HEAD": {
err: errors.New("unknown revision"),
},
"git diff --numstat origin/main": {
"git diff --numstat --ignore-submodules=dirty origin/main": {
output: "3\t0\tREADME.md\n",
},
}}
+5 -5
View File
@@ -17,11 +17,11 @@ import (
const npmPackageName = "@twigpine/zero"
// legacyNpmPackageName is the name zero was published under on npm up to
// 0.9.0. The 0.10.0 release is published once more under it (the legacy step
// in .github/workflows/release-artifacts.yml), so that 0.9.0's `zero upgrade`
// and anyone still installing the old name land on a build that says where
// the package went. Nothing is published under it after that, which is why an
// install of it is told to switch instead of offered an update.
// 0.9.0. The 0.10.0 release was published once more under it, so that 0.9.0's
// `zero upgrade` and anyone still installing the old name land on a build that
// says where the package went (docs/NPM_PACKAGING.md). Nothing is published
// under it after that, which is why an install of it is told to switch instead
// of offered an update.
const legacyNpmPackageName = "@gitlawb/zero"
// InstallMethod identifies how the running zero binary was installed.
+139
View File
@@ -0,0 +1,139 @@
package workspaceindex
import (
"bytes"
"context"
"os"
"os/exec"
"path"
"strings"
"time"
)
// gitIgnoredPathsTimeout bounds how long Scan waits on git before giving up and falling
// back to the non-git behavior. Scan also runs for the per-turn workspace seed, so a slow or
// degenerate tree (network filesystem, huge monorepo) must not stall a turn.
const gitIgnoredPathsTimeout = 5 * time.Second
// gitIgnoredPaths asks git which paths under root it considers ignored, honoring every
// source git itself resolves (nested .gitignore files, .git/info/exclude,
// core.excludesFile). Returns a set of paths relative to root: whole ignored directories
// collapsed to one entry plus individually ignored files, using forward slashes and no
// trailing slash (the `rel` form Scan computes).
//
// Returns (nil, false) when root is not inside a git work tree, git is missing or older
// than the floor below, or a command fails or times out. Scan then behaves as it does
// outside git. A scan must never fail or hang because of this lookup.
//
// Git version floor: 2.16. `status --ignored=<mode>` landed in 2.16 and `--porcelain=v1`
// in 2.11; an older git rejects the arguments, which lands on the (nil, false) path.
// GIT_OPTIONAL_LOCKS (2.15) is an environment variable, so an older git ignores it.
//
// Why `status --ignored=matching` and not `ls-files --others --ignored --directory`:
// ls-files stops descending at the first untracked directory, so an ignored directory or
// file nested inside one (svc/build-out/, app/cache.db) is never reported. status descends,
// collapses a fully ignored directory to a single entry, and lists partially ignored files
// individually. `-uno` is not usable as a speedup: with it, status reports no ignored
// entries at all.
func gitIgnoredPaths(root string) (map[string]bool, bool) {
ctx, cancel := context.WithTimeout(context.Background(), gitIgnoredPathsTimeout)
defer cancel()
// status reports paths relative to the repository top level even when run from a
// subdirectory, while Scan's `rel` is relative to root. show-prefix is root's own
// location inside the repo ("" at the top level, "svc/" for <repo>/svc), and also the
// cheapest way to learn whether this is a git work tree at all.
prefixOut, err := gitCommand(ctx, root, "rev-parse", "--show-prefix").Output()
if err != nil {
return nil, false
}
prefix := strings.TrimSpace(string(prefixOut))
// -z: NUL-terminated and unquoted. Without it git C-quotes non-ASCII bytes
// (core.quotePath), e.g. "caf\303\251/", which would never match the raw `rel`.
// The trailing "." limits the report to root when root is a repository subdirectory.
// --untracked-files=normal: status.showUntrackedFiles=no makes git reject
// --ignored=matching; without an explicit override Scan falls back to the fixed list.
out, err := gitCommand(ctx, root, "status", "--porcelain=v1", "-z", "--untracked-files=normal", "--ignored=matching", ".").Output()
if err != nil {
return nil, false
}
return parseIgnoredStatus(out, prefix), true
}
// parseIgnoredStatus extracts the ignored paths from `git status --porcelain=v1 -z` output,
// made relative to prefix (root's location inside the repository).
//
// Each record is "XY <path>". A rename or copy (R or C in either status column) is followed
// by one more NUL field holding the original path with no XY prefix. That field is consumed,
// not inspected: an original name that itself begins with "!! " would otherwise read as an
// ignored entry and hide a real file from Scan.
func parseIgnoredStatus(out []byte, prefix string) map[string]bool {
paths := make(map[string]bool)
skipOrigPath := false
for entry := range bytes.SplitSeq(out, []byte{0}) {
if skipOrigPath {
skipOrigPath = false
continue
}
if len(entry) < 3 {
continue
}
if x, y := entry[0], entry[1]; x == 'R' || x == 'C' || y == 'R' || y == 'C' {
skipOrigPath = true
continue
}
// Ignored entries are "!! <path>"; untracked "??" and changed entries are skipped.
rest, ok := bytes.CutPrefix(entry, []byte("!! "))
if !ok {
continue
}
rel, ok := strings.CutPrefix(string(rest), prefix)
if !ok {
continue
}
rel = path.Clean(strings.TrimSuffix(rel, "/"))
if rel == "." || rel == "" {
continue
}
paths[rel] = true
}
return paths
}
// gitCommand builds a git invocation with a fixed argv (no shell) rooted at dir.
//
// GIT_OPTIONAL_LOCKS=0 stops `git status` from taking .git/index.lock to refresh the
// index stat cache. Scan runs on every turn, often while the user or the agent is running
// git in the same repository; an opportunistic lock there can make a concurrent
// `git commit` or `git add` fail with "index.lock exists".
func gitCommand(ctx context.Context, dir string, args ...string) *exec.Cmd {
// #nosec G204 -- fixed argv, no shell; dir is the caller's own absolute path.
// core.fsmonitor in .git/config can point at a command git status runs; disable it so
// Scan never executes foreign hooks from the repository under scan.
cmd := exec.CommandContext(ctx, "git", append([]string{"-C", dir, "-c", "core.fsmonitor=false"}, args...)...)
cmd.Env = append(withoutRepoLocationEnv(os.Environ()), "GIT_OPTIONAL_LOCKS=0")
return cmd
}
// withoutRepoLocationEnv drops the repository path overrides from git(1),
// "The Git Repository": GIT_DIR, GIT_WORK_TREE, GIT_INDEX_FILE, GIT_COMMON_DIR,
// GIT_OBJECT_DIRECTORY, and GIT_ALTERNATE_OBJECT_DIRECTORIES. Hooks export
// them (githooks(5)) so git commands inside the hook find that repository.
// Inherited, -C dir still describes the hook's repository, and Scan applies
// its ignore rules instead of the tree under dir. GIT_CEILING_DIRECTORIES is
// not one of these; it only bounds discovery and must be inherited.
func withoutRepoLocationEnv(env []string) []string {
out := make([]string, 0, len(env))
for _, kv := range env {
key, _, _ := strings.Cut(kv, "=")
switch key {
case "GIT_DIR", "GIT_WORK_TREE", "GIT_INDEX_FILE", "GIT_COMMON_DIR",
"GIT_OBJECT_DIRECTORY", "GIT_ALTERNATE_OBJECT_DIRECTORIES":
continue
}
out = append(out, kv)
}
return out
}
+387
View File
@@ -0,0 +1,387 @@
package workspaceindex
import (
"context"
"fmt"
"os"
"os/exec"
"path/filepath"
"reflect"
"strings"
"testing"
)
// isolateGitConfig keeps the git child processes away from the developer's real global and
// system config: a global core.excludesFile would change what counts as ignored and make
// these tests depend on the machine. GIT_CONFIG_GLOBAL needs git >= 2.32; HOME and
// XDG_CONFIG_HOME cover older git and macOS/Windows home resolution.
func isolateGitConfig(t *testing.T) {
t.Helper()
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home)
t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))
t.Setenv("GIT_CONFIG_GLOBAL", os.DevNull)
t.Setenv("GIT_CONFIG_NOSYSTEM", "1")
}
// stopGitDiscoveryAbove keeps git from finding an enclosing repository above root, so a
// TMPDIR that happens to live inside a git work tree cannot turn a "not a repo" fixture
// into one.
func stopGitDiscoveryAbove(t *testing.T, root string) {
t.Helper()
t.Setenv("GIT_CEILING_DIRECTORIES", filepath.Dir(root))
}
// requireGitIgnoredMatching skips below git 2.16, where `status --ignored=matching` landed.
// Older git takes Scan's non-git path, which the fallback tests cover separately.
func requireGitIgnoredMatching(t *testing.T) {
t.Helper()
out, err := exec.Command("git", "version").Output()
if err != nil {
t.Skipf("git unavailable: %v", err)
}
var major, minor int
if _, err := fmt.Sscanf(strings.TrimPrefix(strings.TrimSpace(string(out)), "git version "), "%d.%d", &major, &minor); err != nil {
t.Skipf("cannot parse %q: %v", out, err)
}
if major < 2 || (major == 2 && minor < 16) {
t.Skipf("needs git >= 2.16 for status --ignored=matching, have %d.%d", major, minor)
}
}
func runGit(t *testing.T, dir string, args ...string) {
t.Helper()
cmd := exec.Command("git", append([]string{"-C", dir, "-c", "user.name=zero-test", "-c", "[email protected]", "-c", "commit.gpgsign=false"}, args...)...)
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("git %v: %v: %s", args, err, out)
}
}
func initGitRepo(t *testing.T, root, gitignore string) {
t.Helper()
isolateGitConfig(t)
requireGitIgnoredMatching(t)
stopGitDiscoveryAbove(t, root)
runGit(t, root, "init", "-q")
if gitignore != "" {
writeFile(t, root, ".gitignore", gitignore)
}
}
// A virtualenv or cache directory whose name no fixed list anticipates must still be
// skipped when the repository's own .gitignore excludes it. Reported against a real repo
// where `zero repo-map` spent its whole file budget inside a `.locust_env/` virtualenv and
// never reached the repo's own README.md.
func TestScanHonorsRepoGitignore(t *testing.T) {
root := t.TempDir()
initGitRepo(t, root, ".locust_env/\n*.db\ncafé-cache/\n")
writeFile(t, root, "README.md", "# Example\n")
writeFile(t, root, "app/main.py", "print('hi')\n")
writeFile(t, root, "app/café.py", "x = 1\n")
writeFile(t, root, ".locust_env/lib/python3.13/site-packages/flask/sansio/README.md", "vendored")
writeFile(t, root, "audit_results.db", "sqlite")
// Ignored file inside an untracked directory: `ls-files --directory` would miss it.
writeFile(t, root, "app/cache.db", "sqlite")
// Non-ASCII ignored directory: only matches because the lookup uses -z (no C-quoting).
writeFile(t, root, "café-cache/blob.json", "{}")
got, err := Scan(root, Options{MaxDepth: DefaultMaxDepth})
if err != nil {
t.Fatalf("Scan: %v", err)
}
want := []string{".gitignore", "README.md", "app/café.py", "app/main.py"}
if !reflect.DeepEqual(pathsOf(got.Files), want) {
t.Fatalf("Files=%v want %v", pathsOf(got.Files), want)
}
}
// Scanning a repository subdirectory must resolve ignore rules relative to that
// subdirectory (Scan's `rel`), not relative to the repository top level.
func TestScanHonorsRepoGitignoreFromSubdirectoryRoot(t *testing.T) {
repo := t.TempDir()
initGitRepo(t, repo, "build-out/\n")
writeFile(t, repo, "svc/handler.go", "package svc\n")
writeFile(t, repo, "svc/build-out/gen.go", "package gen\n")
got, err := Scan(filepath.Join(repo, "svc"), Options{MaxDepth: DefaultMaxDepth})
if err != nil {
t.Fatalf("Scan: %v", err)
}
if want := []string{"handler.go"}; !reflect.DeepEqual(pathsOf(got.Files), want) {
t.Fatalf("Files=%v want %v", pathsOf(got.Files), want)
}
}
// Inside a git work tree, git decides. A tracked directory that merely shares a name with
// a build-cache directory (a Go package called "target", say) is real source and must be
// scanned; the fixed list is only a stand-in for when git cannot be asked.
func TestScanKeepsTrackedBuildCacheNamedDirInsideGitRepo(t *testing.T) {
root := t.TempDir()
initGitRepo(t, root, "")
writeFile(t, root, "main.go", "package main\n")
writeFile(t, root, "target/target.go", "package target\n")
runGit(t, root, "add", ".")
runGit(t, root, "commit", "-q", "-m", "fixture")
got, err := Scan(root, Options{MaxDepth: DefaultMaxDepth})
if err != nil {
t.Fatalf("Scan: %v", err)
}
if want := []string{"main.go", "target/target.go"}; !reflect.DeepEqual(pathsOf(got.Files), want) {
t.Fatalf("Files=%v want %v", pathsOf(got.Files), want)
}
}
// Scanning a directory that the enclosing repository ignores as a whole must still list
// its contents: git reports the root itself as ignored, which is not a reason to skip
// everything under it.
func TestScanOfIgnoredRootStillListsItsFiles(t *testing.T) {
repo := t.TempDir()
initGitRepo(t, repo, "out/\n")
writeFile(t, repo, "out/report.md", "# report\n")
writeFile(t, repo, "out/data/rows.csv", "a,b\n")
got, err := Scan(filepath.Join(repo, "out"), Options{MaxDepth: DefaultMaxDepth})
if err != nil {
t.Fatalf("Scan: %v", err)
}
if want := []string{"data/rows.csv", "report.md"}; !reflect.DeepEqual(pathsOf(got.Files), want) {
t.Fatalf("Files=%v want %v", pathsOf(got.Files), want)
}
}
// Outside a git work tree, Scan falls back to the fixed build/cache list so the file
// budget is not spent on build output. Reported against a repo where
// rust/*/target/debug/.fingerprint alone held hundreds of files: with a small budget the
// real source must still be reached and the scan must not report truncation.
func TestScanSkipsBuildCacheDirsOutsideGit(t *testing.T) {
isolateGitConfig(t)
root := t.TempDir()
stopGitDiscoveryAbove(t, root)
if _, ok := gitIgnoredPaths(root); ok {
t.Fatalf("gitIgnoredPaths(non-repo) reported a work tree")
}
writeFile(t, root, "src/lib.rs", "pub fn f() {}\n")
writeFile(t, root, "app/main.py", "print('hi')\n")
for i := range 10 {
writeFile(t, root, fmt.Sprintf("rust/crate/target/debug/.fingerprint/foo-%d/lib-foo.d", i), "x")
}
writeFile(t, root, "app/__pycache__/main.cpython-312.pyc", "x")
writeFile(t, root, ".venv/lib/site-packages/pkg.py", "x")
writeFile(t, root, "Venv/lib/pkg.py", "x")
writeFile(t, root, ".terraform/providers/provider.json", "x")
writeFile(t, root, ".pytest_cache/v/cache/nodeids", "x")
writeFile(t, root, ".mypy_cache/3.12/mod.json", "x")
writeFile(t, root, ".ruff_cache/content", "x")
got, err := Scan(root, Options{MaxFiles: 3, MaxDepth: DefaultMaxDepth})
if err != nil {
t.Fatalf("Scan: %v", err)
}
if want := []string{"app/main.py", "src/lib.rs"}; !reflect.DeepEqual(pathsOf(got.Files), want) {
t.Fatalf("Files=%v want %v", pathsOf(got.Files), want)
}
if got.Truncated {
t.Fatalf("Truncated=true: build-cache files consumed the file budget")
}
}
// When git cannot run at all, Scan must not error and must behave exactly as outside a
// repository, even inside one: .gitignore is not consulted, the fixed list applies.
func TestScanFallsBackWhenGitUnavailable(t *testing.T) {
root := t.TempDir()
initGitRepo(t, root, ".locust_env/\n")
writeFile(t, root, "main.go", "package main\n")
writeFile(t, root, ".locust_env/lib/x.py", "kept: git is not available to honor .gitignore")
writeFile(t, root, "target/debug/app.d", "skipped by the fixed list")
t.Setenv("PATH", t.TempDir())
if paths, ok := gitIgnoredPaths(root); ok || paths != nil {
t.Fatalf("gitIgnoredPaths without git = (%v, %v), want (nil, false)", paths, ok)
}
got, err := Scan(root, Options{MaxDepth: DefaultMaxDepth})
if err != nil {
t.Fatalf("Scan: %v", err)
}
if want := []string{".gitignore", ".locust_env/lib/x.py", "main.go"}; !reflect.DeepEqual(pathsOf(got.Files), want) {
t.Fatalf("Files=%v want %v", pathsOf(got.Files), want)
}
}
// ShouldSkipDir is shared with glob, grep, list_directory and path autocomplete. The
// build/cache names Scan skips must stay visible there, so a question about a dependency in
// .venv or generated code under target can still be answered.
func TestShouldSkipDirLeavesScanOnlyDirsVisibleToTools(t *testing.T) {
for _, name := range []string{"target", "__pycache__", ".venv", "venv", ".pytest_cache", ".terraform", ".mypy_cache", ".ruff_cache"} {
if ShouldSkipDir(name) {
t.Fatalf("ShouldSkipDir(%q)=true: tools would stop seeing it", name)
}
if !isScanOnlySkipDir(name) {
t.Fatalf("isScanOnlySkipDir(%q)=false want true", name)
}
}
for _, name := range []string{"Target", " target ", "__PYCACHE__", ".VENV"} {
if !isScanOnlySkipDir(name) {
t.Fatalf("isScanOnlySkipDir(%q)=false want true (case/whitespace-insensitive)", name)
}
}
for _, name := range []string{"src", "targets", "venv2", "internal"} {
if isScanOnlySkipDir(name) {
t.Fatalf("isScanOnlySkipDir(%q)=true want false", name)
}
}
}
// A repository's core.fsmonitor can run an external command during status; the
// ignored-path lookup must not trigger it. Git for Windows hands both that
// command and the hook body to sh, which consumes backslashes. The script path
// stored in core.fsmonitor and the marker path inside the hook are quoted and
// use forward slashes: an unquoted backslash script path never starts the
// hook, and an unquoted backslash marker (touch C:\tmp\run\marker) creates
// C:tmprunmarker, so the file this test stats is never created. A status
// without the override must create the marker first; otherwise a hook that
// does not run would still pass.
func TestGitIgnoredPathsDisablesRepoFsmonitor(t *testing.T) {
root := t.TempDir()
initGitRepo(t, root, ".locust_env/\n")
writeFile(t, root, "README.md", "# x\n")
writeFile(t, root, ".locust_env/lib/x.py", "x\n")
marker := filepath.Join(root, ".fsmonitor-ran")
script := filepath.Join(root, "fsmonitor-hook.sh")
hook := "#!/bin/sh\ntouch '" + filepath.ToSlash(marker) + "'\n"
if err := os.WriteFile(script, []byte(hook), 0o755); err != nil {
t.Fatalf("write fsmonitor hook: %v", err)
}
runGit(t, root, "config", "core.fsmonitor", "'"+filepath.ToSlash(script)+"'")
runGit(t, root, "status", "--porcelain=v1", "-z", "--untracked-files=normal", "--ignored=matching", ".")
if _, err := os.Stat(marker); err != nil {
t.Fatalf("fsmonitor hook did not run without the override: %v", err)
}
if err := os.Remove(marker); err != nil {
t.Fatalf("remove marker: %v", err)
}
if _, ok := gitIgnoredPaths(root); !ok {
t.Fatal("gitIgnoredPaths failed")
}
if _, err := os.Stat(marker); err == nil {
t.Fatalf("fsmonitor hook ran during gitIgnoredPaths (marker %q exists)", marker)
} else if !os.IsNotExist(err) {
t.Fatalf("stat marker: %v", err)
}
}
// status.showUntrackedFiles=no makes git reject --ignored=matching unless
// --untracked-files=normal is passed explicitly; ignored paths must still come from git.
func TestScanHonorsRepoGitignoreWhenShowUntrackedFilesNo(t *testing.T) {
root := t.TempDir()
initGitRepo(t, root, ".locust_env/\n")
runGit(t, root, "config", "status.showUntrackedFiles", "no")
writeFile(t, root, "README.md", "# Example\n")
writeFile(t, root, "app/main.py", "print('hi')\n")
writeFile(t, root, ".locust_env/lib/x.py", "vendored\n")
got, err := Scan(root, Options{MaxDepth: DefaultMaxDepth})
if err != nil {
t.Fatalf("Scan: %v", err)
}
want := []string{".gitignore", "README.md", "app/main.py"}
if !reflect.DeepEqual(pathsOf(got.Files), want) {
t.Fatalf("Files=%v want %v", pathsOf(got.Files), want)
}
}
// Git exports repository-location variables to hooks (githooks(5)). With them
// pointing at another repository, Scan must still apply root's own .gitignore
// and match a lookup made without those variables.
func TestScanIgnoresInheritedRepositoryLocation(t *testing.T) {
root := t.TempDir()
initGitRepo(t, root, ".locust_env/\n")
writeFile(t, root, "README.md", "# Example\n")
writeFile(t, root, ".locust_env/lib/x.py", "vendored\n")
clean, err := Scan(root, Options{MaxDepth: DefaultMaxDepth})
if err != nil {
t.Fatalf("Scan: %v", err)
}
other := t.TempDir()
initGitRepo(t, other, "")
writeFile(t, other, "README.md", "other\n")
gitDir := filepath.Join(other, ".git")
t.Setenv("GIT_DIR", gitDir)
t.Setenv("GIT_WORK_TREE", other)
t.Setenv("GIT_INDEX_FILE", filepath.Join(gitDir, "index"))
t.Setenv("GIT_COMMON_DIR", gitDir)
t.Setenv("GIT_OBJECT_DIRECTORY", filepath.Join(gitDir, "objects"))
t.Setenv("GIT_ALTERNATE_OBJECT_DIRECTORIES", filepath.Join(gitDir, "objects"))
got, err := Scan(root, Options{MaxDepth: DefaultMaxDepth})
if err != nil {
t.Fatalf("Scan with inherited repository location: %v", err)
}
if !reflect.DeepEqual(pathsOf(got.Files), pathsOf(clean.Files)) {
t.Fatalf("Files=%v want clean environment %v", pathsOf(got.Files), pathsOf(clean.Files))
}
}
// The ignored-path lookup runs every turn, often next to a git command the user or agent
// is running in the same repository. It must not take the optional index lock, and a
// GIT_OPTIONAL_LOCKS value inherited from the environment must not re-enable it.
func TestGitCommandDisablesOptionalLocks(t *testing.T) {
t.Setenv("GIT_OPTIONAL_LOCKS", "1")
cmd := gitCommand(context.Background(), t.TempDir(), "status")
value := ""
for _, entry := range cmd.Env {
if after, ok := strings.CutPrefix(entry, "GIT_OPTIONAL_LOCKS="); ok {
value = after
}
}
if value != "0" {
t.Fatalf("effective GIT_OPTIONAL_LOCKS=%q want \"0\"", value)
}
}
// A rename or copy record carries its original path in the next NUL field, with no XY
// prefix. An original name that starts with "!! " must not be read as an ignored entry.
func TestParseIgnoredStatusSkipsRenameAndCopyOrigPaths(t *testing.T) {
out := []byte("R b\x00!! a\x00" +
"C d\x00!! c\x00" +
" R e\x00!! f\x00" +
"!! out/\x00" +
"?? new.txt\x00" +
" M main.go\x00" +
"!! svc/build/\x00")
if got, want := parseIgnoredStatus(out, ""), map[string]bool{"out": true, "svc/build": true}; !reflect.DeepEqual(got, want) {
t.Fatalf("prefix \"\": got %v want %v", got, want)
}
if got, want := parseIgnoredStatus(out, "svc/"), map[string]bool{"build": true}; !reflect.DeepEqual(got, want) {
t.Fatalf("prefix \"svc/\": got %v want %v", got, want)
}
}
// End to end: after `git mv "!! a" b`, status reports "R b\0!! a\0". The tracked file "a"
// must still be scanned.
func TestScanKeepsFileNamedLikeRenameOrigPath(t *testing.T) {
root := t.TempDir()
initGitRepo(t, root, "")
writeFile(t, root, "!! a", "moved\n")
writeFile(t, root, "a", "kept\n")
runGit(t, root, "add", ".")
runGit(t, root, "commit", "-q", "-m", "fixture")
runGit(t, root, "mv", "!! a", "b")
got, err := Scan(root, Options{MaxDepth: DefaultMaxDepth})
if err != nil {
t.Fatalf("Scan: %v", err)
}
if want := []string{"a", "b"}; !reflect.DeepEqual(pathsOf(got.Files), want) {
t.Fatalf("Files=%v want %v", pathsOf(got.Files), want)
}
}
+26 -2
View File
@@ -63,6 +63,18 @@ func Scan(root string, options Options) (Summary, error) {
maxDepth = DefaultMaxDepth
}
// Inside a git work tree, git's own ignore rules decide what else to skip; outside one
// (or when git is unavailable or too old), a fixed list of build/cache directory names
// stands in. Both apply to Scan only: ShouldSkipDir is shared with glob, grep,
// list_directory and path autocomplete, which must keep seeing these directories.
ignored, hasIgnored := gitIgnoredPaths(cleanRoot)
skipScanDir := func(rel, name string) bool {
if hasIgnored {
return ignored[rel]
}
return isScanOnlySkipDir(name)
}
files := []File{}
dirs := map[string]struct{}{}
maxDepthSeen := 0
@@ -83,7 +95,7 @@ func Scan(root string, options Options) (Summary, error) {
rel = filepath.ToSlash(rel)
if entry.IsDir() {
if ShouldSkipDir(entry.Name()) || isSymlink(entry) {
if ShouldSkipDir(entry.Name()) || isSymlink(entry) || skipScanDir(rel, entry.Name()) {
return filepath.SkipDir
}
depth := pathDepth(rel)
@@ -98,7 +110,7 @@ func Scan(root string, options Options) (Summary, error) {
return nil
}
if isSymlink(entry) || ShouldSkipFile(rel) {
if isSymlink(entry) || ShouldSkipFile(rel) || (hasIgnored && ignored[rel]) {
return nil
}
depth := FileDepth(rel)
@@ -175,6 +187,18 @@ func ShouldSkipDir(name string) bool {
}
}
// isScanOnlySkipDir names build and cache directories Scan skips when it cannot ask git
// what is ignored. It is deliberately separate from ShouldSkipDir: a tracked package named
// "target" or a question about a dependency inside .venv must stay visible to the tools.
func isScanOnlySkipDir(name string) bool {
switch strings.ToLower(strings.TrimSpace(name)) {
case "target", "__pycache__", ".venv", "venv", ".pytest_cache", ".terraform", ".mypy_cache", ".ruff_cache":
return true
default:
return false
}
}
func ShouldSkipFile(rel string) bool {
base := strings.ToLower(path.Base(filepath.ToSlash(rel)))
switch base {
+3 -1
View File
@@ -83,7 +83,9 @@ func Build(input Input) Seed {
}
// BuildFromWorkspace scans the local filesystem with workspaceindex and builds
// a seed from the resulting file list. It performs no git operations.
// a seed from the resulting file list. Its only git operation is the read-only
// ignored-path lookup inside workspaceindex.Scan; branch and dirty state come
// from the caller's GitInfo.
func BuildFromWorkspace(root string, git GitInfo) (Seed, error) {
summary, err := workspaceindex.Scan(root, workspaceindex.Options{
MaxFiles: workspaceindex.DefaultMaxFiles,
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "@twigpine/zero",
"version": "0.10.0",
"version": "0.11.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@twigpine/zero",
"version": "0.10.0",
"version": "0.11.0",
"cpu": [
"x64",
"arm64"
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@twigpine/zero",
"version": "0.10.0",
"version": "0.11.0",
"description": "Zero — a fast, multi-provider terminal coding agent.",
"type": "module",
"module": "bin/zero.js",
+5 -4
View File
@@ -25,10 +25,11 @@
// CI must always assemble the full matrix, and the wrapper's
// optionalDependencies always reference the full matrix regardless).
//
// --package-name assembles the same artifacts under another name. It exists
// for the one final release under the pre-rename name @gitlawb/zero (see
// docs/NPM_PACKAGING.md); everything in the payloads that names the package
// follows it, so the binary and wrapper know which name they were installed as.
// --package-name assembles the same artifacts under another name. It was
// added for the one release (0.10.0) also published under the pre-rename name
// @gitlawb/zero (see docs/NPM_PACKAGING.md); everything in the payloads that
// names the package follows it, so the binary and wrapper know which name they
// were installed as.
//
// Usage:
// node scripts/npm/build-platform-packages.mjs \
+67 -16
View File
@@ -4,8 +4,15 @@
# with the dist-tags docs/NPM_PACKAGING.md requires:
#
# 1. every platform payload, under the `platform` dist-tag, never `latest`;
# 2. check that `latest` was not moved onto a platform version;
# 3. the wrapper, which takes `latest`, then confirm `latest` is its version.
# 2. wait until npm shows every platform version, then check that `latest`
# was not moved onto one;
# 3. the wrapper, which takes `latest`, then wait until `latest` is its
# version.
#
# The waits exist because npm applies a publish asynchronously: `npm publish`
# returns once the upload is accepted, and the version can appear minutes
# later (ten minutes for a platform payload of 0.10.0). Ordering the uploads
# alone would still let the wrapper appear first, with aliases that 404.
#
# The package name comes from the assembled wrapper's package.json, so the
# same script publishes whatever name the payloads were built under.
@@ -19,7 +26,9 @@
# Usage: scripts/npm/publish-packages.sh <npm-dir>
#
# Auth comes from the environment (npm OIDC trusted publishing in CI).
# ZERO_NPM_PUBLISH_RETRY_DELAY overrides the seconds between dist-tag lookups.
# ZERO_NPM_PUBLISH_RETRY_DELAY overrides the seconds between registry lookups,
# and ZERO_NPM_PUBLISH_WAIT_TIMEOUT the seconds to wait for npm to apply the
# publishes (default 1800).
set -euo pipefail
fail() {
@@ -39,7 +48,24 @@ manifest_field() {
name="$(manifest_field "$wrapper_dir" name)"
version="$(manifest_field "$wrapper_dir" version)"
[ -n "$name" ] && [ -n "$version" ] || fail "${wrapper_dir}/package.json has no name or version"
delay="${ZERO_NPM_PUBLISH_RETRY_DELAY:-5}"
# whole_seconds <variable> <value> <max>: the value as a decimal number of
# seconds. Checked here, before anything is published: Bash arithmetic reads a
# leading zero as octal (0600 would wait 384 seconds, and 08 is an error), a
# negative timeout would end the wait before it starts, and a bad delay would
# only fail at the first sleep, possibly after the platform payloads went out.
whole_seconds() {
case "$2" in
'' | *[!0-9]*) fail "$1 must be a whole number of seconds, got '$2'" ;;
esac
# Bounded before the conversion so it cannot overflow.
[ "${#2}" -le 18 ] || fail "$1 must be at most $3 seconds, got '$2'"
local seconds=$((10#$2))
[ "$seconds" -le "$3" ] || fail "$1 must be at most $3 seconds, got '$2'"
echo "$seconds"
}
delay="$(whole_seconds ZERO_NPM_PUBLISH_RETRY_DELAY "${ZERO_NPM_PUBLISH_RETRY_DELAY:-5}" 600)"
wait_timeout="$(whole_seconds ZERO_NPM_PUBLISH_WAIT_TIMEOUT "${ZERO_NPM_PUBLISH_WAIT_TIMEOUT:-1800}" 86400)"
# The wrapper pins every platform version exactly, so a wrapper whose aliases
# do not match the payloads beside it would install aliases that 404. Check
@@ -105,16 +131,49 @@ publish() {
(cd "$dir" && npm publish --access public --ignore-scripts "$@")
}
# Unlike already_published, these lookups may fail without stopping the job:
# while npm has not applied a publish, an error or an E404 both mean "not
# yet", and the deadline bounds the wait.
version_visible() {
[ "$(npm view "${name}@$1" version 2>/dev/null || true)" = "$1" ]
}
latest_is() {
[ "$(npm view "$name" dist-tags.latest 2>/dev/null || true)" = "$1" ]
}
# One deadline covers every wait in this run, so the job cannot wait
# wait_timeout once per version.
deadline=$((SECONDS + wait_timeout))
# wait_until <what we are waiting for> <command...>
wait_until() {
local what="$1"
shift
until "$@"; do
if [ "$SECONDS" -ge "$deadline" ]; then
fail "npm accepted the upload but still does not show ${what} after ${wait_timeout}s. Re-run once it does: versions already on the registry are skipped."
fi
sleep "$delay"
done
}
for dir in "${platform_dirs[@]}"; do
dir="${dir%/}"
publish "$dir" "$(manifest_field "$dir" version)" --tag platform
done
for dir in "${platform_dirs[@]}"; do
platform_version="$(manifest_field "${dir%/}" version)"
wait_until "${name}@${platform_version}" version_visible "$platform_version"
echo "${name}@${platform_version} is on the registry."
done
# Platform versions are semver prereleases of the wrapper version, so a
# platform publish that took `latest` would make a plain install deliver a
# payload with no bin. Stop before the wrapper goes out. A failed lookup is
# tolerated (registry lag, or no `latest` yet on a package's first release):
# failing here would strand the platform payloads without their wrapper.
# tolerated (no `latest` yet on a package's first release): failing here would
# strand the platform payloads without their wrapper.
latest="$(npm view "$name" dist-tags.latest || true)"
case "$latest" in
"${version}-"*)
@@ -125,13 +184,5 @@ echo "dist-tag latest is ${latest:-unset} (unchanged by the platform publishes).
publish "$wrapper_dir" "$version"
# Registry metadata can lag a publish by a few seconds.
for _ in 1 2 3 4 5; do
latest="$(npm view "$name" dist-tags.latest || true)"
if [ "$latest" = "$version" ]; then
echo "dist-tag latest is ${version}."
exit 0
fi
sleep "$delay"
done
fail "dist-tag latest of ${name} is ${latest:-unset}, expected ${version} after the wrapper publish."
wait_until "${name}@${version} as the latest dist-tag" latest_is "$version"
echo "dist-tag latest is ${version}."