edit_file on a UTF-8 file carrying CJK text rewrote bytes it never touched:
continuation bytes of characters far from the edit came back as 84 31 A4 37.
Detect took a nil error from the GB18030 decoder as proof the file was
GB18030. The decoder never fails: it maps every invalid sequence to U+FFFD.
So any file that was not valid UTF-8 -- one stray byte is enough -- was read
as GB18030, and the read-modify-write tools (edit_file, multi_edit,
delete_range, write_file over an existing file), checkpoint restore and the
config migrations then re-encoded every unreadable sequence as the GB18030
form of U+FFFD.
Detect now names a charset only when decoding and re-encoding restores the
input byte for byte. GB18030 is tried first, then GBK, which restores what
GB18030 cannot such as CP936's single-byte euro 0x80, so such a file stays
editable in its own encoding; anything else is LossyUTF8, whose bytes pass
through untouched. The new Kind is appended because checkpoints persist it
by value.
GBK cannot hold every character, so Encode now returns an error instead of
falling back to UTF-8, which rewrote a whole CP936 file as UTF-8 when an edit
added a character such as an emoji. A legacy charset that cannot represent a
character answers an *UnencodableError naming it, its byte offset and the
charset, with ErrUnencodable as its identity; the file tools, checkpoint
restore and the config migrations refuse the write and leave the file as it
was. Fixtures encode through MustEncode.
Readers that hold only part of a stream say so rather than inherit the
whole-file answer:
- DetectFragment runs the round trip on the prefix the decoder consumes when
not at EOF, so a character the cut split is dropped structurally. grep's
8 KiB peek, read_file's 256 KiB sample and the desktop's 2 MiB file preview
use it; the preview's next page starts at the split character, and lossy
text is shown as text rather than as binary.
- DecodeOutput is the one entry point for bytes that are only displayed.
The producer says which ends its bound cut (Cut{Head, Tail}); UTF-8 is read
without a character split at a cut end, and an uncut end keeps its bytes,
since a GBK pair there is output rather than half a rune. What no charset
restores is still read as GB18030. Hook output passes its tail cut when
truncated.
- DetectAndDecode decodes a legacy charset once; the edit tools and
DecodeToUTF8 use it instead of detecting and decoding separately.
Tests that fail before this change (on the base or an earlier revision of it)
and pass here:
- TestE2EEditKeepsBytesOfUTF8FileWithStrayByte: e4 b8 ad became
e4 b8 84 31 a4 37 outside the edit (fails on origin/main-v2).
- TestDetectUTF8WithStrayByteIsNotGB18030 (fails on origin/main-v2).
- TestWritesRefuseCharacterCP936CannotHold (edit_file, multi_edit,
write_file): nil error and the whole CP936 file rewritten as UTF-8.
- TestDecodeOutputTrimsCutUTF8Edges: UTF-8 cut at the front read as mojibake.
- TestDecodeOutputKeepsUncutCodePageEdges, TestDecodeHookOutputKeepsUncutCodePageEdges:
GBK pairs at an uncut edge (bc db, b0 a1 0d 0a, 6f 6b 20 e4 a1,
b0 a1 20 6f 6b) were dropped as partial UTF-8 runes.
- TestEditKeepsCP936FileWithEuroByte: an edit adding Chinese wrote UTF-8
into a CP936 file.
- TestGrepGB18030PeekWithoutNewline, TestGrepGB18030PeekCutMidCharacter:
no matches past a peek cut mid-character.
- TestReadFileGB18030SampleWithoutNewline: garbled text past the sample.
- TestReadFileGB18030TruncatedWithoutNewline,
TestReadFileGB18030TruncatedMidCharacter (desktop): the 2 MiB preview of a
GB18030 file was garbled or ended in U+FFFD.
- TestDecodeHookOutputReadsTruncatedCodePageText,
TestDecodeHookOutputReadsCP936Euro: mojibake for cut or euro-carrying cmd.exe output.
- TestEncodeRefusesCharacterTheCharsetCannotHold, TestDetectGBKWithEuroByte,
TestDetectFragmentDropsTheSplitCharacter, TestDecodeOutputReadsCutCodePageText:
the API's contract.
Problem: doctor reports built-in review skills as referencing a missing
use_capability tool, while capability diagnostics do not check those references.
Root cause: the ordinary doctor passes no runtime tool inventory and relies on
an incomplete name whitelist; its skill discovery also omits configured filters.
Fix: share static host tool identities with runtime names, reuse compile-time
built-ins, and route both diagnostic entrypoints through the same skill discovery
and reference checks. Preserve unknown-name and MCP dependency warnings while
marking unresolved dynamic references as unverified. Reuse existing MCP parsing
semantics and observed host bindings without starting servers during static checks.
Verification: full root Go tests, affected diagnostic package race tests,
affected-package go vet, repolint, and isolated CLI checks passed. CLI checks
observed zero provider requests and zero MCP starts. English and Chinese
diagnostic documentation updated. Tool names, schemas, and permissions unchanged.
* refactor(go): adopt modern stdlib idioms across kernel and desktop
Mechanical sweep with golangci-lint's modernize/intrange/copyloopvar/
usestdlibvars fixers plus hand-fixes for what they could not rewrite:
slices.Contains, min/max, slices.Backward, WaitGroup.Go, reflect.TypeFor,
maps.Copy, strings.Cut/CutPrefix/SplitSeq, strings.Builder, range-over-int.
Net -549 lines with no behavior change.
Three loops in session.go stay index-based: slices.Backward yields element
copies, so the fixer's rewrite silently dropped writes to DecisionReceipts
and ToolCalls (caught by TestDelivery* and TestDependentSameBatch*). All 49
slices.Backward sites were audited for that shape; only these three write
through the index, and they carry a nolint with the reason.
CaptureAfter's scan loop had an empty body and never ran any logic, so the
fixer could not express it. Drop it and the accessor it was the only caller
of. omitzero is deliberately untouched: turning omitempty into omitzero
changes the JSON written for nested structs, which is a wire-format change
to session files and the extension protocol.
* build(lint): gate the modern-idiom linters
Nothing stopped the codebase from drifting back to pre-1.22 idioms. Both
modules are now clean for modernize, intrange, copyloopvar, and
usestdlibvars, so enable them rather than leaving the sweep as a one-off.
omitzero stays disabled with the reason inline; usetesting, perfsprint, and
errorlint still have residue and land separately.
312 banner separators across 63 files, the thing REASONIX.md bans outright and
nothing has ever checked. 64 were pure rules with no content and are gone; the
other 248 carried a label inside the dashes, so the decoration is stripped and
the label kept -- `// --- session/resume ---` becomes `// session/resume`, which
still orients a reader in a 1000-line test file without drawing a line across it.
The repolint ceiling for this rule drops 312 -> 0, so it is now zero-tolerance
rather than merely frozen.
Add an on-demand builtin skill and a shared read-only diagnostics model for
Skills, Commands, Hooks, plugin packages, MCP, and instruction docs. CLI
defaults to static probes; --live uses an isolated Host. Desktop Settings
exposes the same report without starting MCP or mutating config.
Lets the model invoke a project slash command (a custom command or a skill) by
name, the way a user types "/name". A slash command is a prompt template, so the
tool returns the expanded prompt text for the model to act on in the same turn —
it does not run anything on its own. Calling with an empty/"list" command returns
the available commands.
- internal/command: slash_command tool over a uniform SlashEntry view (Name,
Description, ArgHint, Render). Read-only; tolerates a leading slash; a later
entry wins a name clash. command does not import skill (no cycle).
- boot: adapt loaded skills then custom commands into SlashEntry (commands win
clashes, matching the prompt's precedence) and register the tool after the
command load.
Tests: expand-with-args, leading-slash resolution, list (empty/"list"/"LIST"),
unknown-command error, empty registry, name-clash precedence, basics. Full
go test ./... green (29 pkgs); gofmt clean. Live (DeepSeek): the model calls
slash_command{command:"haiku",arguments:"autumn rain"} and produces the haiku
from the expanded template.
Replace three copy-pasted splitFrontmatter implementations (command, skill, memory) with a single internal/frontmatter.Split. The shared parser adopts the most complete variant (the memory one): CRLF normalization and skipping empty-value section headers. This aligns command/skill onto that behavior — a net improvement (CRLF robustness) with no consumer impact (callers read values, where empty == absent). Local thin wrappers kept for test compatibility; 11 tests cover Split's edge cases. Thanks @HUQIANTAO.
Skills (internal/skill): discover SKILL.md / <name>.md across .reasonix /
.agents / .agent / .claude (project + home) plus custom roots; symlink-aware;
fold a names+descriptions index into the cache-stable prompt (bodies load on
demand). Tools: run_skill, install_skill, and dedicated explore / research /
review / security_review subagent wrappers reusing an extracted
agent.RunSubAgent / FilterRegistry. Built-ins: explore, research, review,
security-review, test, init.
Hooks (internal/hook): PreToolUse / PostToolUse / UserPromptSubmit / Stop from
settings.json — project (.reasonix/settings.json, runs only after `/hooks
trust`) and global (~/.reasonix/settings.json, always). exit 2 blocks on gating
events; output capped; per-event timeouts. Fired in agent.executeOne and at the
controller turn boundary; trust stored in ~/.reasonix/trust.json.
Slash: each skill appears as /<name>; /skill (list/show/new/paths) and /hooks
(list/trust) management commands; fix /mcp list. Custom-command discovery now
follows symlinks too.
Conventions: config.ConventionDirs is the single source for skills + commands
(CommandDirs scans the four convention dirs so commands authored for other
tools migrate in). Memory docs discover REASONIX.md / AGENTS.md / CLAUDE.md
(+ .local); new docs default to AGENTS.md, preferring an existing doc so notes
don't split.
CLI: `reasonix setup` is the config wizard (renamed from `init`). Project memory
is model-generated in-session — `/init` is a built-in inline skill that has the
model analyze the codebase and write AGENTS.md; `reasonix init` prints a pointer
to it. usage / README / SPEC / auth-error updated.
Bring in the finished Go rewrite (developed under the placeholder name
"duo") as the v2 codebase. Code-only import — no upstream git history.
Config- and plugin-driven coding agent: registry core, OpenAI-compatible
providers, builtin tools, MCP (stdio + HTTP) plugins, permission + sandbox,
custom slash commands, @-references, two-model collaboration.
Renamed throughout: duo→reasonix, Duo→Reasonix, DUO→REASONIX — module path,
imports, CLI name, cmd/ dirs, reasonix.toml, .reasonix/commands, and
$REASONIX_* env prefixes. Kept the v2 MIT LICENSE (the Go project had none).
Verified: go build ./... · go vet ./... · go test ./... all green; gofmt clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
esengine