mirror of
https://github.com/esengine/DeepSeek-Reasonix.git
synced 2026-11-07 10:45:59 -10:00
feat(proc): expose tracked root wait completion
Problem: A foreground command can return bounded cancellation while cmd.Wait is still blocked, so call return cannot establish resource-lifetime completion. Root cause: The tracked wait owner exposes its logical result and cleanup diagnostics but no separate observation of the still-running root Wait. Fix: Register a stable root-wait channel before the waiter starts and close it after Wait returns, before result publication. Expose it with the explicit RootWaitCompleted name. Preserve existing execution, cancellation, retry and error behavior. No production lease consumer or whole-scope claim is added. Verification: Deterministic cancellation/retry/publication tests, portable Go child execution and a real blocked-output process/lease regression pass. Root Wait remains distinct from descendant or external-scope completion; nil means no registered observation. Full local/static and platform evidence is recorded in the PR without inheriting previous heads' native results.
This commit is contained in:
1 parent
540c14998b
commit
5d484c4fe2
6 files changed
+475
-6
No files matched your search
@@ -0,0 +1,118 @@
|
||||
---
|
||||
owner: @SivanCola
|
||||
backup: @esengine
|
||||
status: active
|
||||
reviewed: 2026-10-25
|
||||
---
|
||||
|
||||
# Root wait observation / 根进程等待观察
|
||||
|
||||
## Cause and scope / 根因与范围
|
||||
|
||||
Tracked foreground execution deliberately returns after a cancellation grace
|
||||
period even when `cmd.Wait` has not returned. That bounded result is useful for
|
||||
responsive cancellation; it cannot also prove physical completion.
|
||||
|
||||
`TrackedCommand.RootWaitCompleted` exposes only the registered root wait. Its
|
||||
channel closes after the wait function returns and before its result is sent.
|
||||
Cancellation and cleanup-retry expiry do not close it. No production lease,
|
||||
tool, hook or resource consumer uses the new method in this unit.
|
||||
|
||||
前台取消采用有界返回以保持响应,但返回时 `cmd.Wait` 仍可能未完成。本次保留
|
||||
这一行为,新增明确命名的根等待完成信号,不把调用返回等同于物理执行结束。
|
||||
|
||||
## Source, sink and authority / 输入、副作用与权威
|
||||
|
||||
- The existing tracked-command owner registers a channel before starting its
|
||||
wait goroutine. Access to channel identity uses its existing mutex.
|
||||
- The goroutine closes the channel only after `wait()` returns. It closes
|
||||
before publishing the buffered result, so receiving the result cannot race
|
||||
ahead of notification. A blocked result receiver cannot delay notification.
|
||||
- The buffered result remains available to the existing foreground or retry
|
||||
consumer. The signal adds no goroutine, timer, command, path or OS handle.
|
||||
- Root completion preserves success, exit errors and cancellation error
|
||||
identities. Existing launch, tracking, kill and retry policies stay unchanged.
|
||||
- Nil handles and handles without a registered wait return nil. Nil is not
|
||||
completion evidence or permission to release a resource.
|
||||
|
||||
信号身份沿现有 mutex 发布,在等待返回后、结果发送前关闭。既有取消、重试、
|
||||
错误分类和进程操作保持原样,没有新命令、路径、句柄或后台 goroutine。
|
||||
|
||||
## Meaning of each boundary / 各边界的含义
|
||||
|
||||
| Observation | What it proves | What it does not prove |
|
||||
| --- | --- | --- |
|
||||
| Foreground cancellation returns | Grace policy returned control | Root wait or descendants ended |
|
||||
| Retry window expires | Automatic retry budget ended | Root wait completed |
|
||||
| RootWaitCompleted closes | Registered root `cmd.Wait` returned | Descendants or external scope ended |
|
||||
| Method returns nil | No wait was registered on this handle | Command never ran or all effects ended |
|
||||
| ProcessState exists | os/exec observed a root process result | Output copies or descendants ended |
|
||||
|
||||
A root wait may return an error and still be completed. Conversely a failed
|
||||
launch can return a tracked handle without registering a wait. In particular,
|
||||
Windows launch preparation may start and reap a suspended child before returning
|
||||
an error. The nil signal makes no stronger claim about that path.
|
||||
|
||||
完成可以伴随错误;失败启动也可能产生未登记等待的句柄。Windows 的启动准备
|
||||
可能先创建再回收挂起子进程,因此 nil 不能被解释为“肯定未执行”。
|
||||
|
||||
## Verification / 验证
|
||||
|
||||
- Deterministic synthetic-time tests keep Wait blocked through cancellation,
|
||||
grace expiry and retry expiry, then release it and observe the same channel.
|
||||
- An unbuffered receiver control proves completion notification precedes result
|
||||
publication without depending on goroutine scheduling speed.
|
||||
- Actual portable Go child commands cover success, nonzero exit, missing
|
||||
executable, cancellation before launch and unchanged untracked behavior.
|
||||
- An actual child writes into a deliberately blocked output sink. Bounded
|
||||
cancellation returns while `cmd.Wait` remains pending. A test-only consumer
|
||||
binds the signal to an existing mutation lease and ends it logically.
|
||||
- Conflicting admission remains busy, disjoint admission proceeds, and releasing
|
||||
the output sink lets the lease close. Repeating old cleanup does not release
|
||||
the next transaction. All fixture processes and waits are joined.
|
||||
|
||||
真实子进程测试固定阻塞输出处理,因此不靠时间快慢猜测 Wait 是否完成。将信号
|
||||
接到租约的调用仅存在于回归测试,不代表运行时已启用写事务。
|
||||
|
||||
## Compatibility / 兼容性
|
||||
|
||||
| Contract | Before | After |
|
||||
| --- | --- | --- |
|
||||
| Tracked foreground return/error | Existing bounded cancellation | Unchanged |
|
||||
| Untracked RunCommand | Nil tracked handle | Unchanged |
|
||||
| Root wait observation method | Absent | Additive, local-only method |
|
||||
| Diagnostics/wire/session records | Existing fields | Unchanged |
|
||||
| OS tracking and sandbox policy | Existing backend behavior | Unchanged |
|
||||
|
||||
There is no new persisted field, migration, model-visible schema or prompt
|
||||
prefix. This method has no preceding API implementation to compare. Deliberately
|
||||
weakened completion ordering or cancellation closure controls validate the new
|
||||
contract; they are not claims that an older API already promised this signal.
|
||||
|
||||
本次没有持久化或 wire 变更。新增接口此前不存在,故意削弱信号规则的对照用于
|
||||
验证新测试,不能冒称旧版已经承诺这个接口却违反它。
|
||||
|
||||
## Limits and next integration / 边界与后续接入
|
||||
|
||||
Do not use this signal alone to release a lease for an unresolved shell or hook.
|
||||
The command may spawn descendants, detach, delegate work to a service or return
|
||||
`exec.ErrWaitDelay`. Root Wait is not a process-scope termination protocol.
|
||||
|
||||
Full write-lifetime integration must separately establish the backend's supported
|
||||
scope, distinguish unknown termination, retain ownership when completion cannot
|
||||
be proved, and provide attributable diagnostics and recovery. Closing every
|
||||
channel is not a valid substitute for proving completion.
|
||||
|
||||
Until those boundaries are implemented, this unit does not activate shared
|
||||
transactions, settle unknown scopes, recover process crashes or guarantee every
|
||||
descendant is gone. Existing cancellation cleanup remains best-effort where the
|
||||
backend is best-effort. Native platform evidence is recorded separately.
|
||||
|
||||
不能仅凭根等待完成就释放未知 shell/hook 写入租约。后代、脱离进程、服务委托和
|
||||
输出超时都有独立边界。无法证明范围终止时必须保留占用并给出诊断及恢复路径,
|
||||
不能为了关闭 channel 而伪造完成。本次不启用共享事务,也不完成未知范围恢复。
|
||||
|
||||
This is the author's security assessment, not independent review or disposition
|
||||
of the original large PR's CodeQL findings.
|
||||
|
||||
本说明为作者评估,不替代独立安全审查或原大 PR 的告警逐项判断。
|
||||
@@ -0,0 +1,18 @@
|
||||
package proc
|
||||
|
||||
func publishRootWait(done chan struct{}, results chan<- error, err error) {
|
||||
close(done)
|
||||
results <- err
|
||||
}
|
||||
|
||||
// RootWaitCompleted closes after the registered cmd.Wait returns, before its
|
||||
// result is received. Nil means no registered wait, not completed execution.
|
||||
// It does not prove that descendants or an external execution scope have ended.
|
||||
func (p *TrackedCommand) RootWaitCompleted() <-chan struct{} {
|
||||
if p == nil {
|
||||
return nil
|
||||
}
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
return p.rootWait
|
||||
}
|
||||
@@ -0,0 +1,194 @@
|
||||
package proc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"os"
|
||||
"os/exec"
|
||||
"sync"
|
||||
"testing"
|
||||
"testing/synctest"
|
||||
"time"
|
||||
)
|
||||
|
||||
func assertRootWaitClosed(t *testing.T, done <-chan struct{}) {
|
||||
t.Helper()
|
||||
if done == nil {
|
||||
t.Fatal("registered root wait has no completion signal")
|
||||
}
|
||||
select {
|
||||
case <-done:
|
||||
default:
|
||||
t.Fatal("root wait completed without closing its signal")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRootWaitCompletionPrecedesResultAndRetainsWaitError(t *testing.T) {
|
||||
for _, errWait := range []error{nil, errors.New("root wait failed")} {
|
||||
tracked := &TrackedCommand{}
|
||||
err := waitForTrackedCommand(t.Context(), tracked, func() error { return errWait }, time.Second, time.Second, time.Second)
|
||||
if !errors.Is(err, errWait) {
|
||||
t.Fatalf("wait error changed: %v", err)
|
||||
}
|
||||
done := tracked.RootWaitCompleted()
|
||||
assertRootWaitClosed(t, done)
|
||||
if tracked.RootWaitCompleted() != done {
|
||||
t.Fatal("completion identity changed")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRootWaitPublicationDoesNotDependOnReceiver(t *testing.T) {
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
done, results := make(chan struct{}), make(chan error)
|
||||
expected := errors.New("root wait result")
|
||||
go publishRootWait(done, results, expected)
|
||||
defer func() {
|
||||
if err := <-results; !errors.Is(err, expected) {
|
||||
t.Fatalf("result changed: %v", err)
|
||||
}
|
||||
}()
|
||||
synctest.Wait()
|
||||
assertRootWaitClosed(t, done)
|
||||
})
|
||||
}
|
||||
|
||||
func TestRootWaitCompletionSurvivesCancellationAndRetryExpiry(t *testing.T) {
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(t.Context())
|
||||
defer cancel()
|
||||
tracked := &TrackedCommand{}
|
||||
started, unblock := make(chan struct{}), make(chan struct{})
|
||||
release := sync.OnceFunc(func() { close(unblock) })
|
||||
defer release()
|
||||
returned := make(chan error, 1)
|
||||
go func() {
|
||||
returned <- waitForTrackedCommand(ctx, tracked, func() error {
|
||||
close(started)
|
||||
<-unblock
|
||||
return nil
|
||||
}, time.Second, time.Second, 3*time.Second)
|
||||
}()
|
||||
<-started
|
||||
done := tracked.RootWaitCompleted()
|
||||
if done == nil {
|
||||
t.Fatal("wait signal was not published before entering Wait")
|
||||
}
|
||||
cancel()
|
||||
if err := <-returned; !errors.Is(err, context.Canceled) {
|
||||
t.Fatalf("logical cancellation result changed: %v", err)
|
||||
}
|
||||
synctest.Wait()
|
||||
select {
|
||||
case <-done:
|
||||
t.Fatal("bounded cancellation was mistaken for root completion")
|
||||
default:
|
||||
}
|
||||
time.Sleep(4 * time.Second)
|
||||
synctest.Wait()
|
||||
select {
|
||||
case <-done:
|
||||
t.Fatal("retry expiry was mistaken for root completion")
|
||||
default:
|
||||
}
|
||||
release()
|
||||
synctest.Wait()
|
||||
assertRootWaitClosed(t, done)
|
||||
if done != tracked.RootWaitCompleted() {
|
||||
t.Fatal("late completion replaced the registered signal")
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestRootWaitCompletionDoesNotChangeCancelledWaitErrors(t *testing.T) {
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(t.Context())
|
||||
defer cancel()
|
||||
tracked := &TrackedCommand{}
|
||||
entered, release := make(chan struct{}), make(chan struct{})
|
||||
errWait := errors.New("root exited after cancellation")
|
||||
returned := make(chan error, 1)
|
||||
go func() {
|
||||
returned <- waitForTrackedCommand(ctx, tracked, func() error {
|
||||
close(entered)
|
||||
<-release
|
||||
return errWait
|
||||
}, time.Hour, time.Hour, time.Hour)
|
||||
}()
|
||||
<-entered
|
||||
cancel()
|
||||
synctest.Wait()
|
||||
close(release)
|
||||
if err := <-returned; !errors.Is(err, context.Canceled) || !errors.Is(err, errWait) {
|
||||
t.Fatalf("cancel/wait identities changed: %v", err)
|
||||
}
|
||||
assertRootWaitClosed(t, tracked.RootWaitCompleted())
|
||||
})
|
||||
}
|
||||
|
||||
func TestRootWaitUnregisteredIsNotCompleted(t *testing.T) {
|
||||
for _, tracked := range []*TrackedCommand{nil, {}} {
|
||||
if done := tracked.RootWaitCompleted(); done != nil {
|
||||
t.Fatal("unregistered wait fabricated completion")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRootWaitCommandHelper(t *testing.T) {
|
||||
if mode := os.Getenv("REASONIX_ROOT_WAIT_HELPER"); mode != "" {
|
||||
if mode == "error" {
|
||||
os.Exit(7)
|
||||
}
|
||||
os.Exit(0)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRootWaitCompletionOnRealCommands(t *testing.T) {
|
||||
for _, mode := range []string{"success", "error"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
cmd := exec.CommandContext(t.Context(), os.Args[0], "-test.run=^TestRootWaitCommandHelper$")
|
||||
cmd.Env = append(os.Environ(), "REASONIX_ROOT_WAIT_HELPER="+mode)
|
||||
tracked, err := RunCommand(t.Context(), cmd, RunOptions{Track: true})
|
||||
if mode == "success" && err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if mode == "error" {
|
||||
var exited *exec.ExitError
|
||||
if !errors.As(err, &exited) || exited.ExitCode() != 7 {
|
||||
t.Fatalf("exit error changed: %v", err)
|
||||
}
|
||||
}
|
||||
if cmd.ProcessState == nil {
|
||||
t.Fatal("real command was not reaped")
|
||||
}
|
||||
assertRootWaitClosed(t, tracked.RootWaitCompleted())
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRootWaitDoesNotRegisterBeforeStartOrForUntrackedCommands(t *testing.T) {
|
||||
for _, scenario := range []string{"cancelled_before_start", "missing_executable", "untracked"} {
|
||||
t.Run(scenario, func(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(t.Context())
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, os.Args[0], "-test.run=^TestRootWaitCommandHelper$")
|
||||
cmd.Env = append(os.Environ(), "REASONIX_ROOT_WAIT_HELPER=success")
|
||||
if scenario == "cancelled_before_start" {
|
||||
cancel()
|
||||
}
|
||||
if scenario == "missing_executable" {
|
||||
cmd.Path = t.TempDir() + string(os.PathSeparator) + "missing"
|
||||
}
|
||||
tracked, err := RunCommand(ctx, cmd, RunOptions{Track: scenario != "untracked"})
|
||||
if scenario != "untracked" && err == nil {
|
||||
t.Fatal("unexpected successful launch")
|
||||
}
|
||||
if scenario == "untracked" && (tracked != nil || err != nil) {
|
||||
t.Fatalf("untracked contract changed: %v", err)
|
||||
}
|
||||
if tracked.RootWaitCompleted() != nil {
|
||||
t.Fatal("unregistered wait has completion proof")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -46,11 +46,12 @@ type RunDiagnostics struct {
|
||||
type TrackedCommand struct {
|
||||
cmd *exec.Cmd
|
||||
|
||||
mu sync.Mutex
|
||||
job *TrackedJob
|
||||
tree *TreeTracker
|
||||
killed bool
|
||||
diag RunDiagnostics
|
||||
mu sync.Mutex
|
||||
job *TrackedJob
|
||||
tree *TreeTracker
|
||||
killed bool
|
||||
diag RunDiagnostics
|
||||
rootWait <-chan struct{}
|
||||
}
|
||||
|
||||
// RunCommand starts cmd and waits for it. When tracking is enabled, cancellation
|
||||
@@ -138,7 +139,13 @@ func normalizeRunOptions(opts RunOptions) RunOptions {
|
||||
|
||||
func waitForTrackedCommand(ctx context.Context, tracked *TrackedCommand, wait func() error, grace, retryEvery, retryFor time.Duration) error {
|
||||
waitCh := make(chan error, 1)
|
||||
go func() { waitCh <- wait() }()
|
||||
done := make(chan struct{})
|
||||
tracked.mu.Lock()
|
||||
tracked.rootWait = done
|
||||
tracked.mu.Unlock()
|
||||
go func() {
|
||||
publishRootWait(done, waitCh, wait())
|
||||
}()
|
||||
|
||||
select {
|
||||
case err := <-waitCh:
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
package mutation
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"reasonix/internal/base/proc"
|
||||
)
|
||||
|
||||
func TestMutationRootWaitHelper(t *testing.T) {
|
||||
if os.Getenv("REASONIX_MUTATION_ROOT_WAIT_HELPER") == "1" {
|
||||
fmt.Print("root-output")
|
||||
os.Exit(0)
|
||||
}
|
||||
}
|
||||
|
||||
type rootWaitOutput struct {
|
||||
once sync.Once
|
||||
entered chan struct{}
|
||||
release chan struct{}
|
||||
}
|
||||
|
||||
func (w *rootWaitOutput) Write(p []byte) (int, error) {
|
||||
w.once.Do(func() { close(w.entered) })
|
||||
<-w.release
|
||||
return len(p), nil
|
||||
}
|
||||
|
||||
func TestRootWaitRetainsLeaseAfterRealBoundedCancellation(t *testing.T) {
|
||||
root, state := t.TempDir(), t.TempDir()
|
||||
released := make(chan struct{}, 1)
|
||||
c, err := New(root, state, func(n Notice) {
|
||||
if n.State == "released" && n.CallID == "root-wait" {
|
||||
released <- struct{}{}
|
||||
}
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
request := intent("held")
|
||||
request.CallID = "root-wait"
|
||||
lease, err := c.Acquire(t.Context(), request)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer lease.End(NotExecuted)
|
||||
ctx, cancel := context.WithTimeout(t.Context(), 20*time.Second)
|
||||
defer cancel()
|
||||
output := &rootWaitOutput{entered: make(chan struct{}), release: make(chan struct{})}
|
||||
unblock := sync.OnceFunc(func() { close(output.release) })
|
||||
cmd := exec.CommandContext(ctx, os.Args[0], "-test.run=^TestMutationRootWaitHelper$")
|
||||
cmd.Env = append(os.Environ(), "REASONIX_MUTATION_ROOT_WAIT_HELPER=1")
|
||||
cmd.Stdout = output
|
||||
type result struct {
|
||||
tracked *proc.TrackedCommand
|
||||
err error
|
||||
}
|
||||
returned, joined := make(chan result, 1), make(chan struct{})
|
||||
go func() {
|
||||
defer close(joined)
|
||||
tracked, err := proc.RunCommand(ctx, cmd, proc.RunOptions{Track: true,
|
||||
CancelWaitGrace: time.Millisecond, CancelRetryInterval: time.Millisecond, CancelRetryFor: time.Millisecond})
|
||||
returned <- result{tracked, err}
|
||||
}()
|
||||
t.Cleanup(func() { unblock(); cancel(); <-joined })
|
||||
select {
|
||||
case <-output.entered:
|
||||
case <-ctx.Done():
|
||||
t.Fatal(ctx.Err())
|
||||
}
|
||||
cancel()
|
||||
res := <-returned
|
||||
if !errors.Is(res.err, context.Canceled) {
|
||||
t.Fatalf("bounded cancellation result: %v", res.err)
|
||||
}
|
||||
done := res.tracked.RootWaitCompleted()
|
||||
if done == nil {
|
||||
t.Fatal("real tracked wait was not registered")
|
||||
}
|
||||
t.Cleanup(func() { unblock(); <-done })
|
||||
select {
|
||||
case <-done:
|
||||
t.Fatal("blocked output copy was not a pending cmd.Wait")
|
||||
default:
|
||||
}
|
||||
if err := lease.HoldUntil(done); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
lease.End(Unknown)
|
||||
other := testCoordinator(t, root, state)
|
||||
if next, err := other.TryAcquire(t.Context(), request); !errors.Is(err, ErrBusy) {
|
||||
if next != nil {
|
||||
next.End(NotExecuted)
|
||||
}
|
||||
t.Fatalf("logical return released physical ownership: %v", err)
|
||||
}
|
||||
disjoint, err := other.TryAcquire(t.Context(), intent("disjoint"))
|
||||
if err != nil {
|
||||
t.Fatalf("unrelated file blocked: %v", err)
|
||||
}
|
||||
disjoint.End(NotExecuted)
|
||||
unblock()
|
||||
select {
|
||||
case <-released:
|
||||
case <-time.After(10 * time.Second):
|
||||
t.Fatal("completed root wait did not release the lease")
|
||||
}
|
||||
next, err := other.TryAcquire(t.Context(), request)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer next.End(NotExecuted)
|
||||
lease.End(NotExecuted)
|
||||
if probe, err := c.TryAcquire(t.Context(), request); !errors.Is(err, ErrBusy) {
|
||||
if probe != nil {
|
||||
probe.End(NotExecuted)
|
||||
}
|
||||
t.Fatalf("old cleanup released a new transaction: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
## 变更
|
||||
|
||||
- 为受跟踪命令增加根进程 Wait 完成信号,明确区分取消返回与等待真正完成;
|
||||
不改变取消策略,不把该信号当成整个进程范围结束的证明。Refs #14025
|
||||
- Add a root-wait completion observation for tracked commands, separate from
|
||||
bounded cancellation return; it does not prove descendant completion. Refs #14025
|
||||
Reference in new issue
Block a user