Files
zero/internal/cli/usage.go
9c2e14272b CLI: doctor config validation, autonomy ceiling, changes --base, usage report (#142)
* config: add ValidateFile for structured config diagnostics

* doctor: add offsetToLineCol helper for JSON error positions

* doctor: validate config files in config.validation check

* cli: thread config paths into doctor and degrade on bad JSON

* doctor: inline read, single-parse config validation + utf8 col test

- Remove the osReadFile wrapper; call os.ReadFile directly at its one call site
- Add config.ValidateBytes([]byte) so each config file is read and JSON-parsed
  once in configValidationCheck instead of twice (once for position, once via
  ValidateFile); ValidateFile now reads then delegates to its own unmarshal for
  the path-prefixed error message, keeping its public signature unchanged
- Fix the misleading skip comment in configValidationCheck: configFilesCheck
  only checks path-string presence; the skip is a defensive guard for the
  unreachable case where DefaultResolveOptions passed a non-empty path that
  still fails to read
- Add utf8 multibyte case to TestOffsetToLineCol documenting byte-column
  semantics (column counts bytes, not runes)

* sandbox: add Policy.MaxAutonomy ceiling defaulting to high

* sandbox: normalize both operands in autonomyAllowed to fail closed

* sandbox: clamp grant-allow and unsafe escalation to policy autonomy ceiling

* sandbox: add RequiredAutonomyForBatch helper with fail-closed mapping

* config: add sandbox.maxAutonomy field to file and resolved config

* config: merge sandbox.maxAutonomy across layers, env, and overrides

* cli: thread configured autonomy ceiling into sandbox engine construction

* cli: surface max_autonomy in sandbox policy formatters

* zerocommands: carry max autonomy in sandbox policy snapshot

* sandbox: default empty policy ceiling to high + reason const

Evaluate now treats an empty Policy.MaxAutonomy (from a directly-constructed
Policy that bypasses DefaultPolicy) as High instead of letting it normalize to
Low, which would silently clamp every Medium/High decision to Prompt. The
ceiling is a no-op unless explicitly configured.

Also extracts the "above policy ceiling" clamp reason into a package const
referenced at both engine clamp sites and the related tests, and adds a
non-vacuous engine test proving the empty-ceiling trap is closed.

* zerogit: add base-ref three-dot diff to changes inspect

* zerogit: cover base-ref backward-compat and real-git diff

* zerogit: propagate base ref through change snapshot

* cli: parse changes --base ref and reject on commit

* cli: thread base ref into changes inspect output and help

* changes: redact base in cli summary + guard --base= + rename test

- redactChangeSummary now redacts the Base field alongside Root/Branch/Commit/etc.
- --base= equals-form rejects leading-dash values via flagValueLooksLikeOption, closing the option-smuggling gap
- TestParseNameStatusRenameAndCopy asserts rename/copy three-field lines use the new/destination path with correct status, and confirms two-field modify is unaffected

* Add ParseDiffStat helper for git diff --stat summaries

* Add usage report aggregation over persisted session usage events

* Add zero usage report command over persisted session usage

* Wire usage command into CLI dispatch and help

* usage: validate --since format, fix redaction comments, add tests

Fix 1 (BLOCKING): parseUsageArgs now validates --since with time.Parse
against YYYY-MM-DD; returns exitUsage + "invalid --since %q: expected
YYYY-MM-DD" for bare strings like "foo", unpadded "2026-6-1", or
wrong-separator "06/01/2026". Valid dates are stored unchanged for the
existing lexical comparison.

Fix 2: Replace false "pre-redaction stat" claims in diffstat.go and
usage.go with accurate comments: the --stat summary line carries no
secret-bearing tokens so parsing the already-redacted DiffStat is safe.

Fix 3: Add three tests to usage_test.go —
  - TestRunUsageDaysFilter: stubs deps.now, seeds events on two dates,
    asserts --days 3 includes the recent date and excludes the old one.
  - TestRunUsageInvalidSince: table-driven; verifies "foo", "2026-6-1",
    "06/01/2026" each return exitUsage + validation message, while
    "2026-06-01" returns exitSuccess.
  - TestRunUsageEmptyStore: zero usage events → exitSuccess, output
    contains header and "total" row, no panic.

* doctor: surface type-mismatch offsets and drop colliding flat line/col keys

Probe config validation against config.FileConfig instead of a bare any so
a structurally-valid document with a wrong field type (e.g. maxTurns as a
string) surfaces a *json.UnmarshalTypeError carrying the offset, instead of
losing line/col. Remove the flat top-level details["line"]/["col"] keys
that were overwritten by the last malformed file when several were present;
keep only the unambiguous per-path map entry. Tighten the existing malformed
test to assert concrete per-path line/col and add a type-mismatch test that
exercises the previously-dead UnmarshalTypeError branch.

* sandbox: fail closed on invalid configured autonomy ceiling

An invalid non-empty sandbox.maxAutonomy (e.g. "moderate") previously
survived Resolve (only trimmed, never validated), failed to normalize at the
sandbox bridge, and left the policy unchanged at the default High ceiling.
A typo therefore silently disabled the admin's intended ceiling: fail-open on
a security boundary.

Validate at resolve time (fail loud): Resolve now rejects a non-empty
maxAutonomy that does not normalize, returning a clear error. As defense in
depth, applyConfiguredAutonomyCeiling now clamps an unrecognised non-empty
value to the most restrictive ceiling (low) instead of returning the policy
unchanged, so a bad value can never widen the ceiling even via direct use.
Empty/unset stays valid and keeps the default High ceiling.

Add resolver tests for invalid-fails / valid-resolves and bridge tests for
the fail-closed clamp and valid mappings.

* sandbox: fail closed on malformed autonomy and surface policy resolve errors

Treat a genuinely-invalid request autonomy as the highest tier in the engine
so it exceeds a Medium/Low ceiling and clamps to Prompt instead of being
sanitized to Low and auto-allowing on the grant/unsafe path. Surface
resolveConfig failures in the policy command instead of silently falling back
to the permissive DefaultPolicy, which would misreport trust posture.

* usage: bucket and filter by UTC calendar day, reject empty --session

Normalize RFC3339 timestamps to their UTC calendar date before day bucketing
(report) and before the --since/--days cutoff comparison (CLI) so offset
timestamps land on the correct UTC day and the bucket and cutoff agree;
malformed timestamps fall back to the leading-10 slice. Reject empty
--session/--session-id values with a value-required error matching the --since
validation style.

* doctor: fail on unreadable config files and stop fabricating positions

Surface a present-but-unreadable config path (permissions, is-a-directory) as
a failing per-path validation detail instead of silently skipping it; a
genuinely-missing path stays a skip. Return ok=false for a JSON parse error
that carries no offset so it routes through the no-position ValidateBytes path
rather than fabricating a (1,1) line/col. Assert empty stderr on the JSON
doctor failure path.

* config: make sandbox maxAutonomy resolver tests hermetic

Pass an explicit empty Env to the maxAutonomy Resolve tests so host
environment variables cannot leak in through the nil-Env os.Getenv fallback.

* sandbox: preserve raw autonomy for ceiling check so invalid clamps under default ceiling

Defaulting invalid request.Autonomy to High only clamped under a Medium/Low
ceiling; under the default High ceiling autonomyAllowed(High, High) returned true
and still auto-allowed (fail-open). Keep the raw requested value for the ceiling
check so autonomyAllowed's unknown-tier guard fails it closed under ANY ceiling.
Adds default-High-ceiling regression tests for the unsafe and grant-allow paths.

---------

Co-authored-by: KRATOS <[email protected]>
Co-authored-by: Vasanthdev2004 <[email protected]>
2026-06-08 17:15:57 +05:30

316 lines
10 KiB
Go

package cli
import (
"context"
"fmt"
"io"
"strings"
"time"
"github.com/Gitlawb/zero/internal/modelregistry"
"github.com/Gitlawb/zero/internal/sessions"
"github.com/Gitlawb/zero/internal/usage"
"github.com/Gitlawb/zero/internal/zerogit"
)
type usageOptions struct {
json bool
days int
since string
sessionID string
}
// usageEventSet keeps the persisted events paired with the session metadata they
// were read alongside, so cost reconstruction can resolve each event's owning
// model id during aggregation.
type usageEventSet struct {
events []sessions.Event
meta []sessions.Metadata
}
// collectUsageData reads every persisted session's events (optionally limited to
// a single session id) and returns them flattened alongside the matching session
// metadata. It mirrors runSearch's persisted-event traversal over the injected
// session store.
func collectUsageData(store *sessions.Store, sessionFilter string) (usageEventSet, error) {
metadata, err := store.List()
if err != nil {
return usageEventSet{}, err
}
set := usageEventSet{}
for _, meta := range metadata {
if sessionFilter != "" && meta.SessionID != sessionFilter {
continue
}
set.meta = append(set.meta, meta)
sessionEvents, err := store.ReadEvents(meta.SessionID)
if err != nil {
return usageEventSet{}, err
}
set.events = append(set.events, sessionEvents...)
}
return set, nil
}
// filterEventsSince drops events whose UTC calendar date precedes the inclusive
// lower bound. An empty since returns the events unchanged.
func filterEventsSince(events []sessions.Event, since string) []sessions.Event {
if since == "" {
return events
}
filtered := make([]sessions.Event, 0, len(events))
for _, event := range events {
if eventUTCDate(event.CreatedAt) >= since {
filtered = append(filtered, event)
}
}
return filtered
}
// eventUTCDate maps an RFC3339 timestamp to its UTC calendar date (YYYY-MM-DD)
// so the --since/--days cutoff compares against the same UTC day the report
// buckets under. On a parse failure it falls back to the leading-10 slice.
func eventUTCDate(createdAt string) string {
if parsed, err := time.Parse(time.RFC3339, createdAt); err == nil {
return parsed.UTC().Format("2006-01-02")
}
if len(createdAt) >= 10 {
return createdAt[:10]
}
return createdAt
}
func runUsage(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) int {
subcommand := "report"
if len(args) > 0 && !strings.HasPrefix(args[0], "-") {
subcommand = strings.ToLower(strings.TrimSpace(args[0]))
args = args[1:]
}
if subcommand == "help" {
if err := writeUsageHelp(stdout); err != nil {
return exitCrash
}
return exitSuccess
}
if subcommand != "report" {
return writeExecUsageError(stderr, fmt.Sprintf("unknown usage command %q", subcommand))
}
options, help, err := parseUsageArgs(args)
if err != nil {
return writeExecUsageError(stderr, err.Error())
}
if help {
if err := writeUsageHelp(stdout); err != nil {
return exitCrash
}
return exitSuccess
}
set, err := collectUsageData(deps.newSessionStore(), options.sessionID)
if err != nil {
return writeAppError(stderr, err.Error(), exitCrash)
}
since := usageSinceCutoff(options, deps)
events := filterEventsSince(set.events, since)
workspaceRoot, err := resolveWorkspaceRoot("", deps)
if err != nil {
return writeExecUsageError(stderr, err.Error())
}
summary, err := deps.inspectChanges(context.Background(), zerogit.InspectOptions{Cwd: workspaceRoot})
if err != nil {
return writeExecUsageError(stderr, err.Error())
}
// The --stat summary line ("N files changed, A insertions(+), B deletions(-)")
// carries no secret-bearing tokens, so parsing the already-redacted DiffStat
// returned by zerogit.Inspect is safe.
diff := zerogit.ParseDiffStat(summary.DiffStat)
registry, err := modelregistry.DefaultRegistry()
if err != nil {
return writeAppError(stderr, err.Error(), exitCrash)
}
report, err := usage.BuildReport(events, set.meta, &registry, diff.NetLOC())
if err != nil {
return writeAppError(stderr, err.Error(), exitCrash)
}
if options.json {
if err := writePrettyJSON(stdout, report); err != nil {
return exitCrash
}
return exitSuccess
}
if _, err := fmt.Fprintln(stdout, FormatReport(report, diff.Insertions, diff.Deletions)); err != nil {
return exitCrash
}
return exitSuccess
}
// usageSinceCutoff resolves the inclusive lower-bound date (YYYY-MM-DD). An
// explicit --since wins; otherwise --days N derives a cutoff relative to the
// injected clock. An empty result means "no lower bound".
func usageSinceCutoff(options usageOptions, deps appDeps) string {
if strings.TrimSpace(options.since) != "" {
return options.since
}
if options.days > 0 {
cutoff := deps.now().UTC().AddDate(0, 0, -(options.days - 1))
return cutoff.Format("2006-01-02")
}
return ""
}
func parseUsageArgs(args []string) (usageOptions, bool, error) {
options := usageOptions{}
for index := 0; index < len(args); index++ {
arg := args[index]
switch {
case arg == "-h" || arg == "--help" || arg == "help":
return options, true, nil
case arg == "--json":
options.json = true
case arg == "--days":
value, next, err := nextFlagValue(args, index, arg)
if err != nil {
return options, false, err
}
days, err := parsePositiveOrZeroInt(value, "--days")
if err != nil {
return options, false, err
}
options.days = days
index = next
case strings.HasPrefix(arg, "--days="):
days, err := parsePositiveOrZeroInt(strings.TrimSpace(strings.TrimPrefix(arg, "--days=")), "--days")
if err != nil {
return options, false, err
}
options.days = days
case arg == "--since":
value, next, err := nextFlagValue(args, index, arg)
if err != nil {
return options, false, err
}
if _, parseErr := time.Parse("2006-01-02", value); parseErr != nil {
return options, false, execUsageError{fmt.Sprintf("invalid --since %q: expected YYYY-MM-DD", value)}
}
options.since = value
index = next
case strings.HasPrefix(arg, "--since="):
value := strings.TrimSpace(strings.TrimPrefix(arg, "--since="))
if _, parseErr := time.Parse("2006-01-02", value); parseErr != nil {
return options, false, execUsageError{fmt.Sprintf("invalid --since %q: expected YYYY-MM-DD", value)}
}
options.since = value
case arg == "--session" || arg == "--session-id":
value, next, err := nextFlagValue(args, index, arg)
if err != nil {
return options, false, err
}
if strings.TrimSpace(value) == "" {
return options, false, execUsageError{fmt.Sprintf("%s requires a value", arg)}
}
options.sessionID = strings.TrimSpace(value)
index = next
case strings.HasPrefix(arg, "--session="):
value := strings.TrimSpace(strings.TrimPrefix(arg, "--session="))
if value == "" {
return options, false, execUsageError{"--session requires a value"}
}
options.sessionID = value
case strings.HasPrefix(arg, "--session-id="):
value := strings.TrimSpace(strings.TrimPrefix(arg, "--session-id="))
if value == "" {
return options, false, execUsageError{"--session-id requires a value"}
}
options.sessionID = value
default:
return options, false, execUsageError{fmt.Sprintf("unknown usage flag %q", arg)}
}
}
return options, false, nil
}
// FormatReport renders a usage Report as a per-day table plus totals and net-LOC
// efficiency ratios. Cost and net-LOC are labeled as estimates: persisted usage
// events carry no model id or cost (cost is reconstructed from session
// metadata), and net-LOC is a working-tree diff proxy. The insertions/deletions
// counts come straight from the parsed working-tree --stat.
func FormatReport(report usage.Report, insertions int, deletions int) string {
var builder strings.Builder
builder.WriteString("Usage report (cost is a reconstructed estimate)\n\n")
builder.WriteString(fmt.Sprintf("%-12s %10s %14s %14s\n", "date", "requests", "tokens", "est. cost"))
for _, bucket := range report.Buckets {
builder.WriteString(fmt.Sprintf("%-12s %10d %14s %14s\n",
bucket.Date, bucket.Requests, groupThousands(bucket.TotalTokens), formatUSD(bucket.TotalCost)))
}
builder.WriteString(fmt.Sprintf("\n%-12s %10d %14s %14s\n",
"total", report.Total.Requests, groupThousands(report.Total.TotalTokens), formatUSD(report.Total.TotalCost)))
builder.WriteString(fmt.Sprintf("\nnet LOC (estimate): +%d / -%d = %d\n",
insertions, deletions, report.NetLOC))
if report.NetLOCPositive {
builder.WriteString(fmt.Sprintf("tokens per net LOC: %.1f\n", report.TokensPerNetLOC))
builder.WriteString(fmt.Sprintf("est. cost per net LOC: %s\n", formatUSD(report.CostPerNetLOC)))
} else {
builder.WriteString("tokens per net LOC: n/a (net LOC <= 0)\n")
builder.WriteString("est. cost per net LOC: n/a (net LOC <= 0)\n")
}
return strings.TrimRight(builder.String(), "\n")
}
func formatUSD(value float64) string {
formatted, err := modelregistry.FormatCostUSD(value)
if err != nil {
return "$0.0000"
}
return formatted
}
// groupThousands renders an integer with comma thousands separators, preserving
// a leading minus sign.
func groupThousands(value int) string {
sign := ""
if value < 0 {
sign = "-"
value = -value
}
digits := fmt.Sprintf("%d", value)
if len(digits) <= 3 {
return sign + digits
}
var out []byte
prefix := len(digits) % 3
if prefix == 0 {
prefix = 3
}
out = append(out, digits[:prefix]...)
for index := prefix; index < len(digits); index += 3 {
out = append(out, ',')
out = append(out, digits[index:index+3]...)
}
return sign + string(out)
}
func writeUsageHelp(w io.Writer) error {
_, err := fmt.Fprint(w, `Usage:
zero usage report [flags]
Summarizes token usage and reconstructed (estimated) cost from persisted local
Zero session usage events, plus a working-tree net-LOC efficiency estimate.
Flags:
--json Print JSON report
--days <number> Only include the most recent N days
--since <YYYY-MM-DD> Only include events on or after this date
--session <id> Limit the report to one session
-h, --help Show this help
Cost is reconstructed from session model metadata and is an estimate; net LOC
is a working-tree diff proxy and is labeled as an estimate.
`)
return err
}