esengine/DeepSeek-Reasonix
1
0 Fork 0
mirror of https://github.com/esengine/DeepSeek-Reasonix.git synced 2026-11-07 10:45:59 -10:00
Code Issues Packages Projects Releases 1 Wiki Activity
13,235 Commits 49 Branches 617 Tags
studio
Commit Graph
13 Commits
Author SHA1 Message Date
esengine 522eb69230 feat(telemetry): a metric is a count, not an install id 
Opt-in metrics carried the anonymous install id so the dashboard could show a
deduplicated daily user per signal. Keeping that answer meant keeping a row per
install per signal per day — ~28M of them — and the 30-day COUNT(DISTINCT
install_id) over that table was the database's entire read bill: the primary
key leads with `date`, so each query could prune only the window and scanned
the whole table, eight signals an hour, ~5.4B rows a day for one module.

The client stops sending the id, the worker stops writing and creating the
tables, the hourly rollup that existed only to make that query survivable is
gone, and the dashboard's users/opens toggle goes with it. The aggregate
tables keep the same signals as event counts in ~180k rows.

migrate-drop-metric-users.sql drops the four retired tables. Deploy the worker
first — it stops creating them at runtime, and a DROP before that is rebuilt on
the next request.
 2026-08-25 10:05:18 -07:00
SivanCola ed00eccf32 Add cross-platform Desktop crash diagnostics and runtime attribution / 增加跨平台桌面崩溃诊断与运行时归因 (#8169) 
* fix(desktop): add Windows crash attribution diagnostics

Problem: Windows LTSC 2019 abrupt exits could not be attributed reliably, test telemetry could reach production, and Diagnostics counted lifecycle events rather than affected installations.

Root cause: legacy startup state was replayable, WebView2 failures were inferred from log text without native process details, and the crash Worker lacked build-, runtime-, and installation-level aggregation.

Fix: fail-close Desktop test endpoints, consume legacy state atomically, add lifecycle v2 and native WebView2 Args2/Args3 observation, and extend privacy-bounded telemetry, D1 rollups, Diagnostics filters, and bilingual release runbooks.

Verification: go test ./... in root and desktop; focused race tests; Worker typecheck and 86 tests; frontend tests and production build; fresh and migrated SQLite schema validation; Windows amd64 Desktop and vendored WebView2 cross-compilation; git diff --check.

* refactor(diagnostics): satisfy owner size budgets

Problem: the initial diagnostics implementation exceeded the repository's incremental file and function size budgets in Desktop startup, Worker ingestion, and dashboard rendering owners.

Root cause: lifecycle initialization, installation aggregation, and group-detail rendering were added directly to already baselined entry-point files.

Fix: move lifecycle setup into its tracker owner, extract diagnostics-v2 storage and query policy, and isolate crash-group rendering without changing wire contracts or runtime behavior.

Verification: repolint; Worker typecheck and 86 tests; focused Desktop and repair tests; Windows amd64 Desktop cross-compilation; the unrelated transient updater test passed three consecutive focused reruns.

* test(repair): use WaitGroup.Go for claim race

Problem: the current PR lint rejected the manual WaitGroup Add/go/Done pattern in the concurrent legacy-state claim regression test.

Root cause: the test used the legacy goroutine setup even though the repository modernize rules require sync.WaitGroup.Go.

Fix: use WaitGroup.Go while preserving the explicit ready barrier and deterministic interleaving.

Verification: go test ./internal/repair; go test -race ./internal/repair; go run ./tools/repolint; git diff --check.

* fix(repair): serialize legacy startup claims

Problem: Windows CI observed zero reports when concurrent launches tried to consume one dead legacy startup record.

Root cause: concurrent pre-claim reads can keep Windows file handles open long enough for every rename attempt to fail.

Fix: acquire the existing non-blocking cross-process file lock before reading and atomically renaming the legacy record, so exactly one observer owns the claim path.

Verification: 100 concurrent-claim repetitions; 20 race-enabled repetitions; full internal/repair normal and race suites; Windows amd64 test compilation; repolint; git diff --check.

* fix(diagnostics): close Windows crash attribution gaps

Start lifecycle tracking after the Wails single-instance gate and serialize anonymous install identity creation across concurrent callers.

Wait for WebView2 navigation completion before recording renderer recovery, commit crash group/sample/aggregate writes in one D1 batch, and align runtime CLI schema and diagnostics ordering.

* fix(diagnostics): address PR review findings

Problem:
- Renderer recovery could consume an unrelated navigation and recoverable COM errors could terminate the desktop.
- Per-install daily rows overwrote WebView2 dimensions, relabeling earlier GPU and runtime events.
- The PR exceeded repository size gates and did not execute vendored patch tests in Windows CI.

Root cause:
- Recovery completion was not correlated with NavigationStarting IDs, and shared fatal error handling was reused.
- Mutable dimension columns and event counts lived in the same deduplicated installation row.

Fix:
- Correlate renderer reloads by navigation ID, keep recovery errors non-fatal, and default failed kind reads to unknown.
- Add immutable event-dimension facts for filters and distributions while retaining installation deduplication.
- Split diagnostics tests, trim size regressions, and run vendored WebView2 tests in Windows CI.

Verification:
- go test ./...
- targeted Desktop and internal/repair race tests
- Worker tests and typecheck
- Frontend 146-suite test run
- repolint, actionlint, gofmt, and Windows cross-compile

* fix(desktop): align raw bundle ratchet

Problem:
The Desktop CI inherited a main-v2 raw startup bundle size of 2204.8 KiB while the ratchet still allowed only 2203 KiB, so frontend builds stopped before all Desktop checks.

Root cause:
Recent always-loaded desktop contracts already moved the baseline beyond the old threshold, and this PR adds the required diagnostics privacy attribution copy without materially changing gzip size.

Fix:
Raise only the raw aggregate allowance to a tightly measured 2205 KiB and document the native diagnostics copy. Keep gzip, CSS, locale, and largest-chunk limits unchanged.

Verification:
- pnpm build
- go run ./tools/repolint
- git diff --check

* feat(desktop): add cross-platform crash diagnostics

Problem: Desktop runtime failures lacked reliable ownership, lifecycle attribution, and affected-install analytics across Windows and Linux.

Root cause: legacy startup records could replay, WebView2 diagnostics were platform-specific, and Worker aggregation mixed recovery outcomes into unstable groups.

Fix: add fail-open primary ownership and lifecycle v2, generic Web Runtime reports, native WebView2 and WebKitGTK observers, privacy-safe platform dimensions, additive D1 aggregates, and cross-platform Dashboard filters.

Verification: go test and race suites, go vet, golangci-lint, Worker typecheck/tests, frontend tests/build/bundle budgets, repolint, actionlint, and Windows amd64 cross-builds.

* fix(diagnostics): preserve future lifecycle schemas

Problem:
A future lifecycle schema that removes v2 PID or phase fields could be treated as an invalid v2 record and pruned after retention.

Root cause:
The consumer validated v2-specific fields before checking the schema version.

Fix:
Reject unknown schemas before applying v2 validation or cleanup, and cover an aged future record without current fields.

Verification:
- cd desktop && go test . -run '^TestDesktopLifecycle'
- cd desktop && go test -race . -run '^TestDesktopLifecycle'
- git diff --check

* fix(diagnostics): correlate WebKit reload events

Problem:
The native WebKitGTK recovery smoke classified a successful renderer reload as failed and could not reach the cooldown assertion.

Root cause:
WebKit can emit load-failed for the terminated navigation after web-process-terminated. The recovery state treated every pending load failure as belonging to the new reload.

Fix:
Track whether the recovery reload has emitted LOAD_STARTED and only accept load-failed and LOAD_FINISHED after that boundary. Reset the marker on every terminal path and extend the source contract test.

Verification:
GOCACHE=/tmp/reasonix-pr8169-go-cache go test . -run '^(TestWebKit|TestWebRuntime)'
git diff --check

* fix(diagnostics): defer WebKit recovery reload

Problem:
WebKitGTK 4.1 still timed out the success recovery smoke after load events were scoped to the recovery navigation.

Root cause:
Starting webkit_web_view_reload synchronously inside the web-process-terminated signal callback is unreliable while WebKit is still dispatching process teardown.

Fix:
Schedule the single recovery reload on the GTK main context's idle queue, retain the WebView until dispatch, and guard the helper with the pending recovery owner. Update the source contract test to enforce this deferred abnormal-only path.

Verification:
GOCACHE=/tmp/reasonix-pr8169-go-cache go test . -run '^(TestWebKit|TestWebRuntime)'
git diff --check

* test(diagnostics): isolate WebKit smoke scenarios

Problem:
After the deferred reload made the success smoke pass, the next scenario aborted inside WebKitGTK while constructing another test WebView in the same process.

Root cause:
The smoke harness repeatedly destroyed and recreated WebViews that share WebKitGTK's process-global default context. That cross-scenario lifecycle does not match the single-WebView Desktop process and leaves asynchronous teardown active between cases.

Fix:
Run success, failure, timeout, and cooldown scenarios in isolated test subprocesses. Keep the cooldown pair in one child and one WebView so repeated native termination and single-reload behavior remain covered.

Verification:
GOCACHE=/tmp/reasonix-pr8169-go-cache go test . -run '^(TestWebKit|TestWebRuntime)'
git diff --check

* test(diagnostics): keep WebKit smoke local

Problem:
The isolated success and cooldown smoke scenarios still reported reload failure while failure and timeout behaved correctly.

Root cause:
The test HTML used https://reasonix.invalid as its base URI. Reloading the terminated view attempted to resolve that unreachable URI, so the smoke measured a synthetic network failure instead of native process recovery.

Fix:
Load the smoke HTML with a NULL base URI, which WebKitGTK maps to local about:blank content and requires no network access.

Verification:
GOCACHE=/tmp/reasonix-pr8169-go-cache go test . -run '^(TestWebKit|TestWebRuntime)'
git diff --check

* test(diagnostics): use reloadable WebKit smoke URI

Problem:
The native success and cooldown scenarios still timed out with load_html content even after removing the unreachable base URI.

Root cause:
A transient document created by load_html does not provide a stable reloadable resource after its WebProcess is terminated on the tested WebKitGTK runtime.

Fix:
Load a fully local data URI before terminating the WebProcess. This provides a stable current URI for the same production reload call without adding network or filesystem I/O.

Verification:
GOCACHE=/tmp/reasonix-pr8169-go-cache go test . -run '^(TestWebKit|TestWebRuntime)'
git diff --check

* test(diagnostics): allow WebKit process startup

Problem:
The native smoke used a one-second recovery timeout and classified cold WebProcess startup under Xvfb as a failed reload.

Root cause:
The smoke-only timeout was substantially shorter than observed headless WebKitGTK process creation, while production correctly allows thirty seconds.

Fix:
Raise only the tagged smoke timeout to five seconds. Keep the production timeout at thirty seconds and the smoke safety timeout at fifteen seconds.

Verification:
GOCACHE=/tmp/reasonix-pr8169-go-cache go test . -run '^(TestWebKit|TestWebRuntime)'
git diff --check

* Automate diagnostics D1 migration before worker deploy

Problem:
The diagnostics-v2 schema had to be applied manually even though the production Cloudflare credential is available only to Actions. Deploying the new Worker first would make ingestion depend on missing D1 columns and tables.

Root cause:
The crash Worker workflow validated and deployed code but had no guarded database migration stage.

Fix:
Add a fail-closed schema inspector that skips a complete migration, applies the SQL only when entirely absent, rejects partial state, records the current D1 Time Travel bookmark, and verifies the final schema. Run it after tests and before Worker deployment.

Verification:
- npm test (96 tests)
- npm run typecheck
- node --check scripts/apply-diagnostics-v2.mjs
- bash scripts/release-workflows.test.sh
- actionlint v1.7.7
- git diff --check

* Keep diagnostics copy within locale bundle budget

Problem:
Merging the latest main-v2 bundle ratchets made the expanded Chinese telemetry disclosure exceed both locale budgets.

Root cause:
The cross-platform disclosure repeated verbose phrases for the anonymous identifier, runtime fields, abnormal-exit payload, and privacy exclusions.

Fix:
Compact the Simplified and Traditional Chinese copy while preserving upload timing, diagnostic dimensions, and excluded sensitive data. Keep the existing bundle budgets unchanged.

Verification:
- cd desktop/frontend && pnpm build
- node scripts/check-bundle-budget.mjs
- git diff --check
 2026-08-11 18:09:41 +08:00
YHH 4f2fb94998 feat(crash-worker): precompute the 30-day preferences deduplication (#7279) 
Closes #7272.

The preferences module's 30-day COUNT(DISTINCT install_id) spans ~28M rows of
metric_users and D1 abandons it mid-query, so #7271 pinned the module to the
7-day window it can still finish. This computes the 30-day answer ahead of time
instead.

It has to be computed, not derived: a cross-day COUNT(DISTINCT) cannot be summed
from per-day totals, because an install active on twelve days would count twelve
times. Benchmarks ruled out the alternatives — storing one JSON row per install
per day and expanding it with json_each took 103s at production shape, worse
than what we have, because parsing JSON per row costs more than the I/O it
saves.

What does work is the query split by signal: measured against production, one
signal over the 30-day window is ~1s. An hourly cron walks the signal list eight
at a time behind a cursor, so a run stays well inside its CPU budget and a full
pass takes about seven hours. Each signal's rows are replaced in a single batch,
so a reader never observes one half-written, and a signal whose query is
abandoned still advances the cursor rather than blocking every signal behind it.

The dashboard reads the rollup for the 30-day desktop window and keeps querying
live everywhere else. Until the first pass completes the panel says the
precompute has not reached every signal yet, rather than showing an empty one.

Both tables are created at runtime, matching how ingest_sentinel_state and the
cli_* tables bootstrap, so no manual migration is needed.
 2026-08-03 15:27:06 +08:00
SivanCola b907a6b097 Add privacy-preserving CLI telemetry and crash reports / 添加隐私友好的 CLI 遥测与崩溃报告 (#6984) 
* feat(cli): add privacy-preserving telemetry and crash reports

Summary:
- add bounded CLI usage metrics with explicit policy controls and silent retry behavior
- capture sanitized panic reports locally and require user confirmation before upload
- separate Desktop and CLI aggregates in the crash-report worker and stats dashboard
- document telemetry fields, privacy guarantees, report commands, and D1 migration requirements

Compatibility:
- default missing client surfaces to Desktop for existing clients and payloads
- migrate existing aggregate rows to the Desktop surface before deploying the new worker

Verification:
- go test ./internal/telemetry ./internal/crashreport ./internal/config ./internal/cli ./cmd/reasonix
- env -u DEEPSEEK_API_KEY go test ./...
- pnpm test (workers/crash-report, 49 tests)
- pnpm run typecheck (workers/crash-report)
- ./scripts/cache-guard.sh

* fix(stats): decouple CLI schema migration from Worker deploy

Problem:
The initial surface migration rebuilt the existing Desktop telemetry tables and changed their primary keys. Running it before the new Worker broke the released Worker's ON CONFLICT clauses, while deploying the Worker first queried columns that did not exist yet.

Root cause:
Desktop and CLI isolation was implemented by changing a shared persisted schema, which made the old and new Worker contracts mutually exclusive during rollout.

Fix:
Keep the released Desktop tables and primary keys unchanged, route CLI telemetry to additive cli_* tables, and let the new Worker initialize those tables with idempotent CREATE IF NOT EXISTS statements. Keep the migration additive and identical to the runtime bootstrap so migration, deployment, and rollback order no longer affect Desktop ingestion.

Verification:
- pnpm test (workers/crash-report, 54 tests)
- pnpm run typecheck (workers/crash-report)
- git diff --check

* fix(test): make crash report mode check portable

Problem:
The Windows CI job failed because os.FileMode reports synthesized POSIX bits on Windows rather than the 0600 mode requested by AtomicWriteFile.

Root cause:
The test treated Unix permission bits as a cross-platform ACL signal even though Windows enforces file access through inherited ACLs.

Fix:
Keep the 0600 assertion on Unix-like systems and skip only that non-portable mode-bit comparison on Windows while retaining all report creation, sanitization, and bounded-retention checks.

Verification:
- go test ./internal/crashreport
- GOOS=windows GOARCH=amd64 go test -c ./internal/crashreport

* fix(cli): preserve actionable crash report groups

Problem:
CLI panic reports used runtime/debug.Stack as their top frame, so unrelated panics collapsed into one fingerprint. Queue pruning also deleted reports written in an unknown future format.

Root cause:
Top-frame extraction selected the first source location without filtering capture/runtime frames, and pruning treated every JSON file as a current report.

Fix:
Select the first application frame with its sanitized function name, retain a safe fallback, and prune only recognized current-schema report files. Reject newer schemas from old-client listing so their fields remain untouched.

Verification:
- env -u DEEPSEEK_API_KEY go test ./...
- go test -race ./internal/crashreport ./cmd/reasonix
- go vet ./...
- pnpm test (workers/crash-report, 54 tests)
- pnpm run typecheck (workers/crash-report)
- Windows cross-compile of internal/crashreport

* fix(cli): request consent before telemetry upload

Problem:
A fresh release-build interactive CLI treated a missing telemetry setting as auto and could start anonymous uploads without first informing the user.

Root cause:
The runtime default and persisted decision state were conflated, and telemetry startup had no consent owner before the reporter launched its background flush.

Fix:
Preserve a missing cli_metrics field as an undecided state, show a localized one-time [Y/n] disclosure before the first eligible upload, persist auto or off before continuing, clean pending counters on rejection, and fail closed when saving fails. Keep CI, Safe Mode, development, environment opt-outs, and non-interactive sessions silent and disabled until a decision exists.

Verification:
- env -u DEEPSEEK_API_KEY go test ./...
- go test -race ./internal/telemetry ./internal/cli
- go vet ./...
- pnpm test (workers/crash-report, 54 tests)
- pnpm run typecheck (workers/crash-report)
- ./scripts/cache-guard.sh
- git diff --check
 2026-07-27 23:00:53 +08:00
SivanCola 6b30958d83 fix(crash-report): detect stalled ingest between checks 2026-07-14 00:44:05 +08:00
YHH 1858974dc7 feat(crash): unify dashboard auth onto id.reasonix.io identity (#5701) 
The crash dashboard ran its own account system — a separate D1 users/
sessions table with its own password login — which fragmented identity
from id.reasonix.io. Make it consume the shared identity instead: it
resolves the .reasonix.io rxid cookie via id.reasonix.io/me (Bearer) and
keeps only a per-dashboard role in a new `access` table. Existing roles
migrate by email (migrate-access.sql); ADMIN_EMAILS force-promotes the
owner so a missed migration can't lock them out. Sign-in redirects to the
shared login; sign-out revokes the shared session.

Also: accounts /auth/logout accepts a Bearer token (so crash can revoke
server-side), and the site login honours a next to any *.reasonix.io.
 2026-07-01 16:26:24 +08:00
SivanCola ffcbac9bf5 Add settings preference metrics 2026-06-16 22:45:45 +08:00
SivanCola ea0c3a9b69 Improve crash reporting diagnostics / 增强崩溃上报诊断 (#4285) 
* feat: enrich crash reporting pipeline

* fix: address crash report worker review
 2026-06-13 23:25:40 +08:00
YHH 89bcd487c7 feat(crash): capture Go panics, breadcrumbs, and group summaries (#4187) 
The crash dashboard only ever saw frontend WebView errors, every group was
an opaque count=1 fingerprint, and minified stacks carried no symbols. This
widens what a report captures and makes the dashboard triageable.

- desktop: capture Go-side panics on the goroutines we own (restore/build/
  snapshot), persist a scrubbed stack to crash-pending.json, and resend on
  next launch under the desktop.telemetry opt-out. recoverToPending re-raises
  so the process still crashes — the stack is recorded, not swallowed.
- worker: store a one-line title per group and surface it in the dashboard
  list so a crash is readable without opening each fingerprint.
- frontend: a breadcrumb ring buffer (console.error/warn, tab navigation)
  folded into the report. Keep console.warn/error in production — terser
  drop_console was deleting the very trail the breadcrumbs collect.
- frontend: preserve function/class names through terser and stamp the build
  commit, normalized in the worker fingerprint so grouping stays build-stable.
 2026-06-12 21:08:37 +08:00
YHH c0aac6412f feat(crash): account system for the crash dashboard (#4119) 
Replace the single shared STATS_PASSWORD with email/password accounts,
sessions, and role-based access. New registrations land as `pending`
with no access until an admin grants `viewer` or `admin`; the
ADMIN_EMAILS secret bootstraps the first admin on register. Admins get
user management, crash-group triage (status / note / delete), and an
audit log. The /v1 ingest endpoints stay public and unchanged.
 2026-06-12 12:14:48 +08:00
YHHandreasonix 653ab079ce feat(crash-worker): add /v1/metrics ingest for opt-in agent metrics (#4030) 
Adds the worker side of the desktop opt-in metrics flush: a generic
(date, version, os, signal, bucket) -> count table, a zod-validated
/v1/metrics POST that upserts per-launch aggregate snapshots, a dedicated
rate-limit bucket, and a stats-page section that breaks each signal down
by bucket.

The signal set is an enum and the bucket is length+charset capped, so the
table can never be polluted with arbitrary keys. No install id and no
content reach this endpoint — only enumerated counters.

Deploy is owner-run: apply schema.sql to D1, then wrangler deploy.

Co-authored-by: reasonix <[email protected]>
 2026-06-11 19:42:55 +08:00
YHHandreasonix c58bcc3a06 feat(desktop): anonymous launch ping, crash device info, /stats page (#3985) 
Crash reports alone answer "what broke" but not "for whom" or "how many
are even running this". Three additions, all on the existing worker:

- Launch ping: one POST per start with a random install id + version +
  OS facts. Gated on new desktop.telemetry config (default on, toggle in
  Settings > Updates, disclosed in desktop/README.md); dev builds never
  ping. Carries no conversation, key, or file data.
- Crash reports now attach coarse device facts (OS version, CPU model,
  cores, RAM) so "only crashes on X" patterns are visible.
- Worker grows /v1/ping (per-day install dedup with opens counter) and a
  Basic-auth /stats page (daily actives, version/platform breakdown,
  recent crash groups) so day-to-day reading needs no SQL.

Co-authored-by: reasonix <[email protected]>
 2026-06-11 15:53:08 +08:00
YHHandreasonix 8feccbfa5b feat(desktop): user-initiated crash reporting via crash.reasonix.io (#3978) 
* feat(desktop): user-initiated crash reporting via crash.reasonix.io

The crash overlay could only ask users to screenshot or copy the error,
so most reports never reached us. Add an opt-in-per-click "Send report"
button wired to a new ReportCrash binding that scrubs user names from
paths, attaches version/GOOS/GOARCH, and POSTs to a Cloudflare Worker
(workers/crash-report): D1-backed, fingerprint-deduped (5 raw samples
per group), per-IP rate limited. Nothing is ever sent automatically.

* chore(workers): pin crash-report D1 database_id from initial deploy

---------

Co-authored-by: reasonix <[email protected]>
 2026-06-11 15:25:55 +08:00