SivanCola
95b5857e41
Problem: Long-range usage queries repeatedly decoded every daily JSONL record, making the statistics page increasingly expensive. Root cause: The authoritative append log had no offset ledger or derived aggregation layer, so exact queries always rescanned source files. Fix: Add a disposable usage catalog with path-offset idempotency, line-hash rewrite detection, tail reconciliation, daily rollups, exact JSONL fallback for incomplete ranges, bounded flush integration, diagnostics, and bilingual documentation. Verification: Covered by duplicate receipt, tail append, file rewrite, corrupt-line, legacy request, SQL-versus-JSONL parity, and million-record warm-query benchmark tests.
1.2 KiB
Usage Catalog
Reasonix stores disposable usage rollups at <cache root>/usage-catalog/v1.sqlite.
Daily statistics JSONL remains authoritative and byte-compatible with older
versions. The catalog records file offsets and line hashes for cross-process
idempotency, then derives daily source/model/provider rollups.
Only a successful JSONL append emits a non-blocking receipt. Duplicate receipts cannot double-count a line. A changed hash, truncated file, replacement, queue overflow, or external append triggers byte-offset reconciliation or a safe per-day rebuild. Torn tails, blank lines, corrupt JSON, legacy request defaults, time zones, date boundaries, and active-day semantics match the existing JSONL decoder.
SQL aggregation is used only when every selected day is complete. Otherwise the existing exact JSONL query runs on explicit statistics-page access while the background index catches up. Catalog failures never block provider events, turn completion, controller startup, or shutdown.
reasonix doctor catalogs [--json]
reasonix catalogs reindex usage [--json]
Diagnostics expose schema, integrity, lag, counts, and failures without model request content. Reindexing never changes daily JSONL.