anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 02:05:26 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

Regression detection and diagnostic reasoning #9112

New Issue
Open
opened 2026-02-16 18:11:40 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 18:11:40 -05:00
Owner
Copy Link
Copy Source

Originally created by @florianleibert on GitHub (Feb 11, 2026).

Originally assigned to: @thdxr on GitHub.

Task: Regression detection and diagnostic reasoning

Description

Implement the regression detection algorithm that compares each response's score against rolling averages and flags quality drops. Includes severity classification (minor/moderate/severe) and diagnostic reasoning that explains why a score was low, correlating sub-score drops with observable signals.

Acceptance Criteria

  • packages/opencode/src/quality/detector.ts created
  • Fetches rolling window (last N responses, configurable via regression.windowSize) from SQLite
  • Computes rolling average for composite score and each sub-score
  • Calculates delta: current_score - rolling_average
  • Severity classification: minor (-5 to -15), moderate (-15 to -30), severe (below -30)
  • Regressions persisted to regressions table with: response_id, score_delta, severity, component, description
  • Diagnostic reasoning generated for each regression: identifies which sub-scores dropped, correlates with signals (e.g., "3 new type errors", "user reverted change", "tool invocation failed")
  • Emits Quality.Event.Regression event on bus when severity >= configured alertLevel
  • Handles edge cases: fewer than windowSize responses (use available data), first response (no comparison, no regression)
  • Regression detection runs as part of the scoring pipeline (after score computed, before response to user)

Technical Details

  • Define Quality.Event.Regression using BusEvent.define() pattern
  • Query rolling averages from SQLite: SELECT AVG(composite_score), AVG(correctness_score), AVG(feedback_score), AVG(analysis_score) FROM responses ORDER BY timestamp DESC LIMIT ?
  • Diagnostic reasoning uses a template system: check each sub-score delta, match to known signal patterns, compose human-readable explanation
  • Regression event payload: { responseId, severity, delta, component, description } — TUI can subscribe to show inline warnings

Dependencies

  • Task 001: SQLite storage layer (for queries)
  • Task 003: Scoring engine (provides scores to compare)

Effort Estimate

  • Size: M
  • Hours: 6-8
  • Parallel: false (depends on 001, 003)
Originally created by @florianleibert on GitHub (Feb 11, 2026). Originally assigned to: @thdxr on GitHub. # Task: Regression detection and diagnostic reasoning ## Description Implement the regression detection algorithm that compares each response's score against rolling averages and flags quality drops. Includes severity classification (minor/moderate/severe) and diagnostic reasoning that explains why a score was low, correlating sub-score drops with observable signals. ## Acceptance Criteria - [ ] `packages/opencode/src/quality/detector.ts` created - [ ] Fetches rolling window (last N responses, configurable via `regression.windowSize`) from SQLite - [ ] Computes rolling average for composite score and each sub-score - [ ] Calculates delta: `current_score - rolling_average` - [ ] Severity classification: minor (-5 to -15), moderate (-15 to -30), severe (below -30) - [ ] Regressions persisted to `regressions` table with: response_id, score_delta, severity, component, description - [ ] Diagnostic reasoning generated for each regression: identifies which sub-scores dropped, correlates with signals (e.g., "3 new type errors", "user reverted change", "tool invocation failed") - [ ] Emits `Quality.Event.Regression` event on bus when severity >= configured `alertLevel` - [ ] Handles edge cases: fewer than windowSize responses (use available data), first response (no comparison, no regression) - [ ] Regression detection runs as part of the scoring pipeline (after score computed, before response to user) ## Technical Details - Define `Quality.Event.Regression` using `BusEvent.define()` pattern - Query rolling averages from SQLite: `SELECT AVG(composite_score), AVG(correctness_score), AVG(feedback_score), AVG(analysis_score) FROM responses ORDER BY timestamp DESC LIMIT ?` - Diagnostic reasoning uses a template system: check each sub-score delta, match to known signal patterns, compose human-readable explanation - Regression event payload: `{ responseId, severity, delta, component, description }` — TUI can subscribe to show inline warnings ## Dependencies - [ ] Task 001: SQLite storage layer (for queries) - [ ] Task 003: Scoring engine (provides scores to compare) ## Effort Estimate - Size: M - Hours: 6-8 - Parallel: false (depends on 001, 003)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#9112
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?