anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-23 02:36:57 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

Scoring engine #9116

New Issue
Open
opened 2026-02-16 18:11:40 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 18:11:40 -05:00
Owner
Copy Link
Copy Source

Originally created by @florianleibert on GitHub (Feb 11, 2026).

Originally assigned to: @thdxr on GitHub.

Task: Scoring engine

Description

Implement the core scoring engine that computes a composite 0-100 quality score for each AI response. The engine calculates three sub-scores (code correctness, user feedback, static analysis) and combines them using configurable weights. Scoring runs asynchronously to avoid blocking response delivery.

Acceptance Criteria

  • packages/opencode/src/quality/scorer.ts created
  • Correctness sub-score (0-100): based on tool invocation success/failure, error presence in response, LSP diagnostic delta after changes
  • User feedback sub-score (0-100): starts at neutral (70), adjusted retroactively when signals arrive (accept: +, revert: -30, redo: -15, edit-after-apply: -5)
  • Static analysis sub-score (0-100): based on LSP diagnostic count delta (new errors: heavy penalty, new warnings: moderate, resolved issues: bonus)
  • Composite score = weighted sum using config weights, clamped to 0-100
  • Scoring is async — uses queueMicrotask() or setTimeout(0) after response delivery
  • Partial scoring supported — if LSP data unavailable, score with available signals only (weight redistributed)
  • Score results persisted to SQLite via store.ts
  • Scoring errors caught and logged, never propagated to main workflow

Technical Details

  • Import config weights from quality config
  • Import store functions from store.ts for persistence
  • Correctness signals extracted from message metadata: tool results (success/error), response errors, token usage patterns
  • Static analysis signals from LSP diagnostics snapshot before/after response application
  • User feedback is deferred — initial score computed without it, updated retroactively
  • Expose Quality.scoreResponse(messageMetadata, lspBefore, lspAfter) as main API

Dependencies

  • Task 001: SQLite storage layer (for persistence)
  • Task 002: Quality configuration (for weights and settings)

Effort Estimate

  • Size: L
  • Hours: 8-10
  • Parallel: false (depends on 001, 002)
Originally created by @florianleibert on GitHub (Feb 11, 2026). Originally assigned to: @thdxr on GitHub. # Task: Scoring engine ## Description Implement the core scoring engine that computes a composite 0-100 quality score for each AI response. The engine calculates three sub-scores (code correctness, user feedback, static analysis) and combines them using configurable weights. Scoring runs asynchronously to avoid blocking response delivery. ## Acceptance Criteria - [ ] `packages/opencode/src/quality/scorer.ts` created - [ ] Correctness sub-score (0-100): based on tool invocation success/failure, error presence in response, LSP diagnostic delta after changes - [ ] User feedback sub-score (0-100): starts at neutral (70), adjusted retroactively when signals arrive (accept: +, revert: -30, redo: -15, edit-after-apply: -5) - [ ] Static analysis sub-score (0-100): based on LSP diagnostic count delta (new errors: heavy penalty, new warnings: moderate, resolved issues: bonus) - [ ] Composite score = weighted sum using config weights, clamped to 0-100 - [ ] Scoring is async — uses `queueMicrotask()` or `setTimeout(0)` after response delivery - [ ] Partial scoring supported — if LSP data unavailable, score with available signals only (weight redistributed) - [ ] Score results persisted to SQLite via store.ts - [ ] Scoring errors caught and logged, never propagated to main workflow ## Technical Details - Import config weights from quality config - Import store functions from store.ts for persistence - Correctness signals extracted from message metadata: tool results (success/error), response errors, token usage patterns - Static analysis signals from LSP diagnostics snapshot before/after response application - User feedback is deferred — initial score computed without it, updated retroactively - Expose `Quality.scoreResponse(messageMetadata, lspBefore, lspAfter)` as main API ## Dependencies - [ ] Task 001: SQLite storage layer (for persistence) - [ ] Task 002: Quality configuration (for weights and settings) ## Effort Estimate - Size: L - Hours: 8-10 - Parallel: false (depends on 001, 002)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#9116
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?