anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 02:05:26 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[PR #7036] perf(tool): add caching and retry logic to web tools #12218

New Issue
Open
opened 2026-02-16 18:17:08 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 18:17:08 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/anomalyco/opencode/pull/7036

State: open
Merged: No

Summary

Add LRU cache with disk persistence and retry with exponential backoff to websearch, codesearch, and webfetch tools to improve performance and reliability.

Problem

The web tools currently:

  1. No caching - Every identical query hits the network, even for repeated queries seconds apart
  2. No retry - Transient network errors or rate limits cause immediate failure
  3. Cold starts - Every OpenCode restart starts fresh with no cached data
  4. No observability - No way to see if caching is working

Solution

1. LRU Cache with Disk Persistence

New utility at packages/opencode/src/util/cache.ts:

  • In-memory LRU cache for fast lookups (<5ms)
  • Disk persistence to ~/.cache/opencode/{namespace}/ for warm restarts
  • TTL-based expiration for freshness
  • SHA256 key hashing for safe filenames
  • INFO-level logging for cache hits/misses (observability)
  • Hit/miss counters with hit rate calculation
Tool TTL Max Entries Notes
websearch 1 hour 500 Skips cache when livecrawl: "preferred"
codesearch 2 hours 500 Longer TTL for stable docs
webfetch 30 min 200 Skips dynamic URLs (api, auth, etc.)

2. Retry with Exponential Backoff

Extended packages/util/src/retry.ts with new exports:

  • isRetryableError() - Combines all retryable error checks
  • isRateLimitError() - Detects 429 errors
  • isServerError() - Detects 5xx errors

Configuration: 3 attempts, 1s → 2s → 4s backoff, max 10s delay

Retries: Network errors, rate limits (429), server errors (5xx)
Does NOT retry: Client errors (4xx) - those indicate request issues

Performance Impact

Scenario Before After
Repeated query ~300ms <5ms
Network hiccup Immediate failure Auto-retry 3x
After restart Cold start Warm from disk

Observability

Cache operations are logged at INFO level:

INFO service=cache namespace=websearch event=hit source=memory
INFO service=cache namespace=websearch event=miss

Stats available via cache.stats():

{
  namespace: "websearch",
  memorySize: 42,
  maxSize: 500,
  hits: 156,
  misses: 23,
  hitRate: 0.87
}

Changes

File Change
packages/opencode/src/util/cache.ts NEW - LRU cache with logging and stats
packages/util/src/retry.ts Add isRetryableError, isRateLimitError, isServerError
packages/opencode/src/tool/websearch.ts Add caching + retry
packages/opencode/src/tool/codesearch.ts Add caching + retry
packages/opencode/src/tool/webfetch.ts Add caching + retry
packages/opencode/test/tool/cache.test.ts NEW - Cache unit tests (7 tests)
packages/opencode/test/util/retry.test.ts NEW - Retry unit tests (10 tests)

Testing

  • 17 new unit tests covering:
    • Cache behavior (LRU, TTL, eviction)
    • Hit/miss counters and hit rate
    • Counter reset on clear
    • Expired entries counting as misses
    • Retry logic with exponential backoff
  • All existing tests pass
  • Manual testing confirms cache hits on repeated queries

Breaking Changes

None. Existing behavior preserved, just faster and more reliable.

**Original Pull Request:** https://github.com/anomalyco/opencode/pull/7036 **State:** open **Merged:** No --- ## Summary Add LRU cache with disk persistence and retry with exponential backoff to `websearch`, `codesearch`, and `webfetch` tools to improve performance and reliability. ## Problem The web tools currently: 1. **No caching** - Every identical query hits the network, even for repeated queries seconds apart 2. **No retry** - Transient network errors or rate limits cause immediate failure 3. **Cold starts** - Every OpenCode restart starts fresh with no cached data 4. **No observability** - No way to see if caching is working ## Solution ### 1. LRU Cache with Disk Persistence New utility at `packages/opencode/src/util/cache.ts`: - In-memory LRU cache for fast lookups (<5ms) - Disk persistence to `~/.cache/opencode/{namespace}/` for warm restarts - TTL-based expiration for freshness - SHA256 key hashing for safe filenames - **INFO-level logging** for cache hits/misses (observability) - **Hit/miss counters** with hit rate calculation | Tool | TTL | Max Entries | Notes | |------|-----|-------------|-------| | websearch | 1 hour | 500 | Skips cache when `livecrawl: "preferred"` | | codesearch | 2 hours | 500 | Longer TTL for stable docs | | webfetch | 30 min | 200 | Skips dynamic URLs (api, auth, etc.) | ### 2. Retry with Exponential Backoff Extended `packages/util/src/retry.ts` with new exports: - `isRetryableError()` - Combines all retryable error checks - `isRateLimitError()` - Detects 429 errors - `isServerError()` - Detects 5xx errors Configuration: 3 attempts, 1s → 2s → 4s backoff, max 10s delay **Retries:** Network errors, rate limits (429), server errors (5xx) **Does NOT retry:** Client errors (4xx) - those indicate request issues ## Performance Impact | Scenario | Before | After | |----------|--------|-------| | Repeated query | ~300ms | <5ms | | Network hiccup | Immediate failure | Auto-retry 3x | | After restart | Cold start | Warm from disk | ## Observability Cache operations are logged at INFO level: ``` INFO service=cache namespace=websearch event=hit source=memory INFO service=cache namespace=websearch event=miss ``` Stats available via `cache.stats()`: ```typescript { namespace: "websearch", memorySize: 42, maxSize: 500, hits: 156, misses: 23, hitRate: 0.87 } ``` ## Changes | File | Change | |------|--------| | `packages/opencode/src/util/cache.ts` | **NEW** - LRU cache with logging and stats | | `packages/util/src/retry.ts` | Add `isRetryableError`, `isRateLimitError`, `isServerError` | | `packages/opencode/src/tool/websearch.ts` | Add caching + retry | | `packages/opencode/src/tool/codesearch.ts` | Add caching + retry | | `packages/opencode/src/tool/webfetch.ts` | Add caching + retry | | `packages/opencode/test/tool/cache.test.ts` | **NEW** - Cache unit tests (7 tests) | | `packages/opencode/test/util/retry.test.ts` | **NEW** - Retry unit tests (10 tests) | ## Testing - 17 new unit tests covering: - Cache behavior (LRU, TTL, eviction) - Hit/miss counters and hit rate - Counter reset on clear - Expired entries counting as misses - Retry logic with exponential backoff - All existing tests pass - Manual testing confirms cache hits on repeated queries ## Breaking Changes None. Existing behavior preserved, just faster and more reliable.
yindo added the pull-request label 2026-02-16 18:17:08 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#12218
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?