langchaingo/internal/httprr

httprr: HTTP Record and Replay for Testing

The httprr package provides deterministic HTTP record and replay functionality for testing. It allows tests to record real HTTP interactions during development and replay them during CI/testing, ensuring consistent and fast test execution.

Quick Start

func TestMyAPI(t *testing.T) {
    // Skip test gracefully if no credentials and no recording exists
    httprr.SkipIfNoCredentialsOrRecording(t, "API_KEY")
    
    // Create recorder/replayer
    rr, err := httprr.OpenForTest(t, http.DefaultTransport)
    if err != nil {
        t.Fatal(err)
    }
    defer rr.Close()
    
    // Use rr.Client() for all HTTP calls
    client := rr.Client()
    resp, err := client.Get("https://api.example.com/data")
    // ... test continues
}

Core Concepts

Recording vs Replay Modes

• Recording Mode (-httprecord=.): Makes real HTTP requests and saves them to .httprr files
• Replay Mode (default): Reads saved .httprr files and replays the responses

Circular Buffer for Identical Requests

When multiple identical requests are recorded (e.g., for cache testing), httprr stores all responses in order and replays them sequentially using a circular buffer. This allows testing scenarios where:

• The same request produces different responses (e.g., cache miss → cache hit)
• Multiple identical requests need to be tested in sequence
• Response ordering matters for test accuracy

Example: Testing Google Gemini's implicit caching where 3 identical requests produce different CachedTokens values (0, then non-zero, then non-zero).

Command-Line Flags

• -httprecord=<regexp>: Re-record traces for files matching the regexp pattern (use "." to match all)
• -httprecord-delay=<ms>: Add delay in milliseconds between HTTP requests during recording (helps avoid rate limits)

File Management

• Recording: Always creates uncompressed .httprr files for easier debugging
• Replay: Automatically handles both .httprr and .httprr.gz files
• Conflict Resolution: Chooses the newer file if both compressed and uncompressed exist
• Auto-cleanup: Recording mode removes conflicting files automatically

API Reference

Core Functions

OpenForTest(t *testing.T, rt http.RoundTripper) (*RecordReplay, error)

The primary API for most test cases. Creates a recorder/replayer for the given test.

• Recording mode: Creates testdata/TestName.httprr
• Replay mode: Loads existing recording
• File naming: Derived automatically from t.Name()
• Directory: Always uses testdata/ subdirectory

SkipIfNoCredentialsOrRecording(t *testing.T, envVars ...string)

Gracefully skips tests when they cannot run (no API keys) and have no recorded data.

// Skip if OPENAI_API_KEY not set AND no recording exists
httprr.SkipIfNoCredentialsOrRecording(t, "OPENAI_API_KEY")

// Skip if neither API_KEY nor BACKUP_KEY is set AND no recording exists  
httprr.SkipIfNoCredentialsOrRecording(t, "API_KEY", "BACKUP_KEY")

Open(file string, rt http.RoundTripper) (*RecordReplay, error)

Low-level API for custom file management. Most tests should use OpenForTest instead.

RecordReplay Methods

Client() *http.Client

Returns an HTTP client that routes through the recorder/replayer.

ScrubReq(scrubs ...func(*http.Request) error)

Adds request scrubbing functions to sanitize sensitive data before recording.

rr.ScrubReq(func(req *http.Request) error {
    req.Header.Set("Authorization", "Bearer test-api-key")
    return nil
})

ScrubResp(scrubs ...func(*bytes.Buffer) error)

Adds response scrubbing functions to sanitize sensitive data in responses.

Recording() bool

Reports whether the recorder is in recording mode.

Close() error

Closes the recorder/replayer. Use with defer for automatic cleanup.

Usage Patterns

Basic API Testing

func TestOpenAIChat(t *testing.T) {
    httprr.SkipIfNoCredentialsOrRecording(t, "OPENAI_API_KEY")
    
    rr, err := httprr.OpenForTest(t, http.DefaultTransport)
    if err != nil {
        t.Fatal(err)
    }
    defer rr.Close()
    
    // Scrub sensitive data
    rr.ScrubReq(func(req *http.Request) error {
        req.Header.Set("Authorization", "Bearer test-api-key")
        return nil
    })
    
    // Create client with recording support
    llm, err := openai.New(openai.WithHTTPClient(rr.Client()))
    require.NoError(t, err)
    
    // Test continues with recorded/replayed HTTP calls
    response, err := llm.GenerateContent(ctx, messages)
    require.NoError(t, err)
}

Helper Functions for Multiple Tests

func createTestClient(t *testing.T) *MyAPIClient {
    t.Helper()
    httprr.SkipIfNoCredentialsOrRecording(t, "MY_API_KEY")
    
    rr, err := httprr.OpenForTest(t, http.DefaultTransport)
    if err != nil {
        t.Fatal(err)
    }
    t.Cleanup(func() { rr.Close() })
    
    return NewMyAPIClient(WithHTTPClient(rr.Client()))
}

func TestFeatureA(t *testing.T) {
    client := createTestClient(t)
    // ... test continues
}

func TestFeatureB(t *testing.T) {
    client := createTestClient(t)
    // ... test continues
}

Multiple API Endpoints

func TestMultiAPIIntegration(t *testing.T) {
    httprr.SkipIfNoCredentialsOrRecording(t, "OPENAI_API_KEY", "SERPAPI_KEY")
    
    rr, err := httprr.OpenForTest(t, http.DefaultTransport)
    if err != nil {
        t.Fatal(err)
    }
    defer rr.Close()
    
    // Both clients will use the same recording
    openaiClient := openai.New(openai.WithHTTPClient(rr.Client()))
    searchClient := serpapi.New(serpapi.WithHTTPClient(rr.Client()))
    
    // All HTTP calls are recorded/replayed together
}

Testing Identical Requests (Circular Buffer)

When testing scenarios that require multiple identical requests with different responses (e.g., cache behavior):

func TestImplicitCaching(t *testing.T) {
    httprr.SkipIfNoCredentialsOrRecording(t, "API_KEY")
    
    rr, err := httprr.OpenForTest(t, http.DefaultTransport)
    if err != nil {
        t.Fatal(err)
    }
    defer rr.Close()
    
    client := NewAPIClient(WithHTTPClient(rr.Client()))
    
    // Make identical requests - each gets recorded with its actual response
    // Request 1: Cache miss (no cached tokens)
    r1, _ := client.Query("same query")
    assert.Equal(t, 0, r1.CachedTokens)
    
    // Request 2: Cache hit (has cached tokens)
    r2, _ := client.Query("same query")
    assert.Greater(t, r2.CachedTokens, 0)
    
    // Request 3: Still cached
    r3, _ := client.Query("same query")
    assert.Greater(t, r3.CachedTokens, 0)
    
    // Replay mode: Responses are returned in order from circular buffer
    // Re-running the test will get: r1 (no cache), r2 (cached), r3 (cached)
    // A 4th identical request would cycle back to r1's response
}

Command Line Usage

Recording New Interactions

# Record all tests
go test ./... -httprecord=.

# Record specific test
go test ./pkg -httprecord=. -run TestSpecificFunction

# Record with pattern matching
go test ./... -httprecord="TestOpenAI.*"

Recording Through a Gateway

The Google AI tests reach the vendor through whatever base URL GOOGLE_BASE_URL names, so fixtures can be re-recorded with a proxy key when no direct vendor key is at hand:

GOOGLE_API_KEY=<gateway key> GOOGLE_BASE_URL=https://gateway.example/gemini \
  go test ./llms/googleai/ -httprecord=. -run TestGoogleAIWithTools

The recorded request still carries the vendor URL, because the rewrite happens below the recorder. Proxy-added X-Litellm-* response headers are scrubbed, one of which reports the key's accumulated spend; TestRecordingsCarryNoProxyHeaders fails if any reaches testdata/.

Running with Recorded Data

# Normal test run (uses recorded data)
go test ./...

# Skip tests that need credentials
OPENAI_API_KEY="" go test ./...  # Tests will skip gracefully

File Management

File Structure

testdata/
├── TestBasicFunction.httprr           # Uncompressed recording
├── TestWithSubtest-subcase.httprr     # Subtest recording  
├── TestOldFunction.httprr.gz          # Compressed recording
└── TestComplexAPI-setup.httprr        # Multi-part test

File Naming Rules

• Test name: TestMyFunction → File: TestMyFunction.httprr
• With subtests: TestMyFunction/subcase → File: TestMyFunction-subcase.httprr
• Special chars: Replaced with hyphens for filesystem compatibility

Compression Management

# Compress all recordings (for repository storage)
go run ./internal/devtools/rrtool pack -r

# Check compression status
go run ./internal/devtools/rrtool check

# Decompress for debugging
go run ./internal/devtools/rrtool unpack -r

Recording with Rate Limit Protection

When recording tests that make many API calls, use the delay flag to avoid hitting rate limits:

# Record with 1 second delay between requests
go test -httprecord=. -httprecord-delay=1000 ./...

# Record specific test with 500ms delay
go test -httprecord=. -httprecord-delay=500 -run TestMyAPI ./mypackage

Best Practices

1. Always Use Graceful Skipping

// ✅ Good: Test skips gracefully when it can't run
httprr.SkipIfNoCredentialsOrRecording(t, "API_KEY")

// ❌ Bad: Test fails when API key missing
rr, err := httprr.OpenForTest(t, http.DefaultTransport)

2. Scrub Sensitive Data

// ✅ Good: Replace real API keys with test values
rr.ScrubReq(func(req *http.Request) error {
    req.Header.Set("Authorization", "Bearer test-api-key")
    return nil
})

// ❌ Bad: Real API keys recorded in files
// (No scrubbing - keys end up in repository)

3. Use Helper Functions

// ✅ Good: Reusable test setup
func createTestLLM(t *testing.T) *openai.LLM {
    t.Helper()
    httprr.SkipIfNoCredentialsOrRecording(t, "OPENAI_API_KEY")
    // ... setup code
}

// ❌ Bad: Duplicate setup in every test
func TestA(t *testing.T) {
    httprr.SkipIfNoCredentialsOrRecording(t, "OPENAI_API_KEY")
    rr, err := httprr.OpenForTest(t, http.DefaultTransport)
    // ... repeated setup
}

4. Handle Cleanup Properly

// ✅ Good: Automatic cleanup
defer rr.Close()

// or
t.Cleanup(func() { rr.Close() })

// ❌ Bad: Manual cleanup (can be forgotten)
// (No defer or cleanup)

Troubleshooting

Common Issues

"cached HTTP response not found"

Problem: Test is trying to make an HTTP request not in the recording.

Solutions:

# Re-record the test
go test ./pkg -httprecord=. -run TestName

# Check if you have required environment variables
export OPENAI_API_KEY="your-key-here"
go test ./pkg -httprecord=. -run TestName

"gzip: invalid header"

Problem: .httprr.gz file is corrupted or not actually compressed.

Solutions:

# Check and fix compression
go run ./internal/devtools/rrtool check
go run ./internal/devtools/rrtool pack -r

# Or remove the corrupted file and re-record
rm testdata/TestName.httprr.gz
go test ./pkg -httprecord=. -run TestName

Test skipped unexpectedly

Problem: Test is skipping when you expect it to run.

Debug steps:

# Check if environment variables are set
echo $OPENAI_API_KEY

# Check if recording exists
ls testdata/TestName.httprr*

# Run with verbose output
go test ./pkg -run TestName -v

File Conflicts

The system automatically handles conflicts, but you can resolve manually:

# Check which file is newer
ls -la testdata/TestName.httprr*

# Remove older file (system will warn and use newer)
rm testdata/TestName.httprr.gz  # if .httprr is newer

# Or compress the newer one
gzip testdata/TestName.httprr

Migration Guide

From OpenForTestWithSkip (Old API)

// ❌ Old API (removed)
rr := httprr.OpenForTestWithSkip(t, http.DefaultTransport, "API_KEY")
defer rr.Close()

// ✅ New API
httprr.SkipIfNoCredentialsOrRecording(t, "API_KEY")

rr, err := httprr.OpenForTest(t, http.DefaultTransport)
if err != nil {
    t.Fatal(err)
}
defer rr.Close()

Benefits of New API

1. Consistent Error Handling: All httprr operations return errors
2. Clear Separation: Skip logic separate from file operations
3. Single Responsibility: Each function has one clear purpose
4. Better Documentation: Self-documenting function names

Advanced Usage

Circular Buffer Internals

When httprr encounters multiple identical requests during recording, it stores all responses in order. During replay:

1. First request: Returns the first recorded response
2. Second request: Returns the second recorded response
3. Subsequent requests: Continue cycling through responses in order
4. After last response: Cycles back to the first response (circular buffer)

This behavior is automatic and transparent - no special configuration needed.

File Format: Identical requests share the same request key but maintain separate response entries in the .httprr file, preserving recording order.

Custom File Locations

// For custom file management (rarely needed)
rr, err := httprr.Open("custom/path/recording.httprr", http.DefaultTransport)
if err != nil {
    t.Fatal(err)
}
defer rr.Close()

Conditional Recording

func TestWithConditionalRecording(t *testing.T) {
    // Only record if we have credentials
    if os.Getenv("API_KEY") != "" {
        // Will record new interactions
        rr, err := httprr.OpenForTest(t, http.DefaultTransport)
        // ...
    } else {
        // Will only replay existing recordings
        httprr.SkipIfNoCredentialsOrRecording(t, "API_KEY")
        rr, err := httprr.OpenForTest(t, http.DefaultTransport)
        // ...
    }
}

Complex Scrubbing

rr.ScrubReq(func(req *http.Request) error {
    // Remove API keys
    req.Header.Set("Authorization", "Bearer test-key")
    
    // Scrub request body
    if req.Body != nil {
        body := req.Body.(*httprr.Body)
        bodyStr := string(body.Data)
        bodyStr = strings.ReplaceAll(bodyStr, "real-secret", "test-secret")
        body.Data = []byte(bodyStr)
    }
    
    return nil
})

rr.ScrubResp(func(buf *bytes.Buffer) error {
    // Remove sensitive data from responses
    content := buf.String()
    content = strings.ReplaceAll(content, "sensitive-data", "redacted")
    buf.Reset()
    buf.WriteString(content)
    return nil
})

Contributing

When adding new tests that use external APIs:

1. Always use SkipIfNoCredentialsOrRecording for graceful degradation
2. Include appropriate scrubbing to avoid committing secrets
3. Record with real credentials initially, then scrub the results
4. Compress recordings before committing to save repository space
5. Document required environment variables in test comments

For questions or issues with the httprr system, see the main project documentation or open an issue.