mirror of
https://github.com/vxcontrol/langchaingo.git
synced 2026-07-21 00:45:22 -04:00
Travis Cline
91fc754372
* llms: add prompt caching and reasoning token support Add comprehensive infrastructure for prompt caching and reasoning/thinking tokens: - Add ReasoningModel interface for models supporting extended reasoning - Add CacheControl and CachedContent types for prompt caching - Add ThinkingConfig with modes (none, low, medium, high, auto) - Add token budget calculation and usage tracking for thinking tokens - Add WithPromptCaching, WithThinking, and related call options - Add model detection for reasoning-capable models (OpenAI o1/o3, Claude 3.7+, DeepSeek) - Update options documentation for better provider compatibility notes - Remove testify imports tracking file Supports provider-specific implementations while maintaining consistent API. * testing: add comprehensive LLM testing framework Add standardized testing infrastructure for all LLM providers following Go's testing/fstest design philosophy: - Add llmtest package with automatic capability discovery and parallel testing - Add TestLLM function that probes and tests all supported LLM features - Add provider-specific test files for all LLM implementations (anthropic, bedrock, cloudflare, cohere, ernie, fake, googleai, huggingface, llamafile, local, maritaca, mistral, ollama, openai, watsonx) - Add comprehensive tests for prompt caching, reasoning tokens, and token utilization - Add MockLLM implementation for testing without API calls - Add ValidateLLM function for basic model validation - Support automatic detection of streaming, tool calls, reasoning, and caching capabilities The framework provides a simple API (TestLLM) that automatically discovers and tests all supported capabilities, enabling consistent testing across all provider implementations. * llms: implement provider-specific prompt caching and reasoning support Add comprehensive provider-specific implementations for prompt caching and reasoning tokens: **Anthropic (Claude)**: - Add prompt caching with ephemeral cache controls and beta headers - Add extended thinking support for Claude 3.7+ with budget token configuration - Add interleaved thinking for tool calls and 128K output support - Add thinking content extraction from <thinking> tags - Add cache token tracking (creation/read input tokens) **Google AI (Gemini)**: - Add CachingHelper for pre-created cached content management - Add reasoning detection for Gemini 2.0+ models - Add cached content support via metadata - Add standardized token usage reporting with cache information **Ollama**: - Add ContextCache for in-memory conversation context caching - Add reasoning support detection for DeepSeek R1, QwQ, and thinking models - Add Think parameter support for reasoning-capable models - Add cache statistics and hit/miss tracking **OpenAI**: - Add reasoning support detection for o1/o3 series and future GPT-5 - Add system message handling for models without system support (o1/o3) - Add thinking content extraction and standardized token reporting - Add metadata filtering to prevent internal fields from reaching API - Add ReasoningEffort parameter preparation for future models **Cross-provider features**: - Implement ReasoningModel interface across all providers - Add standardized GenerationInfo fields (ThinkingContent, ThinkingTokens, etc.) - Add provider-specific options (WithPromptCaching, WithExtendedOutput, etc.) - Add comprehensive test coverage for caching and reasoning features - Add model capability detection and validation All providers now support the unified prompt caching and reasoning interfaces while maintaining their specific implementation details and capabilities. * examples: add comprehensive prompt caching and reasoning examples Add practical examples demonstrating prompt caching and reasoning token features across multiple LLM providers: **Anthropic Examples**: - Add anthropic-extended-capabilities example showing combined extended thinking + 128K output - Add anthropic-interleaved-thinking example demonstrating thinking between tool calls - Show token budget management and thinking content extraction - Include comprehensive README documentation for each example **Multi-Provider Examples**: - Add googleai-reasoning-caching example with Gemini 2.0+ reasoning and cached content - Add ollama-reasoning-caching example with DeepSeek R1/QwQ models and context caching - Add prompt-caching example showing Anthropic's 90% cost reduction on cached tokens - Add reasoning-tokens example comparing o1-mini, Claude 3.7, and standard models **Key Features Demonstrated**: - Prompt caching for cost optimization and faster responses - Reasoning/thinking tokens for improved response quality - Token usage analysis and cache hit/miss tracking - Provider-specific capabilities and configuration options - Real-world use cases like data analysis, Q&A systems, and multi-step reasoning All examples include detailed documentation, error handling, and clear output formatting to help developers understand and implement these advanced LLM features. * gofmt: fix formatting in messages.go
63 lines
2.1 KiB
Go
63 lines
2.1 KiB
Go
// Package llmtest provides utilities for testing LLM implementations.
|
|
//
|
|
// Inspired by Go's testing/fstest package, llmtest offers a simple,
|
|
// backend-independent way to verify that LLM implementations conform
|
|
// to the expected interfaces and behaviors.
|
|
//
|
|
// # Design Philosophy
|
|
//
|
|
// Following the principles of testing/fstest:
|
|
// - Minimal API surface - one main function (TestLLM)
|
|
// - Automatic capability discovery - no configuration required
|
|
// - Comprehensive by default - tests all detected capabilities
|
|
// - Interface testing - works with any llms.Model implementation
|
|
// - Simple usage pattern - just pass the model to test
|
|
//
|
|
// # Usage
|
|
//
|
|
// Testing an LLM implementation is straightforward:
|
|
//
|
|
// func TestMyLLM(t *testing.T) {
|
|
// llm, err := mylllm.New()
|
|
// if err != nil {
|
|
// t.Fatal(err)
|
|
// }
|
|
// llmtest.TestLLM(t, llm)
|
|
// }
|
|
//
|
|
// # Automatic Capability Discovery
|
|
//
|
|
// The package automatically detects and tests supported capabilities:
|
|
// - Basic operations (Call, GenerateContent)
|
|
// - Streaming (if model implements streaming interface)
|
|
// - Tool/Function calling (probed with test tool)
|
|
// - Reasoning/Thinking mode (if supported)
|
|
// - Token counting (if usage information provided)
|
|
// - Context caching (if implemented)
|
|
//
|
|
// # Mock Implementation
|
|
//
|
|
// A MockLLM is provided for testing without making actual API calls:
|
|
//
|
|
// mock := &llmtest.MockLLM{
|
|
// CallFunc: func(ctx context.Context, prompt string, options ...llms.CallOption) (string, error) {
|
|
// return "mocked response", nil
|
|
// },
|
|
// }
|
|
// llmtest.TestLLM(t, mock)
|
|
//
|
|
// # Parallel Testing
|
|
//
|
|
// All tests run in parallel by default for better performance:
|
|
// - Core tests (Call, GenerateContent) run concurrently
|
|
// - Capability tests run in parallel when detected
|
|
// - Safe for concurrent execution with independent contexts
|
|
//
|
|
// # Provider Coverage
|
|
//
|
|
// The package is used to test all LangChain Go providers:
|
|
// anthropic, bedrock, cloudflare, cohere, ernie, fake, googleai,
|
|
// huggingface, llamafile, local, maritaca, mistral, ollama, openai,
|
|
// watsonx, and more.
|
|
package llmtest
|