mirror of
https://github.com/vxcontrol/langchaingo.git
synced 2026-07-19 13:45:47 -04:00
Travis Cline
91fc754372
* llms: add prompt caching and reasoning token support Add comprehensive infrastructure for prompt caching and reasoning/thinking tokens: - Add ReasoningModel interface for models supporting extended reasoning - Add CacheControl and CachedContent types for prompt caching - Add ThinkingConfig with modes (none, low, medium, high, auto) - Add token budget calculation and usage tracking for thinking tokens - Add WithPromptCaching, WithThinking, and related call options - Add model detection for reasoning-capable models (OpenAI o1/o3, Claude 3.7+, DeepSeek) - Update options documentation for better provider compatibility notes - Remove testify imports tracking file Supports provider-specific implementations while maintaining consistent API. * testing: add comprehensive LLM testing framework Add standardized testing infrastructure for all LLM providers following Go's testing/fstest design philosophy: - Add llmtest package with automatic capability discovery and parallel testing - Add TestLLM function that probes and tests all supported LLM features - Add provider-specific test files for all LLM implementations (anthropic, bedrock, cloudflare, cohere, ernie, fake, googleai, huggingface, llamafile, local, maritaca, mistral, ollama, openai, watsonx) - Add comprehensive tests for prompt caching, reasoning tokens, and token utilization - Add MockLLM implementation for testing without API calls - Add ValidateLLM function for basic model validation - Support automatic detection of streaming, tool calls, reasoning, and caching capabilities The framework provides a simple API (TestLLM) that automatically discovers and tests all supported capabilities, enabling consistent testing across all provider implementations. * llms: implement provider-specific prompt caching and reasoning support Add comprehensive provider-specific implementations for prompt caching and reasoning tokens: **Anthropic (Claude)**: - Add prompt caching with ephemeral cache controls and beta headers - Add extended thinking support for Claude 3.7+ with budget token configuration - Add interleaved thinking for tool calls and 128K output support - Add thinking content extraction from <thinking> tags - Add cache token tracking (creation/read input tokens) **Google AI (Gemini)**: - Add CachingHelper for pre-created cached content management - Add reasoning detection for Gemini 2.0+ models - Add cached content support via metadata - Add standardized token usage reporting with cache information **Ollama**: - Add ContextCache for in-memory conversation context caching - Add reasoning support detection for DeepSeek R1, QwQ, and thinking models - Add Think parameter support for reasoning-capable models - Add cache statistics and hit/miss tracking **OpenAI**: - Add reasoning support detection for o1/o3 series and future GPT-5 - Add system message handling for models without system support (o1/o3) - Add thinking content extraction and standardized token reporting - Add metadata filtering to prevent internal fields from reaching API - Add ReasoningEffort parameter preparation for future models **Cross-provider features**: - Implement ReasoningModel interface across all providers - Add standardized GenerationInfo fields (ThinkingContent, ThinkingTokens, etc.) - Add provider-specific options (WithPromptCaching, WithExtendedOutput, etc.) - Add comprehensive test coverage for caching and reasoning features - Add model capability detection and validation All providers now support the unified prompt caching and reasoning interfaces while maintaining their specific implementation details and capabilities. * examples: add comprehensive prompt caching and reasoning examples Add practical examples demonstrating prompt caching and reasoning token features across multiple LLM providers: **Anthropic Examples**: - Add anthropic-extended-capabilities example showing combined extended thinking + 128K output - Add anthropic-interleaved-thinking example demonstrating thinking between tool calls - Show token budget management and thinking content extraction - Include comprehensive README documentation for each example **Multi-Provider Examples**: - Add googleai-reasoning-caching example with Gemini 2.0+ reasoning and cached content - Add ollama-reasoning-caching example with DeepSeek R1/QwQ models and context caching - Add prompt-caching example showing Anthropic's 90% cost reduction on cached tokens - Add reasoning-tokens example comparing o1-mini, Claude 3.7, and standard models **Key Features Demonstrated**: - Prompt caching for cost optimization and faster responses - Reasoning/thinking tokens for improved response quality - Token usage analysis and cache hit/miss tracking - Provider-specific capabilities and configuration options - Real-world use cases like data analysis, Q&A systems, and multi-step reasoning All examples include detailed documentation, error handling, and clear output formatting to help developers understand and implement these advanced LLM features. * gofmt: fix formatting in messages.go