* llms/openai: sanitize HTTP errors to prevent API key exposure (#1393)
Fix security issue where context deadline errors could expose API keys and
sensitive request details in error messages. Added sanitizeHTTPError function
to detect context timeouts and network errors, then return generic error
messages without exposing sensitive information.
Changes:
- Added sanitizeHTTPError() function to sanitize HTTP client errors
- Updated chat.go to use sanitizeHTTPError() for http.Do() errors
- Updated embeddings.go to use sanitizeHTTPError() for http.Do() errors
- Added comprehensive test cases to prevent regression
* agents: fix ChainCallOption silent failure (#1416)
Fix issue where ChainCallOption parameters were silently ignored by Executor.Call() and Agent implementations.
Changes:
- Updated Agent.Plan() interface signature to accept variadic ChainCallOption parameters
- Updated Executor.Call() to accept and propagate options to Agent.Plan()
- Updated Executor.doIteration() to propagate options through the chain
- Updated OneShotZeroAgent.Plan() to accept and pass options to chains.Predict()
- Updated ConversationalAgent.Plan() to accept and pass options to chains.Predict()
- Updated OpenAIFunctionsAgent.Plan() to accept and pass options to LLM.GenerateContent()
- Exported GetLLMCallOptions() function for option conversion (was getLLMCallOptions)
- Updated test mock to match new Agent interface signature
Now users can pass LLM configuration options (temperature, max tokens, etc.) through executors to agents.
llms/anthropic: add streaming thinking/reasoning token support
Implement StreamingReasoningFunc support in the Anthropic client to enable
real-time streaming of thinking tokens during extended thinking responses.
Changes:
- Add StreamingReasoningFunc field to messagePayload, MessageRequest structs
- Modify handleThinkingDelta() to call StreamingReasoningFunc when thinking
chunks arrive during streaming
- Wire up StreamingReasoningFunc from llms.CallOptions through to the
Anthropic client payload
- Update setMessageDefaults to enable streaming when StreamingReasoningFunc
is provided
This follows the same pattern as the OpenAI client (chat.go:638-663) and
enables thinking tokens to stream in real-time at the BEGINNING of the
response, rather than appearing after the response completes.
Fixes issue where thinking_delta events were not calling the streaming
reasoning callback, causing thinking content to only be available after
response completion.
llms: update OpenAI model context sizes
Add support for latest OpenAI models with accurate context windows:
- GPT-4o and GPT-4o-mini: 128K tokens
- GPT-4 Turbo variants: 128K tokens
- GPT-3.5 Turbo: 16K tokens (corrected from 4K)
Includes comprehensive test coverage for all model variants.
Reference: https://platform.openai.com/docs/models
Co-authored-by: paulnegz <paulnegz@example.com>
* llms: add prompt caching and reasoning token support
Add comprehensive infrastructure for prompt caching and reasoning/thinking tokens:
- Add ReasoningModel interface for models supporting extended reasoning
- Add CacheControl and CachedContent types for prompt caching
- Add ThinkingConfig with modes (none, low, medium, high, auto)
- Add token budget calculation and usage tracking for thinking tokens
- Add WithPromptCaching, WithThinking, and related call options
- Add model detection for reasoning-capable models (OpenAI o1/o3, Claude 3.7+, DeepSeek)
- Update options documentation for better provider compatibility notes
- Remove testify imports tracking file
Supports provider-specific implementations while maintaining consistent API.
* testing: add comprehensive LLM testing framework
Add standardized testing infrastructure for all LLM providers following Go's testing/fstest design philosophy:
- Add llmtest package with automatic capability discovery and parallel testing
- Add TestLLM function that probes and tests all supported LLM features
- Add provider-specific test files for all LLM implementations (anthropic, bedrock, cloudflare, cohere, ernie, fake, googleai, huggingface, llamafile, local, maritaca, mistral, ollama, openai, watsonx)
- Add comprehensive tests for prompt caching, reasoning tokens, and token utilization
- Add MockLLM implementation for testing without API calls
- Add ValidateLLM function for basic model validation
- Support automatic detection of streaming, tool calls, reasoning, and caching capabilities
The framework provides a simple API (TestLLM) that automatically discovers and tests all supported capabilities, enabling consistent testing across all provider implementations.
* llms: implement provider-specific prompt caching and reasoning support
Add comprehensive provider-specific implementations for prompt caching and reasoning tokens:
**Anthropic (Claude)**:
- Add prompt caching with ephemeral cache controls and beta headers
- Add extended thinking support for Claude 3.7+ with budget token configuration
- Add interleaved thinking for tool calls and 128K output support
- Add thinking content extraction from <thinking> tags
- Add cache token tracking (creation/read input tokens)
**Google AI (Gemini)**:
- Add CachingHelper for pre-created cached content management
- Add reasoning detection for Gemini 2.0+ models
- Add cached content support via metadata
- Add standardized token usage reporting with cache information
**Ollama**:
- Add ContextCache for in-memory conversation context caching
- Add reasoning support detection for DeepSeek R1, QwQ, and thinking models
- Add Think parameter support for reasoning-capable models
- Add cache statistics and hit/miss tracking
**OpenAI**:
- Add reasoning support detection for o1/o3 series and future GPT-5
- Add system message handling for models without system support (o1/o3)
- Add thinking content extraction and standardized token reporting
- Add metadata filtering to prevent internal fields from reaching API
- Add ReasoningEffort parameter preparation for future models
**Cross-provider features**:
- Implement ReasoningModel interface across all providers
- Add standardized GenerationInfo fields (ThinkingContent, ThinkingTokens, etc.)
- Add provider-specific options (WithPromptCaching, WithExtendedOutput, etc.)
- Add comprehensive test coverage for caching and reasoning features
- Add model capability detection and validation
All providers now support the unified prompt caching and reasoning interfaces while maintaining their specific implementation details and capabilities.
* examples: add comprehensive prompt caching and reasoning examples
Add practical examples demonstrating prompt caching and reasoning token features across multiple LLM providers:
**Anthropic Examples**:
- Add anthropic-extended-capabilities example showing combined extended thinking + 128K output
- Add anthropic-interleaved-thinking example demonstrating thinking between tool calls
- Show token budget management and thinking content extraction
- Include comprehensive README documentation for each example
**Multi-Provider Examples**:
- Add googleai-reasoning-caching example with Gemini 2.0+ reasoning and cached content
- Add ollama-reasoning-caching example with DeepSeek R1/QwQ models and context caching
- Add prompt-caching example showing Anthropic's 90% cost reduction on cached tokens
- Add reasoning-tokens example comparing o1-mini, Claude 3.7, and standard models
**Key Features Demonstrated**:
- Prompt caching for cost optimization and faster responses
- Reasoning/thinking tokens for improved response quality
- Token usage analysis and cache hit/miss tracking
- Provider-specific capabilities and configuration options
- Real-world use cases like data analysis, Q&A systems, and multi-step reasoning
All examples include detailed documentation, error handling, and clear output formatting to help developers understand and implement these advanced LLM features.
* gofmt: fix formatting in messages.go
- Add nil/bounds checks in OpenAI Functions Agent ParseOutput
- Handle multiple tool calls properly instead of just first one
- Fix Anthropic panic when content is nil or empty (fixes#993)
- Prevent index out of bounds errors in agent parsing
test: improve test resilience with better HTTP recording handling
Add hasExistingRecording helper to detect available recordings and skip tests
gracefully when recordings are unavailable. Replace generic credential checks
with specific recording validation and provide clear error messages.
* openai: omit temperature field for reasoning models
Adds special handling for GPT-5, o1, and o3 models that only accept default
temperature values. These reasoning models reject requests with non-default
temperature settings, causing API errors.
- Modify ChatRequest.MarshalJSON to omit temperature field for reasoning models
- Add isReasoningModel helper function to identify affected models
- Add comprehensive tests for temperature field handling and model detection"
* openai: update test recordings for temperature field handling
Update test recordings to reflect the new temperature field serialization behavior
implemented in the reasoning models fix. These recordings show the temperature
field correctly positioned at the end of request JSON objects, ensuring proper
omission when needed for GPT-5, o1, and o3 models.
* test: update OpenAI test recordings
Re-record OpenAI package test recordings to match updated JSON field
ordering from the temperature field positioning changes.
Add comprehensive support for token limits in OpenAI API requests:
- Add WithMaxCompletionTokens() option for explicitly setting the modern field
- Add WithLegacyMaxTokensField() option for compatibility with older servers
- Implement custom JSON marshaling to prevent both fields being sent simultaneously
- Add documentation explaining token limit options and best practices
- Add comprehensive tests for serialization behavior and option handling
This implementation properly handles both max_tokens (legacy) and
max_completion_tokens (modern) fields while preventing API errors
from sending both fields at once.
* build: add examples-updater tool and update examples to v0.1.14-pre.1
Add a new tool to automate updating langchaingo version references in example
projects and use it to update all examples to v0.1.14-pre.1. The tool also
removes temporary replace directives that are no longer needed.
- Add update-examples Makefile target that runs the new tool
- Create internal/devtools/examples-updater implementation
- Update all example go.mod files to reference v0.1.14-pre.1
- Remove replace directives in googleai and vertex examples
- Update dependencies in main go.mod/go.sum
* httprr: improve header normalization and test container setup
Add header normalization to make HTTP recordings more stable:
- Add normalizeGoogleAPIClientHeader and normalizeVersionHeader functions
- Normalize User-Agent, x-goog-api-client and other version headers
- Remove OpenAI-Project header for consistency
- Preserve body for both replay lookup and recording
- Add comprehensive tests for header normalization
Improve testcontainer environment detection:
- Better handling of non-standard Docker socket paths
- Disable Ryuk reaper by default for resource efficiency
- Add verbose logging option via environment variable
* httputil: add ApiKeyTransport for query parameter API keys
Add a reusable ApiKeyTransport that adds API keys to URL query parameters:
- Create ApiKeyTransport implementation in httputil package
- Add comprehensive tests for the new transport
- Refactor googleai_test.go to use the shared transport
- Update chains/llm_test.go to use ApiKeyTransport with proper key scrubbing
This improves how API keys are handled with client libraries that don't
properly set keys when using custom HTTP clients, particularly useful
with httprr for testing Google API integrations.
* openai: remove duplicate MaxTokens field assignment
Remove redundant assignment where both MaxTokens and MaxCompletionTokens
were being set to the same value. MaxCompletionTokens is the preferred
field name that reflects OpenAI's API changes, while MaxTokens was
previously kept for backward compatibility.
* vectorstores/maridadb: add test infrastructure for MariaDB vectorstore
Add TestMain function for MariaDB vectorstore tests that ensures the proper
test environment is set up using the testctr package. This enables consistent
test container setup and teardown for MariaDB integration tests.
* all: normalize all httprr recordings for consistency
Update test recordings across all packages to use normalized headers:
- Standardize User-Agent to 'langchaingo-httprr'
- Normalize version information in x-goog-api-client headers
- Remove OpenAI-Project headers for consistency
- Fix deprecated Anthropic completion API tests
- Update HuggingFace test recordings with valid responses
These changes make test recordings stable across dependency updates
and different environments, preventing test failures from version changes.
* devtools: add utility tool for normalizing httprr test recordings
Add a command-line tool that standardizes version information in httprr recordings:
- Normalizes x-goog-api-client headers to use placeholder versions
- Standardizes x-amz-user-agent headers for consistency
- Replaces Go version strings with generic placeholders
- Supports dry-run mode to preview changes without modifying files
- Includes verbose output option for detailed change reporting
This tool helps maintain consistent test recordings across different environments
and dependency versions, preventing test failures from version changes.
* llms/openai: update OpenAI embedding test to use text-embedding-3-small model
Update the embedding test to use the current text-embedding-3-small model:
- Change model from text-embedding-ada-002 to text-embedding-3-small
- Adjust dimensions parameter from 1234 to 256 to match model capabilities
- Update test recording with appropriate response data
This change ensures tests remain compatible with OpenAI's current embedding
models and prevents test failures from API version changes.
* hook up anthropic tool calls to bedrock
* standardize anthropic bedrock tool calling constants and tests
* add integration tests for bedrock anthropic tool calling
* clean up bedrock anthropic tool integration test formatting
* suppress function length linter warnings in bedrock tool tests
Add //nolint:funlen directives to TestBedrockWithTools and
TestBedrockToolCallMultipleIterations to prevent linter warnings
about function length while maintaining comprehensive test coverage.
---------
Co-authored-by: Ryan Kois <ryank@runreveal.com>
* bedrock anthropic tool calling support
* bedrock: add comprehensive tool calling tests and fix test type usage
- Add integration tests for single and multi-iteration tool calling workflows
- Fix test expectations to match new function/tool role support
- Use proper anthropicContentBlock type instead of inline structs
- Add weather and search tool testing scenarios
* fix: bedrock LLM add modelProvider to resolve "unsupported provider"
* fix: remove version 2 config for golangci-lint v1 compatibility
* Revert "fix: remove version 2 config for golangci-lint v1 compatibility"
This reverts commit fe27edcb4c850a8796a703aa70ca69674a2ed244.
* fix go ci test
---------
Co-authored-by: Dreamans <dreamans@163.com>
- Add WithModelProvider option to explicitly set provider for edge cases
- Improve provider detection to handle Nova models and inference profiles
- Add comprehensive unit tests for Nova provider detection
- Add bedrock-provider-example demonstrating usage
- Fix CreateCompletion to accept optional provider parameter
This allows users to specify the provider explicitly when automatic
detection doesn't work correctly (e.g., custom inference endpoints),
and adds full support for Amazon Nova models.
Fixes: #1345
Co-authored-by: Travis Cline <travis.cline@gmail.com>
llms/anthropic: add tool support for streaming responses
Add support for input_json_delta events in streaming API responses to
properly handle tool use during streaming. The implementation includes:
- Handle input_json_delta events in parseStreamingMessageResponse
- Add handleJSONDelta function for accumulating partial JSON data
- Improve error messages with type information for debugging
- Add comprehensive tests for streaming with tool use scenarios
This enables proper tool calling functionality when using Anthropic's
streaming API, ensuring partial JSON tool inputs are correctly
accumulated and parsed.
Co-authored-by: Travis Cline <travis.cline@gmail.com>
* openaiclient: add dimensions option while creating embeddings
* openaiclient: fix dimensions json tag in embeddings payload
Fix the JSON struct tag for the Dimensions field in embeddingPayload from
"omitzero" to the correct "omitempty". This ensures the dimensions parameter
is properly omitted from API requests when not specified.
* openaiclient: fix test name for dimensions test case
Correct the test name from "without dimensions" to "with dimensions" in
TestMakeEmbeddingRequest to accurately reflect the test's purpose. This
ensures the test name properly describes that it's testing the behavior
with dimensions specified, not without.
---------
Co-authored-by: Travis Cline <travis.cline@gmail.com>
Add proper handling for SSE comment lines (starting with ':') and other
non-data fields in streaming responses according to the SSE specification.
The implementation now explicitly checks for "data:" prefixes before
processing lines, improving compatibility with various providers.
Add comprehensive tests for different SSE comment patterns including
OpenRouter comments, comments without spaces, and other SSE fields.
* llms: fix memory leaks by adding Close methods and improving goroutine handling
Add Close methods to GoogleAI and Vertex clients to properly clean up resources
and prevent memory leaks from underlying gRPC connections. Update examples to
demonstrate proper client cleanup with defer statements.
Improve goroutine management in OpenAI streaming response handling with proper
context cancellation and non-blocking channel operations to prevent goroutine
leaks in error scenarios.
* examples: add replace directives for testing memory leak fixes
Add temporary replace directives to GoogleAI and Vertex example go.mod files
to test the Close() method changes from the previous commit. These directives
allow the examples to use the local version of the library with the memory
leak fixes before they're included in the next release.
This commit adds full support for OpenRouter, fixing streaming response parsing
errors that occurred when providers send non-JSON prefixes in their SSE streams.
Fixes#1172, Fixes#942
- Add recording for TestWithThink showing think:true in request
- Add recording for TestClient_GenerateChatWithThink
- Tests now pass without requiring live Ollama server
- Recordings verify think parameter is properly transmitted
* ollama: Add support for reasoning mode (think parameter)
- Add Think bool field to Options struct for Ollama 0.9.0+
- Add WithThink() option function to enable reasoning mode
- Allows models to show their internal reasoning process
Fixes#1287
* ollama: Add tests for think parameter support
- Add TestClient_GenerateChatWithThink to verify API behavior
- Add TestOptionsJSONMarshalWithThink to verify JSON serialization
- Add TestWithThink to verify option works at LLM level
- Tests verify think parameter is properly passed through the stack
* all: add broad httprr coverage, update dependencies, organize go.mod file, bump to 1.23
update go version to 1.23
add lots of test coverage via httprr recordings
update dependencies and organize go.mod
add testutil/testctr which helps work around a testcontainers-go+colima bug
expand the huggingface implementation and tests
expand capabilities of the ollama package
lif