mirror of
https://github.com/vxcontrol/langchaingo.git
synced 2026-07-16 09:24:26 -04:00
Dmitry Ng
1b1c761d98
Breaking Changes: - Remove Cloudflare Workers AI provider - Remove Cohere provider - Remove Ernie (Baidu) provider - Remove Llamafile provider - Remove Local provider - Remove Maritaca provider - Remove Watsonx (IBM) provider - Remove compliance testing suite This removes dead code that was no longer maintained, tested, or actively used. Users should migrate to actively supported providers like OpenAI, Anthropic, Google AI, Bedrock, or Ollama for local deployments. Documentation: - Remove provider integration pages - Update parity matrix - Update search index - Remove provider examples
61 lines
2.0 KiB
Go
61 lines
2.0 KiB
Go
// Package llmtest provides utilities for testing LLM implementations.
|
|
//
|
|
// Inspired by Go's testing/fstest package, llmtest offers a simple,
|
|
// backend-independent way to verify that LLM implementations conform
|
|
// to the expected interfaces and behaviors.
|
|
//
|
|
// # Design Philosophy
|
|
//
|
|
// Following the principles of testing/fstest:
|
|
// - Minimal API surface - one main function (TestLLM)
|
|
// - Automatic capability discovery - no configuration required
|
|
// - Comprehensive by default - tests all detected capabilities
|
|
// - Interface testing - works with any llms.Model implementation
|
|
// - Simple usage pattern - just pass the model to test
|
|
//
|
|
// # Usage
|
|
//
|
|
// Testing an LLM implementation is straightforward:
|
|
//
|
|
// func TestMyLLM(t *testing.T) {
|
|
// llm, err := mylllm.New()
|
|
// if err != nil {
|
|
// t.Fatal(err)
|
|
// }
|
|
// llmtest.TestLLM(t, llm)
|
|
// }
|
|
//
|
|
// # Automatic Capability Discovery
|
|
//
|
|
// The package automatically detects and tests supported capabilities:
|
|
// - Basic operations (Call, GenerateContent)
|
|
// - Streaming (if model implements streaming interface)
|
|
// - Tool/Function calling (probed with test tool)
|
|
// - Reasoning/Thinking mode (if supported)
|
|
// - Token counting (if usage information provided)
|
|
// - Context caching (if implemented)
|
|
//
|
|
// # Mock Implementation
|
|
//
|
|
// A MockLLM is provided for testing without making actual API calls:
|
|
//
|
|
// mock := &llmtest.MockLLM{
|
|
// CallFunc: func(ctx context.Context, prompt string, options ...llms.CallOption) (string, error) {
|
|
// return "mocked response", nil
|
|
// },
|
|
// }
|
|
// llmtest.TestLLM(t, mock)
|
|
//
|
|
// # Parallel Testing
|
|
//
|
|
// All tests run in parallel by default for better performance:
|
|
// - Core tests (Call, GenerateContent) run concurrently
|
|
// - Capability tests run in parallel when detected
|
|
// - Safe for concurrent execution with independent contexts
|
|
//
|
|
// # Provider Coverage
|
|
//
|
|
// The package is used to test all LangChain Go providers:
|
|
// anthropic, bedrock, fake, googleai, huggingface, mistral, ollama, openai, and more.
|
|
package llmtest
|