Commit Graph

183 Commits

Author SHA1 Message Date
Dmitry Ng 064f09155c chore(deps): bump Go toolchain to 1.26.5 and fix vulnerable dependencies
- Raised the `go` directive from 1.24.1 to 1.26.5 to satisfy pgx/v5 and ollama's minimum Go version requirements and to pick up 15 patched standard library CVEs.
- Updated `pgx/v5`, `otel`/`otel/sdk`/`otlptracehttp`, `x/net`, and the AWS `eventstream` protocol package to their latest requested versions.
- Remediated govulncheck-flagged CVEs in `grpc`, `x/text`, `x/crypto`, `antchfx/xpath`, AWS SDK `bedrock*`/`s3` services, `etcd/server`, and `mongo-driver` v1/v2.
- Fixed ollama's `MainGPU` API type change (`int` -> `*int`) in `llms/ollama/options.go` without breaking the public `WithRunnerMainGPU` signature.
2026-08-04 13:57:26 +03:00
Dmitry Ng 1fc129969f feat(llms): add provider-neutral structured output via WithStructuredOutput
Add a single WithStructuredOutput(StructuredOutputConfig) call option
carrying a raw JSON Schema (Draft 2020-12), so one contract yields the
same guarantee — a normal-final response is a single JSON value matching
the schema — on OpenAI, Anthropic, Bedrock (Converse and legacy), Google
AI, Vertex, and Ollama. Each provider maps it to its native mechanism;
the result is validated locally against the original schema in the new
llms/structuredoutput package. Also fixes a latent Converse gap where the
response stop reason was never surfaced on ContentChoice.StopReason.
2026-07-17 04:39:05 +03:00
Dmitry Ng a17a86c171 chore: update golangci-lint version and refactor test cases
- Upgraded golangci-lint from version v2.1.6 to v2.8.0 in the CI workflow for improved linting capabilities.
- Refactored test cases in conversational_test.go to streamline error handling and improve readability.
- Enhanced predictOptions initialization in conversational.go and mrkl.go for better performance and clarity.
- Updated various agent files to optimize memory allocation and improve overall code efficiency.
- Added deferred response body closure in transport_test.go to ensure proper resource management.
2026-02-22 21:31:38 +03:00
Dmitry Ng 59b196e98a feat: enhance reasoning and caching support across various LLMs
- Updated reasoning handling in the Anthropic and OpenAI clients to support real-time streaming of reasoning tokens.
- Refactored reasoning structures to improve integration with existing models, ensuring consistent behavior across different LLM implementations.
- Added comprehensive tests for reasoning and caching functionalities, validating the expected behavior in various scenarios.
- Removed deprecated files related to previous caching implementations to streamline the codebase.

This update improves the overall performance and reliability of reasoning capabilities in LLM interactions.
2026-01-27 22:56:49 +03:00
Dmitry Ng aa2dd71269 Merge branch 'main' into release/v0.1.14-update.1 2026-01-24 02:07:00 +03:00
Travis Cline 334f4c3689 vectorstores/milvus: complete migration to new SDK v2 client (#1403)
* vectorstores/milvus: complete migration to new SDK v2 client (#1397)

Complete the migration from the archived milvus-sdk-go/v2 to the new
github.com/milvus-io/milvus/client/v2 SDK as tracked in issue #1397.

**New Implementation**:
- Add complete vectorstores/milvus/v2/ package with new SDK client
- Implement all core vectorstore operations (Add, Search, Delete, etc.)
- Add comprehensive test suite with unit and integration tests
- Include migration example and documentation

**Key Changes**:
- New milvus.go with updated client initialization and operations
- Updated options.go with v2 SDK configuration patterns
- Added example_migration.go demonstrating upgrade path
- Comprehensive README.md with migration guide
- Updated go.mod/go.sum with new SDK dependencies

**Documentation**:
- Add docs/package-lock.json for documentation build dependencies
- Provide clear migration path from v1 to v2 implementation
- Maintain backward compatibility where possible

This completes the migration work started in earlier commits and provides
a full replacement for the deprecated SDK while maintaining the same
vectorstore interface.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* style: apply gofmt formatting to v2 package files

* fix: remove unused async field from v2 Store struct

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-15 14:45:48 -07:00
Travis Cline 601dc553d5 docs: fix API references and complete integration documentation (#1402)
* docs: fix outdated API references and examples across documentation

Update documentation to reflect current API patterns and fix outdated code examples:

- Update Error struct definition with Provider and Details fields in architecture docs
- Fix code examples to use CallOptions (WithTemperature, WithMaxTokens) instead of constructor options
- Update chain execution pattern from chain.Run() to chains.Run()
- Add context parameter to memory interface methods
- Update Vertex AI examples to use new googleai/vertex package structure
- Change from vertexai.New() to vertex.New() with updated option names
- Fix import paths for Google AI and Vertex AI integration
- Update minimum Go version requirement from 1.21 to 1.23

These changes ensure documentation examples work with the current codebase and provide accurate guidance for developers.

* docs: expand integration documentation and fix remaining API references

- Add comprehensive examples to chat and embeddings integration pages
- Fix missing context imports in LLM provider configuration
- Update Vertex AI examples to use correct vertex.New() syntax
- Fix broken link in CloudSQL vectorstore README
- Transform placeholder pages into useful developer resources

* docs: fix relative link in embeddings integrations page
2025-09-14 23:37:03 +02:00
Travis Cline 0cc366260b vectorstores/milvus: update deprecation notice for archived SDK (#1398)
* vectorstores/milvus: update deprecation notice for archived SDK

The Milvus SDK at github.com/milvus-io/milvus-sdk-go/v2 has been archived.
Updated the deprecation notice to inform users and reference issue #1334
which tracks the migration to the new SDK at github.com/milvus-io/milvus/client/v2.

The migration will be done in a way that maintains backward compatibility
to avoid breaking existing users.

Updates #1334

* vectorstores/milvus: update deprecation notice with archival date and issue reference

Add specific archival date (March 21, 2025) to the deprecation notice and clarify
that migration to the new SDK will require breaking changes in a future version.
Update issue reference from #1334 to #1397 for migration tracking.

Updates #1397
2025-09-14 20:20:44 +02:00
Travis Cline 480112bee0 internal/httprr, vectorstores/azureaisearch: fix race conditions in tests (#1381)
test: fix race conditions in httprr and azureaisearch tests

Add thread-safe access to global record flag in httprr package and prevent
tests that modify global state from running in parallel. Fix race condition
in Azure AI Search tests by properly capturing request counter in a local
variable before use.
2025-08-29 15:48:17 +02:00
Travis Cline 6c217b3865 test: improve test coverage and optimize storage with gzip compression (#1378)
* embeddings/vertexai,memory: remove deprecated PaLM test and add recordings

Remove deprecated VertexAI PaLM embeddings test (PaLM API deprecated August 2024).
Add missing test recordings for memory/token buffer tests.

* vectorstores/redisvector: regenerate test recordings and compress

Regenerate test recordings for Docker container compatibility.
Compress large recordings (>50KB) with gzip for space efficiency.

* vectorstores/chroma: regenerate test recordings and compress

Regenerate test recordings for Docker container compatibility.
Compress large recordings (>50KB) with gzip for space efficiency.

* vectorstores/milvus: add deprecation notice and compress recordings

Add deprecation notice for Milvus vectorstore.
Compress test recordings with gzip for space efficiency.

* vectorstores/opensearch: regenerate test recordings and compress

Regenerate test recordings for Docker container compatibility.
Fix test to properly set up service_role for local tests.
Compress large recordings with gzip for space efficiency.

* vectorstores/weaviate: regenerate test recordings and compress

Regenerate test recordings for Docker container compatibility.
Compress large recordings (>50KB) with gzip for space efficiency.

* vectorstores/{pgvector,alloydb,cloudsql}: regenerate and compress recordings

Regenerate test recordings for Docker container compatibility.
Compress large recordings (>50KB) with gzip for space efficiency.
Covers pgvector, AlloyDB, and CloudSQL PostgreSQL vectorstores.

* tools/wikipedia: regenerate test recordings

Update test recordings with current Wikipedia API responses.
2025-08-23 19:06:20 +02:00
Matt Dale 78296871ac mongovector: fix expected error string in non-tokenized filter test
Simple fix for error string mismatch in test
2025-08-19 18:33:08 +02:00
Travis Cline 99951bca92 httprr: improve test recording stability and add utilities (#1368)
* build: add examples-updater tool and update examples to v0.1.14-pre.1

Add a new tool to automate updating langchaingo version references in example
projects and use it to update all examples to v0.1.14-pre.1. The tool also
removes temporary replace directives that are no longer needed.

- Add update-examples Makefile target that runs the new tool
- Create internal/devtools/examples-updater implementation
- Update all example go.mod files to reference v0.1.14-pre.1
- Remove replace directives in googleai and vertex examples
- Update dependencies in main go.mod/go.sum

* httprr: improve header normalization and test container setup

Add header normalization to make HTTP recordings more stable:
- Add normalizeGoogleAPIClientHeader and normalizeVersionHeader functions
- Normalize User-Agent, x-goog-api-client and other version headers
- Remove OpenAI-Project header for consistency
- Preserve body for both replay lookup and recording
- Add comprehensive tests for header normalization

Improve testcontainer environment detection:
- Better handling of non-standard Docker socket paths
- Disable Ryuk reaper by default for resource efficiency
- Add verbose logging option via environment variable

* httputil: add ApiKeyTransport for query parameter API keys

Add a reusable ApiKeyTransport that adds API keys to URL query parameters:
- Create ApiKeyTransport implementation in httputil package
- Add comprehensive tests for the new transport
- Refactor googleai_test.go to use the shared transport
- Update chains/llm_test.go to use ApiKeyTransport with proper key scrubbing

This improves how API keys are handled with client libraries that don't
properly set keys when using custom HTTP clients, particularly useful
with httprr for testing Google API integrations.

* openai: remove duplicate MaxTokens field assignment

Remove redundant assignment where both MaxTokens and MaxCompletionTokens
were being set to the same value. MaxCompletionTokens is the preferred
field name that reflects OpenAI's API changes, while MaxTokens was
previously kept for backward compatibility.

* vectorstores/maridadb: add test infrastructure for MariaDB vectorstore

Add TestMain function for MariaDB vectorstore tests that ensures the proper
test environment is set up using the testctr package. This enables consistent
test container setup and teardown for MariaDB integration tests.

* all: normalize all httprr recordings for consistency

Update test recordings across all packages to use normalized headers:
- Standardize User-Agent to 'langchaingo-httprr'
- Normalize version information in x-goog-api-client headers
- Remove OpenAI-Project headers for consistency
- Fix deprecated Anthropic completion API tests
- Update HuggingFace test recordings with valid responses

These changes make test recordings stable across dependency updates
and different environments, preventing test failures from version changes.

* devtools: add utility tool for normalizing httprr test recordings

Add a command-line tool that standardizes version information in httprr recordings:
- Normalizes x-goog-api-client headers to use placeholder versions
- Standardizes x-amz-user-agent headers for consistency
- Replaces Go version strings with generic placeholders
- Supports dry-run mode to preview changes without modifying files
- Includes verbose output option for detailed change reporting

This tool helps maintain consistent test recordings across different environments
and dependency versions, preventing test failures from version changes.

* llms/openai: update OpenAI embedding test to use text-embedding-3-small model

Update the embedding test to use the current text-embedding-3-small model:
- Change model from text-embedding-ada-002 to text-embedding-3-small
- Adjust dimensions parameter from 1234 to 256 to match model capabilities
- Update test recording with appropriate response data

This change ensures tests remain compatible with OpenAI's current embedding
models and prevents test failures from API version changes.
2025-08-18 19:41:52 +02:00
Travis Cline aba0732988 vectorstores: add MariaDB and Dolt vector store implementations (#1363)
* vectorstores: add MariaDB and Dolt vector store implementations

Add new vector store implementations for MariaDB and Dolt databases.
Both support similarity search with metadata filtering and score thresholds.

From PR #1147
Co-authored-by: Dustin Brown <coffeegoddd@users.noreply.github.com>

* go.mod: update testcontainers for MariaDB vector store tests

Update testcontainers-go and testcontainers-go/modules/mariadb to v0.38.0
for MariaDB vector store test support.

* vectorstores: fix database handling in MariaDB and Dolt implementations

Change value receivers to pointer receivers for init() and createOrGetDatabase()
methods to ensure proper state updates. Refactor database creation logic to first
check if database exists before creating or updating, replacing the previous
INSERT ON DUPLICATE KEY approach with a more explicit flow.

---------

Co-authored-by: Dustin Brown <coffeegoddd@users.noreply.github.com>
2025-08-16 19:43:17 +02:00
Dmitry Ng c89af96902 fix: update chroma-go dependency to v0.2.3 and refactor tests to remove OpenAI API key usage there 2025-07-12 15:21:45 +03:00
Dmitry Ng a87f33d1a8 fix: lint and test issues, freeze docker images versions 2025-07-12 15:05:38 +03:00
Dmitry Ng 6317685b67 feat: integrate streaming functionality across LLMs and callbacks
- Updated LLMs and callback interfaces to utilize a new streaming package for handling content chunks.
- Changed function signatures to accept streaming.Chunk instead of byte slices for better type safety and clarity.
- Enhanced examples and tests to demonstrate the new streaming capabilities, ensuring compatibility with existing functionality.
- Added reasoning and tool call handling in streaming responses for improved processing of LLM outputs.
2025-06-23 18:52:06 +03:00
Dmitry Ng b870b20a91 refactor: update Redis vectorstore example and tests
- Updated README to reflect the new Ollama models "gemma3:4b" and "gemma:2b".
- Refactored `getEmbedding` function to use constants for model names.
- Enhanced test container setup for Redis, Chroma, OpenSearch, and Weaviate with improved wait strategies for readiness.
- Removed unused code and streamlined test URI retrieval logic.
2025-06-23 15:57:51 +03:00
Dmitry Ng eee90d9059 fix: multiple linter issues and logic in tests 2025-06-23 13:43:06 +03:00
Dmitry Ng 019056baba feat: setup repository for fork 2025-06-22 02:27:21 +03:00
Dmitry Ng 6da7298807 fix: GoogleAI httprr dumps 2025-06-22 02:21:19 +03:00
Dmitry Ng 1fba44588a Merge branch 'pr/613’ into main-pull-requests
Squashed commit of the following:

Author: filepang <filepang@gmail.com>
Date:   Tue Feb 27 16:18:58 2024 +0900

    refactor: remove unnecessary math32.Pow call
    refactor: Use math32 instead of cast to float64

Author: filepang <filepang@gmail.com>
Date:   Tue Feb 20 18:36:27 2024 +0900

    examples: Add inmemory vectorstore example
    vectorstore: Add initial implementation of inmemory vectorstore
2025-06-22 02:03:40 +03:00
Dmitry Ng fbd4bc67ec fix: open issues in the tests 2025-06-22 01:55:59 +03:00
Barthélémy Antonin 4298f6fb0d vectorstores: add WithEndpoint option for Azure AI Search (#1308) 2025-06-21 15:31:24 +02:00
Travis Cline 77504b877f all: expand test coverage (#1312)
* scripts: cleanup test scripts
* lint: add test pattern and architectural linting
* internal/httprr: expand httprr testing
* test: add comprehensive unit test coverage
* docs: add comprehensive testing guide to CONTRIBUTING.md
* examples: standardize module versions and cleanup dependencies
* all: re-record several httprr recordings
2025-06-16 18:14:02 +02:00
Travis Cline c544fb77bd all: add broad httprr coverage, update dependencies, organize go.mod file, bump to 1.23 (#1299)
* all: add broad httprr coverage, update dependencies, organize go.mod file, bump to 1.23

update go version to 1.23
add lots of test coverage via httprr recordings
update dependencies and organize go.mod
add testutil/testctr which helps work around a testcontainers-go+colima bug
expand the huggingface implementation and tests
expand capabilities of the ollama package
2025-06-04 11:41:45 -07:00
Luca Ronca d89fe333df vectorstores: add Bedrock Knowledge Bases vectorstore support (#1200)
* feat: add Aws Bedrock Knowledge Bases vectorstore support

* doc: add comments to Filter exported symbol
2025-05-13 03:39:22 -07:00
Averi Kitsch 017409785d vectorstore: Add interfaces for Google AlloyDB and Cloud SQL (#1204)
* feat(google): Add Google AlloyDB and Cloud SQL vectorstore and chat history

* vectorstore: Add interfaces for Google AlloyDB and Cloud SQL
2025-04-17 13:03:52 -07:00
Manuel de la Peña 8d4c17fc73 ci: migrate linting golangci-lint to v2.0.2 (#1217)
* chore: use slices and maps packages

* chore: skip embedded type

* chore: use fmt.Fprint instead

* chore: use switch

* chore: reorder imports

* chore: add white lines

* chore: use t.Setenv

* chore: excplicitly omit non used param

* chore: use t.TempDir

* chore: use maps package

* chore: migrate golangci-lint config file

Run: "golangci-lint migrate -c .golangci.yaml --skip-validation"

* chore: bump golangci-lint action to v7
2025-04-12 01:54:50 -07:00
Preston Vasquez 8a5f1ed36c GODRIVER-3336 Remove readme in favor to go docs 2025-01-29 10:27:58 -07:00
Preston Vasquez d686479379 Merge branch 'main' into GODRIVER-3336 2025-01-29 10:09:40 -07:00
Aksel df268eb9d8 Merge pull request #1025 from prestonvasquez/GODRIVER-3345
test: containerize mongovector tests
2025-01-29 20:00:17 +05:30
Tai Groot e0b354c3b0 bugfix/vectorstores: weaviate: use correct client creator 2025-01-20 17:52:39 -08:00
Manuel de la Peña 47f0208d20 chore(vectorstores:pinecone): deprecate not-used error 2025-01-07 08:48:40 +01:00
Manuel de la Peña eff3cb8fa0 fix(vectorstores:pinecone): do not return error when zero documents are retrieved 2025-01-07 08:48:26 +01:00
Manuel de la Peña 6bce8b1862 fix: pre-allocate with items' length 2024-11-28 19:56:45 +01:00
Manuel de la Peña 294f2e750f fix(vectorstores:weaviate): do not return error when zero documents are retrieved 2024-11-28 16:49:35 +01:00
Preston Vasquez 17fccc9ce7 Add README.md and example 2024-11-08 11:44:35 -07:00
avi tal 238d1c713d Fixes #974 - returning score driven by cosine similarity by (1 - distance) instead of distance (#1048)
Fixing the returned value of cosine similarity by (1 - distance)

Co-authored-by: avi.tal <avi.tal@appsflyer.com>
2024-10-23 23:09:51 -07:00
Preston Vasquez eff62dc27a Increase warm up time 2024-09-19 15:00:05 -06:00
Preston Vasquez f8140daf53 GODRIVER-3345 Remove print statement 2024-09-18 09:06:02 -06:00
Preston Vasquez 48fc115df8 Remove empty line 2024-09-17 11:56:14 -06:00
Preston Vasquez f46f0dea1c gofumpt test file 2024-09-17 11:47:05 -06:00
Preston Vasquez 80f05b8dbb Resolve linting errors 2024-09-17 11:44:09 -06:00
Preston Vasquez aebbc204c5 Containerize mongovector tests 2024-09-17 11:34:32 -06:00
Travis Cline bf1667feaf lint: fixup lint settings (#1018) 2024-09-13 04:39:28 +00:00
Preston Vasquez 66d7710b63 vectorstores: add mongovector (#1005)
* mongovector: Add mongo vectorstore implementation

---------

Co-authored-by: Travis Cline <travis.cline@gmail.com>
2024-09-13 03:01:07 +00:00
fatedier 0cab9dfd4e milvus: add skipFlushOnWrite option (#992)
milvus: add disableFlush option
2024-09-12 17:25:06 -07:00
Andrew Crump 47d2d99add redisvector: fix score threshold option (#1003)
Vector range queries expect a radius rather than a score threshold.
2024-09-12 17:24:27 -07:00
Travis Cline 4c89af684b Fix up weaviate tests (#885)
* weaviate: Fix weaviate tests, openai embedding model threading

* testcontainers: Update to 0.31
2024-06-15 03:52:42 +00:00
Adrian Cole 17d5a7a537 readme: add scheme to vectorstores/chroma URL (#846)
Fix drift in vectorstores/chroma README.md

Perhaps this worked before, but now if you omit the protocol in the URL, you'll get an error like this:

```
2024/05/21 10:15:43 new: Get "localhost:8000/api/v1/heartbeat": unsupported protocol scheme "localhost"
```
2024-06-14 22:51:56 -04:00