langchain-ai/langsmith-sdk
1
0
Fork 0
mirror of https://github.com/langchain-ai/langsmith-sdk.git synced 2026-07-19 22:33:51 -04:00
Code Issues 125 Packages Projects Releases 16 Wiki Activity

[PR #2056] fix: enforce type in feedback_config, reject unknown fields, add validation tests (#31802) #2041

New Issue
Open
opened 2026-02-15 23:16:19 -05:00 by yindo · 0 comments
yindo commented 2026-02-15 23:16:19 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/langchain-ai/langsmith-sdk/pull/2056
Author: @kanishk2106
Created: 10/5/2025
Status: 🔄 Open

Base: main ← Head: fix/feedback-config-validation-31802

📝 Commits (1)

  • 57d94eb Fix: enforce 'type' in feedback_config, reject unknown fields, add validation tests (#31802)

📊 Changes

2 files changed (+123 additions, -5 deletions)

View changed files

📝 python/langsmith/evaluation/evaluator.py (+29 -5)
➕ python/tests/evaluation/test_feedback_config.py (+94 -0)

📄 Description

Issue Fixed

Fixes langchain-ai/langchain#31802

This PR improves validation in FeedbackConfigModel to ensure stricter feedback configuration and better error handling.

Changes:

  1. Enforce type field as a required field in FeedbackConfigModel.
  2. Add validation to reject unknown fields using a model_validator before model creation.
  3. Update test cases in test_feedback_config.py to cover:
  4. Valid and invalid configurations
  5. Missing type handling
  6. Extra fields detection
  7. Literal enforcement and category validation

Tests
All tests pass successfully:
pytest tests/evaluation/test_feedback_config.py -v
8 passed in 0.02s
All validation scenarios covered (valid, invalid, missing type, extra fields, literal enforcement).

Notes
This change is backward-compatible and designed to work seamlessly alongside existing pydantic.v1 usage in the repository.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/langchain-ai/langsmith-sdk/pull/2056 **Author:** [@kanishk2106](https://github.com/kanishk2106) **Created:** 10/5/2025 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/feedback-config-validation-31802` --- ### 📝 Commits (1) - [`57d94eb`](https://github.com/langchain-ai/langsmith-sdk/commit/57d94eba5eae1a47f25d6cc88e2289fc0c23bf58) Fix: enforce 'type' in feedback_config, reject unknown fields, add validation tests (#31802) ### 📊 Changes **2 files changed** (+123 additions, -5 deletions) <details> <summary>View changed files</summary> 📝 `python/langsmith/evaluation/evaluator.py` (+29 -5) ➕ `python/tests/evaluation/test_feedback_config.py` (+94 -0) </details> ### 📄 Description Issue Fixed Fixes langchain-ai/langchain#31802 This PR improves validation in FeedbackConfigModel to ensure stricter feedback configuration and better error handling. **Changes:** 1. Enforce type field as a required field in FeedbackConfigModel. 2. Add validation to reject unknown fields using a model_validator before model creation. 3. Update test cases in test_feedback_config.py to cover: 4. Valid and invalid configurations 5. Missing type handling 6. Extra fields detection 7. Literal enforcement and category validation **Tests** All tests pass successfully: pytest tests/evaluation/test_feedback_config.py -v 8 passed in 0.02s All validation scenarios covered (valid, invalid, missing type, extra fields, literal enforcement). **Notes** This change is backward-compatible and designed to work seamlessly alongside existing pydantic.v1 usage in the repository. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-15 23:16:19 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Annotation Queue
Bulk Export
Datasets
Evaluators
Experiments
Feature
Migrated
Perf
Playground
Polish
Prompt Hub
TypeScript
User Feature Request
backend
bug
documentation
enhancement
evaluation
frontend
goalie
insights
integrations
integrations
javascript
multi-modal
needs-design
needs-investigation
needs-scoping
nit
opentelemetry
oss
otel
pull-request
Mirrored from GitHub Pull Request
 python
sdk
token tracking
tracing + observability
ui
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langsmith-sdk#2041
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?