langchain-ai/docs
1
0
Fork 0
mirror of https://github.com/langchain-ai/docs.git synced 2026-08-24 12:23:02 -04:00
Code Issues 490 Packages Projects Releases Wiki Activity

[GH-ISSUE #1504] DOC: Defining evaluation "success" #14

New Issue
Open
opened 2026-02-17 17:18:59 -05:00 by yindo · 1 comment
yindo commented 2026-02-17 17:18:59 -05:00
Owner
Copy Link
Copy Source

Originally created by @angadn on GitHub (Jan 28, 2025).
Original GitHub issue: https://github.com/langchain-ai/docs/issues/1504

This quickstart now shows that evaluator callables can return a dict, optionally naming the metric with the key field. That's great, and saves me the weird fn.__name__ override I was doing previously

However, I notice the UI shows a success badge, and I'm not sure what it means - does this mean no exceptions were thrown, or am I able to somehow define thresholds for each metric to consider an evaluation as a success?

Is this somehow possible with the dict too? I'd expect evaluate/aevaluate to take some kind of lambda that sets the success field, but can't find something to this effect

Originally created by @angadn on GitHub (Jan 28, 2025). Original GitHub issue: https://github.com/langchain-ai/docs/issues/1504 This [quickstart](https://docs.smith.langchain.com/old/evaluation/quickstart) now shows that evaluator callables can return a `dict`, optionally naming the metric with the `key` field. That's great, and saves me the weird `fn.__name__` override I was doing previously However, I notice the UI shows a **success** badge, and I'm not sure what it means - does this mean no exceptions were thrown, or am I able to somehow define thresholds for each metric to consider an evaluation as a success? Is this somehow possible with the dict too? I'd expect `evaluate`/`aevaluate` to take some kind of lambda that sets the `success` field, but can't find something to this effect
yindo added the langsmithexternal labels 2026-02-17 17:18:59 -05:00
yindo commented 2026-02-17 17:18:59 -05:00
Author
Owner
Copy Link
Copy Source

@SeveHo commented on GitHub (Apr 9, 2025):

Same question here

@SeveHo commented on GitHub (Apr 9, 2025): Same question here
yindo changed title from DOC: Defining evaluation "success" to [GH-ISSUE #1504] DOC: Defining evaluation "success" 2026-06-05 17:24:34 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
deepagents
docs-infra
external
external
help wanted
integration
internal
internal
javascript
js/ts
langchain
langgraph
langsmith
needs-title
pull-request
Mirrored from GitHub Pull Request
 python
reference
No Label external langsmith
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/docs#14
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?