langchain-ai/docs
1
0
Fork 0
mirror of https://github.com/langchain-ai/docs.git synced 2026-07-22 12:25:55 -04:00
Code Issues 490 Packages Projects Releases Wiki Activity

[GH-ISSUE #1481] How to add reference contexts on evaluation dataset? #56

New Issue
Open
opened 2026-02-17 17:19:05 -05:00 by yindo · 0 comments
yindo commented 2026-02-17 17:19:05 -05:00
Owner
Copy Link
Copy Source

Originally created by @Donovan0243 on GitHub (Aug 26, 2025).
Original GitHub issue: https://github.com/langchain-ai/docs/issues/1481

When I try to use Langsmith evaluation function to evaluate my RAG system, especially the retrieval steps, I wanna test the precision and recall.
To test these metrics, I need reference docs and retrieval docs, then calculate them.

I know that I can get my retrieval docs from output, but how can I get the reference docs? My idea is that when I define my test dataset, I can add the reference_context just like what we did for ground truth, but it can not be taken from the evaluator.

Image Image
Originally created by @Donovan0243 on GitHub (Aug 26, 2025). Original GitHub issue: https://github.com/langchain-ai/docs/issues/1481 When I try to use Langsmith evaluation function to evaluate my RAG system, especially the retrieval steps, I wanna test the precision and recall. To test these metrics, I need reference docs and retrieval docs, then calculate them. I know that I can get my retrieval docs from output, but how can I get the reference docs? My idea is that when I define my test dataset, I can add the reference_context just like what we did for ground truth, but it can not be taken from the evaluator. <img width="1515" height="641" alt="Image" src="https://github.com/user-attachments/assets/f6ccb5ea-9960-4d50-9b49-d3489e037a69" /> <img width="1219" height="165" alt="Image" src="https://github.com/user-attachments/assets/9c442aef-ec57-4e96-8813-a4304b3ed1ee" />
yindo added the external label 2026-02-17 17:19:05 -05:00
yindo changed title from How to add reference contexts on evaluation dataset? to [GH-ISSUE #1481] How to add reference contexts on evaluation dataset? 2026-06-05 17:24:47 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
deepagents
docs-infra
external
external
help wanted
integration
internal
internal
javascript
js/ts
langchain
langgraph
langsmith
needs-title
pull-request
Mirrored from GitHub Pull Request
 python
reference
No Label external
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/docs#56
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?