langchain-ai/auto-evaluator
1
0
Fork 0
mirror of https://github.com/langchain-ai/auto-evaluator.git synced 2026-07-19 14:43:34 -04:00
Code Issues 21 Packages Projects Releases Wiki Activity

[PR #14] [MERGED] Keep descriptive outputs #70

New Issue
Closed
opened 2026-02-15 19:14:55 -05:00 by yindo · 0 comments
yindo commented 2026-02-15 19:14:55 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/langchain-ai/auto-evaluator/pull/14
Author: @rlancemartin
Created: 4/20/2023
Status: ✅ Merged
Merged: 4/21/2023
Merged by: @benisgold

Base: main ← Head: keep_descriptive_outputs

📝 Commits (1)

📊 Changes

1 file changed (+2 additions, -2 deletions)

View changed files

📝 api/evaluator_app.py (+2 -2)

📄 Description

answerScore and retrievalScore descriptive outputs can be kept as collapsible columns in the table.

They will be verbose if grade_prompt != "Fast".

You can see the templates here.

GRADE_DOCS_PROMPT will ask the model to explain itself:

template = """ 
    Given the question: \n
    {query}
    Decide if the following retreived context is relevant: \n
    {result}
    Answer in the following format: \n
    "Context is relevant: True or False." \n 
    And explain why it supports or does not support the correct answer: {answer}"""

GRADE_DOCS_PROMPT = PromptTemplate(input_variables=["query", "result", "answer"], template=template)

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/langchain-ai/auto-evaluator/pull/14 **Author:** [@rlancemartin](https://github.com/rlancemartin) **Created:** 4/20/2023 **Status:** ✅ Merged **Merged:** 4/21/2023 **Merged by:** [@benisgold](https://github.com/benisgold) **Base:** `main` ← **Head:** `keep_descriptive_outputs` --- ### 📝 Commits (1) - [`506725e`](https://github.com/langchain-ai/auto-evaluator/commit/506725ef6604b86ab247bfb1714bd36d7a9d85b4) keep descriptive outputs ### 📊 Changes **1 file changed** (+2 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `api/evaluator_app.py` (+2 -2) </details> ### 📄 Description `answerScore` and `retrievalScore` descriptive outputs can be kept as collapsible columns in the table. They will be verbose if `grade_prompt != "Fast"`. You can see the templates [here](https://github.com/dankolesnikov/evaluator-app/blob/1fc1831a06cafc4c43bd3a6c9cb01677ecb6a103/api/text_utils.py#L65). `GRADE_DOCS_PROMPT` will ask the model to explain itself: ``` template = """ Given the question: \n {query} Decide if the following retreived context is relevant: \n {result} Answer in the following format: \n "Context is relevant: True or False." \n And explain why it supports or does not support the correct answer: {answer}""" GRADE_DOCS_PROMPT = PromptTemplate(input_variables=["query", "result", "answer"], template=template) ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-15 19:14:55 -05:00
yindo closed this issue 2026-02-15 19:14:55 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
good first issue
pull-request
Mirrored from GitHub Pull Request
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/auto-evaluator#70
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?