[GH-ISSUE #40] Add instructions to the demo landing page #23

New Issue

2026-02-15T18:17:04-05:00

yindo commented

2026-02-15 18:17:04 -05:00

Originally created by @rlancemartin on GitHub (Apr 23, 2023).
Original GitHub issue: https://github.com/langchain-ai/auto-evaluator/issues/40

Originally assigned to: @benisgold on GitHub.

Top (Text box, grey background)

Welcome to the auto-evaluator! This is an app to evaluate the performance of question-answering LLM chains. This demo has pre-loaded two things: (1) a document (the Lex Fridman podcast with Andrej Karpathy) and (2) a "test set" of question-answer pairs for this episode. The aim to evaluate the performance of various question-answering LLM chain configuration against test set. You can build any QA chain using on the components and score its performance.

Button (Text box, green background)

Choose the question-answering chain configuration (left) and launch an experiment using the button below. For more detail on each setting, see full the documentation here.

Color: green
Title: run experiment

Summary

Re-name initial row as baseline

Experiment Results (Text box, grey background)

This table shows the each question-answer pair from the test set along with the model's answer to the question. The app will score two things: (1) the relevance of the retrieved documents relative to the question and (2) the similarity of the LLM generated answer relative to ground truth answer. The prompts for both can be seen here and can be chosen by the user in the drop-down list Grading prompt style. The FAST prompt will only have the LLM grader output the score. The other prompts will also produce an explanation.

Originally created by @rlancemartin on GitHub (Apr 23, 2023). Original GitHub issue: https://github.com/langchain-ai/auto-evaluator/issues/40 Originally assigned to: @benisgold on GitHub. `Top` (Text box, grey background) Welcome to the auto-evaluator! This is an app to evaluate the performance of question-answering LLM chains. This demo has pre-loaded two things: (1) a document (the Lex Fridman podcast with Andrej Karpathy) and (2) a "test set" of question-answer pairs for this episode. The aim to evaluate the performance of various question-answering LLM chain configuration against test set. You can build any QA chain using on the components and score its performance. `Button` (Text box, green background) Choose the question-answering chain configuration (left) and launch an experiment using the button below. For more detail on each setting, see full the documentation [here](https://github.com/dankolesnikov/auto-evaluator-app). * Color: green * Title: run experiment `Summary` * Re-name initial row as `baseline` `Experiment Results` (Text box, grey background) This table shows the each question-answer pair from the test set along with the model's answer to the question. The app will score two things: (1) the relevance of the retrieved documents relative to the question and (2) the similarity of the LLM generated answer relative to ground truth answer. The prompts for both can be seen [here](https://github.com/dankolesnikov/evaluator-app/blob/main/api/text_utils.py) and can be chosen by the user in the drop-down list `Grading prompt style`. The `FAST` prompt will only have the LLM grader output the score. The other prompts will also produce an explanation.

yindo closed this issue

2026-02-15 18:17:04 -05:00

yindo referenced this issue

2026-06-05 17:14:00 -04:00

[GH-ISSUE #23] refactor Body.tsx into multiple sub react components #10

yindo changed title from ~~Add instructions to the demo landing page~~ to [GH-ISSUE #40] Add instructions to the demo landing page

2026-06-05 17:14:04 -04:00

Sign in to join this conversation.