run-llama/llama-datasets
1
0
Fork 0
mirror of https://github.com/run-llama/llama-datasets.git synced 2026-06-30 21:27:58 -04:00
Code Issues 3 Packages Projects Releases Wiki Activity

update readme (#47)

Browse Source
This commit is contained in:
Andrei Fajardo
2023-12-19 13:57:51 -05:00
committed by GitHub
parent da50477940
commit 3b5cc1593c
1 changed files with 10 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+10
View File
@@ -53,8 +53,18 @@ The high-level steps are:
4. Submit a Pull Request to this repo to check in the metadata
5. Submit a Pull Request to the [llama-datasets repository](https://github.com/run-llama/llama-datasets) to check in the `LabelledRagDataset` and the source files
(NOTE: you can use the above process for submitting any of our other supported
types of llama-datasets such as the `LabelledEvaluatorDataset`.)
## Usage Pattern
(NOTE: in what follows we present the pattern for producing a RAG benchmark with
the `RagEvaluatorPack` over a `LabelledRagDataset`. However, there are also other
types of llama-datasets such as `LabelledEvaluatorDataset` and corresponding llama-packs
for producing benchmarks on their respective tasks. They all follow the similar
usage pattern. Please refer to the README's to learn more on each type of
llama-dataset.)
As mentioned earlier, llama-datasets are mainly used for evaluating RAG systems.
To perform the evaluation, the recommended usage pattern involves the application of the
`RagEvaluatorPack`. We recommend reading the [docs](https://docs.llamaindex.ai/en/stable/module_guides/evaluating/root.html)