Using a dataset for evaluation

Attaching datasets while evaluating entities

Select a dataset in the test configuration menu to evaluate prompts, workflows, and prompt chains.

Upon clicking the test buttons in either of these flows a right side panel of test config menu appears, where you can select your dataset to be used for testing.

As demonstrated in the video, in the final report:

1. The input from the dataset serves as a query to the model.
2. If a variable is referenced from the dataset, that column replaces the variable values for each entry.
3. If evaluators chosen require expected output (ground truth), we take this data from the dataset and compare it with the actual generated output.

Evaluating datasets on their own

If you have a dataset with an output column populated, you can run evaluations directly on this data. This could be in cases where you have already run the cases and have the LLM output ready for evaluation.

For such cases, click the 'test' button in the top header of the dataset.

1. We will consider the input and output columns for evaluation.
2. If you would also like to evaluate context, you can select the context column while configuring the test run
3. You can select evaluators which needs expected output if you have that column set up in the dataset.