Adding evaluators post run

By default, you can select evaluators to run while testing a prompt or workflow. However, there could be cases where you missed an evaluator or want to run additional evaluation after seeing the first set of results. To help you do this without having to re-run, we allow adding new evaluators to existing test run reports without modifying the original setup. This can save you time and cost by running the entries only for newly added evaluators instead of everything running again.

To add evaluators to an existing run, follow the steps below:

Navigate to the run entries table at the bottom of the report.

Click on the ‘add evaluator’ button on the table header

In the sheet that opens, choose the evaluators you want to add and confirm

All entries will be run only for these evaluators and results will start getting added as they are completed.

You can also add human evaluators via this process and set up the method and details for the same via the popover that is shown post confirmation