Generate comparison reports
Comparison reports allow you to analyze and compare results across different test runs. This helps you evaluate performance, track improvements, and make data-driven decisions.
Creating a comparison report
Give your comparison report a descriptive name that reflects what you're comparing (e.g., "Comparison for Co-pilot Nov updates").
Choose the runs you want to compare, select multiple runs by clicking the add button next to each run
You can mark a selected run as a base run to compare data against
You can also filter runs using the search bar and filter options
Click the "Create dashboard" button to generate your comparison report.
Understanding the comparison report
The comparison report provides several key metrics and visualizations:
- Summary by Evaluator
- Cost by Prompt used in test run
- Tokens used by the prompts
- Latency metrics
If your comparison report includes a base run, you can view the changes in metrics from the base run to the compared runs.
Updating the comparison report
You can update the comparison report by hovering on the report title section and clicking on the "Edit" button. This allows you to adding new runs to it or removing existing runs or marking another run as a base run.
Sharing the comparison report
Once you create the comparison report, you can share it with others by clicking the "Share report" button on top of the report page.