Curate a golden Dataset from Human Annotation
Learn how to curate a golden Dataset for human annotation
Creating golden datasets is essential for scaling your application effectively. Maxim allows you to curate high-quality datasets directly from human annotations as your application evolves.
Follow these steps to curate dataset entries from human annotations:
Set up a test run on a prompt or workflow and send the results to human raters for annotation. Learn more about human-in-the-loop evaluation in our evaluation guide.
Navigate to the test run report after collecting human ratings.
Locate the human evaluation card in the summary section, which shows rater emails and completion status.
Click the "View Details" button next to completed raters' emails to access their detailed ratings.
Navigate to the test run report after collecting human ratings.
Locate the human evaluation card in the summary section showing rater emails and completion status.
Click the View details button next to completed raters' emails to access their detailed ratings.
Review the ratings, comments, and human-corrected outputs where available.
Select the entries you want to preserve using the row checkboxes, then click the "Add to Dataset" button at the top.
Select your target Dataset and map the relevant data to appropriate columns. For example, map human-corrected outputs to ground truth columns in your golden ataset. - Uncheck any columns you don't want to include in the dataset.