If you select Latest version - then we will pick the latest session of Prompt or prompt chain at each instance of scheduled run.
How to
Scheduled test runs
Learn how to schedule test runs for your prompts, prompt chains and workflows at a regular interval.
Select Prompt or Workflow or Prompt chain to schedule runs for.
Click on "Test" and select from the test run sheet header.
Pick version
Add name of the schedule, version (applicable for Prompts and Prompt chains) to run this scheduled test on.
Decide schedule of the run
Select schedule (every hour, day etc) along with the starting date time for the run. The granularity of repetition is an hour at the moment. Please reach out to us if you want lower granularity than this.
Select evaluators
As a last step, select evaluators you want to run. You can update this config anytime by editing this schedule.
Once you save, Maxim will run these jobs at the given cadence. You can pause and resume these runs anytime. All the scheduled runs will be available in the Runs tab. You can filter these runs by the schedule name on the test run report table.
Evaluate Datasets
Learn how to evaluate your AI outputs against expected results using Maxim's Dataset evaluation tools
Trigger Test Runs using SDK
Learn how to programmatically trigger test runs using Maxim's SDK with custom datasets, flexible output functions, and evaluations for your AI applications.