Agentic evaluation

The Maxim SDK allows attaching evaluators to hierarchical entities, such as traces, spans, and their components (e.g., retrievals and generations). Once the required variables for an evaluator are provided via withVariables, the evaluation automatically begins. Results are visible on the Evaluations Tab inside the trace details sheet.

Setting up evaluators using SDK

This section will provide an understanding of how to attach evaluators, pass variables, and interpret the evaluation results within Maxim.

Attaching evaluators to entities

Evaluators can be attached to a trace, any span inside the trace, or any component within a span. The withEvaluators method is used to attach evaluators to an entity.

trace.evaluate.withEvaluators("clarity", "toxicity");
span.evaluate.withEvaluators("relevance");
retrieval.evaluate.withEvaluators("containsValidURL");
generation.evaluate.withEvaluators("bias", "clarity");

Providing variables to evaluators

Once evaluators are attached to a component, variables can be passed to them via the withVariables method. This method accepts a key-value pair of variables and their values.

trace.evaluate.withVariables(
	{ output: assistantResponse.choices[0].message.content }, // Key-value pair of variables
	["clarity", "toxicity"], // List of evaluators
);

This method sends the specified key-value pairs to the evaluators, ensuring they have the data required to process the evaluation.

trace.evaluate
    .withEvaluators("clarity", "toxicity")
    .withVariables({
    	input: userInput,
    });

Viewing evaluation results on evaluations tab

To monitor and analyze evaluations, navigate to the Evaluations tab in the trace sheet. This tab presents a hierarchical view of entities with evaluations.

• Entity tree (on left): Displays entities (traces, spans, components) in a structured hierarchy enabling intuitive navigation.
• Overview tab (per entity): A concise summary of all the evaluator results for the entity along with rewritten outputs (if available). Includes:
  • Evaluator name
  • Pass/Fail result
  • Score

Navigating individual evaluator details

Clicking on an evaluator within the overview panel opens a detailed view with the following insights:

• Variables used: Displays the list of variables and their values used to evaluate the entity.
• Logs: Shows evaluation logs.
• Time taken: Time taken to evaluate the evaluator (also provides model latency if present).
• Reasoning (if available): Evaluator's reasoning, if present.
• Cost and Tokens used (if available): Breakdown of resource usage for the evaluation.
• Human annotation scores breakdown (if applicable): Shows annotation score per user and comment provided by them. It is only shown for human evaluators.

Code example for agentic evaluation

// ...previous code
 
const generation = trace.generation({
	id: uuid(),
	messages: [
		{
			role: "system",
			content: `You are a helpful assistant that helps people with their questions`,
		},
	],
	model: "gpt-4o",
	provider: "openai",
	modelParameters: {
		temperature: 0.7,
		topP: 1,
		maxTokens: 100,
	},
	name: "user-facing-chatbot",
});
 
// ...user message received
 
generation.addMessages([{
    role: 'user',
    content: userMessage,
}]);
 
generation.evaluate
    .withEvaluators("clarity", "toxicity")
    .withVariables({
    	input: userMessage,
    });
 
// ...retrieve and process context
 
generation.evaluate.withVariables(
    { context: context },
    ["toxicity"] // only toxicity requires context
);
 
// ...generate response
 
generation.result(llmResponse);
 
generation.evaluate.withVariables(
    { output: llmResponse.choices[0].message.content },
    ["clarity", "toxicity"]
);
 
// ...code continues

Best practices

• Use evaluators selectively to monitor key performance metrics.
• Setup sampling and filtering to ensure accurate evaluation processing without eating up too much cost.
• Attach variables reliably to ensure accurate evaluation processing.
• Regularly check the Evaluation Tab for actionable insights directly from production logs.

Conclusion

Agentic evaluation empowers robust monitoring and debugging across nested entities like traces, spans, generations, and retrievals. By leveraging the Maxim SDK’s intuitive methods and analyzing results via the Evaluations Tab, you can ensure the performance and reliability of your AI-powered workflows.