Run a Prompt with tool calls
Ensuring your prompt selects the accurate tool call (function) is crucial for building reliable and efficient AI workflows. Maxim's playground allows you to attach your tools (API, code or schema) and measure tool call accuracy for agentic systems.
Tool call usage is a core part of any agentic AI workflow. Maxim's playground allows you to effectively test if the right tools and are being chosen by the LLM and if they are getting successfully executed.
In Maxim, you can create prompt tools within the library section of your workspace. These could be executable or just the schema and then attached to your prompt for testing.
Attach and run your tools in playground
Create a new tool in the library. Use an API or code for executable tools and schema if you only want to test tool choice.
Select and attach tools to the prompt within the configuration section.
Send your prompt referencing the tool usage instructions.
Check the assistant response with tool choice and arguments.
For executable tools, check the tool response message that is shown post execution.
Edit tool type messages manually to test for different responses.
By experimenting in the playground, you can now make sure your prompt is calling the right tools in specific scenarios and that the execution of the tool leads to the right responses.
To test tool call accuracy at scale across all your use cases, run experiments using a dataset and evaluators as shown below.
Measure tool call accuracy across your test cases
Set up your dataset with input and expected tool calls columns.
For each input, add the JSON of one or more expected tool calls and arguments you expect from the assistant.
Trigger a test on the prompt which has the tools attached.
Select your dataset from the dropdown.
Select the tool call accuracy evaluator under statistical evaluators and trigger the run. Add from evaluator store if not available in your workspace.
Once the test run is completed, the tool call accuracy scores will be 0 or 1 based on assistant output.
To check details of the messages click on any entry and click on the messages tab.
