Evaluating Prompt

Single prompt

Learn how to create, configure, and deploy single prompts in Maxim. Explore features like sessions, versioning, variable handling, and SDK integration for efficient prompt engineering and management.

Single prompts in Maxim provide a powerful way to experiment with system and user prompts. This feature allows you to iterate over prompts, test their effectiveness, and ensure they work well before integrating them into more complex workflows for your application.

Working with Single prompts

  1. Create a prompt

    Start by creating a new prompt in the Maxim platform.
  2. Experiment with sessions

    Use sessions to iterate and save your work as you go.
  3. Publish versions

    When satisfied, publish a version of your prompt.
  4. Test and evaluate

    Run tests on your prompt versions using datasets and evaluate the results.
  5. Deploy

    Deploy your prompt versions with specific conditions using tags.
  6. Integrate

    Use the Maxim SDK to access your deployed prompts in your applications.

Creating a prompt

To create a new Single prompt:

Navigate to the "Prompt" tab on the left side panel.

Click on "Prompt" and choose "Single prompt".

Single prompt tab in navigation

Click the "+" icon.

Choose to create a new prompt directly or create a folder for better organization.

Dropdown menu for creating a new prompt

Name your new prompt and optionally assign it to an existing folder.

Creating a new single prompt

Configuring your prompt

Once you've created your prompt, you can configure it to suit your specific needs:

Selecting a model

Maxim supports a wide range of models, including:

  • Open-source models
  • Closed models
  • Custom models

This flexibility allows you to experiment seamlessly across various model architectures.

Model selection interface

Adding system and user prompts

In the prompt editor, you can add your system and user prompts. The system prompt sets the context or instructions for the AI, while the user prompt represents the input you want the AI to respond to.

Prompt editor interface showing system and user prompt fields

Configuring parameters

Each prompt has a set of parameters that you can configure to control the behavior of the model. These parameters are different for each model and can be found in the model's documentation. Here are some examples of common parameters:

  • Temperature
  • Max tokens
  • topP
  • Logit bias
  • Prompt tools (for function calls)
  • Custom stop sequences

Tags

Prompts can also be configured with Tags which are custom metadata that can be used to identify and retrieve prompts through the Maxim SDK.

Note: Tags are a Maxim-specific feature and available for all prompts on the platform only.

Parameter configuration panel

Using tags

Tags make prompts uniquely identifiable and can be used with the Maxim SDK for targeted prompt retrieval. Below is an example of how to assign tags to a prompt:

Example of using tags with the Maxim SDK:

const prompt = await maxim.getPrompt(
	promptId,
	new QueryBuilder()
		.and()
		.deploymentVar("Environment", "test")
		.tag("CustomerId", 1234)
		.tag("grade", "A")
		.tag("test", true)
		.exactMatch()
		.build(),
);

Working with variables and context

Variables and context are essential components in creating dynamic and adaptable prompts. Let's explore how to use them effectively in Maxim.

Using variables in prompts

Maxim allows you to include variables in your prompts using double curly braces {{ }}. This feature is useful for creating flexible prompts that can adapt to different inputs.

Integrating context sources

You can attach your RAG (Retrieval-Augmented Generation) pipeline to prompts using variables. When Maxim encounters a variable in curly braces with a context source attached, it retrieves the context from the RAG pipeline and replaces the variable with the retrieved context.

Static variables

If no context source is attached, Maxim treats the variable as a static value.

  • You can specify the value for static variables in the variables editor while running the prompt in playground.
  • During testing, the value of the static variable is taken from the dataset's column with the same name as the variable.

Versioning and sessions

Maxim introduces a powerful versioning and session system to help you manage your prompt development process effectively.

Working with sessions

Sessions act as a history, saving the entire state of your prompt as you work. This allows you to experiment freely without fear of losing your progress.

Saving sessions

  • You can also use Cmd+S ( Mac) or Ctrl+S (Windows/Linux) to save your current session.
  • Unsaved sessions are marked with a red asterisk beside their name.

Publishing versions

Versions, as the name suggests, represent different releases of your prompt. Each version has its own system (and optionally a user prompt and description).

When you're ready to test your prompt or deploy it:

  1. Click the "Publish version" button.
  2. Optionally include the second message (user prompt) in the version and add a description to track changes and the reasoning behind this version.

Version publishing interface

The first message (typically the system message) is always saved with the version.

Testing and evaluating prompts

To thoroughly test and evaluate your prompt:

  1. Publish a version of your prompt.
  2. Set up a test run with the appropriate evaluators and dataset.
  3. Analyze the results to ensure your prompt performs as expected.

For detailed instructions on testing, refer to the Testing a Prompt guide.

Deploying prompts

Once you're satisfied with your prompt's performance, you can deploy it for use in your applications.

To deploy a prompt:

  1. Select the version you want to deploy.
  2. Click the "Deploy" button.
  3. Set the appropriate deployment variables and rules.
  4. Confirm the deployment.

Deployment variables allow you to control which prompt version is used in different environments or for specific tenants.

Integrating with the Maxim SDK

After deploying your prompts, you can access them in your applications using the Maxim SDK.

const prompt = await maxim.getPrompt("prompt-id");

For more advanced usage, refer to the Maxim SDK Documentation.

By leveraging the single prompt feature in Maxim, you can efficiently develop, test, and deploy prompts for your AI applications. The combination of flexible configuration, versioning, and seamless SDK integration makes it easier than ever to manage your prompt engineering workflow.

On this page