Notification Channels
Maxim supports multiple notification channels to ensure you stay informed about your alerts. Currently, we support Slack and PagerDuty integrations.You must set up at least one notification channel before creating alerts. Follow the instructions for your preferred notification channel.
Managing Integrations
You can manage your notification channel integrations in the following ways:- Edit an integration: Hover over the integration on the Integrations page and click the edit icon
- Delete an integration: Hover over the integration and click the delete icon
- View integration details: Click on the integration to see its configuration and usage
When you delete an integration, any alerts using that integration will no longer send notifications. Make sure to update or remove those alerts first.
Performance Metrics Alerts
Monitor your application’s performance by setting up alerts for latency, token usage, and cost metrics. Create custom thresholds and receive notifications when metrics exceed your specified limits.Before creating alerts, make sure you have set up at least one notification channel in the Notification Channels section above.
Available Performance Metrics
Set up alerts for:- Latency: Response times for API calls
- Token Usage: Token consumption per request
- Cost: API usage expenses
Creating Performance Alerts
1
Navigate to repository
Open the Logs page and select your repository
2
Access alerts tab
Select the Alerts tab and click Create alert
3
Configure alert settings
- Select Log metrics as the type of alert
- Select a metric (Latency, Token Usage, or Cost)
- Choose an operator (greater than, less than)
- Enter the threshold value
- Set minimum occurrence count
- Define evaluation time range
4
Select notification channels
Select your preferred notification channels from the ones you’ve set up in the Notification Channels section.
5
Save your alert
Click Create alert
Performance Alert Examples
Monitor Response Time
Setting Up Token Consumption Alerts
Monitor Daily Costs
Quality Metrics Alerts
Monitor your AI application’s quality with alerts for evaluation scores and quality checks. Receive notifications when AI responses don’t meet expected quality standards.Before creating alerts, make sure you have set up at least one notification channel in the Notification Channels section above.
Available Quality Metrics
Set up alerts for various evaluation scores, such as:- Bias-check: Monitor potential biases in AI responses
- Toxicity: Check for inappropriate or harmful content
- Clarity: Validate clear and understandable output
- Factual accuracy: Verify generated information accuracy
- Custom evaluators: Monitor your defined evaluation metrics
Creating Quality Alerts
1
Navigate to repository
Open the Logs page and select the repository you want to monitor.
2
Access alerts tab
Select the Alerts tab and click Create alert
3
Select alert type
Select Evaluation scores as the type of alert.
4
Configure alert settings
- Choose an evaluation metric (e.g., “Bias-check”)
- The violation criteria is based on your evaluator’s type and configuration.
- Specify how many times this should occur
- Set the evaluation time range
5
Choose notification channels
Select your preferred notification channels from the ones you’ve set up in the Notification Channels section.
6
Save your alert
Click Create alert to save your configuration.
Quality Alert Examples
Bias Check Alert
Toxicity Alert
Managing Alerts
Manage your alerts in the following ways:- Edit an alert: Click the options icon (three dots) on the alert card and select “Edit alert”
- Delete an alert: Click the options icon and select “Delete alert”
- Pause/Resume an alert: Click the options icon and select “Pause alert” or “Resume alert”
Best practices
- Monitor critical evaluators with immediate alerts for bias and toxicity
- Adjust score thresholds to match application requirements
- Combine evaluation metrics for comprehensive monitoring
- Review and adjust alert criteria based on performance
- Document quality issues and solutions from alerts